mystiq/dex
Archived
4
0
Fork 1
Code Issues Pull requests Packages Projects Releases Wiki Activity

Merge pull request #79 from bobbyrullo/intro_page

Browse source 
README.md/Documentation: Get Docs up-to-date.
This commit is contained in:
bobbyrullo 2015-08-26 15:28:19 -07:00
parent 5abc7633fb eb65555fe7
commit ed0696f16b
6 changed files with 270 additions and 107 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

9
Documentation/deploy-guide.md Normal file
View file

@ -0,0 +1,9 @@
# Deploying
Generate systemd unit files by injecting secrets into the unit file templates located in: `./static/...`.
```
source <path-to-secure>/prod/dex.env.txt
./build-units
```

68
Documentation/dev-guide.md Normal file
View file

@ -0,0 +1,68 @@
# Dev Guide
## No DB mode
When you are working on dex it's convenient to use the "--no-db" flag, which starts up Dex in a mode which uses an in-memory datastore for persistence. It also does not rotate keys, so no overlord is required.
In this mode you provide the binary with paths to files to connectors and users - there are example files you can use inside of `static/fixtures`, named "connectors.json.sample" and "users.json.sample" respectively. If you rename these to the equivalent without the ".sample", the defaults point to this location, making starting dex as simple as:
`./bin/dex-worker --no-db`
*Do not use this flag in production* - it's not thread safe and data is destroyed when the process dies. In addition, there is no key rotation.
## Building
To build using the go binary on your host, use the `./build` script.
You can also use a copy of `go` hosted inside a Docker container if you prefix your command with `go-docker`, as in: `./go-docker ./build`
## Docker Build and Push
Once binaries are compiled you can build and push a dex image to quay.io. Before doing this step binaries must be built above using one of the build tools.
```
export DOCKER_USER=<<your user>>
export DOCKER_PASSWORD=<<your password>>
./build-docker-push
```
By default the script pushes to `quay.io/coreos/dex`; if you want to push to a different repository, override the `DOCKER_REGISTRY` and `DOCKER_REPO` environment variables.
## Rebuild API from JSON schema
Go API bindings are generated from a JSON Discovery file.
To regenerate run:
```
./schema/generator
```
For updating generator dependencies see docs in: `schema/generator_import.go`.
## Running Tests
To run all tests (except functional) use the `./test` script;
If you want to test a single package only, use `PKG=<pkgname> ./test`
The functional tests require a database; create a database (eg. `createdb dex_func_test`) and then pass it as an environment variable to the functional test script, eg. `DEX_TEST_DSN=postgres://localhost/dex_func_test?sslmode=disable ./test-functional`
To run these tests with Docker is a little trickier; you need to have a container running Postgres, and then you need to link that container to the container running your tests:
```
# Run the Postgres docker container, which creates a db called "postgres"
docker run --name dex_postgres -d postgres
# The host name in the DSN is "postgres"; that works because that is what we
# will alias the link as, which causes Docker to modify /etc/hosts with a "postgres"
# entry.
export DEX_TEST_DSN=postgres://postgres@postgres/postgres?sslmode=disable
# Run the test container, linking it to the Postgres container.
DOCKER_LINKS=dex_postgres:postgres DOCKER_ENV=DEX_TEST_DSN ./go-docker ./test-functional docker
# Remove the container after the tests are run.
rm -f dex_postgres
```

142
Documentation/getting-started.md Normal file
View file

@ -0,0 +1,142 @@
# Getting Started
# Introduction
In this document we'll stand up the full dex stack on a single machine. This should demonstrate all the moving parts involved in a dex installation, but is not appropriate for production deployment. Please see the [deployment guide][deployment-guide] for information on production dex setups.
[deployment-guide]: https://github.com/coreos/dex/blob/master/Documentation/deployment-guide.md
We'll also start the example web app, so we can try registering and logging in.
# Pre-requisites
Before continuing, you must have the following installed on your system:
* Go 1.4 or greater
* Postgres 9.0 or greater (this guide also assumes that Postgres is up and running)
In addition, if you wish to try out authenticating against Google's OIDC backend, you must have a new client registered with Google:
* Go to https://console.developers.google.com/project and create a new project.
* Click on credentials, and ask to set up an OAuth 2 client ID. You'll then need to give your project a name.
* Back at the "Create Client ID" screen, choose "Web Application" and enter `http://127.0.0.1:5556/auth/google/callback` for your Redirect URI.
# Create Database
`createdb dex_db`
Let's store the connection string in a shell variable:
`DEX_DB_URL=postgres://localhost/dex_db?sslmode=disable`
# Building
The build script will build all dex components.
`./build`
# Generate a Secret Symmetric Key
dex needs a 32 byte base64-encoded key which will be used to encrypt the private keys in the database. A good way to generate the key is to read from /dev/random:
`DEX_KEY_SECRET=$(dd if=/dev/random bs=1 count=32 2>/dev/null | base64)`
The dex overlord and workers allow multiple key secrets (separated by commas) to be passed but only the first will be used to encrypt data; the rest are there for decryption only; this scheme allows for the rotation of keys without downtime (assuming a rolling restart of workers).
# Start the overlord
The overlord is responsible for creating and rotating keys and some other adminsitrative tasks. In addition, the overlord is responsible for creating the necessary database tables (and when you update, performing schema migrations), so it must be started before we do anything else. Debug logging is turned on so we can see more of what's going on. Start it up.
`./bin/dex-overlord --db-url=$DEX_DB_URL --key-secrets=$DEX_KEY_SECRET --log-debug=true &`
## Environment Variables.
Note that parameters can be passed as flags or environment variables to dex components; an equivalent start with environment variables would be:
```
export DEX_OVERLORD_DB_URL=$DEX_DB_URL
export DEX_OVERLORD_KEY_SECRETS=$DEX_KEY_SECRET
export DEX_OVERLORD_LOG_DEBUG=true
./bin/dex-overlord &
```
# Start the dex-worker
Now start the worker:
`./bin/dex-worker --db-url=$DEX_DB_URL --key-secrets=$DEX_KEY_SECRET --email-cfg=static/fixtures/emailer.json.sample --log-debug=true &`
Now you have a worker which you can authenticate against, listening on `http://0.0.0.0:5556`, which is the default. Note that the default issuer URL (which can be changed on --issuerURL) is `http://127.0.0.1:5556`. The issuer URL is the base URL (i.e. no query or fragments) uniquely identifying your dex installation.
Note: the issuer URL MUST have an `https` scheme in production to meet spec compliance and to be considered reasonably secure.
# Set up Connectors
The worker and overlord are up and running, but we need to tell dex what connectors we want to use to authenticate. For this case we'll set up a local connector, where dex manages credentials and provides a UI for authentication, and a Google OIDC connector.
If you prefer to use the Google OIDC Identity Provider (IdP), just omit the second entry in the JSON connector list. Note that you must replace DEX_GOOGLE_CLIENT_{ID,SECRET} with the client ID and client Secret you got when you registered your project with the Google developer console.
```
cat << EOF > /tmp/dex_connectors.json
[
{
"type": "local",
"id": "local"
},
{
"type": "oidc",
"id": "google",
"issuerURL": "https://accounts.google.com",
"clientID": "$DEX_GOOGLE_CLIENT_ID",
"clientSecret": "$DEX_GOOGLE_CLIENT_SECRET",
"trustedEmailProvider": true
}
]
EOF
./bin/dexctl -db-url=$DEX_DB_URL set-connector-configs /tmp/dex_connectors.json
```
One thing to note here that's a bit confusing here is that in the case of the Google OIDC connector, dex is the client and Google is the IdP, but when you're dealing with your own apps that want to authenticate against dex, your app is the client and dex is the IdP.
# Register a Client
Like all OAuth2/OIDC IdPs, clients must be registered with the IdP (dex), along with their valid redirect URLS.
New clients can be registered with the dexctl CLI tool:
```
eval "$(./bin/dexctl -db-url=$DEX_DB_URL new-client http://127.0.0.1:5555/callback)"
```
The output of this command is eval'able if you are in bash, and sets the following shell variables:
```
DEX_APP_CLIENT_ID
DEX_APP_CLIENT_SECRET
DEX_APP_REDIRECTURL
```
# Start the Example Web App
The included example app demonstrates registering and authenticating with dex. Start it up:
```
./bin/example-app --client-id=$DEX_APP_CLIENT_ID --client-secret=$DEX_APP_CLIENT_SECRET --discovery=http://127.0.0.1:5556 &
```
# Authenticate with dex!
Go to `127.0.0.1:5555`, and click "register"; choose either "Google", if you have a Google Account and would like to use that to authenticate. Otherwise, choose "local".
If you chose Google, enter your credentials (if you are not logged into Google) and click through the authorization screen. If you chose "local", enter a name and password and submit.
After registering you should end up back at the example app, where it will display the claims returned by dex.
# Verify Your Email
If you registered with Google, your email address is already verified, and this should be reflected by the presence of an `email_verified` claim. Otherwise, you need to verify your email address.
In a fully configured production environment an email provider will be set up so that dex can email users email verification links (amongst other things); in this setup, we are using the `FakeEmailer` email provider which simply outputs to stdout. Look for the "Welcome to Dex!" message in your console and copy the link that follows it, and then paste it in your browser; you should end up back at the example app page that displays claims, but this time you'll see a tru `email_verified` claim.
# Standup Dev Script
A script which does almost everything in this guide exists at `contrib/standup-db.sh`. Read the comments inside before attemping to run it - it requires a little setup beforehand.

124
README.md
View file

@ -3,34 +3,48 @@ dex
[![Docker Repository on Quay.io](https://quay.io/repository/coreos/dex/status?token=5a9732e4-53d6-4419-b56b-9f784f7f9233 "Docker Repository on Quay.io")](https://quay.io/repository/coreos/dex)
dex is a federated identity management service.
It provides OpenID Connect (OIDC) to users, while it proxies to multiple remote identity providers (IdP) to drive actual authentication.
dex is a federated identity management service. It provides OpenID Connect (OIDC) to users, and can proxy to multiple remote identity providers (IdP) to drive actual authentication, as well as managing local username/password credentials.
We named the project 'dex' beceause it is a central index of users that other pieces of software can authenticate against.
## Architecture
dex consists of multiple components:
- **dex-worker** is the primary server component of dex
- host a user-facing API that drives the OIDC protocol
- host a user-facing API that drives the OIDC protocol
- proxy to remote identity providers via "connectors"
- provides an API for administrators to manage users.
- **dex-overlord** is an auxiliary process responsible for two things:
- rotation of keys used by the workers to sign identity tokens
- garbage collection of stale data in the database
- provides an API for bootstrapping the system.
- **dexctl** is CLI tool used to manage an dex deployment
- configure identity provider connectors
- administer OIDC client identities
- **database**; a database is used to for persistent storage for keys, users,
OAuth sessions and other data. Currently Postgres is the only supported
database.
A typical dex deployment consists of N dex-workers behind a load balanacer, and one dex-overlord.
The dex-workers directly handle user requests, so the loss of all workers can result in service downtime.
The single dex-overlord runs its tasks periodically, so it does not need to maintain 100% uptime.
## Who Should Use Dex?
**TODO**
## Similar Software
**TODO**
## Connectors
Remote IdPs could implement any auth-N protocol.
*connectors* contain protocol-specific logic and are used to communicate with remote IdPs.
Possible examples of connectors could be: OIDC, LDAP, Local Memory, Basic Auth, etc.
dex ships with an OIDC connector, and a basic "local" connector for in-memory testing purposes.
Remote IdPs could implement any auth-N protocol. *Connectors* contain protocol-specific logic and are used to communicate with remote IdPs. Possible examples of connectors could be: OIDC, LDAP, Local credentials, Basic Auth, etc.
dex ships with an OIDC connector, useful for authenticating with services like Google and Salesforce (or even other dex instances!) and a "local" connector, in which dex itself presents a UI for users to authenticate via dex-stored credentials.
Future connectors can be developed and added as future interoperability requirements emerge.
## Relevant Specifications
@ -52,100 +66,14 @@ OpenID Connect (OIDC) is broken up into several specifications. The following (a
- https://accounts.google.com/.well-known/openid-configuration
- https://login.salesforce.com/.well-known/openid-configuration
# Building
# Next steps:
## With Host Go Environment
If you want to try out dex quickly with a single process and no database (do *not* run this way in production!) take a look at the [dev guide][dev-guide].
`./build`
For running the full stack check out the [getting started guide][getting-started].
## With Docker
`./go-docker ./build`
## Docker Build and Push
Binaries must be compiled first.
Builds a docker image and pushes it to the quay repo.
The image is tagged with the git sha and 'latest'.
```
export QUAY_USER=xxx
export QUAY_PASSWORD=yyy
./build-docker-push
```
## Rebuild API from JSON schema
Go API bindings are generated from a JSON Discovery file.
To regenerate run:
```
./schema/generator
```
For updating generator dependencies see docs in: `schema/generator_import.go`.
## Runing Tests
Run all tests: `./test`
Single package only: `PKG=<pkgname> ./test`
Functional tests: `./test-functional`
Run with docker:
```
./go-docker ./test
./go-docker ./test-functional
```
# Running
Run the main dex server:
After building, run `./bin/dex` and provider the required arguments.
Additionally start `./bin/dex-overlord` for key rotation and database garbage collection.
# Deploying
Generate systemd unit files by injecting secrets into the unit file templates located in: `./static/...`.
```
source <path-to-secure>/prod/dex.env.txt
./build-units
```
Resulting unit files are output to: `./deploy`
# Registering Clients
Like all OAuth2 servers clients must be registered with a callback url.
New clients can be registered with the dexctl CLI tool:
```
dexctl --db-url=postgres://localhost/auth?sslmode=disable new-client http://example.com/auth/callback
```
The tool will print the `client-id` and `client-secret` to stdout; you must save these for use in your client application. The output of this command is "KEY=VALUE" format, so If you `eval` it in your shell, the relevant variables are available to use.
Note that for the initial invocation of `dexctl` you need to provide a DSN URL to create a new-client. Once you have created this initial client, you can use its client-id and client-secret as credentials to dexctl, and make requests via the HTTP API instead of the DB:
```
dexctl --endpoint=http://your-issuer-url --client-id=your_client_id --client-secret=your_client_secret new-client
```
or, if you want to go the eval route:
```
eval "$(dexctl --endpoint=http://your-issuer-url --client-id=your_client_id --client-secret=your_client_secret new-client)"
```
The latter form makes the variables `DEX_APP_CLIENT_ID`, `DEX_APP_CLIENT_SECRET` and `DEX_APP_REDIRECTURL_0` available to your shell.
This will allow you to create new clients from machines that cannot hit the database.
# Standup Dev Script
A script which will create a database, create a client, start an overlord and a worker and start the example app exists at `contrib/standup-db.sh`.
[getting-started]: https://github.com/coreos/dex/blob/master/Documentation/getting-started.md
[dev-guide]: https://github.com/coreos/dex/blob/master/Documentation/dev-guide.md
# Coming Soon

8
build-docker-push
View file

@ -1,13 +1,15 @@
#!/bin/bash -e
registry=quay.io
repo=$registry/coreos/dex
DOCKER_REGISTRY=${DOCKER_REGISTRY:=quay.io}
DOCKER_REPO=${DOCKER_REPO:=coreos/dex}
repo=$DOCKER_REGISTRY/$DOCKER_REPO
if [ -v $DOCKER_USER ] || [ -v $DOCKER_PASSWORD ]; then
echo "env variables not set: DOCKER_USER, DOCKER_PASSWORD. skipping login, assuming creds in .dockercfg"
else
echo logging in as $DOCKER_USER
docker login --username="$DOCKER_USER" --password="$DOCKER_PASSWORD" --email="docker.login@coreos.com" $registry
docker login --username="$DOCKER_USER" --password="$DOCKER_PASSWORD" --email="docker.login@coreos.com" $DOCKER_REGISTRY
fi
git_sha=$(git rev-parse HEAD)

26
contrib/standup-db.sh
View file

@ -24,6 +24,20 @@ export DEX_WORKER_DB_URL=$DEX_DB_URL
# Delete/create DB
dropdb $DEX_DB; createdb $DEX_DB
DEX_KEY_SECRET=$(dd if=/dev/random bs=1 count=32 2>/dev/null | base64)
# Start the overlord
export DEX_OVERLORD_DB_URL=$DEX_DB_URL
export DEX_OVERLORD_KEY_SECRETS=$DEX_KEY_SECRET
export DEX_OVERLORD_KEY_PERIOD=1h
./bin/dex-overlord &
echo "Waiting for overlord to start..."
until $(curl --output /dev/null --silent --fail http://localhost:5557/health); do
printf '.'
sleep 1
done
# Create a client
eval "$(./bin/dexctl -db-url=$DEX_DB_URL new-client http://127.0.0.1:5555/callback)"
@ -49,17 +63,17 @@ EOF
./bin/dexctl -db-url=$DEX_DB_URL set-connector-configs $DEX_CONNECTORS_FILE
# Start the overlord
export DEX_OVERLORD_DB_URL=$DEX_DB_URL
export DEX_OVERLORD_KEY_SECRET=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
export DEX_OVERLORD_KEY_PERIOD=1h
./bin/dex-overlord &
# Start the worker
export DEX_WORKER_DB_URL=$DEX_DB_URL
export DEX_WORKER_KEY_SECRET=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
export DEX_WORKER_KEY_SECRETS=$DEX_KEY_SECRET
export DEX_WORKER_LOG_DEBUG=1
./bin/dex-worker &
echo "Waiting for worker to start..."
until $(curl --output /dev/null --silent --fail http://localhost:5556/health); do
printf '.'
sleep 1
done
# Start the app
./bin/example-app --client-id=$DEX_APP_CLIENT_ID --client-secret=$DEX_APP_CLIENT_SECRET --discovery=http://127.0.0.1:5556 &