debian-mirror-gitlab/doc/user/project/clusters/runbooks/index.md

143 lines
6.1 KiB
Markdown
Raw Normal View History

2018-12-13 13:39:08 +05:30
# Runbooks
2019-09-30 21:07:59 +05:30
Runbooks are a collection of documented procedures that explain how to
carry out a particular process, be it starting, stopping, debugging,
2018-12-13 13:39:08 +05:30
or troubleshooting a particular system.
2019-07-07 11:18:12 +05:30
Using [Jupyter Notebooks](https://jupyter.org/) and the [Rubix library](https://github.com/Nurtch/rubix),
users can get started writing their own executable runbooks.
2018-12-13 13:39:08 +05:30
## Overview
2019-09-30 21:07:59 +05:30
Historically, runbooks took the form of a decision tree or a detailed
step-by-step guide depending on the condition or system.
2018-12-13 13:39:08 +05:30
2019-09-30 21:07:59 +05:30
Modern implementations have introduced the concept of an "executable
runbooks", where, along with a well-defined process, operators can execute
2018-12-13 13:39:08 +05:30
pre-written code blocks or database queries against a given environment.
2019-09-04 21:01:54 +05:30
## Executable Runbooks
2018-12-13 13:39:08 +05:30
> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/45912) in GitLab 11.4.
2019-09-30 21:07:59 +05:30
The JupyterHub app offered via GitLabs Kubernetes integration now ships
with Nurtchs Rubix library, providing a simple way to create DevOps
2019-09-04 21:01:54 +05:30
runbooks. A sample runbook is provided, showcasing common operations. While Rubix makes it
simple to create common Kubernetes and AWS workflows, you can also create them manually without
Rubix.
2018-12-13 13:39:08 +05:30
**<i class="fa fa-youtube-play youtube" aria-hidden="true"></i>
Watch this [video](https://www.youtube.com/watch?v=Q_OqHIIUPjE)
2019-07-07 11:18:12 +05:30
for an overview of how this is accomplished in GitLab!**
2018-12-13 13:39:08 +05:30
## Requirements
To create an executable runbook, you will need:
2019-09-30 21:07:59 +05:30
1. **Kubernetes** - A Kubernetes cluster is required to deploy the rest of the applications.
2019-10-12 21:52:04 +05:30
The simplest way to get started is to add a cluster using [GitLab's GKE integration](../index.md#add-new-gke-cluster).
2019-09-30 21:07:59 +05:30
1. **Helm Tiller** - Helm is a package manager for Kubernetes and is required to install
all the other applications. It is installed in its own pod inside the cluster which
2018-12-13 13:39:08 +05:30
can run the helm CLI in a safe environment.
2019-09-30 21:07:59 +05:30
1. **Ingress** - Ingress can provide load balancing, SSL termination, and name-based
2018-12-13 13:39:08 +05:30
virtual hosting. It acts as a web proxy for your applications.
2019-09-30 21:07:59 +05:30
1. **JupyterHub** - [JupyterHub](https://jupyterhub.readthedocs.io/) is a multi-user service for managing notebooks across
a team. Jupyter Notebooks provide a web-based interactive programming environment
2018-12-13 13:39:08 +05:30
used for data analysis, visualization, and machine learning.
## Nurtch
2019-09-30 21:07:59 +05:30
Nurtch is the company behind the [Rubix library](https://github.com/Nurtch/rubix). Rubix is
an open-source python library that makes it easy to perform common DevOps tasks inside Jupyter Notebooks.
Tasks such as plotting Cloudwatch metrics and rolling your ECS/Kubernetes app are simplified
down to a couple of lines of code. See the [Nurtch Documentation](http://docs.nurtch.com/en/latest)
2018-12-13 13:39:08 +05:30
for more information.
## Configure an executable runbook with GitLab
2019-09-30 21:07:59 +05:30
Follow this step-by-step guide to configure an executable runbook in GitLab using
2018-12-13 13:39:08 +05:30
the components outlined above and the preloaded demo runbook.
### 1. Add a Kubernetes cluster
2019-10-12 21:52:04 +05:30
Follow the steps outlined in [Add new GKE cluster](../index.md#add-new-gke-cluster)
2018-12-13 13:39:08 +05:30
to add a Kubernetes cluster to your project.
### 2. Install Helm Tiller, Ingress, and JupyterHub
Once the cluster has been provisioned in GKE, click the **Install** button next to the **Helm Tiller** app.
![install helm](img/helm-install.png)
Once Tiller has been installed successfully, click the **Install** button next to the **Ingress** app.
![install ingress](img/ingress-install.png)
Once Ingress has been installed successfully, click the **Install** button next to the **JupyterHub** app.
![install jupyterhub](img/jupyterhub-install.png)
### 3. Login to JupyterHub and start the server
2019-09-30 21:07:59 +05:30
Once JupyterHub has been installed successfully, navigate to the displayed **Jupyter Hostname** URL and click
**Sign in with GitLab**. Authentication is automatically enabled for any user of the GitLab instance via OAuth2. This
2018-12-13 13:39:08 +05:30
will redirect to GitLab in order to authorize JupyterHub to use your GitLab account. Click **Authorize**.
![authorize jupyter](img/authorize-jupyter.png)
2019-09-30 21:07:59 +05:30
Once the application has been authorized you will taken back to the JupyterHub application. Click **Start My Server**.
2018-12-13 13:39:08 +05:30
The server will take a couple of seconds to start.
### 4. Configure access
In order for the runbook to access your GitLab project, you will need to enter a
2019-09-04 21:01:54 +05:30
[GitLab Access Token](../../../profile/personal_access_tokens.md)
2018-12-13 13:39:08 +05:30
as well as your Project ID in the **Setup** section of the demo runbook.
Double-click the **DevOps-Runbook-Demo** folder located on the left panel.
![demo runbook](img/demo-runbook.png)
Double-click the "Nurtch-DevOps-Demo.ipynb" runbook.
![sample runbook](img/sample-runbook.png)
2019-09-30 21:07:59 +05:30
The contents on the runbook will be displayed on the right side of the screen. Under the "Setup" section, you will find
2018-12-13 13:39:08 +05:30
entries for both your `PRIVATE_TOKEN` and your `PROJECT_ID`. Enter both these values, conserving the single quotes as follows:
```sql
PRIVATE_TOKEN = 'n671WNGecHugsdEDPsyo'
PROJECT_ID = '1234567'
```
2019-09-30 21:07:59 +05:30
Update the `VARIABLE_NAME` on the last line of this section to match the name of the variable you are using for your
2018-12-13 13:39:08 +05:30
access token. In this example our variable name is `PRIVATE_TOKEN`.
```sql
VARIABLE_VALUE = project.variables.get('PRIVATE_TOKEN').value
```
### 5. Configure an operation
2019-09-30 21:07:59 +05:30
For this example we'll use the "**Run SQL queries in Notebook**" section in the sample runbook to query
a postgres database. The first 4 lines of the section define the variables that are required for this query to function.
2018-12-13 13:39:08 +05:30
```sql
%env DB_USER={project.variables.get('DB_USER').value}
%env DB_PASSWORD={project.variables.get('DB_PASSWORD').value}
%env DB_ENDPOINT={project.variables.get('DB_ENDPOINT').value}
%env DB_NAME={project.variables.get('DB_NAME').value}
```
Create the matching variables in your project's **Settings >> CI/CD >> Variables**
![gitlab variables](img/gitlab-variables.png)
2019-09-30 21:07:59 +05:30
Back in Jupyter, click the "Run SQL queries in Notebook" heading and the click *Run*. The results will be
2018-12-13 13:39:08 +05:30
displayed in-line as follows:
![postgres query](img/postgres-query.png)
2019-09-30 21:07:59 +05:30
You can try other operations such as running shell scripts or interacting with a Kubernetes cluster. Visit the
[Nurtch Documentation](http://docs.nurtch.com/) for more information.