2021-01-29 00:20:46 +05:30
---
stage: none
group: unassigned
2022-11-25 23:54:43 +05:30
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
2021-01-29 00:20:46 +05:30
---
2021-01-03 14:25:43 +05:30
# GitLab architecture overview
2014-09-02 18:07:02 +05:30
## Software delivery
2021-01-03 14:25:43 +05:30
There are two software distributions of GitLab:
2014-09-02 18:07:02 +05:30
2021-01-03 14:25:43 +05:30
- The open source [Community Edition ](https://gitlab.com/gitlab-org/gitlab-foss/ ) (CE).
- The open core [Enterprise Edition ](https://gitlab.com/gitlab-org/gitlab/ ) (EE).
2018-03-17 18:26:18 +05:30
2021-01-03 14:25:43 +05:30
GitLab is available under [different subscriptions ](https://about.gitlab.com/pricing/ ).
2014-09-02 18:07:02 +05:30
2021-09-04 01:27:46 +05:30
New versions of GitLab are released from stable branches, and the `main` branch is used for
2021-01-03 14:25:43 +05:30
bleeding-edge development.
2023-04-23 21:23:45 +05:30
For more information, see the [GitLab release process ](https://about.gitlab.com/handbook/engineering/releases/ ).
2021-01-03 14:25:43 +05:30
Both distributions require additional components. These components are described in the
[Component details ](#components ) section, and all have their own repositories.
2021-09-04 01:27:46 +05:30
New versions of each dependent component are usually tags, but staying on the `main` branch of the
2021-01-03 14:25:43 +05:30
GitLab codebase gives you the latest stable version of those components. New versions are
generally released around the same time as GitLab releases, with the exception of informal security
updates deemed critical.
2014-09-02 18:07:02 +05:30
2019-09-04 21:01:54 +05:30
## Components
2015-04-26 12:48:37 +05:30
2021-01-03 14:25:43 +05:30
A typical install of GitLab is on GNU/Linux, but growing number of deployments also use the
Kubernetes platform. The largest known GitLab instance is on GitLab.com, which is deployed using our
[official GitLab Helm chart ](https://docs.gitlab.com/charts/ ) and the [official Linux package ](https://about.gitlab.com/install/ ).
2015-04-26 12:48:37 +05:30
2021-01-03 14:25:43 +05:30
A typical installation uses NGINX or Apache as a web server to proxy through
2023-04-23 21:23:45 +05:30
[GitLab Workhorse ](https://gitlab.com/gitlab-org/gitlab/tree/master/workhorse ) and into the [Puma ](https://puma.io )
application server. GitLab serves web pages and the [GitLab API ](../api/rest/index.md ) using the Puma
2021-01-03 14:25:43 +05:30
application server. It uses Sidekiq as a job queue which, in turn, uses Redis as a non-persistent
database backend for job information, metadata, and incoming jobs.
2015-04-26 12:48:37 +05:30
2021-01-03 14:25:43 +05:30
By default, communication between Puma and Workhorse is via a Unix domain socket, but forwarding
requests via TCP is also supported. Workhorse accesses the `gitlab/public` directory, bypassing the
Puma application server to serve static pages, uploads (for example, avatar images or attachments),
and pre-compiled assets.
2015-04-26 12:48:37 +05:30
2021-01-03 14:25:43 +05:30
The GitLab application uses PostgreSQL for persistent database information (for example, users,
permissions, issues, or other metadata). GitLab stores the bare Git repositories in the location
2021-09-04 01:27:46 +05:30
defined in [the configuration file, `repositories:` section ](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/gitlab.yml.example ).
2021-01-03 14:25:43 +05:30
It also keeps default branch and hook information with the bare repository.
2015-04-26 12:48:37 +05:30
2021-01-03 14:25:43 +05:30
When serving repositories over HTTP/HTTPS GitLab uses the GitLab API to resolve authorization and
access and to serve Git objects.
2019-09-04 21:01:54 +05:30
2021-01-03 14:25:43 +05:30
The add-on component GitLab Shell serves repositories over SSH. It manages the SSH keys within the
2021-09-04 01:27:46 +05:30
location defined in [the configuration file, `GitLab Shell` section ](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/gitlab.yml.example ).
2021-01-03 14:25:43 +05:30
The file in that location should never be manually edited. GitLab Shell accesses the bare
repositories through Gitaly to serve Git objects, and communicates with Redis to submit jobs to
Sidekiq for GitLab to process. GitLab Shell queries the GitLab API to determine authorization and access.
Gitaly executes Git operations from GitLab Shell and the GitLab web app, and provides an API to the
GitLab web app to get attributes from Git (for example, title, branches, tags, or other metadata),
and to get blobs (for example, diffs, commits, or files).
2019-09-04 21:01:54 +05:30
2020-03-13 15:44:24 +05:30
You may also be interested in the [production architecture of GitLab.com ](https://about.gitlab.com/handbook/engineering/infrastructure/production/architecture/ ).
2019-09-04 21:01:54 +05:30
2021-01-03 14:25:43 +05:30
## Adapting existing and introducing new components
There are fundamental differences in how the application behaves when it is installed on a
traditional Linux machine compared to a containerized platform, such as Kubernetes.
Compared to [our official installation methods ](https://about.gitlab.com/install/ ), some of the
notable differences are:
- Official Linux packages can access files on the same file system with different services.
[Shared files ](shared_files.md ) are not an option for the application running on the Kubernetes
platform.
- Official Linux packages by default have services that have access to the shared configuration and
network. This is not the case for services running in Kubernetes, where services might be running
in complete isolation, or only accessible through specific ports.
In other words, the shared state between services needs to be carefully considered when
architecting new features and adding new components. Services that need to have access to the same
files, need to be able to exchange information through the appropriate APIs. Whenever possible,
this should not be done with files.
Since components written with the API-first philosophy in mind are compatible with both methods, all
new features and services must be written to consider Kubernetes compatibility **first** .
The simplest way to ensure this, is to add support for your feature or service to
[the official GitLab Helm chart ](https://docs.gitlab.com/charts/ ) or reach out to
2022-08-27 11:52:29 +05:30
[the Distribution team ](https://about.gitlab.com/handbook/engineering/development/enablement/systems/distribution/#how-to-work-with-distribution ).
2021-01-03 14:25:43 +05:30
2021-03-11 19:13:27 +05:30
Refer to the [process for adding new service components ](adding_service_component.md ) for more details.
2021-01-03 14:25:43 +05:30
### Simplified component overview
2019-09-04 21:01:54 +05:30
This is a simplified architecture diagram that can be used to
2021-02-22 17:27:13 +05:30
understand the GitLab architecture.
2019-09-04 21:01:54 +05:30
A complete architecture diagram is available in our
[component diagram ](#component-diagram ) below.
2022-05-07 20:08:51 +05:30
2019-09-04 21:01:54 +05:30
2019-09-30 21:07:59 +05:30
<!--
2022-05-07 20:08:51 +05:30
To update this diagram, use and update this source file:
https://miro.com/app/board/uXjVOH3lzXo=/
2019-09-04 21:01:54 +05:30
-->
### Component diagram
```mermaid
2021-09-30 23:02:18 +05:30
%%{init: {"flowchart": { "useMaxWidth": false } }}%%
graph LR
%% Anchor items in the appropriate subgraph.
%% Link them where the destination* is.
subgraph Clients
Browser((Browser))
Git((Git))
end
%% External Components / Applications
Geo{{GitLab Geo}} -- TCP 80, 443 --> HTTP
Geo -- TCP 22 --> SSH
Geo -- TCP 5432 --> PostgreSQL
Runner{{GitLab Runner}} -- TCP 443 --> HTTP
2022-01-26 12:08:38 +05:30
K8sAgent{{GitLab Agent}} -- TCP 443 --> HTTP
2021-09-30 23:02:18 +05:30
%% GitLab Application Suite
subgraph GitLab
subgraph Ingress
HTTP[[HTTP/HTTPS]]
SSH[[SSH]]
NGINX[NGINX]
GitLabShell[GitLab Shell]
%% inbound/internal
Browser -- TCP 80,443 --> HTTP
Git -- TCP 80,443 --> HTTP
Git -- TCP 22 --> SSH
HTTP -- TCP 80, 443 --> NGINX
SSH -- TCP 22 --> GitLabShell
end
subgraph GitLab Services
%% inbound from NGINX
NGINX --> GitLabWorkhorse
NGINX -- TCP 8090 --> GitLabPages
NGINX -- TCP 8150 --> GitLabKas
NGINX --> Registry
%% inbound from GitLabShell
2022-05-07 20:08:51 +05:30
GitLabShell --> GitLabWorkhorse
2021-09-30 23:02:18 +05:30
%% services
Puma["Puma (GitLab Rails)"]
Puma < -- > Registry
GitLabWorkhorse[GitLab Workhorse] < -- > Puma
2022-01-26 12:08:38 +05:30
GitLabKas[GitLab Agent Server] --> GitLabWorkhorse
2021-09-30 23:02:18 +05:30
GitLabPages[GitLab Pages] --> GitLabWorkhorse
Mailroom
Sidekiq
end
subgraph Integrated Services
%% Mattermost
Mattermost
Mattermost ---> GitLabWorkhorse
NGINX --> Mattermost
%% Grafana
Grafana
NGINX --> Grafana
end
subgraph Metadata
%% PostgreSQL
PostgreSQL
PostgreSQL --> Consul
%% Consul and inbound
Consul
Puma ---> Consul
Sidekiq ---> Consul
Migrations --> PostgreSQL
%% PgBouncer and inbound
PgBouncer
PgBouncer --> Consul
PgBouncer --> PostgreSQL
Sidekiq --> PgBouncer
Puma --> PgBouncer
end
subgraph State
%% Redis and inbound
Redis
Puma --> Redis
Sidekiq --> Redis
GitLabWorkhorse --> Redis
Mailroom --> Redis
GitLabKas --> Redis
%% Sentinel and inbound
Sentinel < -- > Redis
Puma --> Sentinel
Sidekiq --> Sentinel
GitLabWorkhorse --> Sentinel
Mailroom --> Sentinel
GitLabKas --> Sentinel
end
subgraph Git Repositories
%% Gitaly / Praefect
Praefect --> Gitaly
GitLabKas --> Praefect
GitLabShell --> Praefect
GitLabWorkhorse --> Praefect
Puma --> Praefect
Sidekiq --> Praefect
Praefect < -- > PraefectPGSQL[PostgreSQL]
%% Gitaly makes API calls
%% Ordered here to ensure placement.
Gitaly --> GitLabWorkhorse
end
subgraph Storage
%% ObjectStorage and inbound traffic
ObjectStorage["Object Storage"]
Puma -- TCP 443 --> ObjectStorage
Sidekiq -- TCP 443 --> ObjectStorage
GitLabWorkhorse -- TCP 443 --> ObjectStorage
Registry -- TCP 443 --> ObjectStorage
GitLabPages -- TCP 443 --> ObjectStorage
end
subgraph Monitoring
%% Prometheus
Grafana -- TCP 9090 --> Prometheus[Prometheus]
Prometheus -- TCP 80, 443 --> Puma
RedisExporter[Redis Exporter] --> Redis
Prometheus -- TCP 9121 --> RedisExporter
PostgreSQLExporter[PostgreSQL Exporter] --> PostgreSQL
PgBouncerExporter[PgBouncer Exporter] --> PgBouncer
Prometheus -- TCP 9187 --> PostgreSQLExporter
Prometheus -- TCP 9100 --> NodeExporter[Node Exporter]
Prometheus -- TCP 9168 --> GitLabExporter[GitLab Exporter]
Prometheus -- TCP 9127 --> PgBouncerExporter
Prometheus --> Alertmanager
GitLabExporter --> PostgreSQL
GitLabExporter --> GitLabShell
GitLabExporter --> Sidekiq
%% Alertmanager
Alertmanager -- TCP 25 --> SMTP
end
%% end subgraph GitLab
end
subgraph External
subgraph External Services
SMTP[SMTP Gateway]
LDAP
%% Outbound SMTP
Sidekiq -- TCP 25 --> SMTP
Puma -- TCP 25 --> SMTP
Mailroom -- TCP 25 --> SMTP
%% Outbound LDAP
Puma -- TCP 369 --> LDAP
Sidekiq -- TCP 369 --> LDAP
%% Elasticsearch
Elasticsearch
Puma -- TCP 9200 --> Elasticsearch
Sidekiq -- TCP 9200 --> Elasticsearch
end
subgraph External Monitoring
%% Sentry
Sidekiq -- TCP 80, 443 --> Sentry
Puma -- TCP 80, 443 --> Sentry
%% Jaeger
Jaeger
Sidekiq -- UDP 6831 --> Jaeger
Puma -- UDP 6831 --> Jaeger
Gitaly -- UDP 6831 --> Jaeger
GitLabShell -- UDP 6831 --> Jaeger
GitLabWorkhorse -- UDP 6831 --> Jaeger
end
%% end subgraph External
end
2020-10-24 23:57:45 +05:30
click Alertmanager "./architecture.html#alertmanager"
click Praefect "./architecture.html#praefect"
click Geo "./architecture.html#gitlab-geo"
click NGINX "./architecture.html#nginx"
click Runner "./architecture.html#gitlab-runner"
click Registry "./architecture.html#registry"
click ObjectStorage "./architecture.html#minio"
click Mattermost "./architecture.html#mattermost"
click Gitaly "./architecture.html#gitaly"
click Jaeger "./architecture.html#jaeger"
click GitLabWorkhorse "./architecture.html#gitlab-workhorse"
click LDAP "./architecture.html#ldap-authentication"
2021-01-03 14:25:43 +05:30
click Puma "./architecture.html#puma"
2020-10-24 23:57:45 +05:30
click GitLabShell "./architecture.html#gitlab-shell"
click SSH "./architecture.html#ssh-request-22"
click Sidekiq "./architecture.html#sidekiq"
click Sentry "./architecture.html#sentry"
click GitLabExporter "./architecture.html#gitlab-exporter"
click Elasticsearch "./architecture.html#elasticsearch"
click Migrations "./architecture.html#database-migrations"
click PostgreSQL "./architecture.html#postgresql"
click Consul "./architecture.html#consul"
click PgBouncer "./architecture.html#pgbouncer"
click PgBouncerExporter "./architecture.html#pgbouncer-exporter"
click RedisExporter "./architecture.html#redis-exporter"
click Redis "./architecture.html#redis"
click Prometheus "./architecture.html#prometheus"
click Grafana "./architecture.html#grafana"
click GitLabPages "./architecture.html#gitlab-pages"
click PostgreSQLExporter "./architecture.html#postgresql-exporter"
click SMTP "./architecture.html#outbound-email"
click NodeExporter "./architecture.html#node-exporter"
2019-02-15 15:39:39 +05:30
```
2015-04-26 12:48:37 +05:30
2019-09-04 21:01:54 +05:30
### Component legend
2019-09-30 21:07:59 +05:30
- ✅ - Installed by default
2021-09-04 01:27:46 +05:30
- ⚙ - Requires additional configuration
2019-09-30 21:07:59 +05:30
- ⤓ - Manual installation required
- ❌ - Not supported or no instructions available
2020-01-01 13:55:28 +05:30
- N/A - Not applicable
2019-09-04 21:01:54 +05:30
Component statuses are linked to configuration documentation for each component.
### Component list
2022-04-04 11:22:00 +05:30
| Component | Description | [Omnibus GitLab ](https://docs.gitlab.com/omnibus/ ) | [GitLab Environment Toolkit (GET) ](https://gitlab.com/gitlab-org/gitlab-environment-toolkit ) | [GitLab chart ](https://docs.gitlab.com/charts/ ) | [minikube Minimal ](https://docs.gitlab.com/charts/development/minikube/#deploying-gitlab-with-minimal-settings ) | [GitLab.com ](https://gitlab.com ) | [Source ](../install/installation.md ) | [GDK ](https://gitlab.com/gitlab-org/gitlab-development-kit ) | [CE/EE ](https://about.gitlab.com/install/ce-or-ee/ ) |
2021-03-11 19:13:27 +05:30
|-------------------------------------------------------|----------------------------------------------------------------------|:--------------:|:--------------:|:------------:|:----------------:|:----------:|:------:|:---:|:-------:|
| [Certificate Management ](#certificate-management ) | TLS Settings, Let's Encrypt | ✅ | ✅ | ✅ | ⚙ | ✅ | ⚙ | ⚙ | CE & EE |
| [Consul ](#consul ) | Database node discovery, failover | ⚙ | ✅ | ❌ | ❌ | ✅ | ❌ | ❌ | EE Only |
| [Database Migrations ](#database-migrations ) | Database migrations | ✅ | ✅ | ✅ | ✅ | ✅ | ⚙ | ✅ | CE & EE |
2021-10-27 15:23:28 +05:30
| [Elasticsearch ](#elasticsearch ) | Improved search within GitLab | ⤓ | ⚙ | ⤓ | ⤓ | ✅ | ⤓ | ⚙ | EE Only |
2021-03-11 19:13:27 +05:30
| [Gitaly ](#gitaly ) | Git RPC service for handling all Git calls made by GitLab | ✅ | ✅ | ✅ | ✅ | ✅ | ⚙ | ✅ | CE & EE |
| [GitLab Exporter ](#gitlab-exporter ) | Generates a variety of GitLab metrics | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | CE & EE |
2022-07-16 23:28:13 +05:30
| [GitLab Geo ](#gitlab-geo ) | Geographically distributed GitLab site | ⚙ | ⚙ | ❌ | ❌ | ✅ | ❌ | ⚙ | EE Only |
2021-03-11 19:13:27 +05:30
| [GitLab Pages ](#gitlab-pages ) | Hosts static websites | ⚙ | ⚙ | ❌ | ❌ | ✅ | ⚙ | ⚙ | CE & EE |
2022-05-07 20:08:51 +05:30
| [GitLab agent ](#gitlab-agent ) | Integrate Kubernetes clusters in a cloud-native way | ⚙ | ⚙ | ⚙ | ❌ | ❌ | ⤓ | ⚙ | EE Only |
2021-03-11 19:13:27 +05:30
| [GitLab self-monitoring: Alertmanager ](#alertmanager ) | Deduplicates, groups, and routes alerts from Prometheus | ⚙ | ⚙ | ✅ | ⚙ | ✅ | ❌ | ❌ | CE & EE |
2021-10-27 15:23:28 +05:30
| [GitLab self-monitoring: Grafana ](#grafana ) | Metrics dashboard | ✅ | ✅ | ⚙ | ⤓ | ✅ | ❌ | ⚙ | CE & EE |
2021-03-11 19:13:27 +05:30
| [GitLab self-monitoring: Jaeger ](#jaeger ) | View traces generated by the GitLab instance | ❌ | ⚙ | ⚙ | ❌ | ❌ | ⤓ | ⚙ | CE & EE |
2021-10-27 15:23:28 +05:30
| [GitLab self-monitoring: Prometheus ](#prometheus ) | Time-series database, metrics collection, and query service | ✅ | ✅ | ✅ | ⚙ | ✅ | ❌ | ⚙ | CE & EE |
2021-03-11 19:13:27 +05:30
| [GitLab self-monitoring: Sentry ](#sentry ) | Track errors generated by the GitLab instance | ⤓ | ⤓ | ⤓ | ❌ | ✅ | ⤓ | ⤓ | CE & EE |
| [GitLab Shell ](#gitlab-shell ) | Handles `git` over SSH sessions | ✅ | ✅ | ✅ | ✅ | ✅ | ⚙ | ✅ | CE & EE |
| [GitLab Workhorse ](#gitlab-workhorse ) | Smart reverse proxy, handles large HTTP requests | ✅ | ✅ | ✅ | ✅ | ✅ | ⚙ | ✅ | CE & EE |
| [Inbound email (SMTP) ](#inbound-email ) | Receive messages to update issues | ⤓ | ⤓ | ⚙ | ⤓ | ✅ | ⤓ | ⤓ | CE & EE |
2021-10-27 15:23:28 +05:30
| [Jaeger integration ](#jaeger ) | Distributed tracing for deployed apps | ⤓ | ⤓ | ⤓ | ⤓ | ⤓ | ⤓ | ⚙ | EE Only |
| [LDAP Authentication ](#ldap-authentication ) | Authenticate users against centralized LDAP directory | ⤓ | ⤓ | ⤓ | ⤓ | ❌ | ⤓ | ⚙ | CE & EE |
| [Mattermost ](#mattermost ) | Open-source Slack alternative | ⚙ | ⚙ | ⤓ | ⤓ | ⤓ | ❌ | ⚙ | CE & EE |
2021-03-11 19:13:27 +05:30
| [MinIO ](#minio ) | Object storage service | ⤓ | ⤓ | ✅ | ✅ | ✅ | ❌ | ⚙ | CE & EE |
2021-10-27 15:23:28 +05:30
| [NGINX ](#nginx ) | Routes requests to appropriate components, terminates SSL | ✅ | ✅ | ✅ | ⚙ | ✅ | ⤓ | ⚙ | CE & EE |
2021-03-11 19:13:27 +05:30
| [Node Exporter ](#node-exporter ) | Prometheus endpoint with system metrics | ✅ | ✅ | N/A | N/A | ✅ | ❌ | ❌ | CE & EE |
| [Outbound email (SMTP) ](#outbound-email ) | Send email messages to users | ⤓ | ⤓ | ⚙ | ⤓ | ✅ | ⤓ | ⤓ | CE & EE |
| [Patroni ](#patroni ) | Manage PostgreSQL HA cluster leader selection and replication | ⚙ | ✅ | ❌ | ❌ | ✅ | ❌ | ❌ | EE Only |
| [PgBouncer Exporter ](#pgbouncer-exporter ) | Prometheus endpoint with PgBouncer metrics | ⚙ | ✅ | ❌ | ❌ | ✅ | ❌ | ❌ | CE & EE |
| [PgBouncer ](#pgbouncer ) | Database connection pooling, failover | ⚙ | ✅ | ❌ | ❌ | ✅ | ❌ | ❌ | EE Only |
| [PostgreSQL Exporter ](#postgresql-exporter ) | Prometheus endpoint with PostgreSQL metrics | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | CE & EE |
| [PostgreSQL ](#postgresql ) | Database | ✅ | ✅ | ✅ | ✅ | ✅ | ⤓ | ✅ | CE & EE |
| [Praefect ](#praefect ) | A transparent proxy between any Git client and Gitaly storage nodes. | ✅ | ✅ | ⚙ | ❌ | ✅ | ⚙ | ✅ | CE & EE |
| [Puma (GitLab Rails) ](#puma ) | Handles requests for the web interface and API | ✅ | ✅ | ✅ | ✅ | ✅ | ⚙ | ✅ | CE & EE |
| [Redis Exporter ](#redis-exporter ) | Prometheus endpoint with Redis metrics | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | CE & EE |
| [Redis ](#redis ) | Caching service | ✅ | ✅ | ✅ | ✅ | ✅ | ⤓ | ✅ | CE & EE |
| [Registry ](#registry ) | Container registry, allows pushing and pulling of images | ⚙ | ⚙ | ✅ | ✅ | ✅ | ⤓ | ⚙ | CE & EE |
| [Runner ](#gitlab-runner ) | Executes GitLab CI/CD jobs | ⤓ | ⤓ | ✅ | ⚙ | ✅ | ⚙ | ⚙ | CE & EE |
| [Sentry integration ](#sentry ) | Error tracking for deployed apps | ⤓ | ⤓ | ⤓ | ⤓ | ⤓ | ⤓ | ⤓ | CE & EE |
| [Sidekiq ](#sidekiq ) | Background jobs processor | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | CE & EE |
2023-04-23 21:23:45 +05:30
| [Token Revocation API ](sec/token_revocation_api.md ) | Receives and revokes leaked secrets | ❌ | ❌ | ❌ | ❌ | ✅ | ❌ | ❌ | EE Only |
2019-09-04 21:01:54 +05:30
### Component details
This document is designed to be consumed by systems administrators and GitLab Support Engineers who want to understand more about the internals of GitLab and how they work together.
When deployed, GitLab should be considered the amalgamation of the below processes. When troubleshooting or debugging, be as specific as possible as to which component you are referencing. That should increase clarity and reduce confusion.
**Layers**
2015-04-26 12:48:37 +05:30
2019-02-15 15:39:39 +05:30
GitLab can be considered to have two layers from a process perspective:
2015-04-26 12:48:37 +05:30
2021-02-22 17:27:13 +05:30
- **Monitoring**: Anything from this layer is not required to deliver GitLab the application, but allows administrators more insight into their infrastructure and what the service as a whole is doing.
- **Core**: Any process that is vital for the delivery of GitLab as a platform. If any of these processes halt, a GitLab outage results. For the Core layer, you can further divide into:
2019-02-15 15:39:39 +05:30
- **Processors**: These processes are responsible for actually performing operations and presenting the service.
- **Data**: These services store/expose structured data for the GitLab service.
2017-08-17 22:00:37 +05:30
2019-09-04 21:01:54 +05:30
#### Alertmanager
2015-04-26 12:48:37 +05:30
2019-09-04 21:01:54 +05:30
- [Project page ](https://github.com/prometheus/alertmanager/blob/master/README.md )
2020-04-22 19:07:51 +05:30
- Configuration:
- [Omnibus ](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-config-template/gitlab.rb.template )
- [Charts ](https://github.com/helm/charts/tree/master/stable/prometheus )
2019-02-15 15:39:39 +05:30
- Layer: Monitoring
2019-09-04 21:01:54 +05:30
- Process: `alertmanager`
2020-04-22 19:07:51 +05:30
- GitLab.com: [Monitoring of GitLab.com ](https://about.gitlab.com/handbook/engineering/monitoring/ )
2019-09-04 21:01:54 +05:30
2021-02-22 17:27:13 +05:30
[Alert manager ](https://prometheus.io/docs/alerting/latest/alertmanager/ ) is a tool provided by Prometheus that _"handles alerts sent by client applications such as the Prometheus server. It takes care of deduplicating, grouping, and routing them to the correct receiver integration such as email, PagerDuty, or Opsgenie. It also takes care of silencing and inhibition of alerts."_ You can read more in [issue #45740 ](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/45740 ) about what we alert on.
2019-09-04 21:01:54 +05:30
#### Certificate management
2020-04-22 19:07:51 +05:30
- Project page:
- [Omnibus ](https://github.com/certbot/certbot/blob/master/README.rst )
- [Charts ](https://github.com/jetstack/cert-manager/blob/master/README.md )
- Configuration:
2023-07-09 08:55:56 +05:30
- [Omnibus ](https://docs.gitlab.com/omnibus/settings/ssl/index.html )
2020-04-22 19:07:51 +05:30
- [Charts ](https://docs.gitlab.com/charts/installation/tls.html )
- [Source ](../install/installation.md#using-https )
2021-06-08 01:23:25 +05:30
- [GDK ](https://gitlab.com/gitlab-org/gitlab-development-kit/blob/main/doc/howto/https.md )
2019-09-04 21:01:54 +05:30
- Layer: Core Service (Processor)
2020-04-22 19:07:51 +05:30
- GitLab.com: [Secrets Management ](https://about.gitlab.com/handbook/engineering/infrastructure/production/architecture/#secrets-management )
2019-09-04 21:01:54 +05:30
#### Consul
- [Project page ](https://github.com/hashicorp/consul/blob/master/README.md )
2020-04-22 19:07:51 +05:30
- Configuration:
2020-10-24 23:57:45 +05:30
- [Omnibus ](../administration/consul.md )
2020-04-22 19:07:51 +05:30
- [Charts ](https://docs.gitlab.com/charts/installation/deployment.html#postgresql )
2019-09-04 21:01:54 +05:30
- Layer: Core Service (Data)
2020-04-22 19:07:51 +05:30
- GitLab.com: [Consul ](../user/gitlab_com/index.md#consul )
2019-09-04 21:01:54 +05:30
Consul is a tool for service discovery and configuration. Consul is distributed, highly available, and extremely scalable.
#### Database migrations
2020-04-22 19:07:51 +05:30
- Configuration:
- [Omnibus ](https://docs.gitlab.com/omnibus/settings/database.html#disabling-automatic-database-migration )
- [Charts ](https://docs.gitlab.com/charts/charts/gitlab/migrations/ )
2021-04-17 20:07:23 +05:30
- [Source ](../update/upgrading_from_source.md#10-install-libraries-migrations-etc )
2019-09-04 21:01:54 +05:30
- Layer: Core Service (Data)
#### Elasticsearch
2020-04-22 19:07:51 +05:30
- [Project page ](https://github.com/elastic/elasticsearch/ )
- Configuration:
2022-07-23 23:45:48 +05:30
- [Omnibus ](../integration/advanced_search/elasticsearch.md )
- [Charts ](../integration/advanced_search/elasticsearch.md )
- [Source ](../integration/advanced_search/elasticsearch.md )
2021-06-08 01:23:25 +05:30
- [GDK ](https://gitlab.com/gitlab-org/gitlab-development-kit/blob/main/doc/howto/elasticsearch.md )
2019-09-04 21:01:54 +05:30
- Layer: Core Service (Data)
2023-05-27 22:25:52 +05:30
- GitLab.com: [Get advanced search working on GitLab.com (Closed) ](https://gitlab.com/groups/gitlab-org/-/epics/153 ) epic.
2019-02-15 15:39:39 +05:30
2019-09-04 21:01:54 +05:30
Elasticsearch is a distributed RESTful search engine built for the cloud.
2019-02-15 15:39:39 +05:30
2019-09-04 21:01:54 +05:30
#### Gitaly
2019-02-15 15:39:39 +05:30
2019-09-04 21:01:54 +05:30
- [Project page ](https://gitlab.com/gitlab-org/gitaly/blob/master/README.md )
2020-04-22 19:07:51 +05:30
- Configuration:
- [Omnibus ](../administration/gitaly/index.md )
- [Charts ](https://docs.gitlab.com/charts/charts/gitlab/gitaly/ )
- [Source ](../install/installation.md#install-gitaly )
2019-02-15 15:39:39 +05:30
- Layer: Core Service (Data)
2019-09-04 21:01:54 +05:30
- Process: `gitaly`
2020-04-22 19:07:51 +05:30
- GitLab.com: [Service Architecture ](https://about.gitlab.com/handbook/engineering/infrastructure/production/architecture/#service-architecture )
2019-02-15 15:39:39 +05:30
2020-06-23 00:09:42 +05:30
Gitaly is a service designed by GitLab to remove our need for NFS for Git storage in distributed deployments of GitLab (think GitLab.com or High Availability Deployments). As of 11.3.0, this service handles all Git level access in GitLab. You can read more about the project [in the project's README ](https://gitlab.com/gitlab-org/gitaly ).
2019-02-15 15:39:39 +05:30
2020-03-13 15:44:24 +05:30
#### Praefect
- [Project page ](https://gitlab.com/gitlab-org/gitaly/blob/master/README.md )
2020-04-22 19:07:51 +05:30
- Configuration:
- [Omnibus ](../administration/gitaly/index.md )
- [Source ](../install/installation.md#install-gitaly )
2020-03-13 15:44:24 +05:30
- Layer: Core Service (Data)
- Process: `praefect`
2020-04-22 19:07:51 +05:30
- GitLab.com: [Service Architecture ](https://about.gitlab.com/handbook/engineering/infrastructure/production/architecture/#service-architecture )
2020-03-13 15:44:24 +05:30
Praefect is a transparent proxy between each Git client and the Gitaly coordinating the replication of
2020-04-22 19:07:51 +05:30
repository updates to secondary nodes.
2020-03-13 15:44:24 +05:30
2019-12-04 20:38:33 +05:30
#### GitLab Geo
2019-02-15 15:39:39 +05:30
2020-04-22 19:07:51 +05:30
- Configuration:
2020-11-24 15:15:51 +05:30
- [Omnibus ](../administration/geo/setup/index.md )
2020-07-28 23:09:34 +05:30
- [Charts ](https://docs.gitlab.com/charts/advanced/geo/ )
2021-06-08 01:23:25 +05:30
- [GDK ](https://gitlab.com/gitlab-org/gitlab-development-kit/blob/main/doc/howto/geo.md )
2019-09-04 21:01:54 +05:30
- Layer: Core Service (Processor)
2021-06-08 01:23:25 +05:30
Geo is a premium feature built to help speed up the development of distributed teams by providing one or more read-only mirrors of a primary GitLab instance. This mirror (a Geo secondary site) reduces the time to clone or fetch large repositories and projects, or can be part of a Disaster Recovery solution.
2019-12-21 20:55:43 +05:30
#### GitLab Exporter
2019-09-04 21:01:54 +05:30
2019-12-04 20:38:33 +05:30
- [Project page ](https://gitlab.com/gitlab-org/gitlab-exporter )
2020-04-22 19:07:51 +05:30
- Configuration:
- [Omnibus ](../administration/monitoring/prometheus/gitlab_exporter.md )
- [Charts ](https://docs.gitlab.com/charts/charts/gitlab/gitlab-exporter/index.html )
2019-07-07 11:18:12 +05:30
- Layer: Monitoring
2019-12-04 20:38:33 +05:30
- Process: `gitlab-exporter`
2020-04-22 19:07:51 +05:30
- GitLab.com: [Monitoring of GitLab.com ](https://about.gitlab.com/handbook/engineering/monitoring/ )
2019-09-04 21:01:54 +05:30
2020-06-23 00:09:42 +05:30
GitLab Exporter is a process designed in house that allows us to export metrics about GitLab application internals to Prometheus. You can read more [in the project's README ](https://gitlab.com/gitlab-org/gitlab-exporter ).
2019-09-04 21:01:54 +05:30
2022-05-07 20:08:51 +05:30
#### GitLab agent
2021-01-03 14:25:43 +05:30
- [Project page ](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent )
- Configuration:
- [Omnibus ](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-config-template/gitlab.rb.template )
- [Charts ](https://docs.gitlab.com/charts/charts/gitlab/kas/index.html )
2022-05-07 20:08:51 +05:30
The [GitLab agent ](../user/clusters/agent/index.md ) is an active in-cluster
2021-01-03 14:25:43 +05:30
component for solving GitLab and Kubernetes integration tasks in a secure and
cloud-native way.
You can use it to sync deployments onto your Kubernetes cluster.
2019-12-04 20:38:33 +05:30
#### GitLab Pages
2019-09-04 21:01:54 +05:30
2020-04-22 19:07:51 +05:30
- Configuration:
- [Omnibus ](../administration/pages/index.md )
2020-06-23 00:09:42 +05:30
- [Charts ](https://gitlab.com/gitlab-org/charts/gitlab/-/issues/37 )
2020-04-22 19:07:51 +05:30
- [Source ](../install/installation.md#install-gitlab-pages )
2021-06-08 01:23:25 +05:30
- [GDK ](https://gitlab.com/gitlab-org/gitlab-development-kit/blob/main/doc/howto/pages.md )
2019-09-04 21:01:54 +05:30
- Layer: Core Service (Processor)
2020-04-22 19:07:51 +05:30
- GitLab.com: [GitLab Pages ](../user/gitlab_com/index.md#gitlab-pages )
2019-09-04 21:01:54 +05:30
GitLab Pages is a feature that allows you to publish static websites directly from a repository in GitLab.
You can use it either for personal or business websites, such as portfolios, documentation, manifestos, and business presentations. You can also attribute any license to your content.
2019-12-04 20:38:33 +05:30
#### GitLab Runner
2019-09-04 21:01:54 +05:30
2022-04-04 11:22:00 +05:30
- [Project page ](https://gitlab.com/gitlab-org/gitlab-runner/blob/main/README.md )
2020-04-22 19:07:51 +05:30
- Configuration:
- [Omnibus ](https://docs.gitlab.com/runner/ )
- [Charts ](https://docs.gitlab.com/runner/install/kubernetes.html )
- [Source ](https://docs.gitlab.com/runner/ )
2021-06-08 01:23:25 +05:30
- [GDK ](https://gitlab.com/gitlab-org/gitlab-development-kit/blob/main/doc/howto/runner.md )
2019-09-04 21:01:54 +05:30
- Layer: Core Service (Processor)
2021-10-27 15:23:28 +05:30
- GitLab.com: [Runners ](../ci/runners/index.md )
2019-02-15 15:39:39 +05:30
2020-04-22 19:07:51 +05:30
GitLab Runner runs jobs and sends the results to GitLab.
2019-02-15 15:39:39 +05:30
2020-04-22 19:07:51 +05:30
GitLab CI/CD is the open-source continuous integration service included with GitLab that coordinates the testing. The old name of this project was `GitLab CI Multi Runner` but please use `GitLab Runner` (without CI) from now on.
2019-02-15 15:39:39 +05:30
2019-12-04 20:38:33 +05:30
#### GitLab Shell
2019-09-04 21:01:54 +05:30
2023-03-17 16:20:25 +05:30
- [Project page ](https://gitlab.com/gitlab-org/gitlab-shell/ )
- [Documentation ](gitlab_shell/index.md )
2020-04-22 19:07:51 +05:30
- Configuration:
- [Omnibus ](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-config-template/gitlab.rb.template )
- [Charts ](https://docs.gitlab.com/charts/charts/gitlab/gitlab-shell/ )
- [Source ](../install/installation.md#install-gitlab-shell )
2021-09-04 01:27:46 +05:30
- [GDK ](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/gitlab.yml.example )
2019-09-04 21:01:54 +05:30
- Layer: Core Service (Processor)
2020-04-22 19:07:51 +05:30
- GitLab.com: [Service Architecture ](https://about.gitlab.com/handbook/engineering/infrastructure/production/architecture/#service-architecture )
2019-09-04 21:01:54 +05:30
2023-03-17 16:20:25 +05:30
[GitLab Shell ](gitlab_shell/index.md ) is a program designed at GitLab to handle SSH-based `git` sessions, and modifies the list of authorized keys. GitLab Shell is not a Unix shell nor a replacement for Bash or Zsh.
2019-09-04 21:01:54 +05:30
2019-12-04 20:38:33 +05:30
#### GitLab Workhorse
2019-09-04 21:01:54 +05:30
2023-06-20 00:43:36 +05:30
- [Project page ](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/development/workhorse/index.md )
2020-04-22 19:07:51 +05:30
- Configuration:
- [Omnibus ](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-config-template/gitlab.rb.template )
2021-01-03 14:25:43 +05:30
- [Charts ](https://docs.gitlab.com/charts/charts/gitlab/webservice/ )
2020-04-22 19:07:51 +05:30
- [Source ](../install/installation.md#install-gitlab-workhorse )
2019-02-15 15:39:39 +05:30
- Layer: Core Service (Processor)
2019-09-04 21:01:54 +05:30
- Process: `gitlab-workhorse`
2020-04-22 19:07:51 +05:30
- GitLab.com: [Service Architecture ](https://about.gitlab.com/handbook/engineering/infrastructure/production/architecture/#service-architecture )
2019-09-04 21:01:54 +05:30
2023-06-20 00:43:36 +05:30
[GitLab Workhorse ](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/development/workhorse ) is a program designed at GitLab to help alleviate pressure from Puma. You can read more about the [historical reasons for developing ](https://about.gitlab.com/blog/2016/04/12/a-brief-history-of-gitlab-workhorse/ ). It's designed to act as a smart reverse proxy to help speed up GitLab as a whole.
2019-09-04 21:01:54 +05:30
#### Grafana
- [Project page ](https://github.com/grafana/grafana/blob/master/README.md )
2020-04-22 19:07:51 +05:30
- Configuration:
- [Omnibus ](../administration/monitoring/performance/grafana_configuration.md )
2020-07-28 23:09:34 +05:30
- [Charts ](https://docs.gitlab.com/charts/charts/globals#configure-grafana-integration )
2019-09-04 21:01:54 +05:30
- Layer: Monitoring
2020-04-22 19:07:51 +05:30
- GitLab.com: [GitLab triage Grafana dashboard ](https://dashboards.gitlab.com/d/RZmbBr7mk/gitlab-triage?refresh=30s )
2019-09-04 21:01:54 +05:30
2020-04-08 14:13:33 +05:30
Grafana is an open source, feature rich metrics dashboard and graph editor for Graphite, Elasticsearch, OpenTSDB, Prometheus, and InfluxDB.
2019-09-04 21:01:54 +05:30
#### Jaeger
- [Project page ](https://github.com/jaegertracing/jaeger/blob/master/README.md )
2020-04-22 19:07:51 +05:30
- Configuration:
2020-06-23 00:09:42 +05:30
- [Omnibus ](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/4104 )
2020-07-28 23:09:34 +05:30
- [Charts ](https://docs.gitlab.com/charts/charts/globals#tracing )
2020-04-22 19:07:51 +05:30
- [Source ](../development/distributed_tracing.md#enabling-distributed-tracing )
- [GDK ](../development/distributed_tracing.md#using-jaeger-in-the-gitlab-development-kit )
2019-09-04 21:01:54 +05:30
- Layer: Monitoring
2020-06-23 00:09:42 +05:30
- GitLab.com: [Configuration to enable Tracing for a GitLab instance ](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/4104 ) issue.
2019-02-15 15:39:39 +05:30
2020-04-22 19:07:51 +05:30
Jaeger, inspired by Dapper and OpenZipkin, is a distributed tracing system.
It can be used for monitoring microservices-based distributed systems.
2019-09-04 21:01:54 +05:30
#### Logrotate
2019-02-15 15:39:39 +05:30
2019-09-04 21:01:54 +05:30
- [Project page ](https://github.com/logrotate/logrotate/blob/master/README.md )
2020-04-22 19:07:51 +05:30
- Configuration:
- [Omnibus ](https://docs.gitlab.com/omnibus/settings/logs.html#logrotate )
2019-02-15 15:39:39 +05:30
- Layer: Core Service
2019-09-04 21:01:54 +05:30
- Process: `logrotate`
2021-11-11 11:23:49 +05:30
GitLab is comprised of a large number of services that all log. We bundle our own Logrotate
to make sure we were logging responsibly. This is just a packaged version of the common open source offering.
2019-09-04 21:01:54 +05:30
#### Mattermost
- [Project page ](https://github.com/mattermost/mattermost-server/blob/master/README.md )
2020-04-22 19:07:51 +05:30
- Configuration:
2021-11-11 11:23:49 +05:30
- [Omnibus ](../integration/mattermost/index.md )
2020-04-22 19:07:51 +05:30
- [Charts ](https://docs.mattermost.com/install/install-mmte-helm-gitlab-helm.html )
2019-09-04 21:01:54 +05:30
- Layer: Core Service (Processor)
2020-04-22 19:07:51 +05:30
- GitLab.com: [Mattermost ](../user/project/integrations/mattermost.md )
2019-09-04 21:01:54 +05:30
2019-09-30 21:07:59 +05:30
Mattermost is an open source, private cloud, Slack-alternative from < https: / / mattermost . com > .
2019-09-04 21:01:54 +05:30
#### MinIO
- [Project page ](https://github.com/minio/minio/blob/master/README.md )
2020-04-22 19:07:51 +05:30
- Configuration:
- [Omnibus ](https://min.io/download )
- [Charts ](https://docs.gitlab.com/charts/charts/minio/ )
2021-06-08 01:23:25 +05:30
- [GDK ](https://gitlab.com/gitlab-org/gitlab-development-kit/blob/main/doc/howto/object_storage.md )
2019-09-04 21:01:54 +05:30
- Layer: Core Service (Data)
2020-04-22 19:07:51 +05:30
- GitLab.com: [Storage Architecture ](https://about.gitlab.com/handbook/engineering/infrastructure/production/architecture/#storage-architecture )
2019-02-15 15:39:39 +05:30
2023-03-04 22:38:38 +05:30
MinIO is an object storage server released under the GNU AGPL v3.0. It is compatible with Amazon S3 cloud storage service. It is best suited for storing unstructured data such as photos, videos, log files, backups, and container / VM images. Size of an object can range from a few KB to a maximum of 5 TB.
2019-02-15 15:39:39 +05:30
2019-09-04 21:01:54 +05:30
#### NGINX
2019-02-15 15:39:39 +05:30
2020-04-22 19:07:51 +05:30
- Project page:
- [Omnibus ](https://github.com/nginx/nginx )
- [Charts ](https://github.com/kubernetes/ingress-nginx/blob/master/README.md )
- Configuration:
- [Omnibus ](https://docs.gitlab.com/omnibus/settings/ )
- [Charts ](https://docs.gitlab.com/charts/charts/nginx/ )
2023-07-09 08:55:56 +05:30
- [Source ](../install/installation.md#10-nginx )
2019-02-15 15:39:39 +05:30
- Layer: Core Service (Processor)
2019-09-04 21:01:54 +05:30
- Process: `nginx`
2020-04-22 19:07:51 +05:30
- GitLab.com: [Service Architecture ](https://about.gitlab.com/handbook/engineering/infrastructure/production/architecture/#service-architecture )
2019-02-15 15:39:39 +05:30
2019-12-21 20:55:43 +05:30
NGINX has an Ingress port for all HTTP requests and routes them to the appropriate sub-systems within GitLab. We are bundling an unmodified version of the popular open source webserver.
2019-02-15 15:39:39 +05:30
2019-09-04 21:01:54 +05:30
#### Node Exporter
2019-02-15 15:39:39 +05:30
2019-09-04 21:01:54 +05:30
- [Project page ](https://github.com/prometheus/node_exporter/blob/master/README.md )
2020-04-22 19:07:51 +05:30
- Configuration:
- [Omnibus ](../administration/monitoring/prometheus/node_exporter.md )
2020-06-23 00:09:42 +05:30
- [Charts ](https://gitlab.com/gitlab-org/charts/gitlab/-/issues/1332 )
2019-02-15 15:39:39 +05:30
- Layer: Monitoring
2019-09-04 21:01:54 +05:30
- Process: `node-exporter`
2020-04-22 19:07:51 +05:30
- GitLab.com: [Monitoring of GitLab.com ](https://about.gitlab.com/handbook/engineering/monitoring/ )
2019-09-04 21:01:54 +05:30
[Node Exporter ](https://github.com/prometheus/node_exporter ) is a Prometheus tool that gives us metrics on the underlying machine (think CPU/Disk/Load). It's just a packaged version of the common open source offering from the Prometheus project.
2021-01-29 00:20:46 +05:30
#### Patroni
- [Project Page ](https://github.com/zalando/patroni )
- Configuration:
- [Omnibus ](../administration/postgresql/replication_and_failover.md#patroni )
- Layer: Core Service (Data)
- Process: `patroni`
- GitLab.com: [Database Architecture ](https://about.gitlab.com/handbook/engineering/infrastructure/production/architecture/#database-architecture )
2019-09-04 21:01:54 +05:30
#### PgBouncer
- [Project page ](https://github.com/pgbouncer/pgbouncer/blob/master/README.md )
2020-04-22 19:07:51 +05:30
- Configuration:
2020-10-24 23:57:45 +05:30
- [Omnibus ](../administration/postgresql/pgbouncer.md )
2020-04-22 19:07:51 +05:30
- [Charts ](https://docs.gitlab.com/charts/installation/deployment.html#postgresql )
2019-09-04 21:01:54 +05:30
- Layer: Core Service (Data)
2020-04-22 19:07:51 +05:30
- GitLab.com: [Database Architecture ](https://about.gitlab.com/handbook/engineering/infrastructure/production/architecture/#database-architecture )
2019-02-15 15:39:39 +05:30
2019-09-04 21:01:54 +05:30
Lightweight connection pooler for PostgreSQL.
2019-02-15 15:39:39 +05:30
2019-09-04 21:01:54 +05:30
#### PgBouncer Exporter
2019-02-15 15:39:39 +05:30
2020-06-23 00:09:42 +05:30
- [Project page ](https://github.com/prometheus-community/pgbouncer_exporter/blob/master/README.md )
2020-04-22 19:07:51 +05:30
- Configuration:
- [Omnibus ](../administration/monitoring/prometheus/pgbouncer_exporter.md )
- [Charts ](https://docs.gitlab.com/charts/installation/deployment.html#postgresql )
2019-02-15 15:39:39 +05:30
- Layer: Monitoring
2020-04-22 19:07:51 +05:30
- GitLab.com: [Monitoring of GitLab.com ](https://about.gitlab.com/handbook/engineering/monitoring/ )
2019-02-15 15:39:39 +05:30
2019-09-04 21:01:54 +05:30
Prometheus exporter for PgBouncer. Exports metrics at 9127/metrics.
2019-02-15 15:39:39 +05:30
2019-12-21 20:55:43 +05:30
#### PostgreSQL
2019-02-15 15:39:39 +05:30
2019-09-04 21:01:54 +05:30
- [Project page ](https://github.com/postgres/postgres/blob/master/README )
2020-04-22 19:07:51 +05:30
- Configuration:
- [Omnibus ](https://docs.gitlab.com/omnibus/settings/database.html )
2020-07-28 23:09:34 +05:30
- [Charts ](https://docs.gitlab.com/charts/installation/deployment.html#postgresql )
2023-07-09 08:55:56 +05:30
- [Source ](../install/installation.md#7-database )
2019-02-15 15:39:39 +05:30
- Layer: Core Service (Data)
2019-09-04 21:01:54 +05:30
- Process: `postgresql`
2023-05-27 22:25:52 +05:30
- GitLab.com: [PostgreSQL ](https://about.gitlab.com/handbook/engineering/infrastructure/database/ )
2019-02-15 15:39:39 +05:30
2019-07-07 11:18:12 +05:30
GitLab packages the popular Database to provide storage for Application meta data and user information.
2019-02-15 15:39:39 +05:30
2020-04-22 19:07:51 +05:30
#### PostgreSQL Exporter
2019-09-04 21:01:54 +05:30
- [Project page ](https://github.com/wrouesnel/postgres_exporter/blob/master/README.md )
2020-04-22 19:07:51 +05:30
- Configuration:
- [Omnibus ](../administration/monitoring/prometheus/postgres_exporter.md )
2020-07-28 23:09:34 +05:30
- [Charts ](https://docs.gitlab.com/charts/installation/deployment.html#postgresql )
2019-09-04 21:01:54 +05:30
- Layer: Monitoring
- Process: `postgres-exporter`
2020-04-22 19:07:51 +05:30
- GitLab.com: [Monitoring of GitLab.com ](https://about.gitlab.com/handbook/engineering/monitoring/ )
2019-09-04 21:01:54 +05:30
2021-02-22 17:27:13 +05:30
[`postgres_exporter` ](https://github.com/wrouesnel/postgres_exporter ) is the community provided Prometheus exporter that delivers data about PostgreSQL to Prometheus for use in Grafana Dashboards.
2019-02-15 15:39:39 +05:30
2019-09-04 21:01:54 +05:30
#### Prometheus
- [Project page ](https://github.com/prometheus/prometheus/blob/master/README.md )
2020-04-22 19:07:51 +05:30
- Configuration:
- [Omnibus ](../administration/monitoring/prometheus/index.md )
2020-07-28 23:09:34 +05:30
- [Charts ](https://docs.gitlab.com/charts/installation/deployment.html#prometheus )
2019-02-15 15:39:39 +05:30
- Layer: Monitoring
2019-09-04 21:01:54 +05:30
- Process: `prometheus`
2020-04-22 19:07:51 +05:30
- GitLab.com: [Prometheus ](../user/gitlab_com/index.md#prometheus )
2019-02-15 15:39:39 +05:30
Prometheus is a time-series tool that helps GitLab administrators expose metrics about the individual processes used to provide GitLab the service.
2019-09-04 21:01:54 +05:30
#### Redis
2019-02-15 15:39:39 +05:30
2019-09-04 21:01:54 +05:30
- [Project page ](https://github.com/antirez/redis/blob/unstable/README.md )
2020-04-22 19:07:51 +05:30
- Configuration:
- [Omnibus ](https://docs.gitlab.com/omnibus/settings/redis.html )
2020-07-28 23:09:34 +05:30
- [Charts ](https://docs.gitlab.com/charts/installation/deployment.html#redis )
2023-07-09 08:55:56 +05:30
- [Source ](../install/installation.md#8-redis )
2019-02-15 15:39:39 +05:30
- Layer: Core Service (Data)
2019-09-04 21:01:54 +05:30
- Process: `redis`
2020-04-22 19:07:51 +05:30
- GitLab.com: [Service Architecture ](https://about.gitlab.com/handbook/engineering/infrastructure/production/architecture/#service-architecture )
2019-02-15 15:39:39 +05:30
2019-07-07 11:18:12 +05:30
Redis is packaged to provide a place to store:
2019-02-15 15:39:39 +05:30
- session data
- temporary cache information
2019-09-04 21:01:54 +05:30
- background job queues
2021-06-08 01:23:25 +05:30
See our [Redis guidelines ](redis.md ) for more information about how GitLab uses Redis.
2019-09-04 21:01:54 +05:30
#### Redis Exporter
- [Project page ](https://github.com/oliver006/redis_exporter/blob/master/README.md )
2020-04-22 19:07:51 +05:30
- Configuration:
- [Omnibus ](../administration/monitoring/prometheus/redis_exporter.md )
2020-07-28 23:09:34 +05:30
- [Charts ](https://docs.gitlab.com/charts/installation/deployment.html#redis )
2019-09-04 21:01:54 +05:30
- Layer: Monitoring
- Process: `redis-exporter`
2020-04-22 19:07:51 +05:30
- GitLab.com: [Monitoring of GitLab.com ](https://about.gitlab.com/handbook/engineering/monitoring/ )
2019-02-15 15:39:39 +05:30
2019-09-04 21:01:54 +05:30
[Redis Exporter ](https://github.com/oliver006/redis_exporter ) is designed to give specific metrics about the Redis process to Prometheus so that we can graph these metrics in Grafana.
2019-02-15 15:39:39 +05:30
2019-09-04 21:01:54 +05:30
#### Registry
- [Project page ](https://github.com/docker/distribution/blob/master/README.md )
2020-04-22 19:07:51 +05:30
- Configuration:
2021-04-17 20:07:23 +05:30
- [Omnibus ](../update/upgrading_from_source.md#10-install-libraries-migrations-etc )
2020-04-22 19:07:51 +05:30
- [Charts ](https://docs.gitlab.com/charts/charts/registry/ )
- [Source ](../administration/packages/container_registry.md#enable-the-container-registry )
2021-06-08 01:23:25 +05:30
- [GDK ](https://gitlab.com/gitlab-org/gitlab-development-kit/blob/main/doc/howto/registry.md )
2019-09-04 21:01:54 +05:30
- Layer: Core Service (Processor)
2023-03-17 16:20:25 +05:30
- GitLab.com: [GitLab Container Registry ](../user/packages/container_registry/build_and_push_images.md#use-gitlab-cicd )
2019-09-04 21:01:54 +05:30
The registry is what users use to store their own Docker images. The bundled
2019-12-21 20:55:43 +05:30
registry uses NGINX as a load balancer and GitLab as an authentication manager.
2021-02-22 17:27:13 +05:30
Whenever a client requests to pull or push an image from the registry, it
returns a `401` response along with a header detailing where to get an
authentication token, in this case the GitLab instance. The client then
requests a pull or push auth token from GitLab and retries the original request
2023-04-23 21:23:45 +05:30
to the registry. For more information, see
[token authentication ](https://docs.docker.com/registry/spec/auth/token/ ).
2019-09-04 21:01:54 +05:30
An external registry can also be configured to use GitLab as an auth endpoint.
#### Sentry
2020-04-22 19:07:51 +05:30
- [Project page ](https://github.com/getsentry/sentry/ )
- Configuration:
- [Omnibus ](https://docs.gitlab.com/omnibus/settings/configuration.html#error-reporting-and-logging-with-sentry )
2020-07-28 23:09:34 +05:30
- [Charts ](https://docs.gitlab.com/charts/charts/globals#sentry-settings )
2021-09-04 01:27:46 +05:30
- [Source ](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/gitlab.yml.example )
- [GDK ](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/gitlab.yml.example )
2019-02-15 15:39:39 +05:30
- Layer: Monitoring
2020-04-22 19:07:51 +05:30
- GitLab.com: [Searching Sentry ](https://about.gitlab.com/handbook/support/workflows/500_errors.html#searching-sentry )
2020-06-23 00:09:42 +05:30
Sentry fundamentally is a service that helps you monitor and fix crashes in real time.
2020-04-22 19:07:51 +05:30
The server is in Python, but it contains a full API for sending events from any language, in any application.
2019-02-15 15:39:39 +05:30
2020-10-24 23:57:45 +05:30
For monitoring deployed apps, see the [Sentry integration docs ](../operations/error_tracking.md )
2019-02-15 15:39:39 +05:30
2019-09-04 21:01:54 +05:30
#### Sidekiq
2019-02-15 15:39:39 +05:30
2019-09-04 21:01:54 +05:30
- [Project page ](https://github.com/mperham/sidekiq/blob/master/README.md )
2020-04-22 19:07:51 +05:30
- Configuration:
- [Omnibus ](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-config-template/gitlab.rb.template )
- [Charts ](https://docs.gitlab.com/charts/charts/gitlab/sidekiq/ )
2022-03-02 08:16:31 +05:30
- [minikube Minimal ](https://docs.gitlab.com/charts/charts/gitlab/sidekiq/index.html )
2021-09-04 01:27:46 +05:30
- [Source ](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/gitlab.yml.example )
- [GDK ](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/gitlab.yml.example )
2019-02-15 15:39:39 +05:30
- Layer: Core Service (Processor)
2019-09-04 21:01:54 +05:30
- Process: `sidekiq`
2020-08-09 01:23:26 +05:30
- GitLab.com: [Sidekiq ](../user/gitlab_com/index.md#sidekiq )
2019-02-15 15:39:39 +05:30
2019-12-21 20:55:43 +05:30
Sidekiq is a Ruby background job processor that pulls jobs from the Redis queue and processes them. Background jobs allow GitLab to provide a faster request/response cycle by moving work into the background.
2019-02-15 15:39:39 +05:30
2021-01-03 14:25:43 +05:30
#### Puma
2021-09-04 01:27:46 +05:30
Starting with GitLab 13.0, Puma is the default web server.
2021-01-03 14:25:43 +05:30
2021-09-04 01:27:46 +05:30
- [Project page ](https://gitlab.com/gitlab-org/gitlab/-/blob/master/README.md )
2021-01-03 14:25:43 +05:30
- Configuration:
2021-11-18 22:05:49 +05:30
- [Omnibus ](../administration/operations/puma.md )
2021-01-03 14:25:43 +05:30
- [Charts ](https://docs.gitlab.com/charts/charts/gitlab/webservice/ )
- [Source ](../install/installation.md#configure-it )
2021-09-04 01:27:46 +05:30
- [GDK ](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/gitlab.yml.example )
2021-01-03 14:25:43 +05:30
- Layer: Core Service (Processor)
- Process: `puma`
- GitLab.com: [Puma ](../user/gitlab_com/index.md#puma )
2021-02-22 17:27:13 +05:30
[Puma ](https://puma.io/ ) is a Ruby application server that is used to run the core Rails Application that provides the user facing features in GitLab. Often this displays in process output as `bundle` or `config.ru` depending on the GitLab version.
2021-01-03 14:25:43 +05:30
2019-09-04 21:01:54 +05:30
#### LDAP Authentication
2019-02-15 15:39:39 +05:30
2020-04-22 19:07:51 +05:30
- Configuration:
2020-06-23 00:09:42 +05:30
- [Omnibus ](../administration/auth/ldap/index.md )
2020-04-22 19:07:51 +05:30
- [Charts ](https://docs.gitlab.com/charts/charts/globals.html#ldap )
2021-09-04 01:27:46 +05:30
- [Source ](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/gitlab.yml.example )
2021-06-08 01:23:25 +05:30
- [GDK ](https://gitlab.com/gitlab-org/gitlab-development-kit/blob/main/doc/howto/ldap.md )
2019-09-04 21:01:54 +05:30
- Layer: Core Service (Processor)
2020-04-22 19:07:51 +05:30
- GitLab.com: [Product Tiers ](https://about.gitlab.com/pricing/#gitlab-com )
2019-02-15 15:39:39 +05:30
2019-09-04 21:01:54 +05:30
#### Outbound Email
2019-02-15 15:39:39 +05:30
2020-04-22 19:07:51 +05:30
- Configuration:
- [Omnibus ](https://docs.gitlab.com/omnibus/settings/smtp.html )
- [Charts ](https://docs.gitlab.com/charts/installation/command-line-options.html#outgoing-email-configuration )
2021-09-04 01:27:46 +05:30
- [Source ](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/gitlab.yml.example )
- [GDK ](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/gitlab.yml.example )
2019-09-04 21:01:54 +05:30
- Layer: Core Service (Processor)
2020-04-22 19:07:51 +05:30
- GitLab.com: [Mail configuration ](../user/gitlab_com/index.md#mail-configuration )
2019-02-15 15:39:39 +05:30
2019-09-04 21:01:54 +05:30
#### Inbound Email
2019-02-15 15:39:39 +05:30
2020-04-22 19:07:51 +05:30
- Configuration:
- [Omnibus ](../administration/incoming_email.md )
- [Charts ](https://docs.gitlab.com/charts/installation/command-line-options.html#incoming-email-configuration )
2021-09-04 01:27:46 +05:30
- [Source ](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/gitlab.yml.example )
- [GDK ](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/gitlab.yml.example )
2019-09-04 21:01:54 +05:30
- Layer: Core Service (Processor)
2020-04-22 19:07:51 +05:30
- GitLab.com: [Mail configuration ](../user/gitlab_com/index.md#mail-configuration )
2019-07-07 11:18:12 +05:30
2021-01-03 14:25:43 +05:30
## GitLab by request type
2019-02-15 15:39:39 +05:30
GitLab provides two "interfaces" for end users to access the service:
- Web HTTP Requests (Viewing the UI/API)
- Git HTTP/SSH Requests (Pushing/Pulling Git Data)
It's important to understand the distinction as some processes are used in both and others are exclusive to a specific request type.
2021-01-03 14:25:43 +05:30
### GitLab Web HTTP request cycle
2019-02-15 15:39:39 +05:30
2021-02-22 17:27:13 +05:30
When making a request to an HTTP Endpoint (think `/users/sign_in` ) the request takes the following path through the GitLab Service:
2019-02-15 15:39:39 +05:30
2019-12-21 20:55:43 +05:30
- NGINX - Acts as our first line reverse proxy.
2021-01-03 14:25:43 +05:30
- GitLab Workhorse - This determines if it needs to go to the Rails application or somewhere else to reduce load on Puma.
2021-02-22 17:27:13 +05:30
- Puma - Since this is a web request, and it needs to access the application, it routes to Puma.
2020-04-22 19:07:51 +05:30
- PostgreSQL/Gitaly/Redis - Depending on the type of request, it may hit these services to store or retrieve data.
2019-02-15 15:39:39 +05:30
2021-01-03 14:25:43 +05:30
### GitLab Git request cycle
2019-02-15 15:39:39 +05:30
2021-02-22 17:27:13 +05:30
Below we describe the different paths that HTTP vs. SSH Git requests take. There is some overlap with the Web Request Cycle but also some differences.
2019-02-15 15:39:39 +05:30
2021-01-03 14:25:43 +05:30
### Web request (80/443)
2019-09-30 21:07:59 +05:30
2020-04-22 19:07:51 +05:30
Git operations over HTTP use the stateless "smart" protocol described in the
[Git documentation ](https://git-scm.com/docs/http-protocol ), but responsibility
for handling these operations is split across several GitLab components.
Here is a sequence diagram for `git fetch` . Note that all requests pass through
NGINX as well as any other HTTP load balancers, but are not transformed in any
way by them. All paths are presented relative to a `/namespace/project.git` URL.
```mermaid
sequenceDiagram
participant Git on client
participant NGINX
participant Workhorse
participant Rails
participant Gitaly
participant Git on server
Note left of Git on client: git fetch< br / > info-refs
Git on client->>+Workhorse: GET /info/refs?service=git-upload-pack
Workhorse->>+Rails: GET /info/refs?service=git-upload-pack
Note right of Rails: Auth check
Rails-->>-Workhorse: Gitlab::Workhorse.git_http_ok
Workhorse->>+Gitaly: SmartHTTPService.InfoRefsUploadPack request
Gitaly->>+Git on server: git upload-pack --stateless-rpc --advertise-refs
Git on server-->>-Gitaly: git upload-pack response
Gitaly-->>-Workhorse: SmartHTTPService.InfoRefsUploadPack response
Workhorse-->>-Git on client: 200 OK
Note left of Git on client: git fetch< br / > fetch-pack
Git on client->>+Workhorse: POST /git-upload-pack
Workhorse->>+Rails: POST /git-upload-pack
Note right of Rails: Auth check
Rails-->>-Workhorse: Gitlab::Workhorse.git_http_ok
Workhorse->>+Gitaly: SmartHTTPService.PostUploadPack request
Gitaly->>+Git on server: git upload-pack --stateless-rpc
Git on server-->>-Gitaly: git upload-pack response
Gitaly-->>-Workhorse: SmartHTTPService.PostUploadPack response
Workhorse-->>-Git on client: 200 OK
```
The sequence is similar for `git push` , except `git-receive-pack` is used
instead of `git-upload-pack` .
2019-02-15 15:39:39 +05:30
2021-01-03 14:25:43 +05:30
### SSH request (22)
2019-09-30 21:07:59 +05:30
2020-04-22 19:07:51 +05:30
Git operations over SSH can use the stateful protocol described in the
[Git documentation ](https://git-scm.com/docs/pack-protocol#_ssh_transport ), but
responsibility for handling them is split across several GitLab components.
No GitLab components speak SSH directly - all SSH connections are made between
Git on the client machine and the SSH server, which terminates the connection.
To the SSH server, all connections are authenticated as the `git` user; GitLab
users are differentiated by the SSH key presented by the client.
Here is a sequence diagram for `git fetch` , assuming [Fast SSH key lookup ](../administration/operations/fast_ssh_key_lookup.md )
is enabled. Note that `AuthorizedKeysCommand` is an executable provided by
[GitLab Shell ](#gitlab-shell ):
```mermaid
sequenceDiagram
participant Git on client
participant SSH server
participant AuthorizedKeysCommand
participant GitLab Shell
participant Rails
participant Gitaly
participant Git on server
Note left of Git on client: git fetch
Git on client->>+SSH server: ssh git fetch-pack request
SSH server->>+AuthorizedKeysCommand: gitlab-shell-authorized-keys-check git AAAA...
AuthorizedKeysCommand->>+Rails: GET /internal/api/authorized_keys?key=AAAA...
Note right of Rails: Lookup key ID
Rails-->>-AuthorizedKeysCommand: 200 OK, command="gitlab-shell upload-pack key_id=1"
AuthorizedKeysCommand-->>-SSH server: command="gitlab-shell upload-pack key_id=1"
SSH server->>+GitLab Shell: gitlab-shell upload-pack key_id=1
GitLab Shell->>+Rails: GET /internal/api/allowed?action=upload_pack& key_id=1
Note right of Rails: Auth check
Rails-->>-GitLab Shell: 200 OK, { gitaly: ... }
GitLab Shell->>+Gitaly: SSHService.SSHUploadPack request
Gitaly->>+Git on server: git upload-pack request
Note over Git on client,Git on server: Bidirectional communication between Git client and server
Git on server-->>-Gitaly: git upload-pack response
Gitaly -->>-GitLab Shell: SSHService.SSHUploadPack response
GitLab Shell-->>-SSH server: gitlab-shell upload-pack response
SSH server-->>-Git on client: ssh git fetch-pack response
```
The `git push` operation is very similar, except `git receive-pack` is used
instead of `git upload-pack` .
If fast SSH key lookups are not enabled, the SSH server reads from the
`~git/.ssh/authorized_keys` file to determine what command to run for a given
2021-09-04 01:27:46 +05:30
SSH session. This is kept up to date by an [`AuthorizedKeysWorker` ](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/workers/authorized_keys_worker.rb )
2020-04-22 19:07:51 +05:30
in Rails, scheduled to run whenever an SSH key is modified by a user.
[SSH certificates ](../administration/operations/ssh_certificates.md ) may be used
instead of keys. In this case, `AuthorizedKeysCommand` is replaced with an
`AuthorizedPrincipalsCommand` . This extracts a username from the certificate
without using the Rails internal API, which is used instead of `key_id` in the
2021-12-11 22:18:48 +05:30
[`/api/internal/allowed` ](internal_api/index.md ) call later.
2020-04-22 19:07:51 +05:30
GitLab Shell also has a few operations that do not involve Gitaly, such as
resetting two-factor authentication codes. These are handled in the same way,
except there is no round-trip into Gitaly - Rails performs the action as part
2021-12-11 22:18:48 +05:30
of the [internal API ](internal_api/index.md ) call, and GitLab Shell streams the
2020-04-22 19:07:51 +05:30
response back to the user directly.
2015-04-26 12:48:37 +05:30
2021-01-03 14:25:43 +05:30
## System layout
2014-09-02 18:07:02 +05:30
2019-12-04 20:38:33 +05:30
When referring to `~git` in the pictures it means the home directory of the Git user which is typically `/home/git` .
2014-09-02 18:07:02 +05:30
2020-07-28 23:09:34 +05:30
GitLab is primarily installed within the `/home/git` user home directory as `git` user. Within the home directory is where the GitLab server software resides as well as the repositories (though the repository location is configurable).
2014-09-02 18:07:02 +05:30
2020-04-22 19:07:51 +05:30
The bare repositories are located in `/home/git/repositories` . GitLab is a Ruby on rails application so the particulars of the inner workings can be learned by studying how a Ruby on rails application works.
2014-09-02 18:07:02 +05:30
2019-12-04 20:38:33 +05:30
To serve repositories over SSH there's an add-on application called GitLab Shell which is installed in `/home/git/gitlab-shell` .
2014-09-02 18:07:02 +05:30
2021-01-03 14:25:43 +05:30
### Installation folder summary
2014-09-02 18:07:02 +05:30
2021-01-03 14:25:43 +05:30
To summarize here's the [directory structure of the `git` user home directory ](../install/installation.md#gitlab-directory-structure ).
2014-09-02 18:07:02 +05:30
### Processes
2020-03-13 15:44:24 +05:30
```shell
2019-09-30 21:07:59 +05:30
ps aux | grep '^git'
```
2014-09-02 18:07:02 +05:30
2019-10-12 21:52:04 +05:30
GitLab has several components to operate. It requires a persistent database
2021-03-11 19:13:27 +05:30
(PostgreSQL) and Redis database, and uses Apache `httpd` or NGINX to `proxypass`
2021-01-03 14:25:43 +05:30
Puma. All these components should run as different system users to GitLab
(for example, `postgres` , `redis` , and `www-data` , instead of `git` ).
2019-10-12 21:52:04 +05:30
2021-01-03 14:25:43 +05:30
As the `git` user it starts Sidekiq and Puma (a simple Ruby HTTP server
2019-10-12 21:52:04 +05:30
running on port `8080` by default). Under the GitLab user there are normally 4
2021-01-03 14:25:43 +05:30
processes: `puma master` (1 process), `puma cluster worker`
2019-10-12 21:52:04 +05:30
(2 processes), `sidekiq` (1 process).
2014-09-02 18:07:02 +05:30
### Repository access
2021-01-29 00:20:46 +05:30
Repositories get accessed via HTTP or SSH. HTTP cloning/push/pull uses the GitLab API and SSH cloning is handled by GitLab Shell (previously explained).
2014-09-02 18:07:02 +05:30
## Troubleshooting
See the README for more information.
### Init scripts of the services
2021-01-03 14:25:43 +05:30
The GitLab init script starts and stops Puma and Sidekiq:
2014-09-02 18:07:02 +05:30
2020-04-08 14:13:33 +05:30
```plaintext
2014-09-02 18:07:02 +05:30
/etc/init.d/gitlab
Usage: service gitlab {start|stop|restart|reload|status}
```
2019-12-04 20:38:33 +05:30
Redis (key-value store/non-persistent database):
2014-09-02 18:07:02 +05:30
2020-04-08 14:13:33 +05:30
```plaintext
2014-09-02 18:07:02 +05:30
/etc/init.d/redis
Usage: /etc/init.d/redis {start|stop|status|restart|condrestart|try-restart}
```
2019-12-04 20:38:33 +05:30
SSH daemon:
2014-09-02 18:07:02 +05:30
2020-04-08 14:13:33 +05:30
```plaintext
2014-09-02 18:07:02 +05:30
/etc/init.d/sshd
Usage: /etc/init.d/sshd {start|stop|restart|reload|force-reload|condrestart|try-restart|status}
```
2019-12-04 20:38:33 +05:30
Web server (one of the following):
2014-09-02 18:07:02 +05:30
2020-04-08 14:13:33 +05:30
```plaintext
2014-09-02 18:07:02 +05:30
/etc/init.d/httpd
Usage: httpd {start|stop|restart|condrestart|try-restart|force-reload|reload|status|fullstatus|graceful|help|configtest}
$ /etc/init.d/nginx
Usage: nginx {start|stop|restart|reload|force-reload|status|configtest}
```
2019-12-04 20:38:33 +05:30
Persistent database:
2014-09-02 18:07:02 +05:30
2020-04-08 14:13:33 +05:30
```plaintext
2014-09-02 18:07:02 +05:30
$ /etc/init.d/postgresql
Usage: /etc/init.d/postgresql {start|stop|restart|reload|force-reload|status} [version ..]
```
### Log locations of the services
2021-01-03 14:25:43 +05:30
GitLab (includes Puma and Sidekiq logs):
2014-09-02 18:07:02 +05:30
2021-01-03 14:25:43 +05:30
- `/home/git/gitlab/log/` contains `application.log` , `production.log` , `sidekiq.log` , `puma.stdout.log` , `git_json.log` and `puma.stderr.log` normally.
2014-09-02 18:07:02 +05:30
2019-12-04 20:38:33 +05:30
GitLab Shell:
2014-09-02 18:07:02 +05:30
- `/home/git/gitlab-shell/gitlab-shell.log`
2019-12-04 20:38:33 +05:30
SSH:
2014-09-02 18:07:02 +05:30
- `/var/log/auth.log` auth log (on Ubuntu).
- `/var/log/secure` auth log (on RHEL).
2019-12-21 20:55:43 +05:30
NGINX:
2014-09-02 18:07:02 +05:30
- `/var/log/nginx/` contains error and access logs.
2020-06-23 00:09:42 +05:30
Apache `httpd` :
2014-09-02 18:07:02 +05:30
2015-11-26 14:37:03 +05:30
- [Explanation of Apache logs ](https://httpd.apache.org/docs/2.2/logs.html ).
2014-09-02 18:07:02 +05:30
- `/var/log/apache2/` contains error and output logs (on Ubuntu).
- `/var/log/httpd/` contains error and output logs (on RHEL).
2019-12-04 20:38:33 +05:30
Redis:
2014-09-02 18:07:02 +05:30
2015-04-26 12:48:37 +05:30
- `/var/log/redis/redis.log` there are also log-rotated logs there.
2014-09-02 18:07:02 +05:30
2019-12-04 20:38:33 +05:30
PostgreSQL:
2014-09-02 18:07:02 +05:30
- `/var/log/postgresql/*`
2020-06-23 00:09:42 +05:30
### GitLab specific configuration files
2014-09-02 18:07:02 +05:30
2021-01-03 14:25:43 +05:30
GitLab has configuration files located in `/home/git/gitlab/config/*` . Commonly referenced
configuration files include:
2014-09-02 18:07:02 +05:30
2023-05-27 22:25:52 +05:30
- `gitlab.yml` : GitLab Rails configuration
2021-01-29 00:20:46 +05:30
- `puma.rb` : Puma web server settings
- `database.yml` : Database connection settings
2014-09-02 18:07:02 +05:30
2019-12-04 20:38:33 +05:30
GitLab Shell has a configuration file at `/home/git/gitlab-shell/config.yml` .
2014-09-02 18:07:02 +05:30
2023-05-27 22:25:52 +05:30
#### Adding a new setting in GitLab Rails
Settings which belong in `gitlab.yml` include those related to:
- How the application is wired together across multiple services. For example, Gitaly addresses, Redis addresses, Postgres addresses, and Consul addresses.
- Distributed Tracing configuration, and some observability configurations. For example, histogram bucket boundaries.
- Anything that needs to be configured during Rails initialization, possibly before the Postgres connection has been established.
Many other settings are better placed in the app itself, in `ApplicationSetting` . Managing settings in UI is usually a better user experience compared to managing configuration files. With respect to development cost, modifying `gitlab.yml` often seems like a faster iteration, but when you consider all the deployment methods below, it may be a poor tradeoff.
When adding a setting to `gitlab.yml` :
1. Ensure that it is also
[added to Omnibus ](https://docs.gitlab.com/omnibus/settings/gitlab.yml#adding-a-new-setting-to-gitlabyml ).
1. Ensure that it is also [added to Charts ](https://docs.gitlab.com/charts/development/style_guide.html ), if needed.
1. Ensure that it is also [added to GDK ](https://gitlab.com/gitlab-org/gitlab-development-kit/-/blob/main/support/templates/gitlab/config/gitlab.yml.erb ).
2021-01-03 14:25:43 +05:30
### Maintenance tasks
2014-09-02 18:07:02 +05:30
2021-09-04 01:27:46 +05:30
[GitLab ](https://gitlab.com/gitlab-org/gitlab/-/tree/master ) provides Rake tasks with which you see version information and run a quick check on your configuration to ensure it is configured properly within the application. See [maintenance Rake tasks ](../administration/raketasks/maintenance.md ).
2014-09-02 18:07:02 +05:30
In a nutshell, do the following:
2020-04-08 14:13:33 +05:30
```shell
2014-09-02 18:07:02 +05:30
sudo -i -u git
cd gitlab
bundle exec rake gitlab:env:info RAILS_ENV=production
bundle exec rake gitlab:check RAILS_ENV=production
```
2021-01-29 00:20:46 +05:30
It's recommended to sign in to the `git` user using either `sudo -i -u git` or
`sudo su - git` . Although the `sudo` commands provided by GitLab work in Ubuntu,
they don't always work in RHEL.
2017-09-10 17:25:29 +05:30
## GitLab.com
2021-01-29 00:20:46 +05:30
The [GitLab.com architecture ](https://about.gitlab.com/handbook/engineering/infrastructure/production/architecture/ )
is detailed for your reference, but this architecture is only useful if you have
millions of users.
2023-07-09 08:55:56 +05:30
### AI architecture
A [SaaS model gateway ](ai_architecture.md ) is available to enable AI-powered features.
