debian-mirror-gitlab/doc/administration/terraform_state.md

215 lines
7.6 KiB
Markdown
Raw Normal View History

2021-01-03 14:25:43 +05:30
---
stage: Configure
group: Configure
2022-11-25 23:54:43 +05:30
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
2021-01-03 14:25:43 +05:30
---
2021-06-08 01:23:25 +05:30
# Terraform state administration **(FREE)**
2020-04-22 19:07:51 +05:30
> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2673) in GitLab 12.10.
GitLab can be used as a backend for [Terraform](../user/infrastructure/index.md) state
files. The files are encrypted before being stored. This feature is enabled by default.
The storage location of these files defaults to:
- `/var/opt/gitlab/gitlab-rails/shared/terraform_state` for Omnibus GitLab installations.
- `/home/git/gitlab/shared/terraform_state` for source installations.
These locations can be configured using the options described below.
2022-07-16 23:28:13 +05:30
Use [external object storage](https://docs.gitlab.com/charts/advanced/external-object-storage/#lfs-artifacts-uploads-packages-external-diffs-terraform-state-dependency-proxy) configuration for [GitLab Helm chart](https://docs.gitlab.com/charts/) installations.
2021-03-11 19:13:27 +05:30
2021-04-17 20:07:23 +05:30
## Disabling Terraform state
To disable terraform state site-wide, follow the steps below.
2021-09-04 01:27:46 +05:30
A GitLab administrator may want to disable Terraform state to reduce disk space or if Terraform is not used in your instance.
2021-04-17 20:07:23 +05:30
To do so, follow the steps below according to your installation's type.
**In Omnibus installations:**
1. Edit `/etc/gitlab/gitlab.rb` and add the following line:
```ruby
gitlab_rails['terraform_state_enabled'] = false
```
1. Save the file and [reconfigure GitLab](restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect.
**In installations from source:**
1. Edit `/home/git/gitlab/config/gitlab.yml` and add or amend the following lines:
```yaml
terraform_state:
enabled: false
```
1. Save the file and [restart GitLab](restart_gitlab.md#installations-from-source) for the changes to take effect.
2020-04-22 19:07:51 +05:30
## Using local storage
2021-03-08 18:12:59 +05:30
The default configuration uses local storage. To change the location where
Terraform state files are stored locally, follow the steps below.
2020-04-22 19:07:51 +05:30
**In Omnibus installations:**
1. To change the storage path for example to `/mnt/storage/terraform_state`, edit
`/etc/gitlab/gitlab.rb` and add the following line:
```ruby
gitlab_rails['terraform_state_storage_path'] = "/mnt/storage/terraform_state"
```
2020-05-24 23:13:21 +05:30
1. Save the file and [reconfigure GitLab](restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect.
2020-04-22 19:07:51 +05:30
**In installations from source:**
1. To change the storage path for example to `/mnt/storage/terraform_state`, edit
`/home/git/gitlab/config/gitlab.yml` and add or amend the following lines:
```yaml
terraform_state:
enabled: true
storage_path: /mnt/storage/terraform_state
```
2020-05-24 23:13:21 +05:30
1. Save the file and [restart GitLab](restart_gitlab.md#installations-from-source) for the changes to take effect.
2020-04-22 19:07:51 +05:30
2021-03-11 19:13:27 +05:30
## Using object storage **(FREE SELF)**
2020-04-22 19:07:51 +05:30
2022-10-11 01:57:18 +05:30
Instead of storing Terraform state files on disk, we recommend the use of
[one of the supported object storage options](object_storage.md#options).
2022-08-27 11:52:29 +05:30
This configuration relies on valid credentials to be configured already.
2020-04-22 19:07:51 +05:30
[Read more about using object storage with GitLab](object_storage.md).
### Object storage settings
The following settings are:
- Nested under `terraform_state:` and then `object_store:` on source installations.
- Prefixed by `terraform_state_object_store_` on Omnibus GitLab installations.
| Setting | Description | Default |
|---------|-------------|---------|
2020-05-24 23:13:21 +05:30
| `enabled` | Enable/disable object storage | `false` |
2021-02-22 17:27:13 +05:30
| `remote_directory` | The bucket name where Terraform state files are stored | |
2020-04-22 19:07:51 +05:30
| `connection` | Various connection options described below | |
2021-04-29 21:17:54 +05:30
### Migrate to object storage
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/247042) in GitLab 13.9.
2022-03-02 08:16:31 +05:30
WARNING:
It's not possible to migrate Terraform state files from object storage back to local storage,
so proceed with caution. [An issue exists](https://gitlab.com/gitlab-org/gitlab/-/issues/350187)
to change this behavior.
2021-04-29 21:17:54 +05:30
To migrate Terraform state files to object storage, follow the instructions below.
- For Omnibus package installations:
```shell
gitlab-rake gitlab:terraform_states:migrate
```
- For source installations:
```shell
sudo -u git -H bundle exec rake gitlab:terraform_states:migrate RAILS_ENV=production
```
For GitLab 13.8 and earlier versions, you can use a workaround for the Rake task:
1. Open the GitLab [Rails console](operations/rails_console.md).
1. Run the following commands:
```ruby
Terraform::StateUploader.alias_method(:upload, :model)
Terraform::StateVersion.where(file_store: ::ObjectStorage::Store::LOCAL). find_each(batch_size: 10) do |terraform_state_version|
puts "Migrating: #{terraform_state_version.inspect}"
terraform_state_version.file.migrate!(::ObjectStorage::Store::REMOTE)
end
```
2023-01-13 00:05:48 +05:30
You can optionally track progress and verify that all Terraform state files migrated successfully using the
2022-01-26 12:08:38 +05:30
[PostgreSQL console](https://docs.gitlab.com/omnibus/settings/database.html#connecting-to-the-bundled-postgresql-database):
2023-01-13 00:05:48 +05:30
- `sudo gitlab-rails dbconsole` for Omnibus GitLab 14.1 and earlier.
- `sudo gitlab-rails dbconsole --database main` for Omnibus GitLab 14.2 and later.
2022-01-26 12:08:38 +05:30
- `sudo -u git -H psql -d gitlabhq_production` for source-installed instances.
2022-05-07 20:08:51 +05:30
Verify `objectstg` below (where `file_store=2`) has count of all states:
2022-01-26 12:08:38 +05:30
```shell
2022-07-16 23:28:13 +05:30
gitlabhq_production=# SELECT count(*) AS total, sum(case when file_store = '1' then 1 else 0 end) AS filesystem, sum(case when file_store = '2' then 1 else 0 end) AS objectstg FROM terraform_state_versions;
2022-01-26 12:08:38 +05:30
total | filesystem | objectstg
------+------------+-----------
15 | 0 | 15
```
Verify that there are no files on disk in the `terraform_state` folder:
```shell
2022-07-16 23:28:13 +05:30
sudo find /var/opt/gitlab/gitlab-rails/shared/terraform_state -type f | grep -v tmp | wc -l
2022-01-26 12:08:38 +05:30
```
2020-04-22 19:07:51 +05:30
### S3-compatible connection settings
2020-07-28 23:09:34 +05:30
See [the available connection settings for different providers](object_storage.md#connection-settings).
2020-04-22 19:07:51 +05:30
**In Omnibus installations:**
1. Edit `/etc/gitlab/gitlab.rb` and add the following lines; replacing with
the values you want:
```ruby
gitlab_rails['terraform_state_object_store_enabled'] = true
2020-07-28 23:09:34 +05:30
gitlab_rails['terraform_state_object_store_remote_directory'] = "terraform"
2020-04-22 19:07:51 +05:30
gitlab_rails['terraform_state_object_store_connection'] = {
'provider' => 'AWS',
'region' => 'eu-central-1',
'aws_access_key_id' => 'AWS_ACCESS_KEY_ID',
'aws_secret_access_key' => 'AWS_SECRET_ACCESS_KEY'
}
```
2021-02-22 17:27:13 +05:30
NOTE:
2020-04-22 19:07:51 +05:30
If you are using AWS IAM profiles, be sure to omit the AWS access key and secret access key/value pairs.
```ruby
gitlab_rails['terraform_state_object_store_connection'] = {
'provider' => 'AWS',
'region' => 'eu-central-1',
'use_iam_profile' => true
}
```
2020-05-24 23:13:21 +05:30
1. Save the file and [reconfigure GitLab](restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect.
2022-01-26 12:08:38 +05:30
1. [Migrate any existing local states to the object storage](#migrate-to-object-storage)
2020-04-22 19:07:51 +05:30
**In installations from source:**
1. Edit `/home/git/gitlab/config/gitlab.yml` and add or amend the following
lines:
```yaml
terraform_state:
enabled: true
object_store:
enabled: true
2020-07-28 23:09:34 +05:30
remote_directory: "terraform" # The bucket name
2020-04-22 19:07:51 +05:30
connection:
provider: AWS # Only AWS supported at the moment
2021-04-17 20:07:23 +05:30
aws_access_key_id: AWS_ACCESS_KEY_ID
2020-04-22 19:07:51 +05:30
aws_secret_access_key: AWS_SECRET_ACCESS_KEY
region: eu-central-1
```
2020-05-24 23:13:21 +05:30
1. Save the file and [restart GitLab](restart_gitlab.md#installations-from-source) for the changes to take effect.
2022-01-26 12:08:38 +05:30
1. [Migrate any existing local states to the object storage](#migrate-to-object-storage)