info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
Before any upgrade, consider all audiences and targets, ordered by how immediately they are affected by Ruby upgrades:
1.**Developers.** We have many contributors to GitLab and related projects both inside and outside the company. Changing files such as `.ruby-version` affects everyone using tooling that interprets these files.
The developers are affected as soon as they pull from the repository containing the merged changes.
1.**GitLab CI/CD.** We heavily lean on CI/CD for code integration and testing. CI/CD jobs do not interpret files such as `.ruby-version`.
Instead, they use the Ruby installed in the Docker container they execute in, which is defined in `.gitlab-ci.yml`.
The container images used in these jobs are maintained in the [`gitlab-build-images`](https://gitlab.com/gitlab-org/gitlab-build-images) repository.
When we merge an update to an image, CI/CD jobs are affected as soon as the [image is built](https://gitlab.com/gitlab-org/gitlab-build-images/#pushing-a-rebuild-image).
1.**GitLab SaaS**. GitLab.com is deployed from customized Helm charts that use Docker images from [Cloud Native GitLab (CNG)](https://gitlab.com/gitlab-org/build/CNG).
Just like CI/CD, `.ruby-version` is meaningless in this environment. Instead, those Docker images must be patched to upgrade Ruby.
GitLab SaaS is affected with the next deployment.
1.**Self-managed GitLab.** Customers installing GitLab via [Omnibus](https://gitlab.com/gitlab-org/omnibus-gitlab) use none of the above.
Instead, their Ruby version is defined by the [Ruby software bundle](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/config/software/ruby.rb) in Omnibus.
Self-managed customers are affected as soon as they upgrade to the release containing this change.
## Ruby upgrade approach
Timing all steps in a Ruby upgrade correctly is critical. As a general guideline, consider the following:
- For smaller upgrades where production behavior is unlikely to change, aim to keep the version gap between
repositories and production minimal. Coordinate with stakeholders to merge all changes closely together
(within a day or two) to avoid drift. In this scenario the likely order is to upgrade developer tooling and
environments first, production second.
- For larger changes, the risk of going to production with a new Ruby is significant. In this case, try to get into a
position where all known incompatibilities with the new Ruby version are already fixed, then work
with production engineers to deploy the new Ruby to a subset of the GitLab production fleet. In this scenario
the likely order is to update production first, developer tooling and environments second. This makes rollbacks
easier in case of critical regressions in production.
Either way, we found that from past experience the following approach works well, with some steps likely only
necessary for minor and major upgrades. Note that some of these steps can happen in parallel or may have their
order reversed as described above.
### Create an epic
Tracking this work in an epic is useful to get a sense of progress. For larger upgrades, include a
timeline in the epic description so stakeholders know when the final switch is expected to go live.
Break changes to individual repositories into separate issues under this epic.
### Communicate the intent to upgrade
Especially for upgrades that introduce or deprecate features,
communicate early that an upgrade is due, ideally with an associated timeline. Provide links to important or
noteworthy changes, so developers can start to familiarize themselves with
changes ahead of time.
GitLab team members should announce the intent in relevant Slack channels (`#backend` and `#development` at minimum)
and Engineering Week In Review (EWIR). Include a link to the upgrade epic in your
### Add new Ruby to CI/CD and development environments
To build and run Ruby gems and the GitLab Rails application with a new Ruby, you must first prepare CI/CD
and developer environments to include the new Ruby version.
At this stage, you *must not make it the default Ruby yet*, but make it optional instead. This allows
for a smoother transition by supporting both old and new Ruby versions for a period of time.
There are two places that require changes:
1.**[GitLab Build Images](https://gitlab.com/gitlab-org/gitlab-build-images).** These are Docker images
we use for runners and other Docker-based pre-production environments. The kind of change necessary
depends on the scope.
- For [patch level updates](https://gitlab.com/gitlab-org/gitlab-build-images/-/merge_requests/418), it should suffice to increment the patch level of `RUBY_VERSION`.
All projects building against the same minor release automatically download the new patch release.
- For [major and minor updates](https://gitlab.com/gitlab-org/gitlab-build-images/-/merge_requests/320), create a new set of Docker images that can be used side-by-side with existing images during the upgrade process. **Important:** Make sure to copy over all Ruby patch files
in the `/patches` directory to a new folder matching the Ruby version you upgrade to, or they aren't applied.
1.**[GitLab Development Kit (GDK)](https://gitlab.com/gitlab-org/gitlab-development-kit).**
Update GDK to add the new Ruby as an additional option for
developers to choose from. This typically only requires it to be appended to `.tool-versions` so `asdf`
users will benefit from this. Other users will have to install it manually
To assess which of these repositories are critical to be updated alongside the main GitLab application consider:
- The Ruby version scope.
- The role that the service or library plays in the overall functioning of GitLab.
Refer to the [list of GitLab projects](https://about.gitlab.com/handbook/engineering/projects/) for a complete
account of which repositories could be affected.
For smaller version upgrades, it can be acceptable to delay updating libraries that are non-essential or where
we are certain that the main application test suite would catch regressions under a new Ruby version.
NOTE:
Consult with the respective code owners whether it is acceptable to merge these changes ahead
of updating the GitLab application. It might be best to get the necessary approvals
but wait to merge the change until everything is ready.
### Prepare the GitLab application MR
With the dependencies updated and the new gem versions released, you can update the main Rails
application with any necessary changes, similar to the gems and related systems.
On top of that, update the documentation to reflect the version change in the installation
and update instructions ([example](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/68363)).
NOTE:
Be especially careful with timing this merge request, since as soon as it is merged, all GitLab contributors
will be affected by it and the changes will be deployed. You must ensure that this MR remains
open until everything else is ready, but it can be useful to get approval early to reduce lead time.
### Give developers time to upgrade (grace period)
With the new Ruby made available as an option, and all merge requests either ready or merged,
there should be a grace period (1 week at minimum) during which developers can
install the new Ruby on their machines. For GDK and `asdf` users this should happen automatically
via `gdk update`.
This pause is a good time to assess the risk of this upgrade for GitLab SaaS.
For Ruby upgrades that are high risk, such as major version upgrades, it is recommended to
coordinate the changes with the infrastructure team through a [change management request](https://about.gitlab.com/handbook/engineering/infrastructure/change-management/).
If you submit a change management request, coordinate the rollout with infrastructure
engineers. When dealing with larger upgrades, involve [Release Managers](https://about.gitlab.com/community/release-managers/)
in the rollout plan.
### Create patch releases and backports for security patches
If the upgrade was a patch release and contains important security fixes, it should be released as a
GitLab patch release to self-managed customers. Consult our [release managers](https://about.gitlab.com/community/release-managers/)
for how to proceed.
## Ruby upgrade tooling
There are several tools that ease the upgrade process.
### Deprecation Toolkit
A common problem with Ruby upgrades is that deprecation warnings turn into errors. This means that every single
deprecation warning must be resolved before making the switch. To avoid new warnings from making it into the
main application branch, we use [`DeprecationToolkitEnv`](https://gitlab.com/gitlab-org/gitlab/blob/master/spec/deprecation_toolkit_env.rb).
This module observes deprecation warnings emitted from spec runs and turns them into test failures. This prevents
developers from checking in new code that would fail under a new Ruby.
Sometimes it cannot be avoided to introduce new warnings, for example when a Ruby gem we use emits these warnings
and we have no control over it. In these cases, add silences, like [this merge request](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/68865) did.
### Deprecation Logger
We also log Ruby and Rails deprecation warnings to a dedicated log file, `log/deprecation_json.log`
(see [GitLab Developers Guide to Logging](logging.md) for where to find GitLab log files),
which can provide clues when there is code that is not adequately covered by tests and hence would slip past `DeprecationToolkitEnv`.
For GitLab SaaS, GitLab team members can inspect these log events in Kibana
postpone the upgrade to the following month, as we [prioritize availability over velocity](https://about.gitlab.com/handbook/engineering/development/principles/#prioritizing-technical-decisions).
