Upgrading self-compiled installations

DETAILS: Tier: Free, Premium, Ultimate Offering: Self-managed

Make sure you view this upgrade guide from the branch (version) of GitLab you would like to install (for example, 16.0). You can select the required version of documentation in the dropdown list in the upper-right corner of GitLab documentation page.

In each of the following examples, replace BRANCH with the branch of the version you upgrading to (for example, 16-0-stable for 16.0). Replace PREVIOUS_BRANCH with the branch for the version you are upgrading from (for example, 15-11-stable for 15.11).

If the highest number stable branch is unclear check the GitLab Blog for installation guide links by version.

If you are changing from GitLab Community Edition to GitLab Enterprise Edition, see the Upgrading from CE to EE documentation.

Upgrading to a new major version

Major versions are reserved for backwards incompatible changes. You should first upgrade to the latest available minor version of your current major version. Follow the Upgrade Recommendations to identify the ideal upgrade path.

Before upgrading to a new major version, you should ensure that any background migration jobs from previous releases have been completed. To see the current size of the background_migration queue, Check for background migrations before upgrading.

Guidelines for all versions

This section contains all the steps necessary to upgrade Community Edition or Enterprise Edition, regardless of the version you are upgrading to. Version specific guidelines (should there be any) are covered separately.

1. Backup

Prerequisites:

Make sure rsync is installed.

Perform the backup:

cd /home/git/gitlab

sudo -u git -H bundle exec rake gitlab:backup:create RAILS_ENV=production

2. Stop server

# For systems running systemd
sudo systemctl stop gitlab.target

# For systems running SysV init
sudo service gitlab stop

3. Update Ruby

From GitLab 16.7, we only support Ruby 3.1.x and dropped support for Ruby 3.0. Be sure to upgrade if necessary. You can check which version you are running with ruby -v.

Install Ruby.

4. Update Node.js

To check the minimum required Node.js version, see Node.js versions.

GitLab also requires the use of Yarn >= v1.10.0 to manage JavaScript dependencies.

In Debian or Ubuntu:

sudo apt-get remove yarn

npm install --global yarn

More information can be found on the Yarn website.

5. Update Go

To check the minimum required Go version, see Go versions.

You can check which version you are running with go version.

Download and install Go (for Linux, 64-bit):

# Remove former Go installation folder
sudo rm -rf /usr/local/go

curl --remote-name --location --progress-bar "https://go.dev/dl/go1.20.8.linux-amd64.tar.gz"
echo 'cc97c28d9c252fbf28f91950d830201aa403836cbed702a05932e63f7f0c7bc4  go1.20.8.linux-amd64.tar.gz' | shasum -a256 -c - && \
  sudo tar -C /usr/local -xzf go1.20.8.linux-amd64.tar.gz
sudo ln -sf /usr/local/go/bin/{go,gofmt} /usr/local/bin/
rm go1.20.8.linux-amd64.tar.gz

6. Update Git

To check you are running the minimum required Git version, see Git versions.

From GitLab 13.6, you should use the Git version provided by Gitaly that:

Is always at the version required by GitLab.
May contain custom patches required for proper operation.

# Install dependencies
sudo apt-get install -y libcurl4-openssl-dev libexpat1-dev gettext libz-dev libssl-dev libpcre2-dev build-essential

# Clone the Gitaly repository
git clone https://gitlab.com/gitlab-org/gitaly.git -b <X-Y-stable> /tmp/gitaly

# Compile and install Git
cd /tmp/gitaly
sudo make git GIT_PREFIX=/usr/local

Replace <X-Y-stable> with the stable branch that matches the GitLab version you want to install. For example, if you want to install GitLab 13.6, use the branch name 13-6-stable.

Remember to set git -> bin_path to /usr/local/bin/git in config/gitlab.yml.

7. Update PostgreSQL

WARNING: GitLab 16.0 requires at least PostgreSQL 13.

The latest version of GitLab might depend on a more recent PostgreSQL version than what you are running. You may also have to enable some extensions. For more information, see the PostgreSQL requirements

To upgrade PostgreSQL, refer to its documentation.

8. Get latest code

cd /home/git/gitlab

sudo -u git -H git fetch --all --prune
sudo -u git -H git checkout -- Gemfile.lock db/structure.sql locale

For GitLab Community Edition:

cd /home/git/gitlab

sudo -u git -H git checkout BRANCH

For GitLab Enterprise Edition:

cd /home/git/gitlab

sudo -u git -H git checkout BRANCH-ee

9. Update configuration files

New configuration options for `gitlab.yml`

There might be configuration options available for gitlab.yml). View them with the command below and apply them manually to your current gitlab.yml:

cd /home/git/gitlab

git diff origin/PREVIOUS_BRANCH:config/gitlab.yml.example origin/BRANCH:config/gitlab.yml.example

New configuration options for `database.yml`

Changed in GitLab 16.0 to have ci: section in config/database.yml.postgresql.

There might be configuration options available for database.yml. View them with the command below and apply them manually to your current database.yml:

cd /home/git/gitlab

git diff origin/PREVIOUS_BRANCH:config/database.yml.postgresql origin/BRANCH:config/database.yml.postgresql

NGINX configuration

Ensure you're still up-to-date with the latest NGINX configuration changes:

cd /home/git/gitlab

# For HTTPS configurations
git diff origin/PREVIOUS_BRANCH:lib/support/nginx/gitlab-ssl origin/BRANCH:lib/support/nginx/gitlab-ssl

# For HTTP configurations
git diff origin/PREVIOUS_BRANCH:lib/support/nginx/gitlab origin/BRANCH:lib/support/nginx/gitlab

If you are using Strict-Transport-Security in your installation, you must enable it in your NGINX configuration to continue using it. This is because the GitLab application no longer sets it.

If you are using Apache instead of NGINX see the updated Apache templates. Also because Apache does not support upstreams behind Unix sockets you must let GitLab Workhorse listen on a TCP port. You can do this via /etc/default/gitlab.

SMTP configuration

If you use SMTP to deliver mail, you must add the following line to config/initializers/smtp_settings.rb:

ActionMailer::Base.delivery_method = :smtp

See smtp_settings.rb.sample as an example.

Configure systemd units

If using the SysV init script, see Configure SysV init script.

Check if the systemd units have been updated:

cd /home/git/gitlab

git diff origin/PREVIOUS_BRANCH:lib/support/systemd origin/BRANCH:lib/support/systemd

Copy them over:

sudo mkdir -p /usr/local/lib/systemd/system
sudo cp lib/support/systemd/* /usr/local/lib/systemd/system/
sudo systemctl daemon-reload

Configure SysV init script

If using systemd units, see Configure systemd units.

There might be new configuration options available for gitlab.default.example. View them with the command below and apply them manually to your current /etc/default/gitlab:

cd /home/git/gitlab

git diff origin/PREVIOUS_BRANCH:lib/support/init.d/gitlab.default.example origin/BRANCH:lib/support/init.d/gitlab.default.example

Ensure you're still up-to-date with the latest init script changes:

cd /home/git/gitlab

sudo cp lib/support/init.d/gitlab /etc/init.d/gitlab

If you are using the init script on a system running systemd as init, because you have not switched to native systemd units yet, run:

sudo systemctl daemon-reload

10. Install libraries, migrations, etc

Make sure you have the required PostgreSQL extensions, then proceed to install the needed libraries:

cd /home/git/gitlab

# If you haven't done so during installation or a previous upgrade already
sudo -u git -H bundle config set --local deployment 'true'
sudo -u git -H bundle config set --local without 'development test kerberos'

# Update gems
sudo -u git -H bundle install

# Optional: clean up old gems
sudo -u git -H bundle clean

# Run database migrations
sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production

# Update node dependencies and recompile assets
sudo -u git -H bundle exec rake yarn:install gitlab:assets:clean gitlab:assets:compile RAILS_ENV=production NODE_ENV=production NODE_OPTIONS="--max_old_space_size=4096"

# Clean up cache
sudo -u git -H bundle exec rake cache:clear RAILS_ENV=production

11. Update GitLab Shell

cd /home/git/gitlab-shell

sudo -u git -H git fetch --all --tags --prune
sudo -u git -H git checkout v$(</home/git/gitlab/GITLAB_SHELL_VERSION)
sudo -u git -H make build

12. Update GitLab Workhorse

Install and compile GitLab Workhorse.

cd /home/git/gitlab

sudo -u git -H bundle exec rake "gitlab:workhorse:install[/home/git/gitlab-workhorse]" RAILS_ENV=production

13. Update Gitaly

If Gitaly is located on its own server, or you use Gitaly Cluster, see Gitaly or Gitaly Cluster on the Zero downtime upgrades page.

Compile Gitaly

# Fetch Gitaly source with Git and compile with Go
cd /home/git/gitlab
sudo -u git -H bundle exec rake "gitlab:gitaly:install[/home/git/gitaly,/home/git/repositories]" RAILS_ENV=production

14. Update GitLab Pages

Only needed if you use GitLab Pages

Install and compile GitLab Pages. GitLab Pages uses GNU Make. If you are not using Linux you may have to run gmake instead of make below.

cd /home/git/gitlab-pages

sudo -u git -H git fetch --all --tags --prune
sudo -u git -H git checkout v$(</home/git/gitlab/GITLAB_PAGES_VERSION)
sudo -u git -H make

15. Start application

# For systems running systemd
sudo systemctl start gitlab.target
sudo systemctl restart nginx.service

# For systems running SysV init
sudo service gitlab start
sudo service nginx restart

16. Check application status

Check if GitLab and its environment are configured correctly:

cd /home/git/gitlab

sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production

To make sure you didn't miss anything run a more thorough check:

cd /home/git/gitlab

sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production

If all items are green, then congratulations, the upgrade is complete!

Version specific changes

Upgrading versions might need some manual intervention. For more information, check the version you are upgrading to for additional steps required for all GitLab installations, and for steps that apply to self-compiled installations.

Troubleshooting

1. Revert the code to the previous version

To revert to a previous version, you must follow the upgrading guides for the previous version.

For example, if you have upgraded to GitLab 12.6 and want to revert back to 12.5, follow the guides for upgrading from 12.4 to 12.5. You can use the version dropdown list at the top of the page to select the right version.

When reverting, you should not follow the database migration guides, as the backup has already been migrated to the previous version.

2. Restore from the backup

cd /home/git/gitlab

sudo -u git -H bundle exec rake gitlab:backup:restore RAILS_ENV=production

If you have more than one backup *.tar file, add BACKUP=timestamp_of_backup to the above.