forgejo-federation/models/forgejo_migrations/migrate.go

157 lines
4.9 KiB
Go
Raw Normal View History

[DB] Forgejo database migrations - Implements https://codeberg.org/forgejo/discussions/issues/32#issuecomment-918737 - Allows to add Forgejo-specific migrations that don't interfere with Gitea's migration logic. Please do note that we cannot liberally add migrations for Gitea tables, as they might do their own migrations in a future version on that table, and that could undo our migrations. Luckily, we don't have a scenario where that's needed and thus not taken into account. Co-authored-by: Gusted <postmaster@gusted.xyz> Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/795 (cherry picked from commit 8ee32978c0af1f8f71679c87f695df2b90b617c8) (cherry picked from commit c240b34f595a7a9763f7b748052ac98f9f18954d) (cherry picked from commit 03936c649243a0a29701393d58e63e33064c7461) (cherry picked from commit a20ed852f8b6d28872c05d688bffe5c6976bfa03) (cherry picked from commit 1dfa82676f1feb745633618fde2d362bf19c4f28) (cherry picked from commit c39ae0bf8abced8fd5dc32589e68515ac308b69b) (cherry picked from commit cfaff08996c9f42592c95a63fe907b45b8a9317a) (cherry picked from commit 94a458835a2b0336b26c1c9df64fdfe2de47f496) (cherry picked from commit 61a3cf77dfe3f612ff110eb19f94dcb08051daf1) (cherry picked from commit abb350fde879cc495761dc4616b7aa0fc5d94d54) (cherry picked from commit 5194829d6b4ed702cf50ff875da57d04d77c8a18) (cherry picked from commit 89239a60f23cad7dad03add744e23a4f3b10d6a4) (cherry picked from commit 683cfd86efc5fa8cc04973ce3115351515a20917) (cherry picked from commit f4546cfed92844e3666b80130eadabb9348b88ae) (cherry picked from commit 86614d5826392b3fbe68355baeab9a0a761883a5) (cherry picked from commit e4b9c32187a039a83686a82856a9a192919c6e82) (cherry picked from commit 8c253719afa9b82f169757df007587d38560c06d) (cherry picked from commit 857365d6c15b5471d63662b1d89d1523151c4f79) (cherry picked from commit a488b3952f58bbf28bfa101a24e52dad7c9662eb) (cherry picked from commit 98313c49109c941426beecc1a3e7887f28b99970) (cherry picked from commit 430d95e8240971e266705d2e7202a5c785379cb2) (cherry picked from commit 08bf9d918fbb67f5ac06c0cfdc24229aa14ff83f) (cherry picked from commit f8a170e2d042fcb8f314e123de6918317ac1e909) (cherry picked from commit d20e325378e67087279496d35b575e566836aaa1) (cherry picked from commit 6c0aa7dd4fd8c234984d455933f69f51abcb2d32) (cherry picked from commit 46c08c26c7bd3260b3ac7678f24566b467f4a2fb) (cherry picked from commit 9ee22153c4ec62392693c9151d5395221d097f70) [DB] Ensure forgejo migration up to date (squash) - Hook Forgejo's `EnsureUpToDate` to Gitea's `EnsureUpToDate`, such that the Forgejo migrations are also being checked to be up to date. - I'm not sure how I missed this and if this has caused any problems, but due to the lack of any open issue about it it seems to not be a big problem. (cherry picked from commit 6c65b6dcf6ab0d58e5c2d03a866e4e38294f72ad) (cherry picked from commit 6d45c37d843147e69b0a27ebe35c617d7f574b76) [DB] Add test for TestEnsureUpToDate (squash) - Add a test for the behavior of `EnsureUpToDate`, to ensure it will error when needed and succeed when the forgejo version is up to date. - Add forgejo_migrations package to GO_TEST_PACKAGES, to avoid running it with `test-unit` and instead test it with `test-*-migration`. (cherry picked from commit b172a506914fee40a50daa51f0c8e547427fd2f8) (cherry picked from commit d8af3088205b592340fd836135ffe97da9cec5a6) (cherry picked from commit e69e64a32c5e38247e94ab880536e3cfeab67cc6) (cherry picked from commit 4e8363fad4e08845960912a3ea3fe7265ee60602) (cherry picked from commit fc9ecd6c533eca864503423cf4a21710984a6b75) (cherry picked from commit e5c446e3dc9bc6e9549862f7b764a634f4fbaaae) (cherry picked from commit 7066a15655a33f57ccfb68cf2cb994ea57ad3666) (cherry picked from commit 9183cdc8354d529a1c2b570551bc1578fb10d58b) (cherry picked from commit 5f93039e0d7c8a7eb79df16ce0d8603f948b1bd2) Conflicts: Makefile https://codeberg.org/forgejo/forgejo/pulls/2245 (cherry picked from commit a039b3b0c9a7016de9e7e71ea0cc7a1185adb8d9)
2023-06-03 14:11:16 +05:30
// Copyright 2023 The Forgejo Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package forgejo_migrations //nolint:revive
import (
"context"
"fmt"
"os"
[SEMVER] store SemVer in ForgejoSemVer after a database upgrade (cherry picked from commit b7fe7cf401f4bddd6455efc651f7ac054f3fe1cf) (cherry picked from commit cf339eed4f4851b18448dbdd83df32d00bc6f45b) (cherry picked from commit 4f3a16168bbeced519a60c32e10e2895d9367238) (cherry picked from commit 6f5bbc53fcebd614f5ee8627c7d6e3c637ffa694) (cherry picked from commit aca42b422e76668387769c15868ef77b073bb7db) (cherry picked from commit 5a7f7580e525694d1f27e12329c8532bd29273f8) (cherry picked from commit 06c383c807ad49e1e35b429a10c6a11c65aeebe5) (cherry picked from commit fe831dcb53b81b6cc632be751bfcbfb9bc00efd3) (cherry picked from commit cd12cd0dbce47c6117ea579ae5019c182155b3a9) (cherry picked from commit cc79163703ce31706c86b88c38fb8a20ed745e20) (cherry picked from commit 0102a5715ea6a03b560cc2f0b6cbe2b2576c255e) (cherry picked from commit 403f7520b3056eace36eae505afbab6a05f597b7) (cherry picked from commit a3b61510a246f61c174ddc3c288e556522a6aab9) (cherry picked from commit f83f0f9feb76a8b62ca4d74dc2785c713fbec282) (cherry picked from commit fd1c3a6d09a057070844cd955d0920518e60f408) (cherry picked from commit f7cdc3d6f1ad1ff01bb1814207fcb41210db80e8) (cherry picked from commit 060121b644e0515a6b673cdd514d52e6fa1e6ec3) (cherry picked from commit 62c847ff0235196e73e860d0d1658f0734985270) (cherry picked from commit 4d051b51c2813828b978ef84b6b72c0ba051741b) (cherry picked from commit 86e6981a936b1937065605892083b2488424bf5e) (cherry picked from commit c1fc9e441b1467033911f3848f82a89aaacf98a9) (cherry picked from commit 8bb2f0871a507d881acf74acf8fb90fd5ebff567) (cherry picked from commit 0cd9fe52511ee06f1b8849325d196f60d8ccd151) (cherry picked from commit b0b44778b4d3e51852d0669dfe6d9a6c334b90e4) (cherry picked from commit 7c2f4f749f1c8986875eebf95254a3db151e5248)
2023-08-09 03:25:25 +05:30
"code.gitea.io/gitea/models/forgejo/semver"
[DB] forgejo migration v2: create the forgejo_sem_ver table (cherry picked from commit 86b26436af85e0eedb732e115e8be024e1d54ca6) (cherry picked from commit 479cba59aca2b3b73a83e5acc0b754906230f0b2) (cherry picked from commit 4765f9a889ce7324416b51e7d4524b2368459752) (cherry picked from commit af771410bfaa511bbc97cbfeefbb279b3836158d) (cherry picked from commit d1ea9305d8d653d637cbde6d383c095bdc4991f0) (cherry picked from commit f77e1bb7abd6054851b811fc0d71bf0130085353) (cherry picked from commit 0b95f8fe899ce14e19e5aeccd3e53e21b8bf8fc1) (cherry picked from commit 4f8fb2390a415d2f9319c9f23ff653c31efd2409) (cherry picked from commit 8ea0e22ff6e09f1e46d9a10f1325f7c6997c018b) (cherry picked from commit 43ac19ac59044210906c96857194231ca7804c92) (cherry picked from commit 0d2f63df4f9579c62ed21c807c53e689b1c71dbb) (cherry picked from commit d02a8036fda2bfcf8791366198e25735547d6b3f) (cherry picked from commit 1fe4c7db941b892df38be12249d4898248ff016e) (cherry picked from commit d641cdeaf0f77ef12fb5fe5c27c20804757a74ef) (cherry picked from commit 10e8a4f8b864961a700f2c3e6c816489bf4e4ac4) (cherry picked from commit 8097bc40b3f90ed959adc9edd1982e5867856397) (cherry picked from commit ee5cb37d7e9567c129fcf2a606175d425dacace1) (cherry picked from commit 26d93b8e49270440a940cd2e12b36c449644af94) (cherry picked from commit bc73195e5945a89bedb2faac579eac2cc403195e) (cherry picked from commit a763fa5de29e6e5986e0f4971e0a9696aa98a024) (cherry picked from commit 4b000cb435b1ddec2aee901b9d257e0af87b7698) (cherry picked from commit 9f6d20e73f73fefd78e64d8fc11af9118bef9557) (cherry picked from commit b5001edeeabf29d5b494da32fc09fa911b38cdb8) (cherry picked from commit 76321718328532a4ed85a04839c339dd42b87d42) (cherry picked from commit 816c5b0c4e7c599d513033bf5eb01bb6259dd144)
2023-08-09 03:22:37 +05:30
forgejo_v1_20 "code.gitea.io/gitea/models/forgejo_migrations/v1_20"
forgejo_v1_22 "code.gitea.io/gitea/models/forgejo_migrations/v1_22"
[DB] Forgejo database migrations - Implements https://codeberg.org/forgejo/discussions/issues/32#issuecomment-918737 - Allows to add Forgejo-specific migrations that don't interfere with Gitea's migration logic. Please do note that we cannot liberally add migrations for Gitea tables, as they might do their own migrations in a future version on that table, and that could undo our migrations. Luckily, we don't have a scenario where that's needed and thus not taken into account. Co-authored-by: Gusted <postmaster@gusted.xyz> Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/795 (cherry picked from commit 8ee32978c0af1f8f71679c87f695df2b90b617c8) (cherry picked from commit c240b34f595a7a9763f7b748052ac98f9f18954d) (cherry picked from commit 03936c649243a0a29701393d58e63e33064c7461) (cherry picked from commit a20ed852f8b6d28872c05d688bffe5c6976bfa03) (cherry picked from commit 1dfa82676f1feb745633618fde2d362bf19c4f28) (cherry picked from commit c39ae0bf8abced8fd5dc32589e68515ac308b69b) (cherry picked from commit cfaff08996c9f42592c95a63fe907b45b8a9317a) (cherry picked from commit 94a458835a2b0336b26c1c9df64fdfe2de47f496) (cherry picked from commit 61a3cf77dfe3f612ff110eb19f94dcb08051daf1) (cherry picked from commit abb350fde879cc495761dc4616b7aa0fc5d94d54) (cherry picked from commit 5194829d6b4ed702cf50ff875da57d04d77c8a18) (cherry picked from commit 89239a60f23cad7dad03add744e23a4f3b10d6a4) (cherry picked from commit 683cfd86efc5fa8cc04973ce3115351515a20917) (cherry picked from commit f4546cfed92844e3666b80130eadabb9348b88ae) (cherry picked from commit 86614d5826392b3fbe68355baeab9a0a761883a5) (cherry picked from commit e4b9c32187a039a83686a82856a9a192919c6e82) (cherry picked from commit 8c253719afa9b82f169757df007587d38560c06d) (cherry picked from commit 857365d6c15b5471d63662b1d89d1523151c4f79) (cherry picked from commit a488b3952f58bbf28bfa101a24e52dad7c9662eb) (cherry picked from commit 98313c49109c941426beecc1a3e7887f28b99970) (cherry picked from commit 430d95e8240971e266705d2e7202a5c785379cb2) (cherry picked from commit 08bf9d918fbb67f5ac06c0cfdc24229aa14ff83f) (cherry picked from commit f8a170e2d042fcb8f314e123de6918317ac1e909) (cherry picked from commit d20e325378e67087279496d35b575e566836aaa1) (cherry picked from commit 6c0aa7dd4fd8c234984d455933f69f51abcb2d32) (cherry picked from commit 46c08c26c7bd3260b3ac7678f24566b467f4a2fb) (cherry picked from commit 9ee22153c4ec62392693c9151d5395221d097f70) [DB] Ensure forgejo migration up to date (squash) - Hook Forgejo's `EnsureUpToDate` to Gitea's `EnsureUpToDate`, such that the Forgejo migrations are also being checked to be up to date. - I'm not sure how I missed this and if this has caused any problems, but due to the lack of any open issue about it it seems to not be a big problem. (cherry picked from commit 6c65b6dcf6ab0d58e5c2d03a866e4e38294f72ad) (cherry picked from commit 6d45c37d843147e69b0a27ebe35c617d7f574b76) [DB] Add test for TestEnsureUpToDate (squash) - Add a test for the behavior of `EnsureUpToDate`, to ensure it will error when needed and succeed when the forgejo version is up to date. - Add forgejo_migrations package to GO_TEST_PACKAGES, to avoid running it with `test-unit` and instead test it with `test-*-migration`. (cherry picked from commit b172a506914fee40a50daa51f0c8e547427fd2f8) (cherry picked from commit d8af3088205b592340fd836135ffe97da9cec5a6) (cherry picked from commit e69e64a32c5e38247e94ab880536e3cfeab67cc6) (cherry picked from commit 4e8363fad4e08845960912a3ea3fe7265ee60602) (cherry picked from commit fc9ecd6c533eca864503423cf4a21710984a6b75) (cherry picked from commit e5c446e3dc9bc6e9549862f7b764a634f4fbaaae) (cherry picked from commit 7066a15655a33f57ccfb68cf2cb994ea57ad3666) (cherry picked from commit 9183cdc8354d529a1c2b570551bc1578fb10d58b) (cherry picked from commit 5f93039e0d7c8a7eb79df16ce0d8603f948b1bd2) Conflicts: Makefile https://codeberg.org/forgejo/forgejo/pulls/2245 (cherry picked from commit a039b3b0c9a7016de9e7e71ea0cc7a1185adb8d9)
2023-06-03 14:11:16 +05:30
"code.gitea.io/gitea/modules/git"
"code.gitea.io/gitea/modules/log"
"code.gitea.io/gitea/modules/setting"
"xorm.io/xorm"
"xorm.io/xorm/names"
)
// ForgejoVersion describes the Forgejo version table. Should have only one row with id = 1.
type ForgejoVersion struct {
ID int64 `xorm:"pk autoincr"`
Version int64
}
type Migration struct {
description string
migrate func(*xorm.Engine) error
}
// NewMigration creates a new migration.
func NewMigration(desc string, fn func(*xorm.Engine) error) *Migration {
return &Migration{desc, fn}
}
// This is a sequence of additional Forgejo migrations.
// Add new migrations to the bottom of the list.
[DB] forgejo migration v2: create the forgejo_sem_ver table (cherry picked from commit 86b26436af85e0eedb732e115e8be024e1d54ca6) (cherry picked from commit 479cba59aca2b3b73a83e5acc0b754906230f0b2) (cherry picked from commit 4765f9a889ce7324416b51e7d4524b2368459752) (cherry picked from commit af771410bfaa511bbc97cbfeefbb279b3836158d) (cherry picked from commit d1ea9305d8d653d637cbde6d383c095bdc4991f0) (cherry picked from commit f77e1bb7abd6054851b811fc0d71bf0130085353) (cherry picked from commit 0b95f8fe899ce14e19e5aeccd3e53e21b8bf8fc1) (cherry picked from commit 4f8fb2390a415d2f9319c9f23ff653c31efd2409) (cherry picked from commit 8ea0e22ff6e09f1e46d9a10f1325f7c6997c018b) (cherry picked from commit 43ac19ac59044210906c96857194231ca7804c92) (cherry picked from commit 0d2f63df4f9579c62ed21c807c53e689b1c71dbb) (cherry picked from commit d02a8036fda2bfcf8791366198e25735547d6b3f) (cherry picked from commit 1fe4c7db941b892df38be12249d4898248ff016e) (cherry picked from commit d641cdeaf0f77ef12fb5fe5c27c20804757a74ef) (cherry picked from commit 10e8a4f8b864961a700f2c3e6c816489bf4e4ac4) (cherry picked from commit 8097bc40b3f90ed959adc9edd1982e5867856397) (cherry picked from commit ee5cb37d7e9567c129fcf2a606175d425dacace1) (cherry picked from commit 26d93b8e49270440a940cd2e12b36c449644af94) (cherry picked from commit bc73195e5945a89bedb2faac579eac2cc403195e) (cherry picked from commit a763fa5de29e6e5986e0f4971e0a9696aa98a024) (cherry picked from commit 4b000cb435b1ddec2aee901b9d257e0af87b7698) (cherry picked from commit 9f6d20e73f73fefd78e64d8fc11af9118bef9557) (cherry picked from commit b5001edeeabf29d5b494da32fc09fa911b38cdb8) (cherry picked from commit 76321718328532a4ed85a04839c339dd42b87d42) (cherry picked from commit 816c5b0c4e7c599d513033bf5eb01bb6259dd144)
2023-08-09 03:22:37 +05:30
var migrations = []*Migration{
[DB] forgejo migration v1: add blocked user migration (cherry picked from commit 66afddd511d2821f648919925ea365cd085e4e77) (cherry picked from commit 19da0dee9df87cb946d1c8e25b917f7283d95302) (cherry picked from commit 0b725af693828bd7705b09433fb25d974fbb820e) (cherry picked from commit 64d4de2b664a9f406b2927c6e6b465ee2b3a8915) (cherry picked from commit 05bc9d3b7f8d00a37b41615f9caccca84a98c0ee) (cherry picked from commit 5958553066d275c767018454fbed9470c2cf473d) (cherry picked from commit c4f77e26c913eb3293bc702e1c65e2d09757cfe1) (cherry picked from commit 3034832c6637965bf70c766d9169ca3c28c27cd3) (cherry picked from commit d48931ec5b677989f8b2ef91752b2e3ef25ff2a2) (cherry picked from commit b1e0d53c28369666367e2629b4226338a111e0e2) (cherry picked from commit e3de35fe15c33d87f1a33e193fa7e038c5e3a724) (cherry picked from commit 3b2712c3d6bd0884166787e8c257305d54247112) (cherry picked from commit 00c6940851ca270934a95de0ccfff2eadf41096f) (cherry picked from commit ac56c7a202b934d8d46d2eb578d83e953f3b50f5) (cherry picked from commit d957fb66de9bb8d04cf184c88b4a3ca2ab7728eb) (cherry picked from commit 01e9125f1b172c8a78a380c611d2244df104178f) (cherry picked from commit 71675caecb545cda069d766f16feb6bd6924e9ec) (cherry picked from commit 0313d02e50ee64729fdd3aa661b481ae5d32c029) (cherry picked from commit d2952769c1217b984936a1bf6a0d4f072217e122) (cherry picked from commit 63d080cdcfcaac34aa5a3040e00d33506224c112) (cherry picked from commit 43ced29b28c2631d0d5ec7320ab7b40d9f8e06c9) (cherry picked from commit e556074abdc39ec02156de47492a35ae3e278b9f) (cherry picked from commit 165409d8ff75a06563447d490d66c69984e4a04a) (cherry picked from commit 0a3f370162dcb6c945095864248b8dc55208120f)
2023-08-15 04:36:13 +05:30
// v0 -> v1
NewMigration("Add Forgejo Blocked Users table", forgejo_v1_20.AddForgejoBlockedUser),
[DB] forgejo migration v2: create the forgejo_sem_ver table (cherry picked from commit 86b26436af85e0eedb732e115e8be024e1d54ca6) (cherry picked from commit 479cba59aca2b3b73a83e5acc0b754906230f0b2) (cherry picked from commit 4765f9a889ce7324416b51e7d4524b2368459752) (cherry picked from commit af771410bfaa511bbc97cbfeefbb279b3836158d) (cherry picked from commit d1ea9305d8d653d637cbde6d383c095bdc4991f0) (cherry picked from commit f77e1bb7abd6054851b811fc0d71bf0130085353) (cherry picked from commit 0b95f8fe899ce14e19e5aeccd3e53e21b8bf8fc1) (cherry picked from commit 4f8fb2390a415d2f9319c9f23ff653c31efd2409) (cherry picked from commit 8ea0e22ff6e09f1e46d9a10f1325f7c6997c018b) (cherry picked from commit 43ac19ac59044210906c96857194231ca7804c92) (cherry picked from commit 0d2f63df4f9579c62ed21c807c53e689b1c71dbb) (cherry picked from commit d02a8036fda2bfcf8791366198e25735547d6b3f) (cherry picked from commit 1fe4c7db941b892df38be12249d4898248ff016e) (cherry picked from commit d641cdeaf0f77ef12fb5fe5c27c20804757a74ef) (cherry picked from commit 10e8a4f8b864961a700f2c3e6c816489bf4e4ac4) (cherry picked from commit 8097bc40b3f90ed959adc9edd1982e5867856397) (cherry picked from commit ee5cb37d7e9567c129fcf2a606175d425dacace1) (cherry picked from commit 26d93b8e49270440a940cd2e12b36c449644af94) (cherry picked from commit bc73195e5945a89bedb2faac579eac2cc403195e) (cherry picked from commit a763fa5de29e6e5986e0f4971e0a9696aa98a024) (cherry picked from commit 4b000cb435b1ddec2aee901b9d257e0af87b7698) (cherry picked from commit 9f6d20e73f73fefd78e64d8fc11af9118bef9557) (cherry picked from commit b5001edeeabf29d5b494da32fc09fa911b38cdb8) (cherry picked from commit 76321718328532a4ed85a04839c339dd42b87d42) (cherry picked from commit 816c5b0c4e7c599d513033bf5eb01bb6259dd144)
2023-08-09 03:22:37 +05:30
// v1 -> v2
NewMigration("create the forgejo_sem_ver table", forgejo_v1_20.CreateSemVerTable),
[SECURITY] Rework long-term authentication - This is a 'front-port' of the already existing patch on v1.21 and v1.20, but applied on top of what Gitea has done to rework the LTA mechanism. Forgejo will stick with the reworked mechanism by the Forgejo Security team for the time being. The removal of legacy code (AES-GCM) has been left out. - The current architecture is inherently insecure, because you can construct the 'secret' cookie value with values that are available in the database. Thus provides zero protection when a database is dumped/leaked. - This patch implements a new architecture that's inspired from: [Paragonie Initiative](https://paragonie.com/blog/2015/04/secure-authentication-php-with-long-term-persistence#secure-remember-me-cookies). - Integration testing is added to ensure the new mechanism works. - Removes a setting, because it's not used anymore. (cherry picked from commit e3d6622a63da9c33eed1e3d102cf28a92ff653d6) (cherry picked from commit fef1a6dac5e25579e42d40209c4cfc06879948b9) (cherry picked from commit b0c5165145fa52f2f7bbec1f50b308bdf1d20ef3) (cherry picked from commit 7ad51b9f8d0647eecacd258f6ee26155da3872e1) (cherry picked from commit 64f053f3834e764112cde26bb0d16c5e88d6b2af) (cherry picked from commit f5e78e4c204ce50b800645d614218b6b6096eecb) Conflicts: services/auth/auth_token_test.go https://codeberg.org/forgejo/forgejo/pulls/2069 (cherry picked from commit f69fc23d4bbadf388c7857040ee0774b824e418e) (cherry picked from commit d955ab3ab02cbb7f1245a8cddec426d64d3ac500) (cherry picked from commit 9220088f902a25c4690bcabf5a40a8d02e784182) (cherry picked from commit c73ac636962c41c71814c273510146f0533264ab) (cherry picked from commit 747a176048ea93085b406429db0e25bb21912eda) Conflicts: models/user/user.go routers/web/user/setting/account.go https://codeberg.org/forgejo/forgejo/pulls/2295
2023-11-22 21:56:21 +05:30
// v2 -> v3
NewMigration("create the forgejo_auth_token table", forgejo_v1_20.CreateAuthorizationTokenTable),
// v3 -> v4
NewMigration("Add default_permissions to repo_unit", forgejo_v1_22.AddDefaultPermissionsToRepoUnit),
[DB] forgejo migration v2: create the forgejo_sem_ver table (cherry picked from commit 86b26436af85e0eedb732e115e8be024e1d54ca6) (cherry picked from commit 479cba59aca2b3b73a83e5acc0b754906230f0b2) (cherry picked from commit 4765f9a889ce7324416b51e7d4524b2368459752) (cherry picked from commit af771410bfaa511bbc97cbfeefbb279b3836158d) (cherry picked from commit d1ea9305d8d653d637cbde6d383c095bdc4991f0) (cherry picked from commit f77e1bb7abd6054851b811fc0d71bf0130085353) (cherry picked from commit 0b95f8fe899ce14e19e5aeccd3e53e21b8bf8fc1) (cherry picked from commit 4f8fb2390a415d2f9319c9f23ff653c31efd2409) (cherry picked from commit 8ea0e22ff6e09f1e46d9a10f1325f7c6997c018b) (cherry picked from commit 43ac19ac59044210906c96857194231ca7804c92) (cherry picked from commit 0d2f63df4f9579c62ed21c807c53e689b1c71dbb) (cherry picked from commit d02a8036fda2bfcf8791366198e25735547d6b3f) (cherry picked from commit 1fe4c7db941b892df38be12249d4898248ff016e) (cherry picked from commit d641cdeaf0f77ef12fb5fe5c27c20804757a74ef) (cherry picked from commit 10e8a4f8b864961a700f2c3e6c816489bf4e4ac4) (cherry picked from commit 8097bc40b3f90ed959adc9edd1982e5867856397) (cherry picked from commit ee5cb37d7e9567c129fcf2a606175d425dacace1) (cherry picked from commit 26d93b8e49270440a940cd2e12b36c449644af94) (cherry picked from commit bc73195e5945a89bedb2faac579eac2cc403195e) (cherry picked from commit a763fa5de29e6e5986e0f4971e0a9696aa98a024) (cherry picked from commit 4b000cb435b1ddec2aee901b9d257e0af87b7698) (cherry picked from commit 9f6d20e73f73fefd78e64d8fc11af9118bef9557) (cherry picked from commit b5001edeeabf29d5b494da32fc09fa911b38cdb8) (cherry picked from commit 76321718328532a4ed85a04839c339dd42b87d42) (cherry picked from commit 816c5b0c4e7c599d513033bf5eb01bb6259dd144)
2023-08-09 03:22:37 +05:30
}
[DB] Forgejo database migrations - Implements https://codeberg.org/forgejo/discussions/issues/32#issuecomment-918737 - Allows to add Forgejo-specific migrations that don't interfere with Gitea's migration logic. Please do note that we cannot liberally add migrations for Gitea tables, as they might do their own migrations in a future version on that table, and that could undo our migrations. Luckily, we don't have a scenario where that's needed and thus not taken into account. Co-authored-by: Gusted <postmaster@gusted.xyz> Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/795 (cherry picked from commit 8ee32978c0af1f8f71679c87f695df2b90b617c8) (cherry picked from commit c240b34f595a7a9763f7b748052ac98f9f18954d) (cherry picked from commit 03936c649243a0a29701393d58e63e33064c7461) (cherry picked from commit a20ed852f8b6d28872c05d688bffe5c6976bfa03) (cherry picked from commit 1dfa82676f1feb745633618fde2d362bf19c4f28) (cherry picked from commit c39ae0bf8abced8fd5dc32589e68515ac308b69b) (cherry picked from commit cfaff08996c9f42592c95a63fe907b45b8a9317a) (cherry picked from commit 94a458835a2b0336b26c1c9df64fdfe2de47f496) (cherry picked from commit 61a3cf77dfe3f612ff110eb19f94dcb08051daf1) (cherry picked from commit abb350fde879cc495761dc4616b7aa0fc5d94d54) (cherry picked from commit 5194829d6b4ed702cf50ff875da57d04d77c8a18) (cherry picked from commit 89239a60f23cad7dad03add744e23a4f3b10d6a4) (cherry picked from commit 683cfd86efc5fa8cc04973ce3115351515a20917) (cherry picked from commit f4546cfed92844e3666b80130eadabb9348b88ae) (cherry picked from commit 86614d5826392b3fbe68355baeab9a0a761883a5) (cherry picked from commit e4b9c32187a039a83686a82856a9a192919c6e82) (cherry picked from commit 8c253719afa9b82f169757df007587d38560c06d) (cherry picked from commit 857365d6c15b5471d63662b1d89d1523151c4f79) (cherry picked from commit a488b3952f58bbf28bfa101a24e52dad7c9662eb) (cherry picked from commit 98313c49109c941426beecc1a3e7887f28b99970) (cherry picked from commit 430d95e8240971e266705d2e7202a5c785379cb2) (cherry picked from commit 08bf9d918fbb67f5ac06c0cfdc24229aa14ff83f) (cherry picked from commit f8a170e2d042fcb8f314e123de6918317ac1e909) (cherry picked from commit d20e325378e67087279496d35b575e566836aaa1) (cherry picked from commit 6c0aa7dd4fd8c234984d455933f69f51abcb2d32) (cherry picked from commit 46c08c26c7bd3260b3ac7678f24566b467f4a2fb) (cherry picked from commit 9ee22153c4ec62392693c9151d5395221d097f70) [DB] Ensure forgejo migration up to date (squash) - Hook Forgejo's `EnsureUpToDate` to Gitea's `EnsureUpToDate`, such that the Forgejo migrations are also being checked to be up to date. - I'm not sure how I missed this and if this has caused any problems, but due to the lack of any open issue about it it seems to not be a big problem. (cherry picked from commit 6c65b6dcf6ab0d58e5c2d03a866e4e38294f72ad) (cherry picked from commit 6d45c37d843147e69b0a27ebe35c617d7f574b76) [DB] Add test for TestEnsureUpToDate (squash) - Add a test for the behavior of `EnsureUpToDate`, to ensure it will error when needed and succeed when the forgejo version is up to date. - Add forgejo_migrations package to GO_TEST_PACKAGES, to avoid running it with `test-unit` and instead test it with `test-*-migration`. (cherry picked from commit b172a506914fee40a50daa51f0c8e547427fd2f8) (cherry picked from commit d8af3088205b592340fd836135ffe97da9cec5a6) (cherry picked from commit e69e64a32c5e38247e94ab880536e3cfeab67cc6) (cherry picked from commit 4e8363fad4e08845960912a3ea3fe7265ee60602) (cherry picked from commit fc9ecd6c533eca864503423cf4a21710984a6b75) (cherry picked from commit e5c446e3dc9bc6e9549862f7b764a634f4fbaaae) (cherry picked from commit 7066a15655a33f57ccfb68cf2cb994ea57ad3666) (cherry picked from commit 9183cdc8354d529a1c2b570551bc1578fb10d58b) (cherry picked from commit 5f93039e0d7c8a7eb79df16ce0d8603f948b1bd2) Conflicts: Makefile https://codeberg.org/forgejo/forgejo/pulls/2245 (cherry picked from commit a039b3b0c9a7016de9e7e71ea0cc7a1185adb8d9)
2023-06-03 14:11:16 +05:30
// GetCurrentDBVersion returns the current Forgejo database version.
func GetCurrentDBVersion(x *xorm.Engine) (int64, error) {
if err := x.Sync(new(ForgejoVersion)); err != nil {
return -1, fmt.Errorf("sync: %w", err)
}
currentVersion := &ForgejoVersion{ID: 1}
has, err := x.Get(currentVersion)
if err != nil {
return -1, fmt.Errorf("get: %w", err)
}
if !has {
return -1, nil
}
return currentVersion.Version, nil
}
// ExpectedVersion returns the expected Forgejo database version.
func ExpectedVersion() int64 {
return int64(len(migrations))
}
// EnsureUpToDate will check if the Forgejo database is at the correct version.
func EnsureUpToDate(x *xorm.Engine) error {
currentDB, err := GetCurrentDBVersion(x)
if err != nil {
return err
}
if currentDB < 0 {
return fmt.Errorf("database has not been initialized")
}
expected := ExpectedVersion()
if currentDB != expected {
return fmt.Errorf(`current Forgejo database version %d is not equal to the expected version %d. Please run "forgejo [--config /path/to/app.ini] migrate" to update the database version`, currentDB, expected)
}
return nil
}
// Migrate Forgejo database to current version.
func Migrate(x *xorm.Engine) error {
// Set a new clean the default mapper to GonicMapper as that is the default for .
x.SetMapper(names.GonicMapper{})
if err := x.Sync(new(ForgejoVersion)); err != nil {
return fmt.Errorf("sync: %w", err)
}
currentVersion := &ForgejoVersion{ID: 1}
has, err := x.Get(currentVersion)
if err != nil {
return fmt.Errorf("get: %w", err)
} else if !has {
// If the version record does not exist we think
// it is a fresh installation and we can skip all migrations.
currentVersion.ID = 0
currentVersion.Version = ExpectedVersion()
if _, err = x.InsertOne(currentVersion); err != nil {
return fmt.Errorf("insert: %w", err)
}
}
v := currentVersion.Version
// Downgrading Forgejo's database version not supported
if v > ExpectedVersion() {
msg := fmt.Sprintf("Your Forgejo database (migration version: %d) is for a newer version of Forgejo, you cannot use the newer database for this old Forgejo release (%d).", v, ExpectedVersion())
msg += "\nForgejo will exit to keep your database safe and unchanged. Please use the correct Forgejo release, do not change the migration version manually (incorrect manual operation may cause data loss)."
if !setting.IsProd {
msg += fmt.Sprintf("\nIf you are in development and really know what you're doing, you can force changing the migration version by executing: UPDATE forgejo_version SET version=%d WHERE id=1;", ExpectedVersion())
}
_, _ = fmt.Fprintln(os.Stderr, msg)
log.Fatal(msg)
return nil
}
// Some migration tasks depend on the git command
if git.DefaultContext == nil {
if err = git.InitSimple(context.Background()); err != nil {
return err
}
}
// Migrate
for i, m := range migrations[v:] {
log.Info("Migration[%d]: %s", v+int64(i), m.description)
// Reset the mapper between each migration - migrations are not supposed to depend on each other
x.SetMapper(names.GonicMapper{})
if err = m.migrate(x); err != nil {
return fmt.Errorf("migration[%d]: %s failed: %w", v+int64(i), m.description, err)
}
currentVersion.Version = v + int64(i) + 1
if _, err = x.ID(1).Update(currentVersion); err != nil {
return err
}
}
[SEMVER] store SemVer in ForgejoSemVer after a database upgrade (cherry picked from commit b7fe7cf401f4bddd6455efc651f7ac054f3fe1cf) (cherry picked from commit cf339eed4f4851b18448dbdd83df32d00bc6f45b) (cherry picked from commit 4f3a16168bbeced519a60c32e10e2895d9367238) (cherry picked from commit 6f5bbc53fcebd614f5ee8627c7d6e3c637ffa694) (cherry picked from commit aca42b422e76668387769c15868ef77b073bb7db) (cherry picked from commit 5a7f7580e525694d1f27e12329c8532bd29273f8) (cherry picked from commit 06c383c807ad49e1e35b429a10c6a11c65aeebe5) (cherry picked from commit fe831dcb53b81b6cc632be751bfcbfb9bc00efd3) (cherry picked from commit cd12cd0dbce47c6117ea579ae5019c182155b3a9) (cherry picked from commit cc79163703ce31706c86b88c38fb8a20ed745e20) (cherry picked from commit 0102a5715ea6a03b560cc2f0b6cbe2b2576c255e) (cherry picked from commit 403f7520b3056eace36eae505afbab6a05f597b7) (cherry picked from commit a3b61510a246f61c174ddc3c288e556522a6aab9) (cherry picked from commit f83f0f9feb76a8b62ca4d74dc2785c713fbec282) (cherry picked from commit fd1c3a6d09a057070844cd955d0920518e60f408) (cherry picked from commit f7cdc3d6f1ad1ff01bb1814207fcb41210db80e8) (cherry picked from commit 060121b644e0515a6b673cdd514d52e6fa1e6ec3) (cherry picked from commit 62c847ff0235196e73e860d0d1658f0734985270) (cherry picked from commit 4d051b51c2813828b978ef84b6b72c0ba051741b) (cherry picked from commit 86e6981a936b1937065605892083b2488424bf5e) (cherry picked from commit c1fc9e441b1467033911f3848f82a89aaacf98a9) (cherry picked from commit 8bb2f0871a507d881acf74acf8fb90fd5ebff567) (cherry picked from commit 0cd9fe52511ee06f1b8849325d196f60d8ccd151) (cherry picked from commit b0b44778b4d3e51852d0669dfe6d9a6c334b90e4) (cherry picked from commit 7c2f4f749f1c8986875eebf95254a3db151e5248)
2023-08-09 03:25:25 +05:30
if err := x.Sync(new(semver.ForgejoSemVer)); err != nil {
return fmt.Errorf("sync: %w", err)
}
return semver.SetVersionStringWithEngine(x, setting.ForgejoVersion)
[DB] Forgejo database migrations - Implements https://codeberg.org/forgejo/discussions/issues/32#issuecomment-918737 - Allows to add Forgejo-specific migrations that don't interfere with Gitea's migration logic. Please do note that we cannot liberally add migrations for Gitea tables, as they might do their own migrations in a future version on that table, and that could undo our migrations. Luckily, we don't have a scenario where that's needed and thus not taken into account. Co-authored-by: Gusted <postmaster@gusted.xyz> Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/795 (cherry picked from commit 8ee32978c0af1f8f71679c87f695df2b90b617c8) (cherry picked from commit c240b34f595a7a9763f7b748052ac98f9f18954d) (cherry picked from commit 03936c649243a0a29701393d58e63e33064c7461) (cherry picked from commit a20ed852f8b6d28872c05d688bffe5c6976bfa03) (cherry picked from commit 1dfa82676f1feb745633618fde2d362bf19c4f28) (cherry picked from commit c39ae0bf8abced8fd5dc32589e68515ac308b69b) (cherry picked from commit cfaff08996c9f42592c95a63fe907b45b8a9317a) (cherry picked from commit 94a458835a2b0336b26c1c9df64fdfe2de47f496) (cherry picked from commit 61a3cf77dfe3f612ff110eb19f94dcb08051daf1) (cherry picked from commit abb350fde879cc495761dc4616b7aa0fc5d94d54) (cherry picked from commit 5194829d6b4ed702cf50ff875da57d04d77c8a18) (cherry picked from commit 89239a60f23cad7dad03add744e23a4f3b10d6a4) (cherry picked from commit 683cfd86efc5fa8cc04973ce3115351515a20917) (cherry picked from commit f4546cfed92844e3666b80130eadabb9348b88ae) (cherry picked from commit 86614d5826392b3fbe68355baeab9a0a761883a5) (cherry picked from commit e4b9c32187a039a83686a82856a9a192919c6e82) (cherry picked from commit 8c253719afa9b82f169757df007587d38560c06d) (cherry picked from commit 857365d6c15b5471d63662b1d89d1523151c4f79) (cherry picked from commit a488b3952f58bbf28bfa101a24e52dad7c9662eb) (cherry picked from commit 98313c49109c941426beecc1a3e7887f28b99970) (cherry picked from commit 430d95e8240971e266705d2e7202a5c785379cb2) (cherry picked from commit 08bf9d918fbb67f5ac06c0cfdc24229aa14ff83f) (cherry picked from commit f8a170e2d042fcb8f314e123de6918317ac1e909) (cherry picked from commit d20e325378e67087279496d35b575e566836aaa1) (cherry picked from commit 6c0aa7dd4fd8c234984d455933f69f51abcb2d32) (cherry picked from commit 46c08c26c7bd3260b3ac7678f24566b467f4a2fb) (cherry picked from commit 9ee22153c4ec62392693c9151d5395221d097f70) [DB] Ensure forgejo migration up to date (squash) - Hook Forgejo's `EnsureUpToDate` to Gitea's `EnsureUpToDate`, such that the Forgejo migrations are also being checked to be up to date. - I'm not sure how I missed this and if this has caused any problems, but due to the lack of any open issue about it it seems to not be a big problem. (cherry picked from commit 6c65b6dcf6ab0d58e5c2d03a866e4e38294f72ad) (cherry picked from commit 6d45c37d843147e69b0a27ebe35c617d7f574b76) [DB] Add test for TestEnsureUpToDate (squash) - Add a test for the behavior of `EnsureUpToDate`, to ensure it will error when needed and succeed when the forgejo version is up to date. - Add forgejo_migrations package to GO_TEST_PACKAGES, to avoid running it with `test-unit` and instead test it with `test-*-migration`. (cherry picked from commit b172a506914fee40a50daa51f0c8e547427fd2f8) (cherry picked from commit d8af3088205b592340fd836135ffe97da9cec5a6) (cherry picked from commit e69e64a32c5e38247e94ab880536e3cfeab67cc6) (cherry picked from commit 4e8363fad4e08845960912a3ea3fe7265ee60602) (cherry picked from commit fc9ecd6c533eca864503423cf4a21710984a6b75) (cherry picked from commit e5c446e3dc9bc6e9549862f7b764a634f4fbaaae) (cherry picked from commit 7066a15655a33f57ccfb68cf2cb994ea57ad3666) (cherry picked from commit 9183cdc8354d529a1c2b570551bc1578fb10d58b) (cherry picked from commit 5f93039e0d7c8a7eb79df16ce0d8603f948b1bd2) Conflicts: Makefile https://codeberg.org/forgejo/forgejo/pulls/2245 (cherry picked from commit a039b3b0c9a7016de9e7e71ea0cc7a1185adb8d9)
2023-06-03 14:11:16 +05:30
}