- When the database returns an error about the SQL query, the error is
logged but not the SQL query and arguments, which is just as valuable as
the vague deeply hidden documented error that the database returns.
It's possible to log the SQL query by logging **all** SQL queries. For
bigger instances such as Codeberg, this is not a viable option.
- Adds a new hook, enabled by default, to log SQL queries with their
arguments and the error returned by the database when the database
returns an error.
- This likely needs some fine tuning in the future to decide when to
enable this, as the error is already logged and if people have the
`[database].LOG_SQL` option enabled, the SQL would be logged twice. But
given that it's an rare occurence for SQL queries to error, it's fine to
leave that as-is.
- Ref: https://codeberg.org/forgejo/forgejo/issues/1998
(cherry picked from commit 866229bc323619bc8686bad99951f95d5d46fe19)
(cherry picked from commit 96dd3e87cf5f75ac33bfad647e44852667c307d0)
(cherry picked from commit e165510317b3fe3be4fe49ad3ff0a055e7a92124)
(cherry picked from commit 1638e2b3f56a508dbf218207bf0c3c4d16493255)
- Databases are one of the most important parts of Forgejo, every
interaction with Forgejo uses the database in one way or another.
Therefore, it is important to maintain the database and recognize when
Forgejo is not doing well with the database. Forgejo already has the
option to log *every* SQL query along with its execution time, but
monitoring becomes impractical for larger instances and takes up
unnecessary storage in the logs.
- Add a QoL enhancement that allows instance administrators to specify a
threshold value beyond which query execution time is logged as a warning
in the xorm logger. The default value is a conservative five seconds to
avoid this becoming a source of spam in the logs.
- The use case for this patch is that with an instance the size of Codeberg, monitoring SQL logs is not very fruitful and most of them are uninteresting. Recently, in the context of persistent deadlock issues (https://codeberg.org/forgejo/forgejo/issues/220), I have noticed that certain queries hold locks on tables like comment and issue for several seconds. This patch helps to identify which queries these are and when they happen.
- Added unit test.
(cherry picked from commit 24bbe7886fb4cb9a38c8dab8c44f4c9cbfa25481)
(cherry picked from commit 6e29145b3c1455498531593d38e6a914941a12cb)
(cherry picked from commit 63731e30712872bd2395eb3cf36d9996e5793645)
(cherry picked from commit 3ce1a097369c132654de70df707b867e47bd1c40)
(cherry picked from commit a64426907de788cc0937a7a2b16af4d2f26f7fe6)
(cherry picked from commit 4b1921569156445c58d9889602733da5934c7b95)
(cherry picked from commit e6356744359fa947c049827d60c2ea0e277e03dc)
(cherry picked from commit 9cf501f1af4cd870221cef6af489618785b71186)
(cherry picked from commit 0d6b934eba1c0e9b27b364791113aae816b6b366)
(cherry picked from commit 4b6c2738795002887844a106f2fed2ef1673eed1)
(cherry picked from commit 89b1315338b0c7a726a36a84e9844013a13560b8)
(cherry picked from commit edd8e66ce991c395bb0af7720631c3cd26caaa51)
[GITEA] Add slow SQL query warning (squash) document the setting
(cherry picked from commit ce38599c5141c7fc6bc054819f5ff1c1b45bda1f)
(cherry picked from commit 794aa67c68c8e24ac7301eb7ef767c6e2499a78d)
(cherry picked from commit a4c2c6b004c21488e90f637ca7920f49108ed75d)
(cherry picked from commit 97912752bc802db79bb26a6591aec885aea30ee4)
(cherry picked from commit 00b5327c9750215a290238516e7b6fb1e6601e14)
(cherry picked from commit 1069c860e78c11225b4d74ff3044df7786562821)
(cherry picked from commit 84241f42c83852918b57c8bd25364697037fe42f)
(cherry picked from commit e4bda0e8457d00c01b83f153ed5a4a8ea4cf85c8)
(cherry picked from commit 7357fb91bff87045b133c3a7ac9fc70eea781bc4)
(cherry picked from commit a8dd7f6da278ae112200b5efa5bf27e3961f5996)
(cherry picked from commit e636e9f4beca7273dd8622baedb2f0c01db30449)
(cherry picked from commit bf04ae86037f5cb5a81d02750aead2742b040367)
(cherry picked from commit 93b19e3568169bd1cf9b8b78c1751c3d2d65a1b6)
(cherry picked from commit 83f91363ad071675c73a1f636271cc043bf69707)
(cherry picked from commit e34a05bc7319072b70d387975342d617b8136655)
(cherry picked from commit 68569aeee9805aaa8e98c6e7fe8058095e290061)
(cherry picked from commit b7fe7cf401f4bddd6455efc651f7ac054f3fe1cf)
(cherry picked from commit cf339eed4f4851b18448dbdd83df32d00bc6f45b)
(cherry picked from commit 4f3a16168bbeced519a60c32e10e2895d9367238)
(cherry picked from commit 6f5bbc53fcebd614f5ee8627c7d6e3c637ffa694)
(cherry picked from commit aca42b422e76668387769c15868ef77b073bb7db)
(cherry picked from commit 5a7f7580e525694d1f27e12329c8532bd29273f8)
(cherry picked from commit 06c383c807ad49e1e35b429a10c6a11c65aeebe5)
(cherry picked from commit fe831dcb53b81b6cc632be751bfcbfb9bc00efd3)
(cherry picked from commit cd12cd0dbce47c6117ea579ae5019c182155b3a9)
(cherry picked from commit cc79163703ce31706c86b88c38fb8a20ed745e20)
(cherry picked from commit 0102a5715ea6a03b560cc2f0b6cbe2b2576c255e)
(cherry picked from commit 403f7520b3056eace36eae505afbab6a05f597b7)
(cherry picked from commit a3b61510a246f61c174ddc3c288e556522a6aab9)
(cherry picked from commit f83f0f9feb76a8b62ca4d74dc2785c713fbec282)
(cherry picked from commit fd1c3a6d09a057070844cd955d0920518e60f408)
(cherry picked from commit f7cdc3d6f1ad1ff01bb1814207fcb41210db80e8)
(cherry picked from commit 060121b644e0515a6b673cdd514d52e6fa1e6ec3)
(cherry picked from commit 62c847ff0235196e73e860d0d1658f0734985270)
(cherry picked from commit 4d051b51c2813828b978ef84b6b72c0ba051741b)
(cherry picked from commit 86e6981a936b1937065605892083b2488424bf5e)
(cherry picked from commit c1fc9e441b1467033911f3848f82a89aaacf98a9)
(cherry picked from commit 8bb2f0871a507d881acf74acf8fb90fd5ebff567)
(cherry picked from commit 0cd9fe52511ee06f1b8849325d196f60d8ccd151)
(cherry picked from commit b0b44778b4d3e51852d0669dfe6d9a6c334b90e4)
(cherry picked from commit 7c2f4f749f1c8986875eebf95254a3db151e5248)
The xorm `Sync2` has already been deprecated in favor of `Sync`,
so let's do the same inside the Gitea codebase.
Command used to replace everything:
```sh
for i in $(ag Sync2 --files-with-matches); do vim $i -c ':%sno/Sync2/Sync/g' -c ':wq'; done
```
Some dbs require that all tables have primary keys, see
- #16802
- #21086
We can add a test to keep it from being broken again.
Edit:
~Added missing primary key for `ForeignReference`~ Dropped the
`ForeignReference` table to satisfy the check, so it closes #21086.
More context can be found in comments.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: zeripath <art27@cantab.net>
Change all license headers to comply with REUSE specification.
Fix #16132
Co-authored-by: flynnnnnnnnnn <flynnnnnnnnnn@github>
Co-authored-by: John Olheiser <john.olheiser@gmail.com>
This PR adds a context parameter to a bunch of methods. Some helper
`xxxCtx()` methods got replaced with the normal name now.
Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
A testing cleanup.
This pull request replaces `os.MkdirTemp` with `t.TempDir`. We can use the `T.TempDir` function from the `testing` package to create temporary directory. The directory created by `T.TempDir` is automatically removed when the test and all its subtests complete.
This saves us at least 2 lines (error check, and cleanup) on every instance, or in some cases adds cleanup that we forgot.
Reference: https://pkg.go.dev/testing#T.TempDir
```go
func TestFoo(t *testing.T) {
// before
tmpDir, err := os.MkdirTemp("", "")
require.NoError(t, err)
defer os.RemoveAll(tmpDir)
// now
tmpDir := t.TempDir()
}
```
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
