Commit graph

9 commits

Author SHA1 Message Date
Loïc Dachary 689c48cdc0
[CI] set PASSWORD_HASH_ALGO = argon2 for integration tests
(cherry picked from commit 1d7ce2a39c841e77492ef08c0e86c3544ecca88d)
(cherry picked from commit 3af8757583ab9bf8fc5eb05ed1e6901a34682512)
(cherry picked from commit 0b818152096d58e402e3c8bbc2415695af0cdfb9)
(cherry picked from commit 7180a3222a00c33ec7dc9bcaa7e5a1e3ae924735)
(cherry picked from commit 069044f4a6dd7457662e5ca296c714275d16becf)
(cherry picked from commit dd1438f38995e397399d8957fca696f82b73707e)
(cherry picked from commit 4ae6b5d64d59902ce003199b5d652d6c4799ffcf)
(cherry picked from commit c572666d78d0f2cf2a33d5a50ea363430ac9a0a9)
(cherry picked from commit de0a4480bc3e40d61aa5342d93a2ef510abe3826)
(cherry picked from commit 60272ef4f73608407ffedd27892376b6c36a7069)
(cherry picked from commit a68fdde4becef625ce4e654ba6ea8ebee55251e5)
(cherry picked from commit ff448cf3a698dd45c713b734fce3e374c99940bf)
(cherry picked from commit 208498caaab339d4543abfb2b77de3c5793a73c8)
(cherry picked from commit 1abd3efad842f276f1c768ab40213200b62a0d3b)
(cherry picked from commit 818c7f4ca9d59afd75d6cb9597c67cb0564cf775)
(cherry picked from commit 1e552bbe44f79d9a2f5567a0180505b9a41d39d3)
(cherry picked from commit 88dbd6da1d7b4a7faf9dda2d25d68f3202c250a5)
(cherry picked from commit b1c86fb7775dd34511f4d5069db1bd3e5ef68e7a)
(cherry picked from commit 185b1e25b12b56c9ae904adb787854da9b74ddc2)
(cherry picked from commit 0af35605ae8a2594ae0b594278748242bce9d8e0)
(cherry picked from commit 350975b63f12df508e7d8d01845032d32f30035c)
(cherry picked from commit 2788142515ff6bb316a120be60556f39659ee9ea)
(cherry picked from commit b418a53717b247e10d2dda26d1d236400d423a94)
(cherry picked from commit 086cde3f10d786cd3c05bf59e359f08a23183d91)
(cherry picked from commit 3e4818c779bd894b4b86a3672d24623fa45e1824)
(cherry picked from commit f2bf1fe64f9ebf8460a93805289fdba2d5cad6d3)
(cherry picked from commit db4eb288ee464c9a799ff2e059c0a5aa632e633b)
(cherry picked from commit f8db31821b8089d5c9390f902854b7817a0381ef)
(cherry picked from commit 2b13c06f499110ba76d3121d1c2f09df3023564f)
(cherry picked from commit 3fd0092056a4e0f43ff59b2123539ddb1ea0afcc)
(cherry picked from commit 6f7662507f26c34a3de54b3e69ea65ab10921163)
(cherry picked from commit 8a15e41887ee3a72fdb5b7718d2750d48615205b)

Conflicts:
	tests/mysql8.ini.tmpl
	https://codeberg.org/forgejo/forgejo/pulls/1573
(cherry picked from commit 56e67e34e2b4d07bc516aa38bf691b5a195821fe)
(cherry picked from commit 6a049a939babf4422ac18f2561a77afea9218861)
(cherry picked from commit 454fe7f5e1113d310375bfbc8ebd44b53ce75e2d)
(cherry picked from commit 93fa4406b74a1019be44f4b40b80d43a777c0e73)
(cherry picked from commit eab1c7222755e06e4308ed3b2c5d7b23c96f54bb)
(cherry picked from commit 8228adfd26616743138de4ab279d2714e8dcc8cf)
(cherry picked from commit ffdbe1bac8d7beddef775bb46a8c33342ee93528)
(cherry picked from commit 7df440771403bae81fa9c0def90bdbe95a4dd53b)
(cherry picked from commit 9d8c4bc7cad4d56528a19d7892b0f08e21dd252a)
(cherry picked from commit 6696e09b1f644bc176d2ca0129d85d8de23d3e4c)
(cherry picked from commit 1bdc0e345807800e2deffc8f9677d708a06a4828)
(cherry picked from commit e2d1598475f5278c7aa14379086d775bbcec7b9a)
(cherry picked from commit 6e0928ae00ba06b14b4d8d3326102ef4f2a91b89)
(cherry picked from commit eede74e7096f200677909d59b32a18808d143605)
(cherry picked from commit b4fb5f64c68089a9fb28f63e705f68fce29c568e)
(cherry picked from commit cf05ec745a0f00dbd7235e91c669f285aa3d3e8e)
2024-02-05 13:33:58 +01:00
Kyle D 33439b733a
Disable query token param in integration tests (#28592)
Follow up to https://github.com/go-gitea/gitea/pull/28484, this PR
enables the setting for integration tests and migrates a few additional
test queries.
2023-12-23 11:29:51 +08:00
wxiaoguang 79c3329502
Do not use deprecated log config options by default (#26592)
Simplify the log config

* Remove unnecessary `ROUTER` config, it defaults to the `MODE`.
* `XORM` config was deprecated
2023-08-20 01:05:29 +00:00
Felix Victor 0533187f28
Update email-setup.en-us.md (#26068)
The setting `MAILER_TYPE` is deprecated.
According to the config cheat sheet, it should be `PROTOCOL`.

---------

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2023-07-26 21:50:15 +08:00
FuXiaoHei c757765a9e
Implement actions artifacts (#22738)
Implement action artifacts server api.

This change is used for supporting
https://github.com/actions/upload-artifact and
https://github.com/actions/download-artifact in gitea actions. It can
run sample workflow from doc
https://docs.github.com/en/actions/using-workflows/storing-workflow-data-as-artifacts.
The api design is inspired by
https://github.com/nektos/act/blob/master/pkg/artifacts/server.go and
includes some changes from gitea internal structs and methods.

Actions artifacts contains two parts:

- Gitea server api and storage (this pr implement basic design without
some complex cases supports)
- Runner communicate with gitea server api (in comming)

Old pr https://github.com/go-gitea/gitea/pull/22345 is outdated after
actions merged. I create new pr from main branch.


![897f7694-3e0f-4f7c-bb4b-9936624ead45](https://user-images.githubusercontent.com/2142787/219382371-eb3cf810-e4e0-456b-a8ff-aecc2b1a1032.jpeg)

Add artifacts list in actions workflow page.
2023-05-19 21:37:57 +08:00
wxiaoguang 6f9c278559
Rewrite queue (#24505)
# ⚠️ Breaking

Many deprecated queue config options are removed (actually, they should
have been removed in 1.18/1.19).

If you see the fatal message when starting Gitea: "Please update your
app.ini to remove deprecated config options", please follow the error
messages to remove these options from your app.ini.

Example:

```
2023/05/06 19:39:22 [E] Removed queue option: `[indexer].ISSUE_INDEXER_QUEUE_TYPE`. Use new options in `[queue.issue_indexer]`
2023/05/06 19:39:22 [E] Removed queue option: `[indexer].UPDATE_BUFFER_LEN`. Use new options in `[queue.issue_indexer]`
2023/05/06 19:39:22 [F] Please update your app.ini to remove deprecated config options
```

Many options in `[queue]` are are dropped, including:
`WRAP_IF_NECESSARY`, `MAX_ATTEMPTS`, `TIMEOUT`, `WORKERS`,
`BLOCK_TIMEOUT`, `BOOST_TIMEOUT`, `BOOST_WORKERS`, they can be removed
from app.ini.

# The problem

The old queue package has some legacy problems:

* complexity: I doubt few people could tell how it works.
* maintainability: Too many channels and mutex/cond are mixed together,
too many different structs/interfaces depends each other.
* stability: due to the complexity & maintainability, sometimes there
are strange bugs and difficult to debug, and some code doesn't have test
(indeed some code is difficult to test because a lot of things are mixed
together).
* general applicability: although it is called "queue", its behavior is
not a well-known queue.
* scalability: it doesn't seem easy to make it work with a cluster
without breaking its behaviors.

It came from some very old code to "avoid breaking", however, its
technical debt is too heavy now. It's a good time to introduce a better
"queue" package.

# The new queue package

It keeps using old config and concept as much as possible.

* It only contains two major kinds of concepts:
    * The "base queue": channel, levelqueue, redis
* They have the same abstraction, the same interface, and they are
tested by the same testing code.
* The "WokerPoolQueue", it uses the "base queue" to provide "worker
pool" function, calls the "handler" to process the data in the base
queue.
* The new code doesn't do "PushBack"
* Think about a queue with many workers, the "PushBack" can't guarantee
the order for re-queued unhandled items, so in new code it just does
"normal push"
* The new code doesn't do "pause/resume"
* The "pause/resume" was designed to handle some handler's failure: eg:
document indexer (elasticsearch) is down
* If a queue is paused for long time, either the producers blocks or the
new items are dropped.
* The new code doesn't do such "pause/resume" trick, it's not a common
queue's behavior and it doesn't help much.
* If there are unhandled items, the "push" function just blocks for a
few seconds and then re-queue them and retry.
* The new code doesn't do "worker booster"
* Gitea's queue's handlers are light functions, the cost is only the
go-routine, so it doesn't make sense to "boost" them.
* The new code only use "max worker number" to limit the concurrent
workers.
* The new "Push" never blocks forever
* Instead of creating more and more blocking goroutines, return an error
is more friendly to the server and to the end user.

There are more details in code comments: eg: the "Flush" problem, the
strange "code.index" hanging problem, the "immediate" queue problem.

Almost ready for review.

TODO:

* [x] add some necessary comments during review
* [x] add some more tests if necessary
* [x] update documents and config options
* [x] test max worker / active worker
* [x] re-run the CI tasks to see whether any test is flaky
* [x] improve the `handleOldLengthConfiguration` to provide more
friendly messages
* [x] fine tune default config values (eg: length?)

## Code coverage:

![image](https://user-images.githubusercontent.com/2114189/236620635-55576955-f95d-4810-b12f-879026a3afdf.png)
2023-05-08 19:49:59 +08:00
wxiaoguang 5b5f7b756b
Clean some legacy files and move some build files (#23699)
* Clean the "tools" directory. The "tools" directory contains only two
files, move them.
* The "external_renderer.go" works like "cat" command to echo Stdin to
Stdout , to help testing.
* The `// gobuild: external_renderer` is incorrect, there should be no
space: `//gobuild: external_renderer`
* The `fmt.Print(os.Args[1])` is not a well-defined behavior, and it's
never used.
* The "watch.sh" is for "make watch", it's somewhat related to "build"
* After this PR, there is no "tools" directory, the project root
directory looks slightly simpler than before.
* Remove the legacy "contrib/autoboot.sh", there is no
"gogs_supervisord.sh"
* Remove the legacy "contrib/mysql.sql", it's never mentioned anywhere.
* Remove the legacy "contrib/pr/checkout.go", it has been broken for
long time, and it introduces unnecessary dependencies of the main code
base.
2023-03-25 16:22:51 -04:00
Lunny Xiao 36a2d2f919
Add a simple test for external renderer (#20033)
Fix #16402
2022-12-12 20:45:21 +08:00
Kyle D c8ded77680
Kd/ci playwright go test (#20123)
* Add initial playwright config

* Simplify Makefile

* Simplify Makefile

* Use correct config files

* Update playwright settings

* Fix package-lock file

* Don't use test logger for e2e tests

* fix frontend lint

* Allow passing TEST_LOGGER variable

* Init postgres database

* use standard gitea env variables

* Update playwright

* update drone

* Move empty env var to commands

* Cleanup

* Move integrations to subfolder

* tests integrations to tests integraton

* Run e2e tests with go test

* Fix linting

* install CI deps

* Add files to ESlint

* Fix drone typo

* Don't log to console in CI

* Use go test http server

* Add build step before tests

* Move shared init function to common package

* fix drone

* Clean up tests

* Fix linting

* Better mocking for page + version string

* Cleanup test generation

* Remove dependency on gitea binary

* Fix linting

* add initial support for running specific tests

* Add ACCEPT_VISUAL variable

* don't require git-lfs

* Add initial documentation

* Review feedback

* Add logged in session test

* Attempt fixing drone race

* Cleanup and bump version

* Bump deps

* Review feedback

* simplify installation

* Fix ci

* Update install docs
2022-09-02 15:18:23 -04:00
Renamed from integrations/sqlite.ini.tmpl (Browse further)