Summary:
- Move existing test under a `testify` Suite as `baseRedisWithServerTestSuite`
- Those tests require real redis server.
- Add `go.uber.org/mock/mockgen@latest` as dependency
- as a tool (Makefile).
- in the `go.mod` file.
- Mock redis client lives under a `mock` directory under the queue module.
- That mock module has an extra hand-written mock in-memory redis-like struct.
- Add tests using the mock redis client.
- Changed the logic around queue provider creation.
- Now the `getNewQueue` returns a Queue provider directly, not an init
function to create it.
The whole Queue module is close to impossible to test properly because
everything is private, everything goes through a struct route. Because
of that, we can't test for example what keys are used for given queue.
To overcome this, as a first step I removed one step from that hard
route by allowing custom calls to create new queue provider. To achieve
this, I moved the creation logic into the `getNewQueue` (previously it
was `getNewQueueFn`). That changes nothing on that side, everything goes
as before, except the `newXXX` call happens directly in that function
and not outside that.
That made it possible to add extra provider specific parameters to those
function (`newXXX`). For example a client on redis. Calling it through
the `getNewQueue` function, it gets `nil`.
- If the provided client is not `nil`, it will use that instead of the
connection string.
- If it's `nil` (default behaviour), it creates a new redis client as it
did before, no changes to that.
The rest of the provider code is unchanged. All these changes were
required to make it possible to generate mock clients for providers and
use them.
For the tests, the existing two test cases are good with redis server,
and they need some extra helpers, for example to start a new redis
server if required, or waiting on a redis server to be ready to use.
These helpers are only required for test cases using real redis server.
For better isolation, moved existing test under a testify Suite, and
moved them into a new test file called `base_redis_with_server_test.go`
because, well they test the code with server. These tests do exactly the
same as before, calling the same sub-tests the same way as before, the
only change is the structure of the test (remove repetition, scope
server related helper functions).
Finally, we can create unit tests without redis server. The main focus of
this group of tests are higher level overview of operations. With the
mock redis client we can set up expectations about used queue names,
received values, return value to simulate faulty state.
These new unit test functions don't test all functionality, at least
it's not aimed for it now. It's more about the possibility of doing that
and add extra tests around parts we couldn't test before, for example
key.
What extra features can test the new unit test group:
- What is the received key for given queue? For example using `prefix`,
or if all the `SXxx` calls are expected to use `queue_unique` if
it's a unique queue.
- If it's not a unique queue, no `SXxx` functions are called, because
those sets are used only to check if a value is unique or not.
- `HasItem` return `false` always if it's a non-unique queue.
- All functions are called exactly `N` times, and we don't have any
unexpected calls to redis from the code.
Signed-off-by: Victoria Nadasdi <victoria@efertone.me>
For security reasons, scoping access to a redis server via ACL rules is
a good practice. Some parts of the codebase handles prefix like cache[^1]
and session[^2], but the queue module doesn't.
This patch adds this missing functionality to the queue module.
Note about relevant test:
I tried to keep the PR as small as possible (and reasonable), and not
change how the test runs. Updated the existing test to use the same
redis address and basically duplicated the test with the extra flag. It
does NOT test if the keys are correct, it ensures only it works as
expected. To make assertions about the keys, the whole test has to be
updated as the general wrapper doesn't allow the main test to check
anything provider (redis) specific property. That's not something I
wanted to take on now.
[^1]: e4c3c039be/modules/cache/cache_redis.go (L139-L150)
[^2]: e4c3c039be/modules/session/redis.go (L122-L129)
Signed-off-by: Victoria Nadasdi <victoria@efertone.me>
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/3836
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Victoria Nadasdi <victoria@efertone.me>
Co-committed-by: Victoria Nadasdi <victoria@efertone.me>
Fix #30923
(cherry picked from commit effb405cae88474c27f5c8322a2627019af1cf64)
Signed-off-by: Gergely Nagy <forgejo@gergo.csillger.hu>
Conflicts:
- modules/setting/oauth2.go
Conflicted due to different ways of logging. Since the log
message is removed anyway, resolved by removing it.
- modules/setting/oauth2_test.go
Manually copied the test added by Gitea.
- routers/install/install.go
Not a conflict per se, but adjusted to use NewJwtSecret().
Resolve #30917
Make the APIs for adding labels and replacing labels support both label
IDs and label names so the
[`actions/labeler`](https://github.com/actions/labeler) action can work
in Gitea.
<img width="600px"
src="https://github.com/go-gitea/gitea/assets/15528715/7835c771-f637-4c57-9ce5-e4fbf56fa0d3"
/>
(cherry picked from commit b3beaed147466739de0c24fd80206b5af8b71617)
Conflicts:
- modules/structs/issue_label.go
Resolved by applying the Gitea change by hand.
- tests/integration/api_issue_label_test.go
Resolved by copying the new tests.
Merging PR may fail because of various problems. The pull request may
have a dirty state because there is no transaction when merging a pull
request. ref
https://github.com/go-gitea/gitea/pull/25741#issuecomment-2074126393
This PR moves all database update operations to post-receive handler for
merging a pull request and having a database transaction. That means if
database operations fail, then the git merging will fail, the git client
will get a fail result.
There are already many tests for pull request merging, so we don't need
to add a new one.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
(cherry picked from commit ebf0c969403d91ed80745ff5bd7dfbdb08174fc7)
Conflicts:
modules/private/hook.go
routers/private/hook_post_receive.go
trivial conflicts because
263a716cb5 * Performance optimization for git push (#30104)
was not cherry-picked and because of
998a431747 Do not update PRs based on events that happened before they existed
More about codespell: https://github.com/codespell-project/codespell .
I personally introduced it to dozens if not hundreds of projects already and so far only positive feedback.
```
❯ grep lint-spell Makefile
@echo " - lint-spell lint spelling"
@echo " - lint-spell-fix lint spelling and fix issues"
lint: lint-frontend lint-backend lint-spell
lint-fix: lint-frontend-fix lint-backend-fix lint-spell-fix
.PHONY: lint-spell
lint-spell: lint-codespell
.PHONY: lint-spell-fix
lint-spell-fix: lint-codespell-fix
❯ git grep lint- -- .forgejo/
.forgejo/workflows/testing.yml: - run: make --always-make -j$(nproc) lint-backend checks-backend # ensure the "go-licenses" make target runs
.forgejo/workflows/testing.yml: - run: make lint-frontend
```
so how would you like me to invoke `lint-codespell` on CI? (without that would be IMHO very suboptimal and let typos sneak in)
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/3270
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Yaroslav Halchenko <debian@onerussian.com>
Co-committed-by: Yaroslav Halchenko <debian@onerussian.com>
Fix #30807
reuse functions in services
(cherry picked from commit a50026e2f30897904704895362da0fb12c7e5b26)
Conflicts:
models/issues/issue_update.go
routers/api/v1/repo/issue.go
trivial context conflict because of 'allow setting the update date on issues and comments'
The test had a dependency on `https://api.pwnedpasswords.com` which
caused many failures on CI recently:
```
--- FAIL: TestPassword (2.37s)
pwn_test.go:41: Get "https://api.pwnedpasswords.com/range/e6b6a": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
FAIL
coverage: 82.9% of statements
```
(cherry picked from commit 9235442ba58524c8d12ae54865d583acfa1f439d)
Co-authored-by: silverwind <me@silverwind.io>
(cherry picked from commit be112c1fc30f87a248b30f48e891d1c8c18e8280)
Conflicts:
routers/web/web.go
trivial conflict because of https://codeberg.org/forgejo/forgejo/pulls/1533
Resolve all cases for `unused parameter` and `unnecessary type
arguments`
Related: #30729
---------
Co-authored-by: Giteabot <teabot@gitea.io>
(cherry picked from commit e80466f7349164ce4cf3c07bdac30d736d20f035)
Conflicts:
modules/markup/markdown/transform_codespan.go
modules/setting/incoming_email.go
routers/api/v1/admin/user_badge.go
routers/private/hook_pre_receive.go
tests/integration/repo_search_test.go
resolved by discarding the change, this is linting only and
for the sake of avoiding future conflicts
Adjust the `anyHashPattern` to match URL query parameters too, and
adjust `fullHashPatternProcessor` accordingly.
Includes a test case, and an update to an existing one to account for
the new capture group.
Fixes #3548.
Signed-off-by: Gergely Nagy <forgejo@gergo.csillger.hu>
When mentioning a user, the markup post-processor did not handle the
case where the mentioned user did not exist well: it tried to skip to
the next node, which in turn, ended up skipping the rest of the line.
To fix this, lets skip just the mentioned, but non-existing user, and
continue processing the current node from there.
Fixes #3535.
Signed-off-by: Gergely Nagy <forgejo@gergo.csillger.hu>
Suggested by logs in #30729
- Remove `math/rand.Seed`
`rand.Seed is deprecated: As of Go 1.20 there is no reason to call Seed
with a random value.`
- Replace `math/rand.Read`
`rand.Read is deprecated: For almost all use cases, [crypto/rand.Read]
is more appropriate.`
- Replace `math/rand` with `math/rand/v2`, which is available since Go
1.22
(cherry picked from commit 7b8e418da1e082786b844562a05864ec1177ce97)
Should resolve #30642.
Before this commit, we were treating an empty `?sort=` query parameter
as the correct sorting type (which is to sort issues in descending order
by their created UNIX time). But when we perform `sort=latest`, we did
not include this as a type so we would sort by the most recently updated
when reaching the `default` switch statement block.
This commit fixes this by considering the empty string, "latest", and
just any other string that is not mentioned in the switch statement as
sorting by newest.
(cherry picked from commit 9b7af4340c36d3e1888788499d16f83feeb1601b)
Noteable additions:
- `redefines-builtin-id` forbid variable names that shadow go builtins
- `empty-lines` remove unnecessary empty lines that `gofumpt` does not
remove for some reason
- `superfluous-else` eliminate more superfluous `else` branches
Rules are also sorted alphabetically and I cleaned up various parts of
`.golangci.yml`.
(cherry picked from commit 74f0c84fa4245a20ce6fb87dac1faf2aeeded2a2)
Conflicts:
.golangci.yml
apply the linter recommendations to Forgejo code as well
This adds a new options to releases to hide the links to the automatically generated archives. This is useful, when the automatically generated Archives are broken e.g. because of Submodules.
Note:
This juts hides the Archives from the UI. Users can still download 5the Archive if they know t correct URL.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/3139
Reviewed-by: Otto <otto@codeberg.org>
Reviewed-by: 0ko <0ko@noreply.codeberg.org>
Co-authored-by: JakobDev <jakobdev@gmx.de>
Co-committed-by: JakobDev <jakobdev@gmx.de>
If `[email.incoming].USE_TLS` is set, but the port isn't, infer the
default from `.USE_TLS`: set the port to 993 if using tls, and to 143
otherwise. Explicitly setting a port overrides this.
Fixes #3357.
Signed-off-by: Gergely Nagy <forgejo@gergo.csillger.hu>
The keys for setting the username and password for incoming and outgoing
mail are inconsisent: one uses `USERNAME` and `PASSWORD`, the other uses
`USER` and `PASSWD`.
To make things simpler, allow both to be configured by either, thus,
`[mailer].USERNAME` and `[mailer.PASSWORD]` will be aliases for `.USER`
and `.PASSWD`, and similarly, `[email.incoming].USER` and
`[email.incoming].PASSWD` will be aliases for `.USERNAME` and
`.PASSWORD`.
Fixes #3355.
Signed-off-by: Gergely Nagy <forgejo@gergo.csillger.hu>
Related to #30375.
It doesn't make sense to import `modules/web/middleware` and
`modules/setting` in `modules/web/session` since the last one is more
low-level.
And it looks like a workaround to call `DeleteLegacySiteCookie` in
`RegenerateSession`, so maybe we could reverse the importing by
registering hook functions.
(cherry picked from commit 61457cdf6b49225ae831fd9fb084deadd8bdb0fb)
Using the API, a user's _source_id_ can be set in the _CreateUserOption_
model, but the field is not returned in the _User_ model.
This PR updates the _User_ model to include the field _source_id_ (The
ID of the Authentication Source).
(cherry picked from commit 58b204b813cd3a97db904d889d552e64a7e398ff)
- Add new `Compare` struct to represent comparison between two commits
- Introduce new API endpoint `/compare/*` to get commit comparison
information
- Create new file `repo_compare.go` with the `Compare` struct definition
- Add new file `compare.go` in `routers/api/v1/repo` to handle
comparison logic
- Add new file `compare.go` in `routers/common` to define `CompareInfo`
struct
- Refactor `ParseCompareInfo` function to use `common.CompareInfo`
struct
- Update Swagger documentation to include the new API endpoint for
commit comparison
- Remove duplicate `CompareInfo` struct from
`routers/web/repo/compare.go`
- Adjust base path in Swagger template to be relative (`/api/v1`)
GitHub API
https://docs.github.com/en/rest/commits/commits?apiVersion=2022-11-28#compare-two-commits
---------
Signed-off-by: Bo-Yi Wu <appleboy.tw@gmail.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
(cherry picked from commit c70e442ce4b99e2a1f1bf216afcfa1ad78d1925a)
Conflicts:
- routers/api/v1/swagger/repo.go
Conflict resolved by manually adding the lines from the Gitea
PR.
