Commit graph

633 commits

Author SHA1 Message Date
zeripath
df44017328
Restart zero worker if there is still work to do (#18658)
* Restart zero worker if there is still work to do

It is possible for the zero worker to timeout before all the work is finished.
This may mean that work may take a long time to complete because a worker will only
be induced on repushing.

Also ensure that requested count is reset after pulls and push mirror sync requests and add some more trace logging to the queue push.

Fix #18607

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-02-08 14:02:32 +00:00
Viktor Suprun
4d939845d2
Added auto-save whitespace behavior if it changed manually (#15566) 2022-02-08 14:15:04 +08:00
Clar Fon
9911b66aea
Be more lenient with label colors (#17752)
Accept 12-bit color specifications.
2022-02-07 21:21:02 +00:00
wxiaoguang
c2a3e38194
Fix the missing i18n key for update checker (#18646) 2022-02-07 15:43:53 +08:00
6543
3043eb36bf
Delete old git.NewCommand() and use it as git.NewCommandContext() (#18552) 2022-02-06 20:01:47 +01:00
silverwind
37ead26e36
Fix make fmt and make fmt-check (#18633)
* Run 'make fmt'

'make fmt' currently produces this change, I'm not sure how CI did not
fail on it, I made sure I have `mvdan.cc/gofumpt@latest`.

* Fix 'make fmt-check'

`make fmt-check` did not run all commands that `make fmt` did, resulting
in missed diffs. Fix that by just depending on the `fmt` target.

Includes: https://github.com/go-gitea/gitea/pull/18633

* Make gitea-fmt work with -l and -d and integrate gofumpt

This implements -l, -w and -d with gitea-fmt and merges gofumpt.

Signed-off-by: Andrew Thornton <art27@cantab.net>

* as per silverwind

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Apply suggestions from code review

* use -l instead of -d for fmt-check

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Andrew Thornton <art27@cantab.net>
2022-02-06 16:44:30 +01:00
singuliere
8bd89ca294
preserve users if restoring a repository on the same Gitea instance (#18604)
When calling DumpRepository and RestoreRepository on the same Gitea
instance, the users are preserved: all labels, issues etc. belong to
the external user who is, in this particular case, the local user.

Dead code verifying g.gitServiceType.Name() == "" (i.e. plain git) is
removed. The function is never called because the plain git downloader
does not migrate anything that is associated to a user, by definition.

Errors returned by GetUserIDByExternalUserID are no longer ignored.

The userMap is used when the external user is not kown, which is the
most common case. It was only used when the external user exists
which happens less often and, as a result, every occurence of an
unknown external user required a SQL query.

Signed-off-by: Loïc Dachary <loic@dachary.org>

Co-authored-by: Loïc Dachary <loic@dachary.org>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-02-06 17:05:29 +08:00
zeripath
1ab44cb01d
Prevent merge messages from being sorted to the top of email chains (#18566)
* Prevent merge messages from being sorted to the top of email chains

Gitea will currrently resend the same message-id for the closed/merged/reopened
messages for issues. This will cause the merged message to leap to the top of an
email chain and become out of sync.

This PR adds specific suffices for these actions.

Fix #18560

Signed-off-by: Andrew Thornton <art27@cantab.net>

* add test

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-02-03 18:01:16 -05:00
Gusted
cafd19c1a3
Replace sync.Map with normal maps (#18584)
* Replace `sync.Map` with normal maps

- These maps aren't being used in any kind of concurrent read/write and
thus don't need `sync.Map` and can instead use normal maps.
- Special thanks to dachary.
- Added in: https://github.com/go-gitea/gitea/pull/6290

* Remove unannounced feature
2022-02-03 21:18:18 +02:00
zeripath
92e81e97e8
Detect conflicts with 3way merge (#18536)
* Detect conflicts with 3way merge

Unforunately git apply --3way reports conflicts differently than standard patches
resulting in conflicts being missed.

Adjust the conflict detection code to account for this different error reporting.

Fix #18514

Signed-off-by: Andrew Thornton <art27@cantab.net>

* and three-way failed

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-02-02 12:46:10 +08:00
singuliere
367894adc8
add test coverage for original author conversion during migrations (#18506)
* add test coverage for original author conversion during migrations

And create a function to factorize a code snippet that is repeated
five times and would otherwise be more difficult to test and maintain
consistently.

Signed-off-by: Loïc Dachary <loic@dachary.org>

* fix variable scope and int64 formatting

* add missing calls to remapExternalUser and fix misplaced %d

Co-authored-by: Loïc Dachary <loic@dachary.org>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2022-02-01 13:20:28 -05:00
Gusted
22d266350d
Use ImagedProvider for gplus oauth2 provider (#18504)
Use `ImagedProvider` for gplus' oauthv2 provider, as the image isn't "gplus.png" but "google.png".

Resolves #18494
2022-02-01 01:02:56 +01:00
e19b9653ea
GitLab reviews may not have the updated_at field set (#18450)
* GitLab reviews may not have the updated_at field set

Fallback to created_at if that the case and to time.Now() if it is
also missing.

Fixes: 18434

* use assert.WithinDuration

Co-authored-by: Loïc Dachary <loic@dachary.org>
2022-01-29 18:33:20 +01:00
3bb028cc46
Validate migration files (#18203)
JSON Schema validation for data used by Gitea during migrations

Discussion at https://forum.forgefriends.org/t/common-json-schema-for-repository-information/563

Co-authored-by: Loïc Dachary <loic@dachary.org>
2022-01-26 10:45:51 +01:00
Lunny Xiao
a717265e47
Fix restore without topic failure (#18387)
Co-authored-by: zeripath <art27@cantab.net>
2022-01-25 10:44:12 +08:00
Gusted
c2e13fb763
Fix partial cloning a repo (#18373)
- Pass the Global command args into serviceRPC.
- Fixes error with partial cloning.
- Add partial clone test
- Include diff

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-01-23 22:19:32 +01:00
Dr. Tobias Quathamer
3349fd8f79
Add packagist webhook (#18224)
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2022-01-23 21:46:30 +08:00
Lunny Xiao
35fdefc1ff
Always use git command but not os.Command (#18363) 2022-01-23 00:57:52 -05:00
zeripath
a82fd98d53
Pause queues (#15928)
* Start adding mechanism to return unhandled data

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Create pushback interface

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Add Pausable interface to WorkerPool and Manager

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Implement Pausable and PushBack for the bytefifos

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Implement Pausable and Pushback for ChannelQueues and ChannelUniqueQueues

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Wire in UI for pausing

Signed-off-by: Andrew Thornton <art27@cantab.net>

* add testcases and fix a few issues

Signed-off-by: Andrew Thornton <art27@cantab.net>

* fix build

Signed-off-by: Andrew Thornton <art27@cantab.net>

* prevent "race" in the test

Signed-off-by: Andrew Thornton <art27@cantab.net>

* fix jsoniter mismerge

Signed-off-by: Andrew Thornton <art27@cantab.net>

* fix conflicts

Signed-off-by: Andrew Thornton <art27@cantab.net>

* fix format

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Add warnings for no worker configurations and prevent data-loss with redis/levelqueue

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Use StopTimer

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2022-01-22 21:22:14 +00:00
qwerty287
1f40933d38
Add config options to hide issue events (#17414)
* Add config option to hide issue events
Adds a config option `HIDE_ISSUE_EVENTS` to hide most issue events (changed labels, milestones, projects...) on the issue detail page.
If this is true, only the following events (comment types) are shown:
* plain comments
* closed/reopned/merged
* reviews

* Make configurable using a list

* Add docs

* Add missing newline

* Fix merge issues

* Allow changes per user settings

* Fix lint

* Rm old docs

* Apply suggestions from code review

* Use bitsets

* Rm comment

* fmt

* Fix lint

* Use variable/constant to provide key

* fmt

* fix lint

* refactor

* Add a prefix for user setting key

* Add license comment

* Add license comment

* Update services/forms/user_form_hidden_comments.go

Co-authored-by: Gusted <williamzijl7@hotmail.com>

* check len == 0

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Gusted <williamzijl7@hotmail.com>
Co-authored-by: 6543 <6543@obermui.de>
2022-01-21 18:59:26 +01:00
zeripath
44deae8f3d
Refactor jwt.StandardClaims to RegisteredClaims (#18344)
* Refactor jwt.StandardClaims to RegisteredClaims

go-jwt/jwt has deprecated the StandardClaims interface to use RegisteredClaims
instead. This PR migrates to use this new format.

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Apply suggestions from code review

Co-authored-by: Gusted <williamzijl7@hotmail.com>

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Gusted <williamzijl7@hotmail.com>
2022-01-20 22:52:56 +01:00
6543
54e9ee37a7
format with gofumpt (#18184)
* gofumpt -w -l .

* gofumpt -w -l -extra .

* Add linter

* manual fix

* change make fmt
2022-01-20 18:46:10 +01:00
Jelle Hulter
bbd30787d3
Updated Chroma to v0.10.0 (#18270) 2022-01-20 17:59:15 +08:00
zeripath
5cb0c9aa0d
Propagate context and ensure git commands run in request context (#17868)
This PR continues the work in #17125 by progressively ensuring that git
commands run within the request context.

This now means that the if there is a git repo already open in the context it will be used instead of reopening it.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-01-19 23:26:57 +00:00
zeripath
6b0a71230d
Stop trimming preceding and suffixing spaces from editor filenames (#18334)
* Stop trimming preceding and suffixing spaces from editor filenames

In #5702 it was decided to trim preceding and suffixed spaces aswell as / from
editing file filenames. This was because at this point in time the url-safety of
Gitea was much poorer.

We can now drop this requirement and file editing should work correctly.

Fix #18176

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-01-19 21:03:15 +00:00
zeripath
4a20eadfd7
Restore propagation of ErrDependenciesLeft (#18325)
Unfortunately #17643 prevented all propagation of ErrDependenciesLeft meaning
that dependency errors that prevent closing of issues get swallowed.

This PR restores propagation of the error but instead swallows the error in the
places where it needs to be swallowed.

Fix #18223

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-01-19 00:26:42 +01:00
zeripath
089b4e6a45
Use indirect comparison when showing pull requests (#18313)
When generating the commits list and number of files changed for PRs and
compare we should use "..." always not "..".

Fix #18303

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-01-18 07:45:43 +00:00
zeripath
851b54ba9f
Correctly upload LFS files (#18316)
We need to use the cached .gitattributes file for checking if a file
should be stored in the lfs.

Fix #18297

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-01-18 07:44:30 +00:00
076cead40d
Fix CheckRepoStats and reuse it during migration (#18264)
The CheckRepoStats function missed the following counters:

- label num_closed_issues & num_closed_pulls
- milestone num_closed_issues & num_closed_pulls

The update SQL statements for updating the repository
num_closed_issues & num_closed_pulls fields were repeated in three
functions (repo.CheckRepoStats, migrate.insertIssues and
models.Issue.updateClosedNum) and were moved to a single helper.

The UpdateRepoStats is implemented and called in the Finish migration method so that it happens immediately instead of wating for the
CheckRepoStats to run.

Signed-off-by: Loïc Dachary loic@dachary.org

---
[source](https://lab.forgefriends.org/forgefriends/forgefriends/-/merge_requests/34)
2022-01-17 19:31:58 +01:00
Lunny Xiao
35c3553870
Support webauthn (#17957)
Migrate from U2F to Webauthn

Co-authored-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2022-01-14 16:03:31 +01:00
6543
ff00b8688b
Fix NPE on try to get tag reference via API (#18245)
* fix npe

* rm gitRepo from Tag
2022-01-12 20:37:46 +00:00
luzpaz
8c647bf0f6
Fix various typos (#18219)
Found via `codespell -q 3 -S ./options/locale,./vendor -L ba,pullrequest,pullrequests,readby,te,unknwon`

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-01-10 17:32:37 +08:00
zeripath
21ed4fd8da
Add warning for BIDI characters in page renders and in diffs (#17562)
Fix #17514

Given the comments I've adjusted this somewhat. The numbers of characters detected are increased and include things like the use of U+300 to make à instead of à and non-breaking spaces.

There is a button which can be used to escape the content to show it.

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Gwyneth Morgan <gwymor@tilde.club>
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2022-01-07 02:18:52 +01:00
zeripath
1514e13bb8
Add option to convert CRLF to LF line endings for sendmail (#18075)
It appears that several versions of sendmail require that the mail is sent to them with
LF line endings instead of CRLF endings - which of course they will then convert back
to CRLF line endings to comply with the SMTP standard.

This PR adds another setting SENDMAIL_CONVERT_CRLF which will pass the message writer
through a filter. This will filter out and convert CRLFs to LFs before writing them
out to sendmail.

Fix #18024

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-01-06 01:43:45 +01:00
KN4CK3R
bf7b083cfe
Add replay of webhooks. (#18191) 2022-01-05 15:00:20 -06:00
6543
0572c78938
refactoring nits (#18188)
* no octal for time values

* rm artifact from refactoring

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-01-05 07:27:53 +01:00
Lunny Xiao
8760af752a
Team permission allow different unit has different permission (#17811)
* Team permission allow different unit has different permission

* Finish the interface and the logic

* Fix lint

* Fix translation

* align center for table cell content

* Fix fixture

* merge

* Fix test

* Add deprecated

* Improve code

* Add tooltip

* Fix swagger

* Fix newline

* Fix tests

* Fix tests

* Fix test

* Fix test

* Max permission of external wiki and issues should be read

* Move team units with limited max level below units table

* Update label and column names

* Some improvements

* Fix lint

* Some improvements

* Fix template variables

* Add permission docs

* improve doc

* Fix fixture

* Fix bug

* Fix some bug

* fix

* gofumpt

* Integration test for migration (#18124)

integrations: basic test for Gitea {dump,restore}-repo
This is a first step for integration testing of DumpRepository and
RestoreRepository. It:

runs a Gitea server,
dumps a repo via DumpRepository to the filesystem,
restores the repo via RestoreRepository from the filesystem,
dumps the restored repository to the filesystem,
compares the first and second dump and expects them to be identical

The verification is trivial and the goal is to add more tests for each
topic of the dump.

Signed-off-by: Loïc Dachary <loic@dachary.org>

* Team permission allow different unit has different permission

* Finish the interface and the logic

* Fix lint

* Fix translation

* align center for table cell content

* Fix fixture

* merge

* Fix test

* Add deprecated

* Improve code

* Add tooltip

* Fix swagger

* Fix newline

* Fix tests

* Fix tests

* Fix test

* Fix test

* Max permission of external wiki and issues should be read

* Move team units with limited max level below units table

* Update label and column names

* Some improvements

* Fix lint

* Some improvements

* Fix template variables

* Add permission docs

* improve doc

* Fix fixture

* Fix bug

* Fix some bug

* Fix bug

Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Aravinth Manivannan <realaravinth@batsense.net>
2022-01-05 11:37:00 +08:00
Gusted
623c93ff46
Increase Salt randomness (#18179)
- The current implementation of `RandomString` doesn't give you a most-possible unique randomness. It gives you 6*`length` instead of the possible 8*`length` bits(or as `length`x bytes) randomness. This is because `RandomString` is being limited to a max value of 63, this in order to represent the random byte as a letter/digit.
- The recommendation of pbkdf2 is to use 64+ bit salt, which the `RandomString` doesn't give with a length of 10, instead of increasing 10 to a higher number, this patch adds a new function called `RandomBytes` which does give you the guarentee of 8*`length` randomness and thus corresponding of `length`x bytes randomness.
- Use hexadecimal to store the bytes value in the database, as mentioned, it doesn't play nice in order to convert it to a string. This will always be a length of 32(with `length` being 16).
- When we detect on `Authenticate`(source: db) that a user has the old format of salt, re-hash the password such that the user will have it's password hashed with increased salt.

Thanks to @zeripath for working out the rouge edges from my first commit 😄.

Co-authored-by: lafriks <lauris@nix.lv>
Co-authored-by: zeripath <art27@cantab.net>
2022-01-04 15:13:52 +00:00
Lunny Xiao
de8e3948a5
Refactor auth package (#17962) 2022-01-02 21:12:35 +08:00
wxiaoguang
e61b390d54
Unify and simplify TrN for i18n (#18141)
Refer: https://github.com/go-gitea/gitea/pull/18135#issuecomment-1003246099

Now we have a unique and simple `TrN`, and make the fix of PR #18135 also use the better `TrN` logic.
2022-01-02 04:33:57 +01:00
singuliere
fb2dc95623
services/repository: fix ListUnadoptedRepositories incorrect total count (#17865)
The total count returned by ListUnadoptedRepositories is incorrectly
calculated.

The code snippet within ListUnadoptedRepositories used to verify
unadopted repositories is repeated three times in the function. It is
moved in the checkUnadoptedRepositories function and a unit test is
added to verify it works as expected.

A unit test is added to verify the total count returned by
ListUnadoptedRepositories is as expected.

Signed-off-by: singuliere <singuliere@autistici.org>
2022-01-01 10:52:00 +08:00
zeripath
431e482e3f
Reset the conflicted files list in testpatch (#18139)
Although #18004 will seriously reduce the likelihood of finding
conflicts in the first place - one bug was introduced whereby the
conflicted files status was not being reset properly. This leads to
conflicted PRs remaining conflicted when the conflict has been resolved.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-12-31 20:33:55 +08:00
KN4CK3R
8fa97a25f0
Set HeadCommit when creating tags. (#18116)
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2021-12-29 19:40:57 +08:00
zjjhot
e34632bb53
Add issue hyperlinks in the webhook of wechatwork (#18102)
Co-authored-by: zjj <2031381130@qq.com>
2021-12-25 21:30:09 +08:00
zeripath
ffc08c1914
Do not read or write git reference files directly (#18079)
Git will and can pack references into packfiles and therefore if you write/read the
files directly you will get false results. Instead you should use update-ref and
show-ref. To that end I have created three new functions in git/repo_commit.go that
will do this correctly.

Related #17191

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-12-23 21:44:00 +08:00
mscherer
cb05d3a23a
Add List-Unsubscribe header (#17804)
Fixes #13283

Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-12-21 22:53:03 +00:00
zeripath
bcc13f3889
Reset Session ID on login (#18018)
* Reset Session ID on login

When logging in the SessionID should be reset and the session cleaned up.

Signed-off-by: Andrew Thornton <art27@cantab.net>

* with new session.RegenerateID function

Signed-off-by: Andrew Thornton <art27@cantab.net>

* update go-chi/session

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Ensure that session id is changed after oauth data is set and between account linking pages too

Signed-off-by: Andrew Thornton <art27@cantab.net>

* placate lint

Signed-off-by: Andrew Thornton <art27@cantab.net>

* as per review

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-12-20 22:12:26 +08:00
Gusted
ff2fd08228
Simplify parameter types (#18006)
Remove repeated type declarations in function definitions.
2021-12-20 04:41:31 +00:00
zeripath
e4e411821d
Abort merge if head has been updated before pressing merge (#18032)
* Abort merge if head has been updated before pressing merge

It is possible that a PR head may be pushed to between the merge page being shown
and the merge button being pressed. Pass the current expected head in as a parameter
and cancel the merge if it has changed.

Fix #18028

Signed-off-by: Andrew Thornton <art27@cantab.net>

* adjust swagger

Signed-off-by: Andrew Thornton <art27@cantab.net>

* fix test

Signed-off-by: Andrew Thornton <art27@cantab.net>

* placate lint

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-12-20 08:32:54 +08:00
Wim
6fe756dc93
Add support for ssh commit signing (#17743)
* Add support for ssh commit signing

* Split out ssh verification to separate file

* Show ssh key fingerprint on commit page

* Update sshsig lib

* Make sure we verify against correct namespace

* Add ssh public key verification via ssh signatures

When adding a public ssh key also validate that this user actually
owns the key by signing a token with the private key.

* Remove some gpg references and make verify key optional

* Fix spaces indentation

* Update options/locale/locale_en-US.ini

Co-authored-by: Gusted <williamzijl7@hotmail.com>

* Update templates/user/settings/keys_ssh.tmpl

Co-authored-by: Gusted <williamzijl7@hotmail.com>

* Update options/locale/locale_en-US.ini

Co-authored-by: Gusted <williamzijl7@hotmail.com>

* Update options/locale/locale_en-US.ini

Co-authored-by: Gusted <williamzijl7@hotmail.com>

* Update models/ssh_key_commit_verification.go

Co-authored-by: Gusted <williamzijl7@hotmail.com>

* Reword ssh/gpg_key_success message

* Change Badsignature to NoKeyFound

* Add sign/verify tests

* Fix upstream api changes to user_model User

* Match exact on SSH signature

* Fix code review remarks

Co-authored-by: Gusted <williamzijl7@hotmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-12-19 00:37:18 -05:00
zeripath
f1e85622da
Improve TestPatch to use git read-tree -m and implement git-merge-one-file functionality (#18004)
The current TestPatch conflict code uses a plain git apply which does not properly
account for 3-way merging. However, we can improve things using `git read-tree -m` to
do a three-way merge then follow the algorithm used in merge-one-file. We can also use 
`--patience` and/or `--histogram` to generate a nicer diff for applying patches too.

Fix #13679
Fix #6417

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-12-19 05:19:25 +01:00
Lunny Xiao
a6d4a8546a
Fix rename notification bug (#18008) 2021-12-17 14:24:47 +08:00
zeripath
8354670708
Prevent hang in git cat-file if repository is not a valid repository and other fixes (#17991)
This PR contains multiple fixes. The most important of which is:

* Prevent hang in git cat-file if the repository is not a valid repository 
    
    Unfortunately it appears that if git cat-file is run in an invalid
    repository it will hang until stdin is closed. This will result in
    deadlocked /pulls pages and dangling git cat-file calls if a broken
    repository is tried to be reviewed or pulls exists for a broken
    repository.

    Fix #14734
    Fix #9271
    Fix #16113

Otherwise there are a few small other fixes included which this PR was initially intending to fix:

* Fix panic on partial compares due to missing PullRequestWorkInProgressPrefixes
* Fix links on pulls pages  due to regression from #17551 - by making most /issues routes match /pulls too - Fix #17983
* Fix links on feeds pages due to another regression from #17551 but also fix issue with syncing tags - Fix #17943
* Add missing locale entries for oauth group claims
* Prevent NPEs if ColorFormat is called on nil users, repos or teams.
2021-12-16 19:01:14 +00:00
zeripath
e0e3ba6c12
Add Reindex buttons to repository settings page (#17494)
This PR adds reindexing request buttons to the repository settings page.

Fix #3796

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-12-16 23:55:12 +08:00
wxiaoguang
4da1d97810
Refactor HTTP request context (#17979) 2021-12-15 14:59:57 +08:00
qwerty287
9d943bf374
Add missing X-Total-Count and fix some related bugs (#17968)
* Add missing `X-Total-Count` and fix some related bugs

Adds `X-Total-Count` header to APIs that return a list but doesn't have it yet.
Fixed bugs:
* not returned after reporting error (39eb82446c/routers/api/v1/user/star.go (L70))
* crash with index out of bounds, API issue/issueSubscriptions

I also found various endpoints that return lists but do not apply/support pagination yet:
```
/repos/{owner}/{repo}/issues/{index}/labels
/repos/{owner}/{repo}/issues/comments/{id}/reactions
/repos/{owner}/{repo}/branch_protections
/repos/{owner}/{repo}/contents
/repos/{owner}/{repo}/hooks/git
/repos/{owner}/{repo}/issue_templates
/repos/{owner}/{repo}/releases/{id}/assets
/repos/{owner}/{repo}/reviewers
/repos/{owner}/{repo}/teams
/user/emails
/users/{username}/heatmap
```
If this is not expected, an new issue should be opened.

Closes #13043

* fmt

* Update routers/api/v1/repo/issue_subscription.go

Co-authored-by: KN4CK3R <admin@oldschoolhack.me>

* Use FindAndCount

Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
Co-authored-by: 6543 <6543@obermui.de>
2021-12-15 13:39:34 +08:00
zeripath
0981ec30c3
Add Option to synchronize Admin & Restricted states from OIDC/OAuth2 along with Setting Scopes (#16766)
* Add setting to OAuth handlers to override local 2FA settings

This PR adds a setting to OAuth and OpenID login sources to allow the source to
override local 2FA requirements.

Fix #13939

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Fix regression from #16544

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Add scopes settings

Signed-off-by: Andrew Thornton <art27@cantab.net>

* fix trace logging in auth_openid

Signed-off-by: Andrew Thornton <art27@cantab.net>

* add required claim options

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Move UpdateExternalUser to externalaccount

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Allow OAuth2/OIDC to set Admin/Restricted status

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Allow use of the same group claim name for the prohibit login value

Signed-off-by: Andrew Thornton <art27@cantab.net>

* fixup! Move UpdateExternalUser to externalaccount

* as per wxiaoguang

Signed-off-by: Andrew Thornton <art27@cantab.net>

* add label back in

Signed-off-by: Andrew Thornton <art27@cantab.net>

* adjust localisation

Signed-off-by: Andrew Thornton <art27@cantab.net>

* placate lint

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-12-14 16:37:11 +08:00
Lunny Xiao
5723240490
Some repository refactors (#17950)
* some repository refactors

* remove unnecessary code

* Fix test

* Remove unnecessary banner
2021-12-12 23:48:20 +08:00
zeripath
f550e356d6
Prevent services/mailer/mailer_test.go tests from deleteing data directory (#17941)
Running `make test-backend` will delete `data/` due to reloading the configuration and resetting the appdatapath.

This PR removes this unnecessary config reload but also adds extra code in to the unittest main to prevent its cleanup from deleting the wrong directory.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-12-10 19:43:28 +00:00
Lunny Xiao
3ca5dc7e32
Move keys to models/asymkey (#17917)
* Move keys to models/keys

* Rename models/keys -> models/asymkey

* change the missed package name

* Fix package alias

* Fix test

* Fix docs

* Fix test

* Fix test

* merge
2021-12-10 16:14:24 +08:00
Lunny Xiao
719bddcd76
Move repository model into models/repo (#17933)
* Some refactors related repository model

* Move more methods out of repository

* Move repository into models/repo

* Fix test

* Fix test

* some improvements

* Remove unnecessary function
2021-12-10 09:27:50 +08:00
zeripath
9e6e1dc950
Improve checkBranchName (#17901)
The current implementation of checkBranchName is highly inefficient
involving opening the repository, the listing all of the branch names
checking them individually before then using using opened repo to get
the tags.

This PR avoids this by simply walking the references from show-ref
instead of opening the repository (in the nogogit case).

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-12-08 19:08:16 +00:00
Garionion
b59875aa12
allways set a message-id on mails (#17900)
* allways set a message-id on mails
* Add unit tests for mailer & Message-ID

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-12-08 15:34:23 +08:00
Lunny Xiao
dcdb4873c8
Move repo archiver to models/repo (#17913)
* Move repo archiver to models/repo

* Move archiver service into services/repository/

* Fix imports

* Fix test

* Fix test
2021-12-06 15:19:28 +08:00
KN4CK3R
87be76213a
Add migrate from Codebase (#16768)
This PR adds [Codebase](https://www.codebasehq.com/) as migration source.

Supported:
- Milestones
- Issues
- Pull Requests
- Comments
- Labels
2021-12-02 23:24:02 +01:00
mscherer
34b5436ae1
Refactor various strings (#17784)
Fixes #16478

Co-authored-by: Gusted <williamzijl7@hotmail.com>

Co-authored-by: Gusted <williamzijl7@hotmail.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-12-02 15:28:08 +08:00
wxiaoguang
042cac5fed
Improve install code to avoid low-level mistakes. (#17779)
* Improve install code to avoid low-level mistakes.

If a user tries to do a re-install in a Gitea database, they gets a warning and double check.
When Gitea runs, it never create empty app.ini automatically.

Also some small (related) refactoring:

* Refactor db.InitEngine related logic make it more clean (especially for the install code)
* Move some i18n strings out from setting.go to make the setting.go can be easily maintained.
* Show errors in CLI code if an incorrect app.ini is used.
* APP_DATA_PATH is created when installing, and checked when starting (no empty directory is created any more).
2021-12-01 15:50:01 +08:00
zeripath
01087e9eef
Make Requests Processes and create process hierarchy. Associate OpenRepository with context. (#17125)
This PR registers requests with the process manager and manages hierarchy within the processes.

Git repos are then associated with a context, (usually the request's context) - with sub commands using this context as their base context.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-11-30 20:06:32 +00:00
zeripath
b5a9ee94fd
Make Co-committed-by and co-authored-by trailers optional (#17848)
This PR adds another option to app.ini make co-committed-by and co-authored-by trailers
optional on a per server basis.

Fix #17194

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-11-29 07:09:55 +00:00
Lunny Xiao
9defddb286
Move more model into models/user (#17826)
* Move more model into models/user

* Remove unnecessary comment

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2021-11-28 22:11:58 +08:00
Lunny Xiao
1fee11d69a
Move accessmode into models/perm (#17828) 2021-11-28 12:58:28 +01:00
mscherer
04c55e97a2
Fix code formating (#17830)
* fix formatter, format imports first, then go fmt
2021-11-27 19:59:51 +08:00
Lunny Xiao
a666829a37
Move user related model into models/user (#17781)
* Move user related model into models/user

* Fix lint for windows

* Fix windows lint

* Fix windows lint

* Move some tests in models

* Merge
2021-11-24 17:49:20 +08:00
Lunny Xiao
c97d66d23c
Move repofiles from modules/repofiles to services/repository/files (#17774)
* Move repofiles from modules to services

* rename services/repository/repofiles -> services/repository/files

* Fix test

Co-authored-by: 6543 <6543@obermui.de>
2021-11-24 15:56:24 +08:00
KN4CK3R
429ef43793
Use new OneDev /milestones endpoint (#17782)
OneDev changed the API which fails every build at the moment.
2021-11-23 20:28:06 +01:00
zeripath
188fd2dd1a
Add PULL_LIMIT and PUSH_LIMIT to cron.update_mirror task (#17568) 2021-11-22 22:09:35 -05:00
Lunny Xiao
baed01f247
Remove unnecessary attributes of User struct (#17745)
* Remove unnecessary functions of User struct

* Move more database methods out of user struct

* Move more database methods out of user struct

* Fix template failure

* Fix bug

* Remove finished FIXME

* remove unnecessary code
2021-11-22 23:21:55 +08:00
Jimmy Praet
a3efd048a7
Improvements to content history (#17746)
* Improvements to content history

* initialize content history when making an edit to an old item created before the introduction of content history
* show edit history for code comments on pull request files tab

* Fix a flaw in keepLimitedContentHistory
Fix a flaw in keepLimitedContentHistory, the first and the last should never be deleted

* Remove obsolete eager initialization of content history
2021-11-22 20:20:16 +08:00
zeripath
8511eec4d4
Allow Loading of Diffs that are too large (#17739)
* Allow Loading of Diffs that are too large

This PR allows the loading of diffs that are suppressed because the file
is too large. It does not handle diffs of files which have lines which
are too long.

Fix #17738

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-11-22 00:51:08 +08:00
Lunny Xiao
0add627182
Fix close issue but time watcher still running (#17643)
* Fix close issue but time watcher still running

* refactor stopwatch codes

* Fix test

* Fix test

* Fix typo

* Fix test
2021-11-21 17:11:48 +08:00
zeripath
f8ec43dbc7
Extract constant names out for the ReverseProxy and Basic authentication methods (#17735)
In order to reduce load on the GC extract out the constant names of the Basic and ReverseProxy methods.

As mentioned in https://github.com/go-gitea/gitea/pull/15119#discussion_r730352176

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-11-20 23:33:18 +08:00
zeripath
931d0cf854
Enable show more files in diff for git <2.31 (#17733)
Unfortunately due to a misread on my behalf I missed that git diff only learned
--skip-to in version 2.31.0. Thus this functionality was not working on older versions
of git.

This PR adds a handler that simply allows for us to skip reading the diffs until
we find the correct file to skip to.

Fix #17731

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-11-20 13:50:00 +00:00
wxiaoguang
013fb73068
Use hostmatcher to replace matchlist, improve security (#17605)
Use hostmacher to replace matchlist.

And we introduce a better DialContext to do a full host/IP check, otherwise the attackers can still bypass the allow/block list by a 302 redirection.
2021-11-20 17:34:05 +08:00
zeripath
38347aa16f
Add settings to allow different SMTP envelope from address (#17479)
* Add settings to allow different SMTP envelope from address

Sometimes it may be advisable to hide or alias the from address on an SMTP mail
envelope. This PR adds two new options to the mailer to allow setting of an overriding
from address.

Fix #17477

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-11-19 23:35:20 +08:00
Lunny Xiao
fc3d082609
Move attachment into models/repo/ (#17650)
* Move attachment into models/repo/

* Fix test

* Fix bug
2021-11-19 21:39:57 +08:00
Lunny Xiao
7a03473159
Use a standalone struct name for Organization (#17632)
* Use a standalone struct name for Organization

* recover unnecessary change

* make the code readable

* Fix template failure

* Fix template failure

* Move HasMemberWithUserID to org

* Fix test

* Remove unnecessary user type check

* Fix test

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2021-11-19 19:41:40 +08:00
KN4CK3R
f34151bdb2
Move user/org deletion to services (#17673) 2021-11-19 01:42:27 +08:00
99rgosse
e1d655991b
Fix Migrate Description (#17692)
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2021-11-18 23:28:10 +08:00
Lunny Xiao
1f1ae57139
Move task from modules to services (#17680) 2021-11-18 14:47:57 +08:00
Lunny Xiao
3c3855a05c
Move user functions into user.go (#17659)
* Move user functions into user.go

* Fix test
2021-11-18 13:58:42 +08:00
zeripath
3c4724d70e
Add .gitattribute assisted language detection to blame, diff and render (#17590)
Use check attribute code to check the assigned language of a file and send that in to
chroma as a hint for the language of the file.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-11-17 20:37:00 +00:00
Lunny Xiao
81a4fc7528
Return 400 but not 500 when request archive with wrong format (#17691) 2021-11-17 14:47:35 -05:00
Lunny Xiao
5233051e64
Move some functions into services/repository (#17677) 2021-11-17 23:17:31 +08:00
wxiaoguang
750a8465f5
A better go code formatter, and now make fmt can run in Windows (#17684)
* go build / format tools
* re-format imports
2021-11-17 20:34:35 +08:00
zeripath
bbffcc3aec
Multiple Escaping Improvements (#17551)
There are multiple places where Gitea does not properly escape URLs that it is building and there are multiple places where it builds urls when there is already a simpler function available to use this.
    
This is an extensive PR attempting to fix these issues.

1. The first commit in this PR looks through all href, src and links in the Gitea codebase and has attempted to catch all the places where there is potentially incomplete escaping.
2. Whilst doing this we will prefer to use functions that create URLs over recreating them by hand.
3. All uses of strings should be directly escaped - even if they are not currently expected to contain escaping characters. The main benefit to doing this will be that we can consider relaxing the constraints on user names and reponames in future. 
4. The next commit looks at escaping in the wiki and re-considers the urls that are used there. Using the improved escaping here wiki files containing '/'. (This implementation will currently still place all of the wiki files the root directory of the repo but this would not be difficult to change.)
5. The title generation in feeds is now properly escaped.
6. EscapePound is no longer needed - urls should be PathEscaped / QueryEscaped as necessary but then re-escaped with Escape when creating html with locales Signed-off-by: Andrew Thornton <art27@cantab.net>

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-11-16 18:18:25 +00:00
Lunny Xiao
7e1ae38097
Move migrations into services and base into modules/migration (#17663)
* Move migrtions into services and base into modules/migration

* Fix imports

* Fix lint
2021-11-16 23:25:33 +08:00
Lunny Xiao
48ccd325a1
Move some functions into services/repository (#17660) 2021-11-16 21:30:11 +08:00
wxiaoguang
81926d61db
Decouple unit test, remove intermediate unittestbridge package (#17662)
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-11-16 16:53:21 +08:00
Gusted
253d9e4158
Remove unnecassary calls to filepath.Join (#17608)
- Partialy resolvess #17596
- Resolves `badCall` errors from go-critic `badCall: suspicious Join on
1 argument`
- When only 1 argument is passed into `filepath.Join`, it won't do
anything special other than `filepath.Clean(...)` will be applied over
it.

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: 6543 <6543@obermui.de>
2021-11-15 14:02:53 +08:00
Michael Grigoryan
47448083a1
Minor readability patch. (#17627) 2021-11-12 18:27:18 -05:00
wxiaoguang
df64fa4865
Decouple unit test code from business code (#17623) 2021-11-12 22:36:47 +08:00
Lunny Xiao
90eb9fb889
Move EmailAddress & UserRedirect into models/user/ (#17607)
* Move EmailAddress into models/user/

* Fix test

* rename user_mail to user_email

* Fix test

* Move UserRedirect into models/user/

* Fix lint & test

* Fix lint

* Fix lint

* remove nolint comment

* Fix lint
2021-11-11 15:03:30 +08:00
Lunny Xiao
0e189eecaa
Move oauth2 error to oauth2 service package (#17603) 2021-11-11 01:51:09 +08:00
Lunny Xiao
33fca2b537
Move webhook into models/webhook/ (#17579) 2021-11-10 13:13:16 +08:00
Lunny Xiao
99b2858e62
Move unit into models/unit/ (#17576)
* Move unit into models/unit/

* Rename unit.UnitType as unit.Type
2021-11-09 20:57:58 +01:00
Gusted
30515f2df3
Make ParsePatch more robust (#17573) 2021-11-07 12:52:50 -05:00
zeripath
9d855bd6a1
Simplify Gothic to use our session store instead of creating a different store (#17507)
* Simplify Gothic to use our session store instead of creating a different store

We have been using xormstore to provide a separate session store for our OAuth2 logins
however, this relies on using gorilla context and some doubling of our session storing.
We can however, simplify and simply use our own chi-based session store. Thus removing
a cookie and some of the weirdness with missing contexts.

Signed-off-by: Andrew Thornton <art27@cantab.net>

* as per review

Signed-off-by: Andrew Thornton <art27@cantab.net>

* as per review

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Handle MaxTokenLength

Signed-off-by: Andrew Thornton <art27@cantab.net>

* oops

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: Lauris BH <lauris@nix.lv>
2021-11-03 08:33:54 +08:00
wxiaoguang
599ff1c054
Only allow webhook to send requests to allowed hosts (#17482) 2021-11-01 16:39:52 +08:00
Richard Mahn
40c8451b7d
Properly determine CSV delimiter (#17459)
* Fixes #16558 CSV delimiter determiner

* Fixes #16558 - properly determine CSV delmiiter

* Moves quoteString to a new function

* Adds big test with lots of commas for tab delimited csv

* Adds comments

* Shortens the text of the test

* Removes single quotes from regexp as only double quotes need to be searched

* Fixes spelling

* Fixes check of length as it probalby will only be 1e4, not greater

* Makes sample size a const, properly removes truncated line

* Makes sample size a const, properly removes truncated line

* Fixes comment

* Fixes comment

* tests for FormatError() function

* Adds logic to find the limiter before or after a quoted value

* Simplifies regex

* Error tests

* Error tests

* Update modules/csv/csv.go

Co-authored-by: delvh <dev.lh@web.de>

* Update modules/csv/csv.go

Co-authored-by: delvh <dev.lh@web.de>

* Adds comments

* Update modules/csv/csv.go

Co-authored-by: delvh <dev.lh@web.de>

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: delvh <dev.lh@web.de>
2021-10-30 23:50:40 +08:00
qwerty287
01fc24c78c
Add appearance section in settings (#17433)
* Add appearance section in settings

* Fix lint

* Fix lint

* Apply suggestions from code review

Co-authored-by: Lauris BH <lauris@nix.lv>

Co-authored-by: Lauris BH <lauris@nix.lv>
2021-10-27 23:40:08 +08:00
KN4CK3R
f99d50fc9f
Read expected buffer size (#17409)
* Read expected buffer size.

* Changed name.
2021-10-24 22:12:43 +01:00
Nico Schieder
870f5fbc41
Add groups scope/claim to OIDC/OAuth2 Provider (#17367)
* Add groups scope/claim to OICD/OAuth2

Add support for groups claim as part of the OIDC/OAuth2 flow.
Groups is a list of "org" and "org:team" strings to allow clients to
authorize based on the groups a user is part of.

Signed-off-by: Nico Schieder <code@nico-schieder.de>
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2021-10-22 17:19:24 +08:00
zeripath
053b2f4dce
Handle broken references in mirror sync (#17013)
* Handle broken references in mirror sync

If there are broken references during a mirror attempt to fix using `git remote prune`.

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-10-21 16:45:25 +08:00
Richard Mahn
98f7013756
Prevent NPE in CSV diff rendering when column removed (#17018)
Fixes #16837 if a column is deleted.

We were clobbering the columns that were added by looping through the aline (base) and then when bline (head) was looped through, it clobbered what was in the "cells" array that is show in the diff, and then left a nil cell because nothing was shifted.

This fix properly shifts the cells, and properly puts the b cell either at its location or after, according to what the aline placed in the cells.

This includes test, adding a new test function since adding/removing cells works best with three columns, not two, which results in 4 columns of the resulting cells because it has a deleted column and an added column. If you try this locally, you can try those cases and others, such as adding a column.

There was no need to do anything special for the rows when `aline == 0 || bline == 0` so that was removed. This allows the same code to be used for removed or added lines, with the bcell text always being the RightCell, acell text being the LeftCell.

I still added the patch zeripath gave at https://github.com/go-gitea/gitea/issues/16837#issuecomment-913007382 so that just in case for some reason a cell is nil (which shouldn't happen now) it doesn't throw a 500 error, so the user can at least view the raw diff.

Also fixes in the [view.go](https://github.com/go-gitea/gitea/pull/17018/files#diff-43a7f4747c7ba8bff888c9be11affaafd595fd55d27f3333840eb19df9fad393L521) file how if a CSV file is empty (either created empty or if you edit it and remove all contents) it throws a huge 500 error when you then save it (when you view the file). Since we allow creating, saving and pushing empty files, we shouldn't throw an error on an empty CSV file, but just show its empty contents. This doesn't happen if it is a Markdown file or other type of file that is empty.
EDIT: Now handled in the markup/csv renderer code
2021-10-20 20:10:03 +01:00
Lunny Xiao
f494776931
Use a variable but a function for IsProd because of a slight performance increment (#17368) 2021-10-20 16:37:19 +02:00
Lunny Xiao
4a57c9ea17
Fix some lints (#17337)
Fix some linting problems.
2021-10-17 20:47:12 +01:00
zeripath
7117c7774a
Make the Mirror Queue a queue (#17326)
Convert the old mirror syncing queue to the more modern queue format.

Fix a bug in the from the repo-archive queue PR - the assumption was made that uniqueness could be enforced with by checking equality in a map in channel unique queues - however this only works for primitive types - which was the initial intention but is an imperfect. This is fixed by marshalling the data and placing the martialled data in the unique map instead.

The documentation is also updated to add information about the deprecated configuration values.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-10-17 12:43:25 +01:00
zeripath
a889d0cc8c
Add buttons to allow loading of incomplete diffs (#16829)
This PR adds two buttons to the stats and the end of the diffs list to load the (some of) the remaining incomplete diff sections.

Contains #16775
    
Signed-off-by: Andrew Thornton <art27@cantab.net>


## Screenshots

### Show more button at the end of the diff
![Screenshot from 2021-09-04 11-12-37](https://user-images.githubusercontent.com/1824502/132091009-b1f6113e-2c04-4be5-8a04-b8ecea56887b.png)

### Show more button at the end of the diff stats box
![Screenshot from 2021-09-04 11-14-54](https://user-images.githubusercontent.com/1824502/132091063-86da5a6d-6628-4b82-bea9-3655cd9f40f6.png)
2021-10-15 17:05:33 +01:00
Lunny Xiao
d47798ccc7
Upgrade chi to v5 (#17298) 2021-10-13 22:50:23 -04:00
wxiaoguang
c5c88f2f18
Save and view issue/comment content history (#16909)
* issue content history

* Use timeutil.TimeStampNow() for content history time instead of issue/comment.UpdatedUnix (which are not updated in time)

* i18n for frontend

* refactor

* clean up

* fix refactor

* re-format

* temp refactor

* follow db refactor

* rename IssueContentHistory to ContentHistory, remove empty model tags

* fix html

* use avatar refactor to generate avatar url

* add unit test, keep at most 20 history revisions.

* re-format

* syntax nit

* Add issue content history table

* Update models/migrations/v197.go

Co-authored-by: 6543 <6543@obermui.de>

* fix merge

Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lauris BH <lauris@nix.lv>
2021-10-10 18:40:03 -04:00
a1012112796
bb39359668
Add a simple way to rename branch like gh (#15870)
- Update default branch if needed
- Update protected branch if needed
- Update all not merged pull request base branch name
- Rename git branch
- Record this rename work and auto redirect for old branch on ui

Signed-off-by: a1012112796 <1012112796@qq.com>
Co-authored-by: delvh <dev.lh@web.de>
2021-10-08 19:03:04 +02:00
wxiaoguang
f0ba87fda8
Avatar refactor, move avatar code from models to models.avatars, remove duplicated code (#17123)
Why this refactor

The goal is to move most files from `models` package to `models.xxx` package. Many models depend on avatar model, so just move this first.

And the existing logic is not clear, there are too many function like `AvatarLink`, `RelAvatarLink`, `SizedRelAvatarLink`, `SizedAvatarLink`, `MakeFinalAvatarURL`, `HashedAvatarLink`, etc. This refactor make everything clear:

* user.AvatarLink()
* user.AvatarLinkWithSize(size)
* avatars.GenerateEmailAvatarFastLink(email, size)
* avatars.GenerateEmailAvatarFinalLink(email, size)

And many duplicated code are deleted in route handler, the handler and the model share the same avatar logic now.
2021-10-06 01:25:46 +02:00
John Olheiser
760d61b411
Add specific event type to header (#17222)
* Add specific event type to header

Signed-off-by: jolheiser <john.olheiser@gmail.com>

* Change single to type
2021-10-05 12:12:17 -05:00
KN4CK3R
3b06675811
Always set a unique Message-ID header. (#17206) 2021-10-01 11:24:43 -04:00
zeripath
ddd3586c06
Fix missing repo link in issue/pull assigned emails (#17183)
There was a mistake in the template file: `templates/mail/issue/assigned.tmpl`
where the repourl was generated from a non-existent release instead of the issue.

This PR changes this to use the issue but also ensure that the issue repo is loaded.

It also slightly improves the English locale string.

Fix #17160

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: delvh <dev.lh@web.de>
2021-09-30 01:20:18 +02:00
Romain
ecfac78f6e
Kanban colored boards (#16647)
Add a column Color in ProjectBoard and color picker in new / edit project board form.
2021-09-29 22:53:12 +02:00
Norwin
f48dce3176
Don't return binary file changes in raw PR diffs by default (#17158)
* return diffs without binary file content change

* ?binary=true option to restore old behaviour

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: zeripath <art27@cantab.net>
2021-09-27 17:09:49 -04:00
zeripath
920608e592
Support direct comparison (git diff a..b) as well merge comparison (a...b) (#16635)
This PR changes the compare page to make the "..." in the between branches a clickable
link. This changes the comparison type from "..." to "..". Similarly it makes the
initial compare icon clickable to switch the head and base branches.

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-09-27 13:19:34 +01:00
zeripath
123f0aea00
Allow LDAP Sources to provide Avatars (#16851)
* Allow LDAP Sources to provide Avatars

Add setting to LDAP source to allow it to provide an Avatar.

Currently this is required to point to the image bytes.

Fix #4144

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Rename as Avatar Attribute (drop JPEG)

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Always synchronize avatar if there is change

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Actually get the avatar from the ldap

Signed-off-by: Andrew Thornton <art27@cantab.net>

* clean-up

Signed-off-by: Andrew Thornton <art27@cantab.net>

* use len()>0 rather than != ""

Signed-off-by: Andrew Thornton <art27@cantab.net>

* slight shortcut in IsUploadAvatarChanged

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-09-26 22:39:36 -04:00
zeripath
7e98cd58dd
Add SkipLocal2FA option to pam and smtp sources (#17078)
* Add SkipLocal2FA option to other pam and smtp sources

Extend #16954 to allow setting skip local 2fa on pam and SMTP authentication sources

Signed-off-by: Andrew Thornton <art27@cantab.net>

* make SkipLocal2FA omitempty

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: 6543 <6543@obermui.de>
2021-09-26 21:02:01 -04:00
Alexey 〒erentyev
7e9bd206fd
Fix bundle creation (#17079)
Signed-off-by: Alexey Terentyev <axifnx@gmail.com>

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Gwyneth Morgan <87623694+gwymor@users.noreply.github.com>
Co-authored-by: Gwyneth Morgan <gwymor@tilde.club>
2021-09-26 00:29:25 +03:00
Lunny Xiao
5842a55b31
Move login related structs and functions to models/login (#17093)
* Move login related structs and functions to models/login

* Fix test

* Fix lint

* Fix lint

* Fix lint of windows

* Fix lint

* Fix test

* Fix test

* Only load necessary fixtures when preparing unit tests envs

* Fix lint

* Fix test

* Fix test

* Fix error log

* Fix error log

* Fix error log

* remove unnecessary change

* fix error log

* merge main branch
2021-09-24 19:32:56 +08:00
zeripath
9302eba971
DBContext is just a Context (#17100)
* DBContext is just a Context

This PR removes some of the specialness from the DBContext and makes it context
This allows us to simplify the GetEngine code to wrap around any context in future
and means that we can change our loadRepo(e Engine) functions to simply take contexts.

Signed-off-by: Andrew Thornton <art27@cantab.net>

* fix unit tests

Signed-off-by: Andrew Thornton <art27@cantab.net>

* another place that needs to set the initial context

Signed-off-by: Andrew Thornton <art27@cantab.net>

* avoid race

Signed-off-by: Andrew Thornton <art27@cantab.net>

* change attachment error

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-09-23 23:45:36 +08:00
Eng Zer Jun
f2e7d5477f
refactor: move from io/ioutil to io and os package (#17109)
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-09-22 13:38:34 +08:00
Lunny Xiao
a4bfef265d
Move db related basic functions to models/db (#17075)
* Move db related basic functions to models/db

* Fix lint

* Fix lint

* Fix test

* Fix lint

* Fix lint

* revert unnecessary change

* Fix test

* Fix wrong replace string

* Use *Context

* Correct committer spelling and fix wrong replaced words

Co-authored-by: zeripath <art27@cantab.net>
2021-09-19 19:49:59 +08:00
wxiaoguang
6532aa2ed3
Open the DingTalk link in browser (#17084)
https://developers.dingtalk.com/document/app/message-link-description

To open the link in browser, we should use this URL: `"dingtalk://dingtalkclient/page/link?pc_slide=false&url=" + url.QueryEscape(singleURL)`

Otherwise the page is displayed inside DingTalk client, it makes users very difficult to visit non-public URLs in DingTalk webhook messages.
2021-09-18 20:35:23 +01:00
zeripath
27b351aba5
Make LDAP be able to skip local 2FA (#16954)
This PR extends #16594 to allow LDAP to be able to be set to skip local 2FA too. The technique used here would be extensible to PAM and SMTP sources.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-09-17 12:43:47 +01:00
zeripath
db6b7db06d
Improve LDAP synchronization efficiency (#16994)
The current LDAP sync routine has order n^2 efficiency. This change reduces this
to order n.log n.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-09-15 10:28:37 +01:00
Jimmy Praet
3d6cb25e31
Support unprotected file patterns (#16395)
Fixes #16381

Note that changes to unprotected files via the web editor still cannot be pushed directly to the protected branch. I could easily add such support for edits and deletes if needed. But for adding, uploading or renaming unprotected files, it is not trivial.

* Extract & Move GetAffectedFiles to modules/git
2021-09-11 16:21:17 +02:00
zeripath
f394019e9d
Report the associated filter if there is an error in LDAP (#17014)
If there is an error when performing the admin or restricted filter search
ensure that the filter and the userDN are reported.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-09-11 00:46:27 +02:00
zeripath
9ca0e7905c
Add setting to OAuth handlers to skip local 2FA authentication (#16594)
This PR adds a setting to OAuth and OpenID login sources to allow the source to
skip local 2FA authentication.

Fix #13939

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-09-10 18:37:57 +02:00
zeripath
248b96d8a3
Use git attributes to determine generated and vendored status for language stats and diffs (#16773)
Replaces #16262
Replaces #16250
Replaces #14833

This PR first implements a `git check-attr` pipe reader - using `git check-attr --stdin -z --cached` - taking account of the change in the output format in git 1.8.5 and creates a helper function to read a tree into a temporary index file for that pipe reader.

It then wires this in to the language stats helper and into the git diff generation.

Files which are marked generated will be folded by default.

Fixes #14786
Fixes #12653
2021-09-09 21:13:36 +01:00
a1012112796
358555f72c
Return correct error response for agit force-push (#16989)
fix a samll nit for agit `force-push` error response

Signed-off-by: a1012112796 <1012112796@qq.com>
2021-09-08 22:20:55 +02:00
Lunny Xiao
ddc709ff7f
Add repo_id for attachment (#16958)
When create a new issue or comment and paste/upload an attachment/image, it will not assign an issue id before submit. So if user give up the creating, the attachments will lost key feature and become dirty content. We don't know if we need to delete the attachment even if the repository deleted.

This PR add a repo_id in attachment table so that even if a new upload attachment with no issue_id or release_id but should have repo_id. When deleting a repository, they could also be deleted.

Co-authored-by: 6543 <6543@obermui.de>
2021-09-08 17:19:30 +02:00
6543
63d7cbcef1
Make mirror feature more configurable (#16957)
Rename`[repository]` `DISABLE_MIRRORS` to `[mirror]` `DISABLE_NEW_PULL`  and add `ENABLED` and `DISABLE_NEW_PUSH` with the below meanings:

- `ENABLED`: **true**: Enables the mirror functionality. Set to **false** to disable all mirrors.
- `DISABLE_NEW_PULL`: **false**: Disable the creation of **new** mirrors. Pre-existing mirrors remain valid.
- `DISABLE_NEW_PUSH`: **false**: Disable the creation of **new** push mirrors. Pre-existing mirrors remain valid.


Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: delvh <dev.lh@web.de>
2021-09-07 17:49:36 +02:00
zeripath
0767fe0d80
Prevent panic on diff generation (#16950)
The lastLeftIdx should be reset at the same time as creating a new section otherwise
it is possible for a second addition to end up attempting to read a nil entry.

Fix #16943

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-09-05 12:16:49 +02:00
zeripath
a75b2f2842
Allow BASIC authentication access to /:owner/:repo/releases/download/* (#16916)
Duplicate #15987 to allow access to releases download through BASIC authentication.

Fix #16914

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-09-02 16:48:48 +01:00
a1012112796
cbf05c3f79
Add option to update pull request by rebase (#16125)
* add option to update pull request by `rebase`

Signed-off-by: a1012112796 <1012112796@qq.com>
2021-08-31 16:03:45 +02:00
KN4CK3R
2bb32006fd
Test if LFS object is accessible (#16865)
* Test if object is accessible.

* Added more logging.
2021-08-31 14:35:08 +01:00
6543
bb4cc876b1
Repare and Improve GetDiffRangeWithWhitespaceBehavior (#16894)
* repare and improve GetDiffRangeWithWhitespaceBehavior

* Context with Timeout
2021-08-31 06:16:23 +02:00
Lunny Xiao
8d7704b5a2
Fix dump and restore respository (#16698)
* Fix dump and restore

* return different error message for get commit

* Fix missing delete release attachment when deleting repository

* Fix ci and add some comments

Co-authored-by: zeripath <art27@cantab.net>
2021-08-30 21:18:15 -04:00
zeripath
f5b0e2c9d2
Simplify split diff view generation and remove JS dependency (#16775)
Gitea has relied on some slow JS code to match up added and deleted lines on the
diff pages. This can cause a considerable slow down on large diff pages.

This PR makes a small change meaning that the matching up can occur much more simply.

Partial fix #1351

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-08-29 15:28:04 +01:00
Kyle Evans
cad70599a6
Refactor the fork service slightly to take ForkRepoOptions (#16744)
* Refactor the fork service slightly to take ForkRepoOptions

This reduces the number of places we need to change if we want to add other
options during fork time.

Signed-off-by: Kyle Evans <kevans@FreeBSD.org>

* Fix integrations and tests after ForkRepository refactor

Signed-off-by: Kyle Evans <kevans@FreeBSD.org>

* Update OldRepo -> BaseRepo

Signed-off-by: Kyle Evans <kevans@FreeBSD.org>

* gofmt pass

Signed-off-by: Kyle Evans <kevans@FreeBSD.org>
2021-08-28 16:37:14 +08:00
KN4CK3R
88abb0dc8a
Decoupled code from DefaultSigningKey (#16743)
Decoupled code from `DefaultSigningKey`. Makes testing a little bit easier and is cleaner.
2021-08-27 20:28:00 +01:00
Aliaksandr Mianzhynski
28ac4a7a87
Add EdDSA JWT signing algorithm (#16786)
* Add EdDSA signing algorithm

* Fix typo

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-08-25 16:50:38 -04:00
John Olheiser
648464b504
Add bundle download for repository (#14538)
* Add bundle download

Signed-off-by: jolheiser <john.olheiser@gmail.com>

* Fix fmt

Signed-off-by: jolheiser <john.olheiser@gmail.com>

* Fix build tags

Signed-off-by: jolheiser <john.olheiser@gmail.com>

* Download specific commit

Signed-off-by: jolheiser <john.olheiser@gmail.com>
2021-08-24 11:47:09 -05:00
zeripath
7f85610942
Fix openidConnect source regression from #16544 (#16759)
Unfortunately there is bug in #16544 meaning that openid connects aren't
being matched properly as the capitalisation in that PR is incorrect.

This PR changes the capitalisation back to what is expected.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-08-22 10:17:05 +01:00
Abner
7844bf1430
Download lfs in git and web workflow from minio/s3 directly (#16731) 2021-08-21 20:22:06 +02:00
KN4CK3R
0bd58d61e5
Added introspection endpoint. (#16752)
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-08-20 22:16:45 -04:00
Lunny Xiao
f9acad82ca
Add proxy settings and support for migration and webhook (#16704)
* Add proxy settings and support for migration and webhook

* Fix default value

* Add newline for example ini

* Add lfs proxy support

* Fix lint

* Follow @zeripath's review

* Fix git clone

* Fix test

* missgin http requests for proxy

* use empty

Co-authored-by: zeripath <art27@cantab.net>

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: zeripath <art27@cantab.net>
2021-08-18 21:10:39 +08:00
techknowlogick
37d0d94843
Use github token URL when fetching oauth token (#16709)
Fixes regression from #16544
2021-08-17 09:07:52 +01:00
Lunny Xiao
74d75eb69c
Fix spelling of HookProcReceiveResult (#16690) 2021-08-14 13:17:10 +02:00
6543
2289580bb7
[API] generalize list header (#16551)
* Add info about list endpoints to CONTRIBUTING.md

* Let all list endpoints return X-Total-Count header 

* Add TODOs for GetCombinedCommitStatusByRef

* Fix models/issue_stopwatch.go

* Rrefactor models.ListDeployKeys

* Introduce helper func and use them for SetLinkHeader related func
2021-08-12 14:43:08 +02:00
zeripath
162c32af7e
Send registration email on user autoregistration (#16523)
When users login and are autoregistered send email notification.

Fix #16178

* Protect public functions within the mailer by testing if the mailer is configured

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: 6543 <6543@obermui.de>
2021-08-12 10:26:33 +03:00
zeripath
e29e163737
Improve SMTP authentication and Fix user creation bugs (#16612)
* Improve SMTP authentication, Fix user creation bugs and add LDAP cert/key options

This PR has two parts:

Improvements for SMTP authentication:

* Default to use SMTPS if port is 465, and allow setting of force SMTPS.
* Always use STARTTLS if available
* Provide CRAM-MD5 mechanism
* Add options for HELO hostname disabling
* Add options for providing certificates and keys
* Handle application specific password response as a failed user login
instead of as a 500.

Close #16104

Fix creation of new users:

* A bug was introduced when allowing users to change usernames which
prevents the creation of external users.
* The LoginSource refactor also broke this page.

Close #16104

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-08-11 21:42:58 +01:00
6543
c4d70a0325
Rename ctx.Form() to ctx.FormString() and move code into own file (#16571)
Followup from #16562 prepare for #16567

* Rename ctx.Form() to ctx.FormString()
* Reimplement FormX func to need less code and cpu cycles
* Move code into own file
2021-08-11 02:31:13 +02:00
KN4CK3R
d9ef43a712
Replace list.List with slices (#16311)
* Replaced list with slice.

* Fixed usage of pointer to temporary variable.

* Replaced LIFO list with slice.

* Lint

* Removed type check.

* Removed duplicated code.

* Lint

* Fixed merge.

Co-authored-by: 6543 <6543@obermui.de>
2021-08-09 14:08:51 -04:00
zeripath
ab9bb54144
Add microsoft oauth2 providers (#16544)
* Clean up oauth2 providers

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Add AzureAD, AzureADv2, MicrosoftOnline OAuth2 providers

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Apply suggestions from code review

* remove unused Scopes

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-08-05 21:11:08 -04:00
zeripath
e51c73ae5c
Fix 500 on first wiki page (#16586)
* Fix 500 on first wiki page

There is a mistake in #16319 and #16487 which means that the first time
a wiki page is created a 500 is reported because the `master` branch is
not in existence in that wiki yet.

This PR simply checks for this error and returns not found.

Fix #16584

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-08-01 19:04:32 +02:00
zeripath
72738f0cb5
Lock goth/gothic and Re-attempt OAuth2 registration on login if registration failed at startup (#16564)
This PR has two parts:

* Add locking to goth and gothic calls with a RWMutex

The goth and gothic calls are currently unlocked and thus are a cause of multiple potential races

* Reattempt OAuth2 registration on login if registration failed

If OAuth2 registration fails at startup we currently disable the login_source however an alternative approach could be to reattempt registration on login attempt.
    
Fix #16096

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-07-29 18:53:18 +01:00
Lunny Xiao
33e0b38287
Rename context.Query to context.Form (#16562) 2021-07-29 03:42:15 +02:00
a1012112796
3705168837
Add agit flow support in gitea (#14295)
* feature: add agit flow support

ref: https://git-repo.info/en/2020/03/agit-flow-and-git-repo/

example:

```Bash
git checkout -b test
echo "test" >> README.md
git commit -m "test"
git push origin HEAD:refs/for/master -o topic=test
```

Signed-off-by: a1012112796 <1012112796@qq.com>

* fix lint

* simplify code add fix some nits

* update merge help message

* Apply suggestions from code review. Thanks @jiangxin

* add forced-update message

* fix lint

* splite writePktLine

* add refs/for/<target-branch>/<topic-branch> support also

* Add test code add fix api

* fix lint

* fix test

* skip test if git version < 2.29

* try test with git 2.30.1

* fix permission check bug

* fix some nit

* logic implify and test code update

* fix bug

* apply suggestions from code review

* prepare for merge

Signed-off-by: Andrew Thornton <art27@cantab.net>

* fix permission check bug

- test code update
- apply suggestions from code review @zeripath

Signed-off-by: a1012112796 <1012112796@qq.com>

* fix bug when target branch isn't exist

* prevent some special push and fix some nits

* fix lint

* try splite

* Apply suggestions from code review

- fix permission check
- handle user rename

* fix version negotiation

* remane

* fix template

* handle empty repo

* ui: fix  branch link under the title

* fix nits

Co-authored-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-07-28 17:42:56 +08:00
zeripath
6a33b290a0
Fix add authentication page (#16543)
* Fix add authentication page

There is a regression in #16199 whereby the add authentication page
fails to react to the change in selected type.

This is due to the String() method on the LoginSourceType which is ameliorated
with an Int() function being added.

Following on from this there are a few other related bugs.

Fix #16541

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-07-25 08:09:52 +01:00
Lunny Xiao
9f31f3aa8a
Add an abstract json layout to make it's easier to change json library (#16528)
* Add an abstract json layout to make it's easier to change json library

* Fix import

* Fix import sequence

* Fix blank lines

* Fix blank lines
2021-07-24 18:03:58 +02:00
techknowlogick
29a22ade82
switch to maintained lib (#16532)
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Andrew Thornton <art27@cantab.net>
2021-07-24 13:00:41 +02:00
zeripath
5d2e11eedb
Refactor: Move login out of models (#16199)
`models` does far too much. In particular it handles all `UserSignin`.

It shouldn't be responsible for calling LDAP, SMTP or PAM for signing in.

Therefore we should move this code out of `models`.

This code has to depend on `models` - therefore it belongs in `services`.

There is a package in `services` called `auth` and clearly this functionality belongs in there.

Plan:

- [x] Change `auth.Auth` to `auth.Method` - as they represent methods of authentication.
- [x] Move `models.UserSignIn` into `auth`
- [x] Move `models.ExternalUserLogin`
- [x] Move most of the `LoginVia*` methods to `auth` or subpackages
- [x] Move Resynchronize functionality to `auth`
  - Involved some restructuring of `models/ssh_key.go` to reduce the size of this massive file and simplify its files.
- [x] Move the rest of the LDAP functionality in to the ldap subpackage
- [x] Re-factor the login sources to express an interfaces `auth.Source`?
  - I've done this through some smaller interfaces Authenticator and Synchronizable - which would allow us to extend things in future
- [x] Now LDAP is out of models - need to think about modules/auth/ldap and I think all of that functionality might just be moveable
- [x] Similarly a lot Oauth2 functionality need not be in models too and should be moved to services/auth/source/oauth2
  - [x] modules/auth/oauth2/oauth2.go uses xorm... This is naughty - probably need to move this into models.
  - [x] models/oauth2.go - mostly should be in modules/auth/oauth2 or services/auth/source/oauth2 
- [x] More simplifications of login_source.go may need to be done
- Allow wiring in of notify registration -  *this can now easily be done - but I think we should do it in another PR*  - see #16178 
- More refactors...?
  - OpenID should probably become an auth Method but I think that can be left for another PR
  - Methods should also probably be cleaned up  - again another PR I think.
  - SSPI still needs more refactors.* Rename auth.Auth auth.Method
* Restructure ssh_key.go

- move functions from models/user.go that relate to ssh_key to ssh_key
- split ssh_key.go to try create clearer function domains for allow for
future refactors here.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-07-24 11:16:34 +01:00
lengyuqu
078e2b2c39
Add support for corporate WeChat webhooks (#15910)
* 企业微信webhook

* 企业微信webhook

* 企业微信webhook

* Update templates/admin/hook_new.tmpl

Co-authored-by: a1012112796 <1012112796@qq.com>

* Update services/webhook/wechatwork.go

Co-authored-by: a1012112796 <1012112796@qq.com>

* 修善wechatwork

* 修善wechatwork

* fix

* Update locale_cs-CZ.ini

fix

* fix build

* fix

* fix build

* make webhooks.zh-cn.md

* delet unnecessary blank line

* delet unnecessary blank line

* 企业微信webhook

* 企业微信webhook

* 企业微信webhook

* Update templates/admin/hook_new.tmpl

Co-authored-by: a1012112796 <1012112796@qq.com>

* Update services/webhook/wechatwork.go

Co-authored-by: a1012112796 <1012112796@qq.com>

* 修善wechatwork

* 修善wechatwork

* fix

* fix build

* fix

* fix build

* make webhooks.zh-cn.md

* delet unnecessary blank line

* delet unnecessary blank line

* 企业微信webhook

* 企业微信webhook

* 企业微信webhook

* 企业微信webhook

* 企业微信webhook

* fix

* fix

* 企业微信webhook

* 企业微信webhook

* 企业微信webhook

* fix wechat

* fix wechat

* fix wechat

* fix wechat

* Fix invalid params and typo of email templates (#16394)

Signed-off-by: Meano <meanocat@gmail.com>

* Add LRU mem cache implementation (#16226)

The current default memory cache implementation is unbounded in size and number of
objects cached. This is hardly ideal.

This PR proposes creating a TwoQueue LRU cache as the underlying cache for Gitea.
The cache is limited by the number of objects stored in the cache (rather than size)
for simplicity. The default number of objects is 50000 - which is perhaps too small
as most of our objects cached are going to be much less than 1kB.

It may be worth considering using a different LRU implementation that actively limits
sizes or avoids GC - however, this is just a beginning implementation.

Signed-off-by: Andrew Thornton <art27@cantab.net>

* [skip ci] Updated translations via Crowdin

* Replace `plugins/docker` with `techknowlogick/drone-docker`in ci (#16407)

* plugins/docker -> techknowlogick/drone-docker

* It is multi-arch

* docs: rewrite email setup (#16404)

* Add intro for both the docs page and mailer methods
  * Fix numbering level in SMTP section
  * Recommends implicit TLS

Signed-off-by: Bagas Sanjaya <bagasdotme@gmail.com>

* Validate Issue Index before querying DB (#16406)

* Fix external renderer (#16401)

* fix external renderer

* use GBackground context as fallback

* no fallback, return error

Co-authored-by: Lauris BH <lauris@nix.lv>

* Add checkbox to delete pull branch after successful merge (#16049)

* Add checkbox to delete pull branch after successful merge

* Omit DeleteBranchAfterMerge field in json

* Log a warning instead of error when PR head branch deleted

* Add DefaultDeleteBranchAfterMerge to PullRequestConfig

* Add support for delete_branch_after_merge via API

* Fix for API: the branch should be deleted from the HEAD repo

If head and base repo are the same, reuse the already opened ctx.Repo.GitRepo

* Don't delegate to CleanupBranch, only reuse branch deletion code

CleanupBranch contains too much logic that has already been performed by the Merge

* Reuse gitrepo in MergePullRequest

Co-authored-by: Andrew Thornton <art27@cantab.net>

* [skip ci] Updated translations via Crowdin

* Detect encoding changes while parsing diff (#16330)

* Detect encoding changes while parsing diff

* Let branch/tag name be a valid ref to get CI status (#16400)

* fix #16384#

* refactor: move shared helper func to utils package

* extend Tests

* use ctx.Repo.GitRepo if not nil

* fix

* fix

* 企业微信webhook

* 企业微信webhook

* 企业微信webhook

* fix build

* fix build

* Apply suggestions from code review

Co-authored-by: a1012112796 <1012112796@qq.com>
Co-authored-by: myheavily <myheavily>
Co-authored-by: zhaoxin <gitea@fake.local>
Co-authored-by: Meano <Meano@foxmail.com>
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: GiteaBot <teabot@gitea.io>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Bagas Sanjaya <bagasdotme@gmail.com>
Co-authored-by: Norwin <noerw@users.noreply.github.com>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: Jimmy Praet <jimmy.praet@telenet.be>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-07-23 12:41:27 +08:00
6543
b26c3b482f
Add TestPrepareWikiFileName (#16487)
* Add TestPrepareWikiFileName

* use LsTree as LsFiles is index only

* ajust other tests

Co-authored-by: Andrew Thornton <art27@cantab.net>
2021-07-20 15:16:20 +02:00
Gary Wang
e01b782f33
fix: support delete non-urlencoded wiki page (#16482)
* fix: support delete non-urlencoded wiki page

* fix: check error
2021-07-19 18:14:00 +02:00
6543
8798e3a098
Use TrN helper for email templates (#16425)
* Add TrN helper

* use TrN

* a nit
2021-07-14 15:06:09 +02:00
zeripath
b82293270c
Add option to provide signature for a token to verify key ownership (#14054)
* Add option to provide signed token to verify key ownership

Currently we will only allow a key to be matched to a user if it matches
an activated email address. This PR provides a different mechanism - if
the user provides a signature for automatically generated token (based
on the timestamp, user creation time, user ID, username and primary
email.

* Ensure verified keys can act for all active emails for the user

* Add code to mark keys as verified

* Slight UI adjustments

* Slight UI adjustments 2

* Simplify signature verification slightly

* fix postgres test

* add api routes

* handle swapped primary-keys

* Verify the no-reply address for verified keys

* Only add email addresses that are activated to keys

* Fix committer shortcut properly

* Restructure gpg_keys.go

* Use common Verification Token code

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-07-13 15:28:07 +02:00
Jimmy Praet
4ce32c9e93
Detect encoding changes while parsing diff (#16330)
* Detect encoding changes while parsing diff
2021-07-13 03:13:52 +02:00
Jimmy Praet
78118a3b02
Add checkbox to delete pull branch after successful merge (#16049)
* Add checkbox to delete pull branch after successful merge

* Omit DeleteBranchAfterMerge field in json

* Log a warning instead of error when PR head branch deleted

* Add DefaultDeleteBranchAfterMerge to PullRequestConfig

* Add support for delete_branch_after_merge via API

* Fix for API: the branch should be deleted from the HEAD repo

If head and base repo are the same, reuse the already opened ctx.Repo.GitRepo

* Don't delegate to CleanupBranch, only reuse branch deletion code

CleanupBranch contains too much logic that has already been performed by the Merge

* Reuse gitrepo in MergePullRequest

Co-authored-by: Andrew Thornton <art27@cantab.net>
2021-07-13 01:26:25 +02:00
luzpaz
e0296b6a6d
Fix various documentation, user-facing, and source comment typos (#16367)
* Fix various doc, user-facing, and source comment typos

Found via `codespell -q 3 -S ./options/locale,./vendor -L ba,pullrequest,pullrequests,readby`
2021-07-08 13:38:13 +02:00
Gary Wang
b06342f29c
fix: not able to update local created non-urlencoded wiki pages (#16139)
* fix: not able to update local created non-urlencoded wiki pages

* tidy code

* as per suggestion

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Don't replace space to dash for unescaped wiki filename

Co-authored-by: zeripath <art27@cantab.net>

* Remove incorrect comment

* Remove NameToUnescapedFilename()

Co-authored-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-07-07 19:23:09 -04:00
KN4CK3R
aac663e0da
Implemented head_commit for webhooks (#16282)
* Removed Len field.

* Added head_commit webhook field.

* Added comment for returns.
2021-06-29 15:34:03 +02:00
KN4CK3R
9b1b4b5433
Refactor Webhook + Add X-Hub-Signature (#16176)
This PR removes multiple unneeded fields from the `HookTask` struct and adds the two headers `X-Hub-Signature` and `X-Hub-Signature-256`.

## ⚠️ BREAKING ⚠️ 

* The `Secret` field is no longer passed as part of the payload.
* "Breaking" change (or fix?): The webhook history shows the real called url and not the url registered in the webhook (`deliver.go`@129).

Close #16115
Fixes #7788
Fixes #11755

Co-authored-by: zeripath <art27@cantab.net>
2021-06-27 20:21:09 +01:00
Sergey Dryabzhinsky
22a0636544
Add Visible modes function from Organisation to Users too (#16069)
You can limit or hide organisations. This pull make it also posible for users

- new strings to translte
- add checkbox to user profile form
- add checkbox to admin user.edit form
- filter explore page user search
- filter api admin and public user searches
- allow admins view "hidden" users
- add app option DEFAULT_USER_VISIBILITY
- rewrite many files to use Visibility field
- check for teams intersection
- fix context output
- right fake 404 if not visible

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Andrew Thornton <art27@cantab.net>
2021-06-26 20:53:14 +01:00
KN4CK3R
e673e42f7e
Fixed issues not updated by commits (#16254)
`UpdateIssuesCommit` may get called with fewer commits because of `FeedMaxCommitNum` and therefore may miss some commands.
2021-06-26 10:13:51 +01:00
ayb
9b33d18899
Added support for gopher URLs. (#14749)
* Added support for gopher URLs.

* Add setting and make this user settable instead

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: Andrew Thornton <art27@cantab.net>
2021-06-25 18:38:27 -04:00
Jimmy Praet
06f483d0c4
Append to existing trailers in generated squash commit message (#15980)
* Remove superfluous newline before Co-authored-by trailers

* Append to existing PR description trailer section

If the existing PR description message already contains a trailer section (e.g. Signed-off-by: ),
append to it instead of creating a new trailer section.

* Reuse compiled regexp

* Simplify regex and deal with trailing \n in PR description

* Add tests for CommitMessageTrailersPattern

- add support for Key:Value (no space after colon)
- add support for whitespace "folding"

* Update services/pull/pull_test.go

Co-authored-by: Norwin <noerw@users.noreply.github.com>

Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Norwin <noerw@users.noreply.github.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-06-25 13:01:43 -04:00
6543
3ef23d5411
Use gitea logging module for git module (#16243)
remove log() func from gogs times and switch to proper logging

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Andrew Thornton <art27@cantab.net>
2021-06-25 17:54:08 +01:00
KN4CK3R
44b8b07631
Add tag protection (#15629)
* Added tag protection in hook.

* Prevent UI tag creation if protected.

* Added settings page.

* Added tests.

* Added suggestions.

* Moved tests.

* Use individual errors.

* Removed unneeded methods.

* Switched delete selector.

* Changed method names.

* No reason to be unique.

* Allow editing of protected tags.

* Removed unique key from migration.

* Added docs page.

* Changed date.

* Respond with 404 to not found tags.

* Replaced glob with regex pattern.

* Added support for glob and regex pattern.

* Updated documentation.

* Changed white* to allow*.

* Fixed edit button link.

* Added cancel button.

Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-06-25 16:28:55 +02:00
Lunny Xiao
b223d36195
Rework repository archive (#14723)
* Use storage to store archive files

* Fix backend lint

* Add archiver table on database

* Finish archive download

* Fix test

* Add database migrations

* Add status for archiver

* Fix lint

* Add queue

* Add doctor to check and delete old archives

* Improve archive queue

* Fix tests

* improve archive storage

* Delete repo archives

* Add missing fixture

* fix fixture

* Fix fixture

* Fix test

* Fix archiver cleaning

* Fix bug

* Add docs for repository archive storage

* remove repo-archive configuration

* Fix test

* Fix test

* Fix lint

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-06-23 17:12:38 -04:00
Viktor Yakovchuk
08f4b3f312
Fix 500 Error with branch and tag sharing the same name #15592 (#16040)
* Fix 500 Error with branch and tag sharing the same name #15592

Fixed 500 error while create Pull request when there are more
than one sources (branch, tag) with the same name

Fix #15592

Signed-off-by: Viktor Yakovchuk <viktor@yakovchuk.net>

* fix logging

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: 6543 <6543@obermui.de>
2021-06-23 17:08:26 -04:00
KN4CK3R
383ffcfa34
Small refactoring of modules/private (#15947)
* Use correct variable name.

* doer is never nil here.

* Use status code constants.

* Replaced generic map with concrete struct.

* Fixed windows lint.

* Removed unused method.

* Changed error codes.

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-06-23 15:38:19 -04:00
6543
be81dc8b20
Localize Email Templates (#16200)
close #14822
2021-06-23 14:33:21 +02:00
Jimmy Praet
17030ced75
Improve notifications for WIP draft PR's (#14663)
* #14559 Reduce amount of email notifications for WIP draft PR's

don't notify repo watchers of WIP draft PR's

* #13190 Notification when WIP Pull Request is ready for review

* Send email notification to repo watchers when WIP PR is created

* Send ui notification to repo watchers when WIP PR is created

* send specific email notification when PR is marked ready for review

instead of reusing the CreatePullRequest action

* Fix lint error

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-06-23 00:14:22 -04:00
KN4CK3R
4fcae3d06d
Add tests for all webhooks (#16214)
* Added tests for MS Teams.

* Added tests for Dingtalk.

* Added tests for Telegram.

* Added tests for Feishu.

* Added tests for Discord.

* Added tests for closed issue and pullrequest comment.

* Added tests for Matrix.

* Trim all spaces.

* Added tests for Slack.

* Added JSONPayload tests.

* Added general tests.

* Replaced duplicated code.

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-06-20 22:12:19 -04:00
parnic
35742d4af7
Reintroduce squash merge default comment as a config setting (#16134)
* Reinstate most of commit 09304db9a5

* Move the behaviour behind a config setting

* Also fix the initial #12365
2021-06-19 00:08:22 +02:00