Commit graph

76 commits

Author SHA1 Message Date
singuliere
54dd0fc88b
migrations: remove dead code in gitea uploader (#18725)
When migrating, g.issues is a map with all issues created during the
migration. If an issue is not found in g.issues when inserting a
comment or a review, it cannot exist in the database and trying to get
it via GetIssueByIndex() will always fail and return an error.

Signed-off-by: singuliere <singuliere@autistici.org>
2022-02-21 14:00:05 +01:00
6543
3043eb36bf
Delete old git.NewCommand() and use it as git.NewCommandContext() (#18552) 2022-02-06 20:01:47 +01:00
singuliere
8bd89ca294
preserve users if restoring a repository on the same Gitea instance (#18604)
When calling DumpRepository and RestoreRepository on the same Gitea
instance, the users are preserved: all labels, issues etc. belong to
the external user who is, in this particular case, the local user.

Dead code verifying g.gitServiceType.Name() == "" (i.e. plain git) is
removed. The function is never called because the plain git downloader
does not migrate anything that is associated to a user, by definition.

Errors returned by GetUserIDByExternalUserID are no longer ignored.

The userMap is used when the external user is not kown, which is the
most common case. It was only used when the external user exists
which happens less often and, as a result, every occurence of an
unknown external user required a SQL query.

Signed-off-by: Loïc Dachary <loic@dachary.org>

Co-authored-by: Loïc Dachary <loic@dachary.org>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-02-06 17:05:29 +08:00
Gusted
cafd19c1a3
Replace sync.Map with normal maps (#18584)
* Replace `sync.Map` with normal maps

- These maps aren't being used in any kind of concurrent read/write and
thus don't need `sync.Map` and can instead use normal maps.
- Special thanks to dachary.
- Added in: https://github.com/go-gitea/gitea/pull/6290

* Remove unannounced feature
2022-02-03 21:18:18 +02:00
singuliere
367894adc8
add test coverage for original author conversion during migrations (#18506)
* add test coverage for original author conversion during migrations

And create a function to factorize a code snippet that is repeated
five times and would otherwise be more difficult to test and maintain
consistently.

Signed-off-by: Loïc Dachary <loic@dachary.org>

* fix variable scope and int64 formatting

* add missing calls to remapExternalUser and fix misplaced %d

Co-authored-by: Loïc Dachary <loic@dachary.org>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2022-02-01 13:20:28 -05:00
e19b9653ea
GitLab reviews may not have the updated_at field set (#18450)
* GitLab reviews may not have the updated_at field set

Fallback to created_at if that the case and to time.Now() if it is
also missing.

Fixes: 18434

* use assert.WithinDuration

Co-authored-by: Loïc Dachary <loic@dachary.org>
2022-01-29 18:33:20 +01:00
3bb028cc46
Validate migration files (#18203)
JSON Schema validation for data used by Gitea during migrations

Discussion at https://forum.forgefriends.org/t/common-json-schema-for-repository-information/563

Co-authored-by: Loïc Dachary <loic@dachary.org>
2022-01-26 10:45:51 +01:00
Lunny Xiao
a717265e47
Fix restore without topic failure (#18387)
Co-authored-by: zeripath <art27@cantab.net>
2022-01-25 10:44:12 +08:00
6543
54e9ee37a7
format with gofumpt (#18184)
* gofumpt -w -l .

* gofumpt -w -l -extra .

* Add linter

* manual fix

* change make fmt
2022-01-20 18:46:10 +01:00
zeripath
5cb0c9aa0d
Propagate context and ensure git commands run in request context (#17868)
This PR continues the work in #17125 by progressively ensuring that git
commands run within the request context.

This now means that the if there is a git repo already open in the context it will be used instead of reopening it.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-01-19 23:26:57 +00:00
076cead40d
Fix CheckRepoStats and reuse it during migration (#18264)
The CheckRepoStats function missed the following counters:

- label num_closed_issues & num_closed_pulls
- milestone num_closed_issues & num_closed_pulls

The update SQL statements for updating the repository
num_closed_issues & num_closed_pulls fields were repeated in three
functions (repo.CheckRepoStats, migrate.insertIssues and
models.Issue.updateClosedNum) and were moved to a single helper.

The UpdateRepoStats is implemented and called in the Finish migration method so that it happens immediately instead of wating for the
CheckRepoStats to run.

Signed-off-by: Loïc Dachary loic@dachary.org

---
[source](https://lab.forgefriends.org/forgefriends/forgefriends/-/merge_requests/34)
2022-01-17 19:31:58 +01:00
luzpaz
8c647bf0f6
Fix various typos (#18219)
Found via `codespell -q 3 -S ./options/locale,./vendor -L ba,pullrequest,pullrequests,readby,te,unknwon`

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-01-10 17:32:37 +08:00
6543
0572c78938
refactoring nits (#18188)
* no octal for time values

* rm artifact from refactoring

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-01-05 07:27:53 +01:00
zeripath
ffc08c1914
Do not read or write git reference files directly (#18079)
Git will and can pack references into packfiles and therefore if you write/read the
files directly you will get false results. Instead you should use update-ref and
show-ref. To that end I have created three new functions in git/repo_commit.go that
will do this correctly.

Related #17191

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-12-23 21:44:00 +08:00
Gusted
ff2fd08228
Simplify parameter types (#18006)
Remove repeated type declarations in function definitions.
2021-12-20 04:41:31 +00:00
Lunny Xiao
5723240490
Some repository refactors (#17950)
* some repository refactors

* remove unnecessary code

* Fix test

* Remove unnecessary banner
2021-12-12 23:48:20 +08:00
Lunny Xiao
719bddcd76
Move repository model into models/repo (#17933)
* Some refactors related repository model

* Move more methods out of repository

* Move repository into models/repo

* Fix test

* Fix test

* some improvements

* Remove unnecessary function
2021-12-10 09:27:50 +08:00
KN4CK3R
87be76213a
Add migrate from Codebase (#16768)
This PR adds [Codebase](https://www.codebasehq.com/) as migration source.

Supported:
- Milestones
- Issues
- Pull Requests
- Comments
- Labels
2021-12-02 23:24:02 +01:00
Lunny Xiao
9defddb286
Move more model into models/user (#17826)
* Move more model into models/user

* Remove unnecessary comment

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2021-11-28 22:11:58 +08:00
Lunny Xiao
a666829a37
Move user related model into models/user (#17781)
* Move user related model into models/user

* Fix lint for windows

* Fix windows lint

* Fix windows lint

* Move some tests in models

* Merge
2021-11-24 17:49:20 +08:00
KN4CK3R
429ef43793
Use new OneDev /milestones endpoint (#17782)
OneDev changed the API which fails every build at the moment.
2021-11-23 20:28:06 +01:00
wxiaoguang
013fb73068
Use hostmatcher to replace matchlist, improve security (#17605)
Use hostmacher to replace matchlist.

And we introduce a better DialContext to do a full host/IP check, otherwise the attackers can still bypass the allow/block list by a 302 redirection.
2021-11-20 17:34:05 +08:00
Lunny Xiao
fc3d082609
Move attachment into models/repo/ (#17650)
* Move attachment into models/repo/

* Fix test

* Fix bug
2021-11-19 21:39:57 +08:00
99rgosse
e1d655991b
Fix Migrate Description (#17692)
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2021-11-18 23:28:10 +08:00
Lunny Xiao
3c3855a05c
Move user functions into user.go (#17659)
* Move user functions into user.go

* Fix test
2021-11-18 13:58:42 +08:00
Lunny Xiao
7e1ae38097
Move migrations into services and base into modules/migration (#17663)
* Move migrtions into services and base into modules/migration

* Fix imports

* Fix lint
2021-11-16 23:25:33 +08:00