Unfortunately the v180 migration picked up a few non-standalone dependencies. This PR
forcibly copies the important parts back into the migration.
Fix#16150
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Add a new table issue_index to store the max issue index so that issue could be deleted with no duplicated index
* Fix pull index
* Add tests for concurrent creating issues
* Fix lint
* Fix tests
* Fix postgres test
* Add test for migration v180
* Rename wrong test file name
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lauris BH <lauris@nix.lv>
* Always store primary email address into email_address table and also the state
* Add lower_email to not convert email to lower as what's added
* Fix fixture
* Fix tests
* Use BeforeInsert to save lower email
* Fix v180 migration
* fix tests
* Fix test
* Remove wrong submited codes
* Fix test
* Fix test
* Fix test
* Add test for v181 migration
* remove change user's email to lower
* Revert change on user's email column
* Fix lower email
* Fix test
* Fix test
* encrypt migration credentials in task persistence
Not sure this is the best approach, we could encrypt the entire
`PayloadContent` instead. Also instead of clearing individual fields in
payload content, we could just delete the task once it has
(successfully) finished..?
* remove credentials of past migrations
* only run DB migration for completed tasks
* fix binding
* add omitempty
* never serialize unencrypted credentials
* fix import order
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Double the avatar size factor
This results on finer Avatar rendering on Hi-DPI display.
* fix test
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Encrypt LDAP bind password in db with SECRET_KEY
The LDAP source bind password are currently stored in plaintext in the db
This PR simply encrypts them with the setting.SECRET_KEY.
Fix#15460
Signed-off-by: Andrew Thornton <art27@cantab.net>
* remove ui warning regarding unencrypted password
Co-authored-by: silverwind <me@silverwind.io>
* Restore PAM user autocreation functionality
PAM autoregistration of users currently fails due to email invalidity.
This PR adds a new setting to PAM to allow an email domain to be set
or just sets the email to the noreply address and if that fails falls
back to uuid@localhost
Fix#15702
Signed-off-by: Andrew Thornton <art27@cantab.net>
* As per KN4CKER
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Unregister non-matching serviceworkers
With the addition of the /assets url, users who visited a previous
version of the site now may have two active service workers, one with
the old scope `/` and one with scope `/assets`. This check for
serviceworkers that do not match the current script path and unregisters
them.
Also included is a small refactor to publicpath.js which was simplified
because AssetUrlPrefix is always present now. Also it makes use of the
new joinPaths helper too.
Fixes: https://github.com/go-gitea/gitea/pull/15823
Unfortunately some old repositories can have tags with empty Tagger, Commit
or Author. Go-Git variants will always have empty values for these whereas
the native git variant leaves them at nil. The simplest solution is just to
always have these set to empty Signatures.
v156 migration also makes the incorrect assumption that these cannot be empty.
Therefore add some handling to this and add logging and adjust broken
logging elsewhere in this migration.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Decouple TestAction_GetRepoLink and TestSizedAvatarLink.
* Load database for TestCheckGPGUserEmail.
* Load database for TestMakeIDsFromAPIAssigneesToAdd.
* Load database for TestGetUserIDsByNames and TestGetMaileableUsersByIDs.
* Load database for TestUser_ToUser.
* Load database for TestRepository_EditWikiPage.
* Include AppSubURL in test.
* Prevent panic with empty slice.
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Use single shared random string generation function
- Replace 3 functions that do the same with 1 shared one
- Use crypto/rand over math/rand for a stronger RNG
- Output only alphanumerical for URL compatibilty
Fixes: #15536
* use const string method
* Update modules/avatar/avatar.go
Co-authored-by: a1012112796 <1012112796@qq.com>
Co-authored-by: a1012112796 <1012112796@qq.com>
Repositories using external issue tracker tend to use numeric issues in
commits. To prevent conflicts during issue reference parsing or inside
commit hooks, this change respects these configuration and uses the !
character to refer to pull requests in merge commit messages.
For repositories using squash merges, this was already handled.
Signed-off-by: JustusBunsi <61625851+justusbunsi@users.noreply.github.com>
Co-authored-by: zeripath <art27@cantab.net>
* Fix setting version table in dump
As noted on Discord there is a problem with gitea dump where the version table
is not being dumped correctly.
This is due to a missing pointer in the TableInfo.
This PR fixes this.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Update models_test.go
* fix some ui bug about draft release
- should not show draft release in tag list because
it will't create real tag
- still show draft release without tag and commit message
for draft release instead of 404 error
- remove tag load for attachement links because it's useless
Signed-off-by: a1012112796 <1012112796@qq.com>
* add test code
* fix test
That's because has added a new release in relaese test database.
* fix dropdown link for draft release
The DB session clean up needs to check expiry not created_unix.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
go panics otherwise with `panic: interface conversion: error is git.ErrNotExist, not *git.ErrNotExist`, thanks to Codeberg/Andi for reporting this.
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
It is possible that tag commits could be deleted or missing from repos. This causes
migration 156 to fail and breaks upgrade.
This PR simply logs the failure.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
If an avatar is requested in a particular size ensure that /avatars also gets the size request
Fix#15453
Signed-off-by: Andrew Thornton <art27@cantab.net>
Some postgres users have logging which logs even failed transactions. So
just query the db before trying to insert.
Fix#15451
Signed-off-by: Andrew Thornton art27@cantab.net
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Refactored handleOAuth2SignIn in routers/user/auth.go
The function handleOAuth2SignIn was called twice but some code path could only
be reached by one of the invocations. Moved the unnecessary code path out of
handleOAuth2SignIn.
* Refactored user creation
There was common code to create a user and display the correct error message.
And after the creation the only user should be an admin and if enabled a
confirmation email should be sent. This common code is now abstracted into
two functions and a helper function to call both.
* Added auto-register for OAuth2 users
If enabled new OAuth2 users will be registered with their OAuth2 details.
The UserID, Name and Email fields from the gothUser are used.
Therefore the OpenID Connect provider needs additional scopes to return
the coresponding claims.
* Added error for missing fields in OAuth2 response
* Linking and auto linking on oauth2 registration
* Set default username source to nickname
* Add automatic oauth2 scopes for github and google
* Add hint to change the openid connect scopes if fields are missing
* Extend info about auto linking security risk
Co-authored-by: Viktor Kuzmin <kvaster@gmail.com>
Signed-off-by: Martin Michaelis <code@mgjm.de>
The Session table must have an Expiry field not a created_unix field - somehow
this migration adds the incorrect named field leading to #15445 reports.
Fix#15445
Signed-off-by: Andrew Thornton <art27@cantab.net>
The issue is that the TestPatch will reset the PR MergeBase - and it is possible for TestPatch to update the MergeBase whilst a merge is ongoing. The ensuing merge will then complete but it doesn't re-set the MergeBase it used to merge the PR.
Fixes the intermittent error in git test.
Signed-off-by: Andrew Thornton art27@cantab.net
* _ to unused func options
* rm useless brakets
* rm trifial non used models functions
* rm dead code
* rm dead global vars
* fix routers/api/v1/repo/issue.go
* dont overload import module
* Implemented LFS client.
* Implemented scanning for pointer files.
* Implemented downloading of lfs files.
* Moved model-dependent code into services.
* Removed models dependency. Added TryReadPointerFromBuffer.
* Migrated code from service to module.
* Centralised storage creation.
* Removed dependency from models.
* Moved ContentStore into modules.
* Share structs between server and client.
* Moved method to services.
* Implemented lfs download on clone.
* Implemented LFS sync on clone and mirror update.
* Added form fields.
* Updated templates.
* Fixed condition.
* Use alternate endpoint.
* Added missing methods.
* Fixed typo and make linter happy.
* Detached pointer parser from gogit dependency.
* Fixed TestGetLFSRange test.
* Added context to support cancellation.
* Use ReadFull to probably read more data.
* Removed duplicated code from models.
* Moved scan implementation into pointer_scanner_nogogit.
* Changed method name.
* Added comments.
* Added more/specific log/error messages.
* Embedded lfs.Pointer into models.LFSMetaObject.
* Moved code from models to module.
* Moved code from models to module.
* Moved code from models to module.
* Reduced pointer usage.
* Embedded type.
* Use promoted fields.
* Fixed unexpected eof.
* Added unit tests.
* Implemented migration of local file paths.
* Show an error on invalid LFS endpoints.
* Hide settings if not used.
* Added LFS info to mirror struct.
* Fixed comment.
* Check LFS endpoint.
* Manage LFS settings from mirror page.
* Fixed selector.
* Adjusted selector.
* Added more tests.
* Added local filesystem migration test.
* Fixed typo.
* Reset settings.
* Added special windows path handling.
* Added unit test for HTTPClient.
* Added unit test for BasicTransferAdapter.
* Moved into util package.
* Test if LFS endpoint is allowed.
* Added support for git://
* Just use a static placeholder as the displayed url may be invalid.
* Reverted to original code.
* Added "Advanced Settings".
* Updated wording.
* Added discovery info link.
* Implemented suggestion.
* Fixed missing format parameter.
* Added Pointer.IsValid().
* Always remove model on error.
* Added suggestions.
* Use channel instead of array.
* Update routers/repo/migrate.go
* fmt
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: zeripath <art27@cantab.net>
* Unexport SendUserMail
* Instead of "[]*models.User" or "[]string" lists infent "[]*MailRecipient" for mailer
* adopt
* code format
* TODOs for "i18n"
* clean
* no fallback for lang -> just use english
* lint
* exec testComposeIssueCommentMessage per lang and use only emails
* rm MailRecipient
* Dont reload from users from db if you alredy have in ram
* nits
* minimize diff
Signed-off-by: 6543 <6543@obermui.de>
* localize subjects
* linter ...
* Tr extend
* start tmpl edit ...
* Apply suggestions from code review
* use translation.Locale
* improve mailIssueCommentBatch
Signed-off-by: Andrew Thornton <art27@cantab.net>
* add i18n to datas
Signed-off-by: Andrew Thornton <art27@cantab.net>
* a comment
Co-authored-by: Andrew Thornton <art27@cantab.net>
/api/v1/repos/issues/search is a highly inefficient search which is unfortunately
the basis for our dependency searching algorithm. In particular it currently loads
all of the repositories and their owners and their primary coding language all of
which is immediately thrown away.
This PR makes one simple change - just get the IDs.
Related #14560
Related #12827
Signed-off-by: Andrew Thornton <art27@cantab.net>
git gc cron could change the size of the repository therefore we should update the
size of the repo stored in our database.
Also significantly improve the efficiency of counting lfs associated with the
repository
* Create Proper Migration tests
Unfortunately our testing regime has so far meant that migrations do not
get proper testing.
This PR begins the process of creating migration tests for this.
* Add test for v176
* fix mssql drop db
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Fix Migration 176 yet again
Whilst creating a test for v176 in the migrations_test PR
it has become clear that this was still wrong.
This is now fixed. Genuinely.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* and fix repo transfer
Signed-off-by: Andrew Thornton <art27@cantab.net>
There is a serious issue with the v176 migration where there is a mistaken missing
label_id selection.
*introduced by #14912*
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Never add labels not from this repository or organisation and remove org labels on transfer
Prevent the addition of labels from outside of the repository or
organisation and remove organisation labels on transfer.
Related #14908
Signed-off-by: Andrew Thornton <art27@cantab.net>
* switch to use sql
Signed-off-by: Andrew Thornton <art27@cantab.net>
* remove AS
Signed-off-by: Andrew Thornton <art27@cantab.net>
* subquery alias
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Give me some AS?
Signed-off-by: Andrew Thornton <art27@cantab.net>
* double AS
Signed-off-by: Andrew Thornton <art27@cantab.net>
* try try again
Signed-off-by: Andrew Thornton <art27@cantab.net>
* once more around the merry go round
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fix api problem
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Add outside label consistency check into doctor
This PR adds another consistency check into doctor in order to detect
labels that have been added from outside of repositories and organisations
Fix#14908
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fix migration
Signed-off-by: Andrew Thornton <art27@cantab.net>
* prep for merge
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lauris BH <lauris@nix.lv>
* Fix postgres ID sequences broken by recreate-table
Unfortunately there is a subtle problem with recreatetable on postgres which
leads to the sequences not being renamed and not being left at 0.
Fix#14725
Signed-off-by: Andrew Thornton <art27@cantab.net>
* let us try information_schema instead
Signed-off-by: Andrew Thornton <art27@cantab.net>
* try again
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: 6543 <6543@obermui.de>
* Ensure validation occurs on clone addresses too
Fix#14984
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fix lint
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fix test
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Fix api tests
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* chore: rewrite format.
* chore: update format
Signed-off-by: Bo-Yi Wu <appleboy.tw@gmail.com>
* chore: update format
Signed-off-by: Bo-Yi Wu <appleboy.tw@gmail.com>
* chore: Adjacent parameters with the same type should be grouped together
* chore: update format.
* API: fix set milestone on PR creation
pr creation via API failed with 404, because we searched
for milestoneID 0, due to uninitialized var usage D:
* add tests
* fix expected status codes
* fix tests
Co-authored-by: 6543 <6543@obermui.de>
* Never add labels not from this repository or organisation and remove org labels on transfer
Prevent the addition of labels from outside of the repository or
organisation and remove organisation labels on transfer.
Related #14908
* switch to use sql
* subquery alias
* once more around the merry go round
* fix api problem
Closed milestones and issues should only be marked overdue if they were
closed after their deadline.
Fix: #14536
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Make auto check manual merge as a chooseable mod and add manual merge way on ui
as title, Before this pr, we use same way with GH to check manually merge.
It good, but in some special cases, misjudgments can occur. and it's hard
to fix this bug. So I add option to allow repo manager block "auto check manual merge"
function, Then it will have same style like gitlab(allow empty pr). and to compensate for
not being able to detect THE PR merge automatically, I added a manual approach.
Signed-off-by: a1012112796 <1012112796@qq.com>
* make swager
* api support
* ping ci
* fix TestPullCreate_EmptyChangesWithCommits
* Apply suggestions from code review
Co-authored-by: zeripath <art27@cantab.net>
* Apply review suggestions and add test
* Apply suggestions from code review
Co-authored-by: zeripath <art27@cantab.net>
* fix build
* test error message
* make fmt
* Fix indentation issues identified by @silverwind
Co-authored-by: silverwind <me@silverwind.io>
* Fix tests and make manually merged disabled error on API the same
Signed-off-by: Andrew Thornton <art27@cantab.net>
* a small nit
* fix wrong commit id error
* fix bug
* simple test
* fix test
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Most DBs apart from SQLite will use a default Collation that is not case insensitive.
This means that SearchIssuesByKeyword becomes case sensitive for db indexing - in
contrast to the bleve and elastic indexers.
This PR simply uses UPPER(...) to do the LIKE - and although it may be more efficient
to change collations this would be a non-trivial task.
Fix#13663
Signed-off-by: Andrew Thornton <art27@cantab.net>
* make repo as "pending transfer" if on transfer start doer has no right to create repo in new destination
* if new pending transfer ocured, create UI & Mail notifications
Instead of causing a log.Fatal, we should handle broken OAuth2
providers by disabling them.
Fix#8930
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Create Xorm session provider
This PR creates a Xorm session provider which creates
the appropriate Session table for macaron/session.
Fix#7137
Signed-off-by: Andrew Thornton <art27@cantab.net>
* extraneous l
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fix lint
Signed-off-by: Andrew Thornton <art27@cantab.net>
* use key instead of ID to be compatible with go-macaron/session
Signed-off-by: Andrew Thornton <art27@cantab.net>
* And change the migration too.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Update spacing of imports
Co-authored-by: 6543 <6543@obermui.de>
* Update modules/session/xorm.go
Co-authored-by: techknowlogick <matti@mdranta.net>
* add xorm provider to the virtual provider
Signed-off-by: Andrew Thornton <art27@cantab.net>
* prep for master merge
* prep for merge master
* As per @lunny
* move migration out of the way
* Move to call this db session as per @lunny
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: techknowlogick <matti@mdranta.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Prevent adding nil label to .AddedLabels or .RemovedLabels
There are possibly a few old databases out there with malmigrated data that can
cause panics with empty labels being migrated.
This PR adds a few tests to prevent nil labels being added.
Fix#14466
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Add doctor command to remove the broken label comments
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
* Fix GPG key deletion when user is deleted
Per #14531, deleting a user account will delete the user's GPG keys
from the `gpg_key` table but not from `gpg_key_import`, which causes
an error when creating an account with the same email and attempting
to re-add the same key. This commit deletes all entries from
`gpg_key_import` that match any GPG key IDs belonging to the user.
* Format added code in models/user.go
* Create a new function for listing GPG keys and apply it
Create a new function `listGPGKeys` and replace a previous use
of `ListGPGKeys`. Thanks to @6543 for the patch.
Co-authored-by: Anton Khimich <anton.khimicha@mail.utoronto.ca>
Co-authored-by: 6543 <6543@obermui.de>
Migrations currently uses the default Xorm mapper which is
not the same as the mapper Gitea actually uses.
This means that there is a difference between the struct
parsing and mapping to database tables in migrations as
compared to normal Sync2.
This was the cause for the catastrophic problem in v168 -
untagged fields are not mapped in the same way in migrations
as compared to outside of migrations.
This is also likely the cause of some weird subtle failures
in other migrations as any untagged field may not be being
mapped exactly the same way.
This PR suggests that we ensure that the mapper is set at
the start of the migrations code - but also enforces a strict
clean mapper between each migration.
Signed-off-by: Andrew Thornton <art27@cantab.net>
