There are multiple places where Gitea does not properly escape URLs that it is building and there are multiple places where it builds urls when there is already a simpler function available to use this.
This is an extensive PR attempting to fix these issues.
1. The first commit in this PR looks through all href, src and links in the Gitea codebase and has attempted to catch all the places where there is potentially incomplete escaping.
2. Whilst doing this we will prefer to use functions that create URLs over recreating them by hand.
3. All uses of strings should be directly escaped - even if they are not currently expected to contain escaping characters. The main benefit to doing this will be that we can consider relaxing the constraints on user names and reponames in future.
4. The next commit looks at escaping in the wiki and re-considers the urls that are used there. Using the improved escaping here wiki files containing '/'. (This implementation will currently still place all of the wiki files the root directory of the repo but this would not be difficult to change.)
5. The title generation in feeds is now properly escaped.
6. EscapePound is no longer needed - urls should be PathEscaped / QueryEscaped as necessary but then re-escaped with Escape when creating html with locales Signed-off-by: Andrew Thornton <art27@cantab.net>
Signed-off-by: Andrew Thornton <art27@cantab.net>
- Partialy resolvess #17596
- Resolves `badCall` errors from go-critic `badCall: suspicious Join on
1 argument`
- When only 1 argument is passed into `filepath.Join`, it won't do
anything special other than `filepath.Clean(...)` will be applied over
it.
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: 6543 <6543@obermui.de>
* Correctly handle failed migrations
There is a bug in handling failed migrations whereby the migration task gets decoupled
from the migration repository. This leads to a failure of the task to get deleted with
the repository and also leads to the migration failed page resulting in a ISE.
This PR removes the zeroing out of the task id from the migration but also makes
the migration handler tolerate missing tasks much nicer.
Fix #17571
Signed-off-by: Andrew Thornton <art27@cantab.net>
* feat: Allow multiple tags on comments
- Allow for multiples tags(Currently Poster + {Owner, Writer}).
- Utilize the Poster tag within the commentTag function and remove the
checking from templates.
- Use bitwise on CommentTags to enable specific tags.
- Don't show poster tag(view_content.tmpl) on the initial issue comment.
* Change parameters naming
* Change function name
* refactor variable wording
* Merge 'master' branch into 'tags-comments' branch
* Change naming
* `tag` -> `role`
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Fix 500 when a comment was deleted which has a notification
* Tolerate missing Comment in other places too
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
This change enables the usage of U2F without being forced to enroll an TOTP authenticator.
The `/user/auth/u2f` has been changed to hide the "use TOTP instead" bar if TOTP is not enrolled.
Fixes #5410
Fixes #17495
* Hide label comments if labels were added and removed immediately
* Add comment and rename var
* Fix unit test
* Add test case
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
* Simplify Gothic to use our session store instead of creating a different store
We have been using xormstore to provide a separate session store for our OAuth2 logins
however, this relies on using gorilla context and some doubling of our session storing.
We can however, simplify and simply use our own chi-based session store. Thus removing
a cookie and some of the weirdness with missing contexts.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* as per review
Signed-off-by: Andrew Thornton <art27@cantab.net>
* as per review
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Handle MaxTokenLength
Signed-off-by: Andrew Thornton <art27@cantab.net>
* oops
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: Lauris BH <lauris@nix.lv>
The call to html.EscapeString in routers/web/repo/blame.go:renderBlame is extraneous
as the commit message is now rendered by the template. The template will correctly
escape strings - therefore we are currently double escaping.
This PR fixes this.
Fix #17492
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Fixes #16558 CSV delimiter determiner
* Fixes #16558 - properly determine CSV delmiiter
* Moves quoteString to a new function
* Adds big test with lots of commas for tab delimited csv
* Adds comments
* Shortens the text of the test
* Removes single quotes from regexp as only double quotes need to be searched
* Fixes spelling
* Fixes check of length as it probalby will only be 1e4, not greater
* Makes sample size a const, properly removes truncated line
* Makes sample size a const, properly removes truncated line
* Fixes comment
* Fixes comment
* tests for FormatError() function
* Adds logic to find the limiter before or after a quoted value
* Simplifies regex
* Error tests
* Error tests
* Update modules/csv/csv.go
Co-authored-by: delvh <dev.lh@web.de>
* Update modules/csv/csv.go
Co-authored-by: delvh <dev.lh@web.de>
* Adds comments
* Update modules/csv/csv.go
Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: delvh <dev.lh@web.de>
* Run Migrate in Install rather than just SyncTables
The underlying problem in #17328 appears to be that users are re-running the install
page during upgrades. The function that tests and creates the db did not intend for
this and thus instead the migration scripts being run - a simple sync tables occurs.
This then causes a weird partially migrated DB which causes, in this release cycle,
the duplicate column in task table error. It is likely the cause of some weird
partial migration errors in other cycles too.
This PR simply ensures that the migration scripts are also run at this point too.
Fix #17328
Signed-off-by: Andrew Thornton <art27@cantab.net>
There was a recent spelling mistake added to the locale file where stared was used
instead of starred.
This PR changes this to starred.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
There is a small bug in the way that repo access is checked in
repoAssignment: Accessibility is checked by checking if the user has a
marked access to the repository instead of checking if the user has any
team granted access.
This PR changes this permissions check to use HasAccess() which does the
correct test. There is also a fix in the release api ListReleases where
it should return draft releases if the user is a member of a team with
write access to the releases.
The PR also adds a testcase.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Unfortunately there was a regression in #17373 which missed that the user is not
for deploy keys. This leads to a panic when pushing with deploy keys.
Fix #17412
Signed-off-by: Andrew Thornton <art27@cantab.net>
closed #17378
Both errors from #17378 were caused by #15175.
Problem 1 (error with added file):
`ToUTF8WithFallbackReader` creates a `MultiReader` from a `byte[2048]` and the remaining reader. `CreateReaderAndGuessDelimiter` tries to read 10000 bytes from this reader but only gets 2048 because that's the first reader in the `MultiReader`. Then the `if size < 1e4` thinks the input is at EOF and just returns that.
Problem 2 (error with changed file):
The blob reader gets defer closed. That was fine because the old version reads the whole file into memory. Now with the streaming version the close needs to defer after the method.
* Improve: make diff result better, make the HTML element fit the full height in the content history diff dialog
* Bug fix: when edit the main issue, the poster is wrongly set to the issue poster
* Add groups scope/claim to OICD/OAuth2
Add support for groups claim as part of the OIDC/OAuth2 flow.
Groups is a list of "org" and "org:team" strings to allow clients to
authorize based on the groups a user is part of.
Signed-off-by: Nico Schieder <code@nico-schieder.de>
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
We have the `AppState` module now, it can store app related data easily. We do not need to create separate tables for each feature.
So the update checker can use `AppState` instead of a one-row dedicate table.
And the code of update checker is moved from `models` to `modules`.
Gitea writes its own AppPath into git hook scripts. If Gitea's AppPath changes, then the git push will fail.
This PR:
* Introduce an AppState module, it can persist app states into database
* During GlobalInit, Gitea will check if the current AppPath is the same as last one. If they don't match, Gitea will sync git hooks.
* Refactor some code to make them more clear.
* Also, "Detect if gitea binary's name changed" #11341 is related, we call models.RewriteAllPublicKeys to update ssh authorized_keys file
Repositories owned by private users and organisations and pulls by restricted users
need to have permissions checked. Previously Serv would simply assumed that if the
user could log in and the repository was not private then it would be visible.
Fix #17364
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Unfortunately #17301 broke the restriction of the dashboard repolist to
the user's repos because it stopped passing in the uid for the current
user. This PR restores this.
There is also a weird alignment problem - not caused by that PR - where
the menu items in the repolist spread over multiple lines. This PR
simply reduces the padding on these items and switches the justification
of the flex elements to space-evenly.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Apache `ProxyPassReverse` only works for Location, Content-Location and URI headers on HTTP redirect responses, it causes more problems than it resolves. Now all URLs generated by Gitee have the correct prefix AppSubURL. We do not need to set `ProxyPassReverse`.
* fix url param
* use AppSubURL instead of AppURL in api/v1
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Don't panic if we fail to parse a U2FRegistration data
Downgrade logging statement from Fatal to Error so that errors parsing
U2FRegistration data does not panic; instead, the invalid key will be
skipped and we will attempt to parse the next one, if available.
Signed-off-by: David Jimenez <dvejmz@sgfault.com>
It makes Admin's life easier to filter users by various status.
* introduce window.config.PageData to pass template data to javascript module and small refactor
move legacy window.ActivityTopAuthors to window.config.PageData.ActivityTopAuthors
make HTML structure more IDE-friendly in footer.tmpl and head.tmpl
remove incorrect <style class="list-search-style"></style> in head.tmpl
use log.Error instead of log.Critical in admin user search
* use LEFT JOIN instead of SubQuery when admin filters users by 2fa. revert non-en locale.
* use OptionalBool instead of status map
* refactor SearchUserOptions.toConds to SearchUserOptions.toSearchQueryBase
* add unit test for user search
* only allow admin to use filters to search users
* issue content history
* Use timeutil.TimeStampNow() for content history time instead of issue/comment.UpdatedUnix (which are not updated in time)
* i18n for frontend
* refactor
* clean up
* fix refactor
* re-format
* temp refactor
* follow db refactor
* rename IssueContentHistory to ContentHistory, remove empty model tags
* fix html
* use avatar refactor to generate avatar url
* add unit test, keep at most 20 history revisions.
* re-format
* syntax nit
* Add issue content history table
* Update models/migrations/v197.go
Co-authored-by: 6543 <6543@obermui.de>
* fix merge
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lauris BH <lauris@nix.lv>
- Update default branch if needed
- Update protected branch if needed
- Update all not merged pull request base branch name
- Rename git branch
- Record this rename work and auto redirect for old branch on ui
Signed-off-by: a1012112796 <1012112796@qq.com>
Co-authored-by: delvh <dev.lh@web.de>
One of the biggest reasons for slow repository browsing is that we wait
until last commit information has been generated for all files in the
repository.
This PR proposes deferring this generation to a new POST endpoint that
does the look up outside of the main page request.
Signed-off-by: Andrew Thornton <art27@cantab.net>
close #17181
* for all pull requests API return permissions of caller
* for all webhook return empty permissions
Signed-off-by: Danila Kryukov <pricly_yellow@dismail.de>
Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: 6543 <6543@obermui.de>
Why this refactor
The goal is to move most files from `models` package to `models.xxx` package. Many models depend on avatar model, so just move this first.
And the existing logic is not clear, there are too many function like `AvatarLink`, `RelAvatarLink`, `SizedRelAvatarLink`, `SizedAvatarLink`, `MakeFinalAvatarURL`, `HashedAvatarLink`, etc. This refactor make everything clear:
* user.AvatarLink()
* user.AvatarLinkWithSize(size)
* avatars.GenerateEmailAvatarFastLink(email, size)
* avatars.GenerateEmailAvatarFinalLink(email, size)
And many duplicated code are deleted in route handler, the handler and the model share the same avatar logic now.
* api: dont open merged PRs
* don't change base branch when already merged
* don't allow any state change
* also validate opening merged PRs in EditIssue
Co-authored-by: John Olheiser <john.olheiser@gmail.com>
Co-authored-by: 6543 <6543@obermui.de>
* Prevent NPE on invalid diff
If ParseCompareInfo returns a nil compare info the defer function needs to ensure
that it does not attempt to close the HeadGitRepo.
Fix #17193
Signed-off-by: Andrew Thornton <art27@cantab.net>
* add TEST
Co-authored-by: 6543 <6543@obermui.de>
Nodeinfo is a way to expose certain metadata about a server for use of discovery regarding functionality of its federation capabilities.
Two endpoints are required:
1. `/.well-known/nodeinfo` which informs client where it can find the location of the location of its metadata (including which version of the schema is used)
2. the endpoint which exposes the metadata in json format according to schema.
Notes:
* `openRegistrations` is a required field, but I propose to set to false as default in case someone writes a crawler to discover "open" gitea instances
* to limit data leakage I also propose to not include the `usage` field (note it is required so it should be included, but left as empty).
More info:
https://github.com/jhass/nodeinfohttps://github.com/jhass/nodeinfo/tree/main/schemas/2.1http://nodeinfo.diaspora.software/protocol.html
This PR changes the compare page to make the "..." in the between branches a clickable
link. This changes the comparison type from "..." to "..". Similarly it makes the
initial compare icon clickable to switch the head and base branches.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Allow LDAP Sources to provide Avatars
Add setting to LDAP source to allow it to provide an Avatar.
Currently this is required to point to the image bytes.
Fix #4144
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Rename as Avatar Attribute (drop JPEG)
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Always synchronize avatar if there is change
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Actually get the avatar from the ldap
Signed-off-by: Andrew Thornton <art27@cantab.net>
* clean-up
Signed-off-by: Andrew Thornton <art27@cantab.net>
* use len()>0 rather than != ""
Signed-off-by: Andrew Thornton <art27@cantab.net>
* slight shortcut in IsUploadAvatarChanged
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Add SkipLocal2FA option to other pam and smtp sources
Extend #16954 to allow setting skip local 2fa on pam and SMTP authentication sources
Signed-off-by: Andrew Thornton <art27@cantab.net>
* make SkipLocal2FA omitempty
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
* DBContext is just a Context
This PR removes some of the specialness from the DBContext and makes it context
This allows us to simplify the GetEngine code to wrap around any context in future
and means that we can change our loadRepo(e Engine) functions to simply take contexts.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fix unit tests
Signed-off-by: Andrew Thornton <art27@cantab.net>
* another place that needs to set the initial context
Signed-off-by: Andrew Thornton <art27@cantab.net>
* avoid race
Signed-off-by: Andrew Thornton <art27@cantab.net>
* change attachment error
Signed-off-by: Andrew Thornton <art27@cantab.net>
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Some people still appear to report unclosed cat-files. This PR simply adds the caller
to the process descriptor for the CatFileBatch and CatFileBatchCheck calls.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
This PR extends #16594 to allow LDAP to be able to be set to skip local 2FA too. The technique used here would be extensible to PAM and SMTP sources.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Clean-up HookPreReceive and restore functionality for pushing non-standard refs
There was an inadvertent breaking change in #15629 meaning that notes refs and other
git extension refs will be automatically rejected.
Further following #14295 and #15629 the pre-recieve hook code is untenably long and
too complex.
This PR refactors the hook code and removes the incorrect forced rejection of
non-standard refs.
Fix #16688
Signed-off-by: Andrew Thornton <art27@cantab.net>
When converting repositories from forks to normal the root NumFork needs to be
decremented too.
Fix #17026
Signed-off-by: Andrew Thornton <art27@cantab.net>
Often when handling problems it is not clear which configuration file Gitea is
using. This PR simply ensures that the configuration file is emitted.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Use common sessioner for API and Web
Instead of creating separate sessioner and doubly initialising the provider just
use the same sessioner for the API and Web routes.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Fixes #16381
Note that changes to unprotected files via the web editor still cannot be pushed directly to the protected branch. I could easily add such support for edits and deletes if needed. But for adding, uploading or renaming unprotected files, it is not trivial.
* Extract & Move GetAffectedFiles to modules/git
This PR adds a setting to OAuth and OpenID login sources to allow the source to
skip local 2FA authentication.
Fix #13939
Signed-off-by: Andrew Thornton <art27@cantab.net>
When create a new issue or comment and paste/upload an attachment/image, it will not assign an issue id before submit. So if user give up the creating, the attachments will lost key feature and become dirty content. We don't know if we need to delete the attachment even if the repository deleted.
This PR add a repo_id in attachment table so that even if a new upload attachment with no issue_id or release_id but should have repo_id. When deleting a repository, they could also be deleted.
Co-authored-by: 6543 <6543@obermui.de>
Rename`[repository]` `DISABLE_MIRRORS` to `[mirror]` `DISABLE_NEW_PULL` and add `ENABLED` and `DISABLE_NEW_PUSH` with the below meanings:
- `ENABLED`: **true**: Enables the mirror functionality. Set to **false** to disable all mirrors.
- `DISABLE_NEW_PULL`: **false**: Disable the creation of **new** mirrors. Pre-existing mirrors remain valid.
- `DISABLE_NEW_PUSH`: **false**: Disable the creation of **new** push mirrors. Pre-existing mirrors remain valid.
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: delvh <dev.lh@web.de>
* Add Cache-Control to avatar redirects
This does seem to do the trick to make the Avatar redirects cachable
in Chrome.
In Firefox, it does not seem to work, thought and I found no way to
suppress the requests to the original URLs, I even tried setting an
Etag to no avail.
Related discussion in https://github.com/go-gitea/gitea/issues/16964.
Co-authored-by: zeripath <art27@cantab.net>
PR #16125 did not update the error handlers to handle conflict errors relating
to rebases. This PR adds them.
Fix #16922
Signed-off-by: Andrew Thornton <art27@cantab.net>
Calculate and return the number of Repositories on the dashboard
Organization list.
This PR restores some of the logic that was removed in #14032 to
calculate the number of repos on the dashboard orgs list.
Fix #16648
Replaces #16799
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* make sure headGitRepo is closed on err too
* refactor
* Fix git.Blob.DataAsync(): exec cancel since we already read all bytes (close pipe since we return a NopCloser)
There are multiple places where wiki git repositories are not properly closed.
This PR ensures they are closed.
Signed-off-by: Andrew Thornton <art27@cantab.net>
