There are multiple places where Gitea does not properly escape URLs that it is building and there are multiple places where it builds urls when there is already a simpler function available to use this.
This is an extensive PR attempting to fix these issues.
1. The first commit in this PR looks through all href, src and links in the Gitea codebase and has attempted to catch all the places where there is potentially incomplete escaping.
2. Whilst doing this we will prefer to use functions that create URLs over recreating them by hand.
3. All uses of strings should be directly escaped - even if they are not currently expected to contain escaping characters. The main benefit to doing this will be that we can consider relaxing the constraints on user names and reponames in future.
4. The next commit looks at escaping in the wiki and re-considers the urls that are used there. Using the improved escaping here wiki files containing '/'. (This implementation will currently still place all of the wiki files the root directory of the repo but this would not be difficult to change.)
5. The title generation in feeds is now properly escaped.
6. EscapePound is no longer needed - urls should be PathEscaped / QueryEscaped as necessary but then re-escaped with Escape when creating html with locales Signed-off-by: Andrew Thornton <art27@cantab.net>
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Apache `ProxyPassReverse` only works for Location, Content-Location and URI headers on HTTP redirect responses, it causes more problems than it resolves. Now all URLs generated by Gitee have the correct prefix AppSubURL. We do not need to set `ProxyPassReverse`.
* fix url param
* use AppSubURL instead of AppURL in api/v1
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Add info about list endpoints to CONTRIBUTING.md
* Let all list endpoints return X-Total-Count header
* Add TODOs for GetCombinedCommitStatusByRef
* Fix models/issue_stopwatch.go
* Rrefactor models.ListDeployKeys
* Introduce helper func and use them for SetLinkHeader related func
Followup from #16562 prepare for #16567
* Rename ctx.Form() to ctx.FormString()
* Reimplement FormX func to need less code and cpu cycles
* Move code into own file
You can limit or hide organisations. This pull make it also posible for users
- new strings to translte
- add checkbox to user profile form
- add checkbox to admin user.edit form
- filter explore page user search
- filter api admin and public user searches
- allow admins view "hidden" users
- add app option DEFAULT_USER_VISIBILITY
- rewrite many files to use Visibility field
- check for teams intersection
- fix context output
- right fake 404 if not visible
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Andrew Thornton <art27@cantab.net>
* _ to unused func options
* rm useless brakets
* rm trifial non used models functions
* rm dead code
* rm dead global vars
* fix routers/api/v1/repo/issue.go
* dont overload import module
Use [chi](https://github.com/go-chi/chi) instead of the forked [macaron](https://gitea.com/macaron/macaron). Since macaron and chi have conflicts with session share, this big PR becomes a have-to thing. According my previous idea, we can replace macaron step by step but I'm wrong. :( Below is a list of big changes on this PR.
- [x] Define `context.ResponseWriter` interface with an implementation `context.Response`.
- [x] Use chi instead of macaron, and also a customize `Route` to wrap chi so that the router usage is similar as before.
- [x] Create different routers for `web`, `api`, `internal` and `install` so that the codes will be more clear and no magic .
- [x] Use https://github.com/unrolled/render instead of macaron's internal render
- [x] Use https://github.com/NYTimes/gziphandler instead of https://gitea.com/macaron/gzip
- [x] Use https://gitea.com/go-chi/session which is a modified version of https://gitea.com/macaron/session and removed `nodb` support since it will not be maintained. **BREAK**
- [x] Use https://gitea.com/go-chi/captcha which is a modified version of https://gitea.com/macaron/captcha
- [x] Use https://gitea.com/go-chi/cache which is a modified version of https://gitea.com/macaron/cache
- [x] Use https://gitea.com/go-chi/binding which is a modified version of https://gitea.com/macaron/binding
- [x] Use https://github.com/go-chi/cors instead of https://gitea.com/macaron/cors
- [x] Dropped https://gitea.com/macaron/i18n and make a new one in `code.gitea.io/gitea/modules/translation`
- [x] Move validation form structs from `code.gitea.io/gitea/modules/auth` to `code.gitea.io/gitea/modules/forms` to avoid dependency cycle.
- [x] Removed macaron log service because it's not need any more. **BREAK**
- [x] All form structs have to be get by `web.GetForm(ctx)` in the route function but not as a function parameter on routes definition.
- [x] Move Git HTTP protocol implementation to use routers directly.
- [x] Fix the problem that chi routes don't support trailing slash but macaron did.
- [x] `/api/v1/swagger` now will be redirect to `/api/swagger` but not render directly so that `APIContext` will not create a html render.
Notices:
- Chi router don't support request with trailing slash
- Integration test `TestUserHeatmap` maybe mysql version related. It's failed on my macOS(mysql 5.7.29 installed via brew) but succeed on CI.
Co-authored-by: 6543 <6543@obermui.de>
* Add organization wide labels
Implement organization wide labels similar to organization wide
webhooks. This lets you create individual labels for organizations that can be used
for all repos under that organization (so being able to reuse the same
label across multiple repos).
This makes it possible for small organizations with many repos to use
labels effectively.
Fixes #7406
* Add migration
* remove comments
* fix tests
* Update options/locale/locale_en-US.ini
Removed unused translation string
* show org labels in issue search label filter
* Use more clear var name
* rename migration after merge from master
* comment typo
* update migration again after rebase with master
* check for orgID <=0 per guillep2k review
* fmt
* Apply suggestions from code review
Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com>
* remove unused code
* Make sure RepoID is 0 when searching orgID per code review
* more changes/code review requests
* More descriptive translation var per code review
* func description/delete comment when issue label deleted instead of hiding it
* remove comment
* only use issues in that repo when calculating number of open issues for org label on repo label page
* Add integration test for IssuesSearch API with labels
* remove unused function
* Update models/issue_label.go
Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com>
* Use subquery in GetLabelIDsInReposByNames
* Fix tests to use correct orgID
* fix more tests
* IssuesSearch api now uses new BuildLabelNamesIssueIDsCondition. Add a few more tests as well
* update comment for clarity
* Revert previous code change now that we can use the new BuildLabelNamesIssueIDsCondition
* Don't sort repos by date in IssuesSearch API
After much debugging I've found a strange issue where in some cases MySQL will return a different result than other enigines if a query is sorted by a null collumn. For example with our integration test data where we don't set updated_unix in repository fixtures:
SELECT `id`, `owner_id`, `owner_name`, `lower_name`, `name`, `description`, `website`, `original_service_type`, `original_url`, `default_branch`, `num_watches`, `num_stars`, `num_forks`, `num_issues`, `num_closed_issues`, `num_pulls`, `num_closed_pulls`, `num_milestones`, `num_closed_milestones`, `is_private`, `is_empty`, `is_archived`, `is_mirror`, `status`, `is_fork`, `fork_id`, `is_template`, `template_id`, `size`, `is_fsck_enabled`, `close_issues_via_commit_in_any_branch`, `topics`, `avatar`, `created_unix`, `updated_unix` FROM `repository` ORDER BY updated_unix DESC LIMIT 15 OFFSET 45
Returns different results for MySQL than other engines. However, the similar query:
SELECT `id`, `owner_id`, `owner_name`, `lower_name`, `name`, `description`, `website`, `original_service_type`, `original_url`, `default_branch`, `num_watches`, `num_stars`, `num_forks`, `num_issues`, `num_closed_issues`, `num_pulls`, `num_closed_pulls`, `num_milestones`, `num_closed_milestones`, `is_private`, `is_empty`, `is_archived`, `is_mirror`, `status`, `is_fork`, `fork_id`, `is_template`, `template_id`, `size`, `is_fsck_enabled`, `close_issues_via_commit_in_any_branch`, `topics`, `avatar`, `created_unix`, `updated_unix` FROM `repository` ORDER BY updated_unix DESC LIMIT 15 OFFSET 30
Returns the same results.
This causes integration tests to fail on MySQL in certain cases but would never show up in a real installation. Since this API call always returns issues based on the optionally provided repo_priority_id or the issueID itself, there is no change to results by changing the repo sorting method used to get ids earlier in the function.
* linter is back!
* code review
* remove now unused option
* Fix newline at end of files
* more unused code
* update to master
* check for matching ids before query
* Update models/issue_label.go
Co-Authored-By: 6543 <6543@obermui.de>
* Update models/issue_label.go
* update comments
* Update routers/org/setting.go
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
Co-authored-by: 6543 <6543@obermui.de>
* introduce `GET /orgs`
* add TEST
* show also other VisibleType's
* update description
* refactor a lot
* SearchUserOptions by default return only public
* API: orgEditTeam make Fields optional
* add TestCase
* Update integrations/api_team_test.go
* suggestions from lafriks
use len() to check if string is empty
Co-Authored-By: Lauris BH <lauris@nix.lv>
* change ...
* use Where not ID to get mssql
* add return and code format
* fix test
* fix test ... null pointer exept
* update specific colums
* only specific colums too
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* use numbers and not http.Status___ enum
* fix test
* add many missing swagger responses
* code format
* Deletion Sould return 204 ...
* error handling improvements
* if special error type ... then add it to swagger too
* one smal nit
* invalidTopicsError is []string
* valid swagger specification 2.0
- if you add responses swagger can tell you if you do it right 👍
* use ctx.InternalServerError
* Revert "use numbers and not http.Status___ enum"
This reverts commit b1ff386e2418ed6a7f183e756b13277d701278ef.
* use http.Status* enum everywhere
* Add team permission setting to allow creating repo in organization.
Signed-off-by: David Svantesson <davidsvantesson@gmail.com>
* Add test case for creating repo when have team creation access.
Signed-off-by: David Svantesson <davidsvantesson@gmail.com>
* build error: should omit comparison to bool constant
Signed-off-by: David Svantesson <davidsvantesson@gmail.com>
* Add comment on exported functions
* Fix fixture consistency, fix existing unit tests
* Fix boolean comparison in xorm query.
* addCollaborator and changeCollaborationAccessMode separate steps
More clear to use different if-cases.
* Create and commit xorm session
* fix
* Add information of create repo permission in team sidebar
* Add migration step
* Clarify that repository creator will be administrator.
* Fix some things after merge
* Fix language text that use html
* migrations file
* Create repository permission -> Create repositories
* fix merge
* fix review comments
* Add field IsAllRepositories to team
* Add AllRepositories to team UI
* Manage team with access to all repositories
* Add field IsAllRepositories to team API
* put backticks around table/column names
* rename IsAllRepositories to IncludesAllRepositories
* do not reload slice if already loaded
* add repo to teams with access to all repositories when changing repo owner
* improve tests for teams with access to all repositories
* Merge branch 'master'
* Change code for adding all repositories
Signed-off-by: David Svantesson <davidsvantesson@gmail.com>
* fmt after merge
* Change code in API EditTeam similar to EditTeamPost web interface
Signed-off-by: David Svantesson <davidsvantesson@gmail.com>
* Clarify that all repositories will be added
Signed-off-by: David Svantesson <davidsvantesson@gmail.com>
* All repositories option under Permissions headline
* New setting group 'Repository access'
* Move check IncludeAllRepositories to removeRepository.
* Revert "Move check IncludeAllRepositories to removeRepository." and add comment instead.
This reverts commit 753b7d205be260b8be465b5291a02975a81f3093.
* Clarify help text what options do.
* Api endpoint for searching teams.
Signed-off-by: dasv <david.svantesson@qrtech.se>
* Move API to /orgs/:org/teams/search
Signed-off-by: David Svantesson <davidsvantesson@gmail.com>
* Regenerate swagger
Signed-off-by: David Svantesson <davidsvantesson@gmail.com>
* Fix search is Get
Signed-off-by: David Svantesson <davidsvantesson@gmail.com>
* Add test for search team API.
Signed-off-by: David Svantesson <davidsvantesson@gmail.com>
* Update routers/api/v1/org/team.go
grammar
Co-Authored-By: Richard Mahn <richmahn@users.noreply.github.com>
* Fix review comments
Signed-off-by: David Svantesson <davidsvantesson@gmail.com>
* Fix some issues in repo collaboration team search, after changes in this PR.
Signed-off-by: David Svantesson <davidsvantesson@gmail.com>
* Remove teamUser which is not used and replace with actual user id.
Signed-off-by: David Svantesson <davidsvantesson@gmail.com>
* Remove unused search variable UserIsAdmin.
* Add paging to team search.
* Re-genereate swagger
Signed-off-by: David Svantesson <davidsvantesson@gmail.com>
* Fix review comments
Signed-off-by: David Svantesson <davidsvantesson@gmail.com>
* fix
* Regenerate swagger
* Add teams to repo on collaboration page.
Signed-off-by: David Svantesson <davidsvantesson@gmail.com>
* Add option for repository admins to change teams access to repo.
Signed-off-by: David Svantesson <davidsvantesson@gmail.com>
* Add comment for functions
Signed-off-by: David Svantesson <davidsvantesson@gmail.com>
* Make RepoAdminChangeTeamAccess default false in xorm and make it default checked in template instead.
Signed-off-by: David Svantesson <davidsvantesson@gmail.com>
* Make proper language strings and fix error redirection.
* Add unit tests for adding and deleting team from repository.
Signed-off-by: David Svantesson <davidsvantesson@gmail.com>
* Add database migration
Signed-off-by: David Svantesson <davidsvantesson@gmail.com>
* Fix redirect
Signed-off-by: David Svantesson <davidsvantesson@gmail.com>
* Fix locale string mismatch.
Signed-off-by: David Svantesson <davidsvantesson@gmail.com>
* Move team access mode text logic to template.
* Move collaborator access mode text logic to template.
when you use gitea as OAuth2 provider, the /api/v1/user should return
user primary email as identifier, which is unique in OAuth2 clients.
this patch use convert.ToUser replace all u.APIFormat in api requests,
return primary email when caller is yourself or admin.
* api: Add an endpoint to list a particular member of team.
* models: Rename `GetUserTeams()` to `GetUserOrgTeams()` in `org_team` model.
`GetUserTeams()` sounds a bit misnomer since it actually returns
the teams that user belongs to in a given organization rather than
all the teams across all the organization that the user has joined.
* models: Add `GetUserTeams()`.
Returns all the teams that a user belongs to.
* api: Add an endpoint for GET '/user/teams'.
A GET request to this endpoint lists all the teams that a user
belongs to.
