Commit graph

38 commits

Author SHA1 Message Date
wxiaoguang 722dab5286
Make HTML template functions support context (#24056)
# Background

Golang template is not friendly for large projects, and Golang template
team is quite slow, related:
* `https://github.com/golang/go/issues/54450`

Without upstream support, we can also have our solution to make HTML
template functions support context.

It helps a lot, the above Golang template issue `#54450` explains a lot:

1. It makes `{{Locale.Tr}}` could be used in any template, without
passing unclear `(dict "root" . )` anymore.
2. More and more functions need `context`, like `avatar`, etc, we do not
need to do `(dict "Context" $.Context)` anymore.
3. Many request-related functions could be shared by parent&children
templates, like "user setting" / "system setting"

See the test `TestScopedTemplateSetFuncMap`, one template set, two
`Execute` calls with different `CtxFunc`.

# The Solution

Instead of waiting for upstream, this PR re-uses the escaped HTML
template trees, use `AddParseTree` to add related templates/trees to a
new template instance, then the new template instance can have its own
FuncMap , the function calls in the template trees will always use the
new template's FuncMap.

`template.New` / `template.AddParseTree` / `adding-FuncMap` are all
quite fast, so the performance is not affected.

The details:

1. Make a new `html/template/Template` for `all` templates
2. Add template code to the `all` template
3. Freeze the `all` template, reset its exec func map, it shouldn't
execute any template.
4. When a router wants to render a template by its `name`
    1. Find the `name` in `all`
    2. Find all its related sub templates
3. Escape all related templates (just like what the html template
package does)
4. Add the escaped parse-trees of related templates into a new (scoped)
`text/template/Template`
    5. Add context-related func map into the new (scoped) text template
    6. Execute the new (scoped) text template
7. To improve performance, the escaped templates are cached to `template
sets`

# FAQ

## There is a `unsafe` call, is this PR unsafe?

This PR is safe. Golang has strict language definition, it's safe to do
so: https://pkg.go.dev/unsafe#Pointer (1) Conversion of a *T1 to Pointer
to *T2


## What if Golang template supports such feature in the future?

The public structs/interfaces/functions introduced by this PR is quite
simple, the code of `HTMLRender` is not changed too much. It's very easy
to switch to the official mechanism if there would be one.

## Does this PR change the template execution behavior?

No, see the tests (welcome to design more tests if it's necessary)

---------

Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: Jason Song <i@wolfogre.com>
Co-authored-by: Giteabot <teabot@gitea.io>
2023-04-20 04:08:58 -04:00
wxiaoguang 7681d582cd
Refactor locale number (#24134)
Before, the `GiteaLocaleNumber.js` was just written as a a drop-in
replacement for old `js-pretty-number`.

Actually, we can use Golang's `text` package to format.

This PR partially completes the TODOs in `GiteaLocaleNumber.js`:

> if we have complete backend locale support (eg: Golang "x/text"
package), we can drop this component.
> tooltip: only 2 usages of this, we can replace it with Golang's
"x/text/number" package in the future.

This PR also helps #24131

Screenshots:

<details>

![image](https://user-images.githubusercontent.com/2114189/232179420-b1b9974b-9d96-4408-b209-b80182c8b359.png)


![image](https://user-images.githubusercontent.com/2114189/232179416-14f36aa0-3f3e-4ac9-b366-7bd3a4464a11.png)

</details>
2023-04-17 11:37:23 +08:00
wxiaoguang fdbd646113
Group template helper functions, remove Printf, improve template error messages (#23982)
Follow #23328 


Major changes:

* Group the function in `templates/help.go` by their purposes. It could
make future work easier.
* Remove the `Printf` helper function, there is already a builtin
`printf`.
* Remove `DiffStatsWidth`, replace with `Eval` in template
* Rename the `NewTextFuncMap` to `mailSubjectTextFuncMap`, it's for
subject text template only, no need to make it support HTML functions.


----

And fine tune template error messages, to make it more friendly to
developers and users.


![image](https://user-images.githubusercontent.com/2114189/230714245-4fd202d1-2b25-41b2-8be5-03c5fee45091.png)


![image](https://user-images.githubusercontent.com/2114189/230714277-66783577-2a03-49d5-8e8c-ceba5e07a2d4.png)

---------

Co-authored-by: silverwind <me@silverwind.io>
2023-04-08 21:15:22 +08:00
wxiaoguang 8f00979f73
Drop "unrolled/render" package (#23965)
None of the features of `unrolled/render` package is used. 

The Golang builtin "html/template" just works well. Then we can improve
our HTML render to resolve the "$.root.locale.Tr" problem as much as
possible.

Next step: we can have a template render pool (by Clone), then we can
inject global functions with dynamic context to every `Execute` calls.
Then we can use `{{Locale.Tr ....}}` directly in all templates , no need
to pass the `$.root.locale` again and again.
2023-04-08 14:21:50 +08:00
yp05327 bd66fa586a
Rename repo.GetOwner to repo.LoadOwner (#22967)
Fixes https://github.com/go-gitea/gitea/issues/22963

---------

Co-authored-by: Yarden Shoham <hrsi88@gmail.com>
2023-02-18 20:11:03 +08:00
Lunny Xiao 0a7d3ff786
refactor some functions to support ctx as first parameter (#21878)
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
Co-authored-by: Lauris BH <lauris@nix.lv>
2022-12-03 10:48:26 +08:00
flynnnnnnnnnn e81ccc406b
Implement FSFE REUSE for golang files (#21840)
Change all license headers to comply with REUSE specification.

Fix #16132

Co-authored-by: flynnnnnnnnnn <flynnnnnnnnnn@github>
Co-authored-by: John Olheiser <john.olheiser@gmail.com>
2022-11-27 18:20:29 +00:00
Lunny Xiao 86c85c19b6
Refactor AssertExistsAndLoadBean to use generics (#20797)
* Refactor AssertExistsAndLoadBean to use generics

* Fix tests

Co-authored-by: zeripath <art27@cantab.net>
2022-08-16 10:22:25 +08:00
Lunny Xiao cbd45471b1
Move access and repo permission to models/perm/access (#19350)
* Move access and repo permission to models/perm/access

* Remove unnecessary code
2022-05-11 12:09:36 +02:00
wxiaoguang 04fc4b7e05
Call MultipartForm.RemoveAll when request finishes (#19606) 2022-05-05 16:13:23 +02:00
6543 06e4687cec
more context for models (#19511)
make more usage of context, to have more db transaction in one session

(make diff of  #9307 smaller)
2022-04-28 13:48:48 +02:00
6543 3e88af898a
Make git.OpenRepository accept Context (#19260)
* OpenRepositoryCtx -> OpenRepository
* OpenRepository -> openRepositoryWithDefaultContext, only for internal usage
2022-03-30 03:13:41 +08:00
wxiaoguang 7a550b3af2
Use ctx instead of db.DefaultContext in some packages(routers/services/modules) (#19163)
* Remove `db.DefaultContext` usage in routers, use `ctx` directly

* Use `ctx` directly if there is one, remove some `db.DefaultContext` in `services`

* Use ctx instead of db.DefaultContext for `cmd` and some `modules` packages

* fix incorrect context usage
2022-03-22 16:22:54 +01:00
KN4CK3R 80fd25524e
Renamed ctx.User to ctx.Doer. (#19161)
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2022-03-22 15:03:22 +08:00
6543 54e9ee37a7
format with gofumpt (#18184)
* gofumpt -w -l .

* gofumpt -w -l -extra .

* Add linter

* manual fix

* change make fmt
2022-01-20 18:46:10 +01:00
zeripath 5cb0c9aa0d
Propagate context and ensure git commands run in request context (#17868)
This PR continues the work in #17125 by progressively ensuring that git
commands run within the request context.

This now means that the if there is a git repo already open in the context it will be used instead of reopening it.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-01-19 23:26:57 +00:00
wxiaoguang e61b390d54
Unify and simplify TrN for i18n (#18141)
Refer: https://github.com/go-gitea/gitea/pull/18135#issuecomment-1003246099

Now we have a unique and simple `TrN`, and make the fix of PR #18135 also use the better `TrN` logic.
2022-01-02 04:33:57 +01:00
Lunny Xiao 719bddcd76
Move repository model into models/repo (#17933)
* Some refactors related repository model

* Move more methods out of repository

* Move repository into models/repo

* Fix test

* Fix test

* some improvements

* Remove unnecessary function
2021-12-10 09:27:50 +08:00
Lunny Xiao a666829a37
Move user related model into models/user (#17781)
* Move user related model into models/user

* Fix lint for windows

* Fix windows lint

* Fix windows lint

* Move some tests in models

* Merge
2021-11-24 17:49:20 +08:00
wxiaoguang 81926d61db
Decouple unit test, remove intermediate unittestbridge package (#17662)
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-11-16 16:53:21 +08:00
Lunny Xiao d47798ccc7
Upgrade chi to v5 (#17298) 2021-10-13 22:50:23 -04:00
Lunny Xiao a4bfef265d
Move db related basic functions to models/db (#17075)
* Move db related basic functions to models/db

* Fix lint

* Fix lint

* Fix test

* Fix lint

* Fix lint

* revert unnecessary change

* Fix test

* Fix wrong replace string

* Use *Context

* Correct committer spelling and fix wrong replaced words

Co-authored-by: zeripath <art27@cantab.net>
2021-09-19 19:49:59 +08:00
Lunny Xiao 5e20fd6dbf
Move middlewares to web/middleware (#14480)
Co-authored-by: 6543 <6543@obermui.de>
2021-01-30 10:55:53 +02:00
Lunny Xiao 6433ba0ec3
Move macaron to chi (#14293)
Use [chi](https://github.com/go-chi/chi) instead of the forked [macaron](https://gitea.com/macaron/macaron). Since macaron and chi have conflicts with session share, this big PR becomes a have-to thing. According my previous idea, we can replace macaron step by step but I'm wrong. :( Below is a list of big changes on this PR.

- [x] Define `context.ResponseWriter` interface with an implementation `context.Response`.
- [x] Use chi instead of macaron, and also a customize `Route` to wrap chi so that the router usage is similar as before.
- [x] Create different routers for `web`, `api`, `internal` and `install` so that the codes will be more clear and no magic .
- [x] Use https://github.com/unrolled/render instead of macaron's internal render
- [x] Use https://github.com/NYTimes/gziphandler instead of https://gitea.com/macaron/gzip
- [x] Use https://gitea.com/go-chi/session which is a modified version of https://gitea.com/macaron/session and removed `nodb` support since it will not be maintained. **BREAK**
- [x] Use https://gitea.com/go-chi/captcha which is a modified version of https://gitea.com/macaron/captcha
- [x] Use https://gitea.com/go-chi/cache which is a modified version of https://gitea.com/macaron/cache
- [x] Use https://gitea.com/go-chi/binding which is a modified version of https://gitea.com/macaron/binding
- [x] Use https://github.com/go-chi/cors instead of https://gitea.com/macaron/cors
- [x] Dropped https://gitea.com/macaron/i18n and make a new one in `code.gitea.io/gitea/modules/translation`
- [x] Move validation form structs from `code.gitea.io/gitea/modules/auth` to `code.gitea.io/gitea/modules/forms` to avoid dependency cycle.
- [x] Removed macaron log service because it's not need any more. **BREAK**
- [x] All form structs have to be get by `web.GetForm(ctx)` in the route function but not as a function parameter on routes definition.
- [x] Move Git HTTP protocol implementation to use routers directly.
- [x] Fix the problem that chi routes don't support trailing slash but macaron did.
- [x] `/api/v1/swagger` now will be redirect to `/api/swagger` but not render directly so that `APIContext` will not create a html render. 

Notices:
- Chi router don't support request with trailing slash
- Integration test `TestUserHeatmap` maybe mysql version related. It's failed on my macOS(mysql 5.7.29 installed via brew) but succeed on CI.

Co-authored-by: 6543 <6543@obermui.de>
2021-01-26 16:36:53 +01:00
techknowlogick c5d5d63c9c
Macaron 1.5 (#12596)
* update macaron to v1.5 of fork

* update macaron to v1.5 of fork

* test gzip PR

* add push method impl to context_tests

* use proper gzip commit

Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2020-08-27 22:47:17 -04:00
mrsdizzie 8d99ee2773
Add Organization Wide Labels (#10814)
* Add organization wide labels

Implement organization wide labels similar to organization wide
webhooks. This lets you create individual labels for organizations that can be used
for all repos under that organization (so being able to reuse the same
label across multiple repos).

This makes it possible for small organizations with many repos to use
labels effectively.

Fixes #7406

* Add migration

* remove comments

* fix tests

* Update options/locale/locale_en-US.ini

Removed unused translation string

* show org labels in issue search label filter

* Use more clear var name

* rename migration after merge from master

* comment typo

* update migration again after rebase with master

* check for orgID <=0 per guillep2k review

* fmt

* Apply suggestions from code review

Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com>

* remove unused code

* Make sure RepoID is 0 when searching orgID per code review

* more changes/code review requests

* More descriptive translation var per code review

* func description/delete comment when issue label deleted instead of hiding it

* remove comment

* only use issues in that repo when calculating number of open issues for org label on repo label page

* Add integration test for IssuesSearch API with labels

* remove unused function

* Update models/issue_label.go

Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com>

* Use subquery in GetLabelIDsInReposByNames

* Fix tests to use correct orgID

* fix more tests

* IssuesSearch api now uses new BuildLabelNamesIssueIDsCondition. Add a few more tests as well

* update comment for clarity

* Revert previous code change now that we can use the new BuildLabelNamesIssueIDsCondition

* Don't sort repos by date in IssuesSearch API

After much debugging I've found a strange issue where in some cases MySQL will return a different result than other enigines if a query is sorted by a null collumn. For example with our integration test data where we don't set updated_unix in repository fixtures:

SELECT `id`, `owner_id`, `owner_name`, `lower_name`, `name`, `description`, `website`, `original_service_type`, `original_url`, `default_branch`, `num_watches`, `num_stars`, `num_forks`, `num_issues`, `num_closed_issues`, `num_pulls`, `num_closed_pulls`, `num_milestones`, `num_closed_milestones`, `is_private`, `is_empty`, `is_archived`, `is_mirror`, `status`, `is_fork`, `fork_id`, `is_template`, `template_id`, `size`, `is_fsck_enabled`, `close_issues_via_commit_in_any_branch`, `topics`, `avatar`, `created_unix`, `updated_unix` FROM `repository` ORDER BY updated_unix DESC LIMIT 15 OFFSET 45

Returns different results for MySQL than other engines. However, the similar query:

SELECT `id`, `owner_id`, `owner_name`, `lower_name`, `name`, `description`, `website`, `original_service_type`, `original_url`, `default_branch`, `num_watches`, `num_stars`, `num_forks`, `num_issues`, `num_closed_issues`, `num_pulls`, `num_closed_pulls`, `num_milestones`, `num_closed_milestones`, `is_private`, `is_empty`, `is_archived`, `is_mirror`, `status`, `is_fork`, `fork_id`, `is_template`, `template_id`, `size`, `is_fsck_enabled`, `close_issues_via_commit_in_any_branch`, `topics`, `avatar`, `created_unix`, `updated_unix` FROM `repository` ORDER BY updated_unix DESC LIMIT 15 OFFSET 30

Returns the same results.

This causes integration tests to fail on MySQL in certain cases but would never show up in a real installation. Since this API call always returns issues based on the optionally provided repo_priority_id or the issueID itself, there is no change to results by changing the repo sorting method used to get ids earlier in the function.

* linter is back!

* code review

* remove now unused option

* Fix newline at end of files

* more unused code

* update to master

* check for matching ids before query

* Update models/issue_label.go

Co-Authored-By: 6543 <6543@obermui.de>

* Update models/issue_label.go

* update comments

* Update routers/org/setting.go

Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
Co-authored-by: 6543 <6543@obermui.de>
2020-04-01 01:14:46 -03:00
6543 084a2b0026
Code Refactor of IssueWatch related things (#10401)
* refactor

* optimize

* remove Iretating function
LoadWatchUsers do not load Users into IW object and it is used only in api ... so move this logic

* remove unessesary

* Apply suggestions from code review

Thx

Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com>

* make Tests more robust

* fix rebase

* restart CI

* CI no dont hit sqlites deadlock

Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
2020-02-26 03:32:22 -03:00
zeripath 722a7c902d
Add Close() method to gogitRepository (#8901)
In investigating #7947 it has become clear that the storage component of go-git repositories needs closing.

This PR adds this Close function and adds the Close functions as necessary.

In TransferOwnership the ctx.Repo.GitRepo is closed if it is open to help prevent the risk of multiple open files.

Fixes #7947
2019-11-13 07:01:19 +00:00
Tamal Saha 171b359877 Use gitea forked macaron (#7933)
Signed-off-by: Tamal Saha <tamal@appscode.com>
2019-08-23 12:40:29 -04:00
Lunny Xiao d578b71d61
move code.gitea.io/git to code.gitea.io/gitea/modules/git (#6364)
* move code.gitea.io/git to code.gitea.io/gitea/modules/git

* fix imports

* fix fmt

* fix misspell

* remove wrong tests data

* fix unit tests

* fix tests

* fix tests

* fix tests

* fix tests

* fix tests

* enable Debug to trace the failure tests

* fix tests

* fix tests

* fix tests

* fix tests

* fix tests

* comment commit count tests since git clone depth is 50

* fix tests

* update from code.gitea.io/git

* revert change to makefile
2019-03-27 17:33:00 +08:00
Lunny Xiao eabbddcd98
Restrict permission check on repositories and fix some problems (#5314)
* fix units permission problems

* fix some bugs and merge LoadUnits to repoAssignment

* refactor permission struct and add some copyright heads

* remove unused codes

* fix routes units check

* improve permission check

* add unit tests for permission

* fix typo

* fix tests

* fix some routes

* fix api permission check

* improve permission check

* fix some permission check

* fix tests

* fix tests

* improve some permission check

* fix some permission check

* refactor AccessLevel

* fix bug

* fix tests

* fix tests

* fix tests

* fix AccessLevel

* rename CanAccess

* fix tests

* fix comment

* fix bug

* add missing unit for test repos

* fix bug

* rename some functions

* fix routes check
2018-11-28 19:26:14 +08:00
Ethan Koenig 7ea4bfc561 API endpoint for testing webhook (#3550)
* API endpoint for testing webhook

* Empty commit to rerun CI
2018-04-29 14:21:33 +08:00
Ethan Koenig befa7445d2 Unit tests for routers/repo/issue_label (#3198) 2017-12-15 23:11:02 +02:00
Lunny Xiao 41d6afe0e7 Fix compile failed without gcc (#3130)
* fix comile failed without gcc

* add sqlite tag for unit test
2017-12-10 16:08:19 +02:00
Ethan Koenig 596308f213 Fix missing branch in release bug (#3108) 2017-12-08 13:22:02 +08:00
Lunny Xiao b3d5ba6f90 Fix missing password length check when change password (#3039)
* fix missing password length check when change password

* add tests for change password
2017-12-02 19:11:22 +02:00
Ethan Koenig 91f3d77ceb Unit tests for wiki routers (#3022) 2017-11-30 17:52:15 +02:00
Ethan Koenig 7e0654bd9e Fix counts on issues dashboard (#2215)
* Fix counts on issues dashboard

* setupSess -> setupSession

* Unit test

* Load repo owners for issues
2017-08-03 13:09:16 +08:00