The current default memory cache implementation is unbounded in size and number of
objects cached. This is hardly ideal.
This PR proposes creating a TwoQueue LRU cache as the underlying cache for Gitea.
The cache is limited by the number of objects stored in the cache (rather than size)
for simplicity. The default number of objects is 50000 - which is perhaps too small
as most of our objects cached are going to be much less than 1kB.
It may be worth considering using a different LRU implementation that actively limits
sizes or avoids GC - however, this is just a beginning implementation.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Following the merging of https://github.com/go-git/go-git/pull/330 we
can now add a setting to avoid go-git reading and caching large objects.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Now that #16069 is merged, some sites may wish to enforce that users are all public, limited or private, and/or disallow users from becoming private.
This PR adds functionality and settings to constrain a user's ability to change their visibility.
Co-authored-by: zeripath <art27@cantab.net>
One of the repeatedly reported issues has been that gitea produces too much console
logging during set up even if the console logger is turned off.
Fundamentally this is due to some otherwise very helpful logging that has to occur
before logging is set up. This has come to a head with the merging of #16243 where
otherwise potentially helpful Trace logging in the git module now appears on the
console.
This PR proposes three things:
1. Change the initial default logger to Info not Trace.
2. Change the logging for the AppPath things to Info in recompense.
3. Add two new command line options to gitea web: --quiet and --verbose
`gitea web -q` or `gitea web --quiet` will only log Fatal level initially.
`gitea web -verbose` will log at Trace.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
You can limit or hide organisations. This pull make it also posible for users
- new strings to translte
- add checkbox to user profile form
- add checkbox to admin user.edit form
- filter explore page user search
- filter api admin and public user searches
- allow admins view "hidden" users
- add app option DEFAULT_USER_VISIBILITY
- rewrite many files to use Visibility field
- check for teams intersection
- fix context output
- right fake 404 if not visible
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Andrew Thornton <art27@cantab.net>
* Added support for gopher URLs.
* Add setting and make this user settable instead
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Andrew Thornton <art27@cantab.net>
* Ensure settings for Service and Mailer are read on the install page
NewContext does not set the mailer or service settings so add
a new function that will run this.
Fix #15894
Signed-off-by: Andrew Thornton <art27@cantab.net>
* placate lint
Signed-off-by: Andrew Thornton <art27@cantab.net>
There are a couple of settings in `[indexer]` relating to the `issue_indexer` queue
which override settings in unpredictable ways. This PR adjusts this hierarchy and makes
explicit that these settings are deprecated.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Set self-adjusting deadline for connection writing
In #16055 it appears that the simple 5s deadline doesn't work for large
file writes. Now we can't - or at least shouldn't just set no deadline
as go will happily let these connections block indefinitely. However,
what seems reasonable is to set some minimum rate we expect for writing.
This PR suggests the following algorithm:
* Every write has a minimum timeout of 5s (adjustable at compile time.)
* If there has been a previous write - then consider its previous
deadline, add half of the minimum timeout + 2s per kb about to written.
* If that new deadline is after the minimum timeout use that.
Fix #16055
* Linearly increase timeout
* Make PerWriteTimeout, PerWritePerKbTimeouts configurable
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
As revealed by #15964 there is inconsistent use of filepath Join and path Join
for these directories. The best thing to do is to use filepath.Join but then ToSlash
them for consistency.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: John Olheiser <john.olheiser@gmail.com>
This PR suggests a change to the default configuration for queues:
* Use a common DATADIR for the queues
* Set starting workers to 0 and make boost a single worker
Signed-off-by: Andrew Thornton <art27@cantab.net>
* add a new internal hook to save ssh log
as title, when a ssh error ocure like #15785.
only when switch ``RUN_MODE`` to dev can we
found which error is ocure. But this way is
not a good idea for production envirment.
this changes try save ssh error mesage to the
log file like other log by a new internal hook.
I think it's usefull for find error message
in production envirment. Thanks.
Signed-off-by: a1012112796 <1012112796@qq.com>
* rename and fix nit
* Update modules/private/hook.go
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: techknowlogick <matti@mdranta.net>
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Refactored handleOAuth2SignIn in routers/user/auth.go
The function handleOAuth2SignIn was called twice but some code path could only
be reached by one of the invocations. Moved the unnecessary code path out of
handleOAuth2SignIn.
* Refactored user creation
There was common code to create a user and display the correct error message.
And after the creation the only user should be an admin and if enabled a
confirmation email should be sent. This common code is now abstracted into
two functions and a helper function to call both.
* Added auto-register for OAuth2 users
If enabled new OAuth2 users will be registered with their OAuth2 details.
The UserID, Name and Email fields from the gothUser are used.
Therefore the OpenID Connect provider needs additional scopes to return
the coresponding claims.
* Added error for missing fields in OAuth2 response
* Linking and auto linking on oauth2 registration
* Set default username source to nickname
* Add automatic oauth2 scopes for github and google
* Add hint to change the openid connect scopes if fields are missing
* Extend info about auto linking security risk
Co-authored-by: Viktor Kuzmin <kvaster@gmail.com>
Signed-off-by: Martin Michaelis <code@mgjm.de>
* _ to unused func options
* rm useless brakets
* rm trifial non used models functions
* rm dead code
* rm dead global vars
* fix routers/api/v1/repo/issue.go
* dont overload import module
Implements request #14320 The rendering of CSV files does match the diff style.
* Moved CSV logic into base package.
* Added method to create a tabular diff.
* Added CSV compare context.
* Added CSV diff template.
* Use new table style in CSV markup.
* Added file size limit for CSV rendering.
* Display CSV parser errors in diff.
* Lazy read single file.
* Lazy read rows for full diff.
* Added unit tests for various CSV changes.
in previouse if a log subsetcion not set level
it will use ``info`` as default value.
this pr will make default value (``[log] -> LEVEL``) useable.
example config:
```INI
[log]
MODE = console
LEVEL = Trace
[log.console]
LEVEL =
STDERR = false
```
previous result:
```JSON
// console:
{
"level": "info",
...................
}
```
after change:
```JSON
// console:
{
"level": "track",
...................
}
```
Signed-off-by: a1012112796 <1012112796@qq.com>
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lauris BH <lauris@nix.lv>
* Add reverse proxy configuration support for remote IP address validation
* Trust all IP addresses in containerized environments by default
* Use single option to specify networks and proxy IP addresses. By default trust all loopback IPs
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
This is an alternative PR to #13687.
Add `[ui.explore]` settings to allow restricting the
explore pages to logged in users only and to disable the users explore page.
The two proposed settings are:
- `REQUIRE_SIGNIN_VIEW`: Only allows access to the explore pages if the
user is signed in. Also restricts
- `/api/v1/user/search`
- `/api/v1/users/{username}`
- `/api/v1/users/{username}/repos`
- but does not restrict `/api/v1/users/{username}/heatmap`
- `DISABLE_USERS_PAGE`: Disables the /explore/users page
Fix #2908
Close #13687
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
* Make SSH server host key path configurable
* make it possible to have multiple keys
* Make gitea.rsa the default key
* Add some more logging
Signed-off-by: Andrew Thornton <art27@cantab.net>
Add SameSite setting for cookies and rationalise the cookie setting code. Switches SameSite to Lax by default.
There is a possible future extension of differentiating which cookies could be set at Strict by default but that is for a future PR.
Fix #5583
Signed-off-by: Andrew Thornton <art27@cantab.net>
MakeAbsoluteAssetURL should just url join the static url prefix on to appurl
if it is not an absolute path - this is because StaticURLPrefix is an absolute
prefix not a relative prefix to the app sub url.
Fix #14422
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Create Xorm session provider
This PR creates a Xorm session provider which creates
the appropriate Session table for macaron/session.
Fix #7137
Signed-off-by: Andrew Thornton <art27@cantab.net>
* extraneous l
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fix lint
Signed-off-by: Andrew Thornton <art27@cantab.net>
* use key instead of ID to be compatible with go-macaron/session
Signed-off-by: Andrew Thornton <art27@cantab.net>
* And change the migration too.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Update spacing of imports
Co-authored-by: 6543 <6543@obermui.de>
* Update modules/session/xorm.go
Co-authored-by: techknowlogick <matti@mdranta.net>
* add xorm provider to the virtual provider
Signed-off-by: Andrew Thornton <art27@cantab.net>
* prep for master merge
* prep for merge master
* As per @lunny
* move migration out of the way
* Move to call this db session as per @lunny
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: techknowlogick <matti@mdranta.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Gitea allows to whitelist email domains so that only email addresses from certain domains are allowed to register an account, but does not currently allows to do the opposite: blacklisting email domains so that addresses from certain domains are *forbidden* to register an account.
The idea has been briefly mentioned in the discussion about issue #6350, but never implemented. This PR does that.
The rationale is that, in my experience of running a Gitea instance, *a single email domain* is responsible for *most* of the spam accounts, and for *all* of the spam accounts that manage to get past the email confirmation step. So on top of the other spam mitigation measures already available (email confirmation, CAPTCHA, etc.), having the option to block a particularly annoying domain would be helpful.
close #13628
* Added option to disable web hooks
This mod introduces DISABLE_WEB_HOOKS parameter in [security] section
of app.ini (by default set to false). If set to true it disables web
hooks feature. Any existing undelivered web hook tasks will be cancelled.
Any existing web hook definitions will be left untouched in db but
its delivery tasks will be ignored.
Author-Change-Id: IB#1105130
* Webhook spelling fixed
Webhook spelling fixed.
Fixes: 07df6614dc84cdd2e9f39c57577fa1062bd70012
Related: https://github.com/go-gitea/gitea/pull/13176#pullrequestreview-510868421
Author-Change-Id: IB#1105174
* Parameter description fixed
Parameter description fixed.
Fixes: 07df6614dc84cdd2e9f39c57577fa1062bd70012
Related: https://github.com/go-gitea/gitea/pull/13176#pullrequestreview-514086107
Author-Change-Id: IB#1105174
Memcached TTL cannot be > 30 days and if it is attempted the TTL is interpreted as
a unix timestamp.
This PR ensures that the TTL is switched to a unix timestamp in those cases.
Fix #14571
Signed-off-by: Andrew Thornton <art27@cantab.net>
