Commit Graph

63 Commits

Author SHA1 Message Date
flynnnnnnnnnn e81ccc406b
Implement FSFE REUSE for golang files (#21840)
Change all license headers to comply with REUSE specification.

Fix #16132

Co-authored-by: flynnnnnnnnnn <flynnnnnnnnnn@github>
Co-authored-by: John Olheiser <john.olheiser@gmail.com>
2022-11-27 18:20:29 +00:00
zeripath 943753f560
Support Proxy protocol (#12527)
This PR adds functionality to allow Gitea to sit behind an
HAProxy and HAProxy protocolled connections directly.

Fix #7508

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-08-21 19:20:43 +01:00
JonRB cb37c6ba5c
call builtinUnused() if internal SSH is disabled (#20877)
The graceful manager waits for 4 listeners to be created or to be told that they are not needed. If it is not told about them it will indefinitely and timeout. 

This leads to SVC hosts not being told of being in the readyState but on Unix would lead to the termination of the process.

There was an unfortunate regression in #20299 which missed this subtly and in the case whereby SSH is disabled the `builtinUnused()` is not called.

This PR adds a call to `builtinUnused()` when not using the builtin ssh to allow `createServerWaitGroup.Done()` to be called. 

In addition it was noted that the if/else clauses for timeout informing of the SVC host were in the wrong order. These have been swapped.

Fix #20609
2022-08-20 22:09:41 +01:00
Gusted 87a7c37ecb
Use `strings.Cut` for GIT_PROTOCOL value (#20638)
- As per https://github.com/go-gitea/gitea/pull/20616#discussion_r935612542
2022-08-03 01:34:50 +03:00
Gusted 8740631b79
Enable Wire 2 for Internal SSH Server (#20616)
- Git only decides to use the Wire 2 protocol when `git
{receive,upload}-pack` receive the `GIT_PROTOCOL` environment with as
value `version=2`. Currently the internal SSH Server wasn't passing this
environment through. The `gitea serv` code already passed all received
environments to the git command, so no code changes there.
2022-08-02 15:56:38 +08:00
zeripath c9c5bd88be
Stop logging EOFs and exit(1)s in ssh handler (#20476)
The code in modules/ssh/ssh.go:sessionHandler() currently cause an error to be
logged if `gitea serv` exits with a exit(1). This logging is useless because the
accompanying stderr is not provided and in any case the exit(1) is most likely due
to permissions errors.

Further it then causes the EOF to be logged - even though this is not helpful.

This PR simply checks the errors returned and stops logging them.

In the case of misconfigurations causing `gitea serv` to fail with exit(1)
the current logging is not helpful at determining this and users should simply
review the message passed over the ssh connection.

Fix #20473

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-07-28 20:56:55 +01:00
wxiaoguang 27e2def5f0
Refactor SSH init code, fix directory creation for TrustedUserCAKeys file (#20299)
* Refactor SSH init code, fix directory creation for TrustedUserCAKeys file

* Update modules/ssh/init.go

Co-authored-by: zeripath <art27@cantab.net>

* fix lint copyright

* Update modules/ssh/init.go

Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-07-10 14:50:26 +08:00
Wim e528e2b435
Implement http signatures support for the API (#17565)
Fixes #12338

This allows use to talk to the API with our ssh certificate (and/or ssh-agent) without needing to fetch an API key or tokens.
It will just automatically work when users have added their ssh principal in gitea.

This needs client code in tea
Update: also support normal pubkeys

ref: https://tools.ietf.org/html/draft-cavage-http-signatures

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: zeripath <art27@cantab.net>
Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-06-05 08:16:14 +01:00
zeripath 1d04e8641d
Set Setpgid on child git processes (#19865)
When Gitea is running as PID 1 git will occassionally orphan child processes leading
to (defunct) processes. This PR simply sets Setpgid to true on these child processes
meaning that these defunct processes will also be correctly reaped.

Fix #19077

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-06-03 15:36:18 +01:00
Lunny Xiao fd7d83ace6
Move almost all functions' parameter db.Engine to context.Context (#19748)
* Move almost all functions' parameter db.Engine to context.Context
* remove some unnecessary wrap functions
2022-05-20 22:08:52 +08:00
zeripath c88547ce71
Add Goroutine stack inspector to admin/monitor (#19207)
Continues on from #19202.

Following the addition of pprof labels we can now more easily understand the relationship between a goroutine and the requests that spawn them. 

This PR takes advantage of the labels and adds a few others, then provides a mechanism for the monitoring page to query the pprof goroutine profile.

The binary profile that results from this profile is immediately piped in to the google library for parsing this and then stack traces are formed for the goroutines.

If the goroutine is within a context or has been created from a goroutine within a process context it will acquire the process description labels for that process. 

The goroutines are mapped with there associate pids and any that do not have an associated pid are placed in a group at the bottom as unbound.

In this way we should be able to more easily examine goroutines that have been stuck.

A manager command `gitea manager processes` is also provided that can export the processes (with or without stacktraces) to the command line.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-03-31 19:01:43 +02:00
Gusted 6ab4a96855
Update golang.org/x/crypto (#19097)
* Update golang.org/x/crypto

- Update dependency to include fix for CVE.
- See https://groups.google.com/g/golang-announce/c/-cp44ypCT5s/m/wmegxkLiAQAJ?utm_medium=email&utm_source=footer

* Fix deprecation notice

* Remove workaround

- Introduced in https://github.com/go-gitea/gitea/pull/17281
- Fixed in x/crypto:
- 5d542ad81a
- & 3147a52a75

* Update Kex Algorithms

- Use standardized name for curve22519-sha256. 9b076918e3
- Prefer SHA256 version over SHA1 version. e4b3678e5f
2022-03-16 02:59:53 +01:00
6543 54e9ee37a7
format with gofumpt (#18184)
* gofumpt -w -l .

* gofumpt -w -l -extra .

* Add linter

* manual fix

* change make fmt
2022-01-20 18:46:10 +01:00
Gusted ff2fd08228
Simplify parameter types (#18006)
Remove repeated type declarations in function definitions.
2021-12-20 04:41:31 +00:00
Lunny Xiao 3ca5dc7e32
Move keys to models/asymkey (#17917)
* Move keys to models/keys

* Rename models/keys -> models/asymkey

* change the missed package name

* Fix package alias

* Fix test

* Fix docs

* Fix test

* Fix test

* merge
2021-12-10 16:14:24 +08:00
Wim bd613c704c
Fix ipv6 parsing for builtin ssh server (#17561) 2021-11-06 14:23:32 +08:00
zeripath 35b918f574
Offer rsa-sha2-512 and rsa-sha2-256 algorithms in internal SSH (#17281)
* Offer rsa-sha2-512 and rsa-sha2-256 algorithms in internal SSH

There is a subtle bug in the SSH library x/crypto/ssh which makes the incorrect
assumption that the public key type is the same as the signature algorithm type.

This means that only ssh-rsa signatures are offered by default.

This PR adds a workaround around this problem.

Fix #17175

Signed-off-by: Andrew Thornton <art27@cantab.net>

* as per review

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-10-20 15:55:33 -04:00
zeripath 3dcb3e9073
Second attempt at preventing zombies (#16326)
* Second attempt at preventing zombies

* Ensure that the pipes are closed in ssh.go
* Ensure that a cancellable context is passed up in cmd/* http requests
* Make cmd.fail return properly so defers are obeyed
* Ensure that something is sent to stdout in case of blocks here

Signed-off-by: Andrew Thornton <art27@cantab.net>

* placate lint

Signed-off-by: Andrew Thornton <art27@cantab.net>

* placate lint 2

Signed-off-by: Andrew Thornton <art27@cantab.net>

* placate lint 3

Signed-off-by: Andrew Thornton <art27@cantab.net>

* fixup

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Apply suggestions from code review

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lauris BH <lauris@nix.lv>
2021-07-14 10:43:13 -04:00
zeripath 302e8b6d02
Prevent zombie processes (#16314)
Unfortunately go doesn't always ensure that execd processes are completely
waited for. On linux this means that zombie processes can occur.

This PR ensures that these are waited for by using signal notifier in serv and
passing a context elsewhere.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-06-30 22:07:23 +02:00
zeripath f825f20d49
Upgrade Gliderlabs SSH to 0.3.3 and add FailedConnectionCallback (#16278)
* Upgrade Gliderlabs SSH to 0.3.3 and add FailedConnectionCallback

Following the merging of https://github.com/gliderlabs/ssh/pull/143 we
can now report connections to the ssh server that have failed before
public key exchange has completed using the standard fail2ban message.

This PR updates Gliderlabs SSH and adds a callback that will provide this
logging.

Signed-off-by: Andrew Thornton <art27@cantab.net>

* move the callback to its own function to make the logging appear little nicer

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-06-28 13:05:27 -04:00
zeripath daa5a23548
Set self-adjusting deadline for connection writing (#16068)
* Set self-adjusting deadline for connection writing

In #16055 it appears that the simple 5s deadline doesn't work for large
file writes. Now we can't - or at least shouldn't just set no deadline
as go will happily let these connections block indefinitely. However,
what seems reasonable is to set some minimum rate we expect for writing.

This PR suggests the following algorithm:

* Every write has a minimum timeout of 5s (adjustable at compile time.)
* If there has been a previous write - then consider its previous
deadline, add half of the minimum timeout + 2s per kb about to written.
* If that new deadline is after the minimum timeout use that.

Fix #16055

* Linearly increase timeout

* Make PerWriteTimeout, PerWritePerKbTimeouts configurable

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: 6543 <6543@obermui.de>
2021-06-11 00:25:25 +03:00
zeripath c03f530212
Make internal SSH server host key path configurable (#14918)
* Make SSH server host key path configurable

* make it possible to have multiple keys

* Make gitea.rsa the default key

* Add some more logging

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-03-08 03:43:59 +01:00
Stefan eea4197cd9
Internal ssh server respect Ciphers, MACs and KeyExchanges settings (#14523) 2021-01-30 14:20:32 +01:00
6543 a19447aed1
migrate from com.* to alternatives (#14103)
* remove github.com/unknwon/com from models

* dont use "com.ToStr()"

* replace "com.ToStr" with "fmt.Sprint" where its easy to do

* more refactor

* fix test

* just "proxy" Copy func for now

* as per @lunny
2020-12-25 11:59:32 +02:00
zeripath 980b0df829
Standardise logging of failed authentication attempts in internal SSH (#13962)
Continuing on from #13953 continue to improve and standardise
logging from internal SSH.

Also updates the fail2ban setup

Signed-off-by: Andrew Thornton <art27@cantab.net>
2020-12-15 08:45:13 +00:00
zeripath e46a638e8f
Report permissions denied in internal SSH (#13953)
This PR standardizes reporting of permission denied from the internal ssh.

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: 6543 <6543@obermui.de>
2020-12-11 23:52:38 +01:00
Eric Lesiuta abb9cffe4a
Log IP on SSH authentication failure for Built-in SSH server (#13150)
* Log IP on SSH authentication failure

fixes https://github.com/go-gitea/gitea/issues/13094

* include string 'Failed authentication attempt' in error

* update fail2ban docs

also match failed authentication over command line

* better logging of authentication errors with IP addresses

* format ...

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: 6543 <6543@obermui.de>
2020-12-08 18:54:33 +01:00
zeripath 742e21aeba
Handle and propagate errors when checking if paths are Dirs, Files or Exist (#13186)
* Ensure errors from IsDir propagate

* Handle errors when checking IsFile

* Handle and propagate errors from IsExist

* Update modules/templates/static.go

* Update modules/templates/static.go

* Return after ctx.ServerError

* Apply suggestions from code review

* Fix tests

The previous merge managed to break repo_form.go

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: Lauris BH <lauris@nix.lv>
2020-11-27 21:42:08 -05:00
Wim 9066d09c57
Add ssh certificate support (#12281)
* Add ssh certificate support

* Add ssh certificate support to builtin ssh

* Write trusted-user-ca-keys.pem based on configuration

* Update app.example.ini

* Update templates/user/settings/keys_principal.tmpl

Co-authored-by: silverwind <me@silverwind.io>

* Remove unused locale string

* Update options/locale/locale_en-US.ini

Co-authored-by: silverwind <me@silverwind.io>

* Update options/locale/locale_en-US.ini

Co-authored-by: silverwind <me@silverwind.io>

* Update models/ssh_key.go

Co-authored-by: silverwind <me@silverwind.io>

* Add missing creation of SSH.Rootpath

* Update cheatsheet, example and locale strings

* Update models/ssh_key.go

Co-authored-by: zeripath <art27@cantab.net>

* Update models/ssh_key.go

Co-authored-by: zeripath <art27@cantab.net>

* Update models/ssh_key.go

Co-authored-by: zeripath <art27@cantab.net>

* Update models/ssh_key.go

Co-authored-by: zeripath <art27@cantab.net>

* Update models/ssh_key.go

* Optimizations based on feedback

* Validate CA keys for external sshd

* Add filename option and change default filename

Add a SSH_TRUSTED_USER_CA_KEYS_FILENAME option which default is
RUN_USER/.ssh/gitea-trusted-user-ca-keys.pem

Do not write a file when SSH_TRUSTED_USER_CA_KEYS is empty.

Add some more documentation.

* Remove unneeded principalkey functions

* Add blank line

* Apply suggestions from code review

Co-authored-by: zeripath <art27@cantab.net>

* Add SSH_AUTHORIZED_PRINCIPALS_ALLOW option

This adds a SSH_AUTHORIZED_PRINCIPALS_ALLOW which is default
email,username this means that users only can add the principals
that match their email or username.

To allow anything the admin need to set the option anything.

This allows for a safe default in gitea which protects against malicious
users using other user's prinicipals. (before that user could set it).

This commit also has some small other fixes from the last code review.

* Rewrite principal keys file on user deletion

* Use correct rewrite method

* Set correct AuthorizedPrincipalsBackup default setting

* Rewrite principalsfile when adding principals

* Add update authorized_principals option to admin dashboard

* Handle non-primary emails

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Add the command actually to the dashboard template

* Update models/ssh_key.go

Co-authored-by: silverwind <me@silverwind.io>

* By default do not show principal options unless there are CA keys set or they are explicitly set

Signed-off-by: Andrew Thornton <art27@cantab.net>

* allow settings when enabled

* Fix typos in TrustedUserCAKeys path

* Allow every CASignatureAlgorithms algorithm

As this depends on the content of TrustedUserCAKeys we should allow all
signature algorithms as admins can choose the specific algorithm on their
signing CA

* Update models/ssh_key.go

Co-authored-by: Lauris BH <lauris@nix.lv>

* Fix linting issue

Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: techknowlogick <matti@mdranta.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2020-10-10 20:38:09 -04:00
zeripath 311ed516ce
log.Fatal on failure to listen to SSH port (#10795)
Fix #10792
2020-03-23 07:59:38 +00:00
zeripath e3c3b33ea7
Graceful: Xorm, RepoIndexer, Cron and Others (#9282)
* Change graceful to use a singleton obtained through GetManager instead of a global.
* Graceful: Make TestPullRequests shutdownable
* Graceful: Make the cron tasks graceful
* Graceful: AddTestPullRequest run in graceful ctx
* Graceful: SyncMirrors shutdown
* Graceful: SetDefaultContext for Xorm to be HammerContext
* Avoid starting graceful for migrate commands and checkout
* Graceful: DeliverHooks now can be shutdown
* Fix multiple syncing errors in modules/sync/UniqueQueue &  Make UniqueQueue closable
* Begin the process of making the repo indexer shutdown gracefully
2019-12-15 09:51:28 +00:00
zeripath cbaa1de9ec Add Graceful shutdown for Windows and hooks for shutdown of goroutines (#8964)
* Graceful Shutdown for windows and others

Restructures modules/graceful, adding shutdown for windows, removing and
replacing the old minwinsvc code.

Creates a new waitGroup - terminate which allows for goroutines to
finish up after the shutdown of the servers.

Shutdown and terminate hooks are added for goroutines.

* Remove unused functions - these can be added in a different PR

* Add startup timeout functionality

* Document STARTUP_TIMEOUT
2019-11-21 13:32:02 -05:00
zeripath 167e8f18da
Restore Graceful Restarting & Socket Activation (#7274)
* Prevent deadlock in indexer initialisation during graceful restart

* Move from gracehttp to our own service to add graceful ssh

* Add timeout for start of indexers and make hammer time configurable

* Fix issue with re-initialization in indexer during tests

* move the code to detect use of closed to graceful

* Handle logs gracefully - add a pid suffix just before restart

* Move to using a cond and a holder for indexers

* use time.Since

* Add some comments and attribution

* update modules.txt

* Use zero to disable timeout

* Move RestartProcess to its own file

* Add cleanup routine
2019-10-15 14:39:51 +01:00
Tamal Saha 171b359877 Use gitea forked macaron (#7933)
Signed-off-by: Tamal Saha <tamal@appscode.com>
2019-08-23 12:40:29 -04:00
techknowlogick d0ec940dd7
switch to use gliderlabs/ssh for builtin server (#7250)
resolves git conflicts from #3896 (credit to @belak, in case github doesn't keep original author during squash)

Co-Authored-By: Matti Ranta <techknowlogick@gitea.io>
2019-07-06 21:28:09 -04:00
kolaente f9ec2f89f2 Add golangci (#6418) 2019-06-12 15:41:28 -04:00
Antoine GIRARD 10ff527fae internal/ssh: ignore env command totally (#6825)
* ssh: ignore env command totally

* Remove commented code 

Needed fix described in issue #6889
2019-05-09 18:41:43 -04:00
zeripath 704da08fdc
Better logging (#6038) (#6095)
* Panic don't fatal on create new logger

Fixes #5854

Signed-off-by: Andrew Thornton <art27@cantab.net>

* partial broken

* Update the logging infrastrcture

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Reset the skip levels for Fatal and Error

Signed-off-by: Andrew Thornton <art27@cantab.net>

* broken ncsa

* More log.Error fixes

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Remove nal

* set log-levels to lowercase

* Make console_test test all levels

* switch to lowercased levels

* OK now working

* Fix vetting issues

* Fix lint

* Fix tests

* change default logging to match current gitea

* Improve log testing

Signed-off-by: Andrew Thornton <art27@cantab.net>

* reset error skip levels to 0

* Update documentation and access logger configuration

* Redirect the router log back to gitea if redirect macaron log but also allow setting the log level - i.e. TRACE

* Fix broken level caching

* Refactor the router log

* Add Router logger

* Add colorizing options

* Adjust router colors

* Only create logger if they will be used

* update app.ini.sample

* rename Attribute ColorAttribute

* Change from white to green for function

* Set fatal/error levels

* Restore initial trace logger

* Fix Trace arguments in modules/auth/auth.go

* Properly handle XORMLogger

* Improve admin/config page

* fix fmt

* Add auto-compression of old logs

* Update error log levels

* Remove the unnecessary skip argument from Error, Fatal and Critical

* Add stacktrace support

* Fix tests

* Remove x/sync from vendors?

* Add stderr option to console logger

* Use filepath.ToSlash to protect against Windows in tests

* Remove prefixed underscores from names in colors.go

* Remove not implemented database logger

This was removed from Gogs on 4 Mar 2016 but left in the configuration
since then.

* Ensure that log paths are relative to ROOT_PATH

* use path.Join

* rename jsonConfig to logConfig

* Rename "config" to "jsonConfig" to make it clearer

* Requested changes

* Requested changes: XormLogger

* Try to color the windows terminal

If successful default to colorizing the console logs

* fixup

* Colorize initially too

* update vendor

* Colorize logs on default and remove if this is not a colorizing logger

* Fix documentation

* fix test

* Use go-isatty to detect if on windows we are on msys or cygwin

* Fix spelling mistake

* Add missing vendors

* More changes

* Rationalise the ANSI writer protection

* Adjust colors on advice from @0x5c

* Make Flags a comma separated list

* Move to use the windows constant for ENABLE_VIRTUAL_TERMINAL_PROCESSING

* Ensure matching is done on the non-colored message - to simpify EXPRESSION
2019-04-02 08:48:31 +01:00
Lunny Xiao 2d213b64d1 use native golang SSH library but ssh-keygen when enable built-in SSH server to remove dependent on that command lines (#5976)
* use native golang SSH library but ssh-keygen when enable built-in SSH server to remove dependent on that command lines

* fix tests and add comment head
2019-02-07 02:13:12 -05:00
Robert Sprunk 65b6ebf243 Log IP of failed ssh connection (#5766)
Fix #5765 by log the IP address of a connecting remote machine in case of a SSH connection error for the built-in ssh server.

Signed-off-by: Robert Sprunk <github@sprunk.me>
2019-01-19 16:29:17 +00:00
Cum Gun eecaba2031 Configurable SSH key exchange algorithm and MAC suite (#2806) 2017-11-02 23:26:41 +08:00
spacetourist 7131c7d40d Configurable SSH cipher suite (#913)
* Configurable SSH cipher suite

* Update configuration file comment

* Add default in settings loading code

* Fix fmt and log messsage

* Remove default from code as this could probably might not be good idea
2017-10-21 16:13:41 +03:00
Gabriel Jackson bf6f61cc69 Cleanup log messaging
This change corrects a few logging issues:

 * Standardized formatting errors with '%v'.
 * Standardized failure warning word usage.
 * Corrected an instance of using the standard log library when
   the gitea log library should be used instead.
2017-02-02 15:24:18 +01:00
Andrey Nering fda4476061 Fix SSH server on Windows when running as service
Closes #680
2017-01-23 20:44:23 -02:00
Lunny Xiao 65b1875d2b New settings option for a custom SSH host (#3763) (#446)
* New settings option for a custom SSH host (#3763)

* let default ssh listen addr empty
2016-12-22 13:26:43 +01:00
Bwko 4ff0db0246 Catch os... errors 2016-12-02 07:41:19 +01:00
Sandro Santilli 4247304f5a Update import paths from github.com/go-gitea to code.gitea.io (#135)
- Update import paths from github.com/go-gitea to code.gitea.io
- Fix import path for travis

See https://docs.travis-ci.com/user/languages/go#Go-Import-Path
2016-11-10 17:24:48 +01:00
Rémy Boulanouar 2d68bd1ef9 Change import reference to match gitea instead of gogs (#37) 2016-11-03 10:29:56 -02:00
Unknwon 9e09e48502 #2850 fix potential SSH commands dislocation
When use builtin SSH server with concurrent operations, there are probabilities
One connection could use the command from another connections.

Fix this by set SSH_ORIGINAL_COMMAND for each command, not set in global scope.
2016-03-18 06:13:16 -04:00
Unknwon 58e004f7da Remove cache avatar support and add its tests 2016-02-14 23:14:55 -05:00