Commit graph

2722 commits

Author SHA1 Message Date
Norwin
f6bec85296
rework heatmap permissions (#14080)
* now uses the same permission model as for the activity feed:
  only include activities in repos, that the doer has access to.
  this might be somewhat slower.

* also improves handling of user.KeepActivityPrivate (still shows
  the heatmap to self & admins)

* extend tests

* adjust integration test to new behaviour

* add access to actions for admins

* extend heatmap unit tests
2020-12-22 03:53:37 +01:00
Mura Li
09304db9a5
Use the text of pull-request as the squash commit's message (#13071)
Originally, it was filled by the commit messages of the involved
commits. In this change, we use the headline comment of the pull
request as the commit message when it is a squash merge.

Thanks to @zeripath for suggesting the idea.

Fixes #12365

Co-authored-by: Mura Li <typeless@users.noreply.github.com>
2020-12-21 17:46:14 +01:00
a1012112796
34df4e5df5
Add mentionable teams to tributeValues and change team mention rules to gh's style (#13198)
* Add mentionable teams to tributeValues

Signed-off-by: a1012112796 <1012112796@qq.com>

* Apply suggestions from code review

Co-authored-by: silverwind <me@silverwind.io>

* Change team mention rules to gh's style

* use org's avator as team avator in ui

Signed-off-by: a1012112796 <1012112796@qq.com>

* Update modules/markup/html.go

* Update models/issue.go

Co-authored-by: Lauris BH <lauris@nix.lv>

* Update models/issue.go

* fix a small nit and update test code

Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: 6543 <6543@obermui.de>
2020-12-21 16:39:28 +01:00
Paweł Bogusławski
839daa85aa
Added option to disable migrations (#13114)
* Added option to disable migrations

This patch introduces DISABLE_MIGRATIONS parameter in [repository]
section of app.ini (by default set to false). If set to true
it blocks access to repository migration feature.

This mod hides also local repo import option in user editor if
local repo importing or migrations is disabled.

* Alter Example config

DISABLE_MIGRATIONS set to false in example config to
match its default value.

* HTTP error 403 instead of 500 on denied access to migration

* Parameter DISABLE_MIGRATIONS exposed via API

Fixes: 04b04cf854bcb3ed7659442bcf79822bdebe29e9
Author-Change-Id: IB#1105130
2020-12-21 15:39:41 +01:00
Cirno the Strongest
f3c4baa84b
Show dropdown with all statuses for commit (#13977)
* Show dropdown with all statuses for commit

* Use popups

* Remove unnecessary change

* Style popup

* Use divided list

* As per @silverwind

* Refactor GetLastCommitStatus

* Missing dropdown on repo home and commit page

* Fix tests

* Make status icon be a part of a link on PR list

* Fix missing translation call

* Indent fix

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2020-12-20 11:13:12 +08:00
Jiri Vlasak
d7c67a9fb2
Manually approve new registration (#13083)
* Add register manual confirm settings option

The new settings option is used when manually approving new
registrations.

* Enable manual confirmation of new registered user

When manual registration confirmation is desired (by default `false`)
create new user in the database that is *not active*. The user must then
be activated manually.

This change speeds up the process of adding new confirmed users for
Gitea instances without external auth mechanism. (Currently the option
is to manually create new user by admin.)

* Update docs/content/doc/advanced/config-cheat-sheet.zh-cn.md

Co-authored-by: a1012112796 <1012112796@qq.com>

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: a1012112796 <1012112796@qq.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2020-12-19 20:31:06 -05:00
Cirno the Strongest
efa9a8a6e3
Show status check for merged PRs (#13975)
* Show status check for merged PRs

* Handle PRs with no commits

* Styling

Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: 6543 <6543@obermui.de>
2020-12-18 13:37:55 +01:00
6543
e483220ea3
[Refactor] CombinedStatus and CommitStatus related functions & structs (#14026)
* RM unused struct

* rename (*CommitStatus) loadRepo() -> loadAttributes()

* move ToCommitStatus into its own file

* use CommitStatusState instead of StatusState

* move CombinedStatus convertion into convert package

* let models.GetLatestCommitStatus use repoID direct and accept ListOptions

* update swagger docs

* fix tests

* Fix swagger docs

* rm page

* fix swagger docs!!!

* return json null

* always return json

* rename api.Status to api.CommitStatus

* fix swagger docs

* sec swagger fix
2020-12-18 11:33:32 +08:00
zeripath
511f6138d4
Use native git variants by default with go-git variants as build tag (#13673)
* Move last commit cache back into modules/git

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Remove go-git from the interface for last commit cache

Signed-off-by: Andrew Thornton <art27@cantab.net>

* move cacheref to last_commit_cache

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Remove go-git from routers/private/hook

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Move FindLFSFiles to pipeline

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Make no-go-git variants

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Submodule RefID

Signed-off-by: Andrew Thornton <art27@cantab.net>

* fix issue with GetCommitsInfo

Signed-off-by: Andrew Thornton <art27@cantab.net>

* fix GetLastCommitForPaths

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Improve efficiency

Signed-off-by: Andrew Thornton <art27@cantab.net>

* More efficiency

Signed-off-by: Andrew Thornton <art27@cantab.net>

* even faster

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Reduce duplication

* As per @lunny

Signed-off-by: Andrew Thornton <art27@cantab.net>

* attempt to fix drone

Signed-off-by: Andrew Thornton <art27@cantab.net>

* fix test-tags

Signed-off-by: Andrew Thornton <art27@cantab.net>

* default to use no-go-git variants and add gogit build tag

Signed-off-by: Andrew Thornton <art27@cantab.net>

* placate lint

Signed-off-by: Andrew Thornton <art27@cantab.net>

* as per @6543

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2020-12-17 22:00:47 +08:00
6543
9e456b5a56
HotFix: Hide private partisipation in Orgs (#13994)
* HotFix: Hide private partisipation in Orgs

* refactor & add node to fuc GetOrganizations
2020-12-17 00:39:12 +01:00
6543
e2aa701812
[API] Add ref to create/edit issue options & deprecated assignee (#13992)
* API: Add ref to create/edit issue options

* deprecate Assignee in favour of Assignees
2020-12-15 19:38:10 +01:00
bobemoe
b35c1b5784
add thumbnail preview section to issue attachments (#13826)
* add thumbnail preview section to attachments

* dont show thumbnail if the image is already shown inline

* update router to pass the `content` to the attachemnts template

* limit attachment preview height to 150px (same as width)

* remove unused css (referance removed in https://github.com/go-gitea/gitea/pull/11141/files#diff-9faae32445ed9673de2830c9fc35e93f44487f0a0068202988adaf00a5bac850L66 )

* dont show divider after edit if no attachemnts

Co-authored-by: James <inbox.dev@jhodges.co.uk>
Co-authored-by: 6543 <6543@obermui.de>
2020-12-13 21:12:27 +02:00
zeripath
287b594803
Whenever the password is updated ensure that the hash algorithm is too (#13966)
`user.HashPassword` may potentially - and in fact now likely does - change
the `passwd_hash_algo` therefore whenever the `passwd` is updated, this
also needs to be updated.

Fix #13832

Thanks @fblaese for the hint

Signed-off-by: Andrew Thornton <art27@cantab.net>
2020-12-12 19:59:49 +01:00
Lunny Xiao
9f100a45c6
Possible fix the webhook API creation (#13960)
* Possible fix the webhook API creation

* Fix api create webhook bug
2020-12-12 16:33:19 +01:00
Lunny Xiao
6edfa6bc88
Fix broken migration on webhook (#13911)
* Fix broken migration on webhook

* Fix lint

Co-authored-by: John Olheiser <john.olheiser@gmail.com>
2020-12-09 17:20:13 +00:00
Lunny Xiao
6a370363d4
log error when login failed (#13903)
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2020-12-08 21:37:47 +01:00
Eric Lesiuta
abb9cffe4a
Log IP on SSH authentication failure for Built-in SSH server (#13150)
* Log IP on SSH authentication failure

fixes https://github.com/go-gitea/gitea/issues/13094

* include string 'Failed authentication attempt' in error

* update fail2ban docs

also match failed authentication over command line

* better logging of authentication errors with IP addresses

* format ...

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: 6543 <6543@obermui.de>
2020-12-08 18:54:33 +01:00
Lunny Xiao
42354dfe45
Move webhook type from int to string (#13664)
* Move webhook type from int to string

* rename webhook_services

* finish refactor

* Fix merge

* Ignore unnecessary ci

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: zeripath <art27@cantab.net>
2020-12-08 11:41:14 +01:00
6543
48a3bb7a32
User Settings: Ignore empty language codes & validate (#13755) 2020-12-04 08:20:30 +02:00
silverwind
9269a038a4
Direct avatar rendering (#13649)
* Direct avatar rendering

This adds new template helpers for avatar rendering which output image
elements with direct links to avatars which makes them cacheable by the
browsers.

This should be a major performance improvment for pages with many avatars.

* fix avatars of other user's profile pages

* fix top border on user avatar name

* uncircle avatars

* remove old incomplete avatar selector

* use title attribute for name and add it back on blame

* minor refactor

* tweak comments

* fix url path join and adjust test to new result

* dedupe functions
2020-12-03 19:46:11 +01:00
John Olheiser
e306c292ac
Expose default theme in meta and API (#13809)
Signed-off-by: jolheiser <john.olheiser@gmail.com>

Co-authored-by: 6543 <6543@obermui.de>
2020-12-03 13:53:21 +01:00
6543
7ac8a770e1
Move Repo APIFormat to convert package (#13787)
* Move Repo APIFormat to convert package

* tweek
2020-12-02 16:38:30 -05:00
6543
4f5ff1ef08
move notification APIFormat (#13783) 2020-12-02 09:24:35 +00:00
silverwind
14e8ef9ecb
Flip RUN_MODE detection in router (#13768)
Missed that part in https://github.com/go-gitea/gitea/pull/13765. It's
already in the 1.13 backport so this forward-ports that change again.
2020-12-01 09:54:44 +08:00
6543
b2435af9be
Add Allow-/Block-List for Migrate & Mirrors (#13610)
* add black list and white list support for migrating repositories

* fix fmt

* fix lint

* fix vendor

* fix modules.txt

* clean diff

* specify log message

* use blocklist/allowlist

* allways use lowercase to match url

* Apply allow/block

* Settings: use existing "migrations" section

* convert domains lower case

* dont store unused value

* Block private addresses for migration by default

* fix lint

* use proposed-upstream func to detect private IP addr

* a nit

* add own error for blocked migration, add tests, imprufe api

* fix test

* fix-if-localhost-is-ipv4

* rename error & error message

* rename setting options

* Apply suggestions from code review

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2020-11-28 19:37:58 -05:00
6543
0f14f69e60
Verify password for local-account activation (#13631)
* Verify passwords for activation

This is to prevent 3rd party activation

* Fix function comment

* only veify password on local-account aktivation

* fix lint

* Update templates/user/auth/activate.tmpl

Co-authored-by: silverwind <me@silverwind.io>

Co-authored-by: Andreas Shimokawa <shimokawa@fsfe.org>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2020-11-28 17:41:06 -05:00
a1012112796
9c26dc1f3a
Add block on official review requests branch protection (#13705)
Signed-off-by: a1012112796 <1012112796@qq.com>

Co-authored-by: Lauris BH <lauris@nix.lv>
2020-11-28 21:30:46 +02:00
Simon
8c996c6505
Fix missing stylesheets on installation page (#13736)
When running gitea for the first time, the stylesheets for the
installation page are broken since the middleware that statically serves
stylesheets does not get executed by chi. This is because if no handlers
are registered in chi, it will drop all middleware.

This commit introduces a "dummy" handler to deal with that quirk.

Closes #13725

Thanks: Lunny Xiao <xiaolunwen@gmail.com> for finding the quirk

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2020-11-28 19:52:30 +02:00
a1012112796
1e5247d424
ui: show 'owner' tag for real owner (#13689)
* ui: show 'owner' tag for real owner

Signed-off-by: a1012112796 <1012112796@qq.com>

* Update custom/conf/app.example.ini

* simplify logic

fix logic
fix a small bug about original author

* remove system manager tag

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: Lauris BH <lauris@nix.lv>
2020-11-28 17:52:29 +02:00
zeripath
742e21aeba
Handle and propagate errors when checking if paths are Dirs, Files or Exist (#13186)
* Ensure errors from IsDir propagate

* Handle errors when checking IsFile

* Handle and propagate errors from IsExist

* Update modules/templates/static.go

* Update modules/templates/static.go

* Return after ctx.ServerError

* Apply suggestions from code review

* Fix tests

The previous merge managed to break repo_form.go

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: Lauris BH <lauris@nix.lv>
2020-11-27 21:42:08 -05:00
6543
f88a2eae97
[API] Add more filters to issues search (#13514)
* Add time filter for issue search

* Add limit option for paggination

* Add Filter for: Created by User, Assigned to User, Mentioning User

* update swagger

* Add Tests for limit, before & since
2020-11-23 20:49:36 +00:00
Lunny Xiao
c5f6f8f2f1
Refactor combine label comments with tests (#13619)
Co-authored-by: Lauris BH <lauris@nix.lv>
2020-11-20 17:29:09 -05:00
6543
1bb5c09b5d
API: Admin EditUser: Make FullName, Email, Website & Location optional (#13562)
* API: Admin EditUser: Make FullName, Email, Website & Location optional

* update swagger docs

* add Tests

Co-authored-by: Lauris BH <lauris@nix.lv>
2020-11-19 20:56:42 -05:00
Karl Heinz Marbaise
702e82d162
Fix issue/pull request list assignee filter (#13647)
* Fixes #13641 - Filtering in Pull Request kept all the time.
 - The URL contains all the time the assignee in cases
   where once a type has been selected.

Signed-off-by: Karl Heinz Marbaise <kama@soebes.de>

* Followup Fixes #13641 - Filtering in Pull Request kept all the time.
 - The URL contains all the time the assignee in cases
   where once a type has been selected.
 - The same behaviour was observed issues viewed via milestones.

Signed-off-by: Karl Heinz Marbaise <kama@soebes.de>
2020-11-19 15:39:55 -06:00
silverwind
12c2efb45c
Remove fetch request from heatmap (#13623)
* Remove fetch request from heatmap

Render heatmap data directly to HTML, eliminating one HTTP request on
frontpage and user profile. Also added min-height to the container so
the page content will no longer move after loading.

* rename and error display

* also log the js error

* add error handler

* remove useless inline style and hide divider on small screens

* Update routers/user/home.go

* Update routers/user/profile.go
2020-11-18 16:00:16 -06:00
Lunny Xiao
5a991ec519
Fix a bug when check if owner is active (#13612) 2020-11-18 11:58:25 +02:00
silverwind
0615b668dc
HTTP cache rework and enable caching for storage assets (#13569)
This enabled HTTP time-based cache for storage assets, primarily
avatars. I have not observed If-Modified-Since from browsers during
tests but I guess it's good to support regardless.

It introduces a new generic httpcache module that can handle both
time-based and etag-based caching.

Additionally, manifest.json and robots.txt are now also cachable.
2020-11-17 17:44:52 -05:00
Lunny Xiao
9ec5e6c40b
Move metrics from macaron to chi (#13601) 2020-11-17 15:50:06 -05:00
silverwind
b83825253e
Fix label display on new issues (#13590)
* Fix label display on new issues

PR #13570 broke label rendering for new issues and pulls because
missed the fact that the code was relying on the DOM elements being
toggled by JavaScript.

On top of that, the label rendering for new issues and pull was using
an outdated template which I consolidated in a new shared template.

* remove wrapper element and style tweaks

* style tweaks

* use shared template for whole label list

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2020-11-17 07:02:41 +02:00
Lunny Xiao
586bfb9f32
Use mount but not register for chi routes (#13555)
* Use mount but not register for chi routes

* Fix test

* Fix test

* Fix test

* Fix comment

* turn back unnecessary change

* Remove the timout middleware since some operations may spend much time.
2020-11-16 15:33:41 +08:00
wULLSnpAXbWZGYDYyhWTKKspEQoaYxXyhoisqHf
3ea237b333
Add markdown support in organization description (#13549)
Similarly to how you can write some markdown in user profile description, you can now use the same feature in organization description.
As discussed in discord.
kudos to @mrsdizzie and @zeripath 

visualization:
![image](https://user-images.githubusercontent.com/61180606/99115919-8004a200-25f3-11eb-8bbd-ab33de2cabb3.png)
2020-11-15 16:28:08 +00:00
Chris Shyi
d025d84d91
Add email validity check (#13475)
* Improve error feedback for duplicate deploy keys

Instead of a generic HTTP 500 error page, a flash message is rendered
with the deploy key page template so inform the user that a key with the
intended title already exists.

* API returns 422 error when key with name exists

* Add email validity checking

Add email validity checking for the following routes:
[Web interface]
1. User registration
2. User creation by admin
3. Adding an email through user settings
[API]
1. POST /admin/users
2. PATCH /admin/users/:username
3. POST /user/emails

* Add further tests

* Add signup email tests

* Add email validity check for linking existing account

* Address PR comments

* Remove unneeded DB session

* Move email check to updateUser

Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2020-11-14 11:53:43 -05:00
6543
7d2700c8be
[API] Only Return Json (#13511)
* Let Branch and Raw Endpoint return json error if not found

* Revert "RM RepoRefByTypeForAPI and move needed parts into GetRawFile directly"

This reverts commit d826d08577b23765cb3c257e7a861191d1aa9a04.

* more similar to RepoRefByType

* dedub-code

* API should just speak JSON

* nice name

Co-authored-by: zeripath <art27@cantab.net>
2020-11-14 11:13:55 -05:00
6543
3f3447a1ea
API: Fix GetQueryBeforeSince (#13559) 2020-11-14 14:05:40 +00:00
silverwind
374ff60465
Use monaco for the git hook editor (#13552)
Migrate git hook editor to monaco, replacing CodeMirror. Had to do a few
refactors to make the monaco instantiation generic enough to be of use.

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2020-11-13 22:57:34 -05:00
Lunny Xiao
c296f4fed6
Introduce go chi web framework as frontend of macaron, so that we can move routes from macaron to chi step by step (#7420)
* When route cannot be found on chi, go to macaron

* Stick chi version to 1.5.0

* Follow router log setting
2020-11-13 14:51:07 +02:00
Lunny Xiao
ff7341b994
Prevent git operations for inactive users (#13527)
* prevent git operations for inactive users

* Some fixes

* Deny push to the repositories which's owner is inactive

* deny operations also when user is ProhibitLogin

Co-authored-by: zeripath <art27@cantab.net>
2020-11-13 01:29:11 +02:00
Mohammad Hadi Hosseinpour
beab1dd337
Added title and action buttons to Project view page (#13437)
* Added title and action buttons to Project view page

* Changed octicon for project close action

* Fix indentation in templates/repo/projects/view.tmpl

Co-authored-by: zeripath <art27@cantab.net>

* Fix indentation templates/repo/projects/view.tmpl (2nd)

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: zeripath <art27@cantab.net>
2020-11-09 22:46:19 -05:00
zeripath
c05a8abc76
Multiple GitGraph improvements: Exclude PR heads, Add branch/PR links, Show only certain branches, (#12766)
* Multiple GitGraph improvements.

Add backend support for excluding PRs, selecting branches and files.

Fix #10327

Signed-off-by: Andrew Thornton <art27@cantab.net>

* as per @silverwind

Signed-off-by: Andrew Thornton <art27@cantab.net>

* as per @silverwind

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Only show refs in dropdown we display on the graph

Signed-off-by: Andrew Thornton <art27@cantab.net>

* as per @silverwind

Signed-off-by: Andrew Thornton <art27@cantab.net>

* use flexbox for ui header

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Move Hide Pull Request button to the dropdown

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Add SHA and user pictures

Signed-off-by: Andrew Thornton <art27@cantab.net>

* fix test

Signed-off-by: Andrew Thornton <art27@cantab.net>

* fix test 2

Signed-off-by: Andrew Thornton <art27@cantab.net>

* fixes

* async

* more tweaks

* use tabs in tmpl

Signed-off-by: Andrew Thornton <art27@cantab.net>

* remove commented thing

Signed-off-by: Andrew Thornton <art27@cantab.net>

* fix linting

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Update web_src/js/features/gitgraph.js

Co-authored-by: silverwind <me@silverwind.io>

* graph tweaks

* more tweaks

* add title

Signed-off-by: Andrew Thornton <art27@cantab.net>

* fix loading indicator z-index and position

Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: Lauris BH <lauris@nix.lv>
2020-11-08 12:21:54 -05:00
Kyle Evans
e461f0854f
[RFC] Make archival asynchronous (#11296)
* Make archival asynchronous

The prime benefit being sought here is for large archives to not
clog up the rendering process and cause unsightly proxy timeouts.
As a secondary benefit, archive-in-progress is moved out of the
way into a /tmp file so that new archival requests for the same
commit will not get fulfilled based on an archive that isn't yet
finished.

This asynchronous system is fairly primitive; request comes in, we'll
spawn off a new goroutine to handle it, then we'll mark it as done.
Status requests will see if the file exists in the final location,
and report the archival as done when it exists.

Fixes #11265

* Archive links: drop initial delay to three-quarters of a second

Some, or perhaps even most, archives will not take all that long to archive.
The archive process starts as soon as the download button is initially
clicked, so in theory they could be done quite quickly.  Drop the initial
delay down to three-quarters of a second to make it more responsive in the
common case of the archive being quickly created.

* archiver: restructure a little bit to facilitate testing

This introduces two sync.Cond pointers to the archiver package. If they're
non-nil when we go to process a request, we'll wait until signalled (at all)
to proceed. The tests will then create the sync.Cond so that it can signal
at-will and sanity-check the state of the queue at different phases.

The author believes that nil-checking these two sync.Cond pointers on every
archive processing will introduce minimal overhead with no impact on
maintainability.

* gofmt nit: no space around binary + operator

* services: archiver: appease golangci-lint, lock queueMutex

Locking/unlocking the queueMutex is allowed, but not required, for
Cond.Signal() and Cond.Broadcast().  The magic at play here is just a little
too much for golangci-lint, as we take the address of queueMutex and this is
mostly used in archiver.go; the variable still gets flagged as unused.

* archiver: tests: fix several timing nits

Once we've signaled a cond var, it may take some small amount of time for
the goroutines released to hit the spot we're wanting them to be at. Give
them an appropriate amount of time.

* archiver: tests: no underscore in var name, ungh

* archiver: tests: Test* is run in a separate context than TestMain

We must setup the mutex/cond variables at the beginning of any test that's
going to use it, or else these will be nil when the test is actually ran.

* archiver: tests: hopefully final tweak

Things got shuffled around such that we carefully build up and release
requests from the queue, so we can validate the state of the queue at each
step. Fix some assertions that no longer hold true as fallout.

* repo: Download: restore some semblance of previous behavior

When archival was made async, the GET endpoint was only useful if a previous
POST had initiated the download. This commit restores the previous behavior,
to an extent; we'll now submit the archive request there and return a
"202 Accepted" to indicate that it's processing if we didn't manage to
complete the request within ~2 seconds of submission.

This lets a client directly GET the archive, and gives them some indication
that they may attempt to GET it again at a later time.

* archiver: tests: simplify a bit further

We don't need to risk failure and use time.ParseDuration to get 2 *
time.Second.

else if isn't really necessary if the conditions are simple enough and lead
to the same result.

* archiver: tests: resolve potential source of flakiness

Increase all timeouts to 10 seconds; these aren't hard-coded sleeps, so
there's no guarantee we'll actually take that long. If we need longer to
not have a false-positive, then so be it.

While here, various assert.{Not,}Equal arguments are flipped around so that
the wording in error output reflects reality, where the expected argument is
second and actual third.

* archiver: setup infrastructure for notifying consumers of completion

This API will *not* allow consumers to subscribe to specific requests being
completed, just *any* request being completed. The caller is responsible for
determining if their request is satisfied and waiting again if needed.

* repo: archive: make GET endpoint synchronous again

If the request isn't complete, this endpoint will now submit the request and
wait for completion using the new API. This may still be susceptible to
timeouts for larger repos, but other endpoints now exist that the web
interface will use to negotiate its way through larger archive processes.

* archiver: tests: amend test to include WaitForCompletion()

This is a trivial one, so go ahead and include it.

* archiver: tests: fix test by calling NewContext()

The mutex is otherwise uninitialized, so we need to ensure that we're
actually initializing it if we plan to test it.

* archiver: tests: integrate new WaitForCompletion a little better

We can use this to wait for archives to come in, rather than spinning and
hoping with a timeout.

* archiver: tests: combine numQueued declaration with next-instruction assignment

* routers: repo: reap unused archiving flag from DownloadStatus()

This had some planned usage before, indicating whether this request
initiated the archival process or not. After several rounds of refactoring,
this use was deemed not necessary for much of anything and got boiled down
to !complete in all cases.

* services: archiver: restructure to use a channel

We now offer two forms of waiting for a request:
- WaitForCompletion: wait for completion with no timeout
- TimedWaitForCompletion: wait for completion with timeout

In both cases, we wait for the given request's cchan to close; in the latter
case, we do so with the caller-provided timeout. This completely removes the
need for busy-wait loops in Download/InitiateDownload, as it's fairly clean
to wait on a channel with timeout.

* services: archiver: use defer to unlock now that we can

This previously carried the lock into the goroutine, but an intermediate
step just added the request to archiveInProgress outside of the new
goroutine and removed the need for the goroutine to start out with it.

* Revert "archiver: tests: combine numQueued declaration with next-instruction assignment"

This reverts commit bcc52140238e16680f2e05e448e9be51372afdf5.

Revert "archiver: tests: integrate new WaitForCompletion a little better"

This reverts commit 9fc8bedb5667d24d3a3c7843dc28a229efffb1e6.

Revert "archiver: tests: fix test by calling NewContext()"

This reverts commit 709c35685eaaf261ebbb7d3420e3376a4ee8e7f2.

Revert "archiver: tests: amend test to include WaitForCompletion()"

This reverts commit 75261f56bc05d1fa8ff7e81dcbc0ccd93fdc9d50.

* archiver: tests: first attempt at WaitForCompletion() tests

* archiver: tests: slight improvement, less busy-loop

Just wait for the requests to complete in order, instead of busy-waiting
with a timeout.  This is slightly less fragile.

While here, reverse the arguments of a nearby assert.Equal() so that
expected/actual are correct in any test output.

* archiver: address lint nits

* services: archiver: only close the channel once

* services: archiver: use a struct{} for the wait channel

This makes it obvious that the channel is only being used as a signal,
rather than anything useful being piped through it.

* archiver: tests: fix expectations

Move the close of the channel into doArchive() itself; notably, before these
goroutines move on to waiting on the Release cond.

The tests are adjusted to reflect that we can't WaitForCompletion() after
they've already completed, as WaitForCompletion() doesn't indicate that
they've been released from the queue yet.

* archiver: tests: set cchan to nil for comparison

* archiver: move ctx.Error's back into the route handlers

We shouldn't be setting this in a service, we should just be validating the
request that we were handed.

* services: archiver: use regex to match a hash

This makes sure we don't try and use refName as a hash when it's clearly not
one, e.g. heads/pull/foo.

* routers: repo: remove the weird /archive/status endpoint

We don't need to do this anymore, we can just continue POSTing to the
archive/* endpoint until we're told the download's complete. This avoids a
potential naming conflict, where a ref could start with "status/"

* archiver: tests: bump reasonable timeout to 15s

* archiver: tests: actually release timedReq

* archiver: tests: run through inFlight instead of manually checking

While we're here, add a test for manually re-processing an archive that's
already been complete. Re-open the channel and mark it incomplete, so that
doArchive can just mark it complete again.

* initArchiveLinks: prevent default behavior from clicking

* archiver: alias gitea's context, golang context import pending

* archiver: simplify logic, just reconstruct slices

While the previous logic was perhaps slightly more efficient, the
new variant's readability is much improved.

* archiver: don't block shutdown on waiting for archive

The technique established launches a goroutine to do the wait,
which will close a wait channel upon termination. For the timeout
case, we also send back a value indicating whether the timeout was
hit or not.

The timeouts are expected to be relatively small, but still a multi-
second delay to shutdown due to this could be unfortunate.

* archiver: simplify shutdown logic

We can just grab the shutdown channel from the graceful manager instead of
constructing a channel to halt the caller and/or pass a result back.

* Style issues

* Fix mis-merge

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Lauris BH <lauris@nix.lv>
2020-11-07 22:27:28 +02:00