Commit graph

2324 commits

Author SHA1 Message Date
mrsdizzie
4563eb873d
Support unicode emojis and remove emojify.js (#11032)
* Support unicode emojis and remove emojify.js

This PR replaces all use of emojify.js and adds unicode emoji support to various areas of gitea.

This works in a few ways:

First it adds emoji parsing support into gitea itself. This allows us to

 * Render emojis from valid alias (😄)
 * Detect unicode emojis and let us put them in their own class with proper aria-labels and styling
 * Easily allow for custom "emoji"
 * Support all emoji rendering and features without javascript
 * Uses plain unicode and lets the system render in appropriate emoji font
 * Doesn't leave us relying on external sources for updates/fixes/features

That same list of emoji is also used to create a json file which replaces the part of emojify.js that populates the emoji search tribute. This file is about 35KB with GZIP turned on and I've set it to load after the page renders to not hinder page load time (and this removes loading emojify.js also)

For custom "emoji" it uses a pretty simple scheme of just looking for /emojis/img/name.png where name is something a user has put in the "allowed reactions" setting we already have. The gitea reaction that was previously hard coded into a forked copy of emojify.js is included and works as a custom reaction under this method.

The emoji data sourced here is from https://github.com/github/gemoji which is the gem library Github uses for their emoji rendering (and a data source for other sites). So we should be able to easily render any emoji and :alias: that Github can, removing any errors from migrated content. They also update it as well, so we can sync when there are new unicode emoji lists released.

I've included a slimmed down and slightly modified forked copy of https://github.com/knq/emoji to make up our own emoji module. The code is pretty straight forward and again allows us to have a lot of flexibility in what happens.

I had seen a few comments about performance in some of the other threads if we render this ourselves, but there doesn't seem to be any issue here. In a test it can parse, convert, and render 1,000 emojis inside of a large markdown table in about 100ms on my laptop (which is many more emojis than will ever be in any normal issue). This also prevents any flickering and other weirdness from using javascript to render some things while using go for others.

Not included here are image fall back URLS. I don't really think they are necessary for anything new being written in 2020. However, managing the emoji ourselves would allow us to add these as a feature later on if it seems necessary.

Fixes: https://github.com/go-gitea/gitea/issues/9182
Fixes: https://github.com/go-gitea/gitea/issues/8974
Fixes: https://github.com/go-gitea/gitea/issues/8953
Fixes: https://github.com/go-gitea/gitea/issues/6628
Fixes: https://github.com/go-gitea/gitea/issues/5130

* add new shared function emojiHTML

* don't increase emoji size in issue title

* Update templates/repo/issue/view_content/add_reaction.tmpl

Co-Authored-By: 6543 <6543@obermui.de>

* Support for emoji rendering in various templates

* Render code and review comments as they should be

* Better way to handle mail subjects

* insert unicode from tribute selection

* Add template helper for plain text when needed

* Use existing replace function I forgot about

* Don't include emoji greater than Unicode Version 12

Only include emoji and aliases in JSON

* Update build/generate-emoji.go

* Tweak regex slightly to really match everything including random invisible characters. Run tests for every emoji we have

* final updates

* code review

* code review

* hard code gitea custom emoji to match previous behavior

* Update .eslintrc

Co-Authored-By: silverwind <me@silverwind.io>

* disable preempt

Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
2020-04-28 15:05:39 -03:00
mrsdizzie
922a239079
Disable new signal-based asynchronous goroutine preemption from GO 1.14 in git env (#11237)
As seen in trouble shooting #11032 the new feature of Go 1.14 is causing several second delays in startup in certain situations. Debugging shows it spending several seconds handling SIGURG commands during init:

```
6922:04:51.984234 trace init() ./modules/queue/unique_queue_wrapped.go
remote: ) = 69 <0.000012>
remote: [pid 15984] 22:04:51 write(1, "\ttime taken: 236.761\302\265s\n\n", 25    time taken: 236.761µs
remote:
remote: ) = 25 <0.000011>
remote: [pid 15984] 22:04:51 --- SIGURG {si_signo=SIGURG, si_code=SI_TKILL, si_pid=15984, si_uid=0} ---
remote: [pid 15984] 22:04:52 --- SIGURG {si_signo=SIGURG, si_code=SI_TKILL, si_pid=15984, si_uid=0} ---
remote: [pid 15984] 22:04:52 --- SIGURG {si_signo=SIGURG, si_code=SI_TKILL, si_pid=15984, si_uid=0} ---
```

This causes up to 20 seconds added to a push in some cases as it happens for each call of the gitea hook command. This is likely the cause of #10661 as well and would start to effect users once we release 1.12 which would be the first release compiled with Go 1.14. I suspect this is just a slight issue with the upstream implementatation as there have been a few very similar bugs fixed and reported:

 https://github.com/golang/go/issues/37741
 https://github.com/golang/go/issues/37942

We should revisit this in the future and see if a newer version of Go has solved it, but for now disable this option in the environment that gitea hook runs in to avoid it.
2020-04-28 11:45:32 -04:00
zeripath
0e799c26ba
Prevent panic during wrappedConn close at hammertime (#11219)
* Prevent panic during wrappedConn close at hammertime

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Update modules/graceful/server.go

* Fix extraneous debug in goldmark.go

Signed-off-by: Andrew Thornton <art27@cantab.net>
2020-04-26 21:01:06 -03:00
zeripath
9f959ac064
Make TaskCheckBox render correctly (#11214)
* Fix checkbox rendering

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Normalize checkbox rendering

Signed-off-by: Andrew Thornton <art27@cantab.net>

* set the checkboxes to readonly instead of disabled

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: Lauris BH <lauris@nix.lv>
2020-04-26 02:09:08 -03:00
6543
4ddfe0d07a
Fix GetContents(): Dont't ignore Executables (#11192)
* Refactor: dont expose help functions

* repofiles GetContents: dont ignore executables

* CI.restart()
2020-04-24 19:20:22 +03:00
zeripath
812cfd0ad9
Use markdown frontmatter to provide Table of contents, language and frontmatter rendering (#11047)
* Add control for the rendering of the frontmatter
* Add control to include a TOC
* Add control to set language - allows control of ToC header and CJK glyph choice.

Signed-off-by: Andrew Thornton art27@cantab.net
2020-04-24 14:22:36 +01:00
赵智超
d3fc9c08c8
[ui] Change icon type for review action in action page (#11191)
* as title, do same changs on action view with #10737
* chage default icon from "invalid type" to "question" , because  "invalid type" is not a meaningfull icon type

Signed-off-by: a1012112796 <1012112796@qq.com>
2020-04-24 01:58:14 -03:00
zeripath
b10c416f9e
Use AJAX for notifications table (#10961)
* Use AJAX for notifications table

Signed-off-by: Andrew Thornton <art27@cantab.net>

* move to separate js

Signed-off-by: Andrew Thornton <art27@cantab.net>

* placate golangci-lint

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Add autoupdating notification count

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Fix wipeall

Signed-off-by: Andrew Thornton <art27@cantab.net>

* placate tests

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Try hidden

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Try hide and hidden

Signed-off-by: Andrew Thornton <art27@cantab.net>

* More auto-update improvements

Only run checker on pages that have a count
Change starting checker to 10s with a back-off to 60s if there is no change

Signed-off-by: Andrew Thornton <art27@cantab.net>

* string comparison!

Signed-off-by: Andrew Thornton <art27@cantab.net>

* as per @silverwind

Signed-off-by: Andrew Thornton <art27@cantab.net>

* add configurability as per @6543

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Add documentation as per @6543

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Use CSRF header not query

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Further JS improvements

Fix @etzelia update notification table request
Fix @silverwind comments

Co-Authored-By: silverwind <me@silverwind.io>
Signed-off-by: Andrew Thornton <art27@cantab.net>

* Simplify the notification count fns

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: silverwind <me@silverwind.io>
2020-04-24 00:57:38 -03:00
zeripath
e74c4e1be9
Add option to increase provided OAuth2 token maximum size (#11180)
Some OAuth2 providers return quite large structured tokens >32767 bytes.
Gitea currently has a fixed maximum of 32767 bytes for these and
unfortunately due to the convoluted nature of the dependent libraries the
error returned is rather opaque.

Here we manage the error a little better - detecting the rather opaque
github.com/gorilla/securecookie.errEncodedValueTooLong and converting
it to a more readable error.

Further we provide a configurable option to increase the maximum size of
the provided OAuth2 tokens.

Fix #9907

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2020-04-23 00:47:23 +02:00
zeripath
b51fd30522
Log the indexer path on failure (#11172)
Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Lauris BH <lauris@nix.lv>
2020-04-22 16:16:58 -04:00
Lunny Xiao
fcc8cdd446
Improve config logging when WrappedQueue times out (#11174)
Before
```sh
Unable to set the internal queue for -wrapper Error: Timedout creating queue redis with cfg []byte{0x7b, 0x22, 0x41, 0x64, 0x64, 0x72, 0x65, 0x73, 0x73, 0x65, 0x73, 0x22, 0x3a, 0x22, 0x31, 0x32, 0x37, 0x2e, 0x30, 0x2e, 0x30, 0x2e, 0x31, 0x3a, 0x36, 0x33, 0x37, 0x39, 0x22, 0x2c, 0x22, 0x42, 0x61, 0x74, 0x63, 0x68, 0x4c, 0x65, 0x6e, 0x67, 0x74, 0x68, 0x22, 0x3a, 0x32, 0x30, 0x2c, 0x22, 0x42, 0x6c, 0x6f, 0x63, 0x6b, 0x54, 0x69, 0x6d, 0x65, 0x6f, 0x75, 0x74, 0x22, 0x3a, 0x31, 0x30, 0x30, 0x30, 0x30, 0x30, 0x30, 0x30, 0x30, 0x30, 0x2c, 0x22, 0x42, 0x6f, 0x6f, 0x73, 0x74, 0x54, 0x69, 0x6d, 0x65, 0x6f, 0x75, 0x74, 0x22, 0x3a, 0x33, 0x30, 0x30, 0x30, 0x30, 0x30
......
```

After
```sh
Unable to set the internal queue for -wrapper Error: Timedout creating queue redis with cfg "{\"Addresses\":\"127.0.0.1:6379\",\"BatchLength\":20,\"BlockTimeout\":1000000000,\"BoostTimeout\":300000000000,\"BoostWorkers\":5,\"DBIndex\":0,\"DataDir\":\".../data/queues/mail\",\"MaxWorkers\":10,\"Name\":\"mail\",\"Network\":\"\",\"Password\":\"\",\"QueueLength\":20,\"QueueName\":\"mail_queue\",\"SetName\":\"\",\"Workers\":1}" in
```
2020-04-22 13:38:40 +01:00
zeripath
4ee70a9ec9
Fix submodule paths when AppSubUrl is not root (#11098)
Fix submodule paths when AppSubUrl is not root

Fix #11002

Signed-off-by: Andrew Thornton <art27@cantab.net>
2020-04-22 11:34:50 +01:00
silverwind
7e20f1cb5b
enable ENABLE_HARD_LINE_BREAK by default (#11162)
enable ENABLE_HARD_LINE_BREAK by default

This matches GitHub's rendering.

Fix #11155
2020-04-21 23:13:56 +01:00
6543
5bfb9bc2b6
When migrating from Gitlab map Approvals to approving Reviews (#11147)
When migrating from Gitlab map Gitlab Approvals to approving Reviews

Co-Authored-By: zeripath <art27@cantab.net>
2020-04-20 13:30:46 +01:00
6543
e37e217602
[Fix] RetryDownloader (add GetReviews()) (#11093)
* add GetReviews() to RetryDownloader
2020-04-20 03:23:28 +01:00
6543
82dbb34c9c
Vendor Update: go-gitlab v0.22.1 -> v0.31.0 (#11136)
* vendor update: go-gitlab to v0.31.0

* migrate client init to v0.31.0

* refactor
2020-04-19 21:23:05 +01:00
Jordan
5c092eb0ef
Add support for migrating from Gitlab (#9084)
* First stab at a Gitlab migrations interface.

* Modify JS to show migration for Gitlab

* Properly strip out #gitlab tag from repo name

* Working Gitlab migrations!
Still need to figure out how to hide tokens/etc from showing up in opts.CloneAddr

* Try #2 at trying to hide credentials.
CloneAddr was being used as OriginalURL.
Now passing OriginalURL through from the form and saving it.

* Add go-gitlab dependency

* Vendor go-gitlab

* Use gitlab.BasicAuthClient
Correct CloneURL.
This should be functioning!
Previous commits fixed "Migrated from"
from including the migration credentials.

* Replaced repoPath with repoID globally.
RepoID is grabbed in NewGitlabDownloader

* Logging touchup

* Properly set private repo status.
Properly set milestone deadline time.
Consistently use Gitlab username for 'Name'.

* Add go-gitlab vendor cache

* Fix PR migrations:
- Count of issues is kept to set a non-conflicting PR.ID
- Bool is used to tell whether to fetch Issue or PR comments

* Ensure merged PRs are closed and set with the proper time

* Remove copyright and some commented code

* Rip out '#gitlab' based self-hosted Gitlab support

* Hide given credentials for migrated repos.

CloneAddr was being saved as OriginalURL.

Now passing OriginalURL through from the form and
saving it in it's place

* Use asset.URL directly, no point in parsing.
Opened PRs should fall through to false.

* Fix importing Milestones.
Allow importing using Personal Tokens or anonymous access.

* Fix Gitlab Milestone migration if DueDate isn't set

* Empty Milestone due dates properly return nil, not zero time

* Add GITLAB_READ_TOKEN to drone unit-test step

* Add working gitlab_test.go.
A Personal Access Token, given in env variable GITLAB_READ_TOKEN
is required to run the test.

* Fix linting issues

* Add modified JS files

* Remove pre-build JS files

* Only merged PRs are marged as merged/closed

* Test topics

* Skip test if gitlab is inaccessible

* Grab personal token from username, not password.
Matches Github migration implementation

* Add SetContext() to GitlabDownloader.

* Checking Updated field in Issues.

* Actually fetch Issue Updated time from Gitlab

* Add Gitlab migration GetReviews() stub

* Fix Patch and Clone URLs

* check Updated too

* fix mod

* make vendor with go1.14

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2020-04-19 23:44:11 +08:00
silverwind
5180deb819
Send 404 immediately for known public requests (#11117)
Instead of further handling requests to public which causes issues like #11088, immediately terminate requests to directories js, css, fomantic if no file is found which is checked against a hardcoded list. Maybe there is a way to retrieve the top-level entries below public in a dynamic fashion.

I also added fomantic to the reserved usernames and sorted the list.

Fixes: #11088
2020-04-18 22:01:06 +01:00
Cornel
6034f8bcaa
Avoid 0 new commits messages to be send (#11082)
Fixes #10498
2020-04-18 21:02:59 +01:00
John Olheiser
4f597b1866
Add single release page and latest redirect (#11102)
* Add single release and latest release routes

Signed-off-by: jolheiser <john.olheiser@gmail.com>

* Update API and move latest search to models

Signed-off-by: jolheiser <john.olheiser@gmail.com>

* Fix swagger

Signed-off-by: jolheiser <john.olheiser@gmail.com>

Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
2020-04-18 11:47:15 -03:00
6543
12960b9d18
[BugFix] remove nil inserts in models (#11096)
* Fix InsertReleases Nil Insert on Attachments

* FIX "No element on slice when insert" & smal refactor

* again

* impruve

* rm useles newline

* Apply suggestions from code review

Co-Authored-By: zeripath <art27@cantab.net>

* process insert as a whole

Co-authored-by: zeripath <art27@cantab.net>
2020-04-17 20:42:57 +03:00
6543
c52d48aae4
Prevent merge of outdated PRs on protected branches (#11012)
* Block PR on Outdated Branch

* finalize

* cleanup

* fix typo and sentences

thanks @guillep2k

Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com>

Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
Co-authored-by: Lauris BH <lauris@nix.lv>
2020-04-16 22:00:36 -03:00
Lauris BH
4dc62dadce
Migrate to go-enry new version (#10906) 2020-04-15 20:40:39 +03:00
zeripath
7c48085ff4
Remove check on username if AccessToken authentication (#11015)
Signed-off-by: Andrew Thornton <art27@cantab.net>
2020-04-14 13:32:03 -05:00
zeripath
0be25e2150
Handle yaml frontmatter (#11016)
Add goldmark-meta to render yaml frontmatter as a table

Fix #5377

Signed-off-by: Andrew Thornton <art27@cantab.net>
2020-04-09 11:54:50 +01:00
L0veSunshine
08208cdfb8
Fix forked repo has no icon and language stat. (#10791)
* Update header.tmpl

* Update _repository.less

* Update pull.go

* Update indexer.go

* Update pull.go

* Update pull.go

* Update indexer.go

* fix forked repo doesn't have language stat

* fix forked repo doesn't have language stat

* fix forked repo doesn't have language stat

* fix forked repo doesn't have language stat

* fix forked repo doesn't have language stat

* fix forked repo doesn't have language stat

* fix forked repo doesn't have language stat

* fix forked repo doesn't have language stat

* fix forked repo doesn't have language stat

* fix forked repo doesn't have language stat

* fix forked repo doesn't have language stat

Thanks the advice from lunny and 6543.

* fix forked repo doesn't have language stat

Modified as suggested by 6543.

* fix forked repo doesn't have language stat

simplify the function.

* fix forked repo doesn't have language stat

update the indexer after the loop

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: Lauris BH <lauris@nix.lv>
2020-04-08 15:13:04 +03:00
6543
3d63caa542
[API] Get a single commit via Ref (#10915)
* GET /repos/:owner/:repo/commits/:ref

* add Validation Checks

* Fix & Extend TEST

* add two new tast cases
2020-04-07 22:54:46 -04:00
techknowlogick
d00ebf445b
upgrade to most recent bluemonday (#11007)
* upgrade to most recent bluemonday

* make vendor

* update tests for bluemonday

* update tests for bluemonday

* update tests for bluemonday
2020-04-07 23:08:47 +03:00
j.yao.SUSE
b252b23f16
[feature] provide OwnerName field for README template (#10981)
Co-Authored-By: 6543 <6543@obermui.de>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2020-04-06 21:40:38 -04:00
6543
ad31d6b5e8
Fix release counter on API repository info (#10968)
Use the same mechanism as the webpage for calculating the release count

Fix #10946
2020-04-06 19:42:30 +01:00
赵智超
ef89e75d0e
add request review from specific reviewers feature in pull request (#10756)
* add request review feature in pull request

add a way to notify specific reviewers to review like github , by add  or delet a  special type
review . The acton is  is similar to Assign ,  so many code reuse the function and items of
Assignee, but the meaning and result is different.

The Permission style is is similar to github, that only writer can add a review request from Reviewers,
but the poster can recall and remove a review request after a reviwer has revied even if he don't have
Write Premission. only manager , the poster and reviewer of a request review can remove it.

The reviewers can be requested to review contain all readers for private repo , for public, contain
all writers and watchers.

The offical Review Request will block merge if Reject can block it.

an other change: add ui otify for Assignees.

Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
Co-authored-by: Lauris BH <lauris@nix.lv>

Signed-off-by: a1012112796 <1012112796@qq.com>

* new change

* add placeholder string

* do some changes follow #10238 to add review requests num on lists also
change icon for review requests to eye

Co-authored-by: Lauris BH <lauris@nix.lv>
2020-04-06 19:33:34 +03:00
zeripath
d26885e2bf
Mulitple Gitea Doctor improvements (#10943)
* Add `gitea doctor --list` flag to list the checks that will be run, including those by default
* Add `gitea doctor --run` to run specific checks
* Add `gitea doctor --all` to run all checks
* Add db version checker
* Add non-default recalculate merge bases check/fixer to doctor
* Add hook checker (Fix #9878) and ensure hooks are executable (Fix #6319)
* Fix authorized_keys checker - slight change of functionality here because parsing the command is fragile and we should just check if the authorized_keys file is essentially the same as what gitea would produce. (This is still not perfect as order matters - we should probably just md5sum the two files.)
* Add SCRIPT_TYPE check (Fix #10977)
* Add `gitea doctor --fix` to attempt to fix what is possible to easily fix
* Add `gitea doctor --log-file` to set the log-file, be it a file, stdout or to switch off completely. (Fixes previously undetected bug with certain xorm logging configurations - see @6543 comment.)

Signed-off-by: Andrew Thornton <art27@cantab.net>
2020-04-06 11:44:47 +01:00
John Olheiser
baadb51445
Add gitea-vet (#10948)
* Add copyright

Signed-off-by: jolheiser <john.olheiser@gmail.com>

* Add gitea-vet and fix non-compliance

Signed-off-by: jolheiser <john.olheiser@gmail.com>

* Combine tools.go into build.go and clean up

Signed-off-by: jolheiser <john.olheiser@gmail.com>

* Remove extra GO111MODULE=on

Signed-off-by: jolheiser <john.olheiser@gmail.com>
2020-04-05 07:20:50 +01:00
Vasil Mikhalenya
240258a3e5
Avoiding directory execution on hook (#10954) (#10955)
* test -x is not enough https://stackoverflow.com/a/39489087
2020-04-04 11:29:58 -05:00
Lunny Xiao
4f63f283c4
Rename scripts to build and add revive command as a new build tool command (#10942)
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2020-04-03 22:29:12 +03:00
mrsdizzie
8d99ee2773
Add Organization Wide Labels (#10814)
* Add organization wide labels

Implement organization wide labels similar to organization wide
webhooks. This lets you create individual labels for organizations that can be used
for all repos under that organization (so being able to reuse the same
label across multiple repos).

This makes it possible for small organizations with many repos to use
labels effectively.

Fixes #7406

* Add migration

* remove comments

* fix tests

* Update options/locale/locale_en-US.ini

Removed unused translation string

* show org labels in issue search label filter

* Use more clear var name

* rename migration after merge from master

* comment typo

* update migration again after rebase with master

* check for orgID <=0 per guillep2k review

* fmt

* Apply suggestions from code review

Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com>

* remove unused code

* Make sure RepoID is 0 when searching orgID per code review

* more changes/code review requests

* More descriptive translation var per code review

* func description/delete comment when issue label deleted instead of hiding it

* remove comment

* only use issues in that repo when calculating number of open issues for org label on repo label page

* Add integration test for IssuesSearch API with labels

* remove unused function

* Update models/issue_label.go

Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com>

* Use subquery in GetLabelIDsInReposByNames

* Fix tests to use correct orgID

* fix more tests

* IssuesSearch api now uses new BuildLabelNamesIssueIDsCondition. Add a few more tests as well

* update comment for clarity

* Revert previous code change now that we can use the new BuildLabelNamesIssueIDsCondition

* Don't sort repos by date in IssuesSearch API

After much debugging I've found a strange issue where in some cases MySQL will return a different result than other enigines if a query is sorted by a null collumn. For example with our integration test data where we don't set updated_unix in repository fixtures:

SELECT `id`, `owner_id`, `owner_name`, `lower_name`, `name`, `description`, `website`, `original_service_type`, `original_url`, `default_branch`, `num_watches`, `num_stars`, `num_forks`, `num_issues`, `num_closed_issues`, `num_pulls`, `num_closed_pulls`, `num_milestones`, `num_closed_milestones`, `is_private`, `is_empty`, `is_archived`, `is_mirror`, `status`, `is_fork`, `fork_id`, `is_template`, `template_id`, `size`, `is_fsck_enabled`, `close_issues_via_commit_in_any_branch`, `topics`, `avatar`, `created_unix`, `updated_unix` FROM `repository` ORDER BY updated_unix DESC LIMIT 15 OFFSET 45

Returns different results for MySQL than other engines. However, the similar query:

SELECT `id`, `owner_id`, `owner_name`, `lower_name`, `name`, `description`, `website`, `original_service_type`, `original_url`, `default_branch`, `num_watches`, `num_stars`, `num_forks`, `num_issues`, `num_closed_issues`, `num_pulls`, `num_closed_pulls`, `num_milestones`, `num_closed_milestones`, `is_private`, `is_empty`, `is_archived`, `is_mirror`, `status`, `is_fork`, `fork_id`, `is_template`, `template_id`, `size`, `is_fsck_enabled`, `close_issues_via_commit_in_any_branch`, `topics`, `avatar`, `created_unix`, `updated_unix` FROM `repository` ORDER BY updated_unix DESC LIMIT 15 OFFSET 30

Returns the same results.

This causes integration tests to fail on MySQL in certain cases but would never show up in a real installation. Since this API call always returns issues based on the optionally provided repo_priority_id or the issueID itself, there is no change to results by changing the repo sorting method used to get ids earlier in the function.

* linter is back!

* code review

* remove now unused option

* Fix newline at end of files

* more unused code

* update to master

* check for matching ids before query

* Update models/issue_label.go

Co-Authored-By: 6543 <6543@obermui.de>

* Update models/issue_label.go

* update comments

* Update routers/org/setting.go

Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
Co-authored-by: 6543 <6543@obermui.de>
2020-04-01 01:14:46 -03:00
zeripath
bf847b9397
Upgrade golangci-lint to 1.24.0 (#10894)
* upgrade golangci-lint to 1.24.0 to allow go 1.14 compatibility
* fix golangci-lint errors
* make make golangci-lint work when out of go-path

Signed-off-by: Andrew Thornton <art27@cantab.net>
2020-03-30 21:52:45 +03:00
zeripath
e83daf77ad
Avoid logging []byte in queue failures - convert to string first (#10865)
Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
2020-03-29 15:12:15 +08:00
zeripath
052bff0cff
Convert plumbing.ErrObjectNotFound to git.ErrNotExist in getCommit (#10862)
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
Co-authored-by: Antoine GIRARD <sapk@users.noreply.github.com>
2020-03-28 20:59:21 +02:00
S7evinK
828a27feeb
Add Matrix webhook (#10831)
* Add Matrix webhook

Signed-off-by: Till Faelligen <tfaelligen@gmail.com>

* Add template and related translations for Matrix hook

Signed-off-by: Till Faelligen <tfaelligen@gmail.com>

* Add actual webhook routes and form

Signed-off-by: Till Faelligen <tfaelligen@gmail.com>

* Add missing file

Signed-off-by: Till Faelligen <tfaelligen@gmail.com>

* Update modules/webhook/matrix_test.go

* Use stricter regex to replace URLs

Signed-off-by: Till Faelligen <tfaelligen@gmail.com>

* Escape url and text

Signed-off-by: Till Faelligen <tfaelligen@gmail.com>

* Remove unnecessary whitespace

* Fix copy and paste mistake

Co-Authored-By: Tulir Asokan <tulir@maunium.net>

* Fix indention inconsistency

* Use Authorization header instead of url parameter

* Add raw commit information to webhook

Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: Tulir Asokan <tulir@maunium.net>
2020-03-28 15:09:55 +02:00
zeripath
7cd47046ea
Handle push rejection in branch and upload (#10854)
* Handle push rejections and push out-of-date in branch creation and
file upload.
* Remove the duplicated sanitize from services/pull/merge
* Move the errors Err(Merge)PushOutOfDate and ErrPushRejected to
modules/git
* Handle errors better in the upload file dialogs

Fix #10460

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
2020-03-28 01:13:18 -03:00
zeripath
e6baa656f7
make avatar lookup occur at image request (#10540)
speed up page generation by making avatar lookup occur at the browser
not at page generation

* Protect against evil email address ".."

* hash the complete email address

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-Authored-By: Lauris BH <lauris@nix.lv>
2020-03-27 14:34:39 +02:00
zeripath
cf4f2cfa3e
Fix internal server error on checkboxes (#10845)
Annoyingly goldmarks SetAttributeString requires that
the value of the attribute is still a []byte but does
not make it clear in the documentation.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2020-03-26 23:47:34 +00:00
Lauris BH
bbd910ed1b
Allow to set protected file patterns that can not be changed under no conditions (#10806)
Co-Authored-By: zeripath <art27@cantab.net>
2020-03-27 00:26:34 +02:00
zeripath
52cfd2743c
Option to set default branch at repository creation (#10803)
* Option to set default branch at repository creation
* Handle template repos with non-default master branch
* Add DefaultBranch handling on creation to API

Fix #9542

Signed-off-by: Andrew Thornton <art27@cantab.net>
2020-03-26 19:14:51 +00:00
zeripath
311ed516ce
log.Fatal on failure to listen to SSH port (#10795)
Fix #10792
2020-03-23 07:59:38 +00:00
zeripath
f036de5a94
Completely fix task-list checkbox styling (#10798)
Continuing on from #10668 this PR makes goldmark apply the task-list styling to task-lists.
2020-03-22 22:25:38 +00:00
Lunny Xiao
dcaa5643d7
Fix branch api canPush and canMerge (#10776)
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2020-03-20 23:41:33 -04:00
Lunny Xiao
83fe7d414e
Fix bug on branch API (#10767) 2020-03-19 11:39:08 -04:00
Lauris BH
43c09134a9
Migrate to go-git/go-git v5.0.0 (#10735) 2020-03-17 12:19:58 -04:00