This PR cleans up the docs in a way to make them simpler to ingest by
our [docs repo](https://gitea.com/gitea/gitea-docusaurus).
1. It includes all of the sed invocations our ingestion did, removing
the need to do it at build time.
2. It replaces the shortcode variable replacement method with
`@variable@` style, simply for easier sed invocations when required.
3. It removes unused files and moves the docs up a level as cleanup.
---------
Signed-off-by: jolheiser <john.olheiser@gmail.com>
Move `public/*` to `public/assets/*`
Some old PRs (like #15219) introduced inconsistent directory system.
For example: why the local directory "public" is accessed by
`http://site/assets`? How to serve the ".well-known" files properly in
the public directory?
For convention rules, the "public" directory is widely used for the
website's root directory. It shouldn't be an exception for Gitea.
So, this PR makes the things consistent:
* `http://site/assets/foo` means `{CustomPath}/public/assets/foo`.
* `{CustomPath}/public/.well-known` and `{CustomPath}/public/robots.txt`
can be used in the future.
This PR is also a prerequisite for a clear solution for:
* #21942
* #25892
* discourse.gitea.io: [.well-known path serving custom files behind
proxy?](https://discourse.gitea.io/t/well-known-path-serving-custom-files-behind-proxy/5445/1)
This PR is breaking for users who have custom "public" files (CSS/JS).
After getting approvals, I will update the documents.
----
## ⚠️ BREAKING ⚠️
If you have files in your "custom/public/" folder, please move them to
"custom/public/assets/".
---------
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Giteabot <teabot@gitea.io>
- Update all JS dependencies
- Enable `declaration-property-unit-disallowed-list` to forbid `em` on
`line-height`
- Rename dependency update targets to `update-js` and `update-py` and
document them
- Remove margin on Asciicast viewer
- Tested Swagger, Katex, Asciicast
<img width="1243" alt="Screenshot 2023-06-27 at 19 51 05"
src="https://github.com/go-gitea/gitea/assets/115237/2d2722a0-2aa7-4f4c-b8bd-17e1f3637b78">
[updates](https://github.com/silverwind/updates) now supports poetry as
well so we can use it for a new `make poetry-update` to update all
poetry dependencies.
As title, this is how it was inferred before
e24f651c86/Makefile (L83-L84)
We moved to actions, however `GITEA_VERSION` should still work to use
`VERSION`
Signed-off-by: jolheiser <john.olheiser@gmail.com>
The [docker/build-push-action@v2
action](https://github.com/docker/build-push-action) by default ignores
the checkout created using the actions/checkout@v2 action. When you pass
a git build context to docker build, it wouldn't include the .git
directory.
By passing `context: .` to the build step then it'll use the Actions git
context which includes the git fetch from the earlier step.
So I found this [linter](https://github.com/Riverside-Healthcare/djlint)
which features a mode for go templates, so I gave it a try and it did
find a number of valid issue, like unbalanced tags etc. It also has a
number of bugs, I had to disable/workaround many issues.
Given that this linter is written in python, this does add a dependency
on `python` >= 3.8 and `poetry` to the development environment to be
able to run this linter locally.
- `e.g.` prefixes on placeholders are removed because the linter had a
false-positive on `placeholder="e.g. cn=Search"` for the `attr=value`
syntax and it's not ideal anyways to write `e.g.` into a placeholder
because a placeholder is meant to hold a sample value.
- In `templates/repo/settings/options.tmpl` I simplified the logic to
not conditionally create opening tags without closing tags because this
stuff confuses the linter (and possibly the reader as well).
It's been disabled by default since 1.17
(https://github.com/go-gitea/gitea/pull/18914), and it never really
delivered any benefit except being another cache layer that has its own
unsolved invalidation issues. HTTP cache works, we don't need two cache
layers at the browser for assets.
## ⚠️ BREAKING
You can remove the config `[ui].USE_SERVICE_WORKER` from your `app.ini`
now.
Inspired by
https://github.com/go-gitea/gitea/pull/24530#issuecomment-1558815301
This PR use a file filter action to do different CI jobs according
changed files types. All types are defined in
`.github/file-filters.yml`. Now there are 4 types, `docs`, `backend`,
`frontend` and `build`. Then if a PR only changed docs files, those CI
jobs which passed the conditions will run, and other types are also like
this.
---------
Co-authored-by: silverwind <me@silverwind.io>
- Remove actions name where command is descriptive enough
- Use kebab-case instead of snake-case for step names
- Use shorter job names because to make PR checks more readable
- Remove duplicate `checks-backend`
---------
Co-authored-by: Yarden Shoham <git@yardenshoham.com>
Proposal found here: https://github.com/go-gitea/gitea/issues/23654
TODO: make non-breaking (can we publish docker image using dev and
nightly prefix? at same time). if anyone has advice please comment :)
If this PR is merged, then I can add redirects to the downloads site.
- Update all tool dependencies to latest tag
- Remove unused errcheck, it is part of golangci-lint
- Include main.go in air
- Enable wastedassign again now that it's
[generics-compatible](https://github.com/golangci/golangci-lint/pull/3689)
- Restructured lint targets to new `lint-*` namespace
- Add new Make target `lint-md`
- Add new Drone pipeline `compliance-docs`
- Add `*.md` to docs exclusion/inclusion
- Consistently quote `path.include` and `path.exclude` statements in
YAML
* Clean the "tools" directory. The "tools" directory contains only two
files, move them.
* The "external_renderer.go" works like "cat" command to echo Stdin to
Stdout , to help testing.
* The `// gobuild: external_renderer` is incorrect, there should be no
space: `//gobuild: external_renderer`
* The `fmt.Print(os.Args[1])` is not a well-defined behavior, and it's
never used.
* The "watch.sh" is for "make watch", it's somewhat related to "build"
* After this PR, there is no "tools" directory, the project root
directory looks slightly simpler than before.
* Remove the legacy "contrib/autoboot.sh", there is no
"gogs_supervisord.sh"
* Remove the legacy "contrib/mysql.sql", it's never mentioned anywhere.
* Remove the legacy "contrib/pr/checkout.go", it has been broken for
long time, and it introduces unnecessary dependencies of the main code
base.
Refactored `deps-docs` out of `docs` because #23629 broke Gitpod's docs
setup (which grepped to get the hugo install command).
Now `make deps` really installs everything to develop (includes `docs`).
- **Installation**: includes how to install Gitea and related other
tools, also includes upgrade Gitea
- **Administration**: includes how to configure Gitea, customize Gitea
and manage Gitea instance out of Gitea admin UI
- **Usage**: includes how to use Gitea's functionalities. A sub
documentation is about packages, in future we could also include CI/CD
and others.
- **Development**: includes how to integrate with Gitea's API, how to
develop new features within Gitea
- **Contributing**: includes how to contribute code to Gitea
repositories.
After this is merged, I think we can have a sub-documentation of `Usage`
part named `Actions` to describe how to use Gitea actions
---------
Co-authored-by: John Olheiser <john.olheiser@gmail.com>
Ran most of the Less files through the Less compiler and Prettier and
then followed up with a round of manual fixes.
The Less compiler had unfortunately stripped all `//` style comments
that I had to restore (It did preserve `/* */` comments). Other fixes
include duplicate selector removal which were revealed after the
transpilation and which weren't caught by stylelint before but now are.
Fixes: https://github.com/go-gitea/gitea/issues/15565
hardcode the version of test_env we use in docker, so that we can use
different major versions of golang between versions of Gitea
---------
Co-authored-by: Lauris BH <lauris@nix.lv>
- This PR attempts to split our various DB tests into separate
pipelines.
- It splits up some of the extra feature-related tests rather than
having most of them in the MySQL test.
- It disables the race detector for some of the pipelines as well, as it
can cause slower runs and is mostly redundant when the pipelines just
swap DBs.
- It builds without SQLite support for any of the non-SQLite pipelines.
- It moves the e2e test to using SQLite rather than PG (partially
because I moved the minio tests to PG and that mucked up the test
config, and partially because it avoids another running service)
- It splits up the `go mod download` task in the Makefile from the tool
installation, as the tools are only needed in the compliance pipeline.
(Arguably even some of the tools aren't needed there, but that could be
a follow-up PR)
- SQLite is now the only arm64 pipeline, moving PG back to amd64 which
can leverage autoscaler
Should resolve #22010 - one thing that wasn't changed here but is
mentioned in that issue, unit tests are needed in the same pipeline as
an integration test in order to form a complete coverage report (at
least as far as I could tell), so for now it remains in a pipeline with
a DB integration test.
Please let me know if I've inadvertently changed something that was how
it was on purpose.
---
I will say sometimes it's hard to pin down the average time, as a
pipeline could be waiting for a runner for X minutes and that brings the
total up by X minutes as well, but overall this does seem to be faster
on average.
---------
Signed-off-by: jolheiser <john.olheiser@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Using `touchstart` for `click` events is a black magic for mobile
browsers (Google: `fastclick`).
However, it causes many UX problems if the fastclick is used without
careful design.
Fomantic UI uses this fastclick for its `dimmer` and `dropdown`, it
makes mobile users feel strange when they "touch" the dropdown menu.
This PR uses a simple patch to fix that behavior. Then the Fomantic
dropdown only uses `click` for click events.
This PR is simple enough and won't cause hidden bugs even if the patch
doesn't work. In the future, if there are more patches for Fomantic UI,
the patches could be placed in a directory like
`web_src/fomantic/patches/001-fix-click-touchstart`, etc.
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
This PR does a bulk update of a lot of our go deps.
I have not included nektos/act and xorm for the following reasons:
* Xorm updates can sometimes be complex and I'd rather do that in a
separate PR
* I think people more update with the actions code should double check
that the latest nektos/act library works correctly.
---------
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Fix #22370 and more.
Before Go 1.19, the `netgo` tag for Windows does nothing.
But Go 1.19 rewrite the net package code for Windows DNS, and there is a
bug:
* https://github.com/golang/go/issues/57757
This PR just removes the `netgo` tag for Windows build, then the Gitea
for Windows can have the old DNS behavior.
Since #20218 introduced internal watching template, template watching
should be removed from `air`. This will prevent restart the whole server
once the template files changed to speed up developing when using `make
watch`.
To ensure `make watch` will reuse template watching, this PR introduced
a new ENV `GITEA_RUN_MODE` to make sure `make watch` will always run in
a dev mode of Gitea so that template watching will open.
This PR also added more exclude testdata directories.
`fmt-check` now simply does `fmt` before and relies on `git diff` like
other checks like 'tidy-check' already do, so we can remove the argument
in the tool that handles printing changed files.
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Even if we are not bundling with `vite` yet, we can use `vitest` in
place of Jest which brings a few benefits like not requiring to use
`NODE_OPTIONS` to run and having sane module resolution.
It's possible to also use `jest-extended` with vitest, but I opted to
not do so for now because it brings heavyweight dependencies and it was
trivial to just rewrite the affected matchers to be compatible.
This PR also removes 153 JS dependencies, which is certainly nice.
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Running it as part of the build is really unnecessary because we have a
valid output file in the repo and assuming go dependencies do not change
unless go.mod also changes, tidy really is the best target to run the
license generation after.
Also, regenerate the file as I missed to do so during the chroma update,
and mark all json files in assets as generated.
- Update all JS dependencies and playwright image
- Add new eslint rules, enable a few more, fix issues
- Regenerate SVGs
- Tested Vue and Swagger
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
We can not have the `frontend` target depend on golang because of they
way drone is set up. Move the `go-licenses` generation back into `tidy`
where it will now also be checked for consistency during `tidy-check`.
(I assume all `main` branch builds should currently fail [like
this](https://drone.gitea.io/go-gitea/gitea/60244/1/11)).
The reasony why it shouldn't be treated the same as for example `go
generate` is because output files are checked in. tidy is imho the
optimal target to run this after.
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
This removes the JS dependency in the checks pipeline. JSON output is
different because the previous JS did indent the license data
differently and a JSON key was changed, but the end result is the same
as it gets re-indented by wepack.
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: zeripath <art27@cantab.net>
The `go-licenses` make task introduced in #21034 is being run on make vendor
and occasionally causes an empty go-licenses file if the vendors need to
change. This should be moved to the generate task as it is a generated file.
Now because of this change we also need to split generation into two separate
steps:
1. `generate-backend`
2. `generate-frontend`
In the future it would probably be useful to make `generate-swagger` part of `generate-frontend` but it's not tolerated with our .drone.yml
Ref #21034
Signed-off-by: Andrew Thornton <art27@cantab.net>
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: delvh <dev.lh@web.de>
`make go-licenses` will generate `assets/go-licenses.json` which is then included in the webpack build.
This step depends on both go and node being present, so unfortunately, I could not automate the generation by hooking it up to `tidy` as that target is triggered on CI where we do not have a docker image with both go an node.
It should be ran from time to time, ideally after each go mod update.
* Remove debug output when coverage fails
When coverage fails, it logs megabytes of debug to stdout, which seems
to break the drone ui as well as the log output download in drone,
presumably because of the size. I think with removal of this print, we
should still see any errors created by gocovmerge.go, but a few CI runs
may be necessary to get it to fail again.
* Update Makefile
* restart ci
* restart ci
* restart ci
* restart ci
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
This should fix some recently seen linter performance issues. There is some log spam, but it's definitely faster.
Ref: https://github.com/golangci/golangci-lint/issues/2997
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Introduce a separate .eslintrc in the Vue components folder to
selectively enable vue-eslint-parser there, so that the rest of the
files can use eslint's core parser which can deal with hashbangs.
The fact that the eslint-disable comments worked in HTML was a
unintended side-effect of the files being parsed via vue-eslint-parser,
so I had to disable the parsing of these files in .eslintrc.yaml to make
it work, and finally decided to remove eslint-plugin-html as it causes
more issues than it solves.
`printf` in busybox emits a ugly 'invalid number' error when formatting
string variables are present. Avoid that by reducing the go version
check to just two digits, which ought to be enough as patch-level go
versions are meant to be compatible. Avoid error on node-check as well.
[spectral](https://github.com/stoplightio/spectral) lints
openapi/swagger files for mistakes of which it has identified a few and
which I've fixed.
I had to put it into `lint-frontend` because it depends on node_modules
so can not run on Drone during the backend target. I plan to refactor
these targets later to `lint-js` and `lint-go` so that they are
categorized based on the tool dependencies.
- Given we use go1.18 for this and don't rely on the Go 1.11 modules
behavior(we use the modern `go run` & `go get` which has the correct
behavior by-default).
Ensures consistent line endings to avoid useless diffs because there is
somthing platform-dependant in that build.
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
- Remove __dirname, use file URLs instead
- Upgrade fabric dependency
- Use fs/promises syntax, this breaks node 12 but we require 14 already
The change in public/img/favicon.svg is not caused by the fabric
upgrade, but it seems it was not properly generated when introduced.
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Remove customized (unmaintained) dropdown, improve aria a11y for dropdown
* fix repo permission
* use action instead of onChange
* re-order the CSS selector
* fix dropdown behavior for repo permissions, make elements inside menu item non-focusable
* use menu/menuitem instead of combobox/option. use tooltip(data-content) for aria-label, prevent from repeated attaching
* click menu item when pressing Enter
* code format
* fix repo permission
* repo setting: prevent from misleading users when error occurs
* fine tune the repo collaboration access mode dropdown (in case the access mode is undefined in the template)
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Convenience command to run 'go mod tidy'. The compat flag is set to the
minimum supported go version instead of min version minus one which is
go's default, which eliminated some checksums in go.sum as a result.
* Update go tool dependencies
Updated all tool dependencies to latest tags, hoping CI will like it.
* fix new lint errors
* handle more strings.Title cases
* remove lint skip
* use go1.18 to build gitea& update min go version to 1.17
* bump in a few more places
* add a few simple tests for isipprivate
* update go.mod
* update URL to https://go.dev/dl/
* golangci-lint
* attempt golangci-lint workaround
* change version
* bump fumpt version
* skip strings.title test
* go mod tidy
* update tests as some aren't private??
* update tests
This ensures the tools only run in the versions we've tested and it also
does not polute PATH with those tools so they are truly isolated. This
syntax of `go run` requires go 1.17, so the minimum version is set
accordingly.
Fixes: https://github.com/go-gitea/gitea/issues/18867
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
We can't depend on `latest` version of gofumpt because the output will
not be stable across versions. Lock it down to the latest version
released yesterday and run it again.
* Move editorconfig-checker to lint-backend
It makes more sense there as templates are considered backend code.
* User golang version of the tool
* remove dependency
Unfortunately credentialIDs in u2f are 255 bytes long which with base32 encoding
becomes 408 bytes. The default size of a xorm string field is only a VARCHAR(255)
This problem is not apparent on SQLite because strings get mapped to TEXT there.
Fix #18727
Signed-off-by: Andrew Thornton <art27@cantab.net>
The CI currently downloads all go modules in each pipeline step because
go modules reside outside the project directory. Fix this by introducing
a volume for the `/go` directory [1] so modules are only downloaded once per
pipeline using a new `deps-backend` make target.
For completeness, I also included new `deps` and `deps-frontend` targets
and the frontend one is also triggered explicitly on CI where needed.
[1] https://docs.drone.io/pipeline/kubernetes/examples/language/golang/#dependencies
* Run 'make fmt'
'make fmt' currently produces this change, I'm not sure how CI did not
fail on it, I made sure I have `mvdan.cc/gofumpt@latest`.
* Fix 'make fmt-check'
`make fmt-check` did not run all commands that `make fmt` did, resulting
in missed diffs. Fix that by just depending on the `fmt` target.
Includes: https://github.com/go-gitea/gitea/pull/18633
* Make gitea-fmt work with -l and -d and integrate gofumpt
This implements -l, -w and -d with gitea-fmt and merges gofumpt.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* as per silverwind
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Apply suggestions from code review
* use -l instead of -d for fmt-check
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Andrew Thornton <art27@cantab.net>
Pretty minor change to prevent error when `$(MAKE)` path is expanded with a space in the path.
```bash
$ TAGS="bindata sqlite sqlite_unlock_notify" make build
/usr/bin/sh: -c: line 0: syntax error near unexpected token `('
/usr/bin/sh: -c: line 0: `C:/Program Files (x86)/GnuWin32/bin/make -v | head -n 1'
```
I believe Program Files (x86) is the default path for GNU make on windows
* Add lockfile-check
This check runs `npm install` which will rewrite the lockfile in case it
is inconsistent with package.json. This check detects this and will fail
the CI in such a case.
* Add new JS linter rules
Adds a few useful rules from eslint-plugin-github. Notable changes:
- Forbid dataset usage, its camel-casing behaviour makes it hard to
grep for attributes.
- Forbid .then() and .catch(), we should generally prefer await for new
code. For rare cases where they are useful, a eslint-disable-line
directive can be set.
- Add docs js to linting
* also enable github/array-foreach
* small tweak
Co-authored-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
* Update golangci-lint in Makefile
- Partially resolvess #17596
- Download specific version(v1.43.0) by default.
- If current installed version is older than the minium version, it will
download the mininium required version.
- Update the install script to avoid deprecated error
`golangci/golangci-lint err this script is deprecated, please do not use
it anymore. check https://github.com/goreleaser/godownloader/issues/207`
* Simplify golangci-lint version check
* Fix version conversion
* Add version that's downloading
Co-authored-by: zeripath <art27@cantab.net>
* Consistency
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Set unit test timeout to 15 minutes
* Update Makefile
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Workaround coverage bug part 2
Just grep away bad lines from coverage files.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* try again
Signed-off-by: Andrew Thornton <art27@cantab.net>
