2022-05-07 22:35:52 +05:30
|
|
|
// Copyright 2021 Gitea. All rights reserved.
|
2022-11-27 23:50:29 +05:30
|
|
|
// SPDX-License-Identifier: MIT
|
2022-05-07 22:35:52 +05:30
|
|
|
|
|
|
|
package automerge
|
|
|
|
|
|
|
|
import (
|
|
|
|
"context"
|
|
|
|
"errors"
|
|
|
|
"fmt"
|
|
|
|
"strconv"
|
|
|
|
"strings"
|
|
|
|
|
2022-05-08 19:16:34 +05:30
|
|
|
"code.gitea.io/gitea/models/db"
|
2022-06-13 15:07:59 +05:30
|
|
|
issues_model "code.gitea.io/gitea/models/issues"
|
2022-05-11 15:39:36 +05:30
|
|
|
access_model "code.gitea.io/gitea/models/perm/access"
|
2022-05-07 22:35:52 +05:30
|
|
|
pull_model "code.gitea.io/gitea/models/pull"
|
|
|
|
repo_model "code.gitea.io/gitea/models/repo"
|
|
|
|
user_model "code.gitea.io/gitea/models/user"
|
|
|
|
"code.gitea.io/gitea/modules/git"
|
Simplify how git repositories are opened (#28937)
## Purpose
This is a refactor toward building an abstraction over managing git
repositories.
Afterwards, it does not matter anymore if they are stored on the local
disk or somewhere remote.
## What this PR changes
We used `git.OpenRepository` everywhere previously.
Now, we should split them into two distinct functions:
Firstly, there are temporary repositories which do not change:
```go
git.OpenRepository(ctx, diskPath)
```
Gitea managed repositories having a record in the database in the
`repository` table are moved into the new package `gitrepo`:
```go
gitrepo.OpenRepository(ctx, repo_model.Repo)
```
Why is `repo_model.Repository` the second parameter instead of file
path?
Because then we can easily adapt our repository storage strategy.
The repositories can be stored locally, however, they could just as well
be stored on a remote server.
## Further changes in other PRs
- A Git Command wrapper on package `gitrepo` could be created. i.e.
`NewCommand(ctx, repo_model.Repository, commands...)`. `git.RunOpts{Dir:
repo.RepoPath()}`, the directory should be empty before invoking this
method and it can be filled in the function only. #28940
- Remove the `RepoPath()`/`WikiPath()` functions to reduce the
possibility of mistakes.
---------
Co-authored-by: delvh <dev.lh@web.de>
2024-01-28 01:39:51 +05:30
|
|
|
"code.gitea.io/gitea/modules/gitrepo"
|
2022-05-07 22:35:52 +05:30
|
|
|
"code.gitea.io/gitea/modules/graceful"
|
|
|
|
"code.gitea.io/gitea/modules/log"
|
|
|
|
"code.gitea.io/gitea/modules/process"
|
|
|
|
"code.gitea.io/gitea/modules/queue"
|
|
|
|
pull_service "code.gitea.io/gitea/services/pull"
|
|
|
|
)
|
|
|
|
|
|
|
|
// prAutoMergeQueue represents a queue to handle update pull request tests
|
Rewrite queue (#24505)
# ⚠️ Breaking
Many deprecated queue config options are removed (actually, they should
have been removed in 1.18/1.19).
If you see the fatal message when starting Gitea: "Please update your
app.ini to remove deprecated config options", please follow the error
messages to remove these options from your app.ini.
Example:
```
2023/05/06 19:39:22 [E] Removed queue option: `[indexer].ISSUE_INDEXER_QUEUE_TYPE`. Use new options in `[queue.issue_indexer]`
2023/05/06 19:39:22 [E] Removed queue option: `[indexer].UPDATE_BUFFER_LEN`. Use new options in `[queue.issue_indexer]`
2023/05/06 19:39:22 [F] Please update your app.ini to remove deprecated config options
```
Many options in `[queue]` are are dropped, including:
`WRAP_IF_NECESSARY`, `MAX_ATTEMPTS`, `TIMEOUT`, `WORKERS`,
`BLOCK_TIMEOUT`, `BOOST_TIMEOUT`, `BOOST_WORKERS`, they can be removed
from app.ini.
# The problem
The old queue package has some legacy problems:
* complexity: I doubt few people could tell how it works.
* maintainability: Too many channels and mutex/cond are mixed together,
too many different structs/interfaces depends each other.
* stability: due to the complexity & maintainability, sometimes there
are strange bugs and difficult to debug, and some code doesn't have test
(indeed some code is difficult to test because a lot of things are mixed
together).
* general applicability: although it is called "queue", its behavior is
not a well-known queue.
* scalability: it doesn't seem easy to make it work with a cluster
without breaking its behaviors.
It came from some very old code to "avoid breaking", however, its
technical debt is too heavy now. It's a good time to introduce a better
"queue" package.
# The new queue package
It keeps using old config and concept as much as possible.
* It only contains two major kinds of concepts:
* The "base queue": channel, levelqueue, redis
* They have the same abstraction, the same interface, and they are
tested by the same testing code.
* The "WokerPoolQueue", it uses the "base queue" to provide "worker
pool" function, calls the "handler" to process the data in the base
queue.
* The new code doesn't do "PushBack"
* Think about a queue with many workers, the "PushBack" can't guarantee
the order for re-queued unhandled items, so in new code it just does
"normal push"
* The new code doesn't do "pause/resume"
* The "pause/resume" was designed to handle some handler's failure: eg:
document indexer (elasticsearch) is down
* If a queue is paused for long time, either the producers blocks or the
new items are dropped.
* The new code doesn't do such "pause/resume" trick, it's not a common
queue's behavior and it doesn't help much.
* If there are unhandled items, the "push" function just blocks for a
few seconds and then re-queue them and retry.
* The new code doesn't do "worker booster"
* Gitea's queue's handlers are light functions, the cost is only the
go-routine, so it doesn't make sense to "boost" them.
* The new code only use "max worker number" to limit the concurrent
workers.
* The new "Push" never blocks forever
* Instead of creating more and more blocking goroutines, return an error
is more friendly to the server and to the end user.
There are more details in code comments: eg: the "Flush" problem, the
strange "code.index" hanging problem, the "immediate" queue problem.
Almost ready for review.
TODO:
* [x] add some necessary comments during review
* [x] add some more tests if necessary
* [x] update documents and config options
* [x] test max worker / active worker
* [x] re-run the CI tasks to see whether any test is flaky
* [x] improve the `handleOldLengthConfiguration` to provide more
friendly messages
* [x] fine tune default config values (eg: length?)
## Code coverage:
![image](https://user-images.githubusercontent.com/2114189/236620635-55576955-f95d-4810-b12f-879026a3afdf.png)
2023-05-08 17:19:59 +05:30
|
|
|
var prAutoMergeQueue *queue.WorkerPoolQueue[string]
|
2022-05-07 22:35:52 +05:30
|
|
|
|
|
|
|
// Init runs the task queue to that handles auto merges
|
|
|
|
func Init() error {
|
Improve queue and logger context (#24924)
Before there was a "graceful function": RunWithShutdownFns, it's mainly
for some modules which doesn't support context.
The old queue system doesn't work well with context, so the old queues
need it.
After the queue refactoring, the new queue works with context well, so,
use Golang context as much as possible, the `RunWithShutdownFns` could
be removed (replaced by RunWithCancel for context cancel mechanism), the
related code could be simplified.
This PR also fixes some legacy queue-init problems, eg:
* typo : archiver: "unable to create codes indexer queue" => "unable to
create repo-archive queue"
* no nil check for failed queues, which causes unfriendly panic
After this PR, many goroutines could have better display name:
![image](https://github.com/go-gitea/gitea/assets/2114189/701b2a9b-8065-4137-aeaa-0bda2b34604a)
![image](https://github.com/go-gitea/gitea/assets/2114189/f1d5f50f-0534-40f0-b0be-f2c9daa5fe92)
2023-05-26 13:01:55 +05:30
|
|
|
prAutoMergeQueue = queue.CreateUniqueQueue(graceful.GetManager().ShutdownContext(), "pr_auto_merge", handler)
|
2022-05-07 22:35:52 +05:30
|
|
|
if prAutoMergeQueue == nil {
|
Improve queue and logger context (#24924)
Before there was a "graceful function": RunWithShutdownFns, it's mainly
for some modules which doesn't support context.
The old queue system doesn't work well with context, so the old queues
need it.
After the queue refactoring, the new queue works with context well, so,
use Golang context as much as possible, the `RunWithShutdownFns` could
be removed (replaced by RunWithCancel for context cancel mechanism), the
related code could be simplified.
This PR also fixes some legacy queue-init problems, eg:
* typo : archiver: "unable to create codes indexer queue" => "unable to
create repo-archive queue"
* no nil check for failed queues, which causes unfriendly panic
After this PR, many goroutines could have better display name:
![image](https://github.com/go-gitea/gitea/assets/2114189/701b2a9b-8065-4137-aeaa-0bda2b34604a)
![image](https://github.com/go-gitea/gitea/assets/2114189/f1d5f50f-0534-40f0-b0be-f2c9daa5fe92)
2023-05-26 13:01:55 +05:30
|
|
|
return fmt.Errorf("unable to create pr_auto_merge queue")
|
2022-05-07 22:35:52 +05:30
|
|
|
}
|
Improve queue and logger context (#24924)
Before there was a "graceful function": RunWithShutdownFns, it's mainly
for some modules which doesn't support context.
The old queue system doesn't work well with context, so the old queues
need it.
After the queue refactoring, the new queue works with context well, so,
use Golang context as much as possible, the `RunWithShutdownFns` could
be removed (replaced by RunWithCancel for context cancel mechanism), the
related code could be simplified.
This PR also fixes some legacy queue-init problems, eg:
* typo : archiver: "unable to create codes indexer queue" => "unable to
create repo-archive queue"
* no nil check for failed queues, which causes unfriendly panic
After this PR, many goroutines could have better display name:
![image](https://github.com/go-gitea/gitea/assets/2114189/701b2a9b-8065-4137-aeaa-0bda2b34604a)
![image](https://github.com/go-gitea/gitea/assets/2114189/f1d5f50f-0534-40f0-b0be-f2c9daa5fe92)
2023-05-26 13:01:55 +05:30
|
|
|
go graceful.GetManager().RunWithCancel(prAutoMergeQueue)
|
2022-05-07 22:35:52 +05:30
|
|
|
return nil
|
|
|
|
}
|
|
|
|
|
|
|
|
// handle passed PR IDs and test the PRs
|
Rewrite queue (#24505)
# ⚠️ Breaking
Many deprecated queue config options are removed (actually, they should
have been removed in 1.18/1.19).
If you see the fatal message when starting Gitea: "Please update your
app.ini to remove deprecated config options", please follow the error
messages to remove these options from your app.ini.
Example:
```
2023/05/06 19:39:22 [E] Removed queue option: `[indexer].ISSUE_INDEXER_QUEUE_TYPE`. Use new options in `[queue.issue_indexer]`
2023/05/06 19:39:22 [E] Removed queue option: `[indexer].UPDATE_BUFFER_LEN`. Use new options in `[queue.issue_indexer]`
2023/05/06 19:39:22 [F] Please update your app.ini to remove deprecated config options
```
Many options in `[queue]` are are dropped, including:
`WRAP_IF_NECESSARY`, `MAX_ATTEMPTS`, `TIMEOUT`, `WORKERS`,
`BLOCK_TIMEOUT`, `BOOST_TIMEOUT`, `BOOST_WORKERS`, they can be removed
from app.ini.
# The problem
The old queue package has some legacy problems:
* complexity: I doubt few people could tell how it works.
* maintainability: Too many channels and mutex/cond are mixed together,
too many different structs/interfaces depends each other.
* stability: due to the complexity & maintainability, sometimes there
are strange bugs and difficult to debug, and some code doesn't have test
(indeed some code is difficult to test because a lot of things are mixed
together).
* general applicability: although it is called "queue", its behavior is
not a well-known queue.
* scalability: it doesn't seem easy to make it work with a cluster
without breaking its behaviors.
It came from some very old code to "avoid breaking", however, its
technical debt is too heavy now. It's a good time to introduce a better
"queue" package.
# The new queue package
It keeps using old config and concept as much as possible.
* It only contains two major kinds of concepts:
* The "base queue": channel, levelqueue, redis
* They have the same abstraction, the same interface, and they are
tested by the same testing code.
* The "WokerPoolQueue", it uses the "base queue" to provide "worker
pool" function, calls the "handler" to process the data in the base
queue.
* The new code doesn't do "PushBack"
* Think about a queue with many workers, the "PushBack" can't guarantee
the order for re-queued unhandled items, so in new code it just does
"normal push"
* The new code doesn't do "pause/resume"
* The "pause/resume" was designed to handle some handler's failure: eg:
document indexer (elasticsearch) is down
* If a queue is paused for long time, either the producers blocks or the
new items are dropped.
* The new code doesn't do such "pause/resume" trick, it's not a common
queue's behavior and it doesn't help much.
* If there are unhandled items, the "push" function just blocks for a
few seconds and then re-queue them and retry.
* The new code doesn't do "worker booster"
* Gitea's queue's handlers are light functions, the cost is only the
go-routine, so it doesn't make sense to "boost" them.
* The new code only use "max worker number" to limit the concurrent
workers.
* The new "Push" never blocks forever
* Instead of creating more and more blocking goroutines, return an error
is more friendly to the server and to the end user.
There are more details in code comments: eg: the "Flush" problem, the
strange "code.index" hanging problem, the "immediate" queue problem.
Almost ready for review.
TODO:
* [x] add some necessary comments during review
* [x] add some more tests if necessary
* [x] update documents and config options
* [x] test max worker / active worker
* [x] re-run the CI tasks to see whether any test is flaky
* [x] improve the `handleOldLengthConfiguration` to provide more
friendly messages
* [x] fine tune default config values (eg: length?)
## Code coverage:
![image](https://user-images.githubusercontent.com/2114189/236620635-55576955-f95d-4810-b12f-879026a3afdf.png)
2023-05-08 17:19:59 +05:30
|
|
|
func handler(items ...string) []string {
|
|
|
|
for _, s := range items {
|
2022-05-07 22:35:52 +05:30
|
|
|
var id int64
|
|
|
|
var sha string
|
Rewrite queue (#24505)
# ⚠️ Breaking
Many deprecated queue config options are removed (actually, they should
have been removed in 1.18/1.19).
If you see the fatal message when starting Gitea: "Please update your
app.ini to remove deprecated config options", please follow the error
messages to remove these options from your app.ini.
Example:
```
2023/05/06 19:39:22 [E] Removed queue option: `[indexer].ISSUE_INDEXER_QUEUE_TYPE`. Use new options in `[queue.issue_indexer]`
2023/05/06 19:39:22 [E] Removed queue option: `[indexer].UPDATE_BUFFER_LEN`. Use new options in `[queue.issue_indexer]`
2023/05/06 19:39:22 [F] Please update your app.ini to remove deprecated config options
```
Many options in `[queue]` are are dropped, including:
`WRAP_IF_NECESSARY`, `MAX_ATTEMPTS`, `TIMEOUT`, `WORKERS`,
`BLOCK_TIMEOUT`, `BOOST_TIMEOUT`, `BOOST_WORKERS`, they can be removed
from app.ini.
# The problem
The old queue package has some legacy problems:
* complexity: I doubt few people could tell how it works.
* maintainability: Too many channels and mutex/cond are mixed together,
too many different structs/interfaces depends each other.
* stability: due to the complexity & maintainability, sometimes there
are strange bugs and difficult to debug, and some code doesn't have test
(indeed some code is difficult to test because a lot of things are mixed
together).
* general applicability: although it is called "queue", its behavior is
not a well-known queue.
* scalability: it doesn't seem easy to make it work with a cluster
without breaking its behaviors.
It came from some very old code to "avoid breaking", however, its
technical debt is too heavy now. It's a good time to introduce a better
"queue" package.
# The new queue package
It keeps using old config and concept as much as possible.
* It only contains two major kinds of concepts:
* The "base queue": channel, levelqueue, redis
* They have the same abstraction, the same interface, and they are
tested by the same testing code.
* The "WokerPoolQueue", it uses the "base queue" to provide "worker
pool" function, calls the "handler" to process the data in the base
queue.
* The new code doesn't do "PushBack"
* Think about a queue with many workers, the "PushBack" can't guarantee
the order for re-queued unhandled items, so in new code it just does
"normal push"
* The new code doesn't do "pause/resume"
* The "pause/resume" was designed to handle some handler's failure: eg:
document indexer (elasticsearch) is down
* If a queue is paused for long time, either the producers blocks or the
new items are dropped.
* The new code doesn't do such "pause/resume" trick, it's not a common
queue's behavior and it doesn't help much.
* If there are unhandled items, the "push" function just blocks for a
few seconds and then re-queue them and retry.
* The new code doesn't do "worker booster"
* Gitea's queue's handlers are light functions, the cost is only the
go-routine, so it doesn't make sense to "boost" them.
* The new code only use "max worker number" to limit the concurrent
workers.
* The new "Push" never blocks forever
* Instead of creating more and more blocking goroutines, return an error
is more friendly to the server and to the end user.
There are more details in code comments: eg: the "Flush" problem, the
strange "code.index" hanging problem, the "immediate" queue problem.
Almost ready for review.
TODO:
* [x] add some necessary comments during review
* [x] add some more tests if necessary
* [x] update documents and config options
* [x] test max worker / active worker
* [x] re-run the CI tasks to see whether any test is flaky
* [x] improve the `handleOldLengthConfiguration` to provide more
friendly messages
* [x] fine tune default config values (eg: length?)
## Code coverage:
![image](https://user-images.githubusercontent.com/2114189/236620635-55576955-f95d-4810-b12f-879026a3afdf.png)
2023-05-08 17:19:59 +05:30
|
|
|
if _, err := fmt.Sscanf(s, "%d_%s", &id, &sha); err != nil {
|
|
|
|
log.Error("could not parse data from pr_auto_merge queue (%v): %v", s, err)
|
2022-05-07 22:35:52 +05:30
|
|
|
continue
|
|
|
|
}
|
|
|
|
handlePull(id, sha)
|
|
|
|
}
|
|
|
|
return nil
|
|
|
|
}
|
|
|
|
|
2022-06-13 15:07:59 +05:30
|
|
|
func addToQueue(pr *issues_model.PullRequest, sha string) {
|
Rewrite queue (#24505)
# ⚠️ Breaking
Many deprecated queue config options are removed (actually, they should
have been removed in 1.18/1.19).
If you see the fatal message when starting Gitea: "Please update your
app.ini to remove deprecated config options", please follow the error
messages to remove these options from your app.ini.
Example:
```
2023/05/06 19:39:22 [E] Removed queue option: `[indexer].ISSUE_INDEXER_QUEUE_TYPE`. Use new options in `[queue.issue_indexer]`
2023/05/06 19:39:22 [E] Removed queue option: `[indexer].UPDATE_BUFFER_LEN`. Use new options in `[queue.issue_indexer]`
2023/05/06 19:39:22 [F] Please update your app.ini to remove deprecated config options
```
Many options in `[queue]` are are dropped, including:
`WRAP_IF_NECESSARY`, `MAX_ATTEMPTS`, `TIMEOUT`, `WORKERS`,
`BLOCK_TIMEOUT`, `BOOST_TIMEOUT`, `BOOST_WORKERS`, they can be removed
from app.ini.
# The problem
The old queue package has some legacy problems:
* complexity: I doubt few people could tell how it works.
* maintainability: Too many channels and mutex/cond are mixed together,
too many different structs/interfaces depends each other.
* stability: due to the complexity & maintainability, sometimes there
are strange bugs and difficult to debug, and some code doesn't have test
(indeed some code is difficult to test because a lot of things are mixed
together).
* general applicability: although it is called "queue", its behavior is
not a well-known queue.
* scalability: it doesn't seem easy to make it work with a cluster
without breaking its behaviors.
It came from some very old code to "avoid breaking", however, its
technical debt is too heavy now. It's a good time to introduce a better
"queue" package.
# The new queue package
It keeps using old config and concept as much as possible.
* It only contains two major kinds of concepts:
* The "base queue": channel, levelqueue, redis
* They have the same abstraction, the same interface, and they are
tested by the same testing code.
* The "WokerPoolQueue", it uses the "base queue" to provide "worker
pool" function, calls the "handler" to process the data in the base
queue.
* The new code doesn't do "PushBack"
* Think about a queue with many workers, the "PushBack" can't guarantee
the order for re-queued unhandled items, so in new code it just does
"normal push"
* The new code doesn't do "pause/resume"
* The "pause/resume" was designed to handle some handler's failure: eg:
document indexer (elasticsearch) is down
* If a queue is paused for long time, either the producers blocks or the
new items are dropped.
* The new code doesn't do such "pause/resume" trick, it's not a common
queue's behavior and it doesn't help much.
* If there are unhandled items, the "push" function just blocks for a
few seconds and then re-queue them and retry.
* The new code doesn't do "worker booster"
* Gitea's queue's handlers are light functions, the cost is only the
go-routine, so it doesn't make sense to "boost" them.
* The new code only use "max worker number" to limit the concurrent
workers.
* The new "Push" never blocks forever
* Instead of creating more and more blocking goroutines, return an error
is more friendly to the server and to the end user.
There are more details in code comments: eg: the "Flush" problem, the
strange "code.index" hanging problem, the "immediate" queue problem.
Almost ready for review.
TODO:
* [x] add some necessary comments during review
* [x] add some more tests if necessary
* [x] update documents and config options
* [x] test max worker / active worker
* [x] re-run the CI tasks to see whether any test is flaky
* [x] improve the `handleOldLengthConfiguration` to provide more
friendly messages
* [x] fine tune default config values (eg: length?)
## Code coverage:
![image](https://user-images.githubusercontent.com/2114189/236620635-55576955-f95d-4810-b12f-879026a3afdf.png)
2023-05-08 17:19:59 +05:30
|
|
|
log.Trace("Adding pullID: %d to the pull requests patch checking queue with sha %s", pr.ID, sha)
|
|
|
|
if err := prAutoMergeQueue.Push(fmt.Sprintf("%d_%s", pr.ID, sha)); err != nil {
|
2022-05-07 22:35:52 +05:30
|
|
|
log.Error("Error adding pullID: %d to the pull requests patch checking queue %v", pr.ID, err)
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
// ScheduleAutoMerge if schedule is false and no error, pull can be merged directly
|
2022-06-13 15:07:59 +05:30
|
|
|
func ScheduleAutoMerge(ctx context.Context, doer *user_model.User, pull *issues_model.PullRequest, style repo_model.MergeStyle, message string) (scheduled bool, err error) {
|
2022-11-13 01:48:50 +05:30
|
|
|
err = db.WithTx(ctx, func(ctx context.Context) error {
|
2022-05-08 19:16:34 +05:30
|
|
|
lastCommitStatus, err := pull_service.GetPullRequestCommitStatusState(ctx, pull)
|
|
|
|
if err != nil {
|
|
|
|
return err
|
|
|
|
}
|
2022-05-07 22:35:52 +05:30
|
|
|
|
2022-05-08 19:16:34 +05:30
|
|
|
// we don't need to schedule
|
|
|
|
if lastCommitStatus.IsSuccess() {
|
|
|
|
return nil
|
|
|
|
}
|
|
|
|
|
|
|
|
if err := pull_model.ScheduleAutoMerge(ctx, doer, pull.ID, style, message); err != nil {
|
|
|
|
return err
|
|
|
|
}
|
|
|
|
scheduled = true
|
2022-05-07 22:35:52 +05:30
|
|
|
|
2022-06-13 15:07:59 +05:30
|
|
|
_, err = issues_model.CreateAutoMergeComment(ctx, issues_model.CommentTypePRScheduledToAutoMerge, pull, doer)
|
2022-05-08 19:16:34 +05:30
|
|
|
return err
|
2022-11-13 01:48:50 +05:30
|
|
|
})
|
2022-06-20 15:32:49 +05:30
|
|
|
return scheduled, err
|
2022-05-08 19:16:34 +05:30
|
|
|
}
|
|
|
|
|
|
|
|
// RemoveScheduledAutoMerge cancels a previously scheduled pull request
|
2022-06-13 15:07:59 +05:30
|
|
|
func RemoveScheduledAutoMerge(ctx context.Context, doer *user_model.User, pull *issues_model.PullRequest) error {
|
2022-11-13 01:48:50 +05:30
|
|
|
return db.WithTx(ctx, func(ctx context.Context) error {
|
2022-05-08 19:16:34 +05:30
|
|
|
if err := pull_model.DeleteScheduledAutoMerge(ctx, pull.ID); err != nil {
|
|
|
|
return err
|
|
|
|
}
|
|
|
|
|
2022-06-13 15:07:59 +05:30
|
|
|
_, err := issues_model.CreateAutoMergeComment(ctx, issues_model.CommentTypePRUnScheduledToAutoMerge, pull, doer)
|
2022-05-08 19:16:34 +05:30
|
|
|
return err
|
2022-11-13 01:48:50 +05:30
|
|
|
})
|
2022-05-07 22:35:52 +05:30
|
|
|
}
|
|
|
|
|
|
|
|
// MergeScheduledPullRequest merges a previously scheduled pull request when all checks succeeded
|
|
|
|
func MergeScheduledPullRequest(ctx context.Context, sha string, repo *repo_model.Repository) error {
|
2022-06-13 15:07:59 +05:30
|
|
|
pulls, err := getPullRequestsByHeadSHA(ctx, sha, repo, func(pr *issues_model.PullRequest) bool {
|
2022-05-07 22:35:52 +05:30
|
|
|
return !pr.HasMerged && pr.CanAutoMerge()
|
|
|
|
})
|
|
|
|
if err != nil {
|
|
|
|
return err
|
|
|
|
}
|
|
|
|
|
|
|
|
for _, pr := range pulls {
|
|
|
|
addToQueue(pr, sha)
|
|
|
|
}
|
|
|
|
|
|
|
|
return nil
|
|
|
|
}
|
|
|
|
|
2022-06-13 15:07:59 +05:30
|
|
|
func getPullRequestsByHeadSHA(ctx context.Context, sha string, repo *repo_model.Repository, filter func(*issues_model.PullRequest) bool) (map[int64]*issues_model.PullRequest, error) {
|
Simplify how git repositories are opened (#28937)
## Purpose
This is a refactor toward building an abstraction over managing git
repositories.
Afterwards, it does not matter anymore if they are stored on the local
disk or somewhere remote.
## What this PR changes
We used `git.OpenRepository` everywhere previously.
Now, we should split them into two distinct functions:
Firstly, there are temporary repositories which do not change:
```go
git.OpenRepository(ctx, diskPath)
```
Gitea managed repositories having a record in the database in the
`repository` table are moved into the new package `gitrepo`:
```go
gitrepo.OpenRepository(ctx, repo_model.Repo)
```
Why is `repo_model.Repository` the second parameter instead of file
path?
Because then we can easily adapt our repository storage strategy.
The repositories can be stored locally, however, they could just as well
be stored on a remote server.
## Further changes in other PRs
- A Git Command wrapper on package `gitrepo` could be created. i.e.
`NewCommand(ctx, repo_model.Repository, commands...)`. `git.RunOpts{Dir:
repo.RepoPath()}`, the directory should be empty before invoking this
method and it can be filled in the function only. #28940
- Remove the `RepoPath()`/`WikiPath()` functions to reduce the
possibility of mistakes.
---------
Co-authored-by: delvh <dev.lh@web.de>
2024-01-28 01:39:51 +05:30
|
|
|
gitRepo, err := gitrepo.OpenRepository(ctx, repo)
|
2022-05-07 22:35:52 +05:30
|
|
|
if err != nil {
|
|
|
|
return nil, err
|
|
|
|
}
|
|
|
|
defer gitRepo.Close()
|
|
|
|
|
|
|
|
refs, err := gitRepo.GetRefsBySha(sha, "")
|
|
|
|
if err != nil {
|
|
|
|
return nil, err
|
|
|
|
}
|
|
|
|
|
2022-06-13 15:07:59 +05:30
|
|
|
pulls := make(map[int64]*issues_model.PullRequest)
|
2022-05-07 22:35:52 +05:30
|
|
|
|
|
|
|
for _, ref := range refs {
|
|
|
|
// Each pull branch starts with refs/pull/ we then go from there to find the index of the pr and then
|
|
|
|
// use that to get the pr.
|
|
|
|
if strings.HasPrefix(ref, git.PullPrefix) {
|
|
|
|
parts := strings.Split(ref[len(git.PullPrefix):], "/")
|
|
|
|
|
|
|
|
// e.g. 'refs/pull/1/head' would be []string{"1", "head"}
|
|
|
|
if len(parts) != 2 {
|
|
|
|
log.Error("getPullRequestsByHeadSHA found broken pull ref [%s] on repo [%-v]", ref, repo)
|
|
|
|
continue
|
|
|
|
}
|
|
|
|
|
|
|
|
prIndex, err := strconv.ParseInt(parts[0], 10, 64)
|
|
|
|
if err != nil {
|
|
|
|
log.Error("getPullRequestsByHeadSHA found broken pull ref [%s] on repo [%-v]", ref, repo)
|
|
|
|
continue
|
|
|
|
}
|
|
|
|
|
2022-06-13 15:07:59 +05:30
|
|
|
p, err := issues_model.GetPullRequestByIndex(ctx, repo.ID, prIndex)
|
2022-05-07 22:35:52 +05:30
|
|
|
if err != nil {
|
|
|
|
// If there is no pull request for this branch, we don't try to merge it.
|
2022-06-13 15:07:59 +05:30
|
|
|
if issues_model.IsErrPullRequestNotExist(err) {
|
2022-05-07 22:35:52 +05:30
|
|
|
continue
|
|
|
|
}
|
|
|
|
return nil, err
|
|
|
|
}
|
|
|
|
|
|
|
|
if filter(p) {
|
|
|
|
pulls[p.ID] = p
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
return pulls, nil
|
|
|
|
}
|
|
|
|
|
|
|
|
func handlePull(pullID int64, sha string) {
|
|
|
|
ctx, _, finished := process.GetManager().AddContext(graceful.GetManager().HammerContext(),
|
2023-02-04 04:41:48 +05:30
|
|
|
fmt.Sprintf("Handle AutoMerge of PR[%d] with sha[%s]", pullID, sha))
|
2022-05-07 22:35:52 +05:30
|
|
|
defer finished()
|
|
|
|
|
2022-06-13 15:07:59 +05:30
|
|
|
pr, err := issues_model.GetPullRequestByID(ctx, pullID)
|
2022-05-07 22:35:52 +05:30
|
|
|
if err != nil {
|
|
|
|
log.Error("GetPullRequestByID[%d]: %v", pullID, err)
|
|
|
|
return
|
|
|
|
}
|
|
|
|
|
|
|
|
// Check if there is a scheduled pr in the db
|
|
|
|
exists, scheduledPRM, err := pull_model.GetScheduledMergeByPullID(ctx, pr.ID)
|
|
|
|
if err != nil {
|
2023-02-04 04:41:48 +05:30
|
|
|
log.Error("%-v GetScheduledMergeByPullID: %v", pr, err)
|
2022-05-07 22:35:52 +05:30
|
|
|
return
|
|
|
|
}
|
|
|
|
if !exists {
|
|
|
|
return
|
|
|
|
}
|
|
|
|
|
|
|
|
// Get all checks for this pr
|
|
|
|
// We get the latest sha commit hash again to handle the case where the check of a previous push
|
|
|
|
// did not succeed or was not finished yet.
|
|
|
|
|
2022-11-19 13:42:33 +05:30
|
|
|
if err = pr.LoadHeadRepo(ctx); err != nil {
|
2023-02-04 04:41:48 +05:30
|
|
|
log.Error("%-v LoadHeadRepo: %v", pr, err)
|
2022-05-07 22:35:52 +05:30
|
|
|
return
|
|
|
|
}
|
|
|
|
|
Simplify how git repositories are opened (#28937)
## Purpose
This is a refactor toward building an abstraction over managing git
repositories.
Afterwards, it does not matter anymore if they are stored on the local
disk or somewhere remote.
## What this PR changes
We used `git.OpenRepository` everywhere previously.
Now, we should split them into two distinct functions:
Firstly, there are temporary repositories which do not change:
```go
git.OpenRepository(ctx, diskPath)
```
Gitea managed repositories having a record in the database in the
`repository` table are moved into the new package `gitrepo`:
```go
gitrepo.OpenRepository(ctx, repo_model.Repo)
```
Why is `repo_model.Repository` the second parameter instead of file
path?
Because then we can easily adapt our repository storage strategy.
The repositories can be stored locally, however, they could just as well
be stored on a remote server.
## Further changes in other PRs
- A Git Command wrapper on package `gitrepo` could be created. i.e.
`NewCommand(ctx, repo_model.Repository, commands...)`. `git.RunOpts{Dir:
repo.RepoPath()}`, the directory should be empty before invoking this
method and it can be filled in the function only. #28940
- Remove the `RepoPath()`/`WikiPath()` functions to reduce the
possibility of mistakes.
---------
Co-authored-by: delvh <dev.lh@web.de>
2024-01-28 01:39:51 +05:30
|
|
|
headGitRepo, err := gitrepo.OpenRepository(ctx, pr.HeadRepo)
|
2022-05-07 22:35:52 +05:30
|
|
|
if err != nil {
|
2023-02-04 04:41:48 +05:30
|
|
|
log.Error("OpenRepository %-v: %v", pr.HeadRepo, err)
|
2022-05-07 22:35:52 +05:30
|
|
|
return
|
|
|
|
}
|
|
|
|
defer headGitRepo.Close()
|
|
|
|
|
|
|
|
headBranchExist := headGitRepo.IsBranchExist(pr.HeadBranch)
|
|
|
|
|
|
|
|
if pr.HeadRepo == nil || !headBranchExist {
|
2023-02-04 04:41:48 +05:30
|
|
|
log.Warn("Head branch of auto merge %-v does not exist [HeadRepoID: %d, Branch: %s]", pr, pr.HeadRepoID, pr.HeadBranch)
|
2022-05-07 22:35:52 +05:30
|
|
|
return
|
|
|
|
}
|
|
|
|
|
|
|
|
// Check if all checks succeeded
|
|
|
|
pass, err := pull_service.IsPullCommitStatusPass(ctx, pr)
|
|
|
|
if err != nil {
|
2023-02-04 04:41:48 +05:30
|
|
|
log.Error("%-v IsPullCommitStatusPass: %v", pr, err)
|
2022-05-07 22:35:52 +05:30
|
|
|
return
|
|
|
|
}
|
|
|
|
if !pass {
|
2023-02-04 04:41:48 +05:30
|
|
|
log.Info("Scheduled auto merge %-v has unsuccessful status checks", pr)
|
2022-05-07 22:35:52 +05:30
|
|
|
return
|
|
|
|
}
|
|
|
|
|
|
|
|
// Merge if all checks succeeded
|
2022-12-03 08:18:26 +05:30
|
|
|
doer, err := user_model.GetUserByID(ctx, scheduledPRM.DoerID)
|
2022-05-07 22:35:52 +05:30
|
|
|
if err != nil {
|
2023-02-04 04:41:48 +05:30
|
|
|
log.Error("Unable to get scheduled User[%d]: %v", scheduledPRM.DoerID, err)
|
2022-05-07 22:35:52 +05:30
|
|
|
return
|
|
|
|
}
|
|
|
|
|
2022-05-11 15:39:36 +05:30
|
|
|
perm, err := access_model.GetUserRepoPermission(ctx, pr.HeadRepo, doer)
|
2022-05-07 22:35:52 +05:30
|
|
|
if err != nil {
|
2023-02-04 04:41:48 +05:30
|
|
|
log.Error("GetUserRepoPermission %-v: %v", pr.HeadRepo, err)
|
2022-05-07 22:35:52 +05:30
|
|
|
return
|
|
|
|
}
|
|
|
|
|
2023-02-21 20:12:07 +05:30
|
|
|
if err := pull_service.CheckPullMergable(ctx, doer, &perm, pr, pull_service.MergeCheckTypeGeneral, false); err != nil {
|
2022-05-07 22:35:52 +05:30
|
|
|
if errors.Is(pull_service.ErrUserNotAllowedToMerge, err) {
|
2023-02-04 04:41:48 +05:30
|
|
|
log.Info("%-v was scheduled to automerge by an unauthorized user", pr)
|
2022-05-07 22:35:52 +05:30
|
|
|
return
|
|
|
|
}
|
2023-02-04 04:41:48 +05:30
|
|
|
log.Error("%-v CheckPullMergable: %v", pr, err)
|
2022-05-07 22:35:52 +05:30
|
|
|
return
|
|
|
|
}
|
|
|
|
|
|
|
|
var baseGitRepo *git.Repository
|
|
|
|
if pr.BaseRepoID == pr.HeadRepoID {
|
|
|
|
baseGitRepo = headGitRepo
|
|
|
|
} else {
|
2022-11-19 13:42:33 +05:30
|
|
|
if err = pr.LoadBaseRepo(ctx); err != nil {
|
2023-02-04 04:41:48 +05:30
|
|
|
log.Error("%-v LoadBaseRepo: %v", pr, err)
|
2022-05-07 22:35:52 +05:30
|
|
|
return
|
|
|
|
}
|
|
|
|
|
Simplify how git repositories are opened (#28937)
## Purpose
This is a refactor toward building an abstraction over managing git
repositories.
Afterwards, it does not matter anymore if they are stored on the local
disk or somewhere remote.
## What this PR changes
We used `git.OpenRepository` everywhere previously.
Now, we should split them into two distinct functions:
Firstly, there are temporary repositories which do not change:
```go
git.OpenRepository(ctx, diskPath)
```
Gitea managed repositories having a record in the database in the
`repository` table are moved into the new package `gitrepo`:
```go
gitrepo.OpenRepository(ctx, repo_model.Repo)
```
Why is `repo_model.Repository` the second parameter instead of file
path?
Because then we can easily adapt our repository storage strategy.
The repositories can be stored locally, however, they could just as well
be stored on a remote server.
## Further changes in other PRs
- A Git Command wrapper on package `gitrepo` could be created. i.e.
`NewCommand(ctx, repo_model.Repository, commands...)`. `git.RunOpts{Dir:
repo.RepoPath()}`, the directory should be empty before invoking this
method and it can be filled in the function only. #28940
- Remove the `RepoPath()`/`WikiPath()` functions to reduce the
possibility of mistakes.
---------
Co-authored-by: delvh <dev.lh@web.de>
2024-01-28 01:39:51 +05:30
|
|
|
baseGitRepo, err = gitrepo.OpenRepository(ctx, pr.BaseRepo)
|
2022-05-07 22:35:52 +05:30
|
|
|
if err != nil {
|
2023-02-04 04:41:48 +05:30
|
|
|
log.Error("OpenRepository %-v: %v", pr.BaseRepo, err)
|
2022-05-07 22:35:52 +05:30
|
|
|
return
|
|
|
|
}
|
|
|
|
defer baseGitRepo.Close()
|
|
|
|
}
|
|
|
|
|
2022-11-03 21:19:00 +05:30
|
|
|
if err := pull_service.Merge(ctx, pr, doer, baseGitRepo, scheduledPRM.MergeStyle, "", scheduledPRM.Message, true); err != nil {
|
2022-05-07 22:35:52 +05:30
|
|
|
log.Error("pull_service.Merge: %v", err)
|
|
|
|
return
|
|
|
|
}
|
|
|
|
}
|