feat: readme with dev notes
This commit is contained in:
parent
8e95c017c9
commit
7ac5019dc2
1 changed files with 51 additions and 0 deletions
51
README.md
Normal file
51
README.md
Normal file
|
@ -0,0 +1,51 @@
|
|||
# ftest: compliance suite for software forge federation
|
||||
|
||||
## Goals
|
||||
|
||||
1. Flexible: should be able to define new test suites with ease
|
||||
2. Transparent: test results should be transparently available
|
||||
3. Reproducible: test results should reproducible
|
||||
4. Ease of deployment
|
||||
5. Ease of use for developers and forge developers
|
||||
6. Ease of use for general public
|
||||
|
||||
## Development Notes
|
||||
|
||||
### Test Runner
|
||||
|
||||
Choice of test runner should not be hard-codded, it should be possible to
|
||||
define new runner interfaces. But for now, we are going to go with
|
||||
Docker.
|
||||
|
||||
### Test suites
|
||||
|
||||
Each test suite will be it's own Docker image. This allows for polyglot
|
||||
test suites, which should make it easy to define new test suites. Also,
|
||||
containerization will offer some level of security when running
|
||||
untrusted code.
|
||||
|
||||
### Test jobs
|
||||
|
||||
A new container of of the test suite image will be
|
||||
deployed whenever a new test job is scheduled. The container will be
|
||||
given a secret associated with the job ID. When the job is complete, the
|
||||
test suite container should upload the results to the `ftest` server.
|
||||
|
||||
### Test result schema
|
||||
|
||||
```json
|
||||
"tests": [{
|
||||
"test_id": string,
|
||||
"success": boolean,
|
||||
"logs": string,
|
||||
"raw_logs": string
|
||||
}]
|
||||
```
|
||||
|
||||
### Scheduling new test jobs
|
||||
|
||||
The test runner will accept new jobs through a Git repository. To
|
||||
schedule a new test job(for instance when a forge instance is updated),
|
||||
forge developers can send a patch to the Git repository with the
|
||||
`docker-compsoe` definition file to spin up their software, and a job
|
||||
file that enumerates the test suits that must be run against it.
|
Loading…
Reference in a new issue