new deploy: 2023-10-02T07:46:01+00:00

ftest/about/index.html

@@ -53,6 +53,9 @@
     
     
   
+    
+    
+  
 
   
 
@@ -224,9 +227,29 @@ extension, <a href="https://forgefed.org">ForgeFed</a></li>
 <a href="https://github.com/matrix-org/sytest">matrix-org/sytest</a> but for ActivityPub and ForgeFed.</p>
 <p>An instance (will be) is hosted by ForgeFlux, that is run on renewable
 energy.</p>
+<h2 id="mechanism">Mechanism&nbsp;<a class="anchor" href="#mechanism">&#xE732;</a></h2>
+<ol>
+<li>New jobs are fetched from the <a href="https://docs.forgeflux.org/ftest/terminology/#control-repository">control repository</a>. At any given time, ftest will run only one job. Therefore, each commit on the control repository must modify one <a href="https://docs.forgeflux.org/ftest/terminology/#specimen">specimen</a> only.</li>
+<li>TODO: create <code>ftest</code> Docker network (right now, this is done by hand)</li>
+<li>The specimen test environment is deployed using <code>docker-compose.yml</code>
+available in the test network. The specimen might have dependency
+services (like a database or a chase), but the specimen service
+should be added to the <code>ftest</code> network.</li>
+<li>Get a list of initialization workflows from <code>ftest.toml</code> and run them
+sequentially.</li>
+<li>Run test suites sequentially. A test suite might contain more than
+one tests, all of them are launched in parallel. They are added to
+the <code>ftest</code> Docker net by the <code>ftest</code> server.</li>
+<li>Wait for tests from previous test to run to completion, collect logs
+and delete containers. And run next test suite.</li>
+<li>Aggregate logs and write to <a href="https://docs.forgeflux.org/ftest/terminology/#result-repository">result repository</a>.</li>
+<li>Push result repository.</li>
+<li>Wait for next test job.</li>
+</ol>
 <h2 id="quick-links">Quick links&nbsp;<a class="anchor" href="#quick-links">&#xE732;</a></h2>
 <ul>
 <li><a href="https://docs.forgeflux.org/ftest/tutorial-new-specimen/">Add new specimen to ftest</a></li>
+<li><a href="https://docs.forgeflux.org/ftest/terminology/">Terminology</a></li>
 </ul>
 
     

ftest/terminology/index.html

@@ -72,6 +72,9 @@
     
       
       
+    
+  
+    
     
   
 
@@ -92,12 +95,16 @@
             
             <li><a href="https://docs.forgeflux.org/ftest/terminology/#directory-structure">Directory structure</a></li>
             
-            <li><a href="https://docs.forgeflux.org/ftest/terminology/#specimen-docker-compose-yml">specimen docker-compose.yml</a></li>
+            <li><a href="https://docs.forgeflux.org/ftest/terminology/#specimen-docker-compose-yml">Specimen docker-compose.yml</a></li>
             
          </ul>
     
         </li>
     
+          <li><a href="https://docs.forgeflux.org/ftest/terminology/#result-repository">Result Repository</a>
+          
+        </li>
+    
       </ul>
     
 
@@ -251,8 +258,8 @@
   <div id="wrap">
     
         <h2 id="specimen">Specimen&nbsp;<a class="anchor" href="#specimen">&#xE732;</a></h2>
-<p>A specimen is an ActityPub or a ForgeFed instance that has to be tested
+<p>The target of a ftest test job. These are the ActityPub or ForgeFlux
-on ftest.</p>
+implementations that must be tested for compliance.</p>
 <h2 id="job-manifest">Job Manifest&nbsp;<a class="anchor" href="#job-manifest">&#xE732;</a></h2>
 <p>or <code>ftest.toml</code></p>
 <p>Contains information on the test environment, initialization code and
@@ -281,10 +288,13 @@ the test suites that has to be run on the <a href="/ftest/terminology.md#specime
 </span><span>        </span><span style="color:#bf616a;">├──</span><span> docker-compose.yaml
 </span><span>        </span><span style="color:#bf616a;">└──</span><span> Dockerfile
 </span></code></pre>
-<h3 id="specimen-docker-compose-yml">specimen docker-compose.yml&nbsp;<a class="anchor" href="#specimen-docker-compose-yml">&#xE732;</a></h3>
+<h3 id="specimen-docker-compose-yml">Specimen docker-compose.yml&nbsp;<a class="anchor" href="#specimen-docker-compose-yml">&#xE732;</a></h3>
 <p>The control repository <em>MUST</em> contain a <code>docker-compose.yml</code> to spin up
 the specimen. It may include configuration files in the same directory,
 which to docker-compose file can rely on using bind mounts within the container.</p>
+<h2 id="result-repository">Result Repository&nbsp;<a class="anchor" href="#result-repository">&#xE732;</a></h2>
+<p>Result repository contains the results of all the targets' jobs in JSON
+format.</p>
 
     
   </div>