283 lines
9.2 KiB
HTML
283 lines
9.2 KiB
HTML
<!DOCTYPE HTML>
|
|
<html>
|
|
<head>
|
|
<meta charset="UTF-8">
|
|
<link rel="stylesheet" href="https://docs.forgeflux.org/main.css">
|
|
|
|
<link rel="icon" href="https://forgeflux.org/favicon.ico">
|
|
|
|
<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1">
|
|
<title> Docs | Tests </title>
|
|
</head>
|
|
<body>
|
|
|
|
|
|
<main>
|
|
|
|
|
|
|
|
<nav>
|
|
|
|
<a href="https://docs.forgeflux.org">
|
|
<img src="/logo.svg" alt="" />
|
|
</a>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<a href="javascript:void(0);" onclick="burger()" id="mobile" class="ms-Icon--GlobalNavButton"></a>
|
|
<div id="trees">
|
|
|
|
|
|
|
|
|
|
|
|
<input class="tree-toggle" type="checkbox" id="ftest"
|
|
checked />
|
|
<label class="tree-toggle-label" for="ftest">ftest</label>
|
|
|
|
<ul class="subtree">
|
|
|
|
<li >
|
|
<a href="https://docs.forgeflux.org/ftest/introduction/">Introducing ftest</a>
|
|
</li>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<li >
|
|
<a href="https://docs.forgeflux.org/ftest/cli/">Using ftest CLI</a>
|
|
</li>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<li >
|
|
<a href="https://docs.forgeflux.org/ftest/terminology/">Terminology</a>
|
|
</li>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<li >
|
|
<a href="https://docs.forgeflux.org/ftest/quick-links/">Quick Links</a>
|
|
</li>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<li >
|
|
<a href="https://docs.forgeflux.org/ftest/ftest-toml/">ftest.toml</a>
|
|
</li>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<li class="active">
|
|
<a href="https://docs.forgeflux.org/ftest/tests/">Tests</a>
|
|
</li>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<li >
|
|
<a href="https://docs.forgeflux.org/ftest/tutorial-new-specimen/">[Tutorial] Add new specimen</a>
|
|
</li>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
</ul>
|
|
|
|
|
|
|
|
|
|
<input class="tree-toggle" type="checkbox" id="interface"
|
|
/>
|
|
<label class="tree-toggle-label" for="interface">Interface</label>
|
|
|
|
<ul class="subtree">
|
|
|
|
<li >
|
|
<a href="https://docs.forgeflux.org/Interface/about/">About Interface</a>
|
|
</li>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
</ul>
|
|
|
|
|
|
|
|
|
|
<input class="tree-toggle" type="checkbox" id="northstar"
|
|
/>
|
|
<label class="tree-toggle-label" for="northstar">Northstar</label>
|
|
|
|
<ul class="subtree">
|
|
|
|
<li >
|
|
<a href="https://docs.forgeflux.org/Northstar/about/">About Northstar</a>
|
|
</li>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
</ul>
|
|
|
|
|
|
|
|
|
|
<input class="tree-toggle" type="checkbox" id="updates"
|
|
/>
|
|
<label class="tree-toggle-label" for="updates">Updates</label>
|
|
|
|
<ul class="subtree">
|
|
|
|
<li >
|
|
<a href="https://docs.forgeflux.org/updates/december2020/">December 2020</a>
|
|
</li>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<li >
|
|
<a href="https://docs.forgeflux.org/updates/november2020/">November 2020</a>
|
|
</li>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<li >
|
|
<a href="https://docs.forgeflux.org/updates/october2020/">October 2020</a>
|
|
</li>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<li >
|
|
<a href="https://docs.forgeflux.org/updates/september2020/">September 2020</a>
|
|
</li>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
</ul>
|
|
|
|
</div>
|
|
</nav>
|
|
|
|
|
|
<article>
|
|
|
|
|
|
<div id="on_right">
|
|
<span id="search-ico" class="ms-Icon--Search"></span>
|
|
</div>
|
|
<div class="search-container">
|
|
<input id="search" type="search" placeholder="Search as you type...">
|
|
<div class="search-results">
|
|
<div class="search-results__header"></div>
|
|
<ul class="search-results__items"></ul>
|
|
</div>
|
|
</div>
|
|
|
|
|
|
<div id="wrap">
|
|
|
|
<p>Tests are Docker containers that test
|
|
<a href="https://docs.forgeflux.org/ftest/terminology/#specimen">specimens</a> a very specific protocol
|
|
or a feature. They integrate with ftest by accepting configuration
|
|
parameters through environment variables and uploading results through a
|
|
REST API.</p>
|
|
<h2 id="configuration-environment-variables">Configuration (Environment Variables) <a class="anchor" href="#configuration-environment-variables"></a></h2>
|
|
<h3 id="auto-generated">Auto-generated <a class="anchor" href="#auto-generated"></a></h3>
|
|
<p>Some environment variables are automatically generated by ftest</p>
|
|
<table><thead><tr><th>Name</th><th>Purpose</th></tr></thead><tbody>
|
|
<tr><td><code>FTEST_AUTH</code></td><td>Authentication token that should be used when uploading results to ftest server. It is ephemeral, and randomly generated for each test.</td></tr>
|
|
<tr><td><code>FTEST_HOST</code></td><td>The hostname of the ftest server. Results must be uploaded to here. Will be in format http://ftest:9000</td></tr>
|
|
<tr><td><code>FTEST_TARGET_HOST</code></td><td>The hostname of the specimen server. Tests must be run against this hostname. Will be in format http://specimen:8080</td></tr>
|
|
</tbody></table>
|
|
<h3 id="user-provided">User provided <a class="anchor" href="#user-provided"></a></h3>
|
|
<p>There's option to fetch custom environment variables also. Please
|
|
specify them in your test's documentation clearly. The README file of
|
|
the test is recommended.</p>
|
|
<h2 id="results-api">Results API: <a class="anchor" href="#results-api"></a></h2>
|
|
<p>A formal OpenAPI specification for the ftest server is yet to be
|
|
created. For now, this command should upload test results to the ftest
|
|
server:</p>
|
|
<pre data-lang="python" style="background-color:#eff1f5;color:#4f5b66;" class="language-python "><code class="language-python" data-lang="python"><span style="color:#b48ead;">def </span><span style="color:#8fa1b3;">upload_logs_to_ftest</span><span>(</span><span style="color:#bf616a;">auth</span><span>, </span><span style="color:#bf616a;">success</span><span>: bool, </span><span style="color:#bf616a;">logs</span><span>: str):
|
|
</span><span> ftest = </span><span style="color:#b48ead;">f</span><span>"</span><span style="color:#a3be8c;">http://ftest.example.org/api/v1/</span><span>{auth}</span><span style="color:#a3be8c;">/results</span><span>"
|
|
</span><span> payload = {"</span><span style="color:#a3be8c;">success</span><span>": success, "</span><span style="color:#a3be8c;">logs</span><span>": logs}
|
|
</span><span> res = requests.</span><span style="color:#bf616a;">post</span><span>(
|
|
</span><span> ftest, </span><span style="color:#bf616a;">json</span><span>=payload, </span><span style="color:#bf616a;">headers</span><span>={"</span><span style="color:#a3be8c;">Origin</span><span>": "</span><span style="color:#a3be8c;">http://example.org</span><span>"}
|
|
</span><span> )
|
|
</span><span> </span><span style="color:#b48ead;">if </span><span>res.status_code == </span><span style="color:#d08770;">200</span><span>:
|
|
</span><span> logger.</span><span style="color:#bf616a;">info</span><span>("</span><span style="color:#a3be8c;">Upload successful</span><span>")
|
|
</span><span> </span><span style="color:#b48ead;">else</span><span>:
|
|
</span><span> </span><span style="color:#96b5b4;">print</span><span>(res)
|
|
</span><span>
|
|
</span><span style="color:#bf616a;">upload_logs_to_ftest</span><span>("</span><span style="color:#a3be8c;">supersecretauthtoekn_provided_by_ftest</span><span>", </span><span style="color:#d08770;">True</span><span>, "")
|
|
</span></code></pre>
|
|
|
|
|
|
</div>
|
|
|
|
</article>
|
|
</main>
|
|
|
|
|
|
<script type="text/javascript" src="https://docs.forgeflux.org/elasticlunr.min.js" defer></script>
|
|
<script type="text/javascript" src="https://docs.forgeflux.org/search_index.en.js" defer></script>
|
|
<script type="text/javascript" src="https://docs.forgeflux.org/js.js" defer></script>
|
|
|
|
|
|
</body>
|
|
</html>
|