375 lines
13 KiB
HTML
375 lines
13 KiB
HTML
<!DOCTYPE HTML>
|
|
<html>
|
|
<head>
|
|
<meta charset="UTF-8">
|
|
<link rel="stylesheet" href="https://docs.forgeflux.org/main.css">
|
|
|
|
<link rel="icon" href="https://forgeflux.org/favicon.ico">
|
|
|
|
<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1">
|
|
<title> Docs | Using ftest CLI </title>
|
|
</head>
|
|
<body>
|
|
|
|
|
|
<main>
|
|
|
|
|
|
|
|
<nav>
|
|
|
|
<a href="https://docs.forgeflux.org">
|
|
<img src="/logo.svg" alt="" />
|
|
</a>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<a href="javascript:void(0);" onclick="burger()" id="mobile" class="ms-Icon--GlobalNavButton"></a>
|
|
<div id="trees">
|
|
|
|
|
|
|
|
|
|
|
|
<input class="tree-toggle" type="checkbox" id="ftest"
|
|
checked />
|
|
<label class="tree-toggle-label" for="ftest">ftest</label>
|
|
|
|
<ul class="subtree">
|
|
|
|
<li >
|
|
<a href="https://docs.forgeflux.org/ftest/introduction/">Introducing ftest</a>
|
|
</li>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<li class="active">
|
|
<a href="https://docs.forgeflux.org/ftest/cli/">Using ftest CLI</a>
|
|
</li>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<ul id="toc">
|
|
|
|
<li><a href="https://docs.forgeflux.org/ftest/cli/#0-install-runtime-dependencies">0. Install Runtime Dependencies</a>
|
|
|
|
</li>
|
|
|
|
<li><a href="https://docs.forgeflux.org/ftest/cli/#1-setup-networking">1. Setup networking</a>
|
|
|
|
<ul>
|
|
|
|
<li><a href="https://docs.forgeflux.org/ftest/cli/#i-create-docker-network-ftest">i. Create Docker network ftest</a></li>
|
|
|
|
<li><a href="https://docs.forgeflux.org/ftest/cli/#ii-allow-traffic-from-ftest-net-to-host-machine-on-port-29130">ii. Allow traffic from ftest net to host machine on port 29130</a></li>
|
|
|
|
</ul>
|
|
|
|
</li>
|
|
|
|
<li><a href="https://docs.forgeflux.org/ftest/cli/#2-install-ftest">2. Install ftest</a>
|
|
|
|
<ul>
|
|
|
|
<li><a href="https://docs.forgeflux.org/ftest/cli/#i-download-assets">i. Download assets</a></li>
|
|
|
|
<li><a href="https://docs.forgeflux.org/ftest/cli/#ii-verify-checksum">ii Verify checksum</a></li>
|
|
|
|
<li><a href="https://docs.forgeflux.org/ftest/cli/#iii-verify-gpg-signature">iii Verify GPG signature</a></li>
|
|
|
|
<li><a href="https://docs.forgeflux.org/ftest/cli/#iv-install-binary">iv. Install binary</a></li>
|
|
|
|
</ul>
|
|
|
|
</li>
|
|
|
|
<li><a href="https://docs.forgeflux.org/ftest/cli/#2-run">2. Run</a>
|
|
|
|
</li>
|
|
|
|
</ul>
|
|
|
|
|
|
|
|
|
|
|
|
<li >
|
|
<a href="https://docs.forgeflux.org/ftest/terminology/">Terminology</a>
|
|
</li>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<li >
|
|
<a href="https://docs.forgeflux.org/ftest/quick-links/">Quick Links</a>
|
|
</li>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<li >
|
|
<a href="https://docs.forgeflux.org/ftest/ftest-toml/">ftest.toml</a>
|
|
</li>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<li >
|
|
<a href="https://docs.forgeflux.org/ftest/tests/">Tests</a>
|
|
</li>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<li >
|
|
<a href="https://docs.forgeflux.org/ftest/tutorial-new-specimen/">[Tutorial] Add new specimen</a>
|
|
</li>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
</ul>
|
|
|
|
|
|
|
|
|
|
<input class="tree-toggle" type="checkbox" id="interface"
|
|
/>
|
|
<label class="tree-toggle-label" for="interface">Interface</label>
|
|
|
|
<ul class="subtree">
|
|
|
|
<li >
|
|
<a href="https://docs.forgeflux.org/Interface/about/">About Interface</a>
|
|
</li>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
</ul>
|
|
|
|
|
|
|
|
|
|
<input class="tree-toggle" type="checkbox" id="northstar"
|
|
/>
|
|
<label class="tree-toggle-label" for="northstar">Northstar</label>
|
|
|
|
<ul class="subtree">
|
|
|
|
<li >
|
|
<a href="https://docs.forgeflux.org/Northstar/about/">About Northstar</a>
|
|
</li>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
</ul>
|
|
|
|
|
|
|
|
|
|
<input class="tree-toggle" type="checkbox" id="updates"
|
|
/>
|
|
<label class="tree-toggle-label" for="updates">Updates</label>
|
|
|
|
<ul class="subtree">
|
|
|
|
<li >
|
|
<a href="https://docs.forgeflux.org/updates/december2020/">December 2020</a>
|
|
</li>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<li >
|
|
<a href="https://docs.forgeflux.org/updates/november2020/">November 2020</a>
|
|
</li>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<li >
|
|
<a href="https://docs.forgeflux.org/updates/october2020/">October 2020</a>
|
|
</li>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<li >
|
|
<a href="https://docs.forgeflux.org/updates/september2020/">September 2020</a>
|
|
</li>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
</ul>
|
|
|
|
</div>
|
|
</nav>
|
|
|
|
|
|
<article>
|
|
|
|
|
|
<div id="on_right">
|
|
<span id="search-ico" class="ms-Icon--Search"></span>
|
|
</div>
|
|
<div class="search-container">
|
|
<input id="search" type="search" placeholder="Search as you type...">
|
|
<div class="search-results">
|
|
<div class="search-results__header"></div>
|
|
<ul class="search-results__items"></ul>
|
|
</div>
|
|
</div>
|
|
|
|
|
|
<div id="wrap">
|
|
|
|
<p>The ftest binary can run in the following modes:</p>
|
|
<ol>
|
|
<li>
|
|
<p>Daemon: will act as a job server, listening for webhook events to
|
|
schedule new jobs. This mode is to be used on servers.</p>
|
|
</li>
|
|
<li>
|
|
<p>CLI: Useful for running compliance tests locally on one's own
|
|
computer.</p>
|
|
</li>
|
|
</ol>
|
|
<h2 id="0-install-runtime-dependencies">0. Install Runtime Dependencies <a class="anchor" href="#0-install-runtime-dependencies"></a></h2>
|
|
<p>ftest invokes a few binaries that are listed bellow:</p>
|
|
<ol>
|
|
<li><a href="https://packages.debian.org/bookworm/iproute2">iproute2</a></li>
|
|
<li>Docker</li>
|
|
<li>docker-compose</li>
|
|
<li>git</li>
|
|
</ol>
|
|
<p><em>NOTE: The user running ftest binary must be part of the <code>docker</code> UNIX
|
|
group</em></p>
|
|
<h2 id="1-setup-networking">1. Setup networking <a class="anchor" href="#1-setup-networking"></a></h2>
|
|
<p>ftest spawns a HTTP server to collect results from test suites. Right
|
|
now, it is hard-coded to listen on port <code>29130</code>. And, since test jobs
|
|
are run in containers and the ftest binary is going to be run on host
|
|
machine, we have to setup Docker and host machine's firewall to allow
|
|
traffic. We use an <a href="https://git.batsense.net/ForgeFlux/ftest-nginx-proxy">nginx
|
|
container</a> to
|
|
achieve this.</p>
|
|
<p>This guide has instructions for <code>ufw</code>, please adopt the same for your
|
|
setup.</p>
|
|
<h3 id="i-create-docker-network-ftest">i. Create Docker network <code>ftest</code> <a class="anchor" href="#i-create-docker-network-ftest"></a></h3>
|
|
<pre style="background-color:#eff1f5;color:#4f5b66;"><code><span>docker network create --attachable -d bridge --subnet 192.168.32.0/20 ftest
|
|
</span></code></pre>
|
|
<p><em>NOTE: if choosing custom subnet, adjust instructions in the section to match it</em></p>
|
|
<h3 id="ii-allow-traffic-from-ftest-net-to-host-machine-on-port-29130">ii. Allow traffic from <code>ftest</code> net to host machine on port <code>29130</code> <a class="anchor" href="#ii-allow-traffic-from-ftest-net-to-host-machine-on-port-29130"></a></h3>
|
|
<pre data-lang="bash" style="background-color:#eff1f5;color:#4f5b66;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#bf616a;">sudo</span><span> ufw allow from 192.168.32.0/20 proto tcp to any port 29130
|
|
</span></code></pre>
|
|
<h2 id="2-install-ftest">2. Install ftest <a class="anchor" href="#2-install-ftest"></a></h2>
|
|
<h3 id="i-download-assets">i. Download assets <a class="anchor" href="#i-download-assets"></a></h3>
|
|
<pre data-lang="bash" style="background-color:#eff1f5;color:#4f5b66;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#bf616a;">wget</span><span> https://dl.forgeflux.org/forgeflux/ftest/master/ftest-master-linux-amd64.tar.gz.asc
|
|
</span><span style="color:#bf616a;">wget</span><span> https://dl.forgeflux.org/forgeflux/ftest/master/ftest-master-linux-amd64.tar.gz.sha256
|
|
</span><span style="color:#bf616a;">wget</span><span> https://dl.forgeflux.org/forgeflux/ftest/master/ftest-master-linux-amd64.tar.gz
|
|
</span></code></pre>
|
|
<h3 id="ii-verify-checksum">ii Verify checksum <a class="anchor" href="#ii-verify-checksum"></a></h3>
|
|
<pre data-lang="bash" style="background-color:#eff1f5;color:#4f5b66;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#bf616a;">sha256sum -c</span><span> ftest-master-linux-amd64.tar.gz.sha256
|
|
</span></code></pre>
|
|
<h3 id="iii-verify-gpg-signature">iii Verify GPG signature <a class="anchor" href="#iii-verify-gpg-signature"></a></h3>
|
|
<p>All ForgeFlux binaries are signed with <a href="http://keyserver.ubuntu.com/pks/lookup?search=56861597C15E54520A36983A6828E4A2E5B91DB9&fingerprint=on&op=index">our GPG
|
|
key</a>.
|
|
Please verify signatures to verify authenticity.</p>
|
|
<pre data-lang="bash" style="background-color:#eff1f5;color:#4f5b66;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#bf616a;">gpg --keyserver</span><span> keyserver.ubuntu.com</span><span style="color:#bf616a;"> --recv</span><span> 56861597C15E54520A36983A6828E4A2E5B91DB9
|
|
</span><span style="color:#bf616a;">gpg --verify</span><span> ftest-master-linux-amd64.tar.gz.asc
|
|
</span></code></pre>
|
|
<h3 id="iv-install-binary">iv. Install binary <a class="anchor" href="#iv-install-binary"></a></h3>
|
|
<pre data-lang="bash" style="background-color:#eff1f5;color:#4f5b66;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#bf616a;">tar -xvzf</span><span> ftest-master-linux-amd64.tar.gz && </span><span style="color:#bf616a;">sudo</span><span> cp ftest-master-linux-amd64/ftest /usr/local/bin
|
|
</span></code></pre>
|
|
<h2 id="2-run">2. Run <a class="anchor" href="#2-run"></a></h2>
|
|
<p>The <a href="https://docs.forgeflux.org/ftest/terminology/#control-repository">control repository</a> has
|
|
a few pre-configured targets. We recommend starting from there. Please
|
|
see <a href="https://docs.forgeflux.org/ftest/tutorial-new-specimen/">here</a> to configure ftest for new <a href="https://docs.forgeflux.org/ftest/terminology/#specimen">specimens</a>.</p>
|
|
<pre data-lang="bash" style="background-color:#eff1f5;color:#4f5b66;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#bf616a;">git</span><span> clone https://git.batsense.net/ForgeFlux/ftest-control
|
|
</span><span style="color:#96b5b4;">cd</span><span> ftest-control/targets/forgejo
|
|
</span><span style="color:#bf616a;">ftest</span><span> test .
|
|
</span></code></pre>
|
|
<p>The results will be available in JSON format and HTML format (like this
|
|
one <a href="/example/ftest/results/targets/forgejo/results">here</a>) in the same
|
|
directory.</p>
|
|
|
|
|
|
</div>
|
|
|
|
</article>
|
|
</main>
|
|
|
|
|
|
<script type="text/javascript" src="https://docs.forgeflux.org/elasticlunr.min.js" defer></script>
|
|
<script type="text/javascript" src="https://docs.forgeflux.org/search_index.en.js" defer></script>
|
|
<script type="text/javascript" src="https://docs.forgeflux.org/js.js" defer></script>
|
|
|
|
|
|
</body>
|
|
</html>
|