realaravinth
c637be4499
SUMMARY When OptionalPage.page is None, the count was incremented twice leading to skipping next_page_link=2. Increment operation is now movied into `impl From<OptionalPage> for Page` for centralized incrementing, which also fixes this bug |
||
---|---|---|
.github | ||
config | ||
db | ||
docs | ||
federate | ||
forge | ||
scripts | ||
src | ||
static/cache/css | ||
templates | ||
utils/cache-bust | ||
.dockerignore | ||
.env-sample | ||
.gitignore | ||
build.rs | ||
Cargo.lock | ||
Cargo.toml | ||
docker-compose-dev-deps.yml | ||
Dockerfile | ||
LICENSE.md | ||
Makefile | ||
README.md | ||
spider.rs |
Why
There are several small, private forges that host Free Software projects. Some of these Forges might one day participate in the federated ecosystem. So it would make sense to have a system(see spider mechanism) that would map and advertise these instances and the projects that they host.
Consensual Spidering
We are aware that spiders some very aggressive and small forges are often running on resource-constrained environments. Therefore, StarChart(this spider) will only crawl a service if the crawl is requested by the admin of the forge(more accurately, folks that have access to the DNS associated with the forge's hostname though).
StarChart will rate limit API calls to one call every 10 seconds. For instance, a Gitea API call would resemble:
curl -X 'GET' \
'https://gitea.example.org/api/v1/repos/search?page=2&limit=20' \
-H 'accept: application/json'
Contributing
Thanks for considering contributing on GitHub. If you are not an GitHub but would like to contribute to ForgeFlux sub-projects(all repositories under this organisation), I would be happy to manually mirror this repository on my Gitea instance, which has a much more respectful privacy policy