Commit graph

194 commits

Author SHA1 Message Date
e726f2234d
feat: use fixed value for an instance for DNS challenge
DESCRIPTION
    Randomly generated values for forge ownership verification through
    DNS TXT records was making development process complicated. Using
    starchart instance's hostname for the TXT record's value is secure
    enough for our use case.

    This patch gets rid of all the code necessary to implement random
    value challenges
2023-02-22 12:05:49 +05:30
38eb9c74ec
feat: get latest tarball from REST API 2023-02-11 19:42:48 +05:30
cb2dfa5e19
fix: gitea check is broken, disable check and return default true to
avoid breakage
2023-02-11 19:40:48 +05:30
f8d33e04e8
feat: mark entries as imported to differentiate b/w native crawls and
imported records
2023-02-11 19:40:04 +05:30
3f088a8cb8
fix: return string instead of pathbuf 2023-02-11 19:39:50 +05:30
df1edefb9d
feat: implement latest_tar for publiccodeyml 2023-01-08 00:32:15 +05:30
202600696a
feat: define latest_tar on federate trait 2023-01-08 00:31:58 +05:30
d8126a9268
hotfix: remove references to custom repository page and link directly to repository's page 2022-07-23 22:20:40 +05:30
3c6ccc278b
fix: adapt repository.hostname -> repository.url name change 2022-07-23 22:10:13 +05:30
0d1f42d5e4
feat: async crawl 2022-07-15 20:59:35 +05:30
993ba2fe25
feat: periodic deletion of old archives 2022-07-15 20:59:35 +05:30
31fd0f0c51
chore: clippy lints 2022-07-15 20:59:32 +05:30
9a3425eabe
fix & chore: handle hostname/URL cleaning within trait implementation
SUMMARY
    Renamed function parameters from "hostname" to "url" wherever a
    url::Url is received

    db_core::get_hostname is renamed to db_core::clean_url, better
    describing its new implementation

    forge_core::get_hostname is renamed to forge_core::get_url to better
    describe its new implementation

URL PROCESSING METHODS

    federate/federate-core
	Parses URL and returns only the hostname

    db/db-core
	Parses URL, cleans it by removing path and query parameters and
	returns the end result
2022-07-15 20:58:40 +05:30
fe8bd2fb26
fix & chore: handle hostname/URL cleaning within trait implementation
SUMMARY
    Renamed function parameters from "hostname" to "url" wherever a
    url::Url is received

    db_core::get_hostname is renamed to db_core::clean_url, better
    describing its new implementation

    forge_core::get_hostname is renamed to forge_core::get_url to better
    describe its new implementation

URL PROCESSING METHODS

    federate/federate-core
	Parses URL and returns only the hostname

    db/db-core
	Parses URL, cleans it by removing path and query parameters and
	returns the end result
2022-07-15 20:52:47 +05:30
29f5556586
Merge pull request #7 from dat-adi/master
docs: Minor update in documentation and variable refactoring
2022-07-05 15:47:36 +05:30
Dat Adithya
eb97ded32c
Merge branch 'forgeflux-org:master' into master 2022-07-05 14:26:55 +05:30
dat-adi
3bb4bf2e7d docs: Added in a section for workspace setup 2022-07-05 14:26:29 +05:30
dat-adi
a65115bfeb fix: Refactored grammatical errors 2022-07-05 14:05:57 +05:30
aba71115a7
fix: page counter increment 2022-06-09 15:03:14 +05:30
7e1903f807
feat: sqlx postgres: impl interface to get all forges 2022-06-09 14:38:52 +05:30
1ec46670ec
feat: def interface to get all forges 2022-06-09 14:36:22 +05:30
ef2707e4ee
feat: crawl generic over SCForge implementaiton and add forge type checks 2022-06-09 01:22:23 +05:30
1a3206b57a
chore: rm forge dead code 2022-06-04 21:33:13 +05:30
db1d7657a7
feat: create repo file if doesn't exist 2022-06-04 21:30:50 +05:30
d3066683da
feat: impl repository_exists for publiccodeyml 2022-06-04 21:30:14 +05:30
76cd9a0fd6
feat: def interface to check if repository exists 2022-06-04 21:29:59 +05:30
5e8c3a10cf
fix: explicit forge_core::User usage to avoid namespace conflicts 2022-06-04 20:58:31 +05:30
7b30c08f5f
feat: impl interface to get user data sqlx sqlite 2022-06-04 20:58:05 +05:30
653088376a
feat: def interface to get user data 2022-06-04 20:57:48 +05:30
54a1d28395
feat: impl archive user if not exists 2022-06-04 20:47:02 +05:30
2ebd53e612
feat: impl interface to check if user exists for publiccode 2022-06-04 20:45:16 +05:30
e2cf9460e5
feat: def interface to check if user exists 2022-06-04 20:45:01 +05:30
3edafd4f6d
fix: set rate-limit=0 for tests 2022-06-04 20:44:43 +05:30
399ef34f08
feat: migrate crawl accepts rate-limit configuratoin 2022-06-04 20:35:24 +05:30
b3d9dc38d9
feat: impl crawl accepts rate-limit configuration for gitea 2022-06-04 20:35:09 +05:30
7d60189b25
feat: crawl accepts rate-limit config 2022-06-04 20:34:41 +05:30
7e0c235275
feat: archive forge data if it doesn't exist 2022-06-03 23:14:57 +05:30
8d222d185b
feat: impl forge_exists for publiccodeyml 2022-06-03 23:10:54 +05:30
0d0f5270fd
feat: def forge_exists interface 2022-06-03 23:10:31 +05:30
6fb71e0641
feat: impl get_forge for sqlx sqlite 2022-06-03 23:05:29 +05:30
7b343a2cd5
feat: def get_forge interface 2022-06-03 23:05:09 +05:30
036a90e74a
feat: add CC0(public domain) license to published data 2022-05-26 22:50:40 +05:30
664f03d6eb
feat: document craweled data format and dir structure 2022-05-26 19:58:40 +05:30
8f51205d64
feat: dns TXT record-based configuration parser 2022-05-25 13:21:09 +05:30
99d14764ab
feat: CI: store build cache 2022-05-25 12:53:38 +05:30
c637be4499
fix: pagination link generator and related tests
SUMMARY
    When OptionalPage.page is None, the count was incremented twice
    leading to skipping next_page_link=2. Increment operation is now
    movied into `impl From<OptionalPage> for Page` for centralized
    incrementing, which also fixes this bug
2022-05-25 12:49:13 +05:30
11f98f09ea
feat: use cargo-chef for caching 2022-05-24 20:43:18 +05:30
99ddee7e8c
feat: use entrypoint for starting docker container 2022-05-24 20:31:50 +05:30
d8cd8db857
fix: dont serialize legal section if all sub-fields are none 2022-05-24 20:31:04 +05:30
10f33d8ea6
feat: sqlx sqlite create db file if not present 2022-05-24 20:30:43 +05:30