Starchart: publiccodeyml deficiencies

[realaravinth]

@ruphy kindly prompted me to start a thread on here

Hello, I'm from the Forgeflux project(software forge federation) currently involved in writing a software forge spider for crawling and indexing public, Free(as in Freedom) software repositories hosted on independent forge instances. We have plans to implement official support for Gitea(WIP), GitLab and SourceHut.

Starchart will use forge-provided APIs(like the Gitea API) to request and store information on the publicly available repositories hosted on the instance. It will also periodically crawl indexed instances to update the data. The goal
is to create a Free, federated alternative to third-party indexes like awesomeopensource.com to improve visibility for projects hosted on
independently run forges.

Each Starchart repository, in addition to storing data in a database, will store indexed data in a Git repository. This Git repository will be
publicly available and will be used as the primary federation mechanism: federating Starchart instances will git pull each other's indexed data.

@6543 kindly referred me to this project for use as Starchart's file storage format. publiccodeyml requires(mandatory fields) data that can't be retrieved/automatically inferred from data that is provided by the forges' APIs. The following is a list of mandatory fields that can't be supported by Starchart:

1. platforms describes the platform the software will run on, which is hard to determine automatically by Starchart.
2. categories: defined as the list of words used to describe the software. It may be derived from repository tags, but tags on forges are optional so Starchart will be breaking the mandatory requirement.
3. developmentStatus: concept, development, beta, etc. (see full list in link), are not possible to determine.
4. softwareType is similar to platform but is more fine-grained, again, something that Starchart can't auto-generate.
5. description/[lang]/genericName
6. description/[lang]/longDescription may be derived from repository description, but it is an optional field in forges
7. description/[lang]/features mandatory per schema but not possible to auto generate
8. legal/licence hard to generate when repository doesn't contain a licence file
9. maintainance/type is hard to determine
10. maintenance/contractors: conditionally mandatory if maintainance/type is contract and is impossible for the same reasons as maintainance/type.
11. maintainance/contacts: can be generated from the repository owner's email address but isn't ethical to re-share when the owner hasn't explicitly authorized Starchart to share their email.
12. localisation/localisationReady
13. localisation/availableLanguages

Solution

I propose the following solution, but I'm open for suggestions:

1. Support fields that are possible fields that can be retrieved.
2. Check for the existence of a publiccode.yml in the repository at the file locations mandated by the publiccodeyml standard.
3. If publiccode.yml exists, store a link to the file in the database and the repository information file.

Thoughts? :)

[realaravinth]

I'm happy to report that partial support for publiccodeyml, as proposed above, is implemented in starchart! 🎉

The first instance of starchart is online at starchart.forgeflux.org with support for only Gitea at the moment(support for GitLab and Sourcehut will be added soon).

The crawled data is shared publicly from day 0: to download data, head over to /federate and grab the latest tarball. Please see here for more information about the format and structure of the published data. The format may change before starchart stabilizes, but I will keep the linked document updated.

As always, I'm open for any suggestions that you might have :)

Cheers!

[bzg]

Congrats! What is the license for published data? I suggest CC0 but maybe you have some constraints.

[realaravinth]

Thanks!

I didn't think about the license because starchart is simply republishing data that is available in public domain, but I'll explicitly mention that the information is in public domain :)

[realaravinth]

Done! CC0 is included with published data.

[bzg]

Great, thanks!