Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement pruning mechanism #6

Open
cmelone opened this issue Jan 26, 2024 · 2 comments
Open

Implement pruning mechanism #6

cmelone opened this issue Jan 26, 2024 · 2 comments
Assignees
Labels
feature New feature or request

Comments

@cmelone
Copy link
Collaborator

cmelone commented Jan 26, 2024

Problem/Opportunity Statement

We are going to be collecting every single successful build on AWS, but for the prediction, we are only going to need the past 4-5 builds for each potential build. We should remove extraneous entries in the database on a regular basis

What would success / a fix look like?

Cron job that prunes the database. Make sure to pay special attention to what you decide to remove. We probably want to come up with each combination of pkg+ver, compiler+ver, variants to make sure we don't inadvertently delete something that's useful.

@cmelone cmelone self-assigned this Jan 26, 2024
@cmelone cmelone added the feature New feature or request label Jan 26, 2024
@cmelone
Copy link
Collaborator Author

cmelone commented Apr 25, 2024

can do this through a k8s cron job, will need to create a container image with the script and access to the db/pvc

see this for an example

https://github.com/spack/spack-infrastructure/blob/main/k8s/production/custom/prune-buildcache/cron-jobs.yaml

@cmelone
Copy link
Collaborator Author

cmelone commented Sep 17, 2024

or do this in the existing app and essentially wait for 12 hours before running the pruning function

don't keep longer than 3 months

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant