Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Setup automated workflow to test bumping versions #320

Open
timsnyder opened this issue Mar 31, 2022 · 1 comment
Open

Setup automated workflow to test bumping versions #320

timsnyder opened this issue Mar 31, 2022 · 1 comment
Labels
enhancement Enhancement or request

Comments

@timsnyder
Copy link
Contributor

In our CI setup which is using conda-forge ci infrastructure, versions are controlled in several layers:

  1. hardcoded immediate versions in map/conda.recipe/meta.yaml. As of 3b1b571 we don't have any but we used to have linux clang version pinned to an immediate value
  2. jinja 2 references variables defined by map/conda.recipe/conda_build_config.yaml and the global (https://github.com/conda-forge/conda-forge-pinning-feedstock/blob/main/recipe/conda_build_config.yaml)

Each time changes are made to the conda-forge-pinning-feedstock a new datestamp versioned package is uploaded to anaconda.org
. We receive updates of those versions whenever we run conda smithy rerender.

Currently, only developers make changes to map/conda.recipe/conda_build_config.yaml whenever we happen to notice that it has been a long time and we probably should bump versions. We currently constrain two things that might need to be bumped:

  1. version of clang used on linux. Since conda-forge doesn't use clang to build on linux, it's version isn't pinned in the global conda-forge-pinnings-feedstock. I recently constrained it to the current "absolute latest" version available, which at that time was 13. I did that because conda-forge is still using 12 to build all of it's packages on macos and I thought using 13 on linux and 12 on mac would give increased coverage of clang versions without increasing the number of builds that map CI does for every PR.
  2. a single version of python. conda-forge typically supports three releases of python in a rolling fashion. Today, that is 3.7, 3.8 and 3.9. They are in the process of preparing to roll off 3.7 and onto 3.10. We don't care to regress against three versions of Python, so I constrained it to one in map/conda.recipe/conda_build_config.yaml to reflect the manual changes that had been made to the conda smithy rerender generated CI files

You can see the combined results of the two levels of conda_build_config.yaml, applied against map/conda.recipe/meta.yaml in the 'variant' files that are generated and committed into https://github.com/sparcians/map/tree/master/.ci_support. Each YAML file in that directory represents a single configuration that will be run in CI and that the other platform specific ci support files (currently https://github.com/sparcians/map/tree/master/.azure-pipelines and https://github.com/sparcians/map/tree/master/.circleci) will reference. In other words, when you use a variable like linux_clang_version in map/conda.recipe/meta.yaml, if you provide a list of versions to use in one of the conda_build_config.yaml files, conda smithy rerender will calculate the matrix of builds and generate a single yaml file with a single version for everything in map/.ci_support.

@knute-sifive, if you want something automated, I could setup something that:

  1. analyzes contents of map/conda.recipe/conda_build_config.yaml and map/conda.recipe/meta.yaml to determine whether there are newer versions of the packages pinned there. We will need some kind of configuration to tell this part of the script if there are versions that it should not attempt to bump because have pinned them knowingly because of incompatibility issues (that may not be caught by map CI).
  2. runs conda smithy rerender
  3. applies a set of patches to maintain manual edits we've made to the CI files (I'll also need to document how to maintain these patches).
    if it sees changes were made, creates a PR that runs CI using the new versions that you can see the results of running with the new version and choose to merge it and pick them up

This wouldn't have to run often. We could do it monthly or quarterly but it would at least give an automated nudge when new versions of the compilers or boost become available on conda-forge without someone needing to be proactive.

@ghost
Copy link

ghost commented Mar 31, 2022

I like the idea of running something monthly or even quarterly. I've already been caught a couple of times (and had to scramble) to get sparta up to par with the latest packages. I'd love to get a "heads up."

@klingaard klingaard added the enhancement Enhancement or request label Dec 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Enhancement or request
Projects
None yet
Development

No branches or pull requests

2 participants