Some metadata validation is missing from flit_core?

[woodruffw]

First of all, thanks a ton for flit! We are very, very happy with it on pip-audit 🙂

Someone raised an issue with us recently (pypa/pip-audit#353), noting a discrepancy between these two commands:

python -m flit build

# using flit_core's build backend internally
python -m build

In particular, the former (correctly) warns about invalid classifiers, while the latter doesn't:

$ python -m flit build --format wheel
Fetching list of valid trove classifiers                                   I-flit.validate
Unrecognised classifier: 'Programming Language :: Python :: 3 :: 3.10'     E-flit.validate
Unrecognised classifier: 'Programming Language :: Python :: 3 :: 3.7'      E-flit.validate
Unrecognised classifier: 'Programming Language :: Python :: 3 :: 3.8'      E-flit.validate
Unrecognised classifier: 'Programming Language :: Python :: 3 :: 3.9'      E-flit.validate
Config error: Invalid config values (see log)

and:

$ python -m build
* Creating venv isolated environment...
* Installing packages in isolated environment... (flit_core >=3.2,<4)
* Getting dependencies for sdist...
* Building sdist...
* Building wheel from sdist
* Creating venv isolated environment...
* Installing packages in isolated environment... (flit_core >=3.2,<4)
* Getting dependencies for wheel...
* Building wheel...
Successfully built pip_audit-2.4.3.tar.gz and pip_audit-2.4.3-py3-none-any.whl

In particular, this seems to be happening since flit.validate is not part of flit_core.

Is this intended behavior and, if so, is it documented anywhere? It makes sense to me that the flit package would contain more functionality than flit_core, but the discrepancy in validation was slightly surprising as an end user 🙂

[takluyver]

Thanks! It's sort of intended behaviour - the net result obviously isn't desirable, but the pieces that go into it are deliberate.

My expectation when I designed this was that package producers - the people who choose to use Flit to publish their package - would use the flit command line directly, and package consumers would use tools like pip that may call Flit as a backend interface (using the APIs from PEP 517). So the command-line interface does a bunch of checks, aiming to ensure that your package is A-OK before you publish it, whereas the backend API (which python -m build calls) assumes that you want to use a package, and therefore if it can build a wheel, it will.

build (the project providing python -m build) also hides the backend's (Flit's) output unless it fails. So we can't warn and carry on with building: the only way to tell you something is wrong is to error out completely. And the backend doesn't know who is using it or what they want to do. 🤔

[claui]

Thanks @takluyver! Your explanation helps a lot.

To rephrase from the perspective of a package repository maintainer:

• The up-front validation that flit performs is mainly intended for upstream project maintainers so they have a chance to learn about issues before they proceed to cut a release.

• As a maintainer of a (binary) package repository, I start with an already-released upstream source package in hand, and my goal is to just get the thing to build.
  In that context, I’m basically in the role of a package consumer, because I roughly have the same goal as a regular user who would run pip install.

So my take-away for future work as a package repository maintainer is:
I’m going to accept the (already-released) upstream source package as is, and just run build instead of invoking flit directly.
Makes sense?

[takluyver]

As a maintainer of a (binary) package repository, I start with an already-released upstream source package in hand, and my goal is to just get the thing to build. In that context, I’m basically in the role of a package consumer...

That sounds right. It depends a bit on the nature of your repository, but if there are problems with the package, you probably can't fix them as easily as the package authors (even if you have a patching mechanism, the downstream packaging ecosystems I've seen generally prefer not to use this unless necessary).

I’m going to accept the (already-released) upstream source package as is, and just run build instead of invoking flit directly.
Makes sense?

Makes sense. It also has the advantage that the same build command should work for any Python package, regardless of whether they use Flit, Setuptools, Poetry, Hatch, or other packaging tools.

Depending on how tightly your repository likes to control things, this can be a big simplification or a small one. python -m build will make a temporary build environment by default and install build dependencies into it. If your repository is relaxed enough to allow that, then you can build and install most Python packages with a simple formula. If your repository is stricter, then you'll need to work out which of your own packages build dependencies live in, and run build with the --no-isolation option to stop it using a temporary build environment.