Never use the pkg_resources metadata backend on python 3.13+

[sbidoul]

On python 3.13+ we don't support the fallback to the pkg_resources metadata backend, and don't attempt to detect .egg installed distributions (#12330).

In effect, this means pkg_resources is never used at all by pip on python 3.13 (to the best of my knowledge).

This does not mean we will not want to drop support for the pkg_resources backend before we drop support for python 3.12, but at least this places a limit on it.

@pypa/pip-committers what do you think?

[ichard26]

We should probably check in our redistributors to make sure they're OK with this change. /cc @FFY00 @eli-schwartz @stefanor @doko42 @kitterma @hroncok

TL;DR, we'd like to stop using pkg_resources as our metadata backend (e.g. used to discover installed packages) at some point. It's old and has warts that simply don't exist with the importlib-metadata backend. For more details, please see #7413.

Anyway, for a long time, pip has defaulted to using the importlib-metadata backend on Python 3.11 and higher, while sticking with pkg_resources otherwise. In addition, the pkg_resources backend is still used for all Python versions to discover .egg distributions (not .egg-info!) as importlib-metadata lacks support for them. Finally, there are various ways for end-users and redistributors to opt-out of the importlib-metadata backend.

We'd like to disable these escape hatches on Python 3.13 so we have a migration path to eliminating the pkg_resources backend entirely. Our question is whether you still need the pkg_resources backend on Python 3.13 or higher for any reasons? For example, if you still ship any .egg distributions on Python 3.13, pip would not be able to discover or uninstall those packages.

[eli-schwartz]

Gentoo does not ship any .egg files, which I believe means we shouldn't be affected by this change, correct?

[hroncok]

For example, if you still ship any .egg distributions on Python 3.13, pip would not be able to discover or uninstall those packages.

If we do, it's always some cmake-wrpapped-autotools-bash-script-calling-setup.py-with-weird-arguments situation. Such packages are hardly ever interacted with by the rest of the Python ecosystem.

And we don't want pip to uninstall RPM-installed packages. So that part is more than OK.

---

Anyway, on Fedora 42:

• conda-tests has eggs in /usr/share/conda/tests/data/env_metadata/envpy27osx/lib/python2.7/site-packages -- I don't care about Python 2.7 🙈
• deluge-common has some egg based plugins in /usr/lib/python3.13/site-packages/deluge/plugins/ but they don't seem to depend on pip to load them
• python3-distributed has test egg in /usr/lib/python3.13/site-packages/distributed/tests/
• Python has test egg in /usr/lib/python3.13/test/test_importlib/metadata/data
• /usr/lib64/python3.13/site-packages/urjtag-2021.3-py3.13-linux-x86_64.egg seems like a real egg installed by autotools, I will deal with this one

(based on repoquery -q --repo=rawhide -f '*.egg')

[hroncok]

/usr/lib64/python3.13/site-packages/urjtag-2021.3-py3.13-linux-x86_64.egg is caused by pypa/setuptools#3143 which has been without a response for a couple of years now.

I suggest it would be excellent to make setuptools not create eggs first before you land this, but perhaps you need to land this before you make this a priority for setuptools :/

[pfmoore]

We have no control over what setuptools does, unfortunately, but given that pypa/setuptools#3143 appears to be triggered only by using the deprecated setup.py install invocation, I don't think it's something we should hold pip up for.

[stefanor]

In Debian we have a couple of packages installed as .eggs. Usually these are expanded (.egg directories), but some are zip files:

$ apt-file search .egg | sed -ne 's,\.egg/.*,.egg/, p' | uniq
deluge-common: /usr/lib/python3/dist-packages/deluge/plugins/AutoAdd-1.8.egg/
deluge-common: /usr/lib/python3/dist-packages/deluge/plugins/Blocklist-1.4.egg/
deluge-common: /usr/lib/python3/dist-packages/deluge/plugins/Execute-1.3.egg/
deluge-common: /usr/lib/python3/dist-packages/deluge/plugins/Extractor-0.7.egg/
deluge-common: /usr/lib/python3/dist-packages/deluge/plugins/Label-0.3.egg/
deluge-common: /usr/lib/python3/dist-packages/deluge/plugins/Notifications-0.4.egg/
deluge-common: /usr/lib/python3/dist-packages/deluge/plugins/Scheduler-0.3.egg/
deluge-common: /usr/lib/python3/dist-packages/deluge/plugins/Stats-0.4.egg/
deluge-common: /usr/lib/python3/dist-packages/deluge/plugins/Toggle-0.4.egg/
deluge-common: /usr/lib/python3/dist-packages/deluge/plugins/WebUi-0.2.egg/
python3-astra-toolbox: /usr/lib/python3/dist-packages/astra_toolbox-2.1.0-py3.11-linux-x86_64.egg/
python3-ferret: /usr/lib/python3/dist-packages/pyferret-7.65-py3.12-linux-x86_64.egg/
python3-nose2: /usr/lib/python3/dist-packages/nose2/tests/functional/support/scenario/tests_in_unzipped_eggs/pkgunegg-0.0.0-py2.7.egg/
python3-pkg-resources: /usr/lib/python3/dist-packages/pkg_resources/tests/data/my-test-package_unpacked-egg/my_test_package-1.0-py3.7.egg/
python3-pmix: /usr/lib/python3/dist-packages/pypmix-5.0.5-py3.12-linux-x86_64.egg/
python3-zope.testrunner: /usr/lib/python3/dist-packages/zope/testrunner/tests/testrunner-ex-251759/eggs/foo.bar-1.2-py2.5.egg/
quantlib-python: /usr/lib/python3/dist-packages/QuantLib-1.35-py3.12-linux-x86_64.egg/
quantlib-python: /usr/lib/python3/dist-packages/QuantLib-1.36-py3.12-linux-x86_64.egg/

$ apt-file search .egg | grep '\.egg$'
cappuccino: /usr/lib/python3/dist-packages/cappuccino-0.5.1-py3.12.egg
libpython3.12-testsuite: /usr/lib/python3.12/test/test_importlib/data/example-21.12-py3.6.egg
libpython3.13-testsuite: /usr/lib/python3.13/test/test_importlib/metadata/data/example-21.12-py3.6.egg
librust-python-pkginfo-dev: /usr/share/cargo/registry/python-pkginfo-0.6.3/tests/fixtures/build-0.4.0-py3.9.egg
pypy3-lib-testsuite: /usr/lib/pypy3.10/test/test_importlib/data/example-21.12-py3.6.egg
python3-distributed: /usr/lib/python3/dist-packages/distributed/tests/testegg-1.0.0-py3.4.egg
python3-ferret: /usr/lib/python3/dist-packages/gcircle-7.65-py3.12.egg
python3-ferret: /usr/lib/python3/dist-packages/pipedviewer-7.65-py3.12.egg
python3-nose2: /usr/lib/python3/dist-packages/nose2/tests/functional/support/scenario/tests_in_zipped_eggs/pkgegg-0.0.0-py2.7.egg
python3-pkg-resources: /usr/lib/python3/dist-packages/pkg_resources/tests/data/my-test-package_zipped-egg/my_test_package-1.0-py3.7.egg

Many of these are off sys.path, inside test suites (that in many cases should be packaged somewhere else).

We use EXTERNALLY_MANAGED to disallow pip installs, so I'm not really worried by these remaining corner cases.