How to install build-system requirements depending on extra values given to pip?

[liZe]

When pip builds a wheel from a source package to install a package, I’d like to add build-time requirements depending on the extra values given to pip.

I’ve tried this in pyproject.toml:

[build-system]
requires = ["setuptools", "extra_package; 'xxx' in extra"]

And I’d like:

• pip install mypackage to install only setuptools before building the wheel
• pip install mypackage[xxx] to install setuptools and extra_package before building the wheel

But unfortunately, it looks like xxx is not given to the requirements solving function when pip builds the wheel. Pip logs tell me
Ignoring extra_package: markers '"xxx" in extra' don't match your environment.

Is there a way to do this?

[abravalheri]

Hi @liZe , I am afraid setuptools is not involved in the construction of the environment used for the build or the installation of [build-system] requires. It will only be invoked after that is already done.

If I understood your question correctly, the build-front end (e.g. pypa/build or pypa/pip) would have to take into consideration the extra of the package you are trying to install when evaluating the markers in [build-system] requires, but I am not sure if they support that (or plan to support)...

Probably the best place to ask information would be pypa/build or pypa/pip.

[liZe]

If I understood your question correctly, the build-front end (e.g. pypa/build or pypa/pip) would have to take into consideration the extra of the package you are trying to install when evaluating the markers in [build-system] requires

Exactly.

Probably the best place to ask information would be pypa/build or pypa/pip.

OK! If I have to rely on the build frontend because there’s no standard, my idea is probably bad. I’ll try to find a better way to do what I want.

[webknjaz]

OK! If I have to rely on the build frontend because there’s no standard, my idea is probably bad. I’ll try to find a better way to do what I want.

I think the only way is to call python -m build with an arg that disables venv management and populate the build deps manually. But that's beyond a simple pip install.

Also, extras (the thing in square brackets) aren't exposed to the build frontend as a part of the dist build. They are pulled in much later by the dependency resolver, when the sdist is already built.

Talking about PEP 517, I imagine you could try making a thin wrapper of setuptools.build_meta, adding your own hook that emits extra build deps, dynamically. This is how setuptools itself adds an extra dependency — wheel only when building wheels but not sdist.
Ideally, you could make use of config_settings that's passed to all the hooks and can be set via CLI when calling PEP 517 front-ends. Unfortunately, there's a bug in the tooling that spans across setuptools and pypa/build, at least (not sure about pip) that causes config_settings to always be passed as None to the hook that emits extra sdist build deps.
The only workaround for this I can think of, is defining a project-unique env var and setting it when calling the front-end, then emitting an extra dep from your backend wrapper.
I've had to do something like this recently: ansible/ansible@67bafaf#diff-ceb1ca63c229cc0872efd0416241565f02649322f8c9e03e1be66d16e2595201R116-R119.

[liZe]

Thanks a lot for your help.

I’ve finally used a dirty hack to install the required package during the wheel generation, and silently hide the problem if it fails (as it should only fail on systems that actually don’t require this step).