Skip to content

feat(pypi): expose the pyproject.toml extras with pip.configure - #4522

Draft
aignas wants to merge 38 commits into
bazel-contrib:mainfrom
aignas:feat.aignas.pip-configure-extras
Draft

aignas wants to merge 38 commits into
bazel-contrib:mainfrom
aignas:feat.aignas.pip-configure-extras

Conversation

@aignas

@aignas aignas commented Nov 17, 2026

Copy link
Copy Markdown
Collaborator

Stacked on #4395 and #4490.

Lock files often do not record the extras that a project requests, e.g.
uv pip compile and pip-compile --strip-extras drop them, so with the
per-extra hub targets the @pypi//requests:[socks] target does not exist
and @pypi//requests does not include the extra, even though the
pyproject.toml asks for requests[socks].

With this change pip.configure reads the extras from the requirements in
the selected pyproject.toml dependency groups ([project] dependencies,
[project.optional-dependencies] and [dependency-groups]) and adds them
to the hub, so the user only has to point the hub at the pyproject.toml:

pip.parse(
    hub_name = "pypi",
    pyproject_toml = "//:pyproject.toml",
    requirements_lock = "//:requirements_lock.txt",
)
pip.configure(
    hub_name = "pypi",
    dependency_groups = ["default", "group:dev"],
)

select_dependency_groups now also returns the extras of the selected
requirements, hub_exposure returns them for each configured hub and
add_extras adds them to the whl_config_setting values of the hub
whl_map, which the hub already uses to render the [<extra>] targets and
the available extras for the spoke repositories. With
dependency_groups = ["*", "transitive"] all of the packages stay public
and the extras of all of the groups are added. The environment markers of
the requirements are ignored, so the extras are added on all of the
platforms.

Work towards #2786

aignas added 30 commits November 8, 2026 02:07
…oups

Read the dependencies, the optional dependencies and the PEP 735
dependency groups from the pyproject.toml and add a helper to select the
packages using dependency group specifiers.
Add the `hide_unexposed_packages` attribute to the hub repository. When set,
the aliases of the packages that are not exposed are only visible to the
wheel repositories that depend on them.
Add the `pip.configure` tag class that limits the packages that a hub
exposes to the ones listed in the dependency groups of a `pyproject.toml`
file. The other packages of the hub are still used by the wheels that need
them, but their aliases are not public.
Otherwise the unified hub would point to the hidden aliases of the hub and
fail with a visibility error instead of the missing package error.
pip.configure with ["*", "transitive"] used the packages that are
available on all platforms, which made the platform-specific packages
private. Treat it as if the hub was not configured instead, which keeps
these packages public and out of all_requirements as before.
…igure

Replace the nested comprehensions and the long inline conditions with
explicit loops and use types.is_string.
Keep a single wiring test for pip.configure in the extension tests, as the
selection is tested in hub_exposure_tests, and test that the pip.default
pyproject_toml is used. Also link the issue in the news entry.
…igure-extras

# Conflicts:
#	python/private/pypi/BUILD.bazel
#	python/private/pypi/hub_repository.bzl
#	python/private/pypi/render_pkg_aliases.bzl
select_dependency_groups now also returns the extras that the selected
requirements request, e.g. requests[socks], so that pip.configure can
expose them through the hub.
The extras requested in the selected pyproject.toml dependency groups are
added to the hub, so @pypi//requests:[socks] exists and @pypi//requests
includes the extra even if the lock file does not request it.

hub_exposure now returns the extras together with the exposed packages
and add_extras adds them to the config settings of the hub whl_map.
@aignas

aignas commented Nov 17, 2026

Copy link
Copy Markdown
Collaborator Author

This PR was drafted by a Claude Code agent as a design proposal for exposing the pyproject.toml extras through pip.configure. It is stacked on #4395 and #4490, so the diff against main includes those changes. The human author (aignas) will handle the review comments; no agent is monitoring this PR to reply automatically.

This is an agent-generated comment, automatically generated.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant