Repository navigation
Conversation
The facts keep each sdist and each wheel that a configured pip.default platform can select for a requested Python version. The host is not used, so all hosts write the same lockfile. The facts record the platforms in wheel_filter, and facts filtered for other platforms are not read.
Wheel tags do not encode the patch version, so a new patch version must not invalidate the facts. Update the facts of the bzlmod_lockfile integration test.
Whether a wheel is selectable depends only on its python, ABI, and platform tags. Cache the result per tag set for the whole extension run, so that select_whl runs once per tag set instead of once per wheel.
Check that a wheel that only one platform selects is kept, that yank reasons of kept wheels survive the facts round trip, and that wheels with the same tags get the same result across packages.
Update the wheel_filter entries for the current default pip platforms.
simpleapi_download rebuilt each result without the flag, so parse_requirements warned about every wheel digest that the facts left out on purpose.
If the filter dropped every dist of a requested version, the facts lacked that version and every run fetched the index again. Keep the dists of such a version, so that the facts stay complete.
A hub with a `uv_lock` does not read the index, so its Python version only added wheels that no hub uses. It also tied the facts of every user to the Python version of the internal `rules_python_publish_deps` hub.
The facts now lack the wheels that no configured platform selects. An older rules_python ignores `wheel_filter`, so it must not read these facts as complete.
Name `hub_builder.bzl%hub_platforms` in the docstring, say that all sdists are kept, and note the one-time fetch after the upgrade.
Keep the new lines within the 100 column limit.
… facts Fact version v5 stores the longest common directory of each package's dist urls under the "" key, and the urls relative to it. The facts in memory keep the full urls, so only the facts written to the lockfile change.
The common directory of the dist urls now ends before any query string or fragment. A url that ends with "/" no longer becomes the common directory, so its relative key is never "" and does not overwrite the common directory.
A dist fact that is not a dict, or has no string common directory under the "" key, is now treated as absent. Before, its urls were read as relative urls.
Bazel's lockfile merge driver merges the facts of an extension shallowly, so it replaces a package's dist fact as a whole.
…tory Store each package's dist urls as a dict keyed by their common directory instead of under a `""` key. The layout explains itself and needs no special key.
The facts_cache docstring said to persist its in-memory facts, which keep the full URLs. facts_cache now returns the lockfile facts itself, so no caller can persist the uncompacted shape.
Use "URL" as the rest of the file does, and wrap the long schema line. State the real merge invariant: each group has its own common directory, so any union of groups expands to the full URLs.
The facts move to v5, so the first run after the upgrade fetches the index again. Tell --lockfile_mode=error users to update the lockfile.
Keep the code and docstrings within 100 columns, and use "URL" as the rest of the file does.
Check that different schemes, or absolute and relative URLs, give the empty common directory. Check that a dist fact with more than one directory group, as a lockfile merge can make, expands to the full URLs.
Store the facts with fact version v5.
Author
|
I prepared this PR with help from an AI coding agent. I, the human author, review and approve every agent-drafted reply and change before it is posted. No agent replies on this PR on its own. |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
DO NOT MERGE: awaiting #4534
Stacked on #4534. Review only the commits from
feat(pypi): store dist urls relative to their common directory in the factsonward.The
MODULE.bazel.lockfacts key each distribution file by its full URL. An index usually serves all files of a package from one directory, so every entry repeats the same prefix. This makes the lockfile larger.Fact version v5 maps the longest common directory of each package's URLs to the entries. Each entry is keyed by its URL relative to that directory. The extension expands the URLs when it reads the facts and compacts them only when it writes the lockfile. The rest of the code still sees full URLs. A malformed dist fact, for example after a botched merge, counts as absent, so the extension fetches the index again.
The common directory depends on all URLs of a package. If a package goes from one file to more than one, the directory can get shorter, and every entry of that package changes in the lockfile. On PyPI, the files of a package are almost always in different hash directories, so the common directory of a package with more than one file is almost always
https://files.pythonhosted.org/packages/. A version change then keeps that directory.The extension does not read v4 facts, so the first run after the upgrade fetches the index again. If you use
--lockfile_mode=error, update the lockfile once.Example
The numbers come from the same root module as in #4534: a copy of
tests/integration/bzlmod_lockfilewith Python 3.13, the default platforms, and a requirements lock for the newest release of twelve packages. The baseline is #4534, so the table shows only the effect of this change.MODULE.bazel.lockpipextension factsThe facts for
requests2.34.2:Before
After