Skip to content

refactor(entry_points): convert py_console_script_gen build tool to Rust - #4512

Open
rickeylev wants to merge 5 commits into
mainfrom
rickeylev/shrubbery/02-py-console-script-gen
Open

rickeylev wants to merge 5 commits into
mainfrom
rickeylev/shrubbery/02-py-console-script-gen

Conversation

@rickeylev

@rickeylev rickeylev commented Nov 16, 2026 •

Copy link
Copy Markdown
Collaborator

Generating console script entry points via Python requires executing a
full Python binary during py_console_script_gen actions.

Add //crates/py_console_script_gen:py_console_script_gen_lib with an
entry_points.txt reader, register py_console_script_gen on
shrubbery, and wire the rule to get_shrubbery_tool.

Work towards #4250

@rickeylev

rickeylev commented Nov 16, 2026 •

Copy link
Copy Markdown
Collaborator Author

This PR was prepared by an agent under human direction. Review comments are triaged by the human author; agent-drafted replies and changes are reviewed and approved by the author before posting.

This is an agent-generated comment unreviewed by the user.

Comment thread crates/py_console_script_gen/src/lib.rs Resolved
Comment thread crates/py_console_script_gen/src/lib.rs Resolved
Comment thread crates/py_console_script_gen/src/lib.rs Resolved
Comment thread crates/py_console_script_gen/src/lib.rs Resolved
Comment thread crates/py_console_script_gen/src/lib.rs Resolved
@rickeylev
rickeylev force-pushed the rickeylev/shrubbery/02-py-console-script-gen branch from 727ea34 to fe584c3 Compare November 16, 2026 05:09
@rickeylev

Copy link
Copy Markdown
Collaborator Author

AGENT: fix ci failiure

Base automatically changed from rickeylev/shrubbery/01-py-wheel-dist to main November 17, 2026 17:38
Generating console script entry points via Python requires executing a
full Python binary during `py_console_script_gen` actions.

Add `//crates/py_console_script_gen:py_console_script_gen_lib` with an
`entry_points.txt` reader and PEP 503 distribution name normalizer,
register `py_console_script_gen` on `shrubbery`, and wire the rule to
`get_shrubbery_tool`.
Use argfile's parse_fromfile directly instead of a per-crate flagfile
wrapper, adopt the USAGE constant, flags module, anyhow errors, and
long-only arguments used by the other shrubbery subcommands, and fold
run_with_options into run(&Options).
Pre-filter blank and comment lines via lines().map().filter() instead of
early continue guards, and point the generated script's comment at this
crate (with the sys.path explanation carried over into the crate docs)
now that the Python module is only a fallback.
…nc CLI

Use a short output_base under the outer Bazel output_base on Windows in tests/integration/runner.py so nested Rust tool builds stay within MAX_PATH, unwrap tool.executable[DefaultInfo].files_to_run in py_console_script_gen.bzl, and align ParsedArgs and definition ordering in crates/py_console_script_gen.
@rickeylev
rickeylev force-pushed the rickeylev/shrubbery/02-py-console-script-gen branch from 7e165fd to 6a934f9 Compare November 17, 2026 17:38
@rickeylev
rickeylev marked this pull request as ready for review November 17, 2026 17:39
@rickeylev
rickeylev requested a review from aignas as a code owner November 17, 2026 17:39
@rickeylev
rickeylev marked this pull request as draft November 17, 2026 17:39
On local Bazel test executions, pytest tmp_path already lives inside the outer Bazel execroot, causing _windows_short_base_dir(tmp_path / 'no_execroot') to match the outer execroot parent. Use an explicit synthetic path with no execroot ancestor.
@rickeylev
rickeylev marked this pull request as ready for review November 19, 2026 04:26
@rickeylev
rickeylev removed this pull request from stack #4513 November 19, 2026 04:30
@rickeylev

Copy link
Copy Markdown
Collaborator Author

@Solumin more rust. A conceptualy simple one (basically generate from X import Y; Y()), but has to do so by parsing entry_points.txt, which is an ini-style config file.

@rickeylev
rickeylev added this pull request to stack #4536 November 19, 2026 04:30

@aignas aignas left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving because the Python code may have the template in-lined, but since this is a compiled language that we are talking, I think we may want to keep this as a file with something like:

shrubbery template template.txt entry_points.txt --console-script --console-script-guess --shebang.

The split can be done in a followup.

Comment on lines +96 to +117
format!(
"{shebang_prefix}import sys

# See @rules_python//crates/py_console_script_gen for explanation
if getattr(sys.flags, \"safe_path\", False):
# We are running on Python 3.11 and we don't need this workaround
pass
elif \".runfiles\" not in sys.path[0]:
sys.path = sys.path[1:]

try:
from {module} import {attr}
except ImportError:
entries = \"\\n\".join(sys.path)
print(\"Printing sys.path entries for easier debugging:\", file=sys.stderr)
print(f\"sys.path is:\\n{{entries}}\", file=sys.stderr)
raise

if __name__ == \"__main__\":
sys.exit({entry_point}()) # pyrefly: ignore[not-callable]
"
)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I'd prefer this template to be a file in the source code. We should not need to recompile shrubbery to just change the template as long as the variables stay the same.

Any reasons to not do this?

.map(|raw_line| (raw_line, raw_line.trim()))
.filter(|(_, trimmed)| !trimmed.is_empty() && !trimmed.starts_with(['#', ';']));
for (raw_line, trimmed) in lines {
if let Some(section) = trimmed.strip_prefix('[').and_then(|s| s.strip_suffix(']')) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment on lines +64 to +65
.map(|raw_line| (raw_line, raw_line.trim()))
.filter(|(_, trimmed)| !trimmed.is_empty() && !trimmed.starts_with(['#', ';']));

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Carrying around the raw line and the trimmed line smells very weird. Use one or the other.

Or perhaps just use the configparser crate, which matches the Python configparser library?

.map(|raw_line| (raw_line, raw_line.trim()))
.filter(|(_, trimmed)| !trimmed.is_empty() && !trimmed.starts_with(['#', ';']));
for (raw_line, trimmed) in lines {
if let Some(section) = trimmed.strip_prefix('[').and_then(|s| s.strip_suffix(']')) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For example, instead of using trimmed here --- which is the only usage of trimmed in this whole thing --- you could instead do:

raw_line.split_whitespace().collect::<Vec<_>>() == &["[", "console_scripts", "]"]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you wanted to be really fancy, you could do this "scan until we find [console_scripts]" thing in the lines iterator with skip_while.

Comment on lines +76 to +82
if let Some((raw_key, raw_val)) = raw_line.split_once(['=', ':']) {
let key = raw_key.trim();
let val = raw_val.trim();
if !key.is_empty() && !val.is_empty() {
console_scripts.insert(key.to_owned(), val.to_owned());
}
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So everything from [console_scripts] to the end of the file is console script?

};

format!(
"{shebang_prefix}import sys

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: more idiomatic to use raw strings when there's a bunch of quotation marks in a string.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants