Repository navigation
Conversation
## Problem
`ebuild build` fails for any project whose path contains a space -- on
Windows that includes most home directories:
ninja: error: build\build.ninja:24: expected build command name
build C:\Users\Prakhar Maheshwari\AppData\...\obj\app\main.o: cc main.c
^ near here
Paths are interpolated into build statements as-is. Ninja ends the output
list at the first unescaped `:` and splits on unescaped spaces, so neither a
Windows drive letter nor a directory with a space survives.
The failure mode is worse than the error above. Escaping only the colon makes
the manifest parse, and `ninja -t targets all` then reports:
C:/Users/Prakhar: touchit
Maheshwari/AppData/.../dir: touchit
with: touchit
space/out.o: touchit
Four targets where one was meant, no error, no warning. The build then fails
later with a message pointing nowhere near the cause, or quietly builds
something other than what was asked for.
## Fix
Add `escape_ninja_path()` and apply it to every path written into a build
statement: compile edge outputs and sources, link inputs, dependency archives,
and executable/library outputs.
`$` is replaced first. The space and colon replacements introduce `$`
characters, and escaping those again would yield `$$ ` -- a literal dollar
followed by a separator -- rather than an escaped space. This matches what
CMake's Ninja generator does (`cmGlobalNinjaGenerator::EncodePath`).
Deliberately *not* escaped:
- `compile_commands.json`. It is JSON consumed by clang tooling, not a Ninja
manifest; `$:` in a path there would be corruption. A test pins this.
- Command lines. Verified with `ninja -v` that Ninja shell-quotes `$in`/`$out`
itself, so an escaped path still reaches the compiler as one argument:
python argv.py -o "C:\Users\Prakhar Maheshwari\...\out.o"
ARG[0]=-o
ARG[1]=C:\Users\Prakhar Maheshwari\...\out.o
## Testing
The escaping rules were established empirically against ninja 1.13, not from
the manual: unescaped and space-only-escaped manifests fail to parse,
colon-only escaping parses into four wrong targets, and only colon + space
escaping yields the single intended target.
Six tests added as `TestNinjaPathEscaping`:
- three on the helper -- `$`/space/`:` are escaped, `$` is escaped before the
others, ordinary paths are untouched;
- `test_build_dir_with_a_space_is_escaped_in_the_manifest` -- no raw space
survives in a build statement's output list;
- `test_ninja_parses_a_spacey_path_as_one_target` -- runs real
`ninja -t targets all` and asserts 2 targets, not one per path fragment.
This is the check a string assertion cannot make: an unescaped path parses
cleanly and is simply wrong;
- `test_escaping_does_not_leak_into_compile_commands` -- guards the boundary
above.
Verified the tests detect the bug: with escaping replaced by a passthrough,
5 of the 6 fail, along with the end-to-end
`test_editing_a_header_triggers_a_rebuild`. With the fix, all pass.
`test_editing_a_header_triggers_a_rebuild` now passes for the first time. It
had been skipping because the declared `ninja` dependency was absent from the
environment, and failing once installed -- its `tmp_path` contains a space.
Full suite: 212 passed, 5 failed (was 6). The 5 remaining are pre-existing and
unrelated: 3 initramfs, 2 shared-library link rule.
## Known, not addressed here
Flag values are still interpolated unquoted: `cflags = ... -I<dir>` and
`cc = <path>`. A compiler path or include directory containing a space will
split into two shell arguments, and a `$` in one would be read as a Ninja
variable reference. That is shell quoting of user-supplied values rather than
manifest-path escaping -- a different mechanism, sent separately.
Signed-off-by: Prakhar Maheshwari <mpr@stordocktech.com>
Signed-off-by: Prakhar Maheshwari <prakharmaheshwari96@gmail.com>
srpatcha
left a comment
There was a problem hiding this comment.
Thanks for this — the diagnosis is right, but the fix has already landed by
another route, so I want to save you the rebase.
This carries two things, and both are already in the tree.
The object-path restore is #66's, with the same target namespacing.
The escaping is commit 49cd368, "fix(ninja): escape paths in build statements
so Windows builds are valid", already merged. Your escape_ninja_path() and the
merged _ninja_path() are the same function with the same semantics, including
escaping $ first so the escapes introduced afterwards are not re-escaped.
Your docstring is better, though, and concretely so — it names what actually goes
wrong rather than only that something does:
C:\\Users\\Jane Doe\\build\\app.obecomes four targets (C,
...\\Jane,Doe,build\\app.o) instead of one.
"Silently parses into several wrong targets" is the part that makes this bug
frightening rather than annoying, and the merged version does not say it. Happy
to take that wording across if you want it kept.
Where that leaves this PR
It also touches dispatch.py, which conflicts with #66 — already approved, and
the PR that repairs master's SyntaxError (#87). Verified: this branch does not
apply cleanly on top of it.
Rather than resolving the conflict, I would close this. Nothing in it is wrong;
it is second to arrive at the same place.
If part of this does something #66 and 49cd368 do not, say which and I will
look again — I compared the object-path namespacing and the escaping semantics,
not every line.
Something open, if you want it
#85 — eos and eBoot both define CMake targets named test_crypto and
test_multicore, and since ebuild composes them with add_subdirectory, the
integration build cannot configure at all. Same class of bug as the ones you have
been finding, one repository wider.
Description
Problem
ebuild buildfails for any project whose path contains a space — on Windowsthat includes most home directories:
Paths are interpolated into build statements as-is. Ninja ends the output list
at the first unescaped
:and splitther a Windowsdrive letter nor a directory with a space survives.
The failure mode is worse than the error above. Escaping only the colon
makes the manifest parse, and `ninja:
Four targets where one was meant — no error, no warning. The build then fails
later with a message pointing nowherbuilds
something other than what was asked for.
Changes
ebuild/build/ninja_backend.py— addescape_ninja_path()and apply it toevery path written into a build statnd sources,
link inputs, dependency archives, and executable/library outputs.
$is replaced first. The space and colon replacements introduce$characters, and escaping those againl dollar
followed by a separator — rather than an escaped space. This matches what
CMake's Ninja generator does (
cmGlo).Deliberately not escaped:
compile_commands.json— JSONot a Ninjamanifest; a
$:in a path there would be corruption. A test pins this.Command lines — verified with uotes
$in/$outitself, so an escaped path still reaches the compiler as oneargument:
CHANGELOG.md— oneFixedentTesting
The escaping rules were established .13**, not
from the manual:
$)$:)Six tests added as `TestNinjaPathEsc
$/space/`: before theothers, ordinary paths are untouched;
survives in a build statement's output list;
test_ninja_parses_a_spacey_path_aninja -t targets all` and asserts 2 targets, not one per path fragment.**This is the check a string asserped path
parses cleanly and is simply wrong;
above.
Verified the tests detect the bug: with escaping replaced by a passthrough,
5 of the 6 fail, along with the end-
test_editing_a_header_triggers_a_rebuild. With the fix, all pass.test_editing_a_header_triggers_a_rebuildnow passes for the first time.It had been skipping because the decabsent from
the environment, and failing once installed — its
tmp_pathcontains a space.Full suite: 212 passed, 5 failed (was 6). The 5 remaining are pre-existing
and unrelated: 3 initramfs, 2 shared