Skip to content

fix(ebuild): escape ninja-special characters in generated build paths - #80

Merged
srpatcha merged 2 commits into
embeddedos-org:masterfrom
prakhar7017:fix/ninja-escape-paths-with-spaces
Sep 1, 2026
Merged

srpatcha merged 2 commits into
embeddedos-org:masterfrom
prakhar7017:fix/ninja-escape-paths-with-spaces

Conversation

@prakhar7017

Copy link
Copy Markdown
Contributor

Description

Stacked on #79 (fix/ninja-object-path-missing), which is stacked on #78 .
Merge those first.

Problem

ebuild build fails for any project whose path contains a space — on Windows
that includes most home directories:

ninja: error: build\build.ninja:24:
build C:\Users\Prakhar Maheshwari\AppData\...\obj\app\main.o: cc main.c
        ^ near here

Paths are interpolated into build statements as-is. Ninja ends the output list
at the first unescaped : and splitther a Windows
drive letter nor a directory with a space survives.

The failure mode is worse than the error above. Escaping only the colon
makes the manifest parse, and `ninja:

C:/Users/Prakhar: touchit
Maheshwari/AppData/.../dir: touchit
with: touchit
space/out.o: touchit

Four targets where one was meant — no error, no warning. The build then fails
later with a message pointing nowherbuilds
something other than what was asked for.

Changes

ebuild/build/ninja_backend.py — add escape_ninja_path() and apply it to
every path written into a build statnd sources,
link inputs, dependency archives, and executable/library outputs.

$ is replaced first. The space and colon replacements introduce $
characters, and escaping those againl dollar
followed by a separator — rather than an escaped space. This matches what
CMake's Ninja generator does (cmGlo).

Deliberately not escaped:

  • compile_commands.json — JSONot a Ninja
    manifest; a $: in a path there would be corruption. A test pins this.

  • Command lines — verified with uotes
    $in/$out itself, so an escaped path still reaches the compiler as one
    argument:

    python argv.py -o "C:\Users\Prakhar Maheshwari\...\out.o"
    ARG[0]=-o
    ARG[1]=C:\Users\Prakhar Maheshwari\...\out.o
    

CHANGELOG.md — one Fixed ent

Testing

The escaping rules were established .13**, not
from the manual:

Manifest Result
no escaping fails to parse
space only ($ ) fails to parse
colon only ($:) parses — into 4 wrong targets
colon + space parses into the 1

Six tests added as `TestNinjaPathEsc

  • three on the helper — $/space/`: before the
    others, ordinary paths are untouched;
  • `test_build_dir_with_a_space_is_esaw space
    survives in a build statement's output list;
  • test_ninja_parses_a_spacey_path_a ninja -t targets all` and asserts 2 targets, not one per path fragment.
    **This is the check a string asserped path
    parses cleanly and is simply wrong;
  • `test_escaping_does_not_leak_into_e boundary
    above.

Verified the tests detect the bug: with escaping replaced by a passthrough,
5 of the 6 fail, along with the end-
test_editing_a_header_triggers_a_rebuild. With the fix, all pass.

test_editing_a_header_triggers_a_rebuild now passes for the first time.
It had been skipping because the decabsent from
the environment, and failing once installed — its tmp_path contains a space.

Full suite: 212 passed, 5 failed (was 6). The 5 remaining are pre-existing
and unrelated: 3 initramfs, 2 shared

## Problem

`ebuild build` fails for any project whose path contains a space -- on
Windows that includes most home directories:

    ninja: error: build\build.ninja:24: expected build command name
    build C:\Users\Prakhar Maheshwari\AppData\...\obj\app\main.o: cc main.c
            ^ near here

Paths are interpolated into build statements as-is. Ninja ends the output
list at the first unescaped `:` and splits on unescaped spaces, so neither a
Windows drive letter nor a directory with a space survives.

The failure mode is worse than the error above. Escaping only the colon makes
the manifest parse, and `ninja -t targets all` then reports:

    C:/Users/Prakhar: touchit
    Maheshwari/AppData/.../dir: touchit
    with: touchit
    space/out.o: touchit

Four targets where one was meant, no error, no warning. The build then fails
later with a message pointing nowhere near the cause, or quietly builds
something other than what was asked for.

## Fix

Add `escape_ninja_path()` and apply it to every path written into a build
statement: compile edge outputs and sources, link inputs, dependency archives,
and executable/library outputs.

`$` is replaced first. The space and colon replacements introduce `$`
characters, and escaping those again would yield `$$ ` -- a literal dollar
followed by a separator -- rather than an escaped space. This matches what
CMake's Ninja generator does (`cmGlobalNinjaGenerator::EncodePath`).

Deliberately *not* escaped:

- `compile_commands.json`. It is JSON consumed by clang tooling, not a Ninja
  manifest; `$:` in a path there would be corruption. A test pins this.
- Command lines. Verified with `ninja -v` that Ninja shell-quotes `$in`/`$out`
  itself, so an escaped path still reaches the compiler as one argument:

      python argv.py -o "C:\Users\Prakhar Maheshwari\...\out.o"
      ARG[0]=-o
      ARG[1]=C:\Users\Prakhar Maheshwari\...\out.o

## Testing

The escaping rules were established empirically against ninja 1.13, not from
the manual: unescaped and space-only-escaped manifests fail to parse,
colon-only escaping parses into four wrong targets, and only colon + space
escaping yields the single intended target.

Six tests added as `TestNinjaPathEscaping`:

- three on the helper -- `$`/space/`:` are escaped, `$` is escaped before the
  others, ordinary paths are untouched;
- `test_build_dir_with_a_space_is_escaped_in_the_manifest` -- no raw space
  survives in a build statement's output list;
- `test_ninja_parses_a_spacey_path_as_one_target` -- runs real
  `ninja -t targets all` and asserts 2 targets, not one per path fragment.
  This is the check a string assertion cannot make: an unescaped path parses
  cleanly and is simply wrong;
- `test_escaping_does_not_leak_into_compile_commands` -- guards the boundary
  above.

Verified the tests detect the bug: with escaping replaced by a passthrough,
5 of the 6 fail, along with the end-to-end
`test_editing_a_header_triggers_a_rebuild`. With the fix, all pass.

`test_editing_a_header_triggers_a_rebuild` now passes for the first time. It
had been skipping because the declared `ninja` dependency was absent from the
environment, and failing once installed -- its `tmp_path` contains a space.

Full suite: 212 passed, 5 failed (was 6). The 5 remaining are pre-existing and
unrelated: 3 initramfs, 2 shared-library link rule.

## Known, not addressed here

Flag values are still interpolated unquoted: `cflags = ... -I<dir>` and
`cc = <path>`. A compiler path or include directory containing a space will
split into two shell arguments, and a `$` in one would be read as a Ninja
variable reference. That is shell quoting of user-supplied values rather than
manifest-path escaping -- a different mechanism, sent separately.

Signed-off-by: Prakhar Maheshwari <mpr@stordocktech.com>
Signed-off-by: Prakhar Maheshwari <prakharmaheshwari96@gmail.com>

@srpatcha srpatcha left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this — the diagnosis is right, but the fix has already landed by
another route, so I want to save you the rebase.

This carries two things, and both are already in the tree.

The object-path restore is #66's, with the same target namespacing.

The escaping is commit 49cd368, "fix(ninja): escape paths in build statements
so Windows builds are valid", already merged. Your escape_ninja_path() and the
merged _ninja_path() are the same function with the same semantics, including
escaping $ first so the escapes introduced afterwards are not re-escaped.

Your docstring is better, though, and concretely so — it names what actually goes
wrong rather than only that something does:

C:\\Users\\Jane Doe\\build\\app.o becomes four targets (C,
...\\Jane, Doe, build\\app.o) instead of one.

"Silently parses into several wrong targets" is the part that makes this bug
frightening rather than annoying, and the merged version does not say it. Happy
to take that wording across if you want it kept.

Where that leaves this PR

It also touches dispatch.py, which conflicts with #66 — already approved, and
the PR that repairs master's SyntaxError (#87). Verified: this branch does not
apply cleanly on top of it.

Rather than resolving the conflict, I would close this. Nothing in it is wrong;
it is second to arrive at the same place.

If part of this does something #66 and 49cd368 do not, say which and I will
look again — I compared the object-path namespacing and the escaping semantics,
not every line.

Something open, if you want it

#85 — eos and eBoot both define CMake targets named test_crypto and
test_multicore, and since ebuild composes them with add_subdirectory, the
integration build cannot configure at all. Same class of bug as the ones you have
been finding, one repository wider.

@srpatcha
srpatcha merged commit 28e8532 into embeddedos-org:master Sep 1, 2026
15 of 24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants