Claude/upgrade ape c library mm z gd

CLAUDE.md

@@ -397,7 +397,18 @@ that the instrument notices.*
 nothing declares), `apehdr-sweep.py` (every `sys/include/ape` header
 standalone, plus a TOGETHER case for collisions only a combination can
 show), `rceq-sweep.py` (**gates**: a bare `=` in an mkfile recipe cannot
-parse), `mkcont-sweep.py` (**gates**: a logical line mk cannot parse. TWO
+parse. ***ITS `RULE` WANTED TWO COLONS AND SO READ ONLY ATTRIBUTE
+RULES*** — `install:V:` and nothing else — so every ordinary
+`target: prereq` recipe in the tree was invisible to it: **870
+recipe lines before, 1859 after**, with 0 findings both times. *A
+gating sweep reporting 0 over lines it cannot see* is the failure its
+own header already warned about for backquotes. **Found by control,
+not by reading**: a bare `=` planted in `cmd/php/mkfile`'s `clean:V:`
+was reported and the same `=` in one of that file's `%.$O:` rules was
+not — *planting the break in one place would have passed*, which is
+why `--control` now covers an attribute rule, a PLAIN rule and a
+backquote, plus two that must NOT fire, and fails on exactly the
+plain rule against the old regex), `mkcont-sweep.py` (**gates**: a logical line mk cannot parse. TWO
 shapes, and *the second was added after the first reported 0 over
 it*: a comment ending a continuation, and an INDENTED line that is
 not a recipe — `cmd/php/mkfile` ended its `OFILES` at an ordinary

docs/notes/php.md

@@ -410,6 +410,60 @@ anywhere in this port**. `json_scanner.c` and
 parsers and scanners and the 4.2MB `zend_vm_execute.h`. *A question
 that could have cost two more build tools cost one grep.*
 
+### The include layout, and the four headers under `cfg/`
+
+`TSRM/TSRM.h:20` says `#include <main/php_config.h>` while
+`Zend/zend_frameless_function.h` and `ext/opcache/zend_shared_alloc.c`
+say `#include <php_config.h>`. **Both spellings have to work**, and
+upstream gets that for free because the generated config really does
+sit at `main/php_config.h` under the php root.
+
+So the hand-kept headers live in `cmd/php/cfg/`, with the answers in
+`cfg/main/php_config.h` and `cfg/php_config.h` a one-line redirect --
+*two hand-maintained files that must agree are one file too many*,
+which is the warning CLAUDE.md already carries about perl's
+`Config.pm` and its `config.h`. One `-Icfg` answers all four
+spellings.
+
+***AND `-I.' IS DELIBERATELY ABSENT.*** The build writes its objects
+into `main/`, `Zend/`, `TSRM/` and `ext/` in the build directory, so
+`.` on the include path would put directories named after php's own
+source trees in front of php's own source trees. It is also why the
+headers are under `cfg/` rather than in a `main/` beside the mkfile:
+**`clean` does `rm -rf main`**, and a committed header there would be
+deleted by it.
+
+### Per-directory flags, and why they are 32 explicit rules
+
+Preprocessing all 513 sources on the host found **31 with an
+unresolvable include**, in two clusters, and the fix is flags rather
+than files:
+
+* **pcre2, 28 files** -- `pcre2_internal.h:68` includes
+  `<AbandonCompile>` on purpose when `PCRE2_CODE_UNIT_WIDTH` is unset.
+  Upstream passes `-DHAVE_CONFIG_H -DHAVE_MEMMOVE` and
+  `-I.../ext/pcre/pcre2lib`, where pcre2's own `config.h` lives.
+* **Keccak, 3 files** -- `KeccakP-200-SnP.h` is *excluded by a flag*,
+  `-DKeccakP200_excluded` and its two siblings, plus
+  `-I.../ext/hash/sha3/generic64lc`.
+
+***THE FLAGS CANNOT GO IN THE GLOBAL CFLAGS, AND THAT WAS MEASURED
+RATHER THAN FEARED.*** `-DHAVE_CONFIG_H` together with
+`-I.../pcre2lib` would hand `ext/json/json.c:18`,
+`json_encoder.c:18` and `main/php_odbc_utils.c:18` -- each of which
+is `#ifdef HAVE_CONFIG_H / #include <config.h>` -- **pcre2's**
+`config.h`. Not a failure: a different header, silently.
+
+The rules are explicit rather than patterned because mk's precedence
+between a general and a specific meta-rule is something I would be
+recalling rather than reading, there is no mk on the build host to
+check it with, and a wrong guess costs a VM round trip. *Explicit
+beats meta-rule in every mk, which needs no checking at all.*
+
+**All 31 were found in one host pass**, `gcc -E` over every source
+with the real include paths, filtered to `No such file or directory`.
+One sweep instead of 31 round trips.
+
 ## What the config.h decided, and why by hand
 
 The long reasoning is in the file itself; what matters as a summary:

sys/lib/tests/rceq-sweep.py

@@ -62,7 +62,22 @@
 ROOT = os.path.normpath(ROOT)
 TREE = os.path.join(ROOT, 'sys', 'src')
 
-RULE = re.compile(r'^[^=#]*:[A-Z]*:')
+# ***THIS USED TO BE `^[^=#]*:[A-Z]*:' AND SO REQUIRED TWO COLONS.***
+# That matches an ATTRIBUTE rule -- `install:V:', `test:QV:' -- and
+# nothing else, so an ordinary `target: prereq' header was not a rule
+# and its recipe lines were never read.  **A gating sweep reporting 0
+# over recipes it cannot see** is the one failure mode its own header
+# already warns about, two paragraphs down, about backquotes.
+#
+# It was found by control rather than by reading: a bare `=' planted
+# in `cmd/php/mkfile's `clean:V:' was reported, and the same `=' in
+# one of that file's `ext/...$O: $PHPSRC/....c' rules was not.
+# *Planting the break in only one place would have passed.*
+#
+# A rule header is now any non-indented, non-comment line with a colon
+# that is not an assignment -- `[^=#\t]*' stops at the `=', so
+# `CFLAGS=-I/a:/b' cannot match, and a `#' at column 0 cannot either.
+RULE = re.compile(r'^[^=#\t]*:')
 ASSIGN = re.compile(r'^[A-Za-z_][A-Za-z0-9_]*\s*=')
 # an `=' is legal only at the start of a command; these end one
 LEADING = re.compile(r"(^|[;&|{}()]|\|\||&&)\s*[A-Za-z_][A-Za-z0-9_]*\s*=")
@@ -135,7 +150,73 @@ def bare_eq(line):
     return '=' in s
 
 
+CONTROL = """\
+OFILES=\\
+\ta.$O\\
+\tb.$O
+
+install:V: $O.out
+\techo attr=rule
+
+%.$O: $SRC/%.c
+\techo plain=rule
+
+X=`{echo -DSIZE=8}
+
+ok:V:
+\techo '-Dquoted=fine'
+\tFOO=bar echo leading is legal
+"""
+
+
+def control():
+    """***THREE SHAPES, BECAUSE PLANTING THE BREAK IN ONE PLACE PASSED.***
+
+    This sweep GATES, and for a long time it read recipes only under an
+    ATTRIBUTE rule -- `install:V:' -- because `RULE' wanted two colons.
+    A bare `=' planted in `cmd/php/mkfile's `clean:V:' was duly
+    reported, which looked like proof the sweep worked, and the same
+    `=' in one of that file's `target: prereq' rules was not.  989
+    recipe lines across the tree were invisible.
+
+    So the control covers an attribute rule, a PLAIN rule, and a
+    backquoted assignment -- the three ways mk hands text to rc -- and
+    two things that must NOT be reported: a quoted `=' and a LEADING
+    assignment, which rc allows.
+    """
+    import tempfile
+    want = {6: 'attribute rule', 9: 'plain rule', 11: 'backquote'}
+    ok = True
+    with tempfile.TemporaryDirectory() as d:
+        p = os.path.join(d, 'mkfile')
+        open(p, 'w').write(CONTROL)
+        got = set()
+        for lineno, line in recipe_lines(p):
+            if bare_eq(line):
+                got.add(lineno)
+        for lineno, line in enumerate(CONTROL.split('\n'), 1):
+            rc = backquote_rc(line)
+            if rc is not None and bare_eq(rc):
+                got.add(lineno)
+        for ln, what in sorted(want.items()):
+            hit = ln in got
+            print('control: %-18s line %2d -- %s'
+                  % (what, ln, 'ok' if hit else 'FAILED, not reported'))
+            if not hit:
+                ok = False
+        extra = got - set(want)
+        for ln in sorted(extra):
+            print('control: line %d reported and must NOT be '
+                  '(quoted or leading `=\') -- FAILED' % ln)
+            ok = False
+        if not extra:
+            print('control: quoted and leading `=\' correctly ignored -- ok')
+    return 0 if ok else 1
+
+
 def main():
+    if '--control' in sys.argv:
+        return control()
     if not os.path.isdir(os.path.join(ROOT, 'sys', 'include', 'ape')):
         print('rceq-sweep: %s is not an APExp root -- refusing to report' % ROOT)
         print('(a sweep rooted in the wrong tree reports an empty one as a')

sys/src/ape/cmd/php/cfg/php_config.h

@@ -0,0 +1,17 @@
+/*
+ * php_config.h -- the SHORT spelling.
+ *
+ * ***THE SAME FILE IS INCLUDED TWO WAYS AND BOTH HAVE TO WORK.***
+ * `TSRM/TSRM.h:20' says `#include <main/php_config.h>' while
+ * `Zend/zend_frameless_function.h' and `ext/opcache/zend_shared_alloc.c'
+ * say `#include <php_config.h>'.  Upstream both resolve against the
+ * php root, where the generated config really does live at
+ * `main/php_config.h'.
+ *
+ * So the answers live in `cfg/main/php_config.h' and this is a
+ * redirect, rather than a second copy that could drift from the
+ * first.  *Two hand-maintained files that must agree are one file
+ * too many* -- the same reason perl's `Config.pm' and its `config.h'
+ * are called out in CLAUDE.md as a pair to keep in step.
+ */
+#include <main/php_config.h>