From ef4b78e4bb1371bbb0d18918e202cceb87913897 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 5 Nov 2026 12:10:36 +0000 Subject: [PATCH] php: all 514 translation units compile, via a host-built kencc `sys/lib/tests/hostkencc/php-mirror.sh --compile' answers `compiled 514, errors 0, missing 0'. The eight rounds before this one each cost a VM rebuild to learn about one compile error; the new mirror returned all eight at once, in six files, and all six are fixed and re-measured here without the VM. The flag set and the 514-entry source list are both DERIVED from `cmd/php/mkfile' with sed, so the mirror cannot drift from the build. Three of the eight were kencc's: * a pointer could not be assigned to a `bool'. `sub.c's stcompat refused it -- tasign[TUCHAR] is BNUMBER, which excludes BIND -- before `com.c's boolnorm, which was already there at all four converting sites, could run. C99 6.5.16.1 allows it outright. boolptr-test.c covers the four sites at three pointer values; sections 1 and 2 catch the wrong fix, which keeps the low byte. * `bool visible;' declared nothing. lex.c's indeclname lists every type specifier that can precede a declarator and was missing LBOOL, so the declarator was swallowed as a GNU visibility attribute. LCOMPLEX, LIMAGINARY, LCOMPLEXF and LCOMPLEXD are the rest of that set and were added from the same reading. * the __attribute__/__asm__ argument scanner counted parentheses inside string literals, so php's generated debug_gdb_scripts.c -- one __asm__ holding 1680 .ascii lines with one unmatched `(' in the text -- consumed the whole file. Literals are skipped whole now, escapes included, and the diagnostic names the construct it was actually scanning rather than __attribute__ for all six. agree.c is identical before and after all three, and --probe still answers ten regpair-min and eight regpair-probe with kc and vc failing at :242. clisp's 31 TUs still compile clean. Three were APE headers': * had none of the BSD timeval macros; timerisset, timerclear, timercmp, timeradd and timersub are there now, each guarded, as bash's and libressl's own copies are. * was missing facilities 2, 6, 7, 8 and 11 -- LOG_MAIL, LOG_LPR, LOG_NEWS, LOG_UUCP, LOG_FTP -- four of which are POSIX. * had the ns_ half and no compat half, so HEADER, C_IN, T_A and GETSHORT did not exist. is new and included from it as every other libc does; HEADER's layout is checked byte for byte against the host's own header. Fixing it uncovered that had been defining the same sixteen constants itself, which nothing could notice while no translation unit included both. It now includes and keeps only the resolver's own names. dn_skipname, defined by libap and declared by no header, is declared there too. One was the hand-answered config.h: HAVE_GETHOSTBYNAME_R yes with every arity no, so main/network.c's gethostname_re was defined by no arm. libap's is the glibc six-argument form. One was php's: arithmetic on a `void *' in ext/random/csprng.c, a GNU extension, in the one of that file's four sites this platform compiles. And one was the harness. A host-built 6c exited 1, deleted its own output and printed nothing for one input: go1.4's lib9/await.c mallocs its Waitmsg with a 200-byte message buffer and memsets only the struct, so a child that exits 0 leaves msg[0] as whatever malloc left there, and cc/lex.c:349 reads it as "the child failed". The file is +build !plan9, so nothing on the VM reads it. What is NOT measured is the link: the host has no libap and hostkencc deliberately does not build the linkers. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01WGAwvvTwDg2yknFkmZ3qzs --- CLAUDE.md | 56 ++++++- docs/notes/instruments.md | 61 +++++++ docs/notes/kencc.md | 145 +++++++++++++++++ docs/notes/libap.md | 98 ++++++++++++ docs/notes/php.md | 96 ++++++++++- docs/releases/0.6.md | 17 ++ sys/include/ape/arpa/nameser.h | 24 +++ sys/include/ape/arpa/nameser_compat.h | 130 +++++++++++++++ sys/include/ape/resolv.h | 52 +++--- sys/include/ape/sys/time.h | 55 +++++++ sys/include/ape/syslog.h | 26 +++ sys/lib/tests/boolptr-test.c | 156 ++++++++++++++++++ sys/lib/tests/hostkencc/php-mirror.sh | 185 ++++++++++++++++++++++ sys/src/ape/cmd/php/cfg/main/php_config.h | 17 +- sys/src/cmd/cc/lex.c | 94 ++++++++++- sys/src/cmd/cc/sub.c | 30 ++++ sys/src/external/go1.4/src/lib9/await.c | 19 ++- sys/src/external/php/ext/random/csprng.c | 10 +- 18 files changed, 1237 insertions(+), 34 deletions(-) create mode 100644 sys/include/ape/arpa/nameser_compat.h create mode 100644 sys/lib/tests/boolptr-test.c create mode 100755 sys/lib/tests/hostkencc/php-mirror.sh diff --git a/CLAUDE.md b/CLAUDE.md index 54db9e4ff1..bd1485c955 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -235,7 +235,18 @@ member of the wide-constant family***: x86-64 has `MOV r64, imm64` and NO 64-bit `CMP` immediate and NO 64-bit address DISPLACEMENT. `6c/cgen.c:346` folds `(x<` `#define`s `true`/`false`, so a TU that INCLUDES that header never reaches kencc's own C23 keywords. @@ -340,6 +351,23 @@ the compile line, `--compile` builds all 31 TUs and GATES. *Its `long_bitsize` is patched 64 -> 32 afterwards, and leaving that out makes `oint` 32 bits and wastes a round concluding the object representation is broken. +**`hostkencc/php-mirror.sh` is its php counterpart and `--compile` +answers `compiled 514, errors 0`** -- the whole php build, by a +host-built `6c`, in about a minute. ***UNLIKE clisp's THERE IS NOTHING +TO STAGE***, since php's generated parsers, scanners and +`zend_vm_execute.h` all ship in the tarball -- so the file is the FLAG +SET and the source list, **both DERIVED from `cmd/php/mkfile` with +sed** (`DEFS`, `INCS`, `PCRE2FLAGS`, `KECCAKFLAGS`, `OFILES`), with a +gate if a variable comes back unexpanded. It replaced eight rounds +that each cost a VM rebuild to learn about ONE compile error; its +first run returned all eight at once. **Its own two faults are the +shapes to recognise**: it matched the Keccak flags on the DIRECTORY, +which reads like the mkfile's rule and is not it +(`ext/hash/hash_sha3.c` takes them too, from `ext/hash`), and it +passed `CFLAGS` through, whose leading **`-c` means "compile only" to +`pcc` and "print `const if`" to `6c`** -- dozens of lines that read +exactly like diagnostics. *It cannot ask about the LINK*, which stays +the VM's. ***AND `hostkencc/` BUILDS ALL TEN C COMPILERS WITH gcc***, from the shipped `sys/src/cmd/cc` and `[1-9kqv]c` plus go1.4's `lib9`/`libbio`, already vendored at `sys/src/external/go1.4` — which **compiled with gcc @@ -1165,8 +1193,30 @@ silent: - **itcl** — done. 10 failures, all upstream or Tcl 9's, two settled at source, none ours. -**php — SURVEYED, NOTHING BUILT YET, and the survey is in -`docs/notes/php.md`.** 8.5.12, vendored at `sys/src/external/php`. +**php — ***ALL 514 TRANSLATION UNITS COMPILE***, measured on the HOST, +and the notes are in `docs/notes/php.md`.** 8.5.12, vendored at +`sys/src/external/php`; `sh sys/lib/tests/hostkencc/php-mirror.sh +--compile` answers `compiled 514, errors 0, missing 0`. **The LINK is +untouched and is the VM's**: the host has no libap and `hostkencc` +deliberately does not build the linkers, so the undefined-symbol class +— a `config.h` claiming a name libap only DECLARES — has not been +asked. *That question was checked by hand for the resolver family and +it held*: `res_query.c`'s `weak_alias(res_query, res_search)` is a +no-op on kencc, and `network/res_aliases.c` exists precisely to define +the strong forwarders. +**The first sweep returned EIGHT errors in six files and all six were +fixed without a VM round trip** — three of them kencc's (a pointer +assigned to a `bool`; `bool visible;` declaring nothing, because +`indeclname` had every type specifier but `LBOOL`; and the +`__attribute__`/`__asm__` paren scanner counting delimiters INSIDE +string literals, which ate php's 1680-line inlined gdb script), three +APE headers' (no BSD `timeval` macros, five missing `syslog` +facilities, no ``), one php's (`void *` +arithmetic) and one the `config.h`'s (`HAVE_GETHOSTBYNAME_R` yes with +every arity no). ***And the nameser fix uncovered that `` +had been defining the same sixteen constants itself*** — two +definitions with nothing making them agree, invisible until one TU +included both. **508 objects with `--disable-all`**, derived from a real configure run on the host rather than guessed, and `ext/lexbor` is 186 of them — *`--disable-all` does not mean a small build in 8.5*. **Three things diff --git a/docs/notes/instruments.md b/docs/notes/instruments.md index d380759a2d..37691f0595 100644 --- a/docs/notes/instruments.md +++ b/docs/notes/instruments.md @@ -1148,3 +1148,64 @@ than guessing which sites exist.* **Its controls** are sections 4 and 5: a constant that fits, and an unshifted variable that never reaches the fold. Both pass against a broken compiler. + +## `php-mirror.sh` — php's whole build, compiled on the host + +`sys/lib/tests/hostkencc/php-mirror.sh`. `--flags` prints the compile +line, `--compile` builds all 514 translation units and **gates**. +Depends on `hostkencc.sh` having run. + +**What it came from**: eight consecutive rounds, each costing a VM +rebuild to learn about one compile error, because the error in the next +file is invisible until the first is fixed. Its first run returned all +eight at once, in six files, and all six were fixed and re-measured +without the VM. `docs/notes/php.md` has the list. + +***UNLIKE `clisp-mirror.sh` THERE IS NOTHING TO STAGE.*** clisp needs +its `.d` files converted, three `configure` probes run and an +`intparam.h` patched, and that staging is most of its mirror. php's +sources compile where they lie — the generated parsers, scanners and +`zend_vm_execute.h` all ship in the tarball — so this file is the FLAG +SET and the source list and almost nothing else, which is precisely +what the round trips were being spent on. + +**Both of those are DERIVED from `cmd/php/mkfile` with sed**, not +transcribed: `DEFS`, `INCS`, `PCRE2FLAGS`, `KECCAKFLAGS` and the +514-entry `OFILES`, with a gate of its own if a variable comes back +unexpanded. *A mirror whose flags have drifted from the build reports +failures the build does not have and misses the ones it does*, which +this tree has paid for twice — clisp's `intparam.h` and the missing +`-J`. + +**Its own two faults, both in the first run and both recorded in its +header.** It matched the Keccak flags on the DIRECTORY, which reads +like the mkfile's rule but is not it — there are four explicit Keccak +rules and one of them is `ext/hash/hash_sha3.c`, in `ext/hash` itself +— and it duly reported `Could not find include file "KeccakHash.h"`, +a finding that was the instrument's. And it passed the mkfile's +`CFLAGS` straight through, whose leading **`-c` means "compile only" +to `pcc` and "print `const if`" to `6c`** (`cc/lex.c:40`), which +`pcc` does not forward: dozens of lines per file that read exactly +like diagnostics. *A flag one letter from another flag is checked +rather than recalled, and these two meanings are in two different +programs.* + +**What it does not measure is the LINK** — the host has no libap and +`hostkencc` deliberately does not build the linkers — so the +undefined-symbol class stays the VM's. + +## `boolptr-test` — a pointer converted to `bool` + +`sys/lib/tests/boolptr-test.c`, 0 failures on gcc. The four sites that +CONVERT rather than cast — return, initialiser, assignment, argument — +plus the cast, at three pointer values. + +From `Zend/zend_lazy_objects.c:117`, a `void *` returned from a `bool` +function, which kencc refused; `docs/notes/kencc.md` has the mechanism. + +***SECTIONS 1 AND 2 ARE WHY IT IS A TEST AND NOT A COMPILE.*** The +plausible wrong fix lets the pointer through as a NUMBER, which keeps +the low byte — so section 2 uses `0x0000555500000000`, non-null and +zero in every byte a `char`, `short` or `int` conversion would keep. +*That is the shape of the perl `cBOOL` bug `boolnorm` was written for, +arriving by a different route.* diff --git a/docs/notes/kencc.md b/docs/notes/kencc.md index 4813ca5d00..0dc7c14949 100644 --- a/docs/notes/kencc.md +++ b/docs/notes/kencc.md @@ -3144,3 +3144,148 @@ rule otherwise reads as "do not instrument": the recursion counter is what proved the walk ran 630 times and visited no label, and that pair of facts — a working walk and zero labels — is what finally made the identification untenable. + +## Three bugs php's 514 files found, all three in the front end + +`sys/lib/tests/hostkencc/php-mirror.sh --compile` compiles php's whole +build with the host-built `6c`. Its first run returned eight errors in +six files; three were this compiler's, and each is a case where the +machinery to do the right thing was already present and reached by +nothing. + +**All three are LOUD**, which is why they are recorded as a set rather +than argued over one at a time: none of them can miscompile a program +silently. `agree.c`, the codegen control, is **identical before and +after all three**, and `hostkencc.sh --probe` still answers ten +`regpair-min` and eight `regpair-probe` with `kc` and `vc` failing at +`:242` — the recorded numbers to the digit — so nothing moved that was +not meant to. + +### 1. A pointer could not be assigned to a `bool` + +`Zend/zend_lazy_objects.c:117` is + + static bool zend_lazy_object_has_stale_info(const zend_object *obj) + { + return zend_hash_index_find_ptr(&EG(...), obj->handle); + } + +and answered `incompatible types: "UCHAR" and "IND VOID" for op +"RETURN"`. C99 6.5.16.1 lists "the left operand has `_Bool` type and +the right operand is a pointer" as a simple-assignment constraint of +its own, because 6.3.1.2 converts every SCALAR to bool by comparison +with 0. + +***THE CONVERSION WAS ALREADY WRITTEN AND SAT BEHIND THE CHECK THAT +REFUSED THE OPERAND.*** `com.c` calls `boolnorm` — `!!x`, a comparison +and not a truncation, the function the perl `cBOOL` bug paid for — at +OAS, OASI, ORETURN and in `tcoma`'s argument case. **Every one of the +four runs `stcompat(..., tasign)` first**, and `tasign[TUCHAR]` is +`BNUMBER`, which excludes BIND; kencc has no TBOOL etype, so the +question never reached the `isbool` mark that the next four lines test. +One clause in `sub.c`'s `stcompat` lets a pointer through when the +target is bool, and all four sites then do what they already meant to. + +*The CAST form worked all along* — `tcast[TUCHAR]` carries BIND — which +is why `(bool)p` is everywhere in this tree and nothing had noticed. +`p = b`, an integer assigned to a pointer, is still an error. + +`sys/lib/tests/boolptr-test.c` has the four sites at three pointer +values, and **its sections 1 and 2 are the ones that matter**: the +plausible wrong fix lets the pointer through as a NUMBER, which keeps +the low byte, so a pointer whose low four bytes are zero converts to +`false`. `6c -S` shows `CMPQ AX,$0` at all four, and the test is 0 +failures on gcc. + +### 2. `bool visible;` declared nothing + +`ext/reflection/php_reflection.c:3896` is a function whose whole body +is `bool visible;` and a `zend_parse_parameters` of `&visible`, and it +answered `name not declared: visible`. `bool vis;` compiles, which is +what makes the message read as being about the name. + +`lex.c` swallows the GNU visibility attributes `hidden` and `visible` +— unprefixed, because the Linux kernel spells them that way — only +when the previous token is NOT a position where an identifier must be a +name. That predicate, `indeclname`, lists `LCHAR`, `LINT`, `LSTRUCT`, +`LUNION`, `LENUM`, `LTYPE` and the rest, and **did not list `LBOOL`**. +So after `bool` the declarator was eaten as an attribute. + +The omission is of one member of a set the list already covered: +`LBOOL` is in the grammar's `complex:` rule beside `LSTRUCT`, +`LUNION`, `LENUM` and `LTYPE`, all four of which were listed. +`LCOMPLEX`, `LIMAGINARY` and the internal `LCOMPLEXF`/`LCOMPLEXD` that +`float _Complex` combines to are the rest of that set and were added +from the same reading rather than from a failure. + +**The struct form is loud too**: `struct { bool visible; int x; }` +answers `unnamed structure element must be struct/union`, so no member +was ever silently dropped. + +***AND A SECOND LIMIT OF THE SAME PREDICATE IS RECORDED AND NOT +FIXED.*** A `;` does not reset `lasttok`, so in + + visible int g; + hidden int h; + +the second line is not swallowed — `lasttok` is still `LNAME` from `g` +— and answers `not a function`. Measured against the compiler from +before this change and identical, so it predates it. Nothing in the +tree uses the unprefixed spelling at file scope; widening the predicate +would mean swallowing a name in a position where one is legal, which is +the worse failure. In 0.6's inventory. + +### 3. The attribute scanner counted parentheses inside string literals + +`main/debug_gdb_scripts.c` is a generated file whose entire content is +one `__asm__()` holding 1680 `.ascii` directives — gdb's own python, +inlined into a `.debug_gdb_scripts` section — and it answered + + unexpected EOF in __attribute__ + +`lex.c` makes `__attribute__`, `__attribute`, `__declspec`, +`__extension__`, `__asm__` and `__asm` invisible to the grammar by +consuming from the opening `(` to the matching `)`, tracking depth. It +did that **without honouring quotes or escapes**, and the gdb text has +one more `(` than `)` in it, so the depth never came back to 0 and the +scanner ate the file. The same hazard is an `__attribute__((section( +"a)b")))` or a `'('` anywhere in an attribute argument. A string or +character literal is now skipped whole, backslash escapes included. + +**The message named something the file did not contain**, for the +fourth time in these notes after the `lock.h` shadow, `cmd/mkfile:96` +and the `basename -d` round: all six constructs reported +`__attribute__`. It prints `s->name` now, so the same failure would +have said `__asm__`. + +## The host harness lied about a child's exit status + +Not a compiler bug, but it was indistinguishable from one for a round +and belongs beside them. One php file made the host `6c` **exit 1, +delete its own output file and print nothing at all** — no diagnostic +on either stream, from a compiler that had just written 46 buffers of +object code. + +`strace -f` settled it in one run: the `cpp` child `exit_group(0)`, +the parent's `wait4` returned `WIFEXITED && WEXITSTATUS == 0`, and the +parent then exited 1. The only path that raises `nerrors` without +printing is `lex.c:349`, + + if(nerrors == 0 && mywait(&status) > 0 && status != 0) + nerrors++; + +and `compat-unix.c`'s `mywait` sets `status` from `w->msg[0]`. The +fault is in `go1.4/src/lib9/await.c`: `w = malloc(sizeof *w + 200)` +followed by `memset(w, 0, sizeof *w)`, which stops at the struct and +leaves the message buffer — part of the same allocation — +**uninitialised**. A child that exits 0 writes nothing to it, so +`msg[0]` was whatever `malloc` last left there, and the answer varied +by input. `memset(w, 0, sizeof *w + 200)` now. The file is +`+build !plan9`, so nothing on the VM reads it and the VM's `mywait` +is `cc/compat.c`'s. + +*An instrument that reports a failure the subject does not have is the +one fault that cannot be found by reading the subject* — and this one +also explains why `exits()` losing a buffered `diagbuf` was the first +suspicion and the wrong one: go1.4's `libbio` registers an `atexit` +flush, so a diagnostic would have appeared. diff --git a/docs/notes/libap.md b/docs/notes/libap.md index ccf7381377..ef18f8f0ec 100644 --- a/docs/notes/libap.md +++ b/docs/notes/libap.md @@ -4137,3 +4137,101 @@ other through absolute `/sys/include/ape/...` paths that resolve only inside `apexp-sh` -- so the gcc check symlinks the two ucontext headers into a scratch directory instead and supplies `sigset_t` from glibc's own ``. + +## Four header gaps php's compile sweep found, and one duplicated truth + +`php-mirror.sh --compile` (`docs/notes/php.md`) compiled php's 514 +translation units on the build host; three of the eight errors were +APE headers' and the fourth came out of fixing one of them. + +### `` had none of the BSD `timeval` macros + +`main/network.c:406` is `timercmp(&time_now, &limit_time, <)` and +answered `syntax error, last name: limit_time` — an undefined macro, +so the `<` argument reaches the parser as an operator with nothing on +its left. *The message names the victim*, as it does whenever a missing +macro is called with something that is not an expression. + +glibc, musl and every BSD carry `timerisset`, `timerclear`, +`timercmp`, `timeradd` and `timersub` in ``. All five are +there now and **every one is guarded**, which is existing practice +rather than caution: `bash/include/posixtime.h`, +`readline/posixtime.h` and `libressl/include/compat/sys/time.h` each +define their own under `#ifndef`, so an unguarded definition here +would be a `Macro redefinition` for whichever is reached first — which +is exactly how APE's own `` MIN/MAX answered php one round +earlier. + +### `` was missing five facilities, four of them POSIX + +`ext/standard/basic_functions_arginfo.h:3677` registers the facility +names as PHP constants and answered `name not declared: LOG_MAIL`, +`name not declared: LOG_LPR`. The absent numbers were **exactly 2, 6, +7, 8 and 11** — `LOG_MAIL`, `LOG_LPR`, `LOG_NEWS`, `LOG_UUCP` and +`LOG_FTP`, the facilities with no modern use — and POSIX requires the +first four by name. The numbering is the wire protocol's (RFC 5424 +Table 1), so it is not a choice; the one other definition in the tree, +`perl/os2/os2ish.h`, agrees to the digit. + +*They are enum constants in this header, not `#define`s*, which means a +package that `#define`s its own would not merely redefine it but fail +to compile — the same exposure the list already had for `LOG_KERN`, +and the reason the grep before adding a name is for the BARE WORD. + +### `` had the `ns_` half and no compat half + +php's `ext/standard/dns.c:341` is + + typedef union { + HEADER qb1; + uint8_t qb2[65536]; + } querybuf; + +and answered `syntax error, last name: HEADER`. BIND 8 renamed the +BIND 4 names to the `ns_` forms and kept the old ones in +``, which glibc, musl and the BSDs all include +unconditionally from ``. APExp had the new half only. + +***THE CAPABILITY BEHIND IT IS REAL, AND THAT IS WHY THE ANSWER WAS TO +ADD THE TYPE RATHER THAN SWITCH THE FEATURE OFF.*** php reaches that +block only with `HAVE_RES_SEARCH`, `HAVE_DN_EXPAND` and +`HAVE_DN_SKIPNAME` all yes, and the first question was whether those +three answers were honest — `apfunc-sweep`'s declared-only middle +state is what makes a `config.h` link-fail far from the claim, and +`res_query.c` ends in `weak_alias(res_query, res_search)`, which on +kencc is a **no-op**. They hold anyway: +`network/res_aliases.c` exists precisely because kencc has no weak +symbols and defines strong `res_search` and `dn_expand` forwarders, +and `dn_skipname.c` defines its name directly. *The file that looked +like the trap is the file that prevents it.* + +`HEADER` is a bit-field struct whose layout is the wire format's, so +it is endian-conditional — four of APExp's eleven architectures are +big-endian — and it was checked rather than written and hoped for: +`sizeof` 12, `qr` setting byte 2 to 0x80, `rcode` setting byte 3 to +0x03 and `ancount` landing in bytes 6 and 7, **identical to the host's +own `` under gcc and to APExp's under the host +`6c -J`**. `GETSHORT`/`GETLONG`/`PUTSHORT`/`PUTLONG` are aliases of +the `NS_GET`/`NS_PUT` macros, and the seven `NS_*FIXEDSZ`/`NS_*ADDRSZ` +constants the compat names alias were added to `` +itself, so no number is defined in the compat header alone. + +### And `` had been defining the same constants itself + +Adding the compat header turned up sixteen `Macro redefinition`s: +`` carried its own `MAXDNAME`, `MAXCDNAME`, `MAXLABEL`, four +`NS_*` and the whole `T_*`/`C_*` list beside ``'s — +**two definitions of each constant in two headers with nothing making +them agree**, which is the `PATH_MAX` shape from CLAUDE.md's own rules. +*Nothing could notice while no translation unit included both*, and +php's `dns.c` includes both. `` includes +`` now, as glibc's does, and keeps only what is the +resolver's own: the opcodes, the response codes, `RES_*`, and the +record types and classes `` does not know. One +definition per name, and both include orders compile. + +***AND `dn_skipname` WAS DEFINED BY libap AND DECLARED BY NO HEADER*** +— the `asctime_r` shape again, with the same consequence: every caller +got an implicit `int`. It is declared in `` now. *The +inverse question — what does libap define that no header declares — +still has no instrument.* diff --git a/docs/notes/php.md b/docs/notes/php.md index 4baf80749e..48e07f943f 100644 --- a/docs/notes/php.md +++ b/docs/notes/php.md @@ -686,14 +686,104 @@ upstream never compiles on Linux. It is not: **`grep` says all 37 are referenced in 0 files** of what this port builds. Nothing reads them, so there was nothing to decide. +## ALL 514 TRANSLATION UNITS COMPILE, AND THE MEASUREMENT IS A HOST ONE + +`sh sys/lib/tests/hostkencc/php-mirror.sh --compile` answers +**`compiled 514, errors 0, missing 0`**: the whole build, by a +host-built `6c`, with the mkfile's own flags. + +***THE INSTRUMENT IS THE POINT, NOT THE NUMBER.*** The eight rounds +before this one each cost a VM rebuild to learn about one compile +error, the error in the next file being invisible until the first was +fixed. `php-mirror.sh` reads `DEFS`, `INCS`, `PCRE2FLAGS`, +`KECCAKFLAGS` and the 514-entry `OFILES` **out of `cmd/php/mkfile` +with sed** — derived, not transcribed, so a mirror cannot drift from +the build — stages the one shadowing header the host needs +(`stddef_arch.h` includes ``'s body by absolute path), and +compiles the lot in about a minute. *Eight errors came back in the +first run, in six files, and all six were fixed and re-measured +without the VM.* The fidelity limits are `hostkencc.sh`'s and +unchanged: **sound for "does it compile" and "what does it diagnose", +unsound for anything keyed on `long` being 32 bits**, since the host +build is LP64. + +**What it does NOT measure is the LINK**, which is the next class of +failure and is the VM's: a `config.h` claiming a name libap only +DECLARES fails there, far from the claim. The resolver family below is +exactly that shape caught early — `apfunc-sweep`'s middle state — and +it held, but it had to be checked rather than assumed. + +### The eight, and what each turned out to be + +Three were kencc's, three were APE headers', one was php's and one was +the instrument's own. All of them are in `docs/notes/kencc.md` or +`docs/notes/libap.md` with the mechanism; this is the list. + +1. **`Zend/zend_lazy_objects.c:117`** — `return` of a `void *` from a + `bool` function. `cc/sub.c`'s `stcompat` refused it before + `com.c`'s `boolnorm` could convert it. **kencc, fixed**; + `sys/lib/tests/boolptr-test.c`. +2. **`ext/reflection/php_reflection.c:3896`** — `bool visible;` + declared nothing, because `cc/lex.c`'s `indeclname` list of type + specifiers was missing `LBOOL`, so `visible` was swallowed as a + GNU visibility attribute. **kencc, fixed.** +3. **`main/debug_gdb_scripts.c`** — ` unexpected EOF in + __attribute__`: the attribute/`__asm__` argument scanner counted + parentheses **inside string literals**, and that file is one + `__asm__` holding 1680 `.ascii` lines of gdb's python with one + unmatched `(` in the text. **kencc, fixed.** +4. **`main/network.c:406`** — no `timercmp` in APE's ``. + The five BSD `struct timeval` macros are there now, each guarded. +5. **`ext/standard/basic_functions.c`** — `LOG_MAIL` and `LOG_LPR` + missing from ``, which had skipped facilities 2, 6, 7, 8 + and 11; four of the five are POSIX. +6. **`ext/standard/dns.c:341`** — no `HEADER`: APE had the `ns_` half + of `` and none of the BIND 4 compat half. + `` is new, included from `` + as every other libc does, and its `HEADER` is byte-for-byte the + host's (checked both ways). ***Fixing it uncovered that + `` had its OWN `MAXDNAME`, `T_A`, `C_IN` and the rest*** + — two definitions of the same constants in two headers, invisible + until one translation unit included both. +7. **`main/network.c:1423`** — `HAVE_GETHOSTBYNAME_R` yes with every + arity no, so `gethostname_re` was defined by no arm. libap's is the + six-argument form; `HAVE_FUNC_GETHOSTBYNAME_R_6` is 1. +8. **`ext/random/csprng.c:194`** — arithmetic on a `void *`, a GNU + extension. One cast, in the only one of the file's four sites that + this platform compiles. + +**And the instrument's own faults were two**, both recorded in its +header because they are the shapes to recognise: it matched the Keccak +flags on the DIRECTORY, which read as the mkfile's rule but is not it +(`ext/hash/hash_sha3.c` takes them too), and it passed the mkfile's +`CFLAGS` through, whose leading **`-c` means "compile only" to `pcc` +and "print `const if`" to `6c`** — dozens of lines that read exactly +like diagnostics. + +***THE SILENT ONE WAS IN THE HARNESS AND COST THE MOST.*** One file +made the host `6c` exit 1, delete its own output and print **nothing +at all**. `strace` showed the `cpp` child exiting 0 and being waited +for correctly; the cause is `go1.4/src/lib9/await.c`, whose +`memset(w, 0, sizeof *w)` leaves the message buffer in the same +allocation **uninitialised**, so a child that exits 0 writes nothing +to it and `compat-unix.c`'s `mywait` reads whatever `malloc` left +there as "the child failed". Fixed in the vendored file, which is +`+build !plan9` and so reaches nothing on the VM. *A harness that +reports a failure the subject does not have is the one fault that +cannot be found by reading the subject.* + ## Open, as a list * fibers: **done** — `ucontext` is in libap and `ucontext-test` is 0 failures on the VM (`docs/notes/libap.md`). What remains is pointing php at it: `ZEND_FIBER_UCONTEXT` in the `config.h` -* the first build: `cd sys/src/ape/cmd/php && mk`. Nothing has - compiled yet, so the object list, the include paths and every - answer in the `config.h` are all unmeasured together +* **the LINK.** Every object compiles; nothing has been linked, and + the undefined-symbol class is untouched. `php-mirror.sh` cannot ask + — the host has no libap and `hostkencc` deliberately does not build + the linkers — so this one is the VM's. +* the 474 unanswered questions in the template are still unanswered, + and `cfghdr-sweep` gates on none of them. A compile is not evidence + that an answer is right, only that it is consistent. * `memfd_create` is declared with no implementation, which is a libap trap independent of php * `sys/src/ape/lib/ap/time/asctime_r.c:30` is diff --git a/docs/releases/0.6.md b/docs/releases/0.6.md index 7bd504b0d3..1894cc900b 100644 --- a/docs/releases/0.6.md +++ b/docs/releases/0.6.md @@ -1327,6 +1327,23 @@ distinction is the point of the list. rather than one that fails to compile. The prediction and what would refute it are in `docs/releases/0.7.md`. +- **A bare `hidden` or `visible` specifier is swallowed only when the + previous token is not a name, and a `;` does not clear that.** + `lex.c` drops the unprefixed GNU visibility attributes — the Linux + kernel's spelling — when `lasttok` is not a position where an + identifier must be a name. A `;` does not reset `lasttok`, so in + + visible int g; + hidden int h; + + the second line is not swallowed (`lasttok` is still `LNAME` from + `g`) and answers `not a function`. **Measured against the compiler + from before the `LBOOL` fix and identical, so it predates it**, and + it is loud. *Not fixed on purpose*: widening the predicate means + swallowing an identifier in a position where one is legal, which is + the worse failure, and nothing in the tree uses the unprefixed + spelling at file scope. `docs/notes/kencc.md`. + ### libap - **Every architecture's `fenv.s` is inert, so the fenv API is a stub on diff --git a/sys/include/ape/arpa/nameser.h b/sys/include/ape/arpa/nameser.h index c1e6f7a98c..bb7ad0b006 100644 --- a/sys/include/ape/arpa/nameser.h +++ b/sys/include/ape/arpa/nameser.h @@ -24,6 +24,22 @@ #define NS_MAXMSG 65535 /* max message size */ #define NS_MAXDNSRCH 6 /* max domains in search path */ +/* + * The fixed sizes of the wire format's own records, and the two + * address sizes. Added with , which aliases + * each of them to its BIND 4 name -- so there is one set of numbers + * and the compat header invents none. They are the protocol's: a + * header is 12 octets, a question's fixed part 4, a resource record's + * 10, and 0xc0 is the two high bits that mark a compression pointer. + */ +#define NS_HFIXEDSZ 12 /* fixed data in header */ +#define NS_QFIXEDSZ 4 /* fixed data in query */ +#define NS_RRFIXEDSZ 10 /* fixed data in r record */ +#define NS_INADDRSZ 4 /* IPv4 address */ +#define NS_IN6ADDRSZ 16 /* IPv6 address */ +#define NS_CMPRSFLGS 0xc0 /* flag bits indicating name compression */ +#define NS_DEFAULTPORT 53 /* for both TCP and UDP */ + /* * Macros to read big-endian 16- and 32-bit values from a pointer, * advancing the pointer past the consumed bytes. @@ -146,4 +162,12 @@ void ns_put32(unsigned long, unsigned char *); } #endif +/* + * BIND 4's names, which this header's `ns_' forms replaced. Included + * unconditionally, as glibc, musl and the BSDs all do: `HEADER' and + * `C_IN' are reachable by including and by nothing + * else, which is what portable code expects. + */ +#include + #endif diff --git a/sys/include/ape/arpa/nameser_compat.h b/sys/include/ape/arpa/nameser_compat.h new file mode 100644 index 0000000000..bd5222fd74 --- /dev/null +++ b/sys/include/ape/arpa/nameser_compat.h @@ -0,0 +1,130 @@ +#ifndef _ARPA_NAMESER_COMPAT_H +#define _ARPA_NAMESER_COMPAT_H + +/* + * BIND 4's resolver names, which BIND 8 renamed to the `ns_' forms in + * and kept here for everything already written + * against the old ones. glibc, musl and the BSDs all ship this file + * and all include it from unconditionally, which is + * what this tree does too -- so the names are reachable only by + * including , exactly as on a real system. + * + * WHERE IT CAME FROM. php's `ext/standard/dns.c:341' is + * + * typedef union { + * HEADER qb1; + * uint8_t qb2[65536]; + * } querybuf; + * + * and answered `syntax error, last name: HEADER'. `HEADER' is this + * file's, and APExp had the `ns_' half of and none of + * the compat half. + * + * ***AND THE CAPABILITY BEHIND IT IS REAL, WHICH IS WHY THE ANSWER IS + * TO ADD THE TYPE RATHER THAN TO SWITCH THE FEATURE OFF.*** php + * reaches this block only when `HAVE_RES_SEARCH' and `HAVE_DN_EXPAND' + * and `HAVE_DN_SKIPNAME' are all yes, and in this tree all three are + * **defined for real** -- `lib/ap/network/res_aliases.c' has strong + * `res_search' and `dn_expand' forwarders precisely because kencc has + * no weak symbols, and `dn_skipname.c' defines its name directly. *A + * `config.h' claiming a name that is only DECLARED link-fails far from + * the claim*, and that was the thing to check before believing the + * three answers; they hold. + * + * ***THE `HEADER' BIT ORDER IS THE WIRE PROTOCOL'S AND SO IT IS + * ENDIAN-CONDITIONAL.*** The flags occupy the third and fourth octets + * of a DNS message and a C bit-field is laid out from the low end on a + * little-endian machine, so the two arms below are the same twelve + * bytes read the same way -- which is also why this is a bit-field + * struct rather than two `uint8_t' and some macros: every caller, + * php's included, reads `hp->ancount' and friends. `__BYTE_ORDER' + * comes from <_apetypes.h>, which already reaches + * through ; APExp targets four big-endian architectures, so + * the second arm is not decoration. + */ + +typedef struct { + unsigned id :16; /* query identification number */ +#if defined(__BYTE_ORDER) && defined(__BIG_ENDIAN) && \ + __BYTE_ORDER == __BIG_ENDIAN + /* fields in third byte */ + unsigned qr :1; /* response flag */ + unsigned opcode :4; /* purpose of message */ + unsigned aa :1; /* authoritative answer */ + unsigned tc :1; /* truncated message */ + unsigned rd :1; /* recursion desired */ + /* fields in fourth byte */ + unsigned ra :1; /* recursion available */ + unsigned unused :1; /* unused bits (MBZ as of 4.9.3a3) */ + unsigned ad :1; /* authentic data from named */ + unsigned cd :1; /* checking disabled by resolver */ + unsigned rcode :4; /* response code */ +#else + /* fields in third byte */ + unsigned rd :1; /* recursion desired */ + unsigned tc :1; /* truncated message */ + unsigned aa :1; /* authoritative answer */ + unsigned opcode :4; /* purpose of message */ + unsigned qr :1; /* response flag */ + /* fields in fourth byte */ + unsigned rcode :4; /* response code */ + unsigned cd :1; /* checking disabled by resolver */ + unsigned ad :1; /* authentic data from named */ + unsigned unused :1; /* unused bits (MBZ as of 4.9.3a3) */ + unsigned ra :1; /* recursion available */ +#endif + /* remaining bytes */ + unsigned qdcount :16; /* number of question entries */ + unsigned ancount :16; /* number of answer entries */ + unsigned nscount :16; /* number of authority entries */ + unsigned arcount :16; /* number of resource entries */ +} HEADER; + +/* + * The old spellings of the sizes, limits, opcodes, classes and types. + * Each is the `ns_' name from , so there is one set of + * numbers and this file invents none of them. + * + * *The record-type list is as long as 's and no + * longer*: a `T_' alias for a type that header does not know would be + * a number written here and nowhere else. + */ +#define PACKETSZ NS_PACKETSZ +#define MAXDNAME NS_MAXDNAME +#define MAXCDNAME NS_MAXCDNAME +#define MAXLABEL NS_MAXLABEL +#define HFIXEDSZ NS_HFIXEDSZ +#define QFIXEDSZ NS_QFIXEDSZ +#define RRFIXEDSZ NS_RRFIXEDSZ +#define INT32SZ NS_INT32SZ +#define INT16SZ NS_INT16SZ +#define INADDRSZ NS_INADDRSZ +#define IN6ADDRSZ NS_IN6ADDRSZ +#define INDIR_MASK NS_CMPRSFLGS +#define NAMESERVER_PORT NS_DEFAULTPORT + +#define T_A ns_t_a +#define T_NS ns_t_ns +#define T_CNAME ns_t_cname +#define T_SOA ns_t_soa +#define T_PTR ns_t_ptr +#define T_MX ns_t_mx +#define T_TXT ns_t_txt +#define T_AAAA ns_t_aaaa +#define T_SRV ns_t_srv +#define T_ANY ns_t_any + +#define C_IN ns_c_in +#define C_ANY ns_c_any + +/* + * The BIND 4 accessors, which are 's NS_GET/NS_PUT + * under their old names and argument order. php's `dns.c' reads a + * response with twenty-three of these. + */ +#define GETSHORT NS_GET16 +#define GETLONG NS_GET32 +#define PUTSHORT NS_PUT16 +#define PUTLONG NS_PUT32 + +#endif /* _ARPA_NAMESER_COMPAT_H */ diff --git a/sys/include/ape/resolv.h b/sys/include/ape/resolv.h index aa74e27fba..5ce1d14ed3 100644 --- a/sys/include/ape/resolv.h +++ b/sys/include/ape/resolv.h @@ -5,14 +5,23 @@ #include #include -#define MAXDNAME 1025 -#define MAXCDNAME 255 -#define MAXLABEL 63 -#define NS_MAXDNAME 1025 -#define NS_MAXCDNAME 255 -#define NS_MAXLABEL 63 -#define NS_PACKETSZ 512 -#define NS_MAXMSG 65535 +/* + * ***THE WIRE-FORMAT NAMES COME FROM HERE AND ARE NOT REPEATED, WHICH + * THEY WERE.*** This header carried its own `MAXDNAME', `NS_PACKETSZ', + * `T_A' ... `C_ANY' beside 's -- **two definitions of + * the same constant in two headers, with nothing making them agree**, + * which is the `PATH_MAX' shape from this tree's own rules. Nothing + * noticed while no translation unit included both; php's + * `ext/standard/dns.c' includes both and answered `Macro redefinition + * of MAXDNAME' sixteen times. + * + * glibc's includes for exactly this + * reason, so that is what happens here. What stays below is the + * resolver's own: the opcodes, the response codes, `RES_*', and the + * record types and classes does NOT know -- those are + * still defined in one place, this one. + */ +#include /* DNS opcodes */ #define QUERY 0 @@ -28,25 +37,20 @@ #define NOTIMP 4 #define REFUSED 5 -/* DNS record types */ -#define T_A 1 -#define T_NS 2 +/* + * DNS record types does not list. T_A, T_NS, + * T_CNAME, T_SOA, T_PTR, T_MX, T_TXT, T_AAAA, T_SRV and T_ANY are + * 's, aliased to that header's `ns_t_' numbers. + */ #define T_MD 3 #define T_MF 4 -#define T_CNAME 5 -#define T_SOA 6 #define T_MB 7 #define T_MG 8 #define T_MR 9 #define T_NULL 10 #define T_WKS 11 -#define T_PTR 12 #define T_HINFO 13 #define T_MINFO 14 -#define T_MX 15 -#define T_TXT 16 -#define T_AAAA 28 -#define T_SRV 33 #define T_NAPTR 35 #define T_OPT 41 #define T_TSIG 250 @@ -54,13 +58,10 @@ #define T_AXFR 252 #define T_MAILB 253 #define T_MAILA 254 -#define T_ANY 255 -/* DNS record classes */ -#define C_IN 1 +/* DNS record classes; C_IN and C_ANY likewise come from the compat header. */ #define C_CHAOS 3 #define C_HS 4 -#define C_ANY 255 /* Resolver options */ #define RES_INIT 0x00000001 @@ -123,6 +124,13 @@ int __dn_expand(const unsigned char *, const unsigned char *, const unsigned char *, char *, int); int dn_expand(const unsigned char *, const unsigned char *, const unsigned char *, char *, int); +/* + * `lib/ap/network/dn_skipname.c' has defined this all along and no + * header declared it -- the `asctime_r' shape again, and the same + * consequence: every caller got an implicit `int'. php's + * `ext/standard/dns.c:996' is one. + */ +int dn_skipname(const unsigned char *, const unsigned char *); int res_query(const char *, int, int, unsigned char *, int); int res_search(const char *, int, int, unsigned char *, int); diff --git a/sys/include/ape/sys/time.h b/sys/include/ape/sys/time.h index 2aafd30293..969a9c82b6 100644 --- a/sys/include/ape/sys/time.h +++ b/sys/include/ape/sys/time.h @@ -43,6 +43,61 @@ struct timezone { #endif /* __TIMEVAL__ */ +/* + * The BSD `struct timeval' arithmetic macros. glibc, musl and every + * BSD carry these in ; APExp had none of them, and + * php's `main/network.c:406' answered + * + * syntax error, last name: limit_time + * + * on `timercmp(&time_now, &limit_time, <)' -- an undefined macro, so + * the `<' argument reaches the parser as an operator with nothing on + * its left. *The message names the victim*, as it does whenever a + * missing macro is called with something that is not an expression. + * + * ***EVERY ONE IS GUARDED, AND THAT IS NOT CAUTION BUT THE EXISTING + * PRACTICE IN THIS TREE.*** `bash/include/posixtime.h', + * `readline/posixtime.h' and `libressl/include/compat/sys/time.h' all + * define their own under `#ifndef', so an unguarded definition here + * would be a `Macro redefinition' for whichever of them is reached + * first -- which is how APE's own MIN/MAX answered php. + * + * `timercmp' takes the comparison as a macro argument, so it cannot + * be a function, and the three-line form below is BSD's: comparing + * `tv_usec' only when the seconds are equal. + */ +#ifndef timerisset +#define timerisset(tvp) ((tvp)->tv_sec || (tvp)->tv_usec) +#endif +#ifndef timerclear +#define timerclear(tvp) ((tvp)->tv_sec = (tvp)->tv_usec = 0) +#endif +#ifndef timercmp +#define timercmp(a, b, CMP) \ + (((a)->tv_sec == (b)->tv_sec) ? ((a)->tv_usec CMP (b)->tv_usec) \ + : ((a)->tv_sec CMP (b)->tv_sec)) +#endif +#ifndef timeradd +#define timeradd(a, b, res) do { \ + (res)->tv_sec = (a)->tv_sec + (b)->tv_sec; \ + (res)->tv_usec = (a)->tv_usec + (b)->tv_usec; \ + if((res)->tv_usec >= 1000000) { \ + (res)->tv_sec++; \ + (res)->tv_usec -= 1000000; \ + } \ +} while(0) +#endif +#ifndef timersub +#define timersub(a, b, res) do { \ + (res)->tv_sec = (a)->tv_sec - (b)->tv_sec; \ + (res)->tv_usec = (a)->tv_usec - (b)->tv_usec; \ + if((res)->tv_usec < 0) { \ + (res)->tv_sec--; \ + (res)->tv_usec += 1000000; \ + } \ +} while(0) +#endif + #define ITIMER_REAL 0 #define ITIMER_VIRTUAL 1 #define ITIMER_PROF 3 diff --git a/sys/include/ape/syslog.h b/sys/include/ape/syslog.h index c307a3cff6..f7e9f830de 100644 --- a/sys/include/ape/syslog.h +++ b/sys/include/ape/syslog.h @@ -2,14 +2,40 @@ #define __SYSLOG_H #pragma lib "/$M/lib/ape/libap.a" +/* + * ***THE FACILITIES THAT WERE MISSING WERE EXACTLY THE ONES WITH NO + * MODERN USE, AND FOUR OF THEM ARE POSIX.*** 2, 6, 7, 8 and 11 were + * absent -- LOG_MAIL, LOG_LPR, LOG_NEWS, LOG_UUCP and LOG_FTP -- and + * POSIX requires the first four by name. php's + * `basic_functions_arginfo.h:3677' registers the whole set as PHP + * constants and answered + * + * name not declared: LOG_MAIL + * name not declared: LOG_LPR + * + * The numbering is universal (it is the wire protocol's, RFC 5424 + * Table 1), so these are not a choice; the one external definition in + * the tree, `perl/os2/os2ish.h', agrees to the digit. + * + * *They are enum constants here and not `#define's*, as the rest of + * this header is, which means a package that `#define's its own would + * not merely redefine it but break -- the same exposure the list + * already had for LOG_KERN, and the reason the grep before adding a + * name is for the BARE WORD. + */ enum { /* facility */ LOG_KERN = 0<<3, LOG_USER = 1<<3, + LOG_MAIL = 2<<3, LOG_DAEMON = 3<<3, LOG_AUTH = 4<<3, LOG_SYSLOG = 5<<3, + LOG_LPR = 6<<3, + LOG_NEWS = 7<<3, + LOG_UUCP = 8<<3, LOG_CRON = 9<<3, + LOG_FTP = 11<<3, LOG_AUTHPRIV= 10<<3, LOG_LOCAL0 = 16<<3, LOG_LOCAL1 = 17<<3, diff --git a/sys/lib/tests/boolptr-test.c b/sys/lib/tests/boolptr-test.c new file mode 100644 index 0000000000..79f61fb7e6 --- /dev/null +++ b/sys/lib/tests/boolptr-test.c @@ -0,0 +1,156 @@ +/* + * boolptr-test.c -- a POINTER converted to `bool', at each of the four + * sites that CONVERT rather than cast. + * + * pcc -o boolptr-test boolptr-test.c && ./boolptr-test + * + * Correct on gcc too, and checked there first. + * + * WHERE IT CAME FROM. php's `Zend/zend_lazy_objects.c:117' is + * + * static bool zend_lazy_object_has_stale_info(const zend_object *obj) + * { + * return zend_hash_index_find_ptr(..., obj->handle); + * } + * + * -- a `void *' returned from a `bool' function, which C99 6.5.16.1 + * allows outright -- and kencc answered `incompatible types: "UCHAR" + * and "IND VOID" for op "RETURN"'. **kencc has no TBOOL etype**: bool + * is unsigned char (`cc/lex.c', `typebool' in `cc/cc.h'), so + * `tasign[TUCHAR]' is `BNUMBER', which excludes BIND. + * + * ***AND THE CONVERSION WAS ALREADY THERE, BEHIND THE CHECK THAT + * REFUSED THE OPERAND.*** `cc/com.c' calls `boolnorm' -- `!!x', a + * COMPARISON and not a truncation -- at OAS, OASI, ORETURN and in + * `tcoma', and all four run `stcompat(..., tasign)' first. So the fix + * is one clause in `cc/sub.c's `stcompat' and the four sites then do + * what they already meant to. *The CAST form `(bool)p' worked all + * along*, because `tcast[TUCHAR]' carries BIND. + * + * WHY THE VALUES MATTER AND NOT JUST THE COMPILE. The plausible wrong + * fix is to let the pointer through as a NUMBER, which keeps the low + * byte: every pointer whose low byte is zero -- and on a 2MB-aligned + * mapping that is most of them -- would convert to `false'. **Sections + * 1 and 2 exist to separate a compile from a correct conversion**, and + * use a pointer whose low byte, low two bytes and low four bytes are + * all zero. *That is the shape of the perl `cBOOL' bug `boolnorm' was + * written for, arriving by a different route.* + * + * Exit status is the failure count. + */ + +#include +#include +#include + +static int fails; + +static void +eq(const char *what, int got, int want) +{ + if(got == want) + printf(" PASS %-46s %d\n", what, got); + else { + printf(" FAIL %-46s got %d want %d\n", what, got, want); + fails++; + } +} + +/* The four sites. `q' is opaque to the optimiser on purpose. */ +static void *(*opaque)(void); + +static bool +by_return(void) +{ + return opaque(); +} + +static bool +by_init(void) +{ + bool b = opaque(); + return b; +} + +static bool +by_assign(void) +{ + bool b; + + b = opaque(); + return b; +} + +static bool arg_seen; + +static void +sink(bool b) +{ + arg_seen = b; +} + +static void +by_argument(void) +{ + sink(opaque()); +} + +static bool +by_cast(void) +{ + return (bool)opaque(); +} + +/* ---- the three things `opaque' can answer ------------------- */ + +static char big[1]; + +static void *nullp(void) { return 0; } +static void *realp(void) { return big; } + +/* + * A pointer value whose low FOUR bytes are zero. Not dereferenced -- + * only converted -- so any non-null bit pattern will do, and this one + * is the pattern a byte-wise or int-wise truncation gets wrong. + */ +static void *alignedp(void) { return (void *)(unsigned long long)0x0000555500000000ULL; } + +static int +section(const char *name, void *(*f)(void), int want) +{ + char nm[80]; + + opaque = f; + sprintf(nm, "%s: return", name); eq(nm, by_return(), want); + sprintf(nm, "%s: init", name); eq(nm, by_init(), want); + sprintf(nm, "%s: assign", name); eq(nm, by_assign(), want); + sprintf(nm, "%s: cast", name); eq(nm, by_cast(), want); + arg_seen = !want; + by_argument(); + sprintf(nm, "%s: argument", name); eq(nm, arg_seen, want); + return 0; +} + +int +main(void) +{ + setvbuf(stdout, (char *)0, _IOLBF, 0); + printf("boolptr-test: a pointer converted to bool\n\n"); + + printf("Section 0: a null pointer is false\n"); + section("null", nullp, 0); + + printf("\nSection 1: an ordinary pointer is true\n"); + section("object", realp, 1); + + printf("\nSection 2: a pointer whose low four bytes are ZERO\n"); + /* + * ***THIS IS THE SECTION A TRUNCATING CONVERSION FAILS.*** The + * value is 0x0000555500000000: non-null, and zero in every byte + * a `char', `short' or `int' conversion would keep. + */ + section("aligned", alignedp, 1); + + printf("\nboolptr-test: %d failures\n", fails); + return fails; +} diff --git a/sys/lib/tests/hostkencc/php-mirror.sh b/sys/lib/tests/hostkencc/php-mirror.sh new file mode 100755 index 0000000000..4c4ab3c198 --- /dev/null +++ b/sys/lib/tests/hostkencc/php-mirror.sh @@ -0,0 +1,185 @@ +#!/bin/sh +# +# php-mirror.sh -- compile php's 513 translation units with the +# HOST-BUILT kencc, so a compile error costs a second instead of a VM +# round trip. +# +# sh php-mirror.sh --flags # print the compile flags and exit +# sh php-mirror.sh --compile # all 513; GATES on the error count +# sh php-mirror.sh --compile Zend/zend_lazy_objects.c # just these +# eval "$(sh php-mirror.sh --flags)"; $CC6 $CFL -S .../zend.c +# +# Depends on `hostkencc.sh' having run. `clisp-mirror.sh' is the +# precedent and the fidelity limits in `hostkencc.sh' apply unchanged: +# **sound for "does it compile" and "what does it diagnose", unsound +# for anything keyed on `long' being 32 bits**, because this build is +# LP64 and every kencc target is LLP64. +# +# ***UNLIKE clisp's, THERE IS NOTHING TO STAGE.*** clisp needs its +# `.d' files converted, three `configure' probes run and an +# `intparam.h' patched, and that staging is most of its mirror. php's +# sources compile where they lie: the generated parsers, scanners and +# `zend_vm_execute.h' all ship in the tarball, and the `config.h' is +# hand-kept in `cmd/php/cfg'. **So this file is the FLAG SET and the +# source list and little else** -- which is exactly what cost the VM +# round trips, one error at a time. +# +# ***THE FLAGS AND THE SOURCE LIST ARE BOTH DERIVED FROM `cmd/php/mkfile', +# NOT TRANSCRIBED.*** The list is its `OFILES', read with sed; the +# per-directory flags for pcre2 and Keccak are its explicit rules, read +# the same way. *A mirror whose flags have drifted from the mkfile +# reports failures the real build does not have and misses the ones it +# does*, and this tree has paid for that shape twice -- clisp's +# `intparam.h' and the missing `-J'. +# +# ONE SHADOWING HEADER, for the same reason clisp needs one: +# `amd64/include/ape/stddef_arch.h' includes `'s real body by +# ABSOLUTE path (`"/sys/include/ape/stddef.h"'), which exists on the VM +# and not here. +# +# WHAT IT FOUND ON ITS FIRST RUN is in `docs/notes/php.md'. + +set -e + +HERE=$(cd "$(dirname "$0")" && pwd) +ROOT=$(cd "$HERE/../../../.." && pwd) +PHP=$ROOT/sys/src/external/php +PORT=$ROOT/sys/src/ape/cmd/php +OUT=$HERE/out/php + +# `sys/lib/tests/hostkencc' is FOUR levels below the root; the guard is +# here because that count has been wrong in five instruments now. +test -f "$PHP/Zend/zend.c" || { + echo "php-mirror: no $PHP/Zend/zend.c -- ROOT wrong" >&2; exit 2; } +test -f "$PORT/mkfile" || { + echo "php-mirror: no $PORT/mkfile" >&2; exit 2; } + +# ---- the flags, read out of the mkfile ---------------------------- +# +# `DEFS' and `INCS' are continued over two or three lines each and hold +# `$PHPSRC'; the sed below joins the continuations, drops the variable +# name and substitutes the one variable. Nothing else in either is +# expanded, which is checked by the `$' test afterwards. +mkvar() { + sed -n "/^$1=/,/[^\\\\]\$/p" "$PORT/mkfile" | + tr -d '\\' | tr '\n' ' ' | + sed -e "s/^$1=//" -e "s#\\\$PHPSRC#$PHP#g" -e 's/ */ /g' +} + +DEFS=$(mkvar DEFS) +INCS=$(mkvar INCS) +# `-Icfg' is relative to the port directory in the mkfile. +INCS=$(echo "$INCS" | sed "s#-Icfg#-I$PORT/cfg#") + +case "$DEFS$INCS" in +*'$'*) echo "php-mirror: unexpanded variable in the mkfile's flags:" >&2 + echo " DEFS=$DEFS" >&2 + echo " INCS=$INCS" >&2 + exit 2;; +esac + +CFL="$DEFS $INCS -I$OUT/shadow -I$ROOT/amd64/include/ape -I$ROOT/sys/include/ape" + +# The two directories with flags of their own, from the mkfile's +# explicit rules rather than from memory. +PCRE2FLAGS=$(mkvar PCRE2FLAGS | sed "s#\\\$PHPSRC#$PHP#g") +KECCAKFLAGS=$(mkvar KECCAKFLAGS | sed "s#\\\$PHPSRC#$PHP#g") + +# ***THE MKFILE'S `CFLAGS' OPENS WITH `-c' AND THAT FLAG MUST NOT BE +# PASSED ON.*** `-c' to `pcc' means "compile, do not link"; `-c' to +# `6c' means **print `const if'** (`cc/lex.c:40'), and `pcc' does not +# forward it. The first version of this script handed it to `6c' and +# every run came back with dozens of +# Zend/zend_hash.h:1667[...] const if false +# lines, which read exactly like diagnostics and are a debug trace of +# constant-folded conditions. *A flag one letter from another flag is +# checked rather than recalled*, and the two meanings here are in two +# different programs. So `CFL' above carries `DEFS' and `INCS' only, +# never `CFLAGS'. +if [ "${1:-}" = "--flags" ]; then + echo "CC6='$HERE/out/bin/6c -J -p'" + echo "CFL='$CFL'" + echo "PCRE2FLAGS='$PCRE2FLAGS'" + echo "KECCAKFLAGS='$KECCAKFLAGS'" + echo "PATH=\"$HERE/out/bin:\$PATH\"; export PATH" + echo "export objtype=amd64" + exit 0 +fi + +test -x "$HERE/out/bin/6c" || { + echo "php-mirror: run hostkencc.sh first" >&2; exit 2; } + +mkdir -p "$OUT/shadow" +sed 's#"/sys/include/ape/#"#' \ + "$ROOT/amd64/include/ape/stddef_arch.h" > "$OUT/shadow/stddef_arch.h" + +# ---- the source list, read out of the mkfile's OFILES ------------- +# +# `OFILES' holds object paths with comment lines between the groups; +# each object is the source path with `.$O' for `.c', with one +# exception the mkfile states and this reproduces: `ext/json's parser +# is built from the bison output `json_parser.tab.c'. +sources() { + sed -n '/^OFILES=/,/^$/p' "$PORT/mkfile" | + sed -n 's/^ \([^ ]*\)\.\$O\\*$/\1/p' | + sed 's#^ext/json/json_parser$#ext/json/json_parser.tab#' | + sed 's/$/.c/' +} + +if [ "${1:-}" != "--compile" ]; then + n=$(sources | wc -l) + echo "php-mirror: staged $OUT; $n sources; now --compile" + exit 0 +fi +shift + +PATH="$HERE/out/bin:$PATH"; export PATH +export objtype=amd64 +CC6="$HERE/out/bin/6c -J -p" + +if [ $# -gt 0 ]; then list=$*; else list=$(sources); fi + +ok=0; bad=0; missing=0 +: > "$OUT/errors" +for s in $list; do + # + # Nearly every source is in the vendored tree, and the `%.$O: + # %.c' rule beside the `%.$O: $PHPSRC/%.c' one is there for the + # five files the PORT supplies: `internal_functions_cli.c' is + # configure-generated and kept beside the mkfile so the vendored + # tree stays untouched. + if [ -f "$PHP/$s" ]; then src=$PHP/$s + elif [ -f "$PORT/$s" ]; then src=$PORT/$s + else + missing=$((missing+1)) + echo "MISSING $s" >> "$OUT/errors" + continue + fi + extra= + # + # ***`ext/hash/hash_sha3.c' TAKES THE KECCAK FLAGS TOO, AND + # LEAVING IT OUT MANUFACTURED A FAILURE.*** The first version + # of this script matched on the DIRECTORY, which reads as the + # rule but is not it: the mkfile has four explicit Keccak rules + # and one of them is in `ext/hash' itself. The mirror duly + # answered `Could not find include file "KeccakHash.h"' -- a + # finding that was the instrument's, which is the shape + # `clisp-mirror's `intparam.h' already cost a round for. + # + case "$s" in + ext/pcre/pcre2lib/*) extra=$PCRE2FLAGS;; + ext/hash/sha3/generic64lc/*) extra=$KECCAKFLAGS;; + ext/hash/hash_sha3.c) extra=$KECCAKFLAGS;; + ext/json/*) extra=-I$PHP/ext/json;; + esac + if $CC6 $extra $CFL -o "$OUT/t.6" "$src" > "$OUT/t.log" 2>&1; then + ok=$((ok+1)) + else + bad=$((bad+1)) + echo "==== $s" >> "$OUT/errors" + cat "$OUT/t.log" >> "$OUT/errors" + fi +done +echo "php-mirror: compiled $ok, errors $bad, missing $missing" +echo "php-mirror: diagnostics in $OUT/errors" +test "$bad" -eq 0 && test "$missing" -eq 0 diff --git a/sys/src/ape/cmd/php/cfg/main/php_config.h b/sys/src/ape/cmd/php/cfg/main/php_config.h index fab34768f0..f47968b820 100644 --- a/sys/src/ape/cmd/php/cfg/main/php_config.h +++ b/sys/src/ape/cmd/php/cfg/main/php_config.h @@ -344,7 +344,22 @@ /* #undef HAVE_FOPENCOOKIE */ /* #undef COOKIE_SEEKER_USES_OFF64_T */ /* #undef HAVE_FTOK */ -/* #undef HAVE_FUNC_GETHOSTBYNAME_R_6 */ +/* + * ***THIS HAS TO AGREE WITH `HAVE_GETHOSTBYNAME_R' ABOVE, AND IT DID + * NOT.*** `main/network.c' defines its `gethostname_re' helper in one + * of three arms -- `HAVE_FUNC_GETHOSTBYNAME_R_6', `_5' and `_3' -- and + * calls it whenever `HAVE_GETHOSTBYNAME_R' is set, so saying yes to the + * capability and no to every arity gave + * + * network.c:1423 function not declared: gethostname_re + * network.c:1423 incompatible types: "IND STRUCT hostent" and "INT" + * + * the second line being the implicit `int' return, which is the loud + * half of the undeclared-function rule. libap's is the glibc six- + * argument form -- `netdb.h:131', `network/gethostbyname_r.c' -- so + * this is the arity, and it is 1. + */ +#define HAVE_FUNC_GETHOSTBYNAME_R_6 1 /* #undef HAVE_GETIFADDRS */ /* #undef HAVE_GETLOADAVG */ /* #undef HAVE_GETTID */ diff --git a/sys/src/cmd/cc/lex.c b/sys/src/cmd/cc/lex.c index dd757aa9bb..07ab55d616 100644 --- a/sys/src/cmd/cc/lex.c +++ b/sys/src/cmd/cc/lex.c @@ -1015,12 +1015,75 @@ yylex(void) while(depth > 0) { ac = GETC(); if(ac == EOF) { - yyerror("unexpected EOF in __attribute__"); + /* + * The construct's own name, not + * `__attribute__' for all six: this + * message said `__attribute__' for an + * `__asm__' and so named something + * the file did not contain. + */ + yyerror("unexpected EOF in %s", s->name); break; } + if(ac == '\n') { + lineno++; + continue; + } + /* + * ***A STRING OR CHARACTER LITERAL IS SKIPPED + * WHOLE, AND COUNTING PARENS INSIDE ONE IS + * WHAT THIS LOOP USED TO DO.*** php's + * `main/debug_gdb_scripts.c' is one `__asm__' + * holding 1680 lines of `.ascii' directives -- + * gdb's own python, inlined into a + * `.debug_gdb_scripts' section -- and the text + * has ONE unmatched `(' in it, so `depth' never + * came back to 0 and the whole file was + * consumed: + * + * unexpected EOF in __attribute__ + * + * The same hazard is an `__attribute__(( + * section("a)b") ))' or a `'('' anywhere in an + * attribute argument. *Not php's bug and not + * the asm's*: a C scanner may not count + * delimiters inside a literal. + */ + if(ac == '"' || ac == '\'') { + int q = ac; + + for(;;) { + ac = GETC(); + if(ac == EOF) { + yyerror("unexpected EOF in %s", s->name); + break; + } + if(ac == '\n') { + lineno++; + continue; + } + if(ac == '\\') { + /* the escaped char, + * whatever it is, is + * not a delimiter */ + ac = GETC(); + if(ac == EOF) { + yyerror("unexpected EOF in %s", s->name); + break; + } + if(ac == '\n') + lineno++; + continue; + } + if(ac == q) + break; + } + if(ac == EOF) + break; + continue; + } if(ac == '(') depth++; else if(ac == ')') depth--; - else if(ac == '\n') lineno++; } } else { /* no argument list (e.g. bare __extension__); put char back */ @@ -1099,6 +1162,29 @@ yylex(void) * a typedef/variable name (LNAME), a member-access operator (. or ->), * or a pointer star — all positions where the next identifier is a name, * not an attribute. + * + * ***THE LIST HAS TO HOLD EVERY TYPE SPECIFIER THAT CAN STAND + * IMMEDIATELY BEFORE A DECLARATOR, AND LBOOL WAS MISSING.*** + * `bool visible;' had `lasttok == LBOOL', which is not in the + * list, so `visible' was swallowed as an attribute and the + * declaration declared nothing: + * + * php_reflection.c:3896 name not declared: visible + * + * -- php's `ReflectionMethod::setAccessible', whose whole body + * is `bool visible;' and a `zend_parse_parameters' of it. The + * same shape in a struct answers `unnamed structure element + * must be struct/union', so it is always LOUD; `bool vis;' + * compiles, which is what makes the message read as being about + * the name rather than the type before it. + * + * LBOOL is in the grammar's `complex' rule beside LSTRUCT, + * LUNION, LENUM and LTYPE -- all four of which ARE listed -- so + * the omission was of one member of a set the list already + * covered. LCOMPLEX, LIMAGINARY and the two internal + * LCOMPLEXF/LCOMPLEXD that `float _Complex' combines to are the + * rest of that set; `_Complex visible;' is exotic and was added + * from the same reading rather than from a failure. */ #define indeclname(t) \ ((t)=='.' || (t)==LMG || (t)=='*' || (t)==',' || (t)=='(' || (t)==')' || \ @@ -1106,7 +1192,9 @@ yylex(void) (t)==LCHAR || (t)==LINT || (t)==LFLOAT || (t)==LDOUBLE || \ (t)==LLONG || (t)==LSHORT || (t)==LUNSIGNED || (t)==LSIGNED || \ (t)==LVOID || (t)==LTYPE || (t)==LTYPESTR || (t)==LSTRUCT || \ - (t)==LUNION || (t)==LENUM || (t)==LNAME) + (t)==LUNION || (t)==LENUM || (t)==LNAME || (t)==LBOOL || \ + (t)==LCOMPLEX || (t)==LIMAGINARY || (t)==LCOMPLEXF || \ + (t)==LCOMPLEXD) if(s->lexical == LNAME && ( strcmp(s->name, "__thread") == 0 || strcmp(s->name, "_Thread_local") == 0 || diff --git a/sys/src/cmd/cc/sub.c b/sys/src/cmd/cc/sub.c index 324552b183..bbcd78a636 100644 --- a/sys/src/cmd/cc/sub.c +++ b/sys/src/cmd/cc/sub.c @@ -363,6 +363,36 @@ stcompat(Node *n, Type *t1, Type *t2, long ttab[]) i = 0; if(t1 != T) i = t1->etype; + /* + * ***A POINTER MAY BE ASSIGNED TO A bool, AND THE TABLE CANNOT + * SAY SO.*** C99 6.5.16.1 lists `the left operand has _Bool + * type and the right operand is a pointer' as a simple + * assignment constraint of its own, because 6.3.1.2 converts + * every SCALAR to bool by comparison with 0. kencc has no + * TBOOL etype -- bool is unsigned char (`lex.c', `typebool' in + * `cc.h') -- so `tasign[TUCHAR]' is `BNUMBER', which excludes + * BIND, and the question never reaches the `isbool' mark. + * + * `com.c' already has the CONVERSION for all four sites that + * assign -- OAS, OASI, ORETURN and the argument case in + * `tcoma' -- each calling `boolnorm' when the target is bool. + * **Every one of them runs `stcompat' with `tasign' FIRST**, so + * the code that would have done the right thing was behind a + * check that refused the operand. *The cast form works and + * always did*: `tcast[TUCHAR]' carries BIND, which is how + * `(bool)p' reaches the same `boolnorm'. + * + * php's `Zend/zend_lazy_objects.c:117' is + * `return zend_hash_index_find_ptr(...)' in a `bool' function + * and answered `incompatible types: "UCHAR" and "IND VOID" for + * op "RETURN"'. `sys/lib/tests/boolptr-test.c' has the four + * sites; the host harness reproduces the message exactly. + * + * Only this direction: `p = b' is an integer assigned to a + * pointer and stays an error. + */ + if(ttab == tasign && t1 != T && t1->isbool && b == BIND) + return 0; if(b & ttab[i]) { if(ttab == tasign) if(b == BSTRUCT || b == BUNION) diff --git a/sys/src/external/go1.4/src/lib9/await.c b/sys/src/external/go1.4/src/lib9/await.c index dfb155b243..082010aa91 100644 --- a/sys/src/external/go1.4/src/lib9/await.c +++ b/sys/src/external/go1.4/src/lib9/await.c @@ -122,7 +122,24 @@ _wait(int pid4, int opt) w = malloc(sizeof *w + 200); if(w == nil) return nil; - memset(w, 0, sizeof *w); + /* + * APExp: the message buffer is part of this allocation and was + * NOT cleared -- `memset(w, 0, sizeof *w)' stops at the struct. + * A child that exits 0 has nothing written to `w->msg', so + * `msg[0]' was whatever malloc last left there, and a caller + * that reads it as "did the child fail?" fails at random. + * + * `cc/lex.c:349' is such a caller -- + * if(nerrors == 0 && mywait(&status) > 0 && status != 0) + * nerrors++; + * with `compat-unix.c's mywait setting status from `msg[0]' -- + * so a host-built kencc removed its own output file and exited + * 1 with NO DIAGNOSTIC, for some inputs and not others. It + * took php's `KeccakP-1600-opt64.c' and a full strace to see + * that the child had exited 0 and been waited for correctly. + * This file is `+build !plan9', so nothing on the VM reads it. + */ + memset(w, 0, sizeof *w + 200); w->msg = (char*)&w[1]; for(;;){ diff --git a/sys/src/external/php/ext/random/csprng.c b/sys/src/external/php/ext/random/csprng.c index 73832d7d06..10fbc4a168 100644 --- a/sys/src/external/php/ext/random/csprng.c +++ b/sys/src/external/php/ext/random/csprng.c @@ -191,7 +191,15 @@ ZEND_ATTRIBUTE_NONNULL PHPAPI zend_result php_random_bytes_ex(void *bytes, size_ read_bytes = 0; while (read_bytes < size) { errno = 0; - ssize_t n = read(fd, bytes + read_bytes, size - read_bytes); + /* + * APExp: `bytes' is `void *' and arithmetic on one + * is a GNU extension, which kencc does not have: + * `pointer addition not fully declared: VOID'. The + * three other `bytes + read_bytes' in this file are + * inside getrandom(2) arms this platform does not + * compile, so only this one is changed. + */ + ssize_t n = read(fd, (char *) bytes + read_bytes, size - read_bytes); if (n <= 0) { if (errno != 0) {