diff --git a/CLAUDE.md b/CLAUDE.md index 54db9e4ff1..bd1485c955 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -235,7 +235,18 @@ member of the wide-constant family***: x86-64 has `MOV r64, imm64` and NO 64-bit `CMP` immediate and NO 64-bit address DISPLACEMENT. `6c/cgen.c:346` folds `(x<` `#define`s `true`/`false`, so a TU that INCLUDES that header never reaches kencc's own C23 keywords. @@ -340,6 +351,23 @@ the compile line, `--compile` builds all 31 TUs and GATES. *Its `long_bitsize` is patched 64 -> 32 afterwards, and leaving that out makes `oint` 32 bits and wastes a round concluding the object representation is broken. +**`hostkencc/php-mirror.sh` is its php counterpart and `--compile` +answers `compiled 514, errors 0`** -- the whole php build, by a +host-built `6c`, in about a minute. ***UNLIKE clisp's THERE IS NOTHING +TO STAGE***, since php's generated parsers, scanners and +`zend_vm_execute.h` all ship in the tarball -- so the file is the FLAG +SET and the source list, **both DERIVED from `cmd/php/mkfile` with +sed** (`DEFS`, `INCS`, `PCRE2FLAGS`, `KECCAKFLAGS`, `OFILES`), with a +gate if a variable comes back unexpanded. It replaced eight rounds +that each cost a VM rebuild to learn about ONE compile error; its +first run returned all eight at once. **Its own two faults are the +shapes to recognise**: it matched the Keccak flags on the DIRECTORY, +which reads like the mkfile's rule and is not it +(`ext/hash/hash_sha3.c` takes them too, from `ext/hash`), and it +passed `CFLAGS` through, whose leading **`-c` means "compile only" to +`pcc` and "print `const if`" to `6c`** -- dozens of lines that read +exactly like diagnostics. *It cannot ask about the LINK*, which stays +the VM's. ***AND `hostkencc/` BUILDS ALL TEN C COMPILERS WITH gcc***, from the shipped `sys/src/cmd/cc` and `[1-9kqv]c` plus go1.4's `lib9`/`libbio`, already vendored at `sys/src/external/go1.4` — which **compiled with gcc @@ -1165,8 +1193,30 @@ silent: - **itcl** — done. 10 failures, all upstream or Tcl 9's, two settled at source, none ours. -**php — SURVEYED, NOTHING BUILT YET, and the survey is in -`docs/notes/php.md`.** 8.5.12, vendored at `sys/src/external/php`. +**php — ***ALL 514 TRANSLATION UNITS COMPILE***, measured on the HOST, +and the notes are in `docs/notes/php.md`.** 8.5.12, vendored at +`sys/src/external/php`; `sh sys/lib/tests/hostkencc/php-mirror.sh +--compile` answers `compiled 514, errors 0, missing 0`. **The LINK is +untouched and is the VM's**: the host has no libap and `hostkencc` +deliberately does not build the linkers, so the undefined-symbol class +— a `config.h` claiming a name libap only DECLARES — has not been +asked. *That question was checked by hand for the resolver family and +it held*: `res_query.c`'s `weak_alias(res_query, res_search)` is a +no-op on kencc, and `network/res_aliases.c` exists precisely to define +the strong forwarders. +**The first sweep returned EIGHT errors in six files and all six were +fixed without a VM round trip** — three of them kencc's (a pointer +assigned to a `bool`; `bool visible;` declaring nothing, because +`indeclname` had every type specifier but `LBOOL`; and the +`__attribute__`/`__asm__` paren scanner counting delimiters INSIDE +string literals, which ate php's 1680-line inlined gdb script), three +APE headers' (no BSD `timeval` macros, five missing `syslog` +facilities, no ``), one php's (`void *` +arithmetic) and one the `config.h`'s (`HAVE_GETHOSTBYNAME_R` yes with +every arity no). ***And the nameser fix uncovered that `` +had been defining the same sixteen constants itself*** — two +definitions with nothing making them agree, invisible until one TU +included both. **508 objects with `--disable-all`**, derived from a real configure run on the host rather than guessed, and `ext/lexbor` is 186 of them — *`--disable-all` does not mean a small build in 8.5*. **Three things diff --git a/docs/notes/instruments.md b/docs/notes/instruments.md index d380759a2d..37691f0595 100644 --- a/docs/notes/instruments.md +++ b/docs/notes/instruments.md @@ -1148,3 +1148,64 @@ than guessing which sites exist.* **Its controls** are sections 4 and 5: a constant that fits, and an unshifted variable that never reaches the fold. Both pass against a broken compiler. + +## `php-mirror.sh` — php's whole build, compiled on the host + +`sys/lib/tests/hostkencc/php-mirror.sh`. `--flags` prints the compile +line, `--compile` builds all 514 translation units and **gates**. +Depends on `hostkencc.sh` having run. + +**What it came from**: eight consecutive rounds, each costing a VM +rebuild to learn about one compile error, because the error in the next +file is invisible until the first is fixed. Its first run returned all +eight at once, in six files, and all six were fixed and re-measured +without the VM. `docs/notes/php.md` has the list. + +***UNLIKE `clisp-mirror.sh` THERE IS NOTHING TO STAGE.*** clisp needs +its `.d` files converted, three `configure` probes run and an +`intparam.h` patched, and that staging is most of its mirror. php's +sources compile where they lie — the generated parsers, scanners and +`zend_vm_execute.h` all ship in the tarball — so this file is the FLAG +SET and the source list and almost nothing else, which is precisely +what the round trips were being spent on. + +**Both of those are DERIVED from `cmd/php/mkfile` with sed**, not +transcribed: `DEFS`, `INCS`, `PCRE2FLAGS`, `KECCAKFLAGS` and the +514-entry `OFILES`, with a gate of its own if a variable comes back +unexpanded. *A mirror whose flags have drifted from the build reports +failures the build does not have and misses the ones it does*, which +this tree has paid for twice — clisp's `intparam.h` and the missing +`-J`. + +**Its own two faults, both in the first run and both recorded in its +header.** It matched the Keccak flags on the DIRECTORY, which reads +like the mkfile's rule but is not it — there are four explicit Keccak +rules and one of them is `ext/hash/hash_sha3.c`, in `ext/hash` itself +— and it duly reported `Could not find include file "KeccakHash.h"`, +a finding that was the instrument's. And it passed the mkfile's +`CFLAGS` straight through, whose leading **`-c` means "compile only" +to `pcc` and "print `const if`" to `6c`** (`cc/lex.c:40`), which +`pcc` does not forward: dozens of lines per file that read exactly +like diagnostics. *A flag one letter from another flag is checked +rather than recalled, and these two meanings are in two different +programs.* + +**What it does not measure is the LINK** — the host has no libap and +`hostkencc` deliberately does not build the linkers — so the +undefined-symbol class stays the VM's. + +## `boolptr-test` — a pointer converted to `bool` + +`sys/lib/tests/boolptr-test.c`, 0 failures on gcc. The four sites that +CONVERT rather than cast — return, initialiser, assignment, argument — +plus the cast, at three pointer values. + +From `Zend/zend_lazy_objects.c:117`, a `void *` returned from a `bool` +function, which kencc refused; `docs/notes/kencc.md` has the mechanism. + +***SECTIONS 1 AND 2 ARE WHY IT IS A TEST AND NOT A COMPILE.*** The +plausible wrong fix lets the pointer through as a NUMBER, which keeps +the low byte — so section 2 uses `0x0000555500000000`, non-null and +zero in every byte a `char`, `short` or `int` conversion would keep. +*That is the shape of the perl `cBOOL` bug `boolnorm` was written for, +arriving by a different route.* diff --git a/docs/notes/kencc.md b/docs/notes/kencc.md index 4813ca5d00..0dc7c14949 100644 --- a/docs/notes/kencc.md +++ b/docs/notes/kencc.md @@ -3144,3 +3144,148 @@ rule otherwise reads as "do not instrument": the recursion counter is what proved the walk ran 630 times and visited no label, and that pair of facts — a working walk and zero labels — is what finally made the identification untenable. + +## Three bugs php's 514 files found, all three in the front end + +`sys/lib/tests/hostkencc/php-mirror.sh --compile` compiles php's whole +build with the host-built `6c`. Its first run returned eight errors in +six files; three were this compiler's, and each is a case where the +machinery to do the right thing was already present and reached by +nothing. + +**All three are LOUD**, which is why they are recorded as a set rather +than argued over one at a time: none of them can miscompile a program +silently. `agree.c`, the codegen control, is **identical before and +after all three**, and `hostkencc.sh --probe` still answers ten +`regpair-min` and eight `regpair-probe` with `kc` and `vc` failing at +`:242` — the recorded numbers to the digit — so nothing moved that was +not meant to. + +### 1. A pointer could not be assigned to a `bool` + +`Zend/zend_lazy_objects.c:117` is + + static bool zend_lazy_object_has_stale_info(const zend_object *obj) + { + return zend_hash_index_find_ptr(&EG(...), obj->handle); + } + +and answered `incompatible types: "UCHAR" and "IND VOID" for op +"RETURN"`. C99 6.5.16.1 lists "the left operand has `_Bool` type and +the right operand is a pointer" as a simple-assignment constraint of +its own, because 6.3.1.2 converts every SCALAR to bool by comparison +with 0. + +***THE CONVERSION WAS ALREADY WRITTEN AND SAT BEHIND THE CHECK THAT +REFUSED THE OPERAND.*** `com.c` calls `boolnorm` — `!!x`, a comparison +and not a truncation, the function the perl `cBOOL` bug paid for — at +OAS, OASI, ORETURN and in `tcoma`'s argument case. **Every one of the +four runs `stcompat(..., tasign)` first**, and `tasign[TUCHAR]` is +`BNUMBER`, which excludes BIND; kencc has no TBOOL etype, so the +question never reached the `isbool` mark that the next four lines test. +One clause in `sub.c`'s `stcompat` lets a pointer through when the +target is bool, and all four sites then do what they already meant to. + +*The CAST form worked all along* — `tcast[TUCHAR]` carries BIND — which +is why `(bool)p` is everywhere in this tree and nothing had noticed. +`p = b`, an integer assigned to a pointer, is still an error. + +`sys/lib/tests/boolptr-test.c` has the four sites at three pointer +values, and **its sections 1 and 2 are the ones that matter**: the +plausible wrong fix lets the pointer through as a NUMBER, which keeps +the low byte, so a pointer whose low four bytes are zero converts to +`false`. `6c -S` shows `CMPQ AX,$0` at all four, and the test is 0 +failures on gcc. + +### 2. `bool visible;` declared nothing + +`ext/reflection/php_reflection.c:3896` is a function whose whole body +is `bool visible;` and a `zend_parse_parameters` of `&visible`, and it +answered `name not declared: visible`. `bool vis;` compiles, which is +what makes the message read as being about the name. + +`lex.c` swallows the GNU visibility attributes `hidden` and `visible` +— unprefixed, because the Linux kernel spells them that way — only +when the previous token is NOT a position where an identifier must be a +name. That predicate, `indeclname`, lists `LCHAR`, `LINT`, `LSTRUCT`, +`LUNION`, `LENUM`, `LTYPE` and the rest, and **did not list `LBOOL`**. +So after `bool` the declarator was eaten as an attribute. + +The omission is of one member of a set the list already covered: +`LBOOL` is in the grammar's `complex:` rule beside `LSTRUCT`, +`LUNION`, `LENUM` and `LTYPE`, all four of which were listed. +`LCOMPLEX`, `LIMAGINARY` and the internal `LCOMPLEXF`/`LCOMPLEXD` that +`float _Complex` combines to are the rest of that set and were added +from the same reading rather than from a failure. + +**The struct form is loud too**: `struct { bool visible; int x; }` +answers `unnamed structure element must be struct/union`, so no member +was ever silently dropped. + +***AND A SECOND LIMIT OF THE SAME PREDICATE IS RECORDED AND NOT +FIXED.*** A `;` does not reset `lasttok`, so in + + visible int g; + hidden int h; + +the second line is not swallowed — `lasttok` is still `LNAME` from `g` +— and answers `not a function`. Measured against the compiler from +before this change and identical, so it predates it. Nothing in the +tree uses the unprefixed spelling at file scope; widening the predicate +would mean swallowing a name in a position where one is legal, which is +the worse failure. In 0.6's inventory. + +### 3. The attribute scanner counted parentheses inside string literals + +`main/debug_gdb_scripts.c` is a generated file whose entire content is +one `__asm__()` holding 1680 `.ascii` directives — gdb's own python, +inlined into a `.debug_gdb_scripts` section — and it answered + + unexpected EOF in __attribute__ + +`lex.c` makes `__attribute__`, `__attribute`, `__declspec`, +`__extension__`, `__asm__` and `__asm` invisible to the grammar by +consuming from the opening `(` to the matching `)`, tracking depth. It +did that **without honouring quotes or escapes**, and the gdb text has +one more `(` than `)` in it, so the depth never came back to 0 and the +scanner ate the file. The same hazard is an `__attribute__((section( +"a)b")))` or a `'('` anywhere in an attribute argument. A string or +character literal is now skipped whole, backslash escapes included. + +**The message named something the file did not contain**, for the +fourth time in these notes after the `lock.h` shadow, `cmd/mkfile:96` +and the `basename -d` round: all six constructs reported +`__attribute__`. It prints `s->name` now, so the same failure would +have said `__asm__`. + +## The host harness lied about a child's exit status + +Not a compiler bug, but it was indistinguishable from one for a round +and belongs beside them. One php file made the host `6c` **exit 1, +delete its own output file and print nothing at all** — no diagnostic +on either stream, from a compiler that had just written 46 buffers of +object code. + +`strace -f` settled it in one run: the `cpp` child `exit_group(0)`, +the parent's `wait4` returned `WIFEXITED && WEXITSTATUS == 0`, and the +parent then exited 1. The only path that raises `nerrors` without +printing is `lex.c:349`, + + if(nerrors == 0 && mywait(&status) > 0 && status != 0) + nerrors++; + +and `compat-unix.c`'s `mywait` sets `status` from `w->msg[0]`. The +fault is in `go1.4/src/lib9/await.c`: `w = malloc(sizeof *w + 200)` +followed by `memset(w, 0, sizeof *w)`, which stops at the struct and +leaves the message buffer — part of the same allocation — +**uninitialised**. A child that exits 0 writes nothing to it, so +`msg[0]` was whatever `malloc` last left there, and the answer varied +by input. `memset(w, 0, sizeof *w + 200)` now. The file is +`+build !plan9`, so nothing on the VM reads it and the VM's `mywait` +is `cc/compat.c`'s. + +*An instrument that reports a failure the subject does not have is the +one fault that cannot be found by reading the subject* — and this one +also explains why `exits()` losing a buffered `diagbuf` was the first +suspicion and the wrong one: go1.4's `libbio` registers an `atexit` +flush, so a diagnostic would have appeared. diff --git a/docs/notes/libap.md b/docs/notes/libap.md index ccf7381377..ef18f8f0ec 100644 --- a/docs/notes/libap.md +++ b/docs/notes/libap.md @@ -4137,3 +4137,101 @@ other through absolute `/sys/include/ape/...` paths that resolve only inside `apexp-sh` -- so the gcc check symlinks the two ucontext headers into a scratch directory instead and supplies `sigset_t` from glibc's own ``. + +## Four header gaps php's compile sweep found, and one duplicated truth + +`php-mirror.sh --compile` (`docs/notes/php.md`) compiled php's 514 +translation units on the build host; three of the eight errors were +APE headers' and the fourth came out of fixing one of them. + +### `` had none of the BSD `timeval` macros + +`main/network.c:406` is `timercmp(&time_now, &limit_time, <)` and +answered `syntax error, last name: limit_time` — an undefined macro, +so the `<` argument reaches the parser as an operator with nothing on +its left. *The message names the victim*, as it does whenever a missing +macro is called with something that is not an expression. + +glibc, musl and every BSD carry `timerisset`, `timerclear`, +`timercmp`, `timeradd` and `timersub` in ``. All five are +there now and **every one is guarded**, which is existing practice +rather than caution: `bash/include/posixtime.h`, +`readline/posixtime.h` and `libressl/include/compat/sys/time.h` each +define their own under `#ifndef`, so an unguarded definition here +would be a `Macro redefinition` for whichever is reached first — which +is exactly how APE's own `` MIN/MAX answered php one round +earlier. + +### `` was missing five facilities, four of them POSIX + +`ext/standard/basic_functions_arginfo.h:3677` registers the facility +names as PHP constants and answered `name not declared: LOG_MAIL`, +`name not declared: LOG_LPR`. The absent numbers were **exactly 2, 6, +7, 8 and 11** — `LOG_MAIL`, `LOG_LPR`, `LOG_NEWS`, `LOG_UUCP` and +`LOG_FTP`, the facilities with no modern use — and POSIX requires the +first four by name. The numbering is the wire protocol's (RFC 5424 +Table 1), so it is not a choice; the one other definition in the tree, +`perl/os2/os2ish.h`, agrees to the digit. + +*They are enum constants in this header, not `#define`s*, which means a +package that `#define`s its own would not merely redefine it but fail +to compile — the same exposure the list already had for `LOG_KERN`, +and the reason the grep before adding a name is for the BARE WORD. + +### `` had the `ns_` half and no compat half + +php's `ext/standard/dns.c:341` is + + typedef union { + HEADER qb1; + uint8_t qb2[65536]; + } querybuf; + +and answered `syntax error, last name: HEADER`. BIND 8 renamed the +BIND 4 names to the `ns_` forms and kept the old ones in +``, which glibc, musl and the BSDs all include +unconditionally from ``. APExp had the new half only. + +***THE CAPABILITY BEHIND IT IS REAL, AND THAT IS WHY THE ANSWER WAS TO +ADD THE TYPE RATHER THAN SWITCH THE FEATURE OFF.*** php reaches that +block only with `HAVE_RES_SEARCH`, `HAVE_DN_EXPAND` and +`HAVE_DN_SKIPNAME` all yes, and the first question was whether those +three answers were honest — `apfunc-sweep`'s declared-only middle +state is what makes a `config.h` link-fail far from the claim, and +`res_query.c` ends in `weak_alias(res_query, res_search)`, which on +kencc is a **no-op**. They hold anyway: +`network/res_aliases.c` exists precisely because kencc has no weak +symbols and defines strong `res_search` and `dn_expand` forwarders, +and `dn_skipname.c` defines its name directly. *The file that looked +like the trap is the file that prevents it.* + +`HEADER` is a bit-field struct whose layout is the wire format's, so +it is endian-conditional — four of APExp's eleven architectures are +big-endian — and it was checked rather than written and hoped for: +`sizeof` 12, `qr` setting byte 2 to 0x80, `rcode` setting byte 3 to +0x03 and `ancount` landing in bytes 6 and 7, **identical to the host's +own `` under gcc and to APExp's under the host +`6c -J`**. `GETSHORT`/`GETLONG`/`PUTSHORT`/`PUTLONG` are aliases of +the `NS_GET`/`NS_PUT` macros, and the seven `NS_*FIXEDSZ`/`NS_*ADDRSZ` +constants the compat names alias were added to `` +itself, so no number is defined in the compat header alone. + +### And `` had been defining the same constants itself + +Adding the compat header turned up sixteen `Macro redefinition`s: +`` carried its own `MAXDNAME`, `MAXCDNAME`, `MAXLABEL`, four +`NS_*` and the whole `T_*`/`C_*` list beside ``'s — +**two definitions of each constant in two headers with nothing making +them agree**, which is the `PATH_MAX` shape from CLAUDE.md's own rules. +*Nothing could notice while no translation unit included both*, and +php's `dns.c` includes both. `` includes +`` now, as glibc's does, and keeps only what is the +resolver's own: the opcodes, the response codes, `RES_*`, and the +record types and classes `` does not know. One +definition per name, and both include orders compile. + +***AND `dn_skipname` WAS DEFINED BY libap AND DECLARED BY NO HEADER*** +— the `asctime_r` shape again, with the same consequence: every caller +got an implicit `int`. It is declared in `` now. *The +inverse question — what does libap define that no header declares — +still has no instrument.* diff --git a/docs/notes/php.md b/docs/notes/php.md index 4baf80749e..48e07f943f 100644 --- a/docs/notes/php.md +++ b/docs/notes/php.md @@ -686,14 +686,104 @@ upstream never compiles on Linux. It is not: **`grep` says all 37 are referenced in 0 files** of what this port builds. Nothing reads them, so there was nothing to decide. +## ALL 514 TRANSLATION UNITS COMPILE, AND THE MEASUREMENT IS A HOST ONE + +`sh sys/lib/tests/hostkencc/php-mirror.sh --compile` answers +**`compiled 514, errors 0, missing 0`**: the whole build, by a +host-built `6c`, with the mkfile's own flags. + +***THE INSTRUMENT IS THE POINT, NOT THE NUMBER.*** The eight rounds +before this one each cost a VM rebuild to learn about one compile +error, the error in the next file being invisible until the first was +fixed. `php-mirror.sh` reads `DEFS`, `INCS`, `PCRE2FLAGS`, +`KECCAKFLAGS` and the 514-entry `OFILES` **out of `cmd/php/mkfile` +with sed** — derived, not transcribed, so a mirror cannot drift from +the build — stages the one shadowing header the host needs +(`stddef_arch.h` includes ``'s body by absolute path), and +compiles the lot in about a minute. *Eight errors came back in the +first run, in six files, and all six were fixed and re-measured +without the VM.* The fidelity limits are `hostkencc.sh`'s and +unchanged: **sound for "does it compile" and "what does it diagnose", +unsound for anything keyed on `long` being 32 bits**, since the host +build is LP64. + +**What it does NOT measure is the LINK**, which is the next class of +failure and is the VM's: a `config.h` claiming a name libap only +DECLARES fails there, far from the claim. The resolver family below is +exactly that shape caught early — `apfunc-sweep`'s middle state — and +it held, but it had to be checked rather than assumed. + +### The eight, and what each turned out to be + +Three were kencc's, three were APE headers', one was php's and one was +the instrument's own. All of them are in `docs/notes/kencc.md` or +`docs/notes/libap.md` with the mechanism; this is the list. + +1. **`Zend/zend_lazy_objects.c:117`** — `return` of a `void *` from a + `bool` function. `cc/sub.c`'s `stcompat` refused it before + `com.c`'s `boolnorm` could convert it. **kencc, fixed**; + `sys/lib/tests/boolptr-test.c`. +2. **`ext/reflection/php_reflection.c:3896`** — `bool visible;` + declared nothing, because `cc/lex.c`'s `indeclname` list of type + specifiers was missing `LBOOL`, so `visible` was swallowed as a + GNU visibility attribute. **kencc, fixed.** +3. **`main/debug_gdb_scripts.c`** — ` unexpected EOF in + __attribute__`: the attribute/`__asm__` argument scanner counted + parentheses **inside string literals**, and that file is one + `__asm__` holding 1680 `.ascii` lines of gdb's python with one + unmatched `(` in the text. **kencc, fixed.** +4. **`main/network.c:406`** — no `timercmp` in APE's ``. + The five BSD `struct timeval` macros are there now, each guarded. +5. **`ext/standard/basic_functions.c`** — `LOG_MAIL` and `LOG_LPR` + missing from ``, which had skipped facilities 2, 6, 7, 8 + and 11; four of the five are POSIX. +6. **`ext/standard/dns.c:341`** — no `HEADER`: APE had the `ns_` half + of `` and none of the BIND 4 compat half. + `` is new, included from `` + as every other libc does, and its `HEADER` is byte-for-byte the + host's (checked both ways). ***Fixing it uncovered that + `` had its OWN `MAXDNAME`, `T_A`, `C_IN` and the rest*** + — two definitions of the same constants in two headers, invisible + until one translation unit included both. +7. **`main/network.c:1423`** — `HAVE_GETHOSTBYNAME_R` yes with every + arity no, so `gethostname_re` was defined by no arm. libap's is the + six-argument form; `HAVE_FUNC_GETHOSTBYNAME_R_6` is 1. +8. **`ext/random/csprng.c:194`** — arithmetic on a `void *`, a GNU + extension. One cast, in the only one of the file's four sites that + this platform compiles. + +**And the instrument's own faults were two**, both recorded in its +header because they are the shapes to recognise: it matched the Keccak +flags on the DIRECTORY, which read as the mkfile's rule but is not it +(`ext/hash/hash_sha3.c` takes them too), and it passed the mkfile's +`CFLAGS` through, whose leading **`-c` means "compile only" to `pcc` +and "print `const if`" to `6c`** — dozens of lines that read exactly +like diagnostics. + +***THE SILENT ONE WAS IN THE HARNESS AND COST THE MOST.*** One file +made the host `6c` exit 1, delete its own output and print **nothing +at all**. `strace` showed the `cpp` child exiting 0 and being waited +for correctly; the cause is `go1.4/src/lib9/await.c`, whose +`memset(w, 0, sizeof *w)` leaves the message buffer in the same +allocation **uninitialised**, so a child that exits 0 writes nothing +to it and `compat-unix.c`'s `mywait` reads whatever `malloc` left +there as "the child failed". Fixed in the vendored file, which is +`+build !plan9` and so reaches nothing on the VM. *A harness that +reports a failure the subject does not have is the one fault that +cannot be found by reading the subject.* + ## Open, as a list * fibers: **done** — `ucontext` is in libap and `ucontext-test` is 0 failures on the VM (`docs/notes/libap.md`). What remains is pointing php at it: `ZEND_FIBER_UCONTEXT` in the `config.h` -* the first build: `cd sys/src/ape/cmd/php && mk`. Nothing has - compiled yet, so the object list, the include paths and every - answer in the `config.h` are all unmeasured together +* **the LINK.** Every object compiles; nothing has been linked, and + the undefined-symbol class is untouched. `php-mirror.sh` cannot ask + — the host has no libap and `hostkencc` deliberately does not build + the linkers — so this one is the VM's. +* the 474 unanswered questions in the template are still unanswered, + and `cfghdr-sweep` gates on none of them. A compile is not evidence + that an answer is right, only that it is consistent. * `memfd_create` is declared with no implementation, which is a libap trap independent of php * `sys/src/ape/lib/ap/time/asctime_r.c:30` is diff --git a/docs/releases/0.6.md b/docs/releases/0.6.md index 7bd504b0d3..1894cc900b 100644 --- a/docs/releases/0.6.md +++ b/docs/releases/0.6.md @@ -1327,6 +1327,23 @@ distinction is the point of the list. rather than one that fails to compile. The prediction and what would refute it are in `docs/releases/0.7.md`. +- **A bare `hidden` or `visible` specifier is swallowed only when the + previous token is not a name, and a `;` does not clear that.** + `lex.c` drops the unprefixed GNU visibility attributes — the Linux + kernel's spelling — when `lasttok` is not a position where an + identifier must be a name. A `;` does not reset `lasttok`, so in + + visible int g; + hidden int h; + + the second line is not swallowed (`lasttok` is still `LNAME` from + `g`) and answers `not a function`. **Measured against the compiler + from before the `LBOOL` fix and identical, so it predates it**, and + it is loud. *Not fixed on purpose*: widening the predicate means + swallowing an identifier in a position where one is legal, which is + the worse failure, and nothing in the tree uses the unprefixed + spelling at file scope. `docs/notes/kencc.md`. + ### libap - **Every architecture's `fenv.s` is inert, so the fenv API is a stub on diff --git a/sys/include/ape/arpa/nameser.h b/sys/include/ape/arpa/nameser.h index c1e6f7a98c..bb7ad0b006 100644 --- a/sys/include/ape/arpa/nameser.h +++ b/sys/include/ape/arpa/nameser.h @@ -24,6 +24,22 @@ #define NS_MAXMSG 65535 /* max message size */ #define NS_MAXDNSRCH 6 /* max domains in search path */ +/* + * The fixed sizes of the wire format's own records, and the two + * address sizes. Added with , which aliases + * each of them to its BIND 4 name -- so there is one set of numbers + * and the compat header invents none. They are the protocol's: a + * header is 12 octets, a question's fixed part 4, a resource record's + * 10, and 0xc0 is the two high bits that mark a compression pointer. + */ +#define NS_HFIXEDSZ 12 /* fixed data in header */ +#define NS_QFIXEDSZ 4 /* fixed data in query */ +#define NS_RRFIXEDSZ 10 /* fixed data in r record */ +#define NS_INADDRSZ 4 /* IPv4 address */ +#define NS_IN6ADDRSZ 16 /* IPv6 address */ +#define NS_CMPRSFLGS 0xc0 /* flag bits indicating name compression */ +#define NS_DEFAULTPORT 53 /* for both TCP and UDP */ + /* * Macros to read big-endian 16- and 32-bit values from a pointer, * advancing the pointer past the consumed bytes. @@ -146,4 +162,12 @@ void ns_put32(unsigned long, unsigned char *); } #endif +/* + * BIND 4's names, which this header's `ns_' forms replaced. Included + * unconditionally, as glibc, musl and the BSDs all do: `HEADER' and + * `C_IN' are reachable by including and by nothing + * else, which is what portable code expects. + */ +#include + #endif diff --git a/sys/include/ape/arpa/nameser_compat.h b/sys/include/ape/arpa/nameser_compat.h new file mode 100644 index 0000000000..bd5222fd74 --- /dev/null +++ b/sys/include/ape/arpa/nameser_compat.h @@ -0,0 +1,130 @@ +#ifndef _ARPA_NAMESER_COMPAT_H +#define _ARPA_NAMESER_COMPAT_H + +/* + * BIND 4's resolver names, which BIND 8 renamed to the `ns_' forms in + * and kept here for everything already written + * against the old ones. glibc, musl and the BSDs all ship this file + * and all include it from unconditionally, which is + * what this tree does too -- so the names are reachable only by + * including , exactly as on a real system. + * + * WHERE IT CAME FROM. php's `ext/standard/dns.c:341' is + * + * typedef union { + * HEADER qb1; + * uint8_t qb2[65536]; + * } querybuf; + * + * and answered `syntax error, last name: HEADER'. `HEADER' is this + * file's, and APExp had the `ns_' half of and none of + * the compat half. + * + * ***AND THE CAPABILITY BEHIND IT IS REAL, WHICH IS WHY THE ANSWER IS + * TO ADD THE TYPE RATHER THAN TO SWITCH THE FEATURE OFF.*** php + * reaches this block only when `HAVE_RES_SEARCH' and `HAVE_DN_EXPAND' + * and `HAVE_DN_SKIPNAME' are all yes, and in this tree all three are + * **defined for real** -- `lib/ap/network/res_aliases.c' has strong + * `res_search' and `dn_expand' forwarders precisely because kencc has + * no weak symbols, and `dn_skipname.c' defines its name directly. *A + * `config.h' claiming a name that is only DECLARED link-fails far from + * the claim*, and that was the thing to check before believing the + * three answers; they hold. + * + * ***THE `HEADER' BIT ORDER IS THE WIRE PROTOCOL'S AND SO IT IS + * ENDIAN-CONDITIONAL.*** The flags occupy the third and fourth octets + * of a DNS message and a C bit-field is laid out from the low end on a + * little-endian machine, so the two arms below are the same twelve + * bytes read the same way -- which is also why this is a bit-field + * struct rather than two `uint8_t' and some macros: every caller, + * php's included, reads `hp->ancount' and friends. `__BYTE_ORDER' + * comes from <_apetypes.h>, which already reaches + * through ; APExp targets four big-endian architectures, so + * the second arm is not decoration. + */ + +typedef struct { + unsigned id :16; /* query identification number */ +#if defined(__BYTE_ORDER) && defined(__BIG_ENDIAN) && \ + __BYTE_ORDER == __BIG_ENDIAN + /* fields in third byte */ + unsigned qr :1; /* response flag */ + unsigned opcode :4; /* purpose of message */ + unsigned aa :1; /* authoritative answer */ + unsigned tc :1; /* truncated message */ + unsigned rd :1; /* recursion desired */ + /* fields in fourth byte */ + unsigned ra :1; /* recursion available */ + unsigned unused :1; /* unused bits (MBZ as of 4.9.3a3) */ + unsigned ad :1; /* authentic data from named */ + unsigned cd :1; /* checking disabled by resolver */ + unsigned rcode :4; /* response code */ +#else + /* fields in third byte */ + unsigned rd :1; /* recursion desired */ + unsigned tc :1; /* truncated message */ + unsigned aa :1; /* authoritative answer */ + unsigned opcode :4; /* purpose of message */ + unsigned qr :1; /* response flag */ + /* fields in fourth byte */ + unsigned rcode :4; /* response code */ + unsigned cd :1; /* checking disabled by resolver */ + unsigned ad :1; /* authentic data from named */ + unsigned unused :1; /* unused bits (MBZ as of 4.9.3a3) */ + unsigned ra :1; /* recursion available */ +#endif + /* remaining bytes */ + unsigned qdcount :16; /* number of question entries */ + unsigned ancount :16; /* number of answer entries */ + unsigned nscount :16; /* number of authority entries */ + unsigned arcount :16; /* number of resource entries */ +} HEADER; + +/* + * The old spellings of the sizes, limits, opcodes, classes and types. + * Each is the `ns_' name from , so there is one set of + * numbers and this file invents none of them. + * + * *The record-type list is as long as 's and no + * longer*: a `T_' alias for a type that header does not know would be + * a number written here and nowhere else. + */ +#define PACKETSZ NS_PACKETSZ +#define MAXDNAME NS_MAXDNAME +#define MAXCDNAME NS_MAXCDNAME +#define MAXLABEL NS_MAXLABEL +#define HFIXEDSZ NS_HFIXEDSZ +#define QFIXEDSZ NS_QFIXEDSZ +#define RRFIXEDSZ NS_RRFIXEDSZ +#define INT32SZ NS_INT32SZ +#define INT16SZ NS_INT16SZ +#define INADDRSZ NS_INADDRSZ +#define IN6ADDRSZ NS_IN6ADDRSZ +#define INDIR_MASK NS_CMPRSFLGS +#define NAMESERVER_PORT NS_DEFAULTPORT + +#define T_A ns_t_a +#define T_NS ns_t_ns +#define T_CNAME ns_t_cname +#define T_SOA ns_t_soa +#define T_PTR ns_t_ptr +#define T_MX ns_t_mx +#define T_TXT ns_t_txt +#define T_AAAA ns_t_aaaa +#define T_SRV ns_t_srv +#define T_ANY ns_t_any + +#define C_IN ns_c_in +#define C_ANY ns_c_any + +/* + * The BIND 4 accessors, which are 's NS_GET/NS_PUT + * under their old names and argument order. php's `dns.c' reads a + * response with twenty-three of these. + */ +#define GETSHORT NS_GET16 +#define GETLONG NS_GET32 +#define PUTSHORT NS_PUT16 +#define PUTLONG NS_PUT32 + +#endif /* _ARPA_NAMESER_COMPAT_H */ diff --git a/sys/include/ape/resolv.h b/sys/include/ape/resolv.h index aa74e27fba..5ce1d14ed3 100644 --- a/sys/include/ape/resolv.h +++ b/sys/include/ape/resolv.h @@ -5,14 +5,23 @@ #include #include -#define MAXDNAME 1025 -#define MAXCDNAME 255 -#define MAXLABEL 63 -#define NS_MAXDNAME 1025 -#define NS_MAXCDNAME 255 -#define NS_MAXLABEL 63 -#define NS_PACKETSZ 512 -#define NS_MAXMSG 65535 +/* + * ***THE WIRE-FORMAT NAMES COME FROM HERE AND ARE NOT REPEATED, WHICH + * THEY WERE.*** This header carried its own `MAXDNAME', `NS_PACKETSZ', + * `T_A' ... `C_ANY' beside 's -- **two definitions of + * the same constant in two headers, with nothing making them agree**, + * which is the `PATH_MAX' shape from this tree's own rules. Nothing + * noticed while no translation unit included both; php's + * `ext/standard/dns.c' includes both and answered `Macro redefinition + * of MAXDNAME' sixteen times. + * + * glibc's includes for exactly this + * reason, so that is what happens here. What stays below is the + * resolver's own: the opcodes, the response codes, `RES_*', and the + * record types and classes does NOT know -- those are + * still defined in one place, this one. + */ +#include /* DNS opcodes */ #define QUERY 0 @@ -28,25 +37,20 @@ #define NOTIMP 4 #define REFUSED 5 -/* DNS record types */ -#define T_A 1 -#define T_NS 2 +/* + * DNS record types does not list. T_A, T_NS, + * T_CNAME, T_SOA, T_PTR, T_MX, T_TXT, T_AAAA, T_SRV and T_ANY are + * 's, aliased to that header's `ns_t_' numbers. + */ #define T_MD 3 #define T_MF 4 -#define T_CNAME 5 -#define T_SOA 6 #define T_MB 7 #define T_MG 8 #define T_MR 9 #define T_NULL 10 #define T_WKS 11 -#define T_PTR 12 #define T_HINFO 13 #define T_MINFO 14 -#define T_MX 15 -#define T_TXT 16 -#define T_AAAA 28 -#define T_SRV 33 #define T_NAPTR 35 #define T_OPT 41 #define T_TSIG 250 @@ -54,13 +58,10 @@ #define T_AXFR 252 #define T_MAILB 253 #define T_MAILA 254 -#define T_ANY 255 -/* DNS record classes */ -#define C_IN 1 +/* DNS record classes; C_IN and C_ANY likewise come from the compat header. */ #define C_CHAOS 3 #define C_HS 4 -#define C_ANY 255 /* Resolver options */ #define RES_INIT 0x00000001 @@ -123,6 +124,13 @@ int __dn_expand(const unsigned char *, const unsigned char *, const unsigned char *, char *, int); int dn_expand(const unsigned char *, const unsigned char *, const unsigned char *, char *, int); +/* + * `lib/ap/network/dn_skipname.c' has defined this all along and no + * header declared it -- the `asctime_r' shape again, and the same + * consequence: every caller got an implicit `int'. php's + * `ext/standard/dns.c:996' is one. + */ +int dn_skipname(const unsigned char *, const unsigned char *); int res_query(const char *, int, int, unsigned char *, int); int res_search(const char *, int, int, unsigned char *, int); diff --git a/sys/include/ape/sys/time.h b/sys/include/ape/sys/time.h index 2aafd30293..969a9c82b6 100644 --- a/sys/include/ape/sys/time.h +++ b/sys/include/ape/sys/time.h @@ -43,6 +43,61 @@ struct timezone { #endif /* __TIMEVAL__ */ +/* + * The BSD `struct timeval' arithmetic macros. glibc, musl and every + * BSD carry these in ; APExp had none of them, and + * php's `main/network.c:406' answered + * + * syntax error, last name: limit_time + * + * on `timercmp(&time_now, &limit_time, <)' -- an undefined macro, so + * the `<' argument reaches the parser as an operator with nothing on + * its left. *The message names the victim*, as it does whenever a + * missing macro is called with something that is not an expression. + * + * ***EVERY ONE IS GUARDED, AND THAT IS NOT CAUTION BUT THE EXISTING + * PRACTICE IN THIS TREE.*** `bash/include/posixtime.h', + * `readline/posixtime.h' and `libressl/include/compat/sys/time.h' all + * define their own under `#ifndef', so an unguarded definition here + * would be a `Macro redefinition' for whichever of them is reached + * first -- which is how APE's own MIN/MAX answered php. + * + * `timercmp' takes the comparison as a macro argument, so it cannot + * be a function, and the three-line form below is BSD's: comparing + * `tv_usec' only when the seconds are equal. + */ +#ifndef timerisset +#define timerisset(tvp) ((tvp)->tv_sec || (tvp)->tv_usec) +#endif +#ifndef timerclear +#define timerclear(tvp) ((tvp)->tv_sec = (tvp)->tv_usec = 0) +#endif +#ifndef timercmp +#define timercmp(a, b, CMP) \ + (((a)->tv_sec == (b)->tv_sec) ? ((a)->tv_usec CMP (b)->tv_usec) \ + : ((a)->tv_sec CMP (b)->tv_sec)) +#endif +#ifndef timeradd +#define timeradd(a, b, res) do { \ + (res)->tv_sec = (a)->tv_sec + (b)->tv_sec; \ + (res)->tv_usec = (a)->tv_usec + (b)->tv_usec; \ + if((res)->tv_usec >= 1000000) { \ + (res)->tv_sec++; \ + (res)->tv_usec -= 1000000; \ + } \ +} while(0) +#endif +#ifndef timersub +#define timersub(a, b, res) do { \ + (res)->tv_sec = (a)->tv_sec - (b)->tv_sec; \ + (res)->tv_usec = (a)->tv_usec - (b)->tv_usec; \ + if((res)->tv_usec < 0) { \ + (res)->tv_sec--; \ + (res)->tv_usec += 1000000; \ + } \ +} while(0) +#endif + #define ITIMER_REAL 0 #define ITIMER_VIRTUAL 1 #define ITIMER_PROF 3 diff --git a/sys/include/ape/syslog.h b/sys/include/ape/syslog.h index c307a3cff6..f7e9f830de 100644 --- a/sys/include/ape/syslog.h +++ b/sys/include/ape/syslog.h @@ -2,14 +2,40 @@ #define __SYSLOG_H #pragma lib "/$M/lib/ape/libap.a" +/* + * ***THE FACILITIES THAT WERE MISSING WERE EXACTLY THE ONES WITH NO + * MODERN USE, AND FOUR OF THEM ARE POSIX.*** 2, 6, 7, 8 and 11 were + * absent -- LOG_MAIL, LOG_LPR, LOG_NEWS, LOG_UUCP and LOG_FTP -- and + * POSIX requires the first four by name. php's + * `basic_functions_arginfo.h:3677' registers the whole set as PHP + * constants and answered + * + * name not declared: LOG_MAIL + * name not declared: LOG_LPR + * + * The numbering is universal (it is the wire protocol's, RFC 5424 + * Table 1), so these are not a choice; the one external definition in + * the tree, `perl/os2/os2ish.h', agrees to the digit. + * + * *They are enum constants here and not `#define's*, as the rest of + * this header is, which means a package that `#define's its own would + * not merely redefine it but break -- the same exposure the list + * already had for LOG_KERN, and the reason the grep before adding a + * name is for the BARE WORD. + */ enum { /* facility */ LOG_KERN = 0<<3, LOG_USER = 1<<3, + LOG_MAIL = 2<<3, LOG_DAEMON = 3<<3, LOG_AUTH = 4<<3, LOG_SYSLOG = 5<<3, + LOG_LPR = 6<<3, + LOG_NEWS = 7<<3, + LOG_UUCP = 8<<3, LOG_CRON = 9<<3, + LOG_FTP = 11<<3, LOG_AUTHPRIV= 10<<3, LOG_LOCAL0 = 16<<3, LOG_LOCAL1 = 17<<3, diff --git a/sys/lib/tests/boolptr-test.c b/sys/lib/tests/boolptr-test.c new file mode 100644 index 0000000000..79f61fb7e6 --- /dev/null +++ b/sys/lib/tests/boolptr-test.c @@ -0,0 +1,156 @@ +/* + * boolptr-test.c -- a POINTER converted to `bool', at each of the four + * sites that CONVERT rather than cast. + * + * pcc -o boolptr-test boolptr-test.c && ./boolptr-test + * + * Correct on gcc too, and checked there first. + * + * WHERE IT CAME FROM. php's `Zend/zend_lazy_objects.c:117' is + * + * static bool zend_lazy_object_has_stale_info(const zend_object *obj) + * { + * return zend_hash_index_find_ptr(..., obj->handle); + * } + * + * -- a `void *' returned from a `bool' function, which C99 6.5.16.1 + * allows outright -- and kencc answered `incompatible types: "UCHAR" + * and "IND VOID" for op "RETURN"'. **kencc has no TBOOL etype**: bool + * is unsigned char (`cc/lex.c', `typebool' in `cc/cc.h'), so + * `tasign[TUCHAR]' is `BNUMBER', which excludes BIND. + * + * ***AND THE CONVERSION WAS ALREADY THERE, BEHIND THE CHECK THAT + * REFUSED THE OPERAND.*** `cc/com.c' calls `boolnorm' -- `!!x', a + * COMPARISON and not a truncation -- at OAS, OASI, ORETURN and in + * `tcoma', and all four run `stcompat(..., tasign)' first. So the fix + * is one clause in `cc/sub.c's `stcompat' and the four sites then do + * what they already meant to. *The CAST form `(bool)p' worked all + * along*, because `tcast[TUCHAR]' carries BIND. + * + * WHY THE VALUES MATTER AND NOT JUST THE COMPILE. The plausible wrong + * fix is to let the pointer through as a NUMBER, which keeps the low + * byte: every pointer whose low byte is zero -- and on a 2MB-aligned + * mapping that is most of them -- would convert to `false'. **Sections + * 1 and 2 exist to separate a compile from a correct conversion**, and + * use a pointer whose low byte, low two bytes and low four bytes are + * all zero. *That is the shape of the perl `cBOOL' bug `boolnorm' was + * written for, arriving by a different route.* + * + * Exit status is the failure count. + */ + +#include +#include +#include + +static int fails; + +static void +eq(const char *what, int got, int want) +{ + if(got == want) + printf(" PASS %-46s %d\n", what, got); + else { + printf(" FAIL %-46s got %d want %d\n", what, got, want); + fails++; + } +} + +/* The four sites. `q' is opaque to the optimiser on purpose. */ +static void *(*opaque)(void); + +static bool +by_return(void) +{ + return opaque(); +} + +static bool +by_init(void) +{ + bool b = opaque(); + return b; +} + +static bool +by_assign(void) +{ + bool b; + + b = opaque(); + return b; +} + +static bool arg_seen; + +static void +sink(bool b) +{ + arg_seen = b; +} + +static void +by_argument(void) +{ + sink(opaque()); +} + +static bool +by_cast(void) +{ + return (bool)opaque(); +} + +/* ---- the three things `opaque' can answer ------------------- */ + +static char big[1]; + +static void *nullp(void) { return 0; } +static void *realp(void) { return big; } + +/* + * A pointer value whose low FOUR bytes are zero. Not dereferenced -- + * only converted -- so any non-null bit pattern will do, and this one + * is the pattern a byte-wise or int-wise truncation gets wrong. + */ +static void *alignedp(void) { return (void *)(unsigned long long)0x0000555500000000ULL; } + +static int +section(const char *name, void *(*f)(void), int want) +{ + char nm[80]; + + opaque = f; + sprintf(nm, "%s: return", name); eq(nm, by_return(), want); + sprintf(nm, "%s: init", name); eq(nm, by_init(), want); + sprintf(nm, "%s: assign", name); eq(nm, by_assign(), want); + sprintf(nm, "%s: cast", name); eq(nm, by_cast(), want); + arg_seen = !want; + by_argument(); + sprintf(nm, "%s: argument", name); eq(nm, arg_seen, want); + return 0; +} + +int +main(void) +{ + setvbuf(stdout, (char *)0, _IOLBF, 0); + printf("boolptr-test: a pointer converted to bool\n\n"); + + printf("Section 0: a null pointer is false\n"); + section("null", nullp, 0); + + printf("\nSection 1: an ordinary pointer is true\n"); + section("object", realp, 1); + + printf("\nSection 2: a pointer whose low four bytes are ZERO\n"); + /* + * ***THIS IS THE SECTION A TRUNCATING CONVERSION FAILS.*** The + * value is 0x0000555500000000: non-null, and zero in every byte + * a `char', `short' or `int' conversion would keep. + */ + section("aligned", alignedp, 1); + + printf("\nboolptr-test: %d failures\n", fails); + return fails; +} diff --git a/sys/lib/tests/hostkencc/php-mirror.sh b/sys/lib/tests/hostkencc/php-mirror.sh new file mode 100755 index 0000000000..4c4ab3c198 --- /dev/null +++ b/sys/lib/tests/hostkencc/php-mirror.sh @@ -0,0 +1,185 @@ +#!/bin/sh +# +# php-mirror.sh -- compile php's 513 translation units with the +# HOST-BUILT kencc, so a compile error costs a second instead of a VM +# round trip. +# +# sh php-mirror.sh --flags # print the compile flags and exit +# sh php-mirror.sh --compile # all 513; GATES on the error count +# sh php-mirror.sh --compile Zend/zend_lazy_objects.c # just these +# eval "$(sh php-mirror.sh --flags)"; $CC6 $CFL -S .../zend.c +# +# Depends on `hostkencc.sh' having run. `clisp-mirror.sh' is the +# precedent and the fidelity limits in `hostkencc.sh' apply unchanged: +# **sound for "does it compile" and "what does it diagnose", unsound +# for anything keyed on `long' being 32 bits**, because this build is +# LP64 and every kencc target is LLP64. +# +# ***UNLIKE clisp's, THERE IS NOTHING TO STAGE.*** clisp needs its +# `.d' files converted, three `configure' probes run and an +# `intparam.h' patched, and that staging is most of its mirror. php's +# sources compile where they lie: the generated parsers, scanners and +# `zend_vm_execute.h' all ship in the tarball, and the `config.h' is +# hand-kept in `cmd/php/cfg'. **So this file is the FLAG SET and the +# source list and little else** -- which is exactly what cost the VM +# round trips, one error at a time. +# +# ***THE FLAGS AND THE SOURCE LIST ARE BOTH DERIVED FROM `cmd/php/mkfile', +# NOT TRANSCRIBED.*** The list is its `OFILES', read with sed; the +# per-directory flags for pcre2 and Keccak are its explicit rules, read +# the same way. *A mirror whose flags have drifted from the mkfile +# reports failures the real build does not have and misses the ones it +# does*, and this tree has paid for that shape twice -- clisp's +# `intparam.h' and the missing `-J'. +# +# ONE SHADOWING HEADER, for the same reason clisp needs one: +# `amd64/include/ape/stddef_arch.h' includes `'s real body by +# ABSOLUTE path (`"/sys/include/ape/stddef.h"'), which exists on the VM +# and not here. +# +# WHAT IT FOUND ON ITS FIRST RUN is in `docs/notes/php.md'. + +set -e + +HERE=$(cd "$(dirname "$0")" && pwd) +ROOT=$(cd "$HERE/../../../.." && pwd) +PHP=$ROOT/sys/src/external/php +PORT=$ROOT/sys/src/ape/cmd/php +OUT=$HERE/out/php + +# `sys/lib/tests/hostkencc' is FOUR levels below the root; the guard is +# here because that count has been wrong in five instruments now. +test -f "$PHP/Zend/zend.c" || { + echo "php-mirror: no $PHP/Zend/zend.c -- ROOT wrong" >&2; exit 2; } +test -f "$PORT/mkfile" || { + echo "php-mirror: no $PORT/mkfile" >&2; exit 2; } + +# ---- the flags, read out of the mkfile ---------------------------- +# +# `DEFS' and `INCS' are continued over two or three lines each and hold +# `$PHPSRC'; the sed below joins the continuations, drops the variable +# name and substitutes the one variable. Nothing else in either is +# expanded, which is checked by the `$' test afterwards. +mkvar() { + sed -n "/^$1=/,/[^\\\\]\$/p" "$PORT/mkfile" | + tr -d '\\' | tr '\n' ' ' | + sed -e "s/^$1=//" -e "s#\\\$PHPSRC#$PHP#g" -e 's/ */ /g' +} + +DEFS=$(mkvar DEFS) +INCS=$(mkvar INCS) +# `-Icfg' is relative to the port directory in the mkfile. +INCS=$(echo "$INCS" | sed "s#-Icfg#-I$PORT/cfg#") + +case "$DEFS$INCS" in +*'$'*) echo "php-mirror: unexpanded variable in the mkfile's flags:" >&2 + echo " DEFS=$DEFS" >&2 + echo " INCS=$INCS" >&2 + exit 2;; +esac + +CFL="$DEFS $INCS -I$OUT/shadow -I$ROOT/amd64/include/ape -I$ROOT/sys/include/ape" + +# The two directories with flags of their own, from the mkfile's +# explicit rules rather than from memory. +PCRE2FLAGS=$(mkvar PCRE2FLAGS | sed "s#\\\$PHPSRC#$PHP#g") +KECCAKFLAGS=$(mkvar KECCAKFLAGS | sed "s#\\\$PHPSRC#$PHP#g") + +# ***THE MKFILE'S `CFLAGS' OPENS WITH `-c' AND THAT FLAG MUST NOT BE +# PASSED ON.*** `-c' to `pcc' means "compile, do not link"; `-c' to +# `6c' means **print `const if'** (`cc/lex.c:40'), and `pcc' does not +# forward it. The first version of this script handed it to `6c' and +# every run came back with dozens of +# Zend/zend_hash.h:1667[...] const if false +# lines, which read exactly like diagnostics and are a debug trace of +# constant-folded conditions. *A flag one letter from another flag is +# checked rather than recalled*, and the two meanings here are in two +# different programs. So `CFL' above carries `DEFS' and `INCS' only, +# never `CFLAGS'. +if [ "${1:-}" = "--flags" ]; then + echo "CC6='$HERE/out/bin/6c -J -p'" + echo "CFL='$CFL'" + echo "PCRE2FLAGS='$PCRE2FLAGS'" + echo "KECCAKFLAGS='$KECCAKFLAGS'" + echo "PATH=\"$HERE/out/bin:\$PATH\"; export PATH" + echo "export objtype=amd64" + exit 0 +fi + +test -x "$HERE/out/bin/6c" || { + echo "php-mirror: run hostkencc.sh first" >&2; exit 2; } + +mkdir -p "$OUT/shadow" +sed 's#"/sys/include/ape/#"#' \ + "$ROOT/amd64/include/ape/stddef_arch.h" > "$OUT/shadow/stddef_arch.h" + +# ---- the source list, read out of the mkfile's OFILES ------------- +# +# `OFILES' holds object paths with comment lines between the groups; +# each object is the source path with `.$O' for `.c', with one +# exception the mkfile states and this reproduces: `ext/json's parser +# is built from the bison output `json_parser.tab.c'. +sources() { + sed -n '/^OFILES=/,/^$/p' "$PORT/mkfile" | + sed -n 's/^ \([^ ]*\)\.\$O\\*$/\1/p' | + sed 's#^ext/json/json_parser$#ext/json/json_parser.tab#' | + sed 's/$/.c/' +} + +if [ "${1:-}" != "--compile" ]; then + n=$(sources | wc -l) + echo "php-mirror: staged $OUT; $n sources; now --compile" + exit 0 +fi +shift + +PATH="$HERE/out/bin:$PATH"; export PATH +export objtype=amd64 +CC6="$HERE/out/bin/6c -J -p" + +if [ $# -gt 0 ]; then list=$*; else list=$(sources); fi + +ok=0; bad=0; missing=0 +: > "$OUT/errors" +for s in $list; do + # + # Nearly every source is in the vendored tree, and the `%.$O: + # %.c' rule beside the `%.$O: $PHPSRC/%.c' one is there for the + # five files the PORT supplies: `internal_functions_cli.c' is + # configure-generated and kept beside the mkfile so the vendored + # tree stays untouched. + if [ -f "$PHP/$s" ]; then src=$PHP/$s + elif [ -f "$PORT/$s" ]; then src=$PORT/$s + else + missing=$((missing+1)) + echo "MISSING $s" >> "$OUT/errors" + continue + fi + extra= + # + # ***`ext/hash/hash_sha3.c' TAKES THE KECCAK FLAGS TOO, AND + # LEAVING IT OUT MANUFACTURED A FAILURE.*** The first version + # of this script matched on the DIRECTORY, which reads as the + # rule but is not it: the mkfile has four explicit Keccak rules + # and one of them is in `ext/hash' itself. The mirror duly + # answered `Could not find include file "KeccakHash.h"' -- a + # finding that was the instrument's, which is the shape + # `clisp-mirror's `intparam.h' already cost a round for. + # + case "$s" in + ext/pcre/pcre2lib/*) extra=$PCRE2FLAGS;; + ext/hash/sha3/generic64lc/*) extra=$KECCAKFLAGS;; + ext/hash/hash_sha3.c) extra=$KECCAKFLAGS;; + ext/json/*) extra=-I$PHP/ext/json;; + esac + if $CC6 $extra $CFL -o "$OUT/t.6" "$src" > "$OUT/t.log" 2>&1; then + ok=$((ok+1)) + else + bad=$((bad+1)) + echo "==== $s" >> "$OUT/errors" + cat "$OUT/t.log" >> "$OUT/errors" + fi +done +echo "php-mirror: compiled $ok, errors $bad, missing $missing" +echo "php-mirror: diagnostics in $OUT/errors" +test "$bad" -eq 0 && test "$missing" -eq 0 diff --git a/sys/src/ape/cmd/php/cfg/main/php_config.h b/sys/src/ape/cmd/php/cfg/main/php_config.h index fab34768f0..f47968b820 100644 --- a/sys/src/ape/cmd/php/cfg/main/php_config.h +++ b/sys/src/ape/cmd/php/cfg/main/php_config.h @@ -344,7 +344,22 @@ /* #undef HAVE_FOPENCOOKIE */ /* #undef COOKIE_SEEKER_USES_OFF64_T */ /* #undef HAVE_FTOK */ -/* #undef HAVE_FUNC_GETHOSTBYNAME_R_6 */ +/* + * ***THIS HAS TO AGREE WITH `HAVE_GETHOSTBYNAME_R' ABOVE, AND IT DID + * NOT.*** `main/network.c' defines its `gethostname_re' helper in one + * of three arms -- `HAVE_FUNC_GETHOSTBYNAME_R_6', `_5' and `_3' -- and + * calls it whenever `HAVE_GETHOSTBYNAME_R' is set, so saying yes to the + * capability and no to every arity gave + * + * network.c:1423 function not declared: gethostname_re + * network.c:1423 incompatible types: "IND STRUCT hostent" and "INT" + * + * the second line being the implicit `int' return, which is the loud + * half of the undeclared-function rule. libap's is the glibc six- + * argument form -- `netdb.h:131', `network/gethostbyname_r.c' -- so + * this is the arity, and it is 1. + */ +#define HAVE_FUNC_GETHOSTBYNAME_R_6 1 /* #undef HAVE_GETIFADDRS */ /* #undef HAVE_GETLOADAVG */ /* #undef HAVE_GETTID */ diff --git a/sys/src/cmd/cc/lex.c b/sys/src/cmd/cc/lex.c index dd757aa9bb..07ab55d616 100644 --- a/sys/src/cmd/cc/lex.c +++ b/sys/src/cmd/cc/lex.c @@ -1015,12 +1015,75 @@ yylex(void) while(depth > 0) { ac = GETC(); if(ac == EOF) { - yyerror("unexpected EOF in __attribute__"); + /* + * The construct's own name, not + * `__attribute__' for all six: this + * message said `__attribute__' for an + * `__asm__' and so named something + * the file did not contain. + */ + yyerror("unexpected EOF in %s", s->name); break; } + if(ac == '\n') { + lineno++; + continue; + } + /* + * ***A STRING OR CHARACTER LITERAL IS SKIPPED + * WHOLE, AND COUNTING PARENS INSIDE ONE IS + * WHAT THIS LOOP USED TO DO.*** php's + * `main/debug_gdb_scripts.c' is one `__asm__' + * holding 1680 lines of `.ascii' directives -- + * gdb's own python, inlined into a + * `.debug_gdb_scripts' section -- and the text + * has ONE unmatched `(' in it, so `depth' never + * came back to 0 and the whole file was + * consumed: + * + * unexpected EOF in __attribute__ + * + * The same hazard is an `__attribute__(( + * section("a)b") ))' or a `'('' anywhere in an + * attribute argument. *Not php's bug and not + * the asm's*: a C scanner may not count + * delimiters inside a literal. + */ + if(ac == '"' || ac == '\'') { + int q = ac; + + for(;;) { + ac = GETC(); + if(ac == EOF) { + yyerror("unexpected EOF in %s", s->name); + break; + } + if(ac == '\n') { + lineno++; + continue; + } + if(ac == '\\') { + /* the escaped char, + * whatever it is, is + * not a delimiter */ + ac = GETC(); + if(ac == EOF) { + yyerror("unexpected EOF in %s", s->name); + break; + } + if(ac == '\n') + lineno++; + continue; + } + if(ac == q) + break; + } + if(ac == EOF) + break; + continue; + } if(ac == '(') depth++; else if(ac == ')') depth--; - else if(ac == '\n') lineno++; } } else { /* no argument list (e.g. bare __extension__); put char back */ @@ -1099,6 +1162,29 @@ yylex(void) * a typedef/variable name (LNAME), a member-access operator (. or ->), * or a pointer star — all positions where the next identifier is a name, * not an attribute. + * + * ***THE LIST HAS TO HOLD EVERY TYPE SPECIFIER THAT CAN STAND + * IMMEDIATELY BEFORE A DECLARATOR, AND LBOOL WAS MISSING.*** + * `bool visible;' had `lasttok == LBOOL', which is not in the + * list, so `visible' was swallowed as an attribute and the + * declaration declared nothing: + * + * php_reflection.c:3896 name not declared: visible + * + * -- php's `ReflectionMethod::setAccessible', whose whole body + * is `bool visible;' and a `zend_parse_parameters' of it. The + * same shape in a struct answers `unnamed structure element + * must be struct/union', so it is always LOUD; `bool vis;' + * compiles, which is what makes the message read as being about + * the name rather than the type before it. + * + * LBOOL is in the grammar's `complex' rule beside LSTRUCT, + * LUNION, LENUM and LTYPE -- all four of which ARE listed -- so + * the omission was of one member of a set the list already + * covered. LCOMPLEX, LIMAGINARY and the two internal + * LCOMPLEXF/LCOMPLEXD that `float _Complex' combines to are the + * rest of that set; `_Complex visible;' is exotic and was added + * from the same reading rather than from a failure. */ #define indeclname(t) \ ((t)=='.' || (t)==LMG || (t)=='*' || (t)==',' || (t)=='(' || (t)==')' || \ @@ -1106,7 +1192,9 @@ yylex(void) (t)==LCHAR || (t)==LINT || (t)==LFLOAT || (t)==LDOUBLE || \ (t)==LLONG || (t)==LSHORT || (t)==LUNSIGNED || (t)==LSIGNED || \ (t)==LVOID || (t)==LTYPE || (t)==LTYPESTR || (t)==LSTRUCT || \ - (t)==LUNION || (t)==LENUM || (t)==LNAME) + (t)==LUNION || (t)==LENUM || (t)==LNAME || (t)==LBOOL || \ + (t)==LCOMPLEX || (t)==LIMAGINARY || (t)==LCOMPLEXF || \ + (t)==LCOMPLEXD) if(s->lexical == LNAME && ( strcmp(s->name, "__thread") == 0 || strcmp(s->name, "_Thread_local") == 0 || diff --git a/sys/src/cmd/cc/sub.c b/sys/src/cmd/cc/sub.c index 324552b183..bbcd78a636 100644 --- a/sys/src/cmd/cc/sub.c +++ b/sys/src/cmd/cc/sub.c @@ -363,6 +363,36 @@ stcompat(Node *n, Type *t1, Type *t2, long ttab[]) i = 0; if(t1 != T) i = t1->etype; + /* + * ***A POINTER MAY BE ASSIGNED TO A bool, AND THE TABLE CANNOT + * SAY SO.*** C99 6.5.16.1 lists `the left operand has _Bool + * type and the right operand is a pointer' as a simple + * assignment constraint of its own, because 6.3.1.2 converts + * every SCALAR to bool by comparison with 0. kencc has no + * TBOOL etype -- bool is unsigned char (`lex.c', `typebool' in + * `cc.h') -- so `tasign[TUCHAR]' is `BNUMBER', which excludes + * BIND, and the question never reaches the `isbool' mark. + * + * `com.c' already has the CONVERSION for all four sites that + * assign -- OAS, OASI, ORETURN and the argument case in + * `tcoma' -- each calling `boolnorm' when the target is bool. + * **Every one of them runs `stcompat' with `tasign' FIRST**, so + * the code that would have done the right thing was behind a + * check that refused the operand. *The cast form works and + * always did*: `tcast[TUCHAR]' carries BIND, which is how + * `(bool)p' reaches the same `boolnorm'. + * + * php's `Zend/zend_lazy_objects.c:117' is + * `return zend_hash_index_find_ptr(...)' in a `bool' function + * and answered `incompatible types: "UCHAR" and "IND VOID" for + * op "RETURN"'. `sys/lib/tests/boolptr-test.c' has the four + * sites; the host harness reproduces the message exactly. + * + * Only this direction: `p = b' is an integer assigned to a + * pointer and stays an error. + */ + if(ttab == tasign && t1 != T && t1->isbool && b == BIND) + return 0; if(b & ttab[i]) { if(ttab == tasign) if(b == BSTRUCT || b == BUNION) diff --git a/sys/src/external/go1.4/src/lib9/await.c b/sys/src/external/go1.4/src/lib9/await.c index dfb155b243..082010aa91 100644 --- a/sys/src/external/go1.4/src/lib9/await.c +++ b/sys/src/external/go1.4/src/lib9/await.c @@ -122,7 +122,24 @@ _wait(int pid4, int opt) w = malloc(sizeof *w + 200); if(w == nil) return nil; - memset(w, 0, sizeof *w); + /* + * APExp: the message buffer is part of this allocation and was + * NOT cleared -- `memset(w, 0, sizeof *w)' stops at the struct. + * A child that exits 0 has nothing written to `w->msg', so + * `msg[0]' was whatever malloc last left there, and a caller + * that reads it as "did the child fail?" fails at random. + * + * `cc/lex.c:349' is such a caller -- + * if(nerrors == 0 && mywait(&status) > 0 && status != 0) + * nerrors++; + * with `compat-unix.c's mywait setting status from `msg[0]' -- + * so a host-built kencc removed its own output file and exited + * 1 with NO DIAGNOSTIC, for some inputs and not others. It + * took php's `KeccakP-1600-opt64.c' and a full strace to see + * that the child had exited 0 and been waited for correctly. + * This file is `+build !plan9', so nothing on the VM reads it. + */ + memset(w, 0, sizeof *w + 200); w->msg = (char*)&w[1]; for(;;){ diff --git a/sys/src/external/php/ext/random/csprng.c b/sys/src/external/php/ext/random/csprng.c index 73832d7d06..10fbc4a168 100644 --- a/sys/src/external/php/ext/random/csprng.c +++ b/sys/src/external/php/ext/random/csprng.c @@ -191,7 +191,15 @@ ZEND_ATTRIBUTE_NONNULL PHPAPI zend_result php_random_bytes_ex(void *bytes, size_ read_bytes = 0; while (read_bytes < size) { errno = 0; - ssize_t n = read(fd, bytes + read_bytes, size - read_bytes); + /* + * APExp: `bytes' is `void *' and arithmetic on one + * is a GNU extension, which kencc does not have: + * `pointer addition not fully declared: VOID'. The + * three other `bytes + read_bytes' in this file are + * inside getrandom(2) arms this platform does not + * compile, so only this one is changed. + */ + ssize_t n = read(fd, (char *) bytes + read_bytes, size - read_bytes); if (n <= 0) { if (errno != 0) {