Claude/upgrade ape c library mm z gd

CLAUDE.md

@@ -1149,12 +1149,15 @@ two are the opposite of what it says — `HAVE_TM_ZONE` and
 `HAVE_ST_BLOCKS` are both YES, `struct tm` and `struct stat` having
 those members. *A sweep over function names, asked about a struct
 field, answers `absent` confidently.*
-**What can still stop it, in order**: (1) fibers — **CLOSED on the
-host, unmeasured on the VM**: `zend_fibers.c` is unconditional and
-wants either boost.context's GAS assembly or `<ucontext.h>`, and APExp
-now has the second (`lib/ap/arch/amd64/mcontext.s` + `ucontext.c`,
-`docs/notes/libap.md`). amd64 only; the other ten follow the same
-shape when wanted.
+**What can still stop it, in order**: (1) fibers — **CLOSED and
+MEASURED**: `zend_fibers.c` is unconditional and wants either
+boost.context's GAS assembly or `<ucontext.h>`, and APExp now has the
+second (`lib/ap/arch/amd64/ucontext.s` + `makecontext.c`), with
+`ucontext-test` at **0 failures on the VM**. amd64 only; the other ten
+follow the same shape when wanted. *php has not been pointed at it
+yet* — that is `ZEND_FIBER_UCONTEXT` in the `config.h`.
+`docs/notes/libap.md` has the two bugs it cost, both of them
+`long`-width and both invisible to the host check.
 (2) **`ext/opcache` is mandatory in 8.5** — there is no
 `--disable-opcache`, only `--disable-opcache-jit`, which is what drops
 the 13-file `jit/ir` code generator; opcache only has to COMPILE, since
@@ -1488,6 +1491,21 @@ in the topic file.
   relink -- rounds later, with nothing nearby to blame. *The rule that
   a library fix can sit unused for rounds has a second edge: so can a
   library BREAKAGE.*
+- **RENAMING a source file in a library directory leaves its OBJECT in
+  the archive, and `clean` does not remove it.** `mksyslib`'s `clean`
+  is `rm -f *.[$OS] ...`, which touches the directory and never
+  `$LIB`; only `nuke` removes the archive, and `ar vu` replaces
+  members by NAME, so an object whose source no longer exists is
+  simply never revisited. *Renaming `mcontext.s` to `ucontext.s` left
+  `mcontext.6` defining `_ctxreturn` beside the new `ucontext.6`
+  defining it too*, and the next build said `duplicate text symbol:
+  _ctxreturn`. **The loud case is the lucky one**: when the stale
+  object does NOT collide, its code stays linked into every program
+  in the tree for ever with no diagnostic from anyone. One
+  `ar d <lib> <stale>.$O` fixes it; `mk distclean` fixes it at the
+  cost of a full rebuild. *There is no instrument for "a member of
+  `libap.a` whose source is gone" and it would be cheap* -- `ar t`
+  against the directory -- but the archive exists only on the VM.
 - **Check every object built against a library for the flags that library
   was built with**, not just the ones the linker complained about.
   `nm -g --defined-only x.o | wc -l` is a one-second check.

docs/notes/libap.md

@@ -4098,6 +4098,38 @@ bug the reference implementation is blind to, because the difference IS
 the reference's width. The tree's first invariant, arriving inside a
 file written to check something else.
 
+***MEASURED ON THE VM: `ucontext-test` 0 failures, all six sections.***
+Section 0 prints its PASS line twice, which is the resumed path
+re-entering at `getcontext`'s return and the thing the C wrapper could
+not do. So the switch, the new stack, `uc_link`, all eight argument
+slots, five round trips and a context's own locals are all behaving on
+the machine as they do against glibc.
+
+### Two things the first VM build of it said
+
+**`conversion of pointer to shorter integer`, five times.** The casts
+read `(unsigned long long)(unsigned long)ptr`, and **`unsigned long`
+is four bytes here** -- the stack address, `_ctxreturn` and `func`
+would all have been truncated. `uintptr_t` now.
+
+*The host cross-check could not have caught it and never will*:
+`unsigned long` is eight bytes under glibc, so the conversion is
+lossless there. **That is the second `long`-width bug in this one port
+that gcc was structurally blind to**, after `ucontext-test`'s section
+5. The rule that follows is narrow and worth keeping narrow: a host
+cross-check is blind to `long` by construction, so anything that moves
+a POINTER through an INTEGER is read for that specifically rather than
+left to the host.
+
+**`duplicate text symbol: _ctxreturn`.** Not the code: renaming
+`mcontext.s` to `ucontext.s` left `mcontext.6` in `libap.a`, because
+`mksyslib`'s `clean` removes `*.[$OS]` from the directory and never
+touches the archive, and `ar vu` replaces members by name. One
+`ar d` on the stale member clears it. *The collision was the lucky
+outcome* -- a stale object that does not collide stays linked into
+every program with no diagnostic at all. The rule is in CLAUDE.md's
+build-system list.
+
 **What neither instrument covers**: nothing exercises a context switch
 with signals actually blocked. And the header could not be
 syntax-checked against the APE headers on the host -- they reach each

docs/notes/php.md

@@ -217,6 +217,73 @@ memory — so the limit was Plan 9 segment slots, one spent by
 10MB of heap**, long before any trimming question arose.
 `SEG_THRESHOLD` is 64MB now; `docs/notes/libap.md` has it.
 
+## `__x86_64__` has to be defined, and it is the honest answer
+
+**kencc defines no architecture macro at all**, and php needs one. The
+consequence is not a micro-optimisation:
+
+* `Zend/zend_long.h:26` selects `zend_long` from
+  `__x86_64__ || __LP64__ || _LP64 || _WIN64` -- *not* from
+  `SIZEOF_LONG` -- so without it php's integer type is **32 bits**.
+* `ext/date/lib/timelib.h:112` has the identical test for
+  `timelib_sll`.
+* ***And five arms in `Zend/zend_float.h` read
+  `!defined(__x86_64__)`***, selecting x87 control-word code
+  (`_controlfp`, `_FPU_SETCW`, `fpsetprec`, inline x86 asm) that is
+  wrong for an SSE machine. **Not defining it is itself a positive,
+  wrong choice**, which is the argument that settles this: there is no
+  neutral option.
+
+`__LP64__` would be the available alternative and it is a **lie** --
+kencc is LLP64, `long` is four bytes -- so anything keying on it to
+mean "long is 64 bits" would be silently wrong. `__x86_64__` is simply
+true of the machine.
+
+**The risk is what else it switches on, and that was measured rather
+than assumed.** Of the 30 files mentioning it, every site that emits
+GCC inline assembly is separately gated:
+
+| site | also needs | verdict |
+|---|---|---|
+| `zend_string.h:444`, `.c:394/452`, `zend_multiply.h:45/177`, `zend_execute.c:56`, `zend_cpuinfo.c:31`, `zend_vm_execute.h:364`, `TSRM.c:779` | `__GNUC__` | off |
+| `zend_operators.h:566/645/731/835` | `ZEND_USE_ASM_ARITHMETIC`, which is `HAVE_ASM_GOTO` (`:547`) -- answered **0** | off |
+| `zend_portability.h:561` | `HAVE_TMMINTRIN_H` / `NMMINTRIN` / `WMMINTRIN` -- all absent | off |
+| `php_hash_sha.h:58` | `HAVE_IMMINTRIN_H` -- absent | off |
+| `zend_vm_opcodes.h:52` | `ZEND_VM_KIND == HYBRID` -- it is CALL | off |
+| `zend_cpuinfo.c:76` | nothing, but `get_xcr0_eax` and `__zend_cpuid` both have a `return 0` / zeroing fallback for a non-GCC compiler | **on, and harmless** |
+| `zend_string.h:537` | nothing; a plain C multiplication hash loop | **on, and wanted** |
+| `crypt_blowfish.c:56`, `md5.c:160`, murmur, xxhash | nothing; "unaligned access is cheap" | **on, and true** |
+
+So it goes on the compile line (`-D__x86_64__`) rather than in the
+`config.h`, because `timelib.h` is reached by files that have not
+included `php_config.h` yet and a macro that is defined for only some
+translation units is worse than one that is defined for none.
+
+## The headers: 39 of 91 are present
+
+Mechanical, and checkable: every `HAVE_*_H` name in the generated
+config resolved against `sys/include/ape` and `amd64/include/ape`.
+**39 present**, including everything structural --
+`dirent`, `fcntl`, `poll`, `pwd`, `grp`, `langinfo`, `netinet/in`,
+`resolv`, `syslog`, `sys/{file,ioctl,mman,param,resource,select,socket,
+statvfs,stat,sysmacros,time,uio,utsname,wait}`, `utime`, `wchar`.
+
+The 52 absent are ODBC (14 of them), tidy, the Linux `linux/*` family,
+the SSE intrinsic headers, Windows, and `crypt.h` -- php bundles its
+own crypt, which is why `crypt_blowfish`, `crypt_freesec`,
+`crypt_sha256` and `crypt_sha512` are already in the object list.
+Three absences line up with decisions already taken: no
+`sys/ipc.h` (opcache's SysV shm), no `*mmintrin.h` (every SSE path),
+no `sys/mount.h`/`statfs`/`vfs` (`statvfs` is what we have).
+
+***BUT `HAVE_DLFCN_H` RESOLVES TO YES AND THE ANSWER IS NO.*** APE
+ships a `dlfcn.h`, and `apfunc-sweep.py` reports `dlopen` and
+`dlerror` as **declared-only** -- the middle state, a header promise
+with nothing behind it. *A file-existence check answers this one
+confidently and wrongly*, which is the clearest single argument in
+this port for hand-answering rather than deriving: the mechanical
+method is right 90 times and this is one of the others.
+
 ## Extensions: the ones with a ported library
 
 The user's own framing, and the right one: enable what this tree already
@@ -237,13 +304,106 @@ was tested against and it removes a link dependency from the first build.
 has worked in this tree is: get the thing to compile and print its own
 version, then add one library at a time.
 
+## The port directory
+
+`sys/src/ape/cmd/php/` holds six files and **nothing in
+`sys/src/external/php` is touched** -- the vendored tree stays vendor
+source, and the mkfile puts `-I.` first so our copies win:
+
+| file | what it is |
+|---|---|
+| `mkfile` | 513 objects, derived from upstream's configure |
+| `php_config.h` | the hand-answered one |
+| `zend_config.h` | the one line `zend_portability.h:43` wants |
+| `timelib_config.h` | routes timelib through the Zend allocator |
+| `build-defs.h` | the install paths |
+| `internal_functions_cli.c` | the eleven built-in extension entries |
+
+***THE OBJECTS KEEP THEIR PATHS, AND THAT IS NOT COSMETIC.*** Sixteen
+basenames collide across the 43 source directories -- `array.c` is
+both `ext/standard/array.c` and `ext/lexbor/lexbor/core/array.c`, and
+`hash`, `parser`, `state`, `token`, `url` and ten more are the same.
+A flat `OFILES` would build one and link it twice, silently. php is a
+PROGRAM rather than a library, so there is no archive member to
+collide and nested paths cost only a `mkdir` in the rule -- *the
+`ar`-member collision that a library would have had is the same shape
+as the stale `mcontext.6` in `docs/notes/libap.md`, avoided here by
+not having an archive at all.*
+
+Two mkfile details that are rules rather than taste. **Each comment
+inside the `OFILES` list ends with a backslash**, which is
+`mkcont-sweep.py`'s rule: a comment that ends a continuation ends the
+variable, and the rest of the list would vanish without a word. And
+**`HFILES` names the four hand-kept headers**, because mk does not
+scan `#include`s -- without it, editing `php_config.h` rebuilds
+nothing and the next run measures the old binary, which is how
+`HELP_BUILTIN` was turned on and did not reach bash.
+
+`mkcont-sweep` and `rceq-sweep` both pass; `apehdr-sweep` is
+**identical with and without** the new `<ucontext.h>` (151/152
+headers, the same 8 pre-existing TOGETHER errors), which is a control
+rather than a reading of the messages.
+
+**php is deliberately NOT in `sys/src/ape/cmd/mkfile`.** One failing
+entry aborts the whole tree's `mk`, and a port that has never been
+built has not been measured in that position. `cd sys/src/ape/cmd/php
+&& mk` is how to try it.
+
+## What the config.h decided, and why by hand
+
+The long reasoning is in the file itself; what matters as a summary:
+
+***TWO MACROS ARE ANSWERED NO FOR CAPABILITIES THAT ARE PRESENT, LINK,
+AND RETURN SUCCESS.*** This is clisp's `mprotect` rule, twice:
+
+* **`HAVE_MPROTECT`** -- `plan9/mman.c:44` is `return 0`, a stub
+  answering success. Nothing it is asked to protect is protected.
+* **`HAVE_MREMAP`** -- *and this one is worse, because the caller is
+  told it succeeded at something else*. APExp's `mremap` allocates and
+  copies: it MOVES. `zend_mm_chunk_extend` calls
+  `mremap(addr, old, new, 0)` with no `MREMAP_MAYMOVE` precisely
+  because it needs the chunk to stay put, and returns success on
+  anything that is not `MAP_FAILED`. Zend would record a chunk as
+  extended in place while the memory had moved: not a failure, a
+  silently wrong heap.
+
+A configure probe that compiles and links cannot tell either of these
+from the real thing. *That is the whole argument for hand-answering,
+and php supplies it twice without being asked.*
+
+**`STRERROR_R_CHAR_P` is unset and copying the host answer would have
+been a real bug**: glibc's `strerror_r` returns `char *`, APE's is
+POSIX's returning `int` (`<string.h>:45`). php would have used an
+`int` as a pointer.
+
+**`ZEND_CHECK_STACK_LIMIT` is unset, and the cost is recorded rather
+than hidden**: `zend_call_stack.c` finds the stack bounds through
+`pthread_getattr_np`, `pthread_attr_getstack`, `/proc`, `sysctl` or a
+Windows call, and APExp has none. Deep PHP recursion will fault
+instead of raising an error.
+
+**`HAVE_IPV6` is unset and that is a CHOICE, not a finding.**
+`AF_INET6` and `struct sockaddr_in6` both exist. Nothing in this tree
+has ever exercised IPv6 through APE's socket layer, and a port's first
+build is the wrong place to open an unmeasured path. Revisit with a
+test, not with an edit.
+
+***AND A QUESTION ANSWERED ITSELF.*** The sweep found **37 functions
+APE has that the host configure left `#undef`** -- `clock_gettime`,
+`select`, `strndup`, `times`, `utimes`, `pread`, `getlogin` and so on
+-- which looked like a judgement call about enabling code paths
+upstream never compiles on Linux. It is not: **`grep` says all 37 are
+referenced in 0 files** of what this port builds. Nothing reads them,
+so there was nothing to decide.
+
 ## Open, as a list
 
-* fibers: `ucontext` is written (`docs/notes/libap.md`) and passes on
-  the host against glibc; **not yet run on the VM**, and php has not
-  been pointed at it (`ZEND_FIBER_UCONTEXT` in the `config.h`)
-* `main/php_config.h` not written yet; the inventory is derived, the
-  answers are not
+* fibers: **done** — `ucontext` is in libap and `ucontext-test` is 0
+  failures on the VM (`docs/notes/libap.md`). What remains is pointing
+  php at it: `ZEND_FIBER_UCONTEXT` in the `config.h`
+* the first build: `cd sys/src/ape/cmd/php && mk`. Nothing has
+  compiled yet, so the object list, the include paths and every
+  answer in the `config.h` are all unmeasured together
 * `memfd_create` is declared with no implementation, which is a libap
   trap independent of php
 * `sys/src/ape/lib/ap/time/asctime_r.c:30` is

sys/include/ape/ucontext.h

@@ -2,6 +2,7 @@
 #define __UCONTEXT_H
 
 #include <signal.h>
+#include <stddef.h>
 #include <ucontext_arch.h>
 
 #ifdef __cplusplus
@@ -32,7 +33,7 @@ extern "C" {
  */
 typedef struct {
 	void  *ss_sp;
-	long   ss_size;
+	size_t ss_size;		/* POSIX's type, and `long' is 4 bytes here */
 	int    ss_flags;
 } stack_t;
 

sys/src/ape/cmd/php/build-defs.h

@@ -0,0 +1,35 @@
+/*
+ * build-defs.h -- generated by configure in an upstream build;
+ * hand-answered here.
+ *
+ * ***THE PATHS ARE ARCHITECTURE-INDEPENDENT ON PURPOSE.***  They are
+ * baked into the binary as string constants, so they cannot carry
+ * `$objtype' -- and they do not need to: php's runtime files are
+ * text.  `/sys/lib/ape/php' follows `cmd/bash/mkfile's
+ * `-DLOCALEDIR="/sys/lib/ape/locale"', which is the same shape.
+ * Only the BINARY is per-architecture, and the mkfile installs that
+ * through `$BIN'.
+ *
+ * `PHP_EXTENSION_DIR' and the shared-library names are empty because
+ * **Plan 9 has no dynamic linking** -- the same reason perl here has
+ * no XS extensions and `HAVE_LIBDL' is unset in `php_config.h'.  A
+ * path would be a promise nothing can keep.
+ */
+
+#define CONFIGURE_COMMAND	"hand-answered for APExp; see sys/src/ape/cmd/php/mkfile"
+#define PHP_PROG_SENDMAIL	""
+#define PEAR_INSTALLDIR		""
+#define PHP_INCLUDE_PATH	".:/sys/lib/ape/php"
+#define PHP_EXTENSION_DIR	""
+#define PHP_PREFIX		"/sys/lib/ape/php"
+#define PHP_BINDIR		"/bin/ape"
+#define PHP_SBINDIR		"/bin/ape"
+#define PHP_MANDIR		"/sys/man"
+#define PHP_LIBDIR		"/sys/lib/ape/php"
+#define PHP_DATADIR		"/sys/lib/ape/php"
+#define PHP_SYSCONFDIR		"/sys/lib/ape/php"
+#define PHP_LOCALSTATEDIR	"/tmp"
+#define PHP_CONFIG_FILE_PATH	"/sys/lib/ape/php"
+#define PHP_CONFIG_FILE_SCAN_DIR	""
+#define PHP_SHLIB_SUFFIX	""
+#define PHP_SHLIB_EXT_PREFIX	""

sys/src/ape/cmd/php/internal_functions_cli.c

@@ -0,0 +1,76 @@
+/*
+   +----------------------------------------------------------------------+
+   | Copyright (c) The PHP Group                                          |
+   +----------------------------------------------------------------------+
+   | This source file is subject to version 3.01 of the PHP license,      |
+   | that is bundled with this package in the file LICENSE, and is        |
+   | available through the world-wide-web at the following url:           |
+   | https://www.php.net/license/3_01.txt                                 |
+   | If you did not receive a copy of the PHP license and are unable to   |
+   | obtain it through the world-wide-web, please send a note to          |
+   | license@php.net so we can mail you a copy immediately.               |
+   +----------------------------------------------------------------------+
+   | Authors: Andi Gutmans <andi@php.net>                                 |
+   |          Zeev Suraski <zeev@php.net>                                 |
+   +----------------------------------------------------------------------+
+ */
+
+/*
+ * internal_functions_cli.c -- generated by configure in an upstream
+ * build; kept here so `external/php' stays untouched.
+ *
+ * It is the table of extensions linked into the binary, and it is
+ * DERIVED rather than written: configure produced it from the flags
+ * in this port's mkfile.  The eleven below are what `--disable-all'
+ * leaves, which is to say the ones php 8.5 builds unconditionally.
+ * **`opcache' is in the list and has to be**: there is no
+ * `--disable-opcache'.  It only has to register, since
+ * `opcache.enable_cli' is off by default and the accelerator never
+ * starts -- which is why all three of its shared-memory macros are
+ * answered no in `php_config.h'.
+ *
+ * Adding an extension means adding its header, its `phpext_*_ptr' and
+ * its objects to the mkfile, together.
+ */
+#include "php.h"
+#include "php_main.h"
+#include "zend_modules.h"
+#include "zend_compile.h"
+#include <stdarg.h>
+#include <stdlib.h>
+#include <stdio.h>
+
+#include "ext/date/php_date.h"
+#include "ext/pcre/php_pcre.h"
+#include "ext/hash/php_hash.h"
+#include "ext/json/php_json.h"
+#include "ext/lexbor/php_lexbor.h"
+#include "ext/opcache/zend_accelerator_module.h"
+#include "ext/random/php_random.h"
+#include "ext/reflection/php_reflection.h"
+#include "ext/spl/php_spl.h"
+#include "ext/standard/php_standard.h"
+#include "ext/uri/php_uri.h"
+
+
+static zend_module_entry * const php_builtin_extensions[] = {
+	phpext_date_ptr,
+	phpext_pcre_ptr,
+	phpext_hash_ptr,
+	phpext_json_ptr,
+	phpext_lexbor_ptr,
+	phpext_opcache_ptr,
+	phpext_random_ptr,
+	phpext_reflection_ptr,
+	phpext_standard_ptr,
+	phpext_spl_ptr,
+	phpext_uri_ptr,
+
+};
+
+#define EXTCOUNT (sizeof(php_builtin_extensions)/sizeof(zend_module_entry *))
+
+PHPAPI int php_register_internal_extensions(void)
+{
+	return php_register_extensions(php_builtin_extensions, EXTCOUNT);
+}