php: build the four extensions phoronix-test-suite requires

docs/notes/php.md

@@ -746,6 +746,85 @@ general form is worth keeping**: a header-only helper that calls into
 a file the port does not compile is a link error here and silence on
 gcc.
 
+### The four extensions Phoronix needs, and three derivation faults
+
+`php-smoke.php` is **0 failures** on the VM and Phoronix now reaches
+its own requirement check instead of crashing. What it asks for is in
+`pts-core.php:208`, and the error message is misleading in a way that
+matters:
+
+	array(1, extension_loaded('zip') || extension_loaded('zlib'), 'ZIP', ...)
+
+**"MISSING ZIP" is satisfied by ext/zlib**, so libzip is not involved
+at all. With JSON already built, the required set is **libxml + dom +
+simplexml + zlib** -- 37 + 1 + 3 + 3 = 44 objects against two
+libraries this tree already carries, `lib/xml2` (2.15.2) and `lib/z`.
+dom also depends on `lexbor`, which this build already has for its own
+sake. **559 translation units compile, 0 errors.**
+
+#### Linking is by `#pragma lib`, not by a mkfile variable
+
+`<zlib.h>:34` has always carried `#pragma lib "/$M/lib/ape/libz.a"`;
+`<libxml/xmlversion.h>` carries one now. `cc/macbody:728` reads it and
+records the archive in the object, and `apexp-sh`'s bind makes that
+path this tree's copy -- so no `LIB=` line, and *every* libxml2
+consumer links correctly rather than just this one.
+
+**It goes in `xmlversion.h` because that is the header every other one
+reaches**: 42 of the 46 installed `<libxml/*.h>` include it, and the
+three that do not -- `SAX.h`, `nanoftp.h`, `xmlunicode.h` -- are empty
+deprecation stubs that declare nothing and so cannot create a
+reference. *Checked, not assumed.* **Measured**: a TU including it
+records `<libxml2.a` in the object, exactly as a TU including
+`<zlib.h>` records `<libz.a`, and one including neither records
+nothing.
+
+*The hazard is that the header is otherwise byte-identical to the
+vendored copy*, so a re-sync drops the pragma and every consumer then
+fails to link with no hint why. `<zlib.h>` has the same arrangement
+and the same exposure; both say so in the file.
+
+#### THREE DERIVATION FAULTS IN ONE LIST, EACH FOUND BY THE NEXT CHECK
+
+The object list was *derived* from each extension's own `config.m4`,
+which is this tree's rule -- and the derivation was wrong three times:
+
+1. `^ *[a-z0-9_]+\.c$` **cannot match a path with a slash**, so
+   `parentnode/css_selectors.c` and `parentnode/tree.c` vanished.
+   ***And the completeness check compared the result against
+   `ls *.c`, which cannot see a subdirectory either*** -- 34 against
+   34, agreeing because both sides shared the same wrong assumption.
+   *A derivation and its check have to fail independently, or the
+   check is decoration.* `--undefined` caught it by naming
+   `dom_parent_node_append`.
+
+2. Widening the class to allow `/` found those two and **still missed
+   `lexbor/selectors-adapted/selectors.c`, for the hyphen**.
+   `--undefined` caught that one too, naming `lxb_selectors_init`.
+
+3. Widening it again to `[^ \t"]+\.c` matched fragments mid-token and
+   produced `dom/ion.c`, `dom/ype.c`, `zlib/er.c` -- garbage that an
+   existence check does filter out, but by then the method was
+   plainly wrong.
+
+**The answer was to read the block.** dom's `PHP_NEW_EXTENSION` is 37
+plain lines, and the `PHP_ADD_BUILD_DIR` directly under it names both
+subdirectories -- which is also where the `OBJDIRS` entries come from.
+*Two rounds were spent widening a regular expression against a file
+short enough to read.*
+
+#### The instrument had to grow with the build
+
+`--undefined` asked `apfunc-sweep.py`, which knows **libap and
+nothing else** -- correct while every port linked only libap, and
+wrong the moment php pulled libxml2 and libz: it answered `absent`
+for 241 of 539 names. *The same shape as that sweep's own recorded
+wrong-`ROOT` failure, from the other direction -- not a corpus that
+does not exist, but one that is merely incomplete.* It takes
+`--root DIR` now, repeatable, default unchanged, and `php-mirror`
+passes the two library trees. **Add a root there for every library a
+new extension links.**
+
 ### CLOSED: every multiplication over 2**32 was a float
 
 `exit()` fixed, the smoke test runs to completion and **the exit

sys/include/ape/libxml/xmlversion.h

@@ -13,6 +13,29 @@
 #ifndef __XML_VERSION_H__
 #define __XML_VERSION_H__
 
+/*
+ * APExp: link libxml2 the Plan 9 way, as <zlib.h> and the rest of the
+ * APE headers do.  `cc/macbody:728' reads this and records the archive
+ * in the object, so a program that includes any <libxml/*.h> links
+ * against it without naming it in a mkfile; `apexp-sh' binds this
+ * tree's `$objtype/lib/ape' over `/$M/lib/ape', so the path is ours.
+ *
+ * **THIS HEADER IS OTHERWISE BYTE-IDENTICAL TO THE VENDORED COPY** at
+ * `sys/src/external/libxml2/include/libxml/xmlversion.h', so a re-sync
+ * of `sys/include/ape/libxml' silently drops these three lines and
+ * every libxml2 consumer then fails to link with no hint as to why.
+ * `<zlib.h>' is the same arrangement -- it differs from
+ * `external/zlib/zlib.h' by this pragma and a blank line, and nothing
+ * else -- so re-applying it is the known cost of updating either.
+ *
+ * ***IT GOES IN THIS FILE BECAUSE THIS IS THE ONE EVERY OTHER HEADER
+ * REACHES***: 42 of the 46 installed `<libxml/*.h>' include it, and
+ * the three that do not -- `SAX.h', `nanoftp.h', `xmlunicode.h' -- are
+ * empty deprecation stubs that declare nothing, so they cannot create
+ * a reference to resolve.  *Checked rather than assumed.*
+ */
+#pragma lib "/$M/lib/ape/libxml2.a"
+
 /**
  * the version string like "1.2.3"
  */

sys/lib/tests/apfunc-sweep.py

@@ -56,6 +56,21 @@
 AP = os.path.join(ROOT, 'sys/src/ape/lib/ap')
 HDR = os.path.join(ROOT, 'sys/include/ape')
 
+# ***THE CORPUS IS A LIST BECAUSE A PROGRAM LINKS MORE THAN ONE
+# LIBRARY.***  It was `libap and nothing else', which is the right
+# default and was the whole question while every port linked only
+# libap.  php now pulls libxml2 and libz through `#pragma lib', so
+# `php-mirror.sh --undefined' asked about 539 names and this file
+# confidently answered `absent' for the 241 that live in those two --
+# *the same shape as the wrong-ROOT failure above, arrived at from
+# the other direction*: not a corpus that does not exist, but one
+# that is merely incomplete.
+#
+# `--root DIR' (repeatable, relative to the tree root) adds a library
+# source tree.  The default is unchanged, so every existing caller
+# gets exactly the answers it got before.
+ROOTS = [AP]
+
 if not os.path.isdir(AP) or not os.path.isdir(HDR):
     sys.stderr.write('apfunc-sweep: %s or %s is not a directory -- ROOT is '
                      'wrong, and every answer would be "absent"\n' % (AP, HDR))
@@ -94,11 +109,12 @@ def definition_patterns(name):
 
 def defined_in(name):
     for pat in definition_patterns(name):
-        out = subprocess.run(['grep', '-rlE', pat, AP, '--include=*.c'],
-                             capture_output=True, text=True)
-        files = out.stdout.strip().splitlines()
-        if files:
-            return os.path.relpath(files[0], ROOT)
+        for root in ROOTS:
+            out = subprocess.run(['grep', '-rlE', pat, root, '--include=*.c'],
+                                 capture_output=True, text=True)
+            files = out.stdout.strip().splitlines()
+            if files:
+                return os.path.relpath(files[0], ROOT)
     return None
 
 
@@ -149,14 +165,23 @@ def control():
 
 def main():
     args = sys.argv[1:]
+    while '--root' in args:
+        i = args.index('--root')
+        extra = os.path.join(ROOT, args[i + 1])
+        if not os.path.isdir(extra):
+            sys.stderr.write('apfunc-sweep: --root %s is not a directory\n'
+                             % extra)
+            return 2
+        ROOTS.append(extra)
+        del args[i:i + 2]
     if '--control' in args:
         return control()
     if args[:1] == ['--file']:
         names = [l.strip() for l in open(args[1]) if l.strip()]
     elif args:
         names = args
     else:
-        print(__doc__ or 'usage: apfunc-sweep.py NAME... | --file f | --control')
+        print(__doc__ or 'usage: apfunc-sweep.py [--root DIR]... NAME... | --file f | --control')
         return 1
     run(names)
     return 0

sys/lib/tests/hostkencc/php-mirror.sh

@@ -278,7 +278,17 @@ EOF
 	n=$(wc -l < "$OUT/undefined")
 	echo "php-mirror: $n symbol(s) the link must find outside php"
 	echo "php-mirror: the list is in $OUT/undefined; asking libap --"
-	python3 "$ROOT/sys/lib/tests/apfunc-sweep.py" --file "$OUT/undefined" |
+	# ***THREE LIBRARIES NOW, NOT ONE.***  php pulls libxml2 and libz
+	# through `#pragma lib' in their headers, so a sweep that knows
+	# only libap answers `absent' for every `xmlParseFile' and
+	# `deflateInit_' -- 241 of the 539 the first run with the dom,
+	# simplexml and zlib extensions reported.  `--root' adds their
+	# vendored sources; add one here for every library a new
+	# extension links.
+	python3 "$ROOT/sys/lib/tests/apfunc-sweep.py" \
+		--root sys/src/external/libxml2 \
+		--root sys/src/external/zlib \
+		--file "$OUT/undefined" |
 		grep -v '	*defined   ' || true
 	echo "php-mirror: ^ anything here is READ, not a failure -- the"
 	echo "php-mirror:   sweep sees C function definitions only, so"

sys/src/ape/cmd/php/cfg/main/php_config.h

@@ -360,6 +360,22 @@
  * this is the arity, and it is 1.
  */
 #define HAVE_FUNC_GETHOSTBYNAME_R_6 1
+
+/*
+ * ***THE FOUR EXTENSIONS ADDED FOR PHORONIX.***  Each name is the one
+ * its own `config.m4' AC_DEFINEs -- `libxml/config0.m4:12',
+ * `dom/config.m4:9', `simplexml/config.m4:9', `zlib/config0.m4:8' --
+ * read out of the files rather than recalled, since an extension that
+ * compiles with its HAVE_ unanswered is an extension whose every
+ * `#ifdef'-guarded function quietly vanishes.
+ *
+ * `HAVE_ZLIB' is what satisfies Phoronix's "MISSING ZIP": its check is
+ * `extension_loaded('zip') || extension_loaded('zlib')'.
+ */
+#define HAVE_LIBXML 1
+#define HAVE_DOM 1
+#define HAVE_SIMPLEXML 1
+#define HAVE_ZLIB 1
 /* #undef HAVE_GETIFADDRS */
 /* #undef HAVE_GETLOADAVG */
 /* #undef HAVE_GETTID */

sys/src/ape/cmd/php/internal_functions_cli.c

@@ -31,6 +31,13 @@
  *
  * Adding an extension means adding its header, its `phpext_*_ptr' and
  * its objects to the mkfile, together.
+ *
+ * ***THE FOUR ADDED FOR PHORONIX ARE libxml, dom, simplexml AND
+ * zlib***, which is what `pts_needed_extensions()' asks for once JSON
+ * is counted as present and once `extension_loaded('zip') ||
+ * extension_loaded('zlib')' is read as the disjunction it is -- the
+ * error message says "MISSING ZIP" and ext/zlib answers it, so
+ * libzip is not involved.
  */
 #include "php.h"
 #include "php_main.h"
@@ -45,6 +52,10 @@
 #include "ext/hash/php_hash.h"
 #include "ext/json/php_json.h"
 #include "ext/lexbor/php_lexbor.h"
+#include "ext/libxml/php_libxml.h"
+#include "ext/dom/php_dom.h"
+#include "ext/simplexml/php_simplexml.h"
+#include "ext/zlib/php_zlib.h"
 #include "ext/opcache/zend_accelerator_module.h"
 #include "ext/random/php_random.h"
 #include "ext/reflection/php_reflection.h"
@@ -59,6 +70,16 @@ static zend_module_entry * const php_builtin_extensions[] = {
 	phpext_hash_ptr,
 	phpext_json_ptr,
 	phpext_lexbor_ptr,
+	/*
+	 * libxml comes before dom and simplexml because both declare a
+	 * dependency on it (`PHP_ADD_EXTENSION_DEP' in their config.m4)
+	 * and this array IS the registration order.  dom also depends on
+	 * lexbor, which is above it for its own reasons.
+	 */
+	phpext_libxml_ptr,
+	phpext_dom_ptr,
+	phpext_simplexml_ptr,
+	phpext_zlib_ptr,
 	phpext_opcache_ptr,
 	phpext_random_ptr,
 	phpext_reflection_ptr,

sys/src/ape/cmd/php/mkfile

@@ -424,6 +424,94 @@ OFILES=\
 	ext/lexbor/lexbor/url/url.$O\
 # ext/lexbor\
 	ext/lexbor/php_lexbor.$O\
+# ***THE FOUR EXTENSIONS PHORONIX NEEDS, AND THE LIST IS DERIVED FROM\
+# pts-core's OWN CHECK RATHER THAN FROM ITS ERROR MESSAGE.***\
+# `pts-core.php:208's `pts_needed_extensions()' requires four things:\
+# JSON (already built), DOM, SimpleXML, and -- the one the message\
+# hides -- `extension_loaded('zip') || extension_loaded('zlib')'.\
+# **ext/zlib satisfies the "MISSING ZIP" line**, so libzip is not\
+# needed and `ext/zip' is not built; `lib/z' already is.\
+#\
+# `ext/libxml' is the glue BOTH dom and simplexml declare a dependency\
+# on (`PHP_ADD_EXTENSION_DEP'), so it comes first in this list and\
+# first in `internal_functions_cli.c' -- the module order is the\
+# registration order. dom also depends on `lexbor', which this build\
+# already carries for its own sake.\
+#\
+# Every name below is from the extension's own `config.m4'\
+# `PHP_NEW_EXTENSION' line, read with awk, and each was checked to\
+# exist on disk -- the same derivation as the original 513.\
+#\
+# ***AND THE FIRST DERIVATION DROPPED TWO OF THEM, SILENTLY.***  The\
+# awk pattern was `^ *[a-z0-9_]+\.c$', which cannot match a path with\
+# a SLASH, so `parentnode/css_selectors.c' and `parentnode/tree.c'\
+# vanished.  **And the check that the list was complete compared it\
+# against `ls *.c' in `ext/dom', which cannot see a subdirectory\
+# either** -- 34 against 34, agreeing because both sides shared the\
+# same wrong assumption.  *A derivation and its check have to fail\
+# independently or the check is decoration.*\
+#\
+# `php-mirror.sh --undefined' is what caught it, naming\
+# `dom_parent_node_append' among the unresolved symbols; the VM's\
+# `6l' would have said the same thing one rebuild later.\
+#\
+# ***THEN THE WIDENED PATTERN DROPPED A THIRD, FOR THE HYPHEN.***\
+# Allowing `/' in the path found the two above and still missed\
+# `lexbor/selectors-adapted/selectors.c', whose directory has a `-'\
+# in it, and `--undefined' came back naming `lxb_selectors_init'.\
+# *Two rounds of widening a character class, each time believing the\
+# list was now complete.*  *** READ THE BLOCK, DO NOT PATTERN-MATCH\
+# IT***: dom's `PHP_NEW_EXTENSION' is 37 plain lines, and the\
+# `PHP_ADD_BUILD_DIR' right under it names both subdirectories --\
+# which is where the OBJDIRS entries below come from as well.\
+# ext/libxml\
+	ext/libxml/libxml.$O\
+	ext/libxml/mime_sniff.$O\
+	ext/libxml/image_svg.$O\
+# ext/dom\
+	ext/dom/attr.$O\
+	ext/dom/cdatasection.$O\
+	ext/dom/characterdata.$O\
+	ext/dom/comment.$O\
+	ext/dom/document.$O\
+	ext/dom/documentfragment.$O\
+	ext/dom/documenttype.$O\
+	ext/dom/dom_iterators.$O\
+	ext/dom/domexception.$O\
+	ext/dom/domimplementation.$O\
+	ext/dom/element.$O\
+	ext/dom/entity.$O\
+	ext/dom/entityreference.$O\
+	ext/dom/html5_parser.$O\
+	ext/dom/html5_serializer.$O\
+	ext/dom/html_collection.$O\
+	ext/dom/html_document.$O\
+	ext/dom/infra.$O\
+	ext/dom/inner_outer_html_mixin.$O\
+	ext/dom/namednodemap.$O\
+	ext/dom/namespace_compat.$O\
+	ext/dom/node.$O\
+	ext/dom/nodelist.$O\
+	ext/dom/notation.$O\
+	ext/dom/obj_map.$O\
+	ext/dom/php_dom.$O\
+	ext/dom/private_data.$O\
+	ext/dom/processinginstruction.$O\
+	ext/dom/text.$O\
+	ext/dom/token_list.$O\
+	ext/dom/xml_document.$O\
+	ext/dom/xml_serializer.$O\
+	ext/dom/xpath.$O\
+	ext/dom/xpath_callbacks.$O\
+	ext/dom/parentnode/css_selectors.$O\
+	ext/dom/parentnode/tree.$O\
+	ext/dom/lexbor/selectors-adapted/selectors.$O\
+# ext/simplexml\
+	ext/simplexml/simplexml.$O\
+# ext/zlib\
+	ext/zlib/zlib.$O\
+	ext/zlib/zlib_fopen_wrapper.$O\
+	ext/zlib/zlib_filter.$O\
 # ext/opcache\
 	ext/opcache/ZendAccelerator.$O\
 	ext/opcache/shared_alloc_mmap.$O\
@@ -665,6 +753,20 @@ OFILES=\
 HFILES=cfg/main/php_config.h cfg/php_config.h cfg/zend_config.h\
 	cfg/timelib_config.h cfg/build-defs.h
 
+# ***NO `LIB=' LINE, AND THAT IS DELIBERATE.***  Both libraries this
+# binary needs beyond libap -- libxml2 and libz -- are pulled by
+# `#pragma lib' in their own headers, which is how every other library
+# in this tree is linked: `<zlib.h>:34' has always carried one, and
+# `<libxml/xmlversion.h>' carries one now.  `cc/macbody:728' records
+# the archive in the object, so naming it here as well would be
+# saying the same thing twice in the place where it can rot.
+#
+# *The first version of this mkfile did name libxml2 in `LIB='*, on
+# the grounds that putting a pragma in a public APE header reaches
+# every program in the tree.  It does, and that is the point: a
+# program that includes `<libxml/*.h>' is a program that needs the
+# archive.
+
 <$APEXPROOT/sys/src/cmd/mkone
 
 CC=pcc
@@ -712,6 +814,8 @@ OBJDIRS=\
 	ext/lexbor/lexbor/ports/posix/lexbor/core\
 	ext/lexbor/lexbor/punycode ext/lexbor/lexbor/tag\
 	ext/lexbor/lexbor/unicode ext/lexbor/lexbor/url ext/opcache\
+	ext/libxml ext/dom ext/dom/parentnode\
+	ext/dom/lexbor/selectors-adapted ext/simplexml ext/zlib\
 	ext/pcre ext/pcre/pcre2lib ext/random ext/reflection ext/spl\
 	ext/standard ext/standard/libavifinfo ext/uri\
 	ext/uri/uriparser/src main main/streams sapi/cli