diff --git a/CHANGELOG.md b/CHANGELOG.md index 0d311a44260..029fa287d01 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -27,6 +27,7 @@ This serves two purposes: - Raw HTML in Markdown is now enabled by default. Set `markdown.allow_html` to `false` when compiling untrusted or unreviewed Markdown to strip potentially unsafe HTML tags. - `InMemoryPage` now requires callers to select either `contents` or `view`; configuring both throws an `InvalidArgumentException` instead of silently giving contents precedence. - `InMemoryPage` now treats an empty string as an omitted `view`, matching the existing compile-time behavior and allowing literal contents to be used with an empty view value. +- Pages with non-HTML output paths are now excluded from automatic navigation by default. Set `navigation.visible: true` or `navigation.hidden: false` to include one explicitly. ### Deprecated - for changes that will be removed in upcoming releases. diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 00000000000..f916f76fdf1 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,90 @@ +# Agent instructions for the HydePHP monorepo + +This is the hydephp/develop monorepo. Framework code lives in `packages/framework`, +shared test utilities in `packages/testing`, the dev server in `packages/realtime-compiler`. +We are developing HydePHP v3; the main branch for PRs is `2.x`. + +## Epic-driven workflow + +- Multi-PR efforts are specified in epic documents at the repo root (like + `EPIC_NON_HTML_PAGES.md`). The epic is the source of truth: before implementing, + re-read the relevant PR section and design decisions; after implementing, check the + diff against them line by line. +- Deviations from the design and decisions the epic left open MUST be written back + into the epic in the same PR, and implemented sections marked (like "✅ Implemented"). +- Before deviating, verify it is a *good* deviation: check side effects against the + rest of the epic (later PRs, design decisions) so the change doesn't drift from the + overall design. Also watch for silent under-delivery — implementing a design rule + only for the cases the current PR exercises will break later PRs that rely on it. +- Work on the epic happens in `v3/-*` branches off the epic base branch. + Check `git branch --list 'v3/*'` and the epic's status annotations (which may only + exist on PR branches) before assuming a PR is unimplemented. + +## Commits + +- Make atomic commits as you go: one logical change per commit (implementation, + tests, docs/release notes separately when they are separable). Do not batch a + whole PR into one commit at the end. + +## Testing + +The goal is full confidence: feature tests give 100% coverage by exercising all user +paths end-to-end (e.g. register a page, run the real `build` command, assert the +output file), and unit tests cover every code unit. + +- Run suites with `vendor/bin/pest --testsuite FeatureFramework|UnitFramework|FeatureHyde`. + Use `--filter` for targeted runs. Run `php monorepo/HydeStan/run.php` before finishing. +- Every public method on page classes is part of the `BaseHydePageUnitTest` contract + (`packages/testing/src/Common/`): adding public API to `HydePage` means adding an + abstract test there and implementing it in all page unit tests in + `packages/framework/tests/Unit/Pages/`. `TestAllPageTypesHaveUnitTestsTest` enforces + one unit test class per page class. +- The suites read the real project root and pollute the working tree: they delete + `_media/app.css` and leave untracked junk (`_assets/`, `_docs/docs.md`, `_docs/index.md`, + `_pages/root.md`, `_pages/root1.md`, `_posts/my-new-post.md`, `_media/app.js`, `_site/`). + Dozens of environmental failures follow from that state — they are not regressions. + Restore with `git checkout -- _media/app.css` and delete the junk between runs. +- To prove a change introduces no regressions: run the suite at HEAD, then + `git checkout HEAD~N -- packages` (pre-change baseline), clean the tree, re-run, + and diff the sorted `FAILED` lines. Identical lists means no regressions. Restore + with `git checkout HEAD -- packages`. Don't pipe long pest runs through `tail` — + redirect to a file and grep it. +- Known pre-existing failure: `FeaturedImageUnitTest` on PHP 8.5 (`MediaFile::findHash()`). + +## Release notes and docs + +- Add release-notes entries to `HYDEPHP_V3_PLANNING.md` and upgrade steps to + `UPGRADE.md` as part of the PR that makes the change. +- Breaking-change notes must describe realistic impact. If the note describes a + scenario nobody would plausibly be in (like relying on double-extension output + such as `data.json.html`), say that no real impact is expected instead of + prescribing a migration. + +## Code comments + +If you catch yourself writing a code comment, stop and think about why. A comment is +usually a smell that the code is doing something weird — step back and look at what +you are actually trying to do before reaching for a comment. Comments are only for: + +1. Adding better type support (docblocks, `@param`/`@var` annotations). +2. Documenting public APIs. +3. Explaining why unusual-looking code has to be that way, when there is no other option. + +Never write comments that narrate what the next line does, where code came from, or +why a change is correct — that belongs in the PR description, not the code. + +## Developer Experience + +HydePHP treats Developer Experience as a design constraint, not a layer of polish added after a feature works. The framework exists to make creating content-focused websites feel simple without taking away the power developers expect from Laravel. Its guiding promise is that users should be able to begin with Markdown and sensible defaults, while retaining the freedom to use Blade, customize the frontend, replace conventions, or extend the build process when their project demands it. + +The default path should therefore be the shortest path. A common task should work without configuration, manual registration, or knowledge of internal architecture. HydePHP favors convention over configuration, automatic discovery, appropriate default layouts, generated navigation, scaffolding commands, and ready-to-use frontend assets. Configuration and extension points are still important, but they should remain optional until the user has a reason to reach for them. A new feature is aligned with HydePHP when its basic use feels obvious and its advanced use remains possible. + +HydePHP also aims to reuse mental models its users already know. APIs should follow Laravel conventions where practical, and features should compose naturally with familiar tools such as collections, facades, Blade components, console commands, configuration files, service providers, and lifecycle callbacks. Naming should describe what an operation does rather than expose how Hyde implements it. Before inventing a new abstraction, an agent should look for the closest existing HydePHP or Laravel pattern and extend that vocabulary consistently. + +Good Developer Experience includes the failure path. HydePHP should validate assumptions early, produce actionable error messages, and avoid letting mistakes silently reach the generated site. Recent work on the asset and data systems reflects this approach through automatic validation, clearer exception handling, syntax checking, and helpers that remove repetitive filesystem work. Commands should explain what they are doing, generated files should be predictable, and errors should tell the developer what needs to change. + +Performance and feedback speed matter as well. Improvements such as realtime compilation, Vite integration, hot module replacement, intelligent caching, and faster document processing are Developer Experience features because they shorten the distance between an edit and a trustworthy result. Agents should avoid unnecessary work in normal builds, preserve deterministic output, and prefer lazy or cached computation where it reduces repeated cost without making behavior harder to understand. + +Finally, a feature is not complete when its implementation compiles. HydePHP requires focused changes, tests that demonstrate the intended behavior, and documentation for changes users can observe. Backward compatibility and the appropriate release branch must also be considered. Tests protect the experience from regression, while documentation confirms that the public API can be explained clearly. When an API is difficult to test or document, that is often evidence that it is also difficult to use. + +An AI coding agent working on HydePHP should evaluate every feature with a simple standard: does this make the common case joyful, preserve control for advanced users, behave like the rest of the Laravel ecosystem, fail helpfully, and remain understandable through tests and documentation? The most Hyde-like implementation is rarely the one with the most options or abstractions. It is the one that removes the most friction while introducing the least surprise. diff --git a/EPIC_NON_HTML_PAGES.md b/EPIC_NON_HTML_PAGES.md new file mode 100644 index 00000000000..bdf6779aa25 --- /dev/null +++ b/EPIC_NON_HTML_PAGES.md @@ -0,0 +1,697 @@ +# Epic: First-class non-HTML pages (robots.txt, llms.txt, sitemap, RSS) + +> Status: Draft — v3 development branch +> +> Theme: Make non-HTML output files (txt, xml, json) first-class pages instead of +> build-task side effects, so they flow through routing, the build pipeline, the +> realtime compiler, and user-land extension points like every other page. + +## Motivation + +There is currently no easy way to add plain-text files like `robots.txt` or `llms.txt` +to a Hyde site. While investigating this, we found that the framework already contains +three different mechanisms for emitting non-HTML files, each with its own tradeoffs: + +1. **Post-build tasks** (`GenerateSitemap`, `GenerateRssFeed`) write directly to + `_site/` with `file_put_contents`, bypassing the page/route system entirely. + Consequence: `hyde serve` cannot serve `sitemap.xml` or `feed.xml` at all, and + they are invisible to `route:list` and the build manifest. +2. **Virtual pages** (`DocumentationSearchIndex extends InMemoryPage`) participate in + routing and the build, but only by manually overriding `getOutputPath()` because + `HydePage::outputPath()` hardcodes the `.html` suffix + (`packages/framework/src/Pages/Concerns/HydePage.php:213`). The realtime compiler + needs a hardcoded exemption to serve it + (`packages/realtime-compiler/src/Routing/Router.php:72-75`). +3. **Verbatim source files** (`HtmlPage`) are autodiscovered from `_pages` and copied + as-is, but this is a source-file convenience specific to HTML rather than a + requirement for supporting non-HTML output. + +Making the output format part of the page model gives us `robots.txt`/`llms.txt` +support nearly for free, fixes real bugs, and simplifies the framework. It does not +require every output format to have a matching filesystem-discovered page class. + +### Bugs and gaps this epic fixes + +- **`search.json` leaks into sitemaps in production.** `SitemapGenerator::generate()` + excludes only `Redirect` pages, so every other route is included. Verified live: + `https://hydephp.com/sitemap.xml` contains `docs/1.x/search.json` and + `docs/2.x/search.json`. There is no per-page sitemap exclusion mechanism at all. +- **`hyde serve` does not serve `sitemap.xml` or the RSS feed**, because they only + exist as post-build artifacts. +- **Custom page classes cannot declare a non-HTML output format**, forcing them to + override path resolution and making route-key and output-path drift possible. +- **The realtime compiler special-cases `search.json` by string suffix** instead of + asking the route system. + +### What already works in our favor + +- `PageRouter::getContentType()` already maps `json`/`xml`/`txt` output paths to + correct Content-Type headers (`packages/realtime-compiler/src/Routing/PageRouter.php:59-69`). +- `BuildService::getPageTypes()` derives page classes from the live page collection, + and `StaticPageBuilder` writes whatever `getOutputPath()` returns — dynamic pages + need zero build-service changes. +- `HydeCoreExtension::discoverPages()` is the proven registration point for + feature-gated virtual pages (search index, redirects), and users/packages can do + the same via `Hyde::kernel()->booting()` callbacks or a `HydeExtension`. +- The route-key-with-extension convention is already de facto established: + `DocumentationSearchIndex` uses route key `docs/search.json` equal to its output path. + +## Design decisions + +### D1: Route key equals output path; only `.html` is implicit + +Formalize the convention `DocumentationSearchIndex` already uses: a page's route key +is its output path, except that HTML pages drop the `.html` suffix (pretty URLs). +So `robots.txt`, `sitemap.xml`, and `docs/search.json` are route keys as-is, while +`about.html` keeps route key `about`. + +This is preferred over a "clean route key + separate output extension" model because +it is proven in production, requires no realtime-compiler lookup changes +(`PageRouter::normalizePath()` only strips `.html`), and lets `docs/search` (page) +and `docs/search.json` (index) coexist as distinct routes, which they already do. + +> **Implemented (PR 1):** `RouteKey::fromPage()` appends the page class's configured +> non-HTML output extension to the key (skipping it when the identifier already ends +> with it), so custom page classes configured with a non-HTML output extension are +> D1-compliant out of the box and PR 2 can rely on "route key == request path with +> only `.html` stripped" universally. + +### D2: In-memory output formats are inferred from the identifier + +`InMemoryPage::outputPath()` uses `pathinfo($identifier, PATHINFO_EXTENSION)` directly. +When the result is empty it appends `.html`; otherwise it keeps the identifier unchanged. +The inherited instance `getOutputPath()` calls this same static method, so static and +instance path resolution cannot disagree. + +This final design followed three rejected approaches: + +1. **Extension allowlist inference.** Treating a fixed set such as `.txt`, `.xml`, and + `.json` as files made those formats convenient but merely moved the ambiguity. An + identifier ending in `.webmanifest` or any future format unexpectedly became HTML. +2. **`InMemoryPage::file()` plus `$exactOutputPath`.** The explicit instance flag + supported arbitrary filenames, but violated the static/instance path contract: + `InMemoryPage::outputPath('robots.txt')` produced `robots.txt.html` while the flagged + instance's `getOutputPath()` produced `robots.txt`. Compiler integrations may resolve + output paths statically, so that mismatch was unsafe. +3. **`$outputExtension` subclasses.** A class-level extension restored static/instance + symmetry, but the developer experience was poor: every one-off file required a + boilerplate subclass, callers had to omit the extension from the identifier, and a + configurable RSS filename still needed its own path override. + +Pure `pathinfo` inference keeps arbitrary extensions, needs no flag or subclass, and +preserves symmetry. The earlier concern that versioned documentation paths would be +false positives was based on a misunderstanding: `pathinfo` examines the basename, so +`docs/1.x/index` has no extension and correctly becomes `docs/1.x/index.html`. While +`docs/1.x` itself has extension `x`, that path is not a valid route key for the version's +root index; the valid identifier ends in `index`. + +### D3: Sitemap inclusion becomes a page-level concern + +Replace the `instanceof Redirect` filter in `SitemapGenerator` with a +`HydePage::showInSitemap(): bool` method backed by front matter (`sitemap: false`), +with defaults: `false` for redirects and for pages whose output is not `.html`, +`true` otherwise. This fixes the `search.json` leak, prevents the new +sitemap/feed/robots pages from self-listing, and gives users per-page opt-out — +a standalone feature in its own right. + +> **Implementation constraint (from PR 1) — read before writing `showInSitemap()`:** +> the "output is not `.html`" default MUST be derived from the page's *resolved +> output path* (`getOutputPath()`), not merely from the declared extension. The +> resolved path is the canonical answer and also covers generated pages such as +> the RSS feed, whose configurable filename cannot be represented by one fixed +> extension. Keying the default off `getOutputPath()` makes all generated pages +> self-exclude correctly and prevents the `search.json` leak from returning. + +> **Implemented (PR 4):** `HydePage::showInSitemap()` reads the `sitemap` front +> matter key, defaulting to whether the resolved output path (`getOutputPath()`) +> ends in `.html`, per the constraint above. Front matter wins in both directions, +> so `sitemap: true` opts a non-HTML page back in. One refinement to "defaults +> `false` for redirects": `Redirect` overrides `showInSitemap()` to return `false` +> unconditionally, mirroring its `showInNavigation()` and the already-recorded +> release note that redirect routes are intrinsically excluded from navigation and +> sitemaps — a redirect page has no front matter channel in `hyde.redirects`, and +> listing redirects in a sitemap is an SEO anti-pattern, so an opt-in would only +> be a trap. + +> **Reused, not duplicated (PR 7): llms.txt inclusion *is* sitemap inclusion.** No new +> page method and no new front matter key were added. `LlmsTxtGenerator::shouldListPage()` +> is simply `$page->showInSitemap() && $page->getIdentifier() !== '404'`. +> +> This landed in two cuts. The first PR 7 implementation added a mirror +> `HydePage::showInLlmsTxt()` (plus a `Redirect` override, a `BaseHydePageUnitTest` +> contract entry, and six page unit test implementations); that was cut because +> `showInSitemap()` already answers the exact question llms.txt asks — "is this page part +> of the machine-readable index of my site" — and its resolved-output-path default already +> excludes every generated non-HTML page and redirect for free. The interim version kept an +> `llms` front matter key (`matter('llms', $page->showInSitemap())`) to preserve decoupling; +> that was cut too, on the grounds that **front matter is public API we must support +> forever, so it has to earn its place.** The decisive argument is that llms.txt is not a +> control plane: omitting a page from it does not stop any AI service from reading that +> page (only `robots.txt` speaks to crawler access), so `llms: false` could never mean +> "hide this from AI" — it could only mean "curate my index", which is precisely what +> `sitemap: false` already means. A second key with near-identical semantics would mostly +> generate the question "which one do I use?". +> +> The coupling is therefore a feature, not a compromise, and it is the *less* surprising +> default: a user who hides a page from search engines does not expect it advertised to AI +> agents. +> +> The known cost is the converse case: a page kept out of the sitemap for SEO reasons +> (thin content, a duplicate, a paginated archive) that would still be useful to an agent. +> That site is not stuck today — overriding `shouldListPage()` on the generator and +> rebinding it is exactly the D4 tier, and it is a three-line override. The trade is +> deliberate: the edge case pays at the generator tier instead of every site paying for a +> second front matter key. Should the case turn out to be common, reintroducing `llms:` +> front matter (or promoting it to a `showInLlmsTxt()` method) is additive and +> non-breaking — so waiting +> for that evidence costs nothing, while shipping the key speculatively costs us the +> support burden forever. + +### D4: Generators become container-resolved pages; generator actions stay + +Each generated file is registered as an `InMemoryPage` whose compiled contents +resolve its generator **from the container at build time**, e.g. a `sitemap.xml` +page whose compile step returns `app(SitemapGenerator::class)->generate()`. The XML +generator actions (`SitemapGenerator`, `RssFeedGenerator`) are untouched and remain +the default implementations — this is still the `DocumentationSearchIndex` → +`GeneratesDocumentationSearchIndex` split, but the generator is *resolved*, not +`new`'d. + +Resolving through the container is the point: a user can rebind `SitemapGenerator` +(or the page's content closure) to swap the output without replacing the page — a +lighter-weight customization tier than D5's full page override. Contents must be +produced **lazily** (a `compile` macro / closure or a thin subclass `compile()`), +never eager string content, since generation must run at build time against the +final route set. + +Registration happens in `HydeCoreExtension::discoverPages()` behind the existing +`Features::hasSitemap()` / `Features::hasRss()` conditions, replacing the +registrations in `BuildTaskService::registerFrameworkTasks()`. + +**Decided: plain `InMemoryPage`.** D3 already defaults non-HTML pages out of the +sitemap, and the commands only build registered routes. The temporary thin subclasses +were justified by an early command fallback that instantiated fresh pages, but that +fallback was removed in review. With no type-identity consumer remaining, generated +pages are ordinary `InMemoryPage` instances registered with container-resolved +`compile` macros. The D4 swappability tier is preserved and verified by rebind tests. + +### D5: User-defined pages beat generators + +If the page collection already contains a user-defined page with a route key such +as `robots.txt`, the framework does not register its generated page. +This follows the pattern of `discoverDocumentationRootRedirect()`, which skips when +a user-defined route exists. +Users can register an `InMemoryPage` from a service provider or provide a custom page +class through an extension. Combined with D4's container-resolved generators, this +gives a smooth escalation path: +feature default → config tweaks → rebind the generator (or content closure) in the +container → fully custom page in code. + +> **Timing caveat — the skip check is ordering-sensitive.** "Is a `robots.txt` route +> already registered" is evaluated at `discoverPages()` time, so whether a user's page +> wins depends on it being visible at that moment. A page registered via a late +> `booting()` callback may or may not be present depending on boot ordering. The +> `discoverDocumentationRootRedirect()` precedent suggests this is fine, but "fine and +> ordering-dependent" is exactly what passes in our tests and breaks for the one user +> who registers late. PR 5/6 MUST include an end-to-end test asserting that a +> user-registered `robots.txt` (via both a `HydeExtension` page class and a `booting()` +> `addPage()` callback) suppresses the generated one — this is the D5 contract and the +> most likely silent failure for the power-user audience. + +> **Verified for the sitemap (PR 5 part A):** both user paths win, through two +> different mechanisms that the tests pin down end-to-end. `booting()` callbacks run +> before the page collection boots (`BootsHydeKernel::boot()`), so a callback-registered +> `sitemap.xml` page is visible to the core extension's `hasPageWithRouteKey()` skip +> check. A user `HydeExtension` runs *after* the core extension (registration order), +> so the skip check cannot see its pages; instead the user page replaces the generated +> one under the same collection key (`addPage()` keys by source path). Both are +> asserted through the real `build` command output. The robots.txt equivalent remains +> mandatory for PR 6. *(Part B: both paths verified the same way for the feed page.)* +> *(PR 6: both paths verified the same way for the robots.txt page.)* +> *(PR 7: both paths verified the same way for the llms.txt page.)* + +### D6: No built-in `TextPage` or `.txt` autodiscovery + +First-class non-HTML support is about a page's output path and participation in the +route/build/serve lifecycle; it does not require a dedicated source-backed page class +for each file extension. A plain `InMemoryPage` whose identifier includes the desired +extension provides the full lifecycle integration and is a better fit for dynamic +content, while the generated robots and llms pages cover the common cases without source files. + +A core `TextPage` would add only the convenience of autodiscovering `_pages/*.txt`, +while creating pressure for parallel `XmlPage`, `JsonPage`, and similar classes. +Plain-text files also cannot carry front matter, requiring page-type defaults or +special handling for navigation and sitemap behavior. That framework surface is not +justified by the narrow drop-a-file use case. If demand emerges for filesystem-backed +verbatim files, it should be designed as a generic raw/public-file mechanism instead +of one page class per extension. Custom discoverable page classes remain supported as +an extension point. + +## Work breakdown (planned PR sequence, in dependency order) + +### PR 1 — Foundation: page-class output extensions ✅ Implemented + +Goal: any page class can emit a non-`.html` file without overriding `getOutputPath()`. + +- Add `$outputExtension` (default `'.html'`) to `HydePage`; use it in + `outputPath()` (`HydePage.php:211-214`). +- Route keys follow D1; audit `RouteKey` and `Route` for assumptions. +- Override `InMemoryPage::outputPath()` to infer the output suffix from the identifier. +- Keep `DocumentationSearchIndex`'s `.json` extension in its identifier. +- Pure refactor for existing sites: no compiled-output changes. + +Implementation notes (branch `v3/non-html-pages-foundation`): + +- An `outputExtension()` accessor accompanies the property, matching the other + static accessors, and is part of the `BaseHydePageUnitTest` contract. No setter + was added — the existing setters exist for config-driven source customization, + which does not apply here; subclasses redeclare the property. +- Review outcome: the existing `$fileExtension` API was renamed to `$sourceExtension` + (with `fileExtension()`/`setFileExtension()` becoming `sourceExtension()`/ + `setSourceExtension()`) so the source/output pair reads symmetrically — the old + name really meant the source extension, and fixing the vocabulary before later + non-HTML page types (sitemap, RSS, robots, llms) build on it avoids much larger + churn. Clean break, no compatibility aliases: independently redeclared static + properties cannot alias each other without precedence/synchronization hacks. + The mechanical migration is recorded in `HYDEPHP_V3_PLANNING.md` under + "Upgrade script rules" for the release-time Rector script. +- Page-class output extension handling was placed in `RouteKey::fromPage()` (see D1 note) + rather than only in `outputPath()`, so route keys and output paths cannot drift. +- **Revised before PR 8, then finalized in the subsequent design pivot:** the + allowlist-based inference and later instance-level exact-path factory were removed. + A class-level extension approach briefly replaced them, but was also rejected for + in-memory pages due to its boilerplate and poor call-site ergonomics. The final + implementation uses unrestricted `pathinfo` inference (see D2). +- Sitemap / non-HTML detection reads the resolved output path, not just the static + extension declaration, so specialized static path overrides remain supported. + +### PR 2 — Realtime compiler: route-first resolution for non-HTML paths ✅ Implemented + +Goal: `hyde serve` serves any registered route regardless of extension; no +filename special cases. + +- In `Router::shouldProxy()`, replace the `search.json` suffix check with a generic + "is there a registered route for this path?" check (`PageRouter::hasRoute()`), + so pages win over asset proxying. +- Regression tests: versioned-docs dotted paths (`docs/1.x/...`), media assets, + missing-asset 404s, and `search.json` still served. +- `PageRouter::getContentType()` already handles txt/xml/json; extend the map only + if new types come up. + +Implementation notes (branch `v3/non-html-pages-realtime-compiler`): + +- The route lookup needs the booted application, but `shouldProxy()` ran before + booting, so the predicate was dissolved into `Router::handle()` instead of + booting inside it: the `/media/` prefix remains the only boot-free fast path, + and any other asset-like path is proxied only when no registered route matches. + Missing assets fall through to the 404 in `proxyStatic()`, which absorbed the + previous separate missing-asset branch (same response either way). +- Perf consequence: requests for existing static files outside `media/` now boot + the app before being proxied, since routes must be consulted first. Such files + are rare (Hyde assets live under `media/`), and every non-proxied request + already booted. +- Behavior fix beyond the search.json generalization: a static file whose path + shadowed a registered dotted route (like a `_media/9.x` file next to a + `9.x/index` page) was previously served instead of the page; the page now wins. + Conversely, a routeless file like `_media/search.json` requested as + `/search.json` was previously 404'd by the suffix special case and is now + proxied like any other asset. +- `getContentType()` untouched — no new content types came up. +- **Post-implementation review: no changes required.** Route-first resolution is + clean and the shadowing/`search.json` regression tests cover the behavior changes. + Worth adding (if not already present elsewhere) an explicit versioned + `docs/1.x/search.json` serve test alongside the un-versioned one, since the + versioned dotted path is the case most likely to regress silently. + +### PR 3 — `TextPage` autodiscovery ❌ Removed from scope + +The proposed `TextPage` class was evaluated after the non-HTML foundation landed and +removed from the epic per D6. The foundation already makes a `.txt` `InMemoryPage` +first-class, the common robots/llms cases will have generated pages, and advanced +content is often dynamic. Adding a core class solely for `_pages/*.txt` discovery +would introduce extension-specific framework surface without solving the broader +verbatim-file problem. Documentation will instead show the service provider / +`booting()` registration pattern for custom text output. + +### PR 4 — Sitemap inclusion policy ✅ Implemented + +Goal: pages control their own sitemap presence; fix the production `search.json` leak. + +- `HydePage::showInSitemap()` per D3 + `sitemap: false` front matter support. +- **Derive the non-HTML default from `getOutputPath()`, not `outputExtension()`** + (see the D3 implementation constraint) so InMemoryPage-backed generated pages + self-exclude correctly. +- `SitemapGenerator::generate()` filters on it instead of `instanceof Redirect`. +- Changelog note: search indexes no longer appear in sitemaps (bugfix). +- Independent of PRs 1-3; must land before or with PR 5. + +Implementation notes (branch `v3/non-html-pages-sitemap-inclusion-policy`): + +- Implemented exactly per D3 (see the D3 "Implemented" note for the front matter + semantics and the `Redirect` refinement). `showInSitemap()` joined the + `BaseHydePageUnitTest` contract. +- The non-HTML self-exclusion is verified end-to-end: an `InMemoryPage` with a `.txt` + identifier is built by the real `build` command and asserted absent from the built + `sitemap.xml`, guarding the D3 resolved-output-path constraint + against regression by construction rather than only at the unit level. +- Two existing tests asserted the leak as expected behavior and were flipped: + `SitemapServiceTest` now asserts the docs search *page* stays while the search + *index* is excluded, and the `SitemapFeatureTest` expected XML dropped its + `docs/search.json` entry (it also gained a `sitemap: false` page proving the + front matter opt-out through the `build:sitemap` command). +- No UPGRADE.md entry: the fix requires no user action, and nothing realistic + depended on search indexes appearing in sitemaps. + +### PR 5 — Convert sitemap and RSS from build tasks to pages ✅ Implemented + +Goal: `sitemap.xml` and `feed.xml` are routes — served by `hyde serve`, listed in +`route:list`, included in the build manifest, overridable in user land. + +> **Split during implementation:** part A converted the sitemap, part B converted +> the RSS feed the same way. The bullets below describe both; the notes at the end +> of this section record what landed in each part. + +- Register `sitemap.xml` / `feed.xml` as `InMemoryPage`s per D4, with a lazy + `compile` that resolves the generator from the container + (`app(SitemapGenerator::class)->generate()` / `app(RssFeedGenerator::class)->generate()`), + so the implementation is swappable via container rebind. RSS route key comes from + `RssFeedGenerator::getFilename()` (config `hyde.rss.filename`). +- Use plain `InMemoryPage` instances with container-bound `compile` macros (D4); + D3 already handles sitemap self-exclusion. +- **Verify the generators are actually container-resolvable** before advertising the + rebind: no unresolvable constructor dependencies, not `final` (or the swap can't be + bound). D4's whole swappability tier is a lie if `app(SitemapGenerator::class)` + can't be rebound. Add a test that rebinds the generator and asserts the page's + compiled output changes. +- Ensure first-party generated page identifiers include their output extension per D2; + the configurable RSS filename then uses the shared inference without an override. +- Register in `HydeCoreExtension::discoverPages()` behind `Features::hasSitemap()` / + `Features::hasRss()`; remove `GenerateSitemap`/`GenerateRssFeed` from + `BuildTaskService::registerFrameworkTasks()` (evaluate deprecation vs. removal — + v3 allows breaking changes, but third-party code may reference the task classes). +- Rewire `build:sitemap` / `build:rss` commands to build the same registered page + via `StaticPageBuilder::handle(...)`. +- Verify `GlobalMetadataBag` head links and the `hyde.url` requirements still hold + (`Features::hasSitemap()` already requires a site URL). +- Nice side effect: build output shows them under "Dynamic Pages" with the standard + progress display. + +Implementation notes, part A (branch `v3/non-html-pages-convert-sitemap`): + +- A plain `InMemoryPage` with a container-resolved `compile` macro is hidden from + navigation and self-excludes from the sitemap via the D3 non-HTML default. + Registered at the end of `HydeCoreExtension::discoverPages()` behind + `Features::hasSitemap()` with the D5 skip check (see the D5 note for the verified + override ordering semantics). +- `SitemapGenerator` verified container-resolvable and rebindable: not `final`, no + constructor dependencies, and a test rebinds it and asserts the page's compiled + output changes. +- `GenerateSitemap` was removed rather than deprecated: a kept-but-registered task + would generate the sitemap twice, and a kept-but-unregistered task is dead code + that still breaks anyone re-registering it (double generation) — a clean removal + with release-notes guidance to the rebind/override tiers is more honest. Recorded + as a v3 breaking change with the realistic impact being same-basename user-land + task overrides. +- `build:sitemap` builds the registered route's page via `StaticPageBuilder`. The + route-first lookup means a user-defined `sitemap.xml` page wins here too. Skip + exit code changed from 3 (task-runner semantics) to 1. + *(Revised in review, applies to both commands: an initial implementation fell back + to a fresh page instance when the route was not registered, for strict + backwards compatibility with the old tasks — `build:sitemap` generated even with + `hyde.generate_sitemap` disabled, and `build:rss` had no guard at all, emitting an + empty feed with zero posts. That silently overriding the user's own configuration + or producing a useless file is a trap, not a feature: the commands now fail with + a generic "feature is not enabled" error when the route is not registered. Because + the lookup is route-first rather than feature-flag-first, a user-defined page under + the route key is still built even when the feature conditions are unmet — the only + behavior the fallback enabled that anyone could plausibly want, preserved without + it.)* + *(Revised again in a second review pass: the first revision reported the specific + unmet condition — no base URL, disabled in config, no posts, missing SimpleXML — + by re-checking the `Features::hasSitemap()`/`hasRss()` conditions in the command. + That mirror was dropped for a single static message: its final SimpleXML branch + attributed the failure by elimination rather than observation, so any drift in the + mirrored conditions or an extension removing the page would blame SimpleXML on a + system where it is fine, and these commands fail too rarely to justify carrying + duplicated feature logic for a nicer message.)* +- `GlobalMetadataBag` verified: the sitemap head link is emitted under the same + `Features::hasSitemap()` condition that registers the page — no drift possible. +- Realtime compiler needed no changes (PR 2's route-first resolution); a serve test + asserts `sitemap.xml` returns the generated XML with `application/xml`. +- For part B, `BuildTaskServiceUnitTest`'s framework-task fixtures were + migrated from `GenerateSitemap` to `GenerateBuildManifest` (not `GenerateRssFeed`) + so removing the RSS task would not churn them again. The RSS route key comes from + `RssFeedGenerator::getFilename()`. + +Implementation notes, part B (branch `v3/non-html-pages-convert-rss-feed`): + +- The RSS feed mirrors the sitemap throughout: a plain `InMemoryPage` with a + container-resolved `compile` macro (rebind verified by test), registered behind + `Features::hasRss()` with the D5 skip check, hidden from navigation, D3-excluded + from the sitemap, and both user override paths verified end-to-end. +- One divergence: the route key comes from `RssFeedGenerator::getFilename()` + (config `hyde.rss.filename`). The shared `InMemoryPage` inference keeps configured + filenames with extensions verbatim and gives extensionless identifiers HTML output. +- `build:rss` builds the registered route's page like `build:sitemap`, and fails + with the generic "feature is not enabled" error when the route is not registered + (see the revised-in-review notes in the part A section — the old task's no-guard + semantics, where an explicit invocation emitted an empty feed with zero posts, + were deliberately not preserved). +- `BuildTaskService` no longer registers any feature-gated tasks; the `Features` + facade import went with the last one. The remaining framework tasks + (clean/transfer/manifest) are all config-gated. + +### PR 6 — Generated `robots.txt` ✅ Implemented + +Goal: sensible robots.txt out of the box, zero config. + +- `robots.txt` registered as an `InMemoryPage` per D4; default output + `User-agent: * / Allow: /` plus a `Sitemap:` line when `Features::hasSitemap()`. +- Config (e.g. `hyde.robots`) for disallow rules / disabling; user-defined page + precedence per D5 (an explicitly registered `robots.txt` page wins). +- Depends on PRs 1, 2, 5 patterns. + +Implementation notes (branch `v3/non-html-pages-robots`): + +- The generated robots.txt is a plain `InMemoryPage` with a container-resolved + `compile` macro (rebind verified by test), + registered in `HydeCoreExtension::discoverPages()` with the D5 skip check, hidden + from navigation, D3-excluded from the sitemap via the non-HTML default, and both + user override paths (booting callback and extension) verified end-to-end through + the real `build` command per the D5 mandate. +- `RobotsTxtGenerator` lives in the + `Hyde\Framework\Features\TextGenerators` namespace mirroring `XmlGenerators`. + PR 7's generator follows it as `LlmsTxtGenerator`, superseding the epic's earlier + `GeneratesLlmsTxt` working name. +- Feature gate: `Features::hasRobotsTxt()` reads only `hyde.robots.enabled` + (default `true`). Unlike `hasSitemap()`/`hasRss()` there is no site URL + requirement — robots.txt directives are relative, and the one absolute URL (the + `Sitemap:` line) is gated separately inside the generator by `Features::hasSitemap()`, + the same condition that registers the sitemap page and emits its head link + (the `GlobalMetadataBag` no-drift precedent). Consequence: the page registers + unconditionally on default config, so it appears in zero-config builds — several + existing tests asserting exact collections gained a `hyde.robots.enabled => false` + in their setup, alongside their existing sitemap/RSS switches. +- Generator output: `User-agent: *`, then verbatim `Disallow:` lines from the + `hyde.robots.disallow` config array, or `Allow: /` when there are none. The config entries are + *rule values*, not filesystem paths, and are deliberately not normalized + (no leading-slash fixup or empty-string removal) so valid values like wildcard patterns are supported. + Non-string values (like integers or floats) are safely cast to strings during generation. +- Its `InMemoryPage` identifier includes the `.txt` output extension per D2. +- No `build:robots` command: the sitemap/RSS commands exist only as carry-overs of + the removed post-build tasks; robots.txt never had one, and the standard build + and realtime compiler (serve test asserts `text/plain`) cover the lifecycle. + +### PR 7 — Generated `llms.txt` ✅ Implemented + +Goal: best-in-class llms.txt support — no other SSG generates this well out of the box. + +- `GeneratesLlmsTxt` action per the llms.txt spec: site name as H1, `hyde.description` + ("about" blockquote), sections of route links using page titles and the new + documentation page abstracts (#2523) as link descriptions. +- `llms.txt` registered as an `InMemoryPage` wired like robots.txt (feature-gated, + config for section grouping/exclusions, container-resolved generator, user-defined + page precedence). +- **Make the default state (on vs. off) a deliberate decision with a clean opt-out**, + not an afterthought. Some of our audience is privacy/OPSEC-minded and will have + opinions about surfacing content to AI crawlers; the feature flag (and its default) + should be a first-class, documented choice, mirroring how `robots.txt` disabling + works, rather than something a user has to discover. +- Consider `llms-full.txt` (full page contents) as a follow-up, not in scope. + +Implementation notes (branch `v3/non-html-pages-llms-txt`): + +- `LlmsTxtGenerator` lands in `Hyde\Framework\Features\TextGenerators` + next to the robots.txt generator (superseding the `GeneratesLlmsTxt` working name, + as PR 6 anticipated), and its plain `InMemoryPage` uses a container-resolved + `compile` macro (rebind verified by test), + registered in `HydeCoreExtension::discoverPages()` with the D5 skip check, hidden + from navigation, D3-excluded from the sitemap, and both user override paths verified + end-to-end through the real `build` command. No `build:llms` command, for the same + reason PR 6 added no `build:robots`. +- **Reversed to default off (post-implementation review).** `Features::hasLlmsTxt()` + reads `hyde.llms.enabled` (default `false`). The PR originally shipped default `true`, + reasoning that llms.txt lists only already-published pages and surfaces nothing the + sitemap does not, and that sitemap/RSS/robots are all on by default — so an opt-*in* + would bury the feature for the majority to protect a minority that a `false` in the + config serves just as well. That argument proves too little: sitemap.xml, RSS, and + robots.txt are each an established web convention that fixes a real compatibility or + crawler-control problem, so their on-by-default posture costs a site nothing it wasn't + already exposing through routing and search-engine norms. llms.txt has no such + precedent — it is, by the epic's and the config stub's own description, "an emerging + proposal" whose format may still change in a minor or patch release, and publishing it + is explicitly "a deliberate invitation" for AI services to read the site, not a + neutral discovery mechanism like a sitemap. "Bury the feature" describes a marketing + cost, not a reason to default an unstable, AI-specific invitation to on. The opt-in is + called out in the config stub, the release notes, and its own UPGRADE.md step, mirroring + how the (default-on) `hyde.robots.enabled` opt-out is documented, so the choice is a + first-class one either way rather than something a user has to discover. +- **Emerging-standard caveat, recorded deliberately.** llms.txt is a proposal, not a + ratified standard, so the generated *format* carries no backwards-compatibility + promise: we expect to change it in minor and patch releases as the spec moves. This + is stated in the config stub, the generator docblock, the release notes, and + UPGRADE.md (which points users who need a frozen format at the user-defined page + tier). Shipping an imperfect llms.txt is judged better than shipping none. +- **Deviation — site URL is required** (unlike robots.txt, which deliberately is not + gated on one). `hasLlmsTxt()` requires `Hyde::hasSiteUrl()`, putting llms.txt in the + sitemap/RSS camp: the file's entire payload is links, and relative links in a file + fetched by an arbitrary agent are a degraded product. Consequence: zero-config sites + without a base URL get no llms.txt, exactly as they get no sitemap. Under + `hyde serve` the realtime compiler overrides the site URL, so the page *is* served + locally (asserted by a `text/plain` serve test). +- **Deviation — there is no section configuration at all.** The epic (and the research + doc) asked for "config for section grouping/exclusions", sketched as route-key globs. + An initial implementation shipped a `hyde.llms.sections` map of page class to section + heading; it was cut in review. Hyde already *knows* its page types, so grouping needs + no user input to be correct, and the config bought only heading renames and bulk + exclusion — rare needs, paid for by every user in config-file surface (a five-entry + array, entry validation, an exception path, and a comment explaining that omitting a + class silently drops those pages, which is a trap the framework did not previously + have). The section map now lives as a `protected sections()` method on the generator: + page classes are matched with `instanceof` in declaration order — the same semantics + `PageCollection::getPages()` and `RouteCollection::getRoutes()` use — so a user's + `GuidePage extends MarkdownPage` lands in the `Pages` section, while every + `InMemoryPage` descendant (the generated pages, redirects, and the documentation + search page) is absent from the map and therefore never listed. Users who genuinely + need different sections have the D4 tier already advertised for exactly this: + override the generator and rebind it in the container. The config surface is now two + keys, `enabled` and `description`, matching the size of the `rss` and `robots` blocks. + A method rather than a constant because overriding the generator *is* the advertised + customization tier, and a method lets an override compute its sections from config, + installed extensions, or runtime state, which a constant expression cannot. + *Design rule this records: a configuration option must be justified against the + container-rebind tier that already exists, not merely be useful in principle.* +- **Page ordering is route order, deliberately.** Sections are emitted in the order + `sections()` declares them, and pages within a section in route-collection order — + the same order the sitemap lists and the build compiles them in. This is a chosen and + tested contract, not an accident of discovery: `FileFinder` sorts its results by path, + so the order is deterministic and platform-independent, and because Hyde strips + numeric filename prefixes from route keys while still discovering by path, a + `01-installation.md` / `02-usage.md` docs set lands in the file in its intended reading + order with clean URLs. Navigation priority was considered as the ordering key and + rejected: it would couple the file to navigation config, and it is meaningless for the + blog posts and nav-hidden pages that make up much of the listing. +- **Deviation — `hyde.description` does not exist.** The epic assumed a site-level + description config key; there is none (only `hyde.rss.description`). Added + `hyde.llms.description`, mirroring the RSS key rather than inventing a global one, + which would have pulled in the `hyde.meta` description tag and page metadata + generation — a cross-cutting change that does not belong in this PR. It is nullable, + and the summary blockquote is omitted when unset (only the H1 is required by the + spec). + *Follow-up recorded (out of scope): site identity metadata is fragmenting.* The site + name, base URL, language, and now two separate descriptions (`hyde.rss.description` + and `hyde.llms.description`) all describe the same site identity from different config + keys. A coherent site-metadata object — name, canonical URL, description, language, + author/organization — with feature-specific overrides would consolidate them. That is + its own architectural change, not scope for this epic; this PR deliberately mirrored + the existing RSS key rather than pre-empting that design. +- **Markdown-significant characters in titles are escaped.** A page titled + `Arrays [Advanced]` would otherwise emit `- [Arrays [Advanced]](url)`, a malformed + link. `escapeLinkLabel()` escapes `[`, `]`, and `\` in the label. Link *descriptions* + are not escaped: they are prose trailing the link rather than delimiter-sensitive + syntax. +- **Link descriptions:** the `abstract` front matter added by #2523, falling back to + `description`. #2523 only added `abstract` to the docs *content* — there is no + framework schema support for it, and consistent with PR 4 (which did not add + `sitemap` to `PageSchema::PAGE_SCHEMA` either), `abstract` was not added to the + schema; it is documented on the generator that reads it. Note that this PR adds **no + new front matter keys at all** — it only consumes `abstract`, `description`, and + `sitemap`, which all already existed. Whitespace + in descriptions is collapsed to a single line, since a multi-line YAML block scalar + would otherwise emit a broken list item — this is *not* a "verbatim string" case like + the robots.txt disallow rules, where PR 6 correctly refused to normalize, because + here the value is prose embedded in a line-oriented format rather than an exact-match + rule value. With the sections config gone, no llms config entry needs validation: both + remaining keys are scalars read through the typed `Config` accessors. +- **Deviation — 404 pages are never listed.** An error page is not content, and every + real-world llms.txt excludes it. Filtered in the generator by identifier, mirroring + the `$identifier === '404'` special case `SitemapGenerator` already carries. This is a + generator-level curation concern rather than a page-level default (the sitemap + precedent likewise keeps its 404 handling in the generator), and it is the reason the + sitemap-derived inclusion rule is not a bare alias for `showInSitemap()`. +- Everything else the epic left implicit held: the `llms.txt` page identifier declares + its `.txt` output extension, and the generated page self-excludes from its own listing (and the sitemap) + through the D3 resolved-output-path default. + +> **Scope correction (post-implementation review).** The first cut of this PR was +> overbuilt for the value delivered, and three pieces were cut back before merge: the +> `hyde.llms.sections` config (see the sections deviation above), the +> `HydePage::showInLlmsTxt()` page method, and the `llms` front matter key that briefly +> replaced it (both in the D3 "Reused, not duplicated" note). Between them they added a +> public method to every page class, an entry in the `BaseHydePageUnitTest` contract with +> six implementations, a front matter key we would have to support for the life of v3, a +> config array with its own validation and exception path, and a config comment long +> enough to advertise that the option was not simple. All of it served needs that the +> existing `sitemap: false` front matter and the D4 container-rebind tier already served. +> The feature's user-facing capability is materially unchanged; only the surface shrank. +> The final shape adds **no new front matter, no page-model API, and two scalar config +> keys.** +> +> Three rules worth carrying into PR 8 and any future generated-page work: +> 1. **The D4 rebind tier is the default answer for customization.** A new config key or +> page-model method has to beat it, not merely be useful. +> 2. **Front matter is forever.** A key we introduce is public API we must support and +> document for the life of the major version, so a speculative one is a real liability. +> Adding a key later is additive and non-breaking, which makes "wait for the evidence" +> the cheap option and "ship it just in case" the expensive one. +> 3. **A long explanatory comment in a config stub is a design smell,** not diligence. If +> an option needs paragraphs to explain, the option is usually the problem. + +### PR 8 — Documentation & release notes ✅ Implemented + +- Document in-code virtual pages, `sitemap: false` front matter, robots/llms config, + the container-rebind customization tier for generated pages, and the "user-defined + page beats generator" rule. +- Update `HYDEPHP_V3_PLANNING.md` release notes: new features (robots.txt, llms.txt, + serve support for sitemap/RSS), breaking changes (build task classes + removed/relocated, search.json removed from sitemaps). + +Implementation notes (branch `v3/non-html-pages-documentation`): + +- The public InMemoryPage and customization guides document exact non-HTML paths, + navigation and sitemap defaults, all four generated pages, their feature conditions, + robots/llms configuration, generator rebinding, and user-defined route precedence. +- The Build Tasks guide no longer describes sitemap/RSS generation as post-build tasks. + The console command guide now records that `build:sitemap` and `build:rss` compile + registered pages and fail when no matching page is registered. +- The audit found behavior worth documenting beyond the original checklist: llms.txt + reuses sitemap inclusion, its format has no minor/patch compatibility promise while + the proposal evolves, robots.txt controls crawler access while llms.txt does not, + generated pages are hidden from automatic navigation, and sitemap/RSS registration + depends on SimpleXML in addition to their documented content prerequisites. +- `HYDEPHP_V3_PLANNING.md` and `UPGRADE.md` already contain the feature, breaking-change, + and migration entries added with PRs 1–7; this PR verified them rather than duplicating + those notes. + +## Out of scope (noted for later) + +- Filesystem autodiscovery for verbatim or Blade-processed text files + (`robots.txt`, `robots.blade.txt`) — wait for demand; the in-code `InMemoryPage` + path covers custom and dynamic cases. If added later, prefer a generic mechanism + for raw/public files over extension-specific page classes. +- `llms-full.txt` / per-page markdown exports. +- Generalizing `GenerateBuildManifest` or search-index generation commands beyond + what PR 5 requires. +- Reconsidering the page-type `Feature` enum cases (`HtmlPages`, `BladePages`, etc.) + altogether — arguably redundant since not creating source files has the same + effect. Worth a separate v3 discussion; this epic simply doesn't add new ones. diff --git a/HYDEPHP_V3_PLANNING.md b/HYDEPHP_V3_PLANNING.md index a91417df524..627131b9d85 100644 --- a/HYDEPHP_V3_PLANNING.md +++ b/HYDEPHP_V3_PLANNING.md @@ -24,6 +24,14 @@ Having this document in code lets us know the devlopment state at any given poin - Redirects can now be declared as source and destination path pairs in the `hyde.redirects` configuration array. Hyde registers them with the kernel, includes them in `route:list`, and generates them through the normal site build. - Added Blade Blocks for rendering Blade and Blade components from fenced code blocks in Markdown pages. The supported directives are `blade render` and `blade component(name)`, and the feature is controlled by `markdown.enable_blade`. ([#2504](https://github.com/hydephp/develop/pull/2504)) - Added built-in terminal code blocks using the `terminal` fence language. Command prompts are styled for selection-free copying, and `terminal xml` supports four Symfony-style Console formatter tags. ([#2188](https://github.com/hydephp/develop/issues/2188), [#2485](https://github.com/hydephp/develop/issues/2485)) +- `InMemoryPage` now infers its output format from the identifier: identifiers with an extension keep it, while identifiers without one compile to `.html`. Only the HTML extension is implicit in route keys, so non-HTML paths such as `robots.txt` and `docs/search.json` remain route keys as-is. +- Pages with non-HTML output paths are excluded from automatic navigation by default. Set `navigation.visible: true` or `navigation.hidden: false` to opt a non-HTML page into the generated navigation. +- Pages can now control their own sitemap inclusion. Set `sitemap: false` in a page's front matter to exclude it from the generated `sitemap.xml`, or override the new `HydePage::showInSitemap()` method in custom page classes. Pages compiled to non-HTML output files (like `robots.txt`) are excluded by default, and `sitemap: true` front matter opts such a page back in. +- The sitemap and RSS feed are now first-class pages instead of post-build side effects: when the respective feature is enabled, `sitemap.xml` and the RSS feed (`feed.xml`, or the configured `hyde.rss.filename`) are registered as routes, so they are served by `hyde serve`, listed in `route:list`, included in the build manifest, and compiled through the standard site build. The output can be customized by rebinding the `SitemapGenerator` or `RssFeedGenerator` class in the service container, and registering a user-defined page with the same route key (from a service provider, booting callback, or extension) replaces the generated page entirely. +- Hyde now generates a `robots.txt` file for the site out of the box. The default output allows all crawlers, and links to the sitemap when that feature is enabled. Rule values listed in the new `hyde.robots.disallow` configuration array are written verbatim as `Disallow` rules (so wildcard patterns are supported), and the file can be disabled entirely with `hyde.robots.enabled`. The page is wired like the sitemap and RSS feed: it is a real route (served by `hyde serve`, listed in `route:list`, included in the build manifest), the output can be customized by rebinding the `RobotsTxtGenerator` class in the service container, and a user-defined `robots.txt` page replaces the generated one entirely. +- Hyde can now generate an [`llms.txt`](https://llmstxt.org/) file for the site, opt-in, so that AI services and agents can discover your content without crawling your rendered HTML. Set `hyde.llms.enabled` to `true` to publish it. The file uses the site name as its heading, the optional `hyde.llms.description` as its summary blockquote, and lists your pages as Markdown links, grouped into a section for each page type (Pages, Documentation, and Blog Posts). Each link is described by the page's `abstract` front matter, falling back to its `description`, and pages are listed in the same order the sitemap lists them in, so numerically prefixed source files keep their intended reading order. A page is listed when it is included in the sitemap, as both files are machine-readable indexes of your published pages, so `sitemap: false` front matter leaves a page out of both and no new front matter key is introduced. The file indexes only material you already publish and grants no access to anything private, but publishing it is still a deliberate invitation for AI services to read your site, so unlike the sitemap, RSS feed, and robots.txt, it defaults to off. It requires a site base URL since it needs absolute links. The page is wired like the sitemap, RSS feed, and robots.txt: once enabled, it is a real route (served by `hyde serve`, listed in `route:list`, included in the build manifest), the output can be customized by rebinding the `LlmsTxtGenerator` class in the service container, and a user-defined `llms.txt` page replaces the generated one entirely. + + Please note that llms.txt is an emerging standard which is still subject to change, and we are unable to make a backwards compatibility promise while implementing against a moving specification. We expect to change the format of the generated file in minor and patch releases as the standard evolves. ### Feature Changes @@ -34,7 +42,9 @@ Having this document in code lets us know the devlopment state at any given poin ### Minor Changes and Cleanup +- Fixed documentation search index files leaking into the generated sitemap: `search.json` (and any other page compiled to a non-HTML output file) no longer appears in `sitemap.xml`. The sitemap generator now asks each page through `HydePage::showInSitemap()` instead of only filtering out redirect pages. - The `Redirect` page class constructor now accepts an optional `$matter` parameter, used by the framework to hide the generated documentation root redirect from navigation menus. Existing usages are unaffected. +- The realtime compiler now resolves registered page routes before proxying static assets, replacing the hardcoded `search.json` exemption, so `hyde serve` serves any registered route regardless of its output extension. Registered pages now always win over a static file at the same path; the previous behavior of serving such a shadowing file only affected the dev server and no real setups are expected to be affected. - Removed the legacy `checkForDeprecatedRunMixCommandUsage` check and the placeholder `--run-dev`/`--run-prod` options from the `build` command, which were kept in v2 only to surface a helpful error message. ([#2461](https://github.com/hydephp/develop/pull/2461)) - Removed the deprecated `hyde.server.dashboard` boolean config check from `DashboardController::enabled()`, which was kept in v2 for backwards compatibility but had since then been replaced with `hyde.server.dashboard.enabled`. ([#2461](https://github.com/hydephp/develop/pull/2462)) @@ -42,6 +52,9 @@ Having this document in code lets us know the devlopment state at any given poin ### Breaking Changes +- Renamed the static page class property `$fileExtension` to `$sourceExtension`, and the `fileExtension()` and `setFileExtension()` methods to `sourceExtension()` and `setSourceExtension()`, making it explicit that these APIs describe source files. Custom page classes and code calling these APIs need the mechanical rename, which the planned automated upgrade script will handle (see the upgrade script rules section at the end of this document). +- Removed the `GenerateSitemap` post-build task, as the sitemap is now generated through the page and route system. Sites that just enable or disable the sitemap through configuration are unaffected. Code referencing the task class — like a user-land `GenerateSitemap` build task relying on the same-basename override mechanism to replace the framework task — should register a custom `sitemap.xml` page or rebind `SitemapGenerator` in the container instead. The `build:sitemap` command now compiles the registered page, and fails with an error (exit code 1 instead of 3) when the sitemap cannot be generated — because no base URL is configured or it is disabled in the configuration — instead of generating it anyway in the latter case. +- Removed the `GenerateRssFeed` post-build task, as the RSS feed is now generated through the page and route system. Sites that just enable or disable the feed through configuration are unaffected. Code referencing the task class — like a user-land `GenerateRssFeed` build task relying on the same-basename override mechanism to replace the framework task — should register a custom page with the configured feed route key or rebind `RssFeedGenerator` in the container instead. The `build:rss` command now compiles the registered page, and fails with an error when the feed cannot be generated (no base URL, disabled in the configuration, or no Markdown posts), instead of silently generating an empty feed. A user-defined page registered under the feed route key is still built even when the feature conditions are not met. - Removed `Redirect::create()`, `Redirect::store()`, and the `Redirect` constructor's `showText` argument. Redirects must now be declared in `hyde.redirects`, keeping all generated output inside the kernel-owned build graph. Redirect routes are intrinsically excluded from navigation menus and sitemaps, and always include an accessible fallback link. - Removed the `InMemoryPage` instance macro API. Dynamic contents should now be supplied with a closure, while custom methods and other behavior belong in an `InMemoryPage` subclass. - Removed `InMemoryPage` content-source precedence. Calls that previously supplied both `contents` and `view` must retain only the intended source; positional view calls that used an empty-string contents placeholder must use `null` instead. @@ -58,6 +71,9 @@ Please fill in UPGRADE.md as you make changes. - Move any calls to `Redirect::create()` or `Redirect::store()` into the `redirects` array in `config/hyde.php`, using the old path as the key and the destination as the value. - Move `InMemoryPage` `compile` macro callbacks into the contents argument, and replace other instance macros with methods on an `InMemoryPage` subclass. - Update `InMemoryPage` calls to supply only `contents` or `view`. Replace an empty-string positional contents placeholder with `null`, or use the named `view` argument. +- Add `navigation.visible: true` or `navigation.hidden: false` to non-HTML pages that should remain in automatic navigation. +- Rename `$fileExtension` to `$sourceExtension` in custom page classes, and update any calls to `fileExtension()` or `setFileExtension()` to `sourceExtension()` and `setSourceExtension()`. +- If you referenced the removed `GenerateSitemap` or `GenerateRssFeed` build task classes (for example to override one with a same-basename user-land task), customize the output by rebinding `SitemapGenerator` or `RssFeedGenerator` in the service container or by registering your own page with the same route key instead. ## `InMemoryPage` content-source motivation @@ -73,3 +89,29 @@ mutually exclusive alternatives and ambiguous construction fails immediately. Both arguments use `null` as the omission sentinel so an explicitly empty literal remains distinguishable from an omitted contents argument. Their names and order remain unchanged to keep the upgrade mechanical: existing named calls are unaffected, while positional view calls replace their old `''` placeholder with `null`. + +--- + +## Upgrade script rules + +We will provide an automated upgrade script (likely Rector-based) when we finalize the release. +Until then, this section collects the rules that script needs to implement, so we don't lose +track of them. Add an entry here whenever a change requires mechanical migration of user code. + +- Rename the static page class property `$fileExtension` to `$sourceExtension`, and the + `fileExtension()` and `setFileExtension()` methods to `sourceExtension()` and + `setSourceExtension()`. This covers property declarations in page classes + (`public static string $fileExtension = '.md';`), direct static property access + (`MarkdownPage::$fileExtension`, `$pageClass::$fileExtension`), static method calls + (`MarkdownPage::fileExtension()`), and method declarations that override these methods + in page classes (`public static function fileExtension(): string`) — the methods are + public and non-final, and an un-renamed override would silently stop being called once + the framework calls `sourceExtension()`. The rule must be scoped to + `Hyde\Pages\Concerns\HydePage` subclasses (or known Hyde symbols) — it must not rename + unrelated properties or methods that happen to share the name. +- The rename must also cover named arguments: `Page::setFileExtension(fileExtension: '.md')` + becomes `Page::setSourceExtension(sourceExtension: '.md')`, since the parameter was renamed + along with the method. Include a dedicated Rector fixture for this case. +- Dynamic references cannot be migrated automatically and should be called out as manual + upgrade cases: variable method/property names (`$method = 'fileExtension'; + $pageClass::$method()`), reflection, and string-based access. diff --git a/UPGRADE.md b/UPGRADE.md index 656186d9920..906210e8975 100644 --- a/UPGRADE.md +++ b/UPGRADE.md @@ -2,7 +2,18 @@ ## Overview -// +HydePHP v3 infers an `InMemoryPage` output format from its identifier. Identifiers that already have an extension keep +it, while identifiers without an extension compile to `.html`: + +```php +use Hyde\Pages\InMemoryPage; + +InMemoryPage::make('about', contents: $html); +// _site/about.html + +InMemoryPage::make('robots.txt', contents: $text); +// _site/robots.txt +``` ## Before You Begin @@ -192,8 +203,7 @@ $page->macro( ```php $page = new InMemoryPage( 'sitemap.xml', - ['navigation' => ['hidden' => true]], - fn (): string => app(SitemapGenerator::class)->generate()->getXml(), + contents: fn (): string => app(SitemapGenerator::class)->generate()->getXml(), ); ``` @@ -302,6 +312,134 @@ new InMemoryPage('example', view: ''); new InMemoryPage('example', view: null); ``` +### Review Non-HTML Navigation Visibility + +Pages whose resolved output path does not end in `.html` are no longer included in automatic navigation by default. +This applies both to identifier-based `InMemoryPage` instances such as `feed.xml` and to custom page classes with a +non-HTML `$outputExtension`. Machine-readable resources generally need no migration, and any +`navigation.hidden: true` matter used only for this purpose can be removed. + +If a non-HTML page is intentionally linked from the generated navigation, opt it in explicitly: + +```php +new InMemoryPage( + 'downloads/catalog.pdf', + matter: ['navigation' => ['visible' => true]], + contents: $catalog, +); +``` + +The equivalent `navigation.hidden: false` setting is also supported. Explicit visibility overrides only the new +non-HTML default; existing exclusions for posts, configured route keys, and hidden subdirectories still apply. + +## Step 6: Review Sitemap and RSS Feed Customizations + +The sitemap and RSS feed are now generated as regular pages instead of by post-build tasks, so `sitemap.xml` and +the RSS feed (`feed.xml`, or your configured `hyde.rss.filename`) are served by `php hyde serve`, listed in +`route:list`, and included in the build manifest. Sites that just enable or disable these features through +`hyde.generate_sitemap`, `hyde.rss`, and `hyde.url` need no changes. + +The `GenerateSitemap` and `GenerateRssFeed` post-build task classes have been removed. If you overrode one with a +same-basename build task, or referenced the classes directly, customize the output through one of the replacement +tiers instead: + +- Rebind the generator in the service container to change the output while keeping the page registration: + +```php +use Hyde\Framework\Features\XmlGenerators\SitemapGenerator; + +app()->bind(SitemapGenerator::class, MyCustomSitemapGenerator::class); +``` + +The same works for `RssFeedGenerator`. + +- Or register your own page with the same route key (`sitemap.xml`, or the configured feed filename) from a + service provider, booting callback, or extension, which replaces the generated page entirely: + +```php +use Hyde\Hyde; +use Hyde\Pages\InMemoryPage; + +Hyde::kernel()->booting(function ($kernel) use ($myXml): void { + $kernel->pages()->addPage(InMemoryPage::make('sitemap.xml', contents: $myXml)); +}); +``` + +The `build:sitemap` and `build:rss` commands still work and now compile the registered pages. When the output +cannot be generated (no base URL, disabled in the configuration, or — for the feed — no Markdown posts), they +fail with an error instead of generating an empty or unwanted file. `build:sitemap` reports this +failure with exit code 1 instead of 3. If you registered your own page under the route key, the commands build +it regardless of these conditions. + +## Step 7: Review the New Generated robots.txt + +Hyde now generates a `robots.txt` file by default, allowing all crawlers and linking to the sitemap when that +feature is enabled. Most sites want this and need no changes. If you already publish a `robots.txt` through your +own tooling — a deploy step or web server configuration that would conflict with the generated file — either +disable the feature with `hyde.robots.enabled => false`, or move the contents into Hyde by registering your own +`robots.txt` page (which replaces the generated one, using the same registration pattern as the sitemap example +above). Crawl rules can be added to the `hyde.robots.disallow` configuration array without any custom code. + +## Step 8: Optionally Publish the New llms.txt + +Hyde can now generate an [`llms.txt`](https://llmstxt.org/) file, indexing your site's content for AI services +and agents. Unlike the sitemap, RSS feed, and robots.txt, this is opt-in — set `hyde.llms.enabled` to `true` to +publish it. Publishing it is a deliberate invitation for AI services to read your site, so no action is needed +if you'd rather not extend that. Note that leaving pages out of this file does not stop AI crawlers from reading +them either way; crawler access is governed by your `robots.txt`, not by llms.txt. + +If you enable it, it needs no further configuration. It requires a site base URL, since the file links to your +pages with absolute URLs. Pages are grouped into a section per page type and listed in the same order as your +sitemap, and each link is described by the page's `abstract` front matter, falling back to its `description`, +so filling those in improves the file. A page is listed when it is included in the sitemap — the file indexes +only material you already publish, listing nothing your sitemap does not and granting no access to anything +private — so anything already carrying `sitemap: false` stays out of this file too, and there is no separate +front matter key to learn. As with the sitemap and robots.txt, you can replace the file wholesale by registering +your own `llms.txt` page, or adjust the sections and output by extending the `LlmsTxtGenerator` class and +rebinding it in the service container. + +Be aware that llms.txt is an emerging standard which is still subject to change. We cannot make a backwards +compatibility promise for the generated output while the specification is still moving, and we expect to change +the file format in minor and patch releases as the standard evolves. If you depend on the exact output, pin the +format by registering your own page. + +## Step 9: Rename Page File Extension References + +The static page class property `$fileExtension` has been renamed to `$sourceExtension`, along with the +`fileExtension()` and `setFileExtension()` methods, which are now `sourceExtension()` and `setSourceExtension()`. +The new name makes it explicit that these APIs describe the extension of source files. + +This only affects projects with custom page classes or code calling these APIs. Update property declarations, +call sites, and any methods that override `fileExtension()` or `setFileExtension()` — the methods are public +and non-final, and an un-renamed override silently stops being called now that the framework calls +`sourceExtension()`: + +**Before:** + +```php +class CustomPage extends HydePage +{ + public static string $fileExtension = '.md'; +} + +$extension = MarkdownPage::fileExtension(); +``` + +**After:** + +```php +class CustomPage extends HydePage +{ + public static string $sourceExtension = '.md'; +} + +$extension = MarkdownPage::sourceExtension(); +``` + +The automated upgrade script will handle this rename for ordinary property declarations, property accesses, +method calls, and overridden method declarations. Dynamic references — variable method or property names, +reflection, and string-based access — must be updated manually. + ## Migration Checklist Use this checklist to track your upgrade progress: @@ -311,6 +449,11 @@ Use this checklist to track your upgrade progress: - [ ] Moved calls to `Redirect::create()` or `Redirect::store()` into the `hyde.redirects` configuration array - [ ] Moved `InMemoryPage` `compile` macro callbacks into the contents argument and replaced other macros with subclass methods - [ ] Updated `InMemoryPage` calls to supply only one of `contents` and `view` +- [ ] Explicitly opted in any non-HTML pages that should remain in automatic navigation +- [ ] Replaced any references to the removed `GenerateSitemap` and `GenerateRssFeed` build tasks with a generator container rebind or a user-defined page +- [ ] Confirmed the new generated `robots.txt` does not conflict with an existing one, or disabled it with `hyde.robots.enabled` +- [ ] Decided whether to opt in to the new generated `llms.txt` for AI services with `hyde.llms.enabled` +- [ ] Renamed `$fileExtension`, `fileExtension()`, and `setFileExtension()` to `$sourceExtension`, `sourceExtension()`, and `setSourceExtension()` in custom page classes and call sites ## Troubleshooting diff --git a/config/hyde.php b/config/hyde.php index 0214fd7d1d8..2c0f6380926 100644 --- a/config/hyde.php +++ b/config/hyde.php @@ -134,6 +134,57 @@ 'description' => env('SITE_NAME', 'HydePHP').' RSS Feed', ], + /* + |-------------------------------------------------------------------------- + | Robots.txt Generation + |-------------------------------------------------------------------------- + | + | When enabled, a robots.txt file allowing all crawlers will be generated + | when you compile your static site. A link to your sitemap is included + | when the sitemap feature is enabled. + | + | Values added to the disallow array are written verbatim as Disallow + | rule values for all crawlers, so wildcard patterns are supported. + | + */ + + 'robots' => [ + // Should the robots.txt file be generated? + 'enabled' => true, + + // Disallow rule values asking crawlers not to access matching paths. + 'disallow' => [ + // '/private', + // '/*.pdf$', + ], + ], + + /* + |-------------------------------------------------------------------------- + | Llms.txt Generation + |-------------------------------------------------------------------------- + | + | When enabled, an llms.txt file listing your pages will be generated when + | you compile your static site, helping AI services and agents find your + | content. Set this to true if you would like to publish one. + | + | This feature requires that a site base URL has been set. + | + | Unlike robots.txt, llms.txt is not an established web convention: it's + | an emerging proposal that may still change in minor and patch releases, + | and publishing it is a deliberate invitation for AI services to read + | your site, so it defaults to off until you decide you want that. + | + */ + + 'llms' => [ + // Should the llms.txt file be generated? + 'enabled' => false, + + // An optional summary of your site, added as the introductory blockquote. + 'description' => null, + ], + /* |-------------------------------------------------------------------------- | Source Root Directory diff --git a/docs/_data/partials/hyde-pages-api/hyde-page-methods.md b/docs/_data/partials/hyde-pages-api/hyde-page-methods.md index bb6922fd9cd..4307951e7c1 100644 --- a/docs/_data/partials/hyde-pages-api/hyde-page-methods.md +++ b/docs/_data/partials/hyde-pages-api/hyde-page-methods.md @@ -1,7 +1,7 @@
- + #### `make()` @@ -68,23 +68,33 @@ HydePage::sourceDirectory(): string #### `outputDirectory()` -Get the output subdirectory to store compiled HTML files for the page type. +Get the output subdirectory where compiled files are stored for the page type. ```php HydePage::outputDirectory(): string ``` -#### `fileExtension()` +#### `sourceExtension()` -Get the file extension of the source files for the page type. +Get the file extension of the source files for the page type, such as `.md` or `.blade.php`. ```php -HydePage::fileExtension(): string +HydePage::sourceExtension(): string +``` + +#### `outputExtension()` + +Get the output file extension for the page type, such as `.html` or `.txt`. + +The value includes the leading dot, so it can be used directly as a file name suffix. + +```php +HydePage::outputExtension(): string ``` #### `setSourceDirectory()` -Set the output directory for the page type. +Set the source directory for the page type. ```php HydePage::setSourceDirectory(string $sourceDirectory): void @@ -92,18 +102,18 @@ HydePage::setSourceDirectory(string $sourceDirectory): void #### `setOutputDirectory()` -Set the source directory for the page type. +Set the output directory for the page type. ```php HydePage::setOutputDirectory(string $outputDirectory): void ``` -#### `setFileExtension()` +#### `setSourceExtension()` -Set the file extension for the page type. +Set the source file extension for the page type. ```php -HydePage::setFileExtension(string $fileExtension): void +HydePage::setSourceExtension(string $sourceExtension): void ``` #### `sourcePath()` @@ -193,9 +203,9 @@ $page->getOutputPath(): string // Path relative to the site output directory. Get the route key for the page. -The route key is the page URL path, relative to the site root, but without any file extensions. For example, if the page will be saved to `_site/docs/index.html`, the key is `docs/index`. +The route key is the page URL path, relative to the site root, but without the HTML file extension. For example, if the page will be saved to `_site/docs/index.html`, the key is `docs/index`. Pages compiled to non-HTML files keep their extension in the route key, so a page saved to `_site/docs/search.json` has the route key `docs/search.json`. -Route keys are used to identify page routes, similar to how named routes work in Laravel, only that here the name is not just arbitrary, but also defines the output location, as the route key is used to determine the output path which is `$routeKey.html`. +Route keys are used to identify page routes, similar to how named routes work in Laravel, only that here the name is not just arbitrary, but also defines the output location, as the route key is used to determine the output path which is `$routeKey.html`, or the route key as-is for pages compiled to non-HTML files. ```php $page->getRouteKey(): string @@ -263,6 +273,16 @@ Can the page be shown in the navigation menu? $page->showInNavigation(): bool ``` +#### `showInSitemap()` + +Can the page be shown in the sitemap? + +It can be explicitly set in the front matter using the `sitemap` key, otherwise it defaults to true for pages compiled to HTML files, and false for pages compiled to non-HTML files like `robots.txt`. + +```php +$page->showInSitemap(): bool +``` + #### `navigationMenuPriority()` Get the priority of the page in the navigation menu. diff --git a/docs/_data/partials/hyde-pages-api/in-memory-page-methods.md b/docs/_data/partials/hyde-pages-api/in-memory-page-methods.md index b4842f630d2..aeab59cc739 100644 --- a/docs/_data/partials/hyde-pages-api/in-memory-page-methods.md +++ b/docs/_data/partials/hyde-pages-api/in-memory-page-methods.md @@ -1,7 +1,7 @@
- + #### `make()` @@ -12,6 +12,16 @@ Static alias for the constructor. InMemoryPage::make(string|(Closure(): string)|(Closure(static):, Hyde\Markdown\Models\FrontMatter|array $matter, Closure|string|null $contents, string $view): static ``` +#### `outputPath()` + +Qualify a page identifier into a target output file path. + +Identifiers with a file extension are used verbatim, while identifiers without an extension are compiled to HTML files. + +```php +InMemoryPage::outputPath(string $identifier): string +``` + #### `__construct()` Create a new in-memory (virtual) page instance. @@ -20,7 +30,7 @@ Pass literal contents or a closure to `$contents`, or pass a registered Laravel Contents and views cannot be used together. Omit both to create an empty page. An empty view value is treated as no view. -View values ending in `.blade.php` are treated as Blade file paths. Other values are treated as registered Laravel view keys. +View values ending in `.blade.php` are treated as Blade file paths. Other values are treated as registered Laravel view keys. Identifiers that already have an extension use it as the output path unchanged. ```php /** @param string|(Closure(): string)|(Closure(static): string)|null $contents */ diff --git a/docs/advanced-features/build-tasks.md b/docs/advanced-features/build-tasks.md index b85e9c0c4e7..abb9b8b196c 100644 --- a/docs/advanced-features/build-tasks.md +++ b/docs/advanced-features/build-tasks.md @@ -1,5 +1,5 @@ --- -abstract: "The Build Task API lets you hook into HydePHP's build process to run custom logic, or override built-in tasks like sitemap and RSS generation, whenever the site compiles." +abstract: "The Build Task API lets you hook custom logic into HydePHP's build process before or after the site's pages are compiled." --- # Custom Build Tasks @@ -9,9 +9,10 @@ abstract: "The Build Task API lets you hook into HydePHP's build process to run The Build Task API offers a simple way to hook into the build process. The build tasks are very powerful and allow for limitless customizability. -The built-in Hyde features like sitemap generation and RSS feeds are created using tasks like these. Maybe you want to create your own, to for example upload the site to FTP or copy the files to a public directory? -You can also overload the built-in tasks to customize them to your needs. + +Sitemaps, RSS feeds, robots.txt, and llms.txt are generated as regular pages rather than build tasks. To customize +those files, see [Generated discovery files](customization#generated-discovery-files). ## Good to know before you start diff --git a/docs/advanced-features/hyde-pages.md b/docs/advanced-features/hyde-pages.md index 09e01bdeb7b..e0ed038aaba 100644 --- a/docs/advanced-features/hyde-pages.md +++ b/docs/advanced-features/hyde-pages.md @@ -70,14 +70,19 @@ abstract class HydePage public static string $sourceDirectory; /** - * The output subdirectory to store compiled HTML. Relative to the _site output directory. + * The output subdirectory to store compiled files. Relative to the _site output directory. */ public static string $outputDirectory; /** * The file extension of the source files. */ - public static string $fileExtension; + public static string $sourceExtension; + + /** + * The file extension of the compiled output files. + */ + public static string $outputExtension = '.html'; /** * The default template to use for rendering the page. @@ -139,7 +144,7 @@ abstract class BaseMarkdownPage extends HydePage { public Markdown $markdown; - public static string $fileExtension = '.md'; + public static string $sourceExtension = '.md'; } ``` @@ -156,6 +161,14 @@ autodiscovery, you may benefit from creating a custom page class instead, as tha You can learn more about the InMemoryPage class in the [InMemoryPage documentation](in-memory-pages). +In-memory pages infer their output format from the identifier. Identifiers without an extension compile to `.html`, +while identifiers that already have an extension keep it: + +```php +InMemoryPage::make('robots.txt', contents: $text); +// _site/robots.txt +``` + ### Quick Reference | Class Name | Namespace | Source Code | API Docs | @@ -177,7 +190,7 @@ class InMemoryPage extends HydePage { public static string $sourceDirectory; public static string $outputDirectory; - public static string $fileExtension; + public static string $sourceExtension; protected string|\Closure $contents; protected string $view; @@ -208,7 +221,7 @@ class BladePage extends HydePage { public static string $sourceDirectory = '_pages'; public static string $outputDirectory = ''; - public static string $fileExtension = '.blade.php'; + public static string $sourceExtension = '.blade.php'; } ``` @@ -326,7 +339,7 @@ class HtmlPage extends HydePage { public static string $sourceDirectory = '_pages'; public static string $outputDirectory = ''; - public static string $fileExtension = '.html'; + public static string $sourceExtension = '.html'; } ``` diff --git a/docs/advanced-features/in-memory-pages.md b/docs/advanced-features/in-memory-pages.md index 7ed8b458e53..eb20b7e149a 100644 --- a/docs/advanced-features/in-memory-pages.md +++ b/docs/advanced-features/in-memory-pages.md @@ -36,9 +36,52 @@ The constructor supports three content strategies: literal string contents, lazy Pass a string to the `$contents` parameter when the page contents are already available. Hyde saves the string literally. ```php -$page = new InMemoryPage('robots.txt', contents: "User-agent: *\n"); +InMemoryPage::make('about', contents: $html); +// _site/about.html ``` +The output format is inferred from the identifier. Identifiers without an extension compile to `.html`, while an +identifier that already has an extension keeps it: + +```php +InMemoryPage::make('robots.txt', contents: $text); +// _site/robots.txt +``` + +Static and instance path resolution use the same inference: + +```php +InMemoryPage::outputPath('robots.txt'); +// robots.txt + +InMemoryPage::make('robots.txt')->getOutputPath(); +// robots.txt +``` + +Non-HTML output is excluded from automatic navigation by default. To link one of these files from the generated +navigation, opt it in explicitly with `navigation.visible` (or set `navigation.hidden` to `false`): + +```php +InMemoryPage::make( + 'downloads/catalog.pdf', + matter: ['navigation' => ['visible' => true]], + contents: $catalog, +); +``` + +Non-HTML pages are also excluded from the sitemap by default. You can control sitemap inclusion for any page with +the `sitemap` front matter key. This same setting controls whether a page is listed in Hyde's generated `llms.txt`: + +```php +InMemoryPage::make( + 'api/schema.json', + matter: ['sitemap' => true], + contents: $schema, +); +``` + +For custom page classes, override `showInSitemap()` when the decision cannot be expressed as front matter. + Pass a closure when the contents should be generated lazily during compilation. The closure is invoked again for each compilation, which makes it useful for pages generated from the current application state. @@ -48,8 +91,7 @@ use Hyde\Pages\InMemoryPage; $page = new InMemoryPage( 'sitemap.xml', - ['navigation' => ['hidden' => true]], - fn (): string => app(SitemapGenerator::class)->generate()->getXml(), + contents: fn (): string => app(SitemapGenerator::class)->generate()->getXml(), ); ``` @@ -144,6 +186,10 @@ class AppServiceProvider extends ServiceProvider The page will be written to `_site/hello.html` and can be referenced using the `hello` route key. +Hyde's generated `sitemap.xml`, RSS feed, `robots.txt`, and `llms.txt` are also in-memory pages. Registering your own +page with one of those route keys during booting replaces Hyde's generated page, allowing complete control over the +file. For the RSS feed, use the filename configured in `hyde.rss.filename`. + ### In a package extension Package extensions can register the page directly in the page discovery callback. Pages added at this stage are diff --git a/docs/architecture-concepts/page-models.md b/docs/architecture-concepts/page-models.md index 9debc77d0fb..e5eedbab6ab 100644 --- a/docs/architecture-concepts/page-models.md +++ b/docs/architecture-concepts/page-models.md @@ -36,7 +36,7 @@ class MarkdownPost extends BaseMarkdownPage { public static string $sourceDirectory = '_posts'; public static string $outputDirectory = 'posts'; - public static string $fileExtension = '.md'; + public static string $sourceExtension = '.md'; public static string $template = 'post'; public string $identifier; @@ -63,7 +63,7 @@ class MarkdownPost extends BaseMarkdownPage { public static string $sourceDirectory = '_posts'; public static string $outputDirectory = 'posts'; - public static string $fileExtension = '.md'; + public static string $sourceExtension = '.md'; public static string $template = 'post'; } ``` diff --git a/docs/digging-deeper/customization.md b/docs/digging-deeper/customization.md index ab067fb53f4..a90fc214fee 100644 --- a/docs/digging-deeper/customization.md +++ b/docs/digging-deeper/customization.md @@ -127,26 +127,102 @@ redirects: docs/old-guide: docs/new-guide ``` -### RSS feed generation +### Generated discovery files -When enabled, an RSS feed containing all your Markdown blog posts will be generated when you compile your static site. -Here are the default settings: +Hyde generates a sitemap, RSS feed, `robots.txt`, and `llms.txt` as regular in-memory pages. They are registered routes, +so they are included in a normal site build, shown by `php hyde route:list`, recorded in the build manifest, and served +by `php hyde serve`. Generated non-HTML pages are excluded from navigation and from the sitemap by default. + +The sitemap and llms.txt require a site URL so they can contain absolute links. RSS additionally requires Markdown blog +posts and the SimpleXML extension; sitemap generation also requires SimpleXML. Robots.txt has no site URL requirement, +and includes a `Sitemap:` line only when the sitemap is available. + +Here are the related default settings: ```php // filepath config/hyde.php +'generate_sitemap' => true, + 'rss' => [ - // Should the RSS feed be generated? 'enabled' => true, - - // What filename should the RSS file use? 'filename' => 'feed.xml', - - // The channel description. 'description' => env('SITE_NAME', 'HydePHP').' RSS Feed', ], + +'robots' => [ + 'enabled' => true, + 'disallow' => [ + // '/private', + // '/*.pdf$', + ], +], + +'llms' => [ + 'enabled' => false, + 'description' => null, +], ``` ->warning Note that this feature requires that a `site_url` is set! +Each robots.txt `disallow` value is written verbatim as a `Disallow:` rule, allowing patterns such as `/*.pdf$`. +The llms.txt description becomes its introductory blockquote. The file groups published pages by type and uses each +page's `abstract` front matter, falling back to `description`, as the link description. A page is included in llms.txt +when it is included in the sitemap, so `sitemap: false` excludes it from both indexes. This does not prevent an AI +crawler from accessing the page; use robots.txt rules for crawler access control. + +Unlike the sitemap, RSS feed, and robots.txt, llms.txt is disabled by default: publishing it is a deliberate +invitation for AI services to read your site, so set `hyde.llms.enabled` to `true` to opt in. + +>warning Llms.txt is an emerging standard. Hyde may change the generated format in minor or patch releases as the +> specification evolves. Replace the page if you need a fixed format. + +#### Customizing generated output + +For small output changes, extend the corresponding generator and bind your implementation in a service provider. Hyde +resolves generators from the service container when the page is compiled, after route discovery is complete: + +```php +// filepath app/Providers/AppServiceProvider.php + +use Hyde\Framework\Features\TextGenerators\RobotsTxtGenerator; +use Illuminate\Support\ServiceProvider; + +class CustomRobotsTxtGenerator extends RobotsTxtGenerator +{ + public function generate(): string + { + return parent::generate()."\nHost: example.com\n"; + } +} + +class AppServiceProvider extends ServiceProvider +{ + public function register(): void + { + $this->app->bind(RobotsTxtGenerator::class, CustomRobotsTxtGenerator::class); + } +} +``` + +The available generators are `SitemapGenerator`, `RssFeedGenerator`, `RobotsTxtGenerator`, and `LlmsTxtGenerator` in +their respective `Hyde\Framework\Features\XmlGenerators` and `Hyde\Framework\Features\TextGenerators` namespaces. +The protected `LlmsTxtGenerator::sections()` method can be overridden to change its page groups. + +To replace a generated file completely, register your own [`InMemoryPage`](in-memory-pages) with +the same route key during a kernel booting callback or extension discovery. User-defined pages take precedence over +Hyde's generators, even when the corresponding feature is disabled. For example: + +```php +use Hyde\Hyde; +use Hyde\Pages\InMemoryPage; +use Hyde\Foundation\HydeKernel; + +Hyde::booting(function (HydeKernel $kernel): void { + $kernel->pages()->addPage(InMemoryPage::make( + 'robots.txt', + contents: "User-agent: *\nDisallow: /private\n", + )); +}); +``` ### Authors diff --git a/docs/getting-started/console-commands.md b/docs/getting-started/console-commands.md index 2f4c8f55be7..32194760945 100644 --- a/docs/getting-started/console-commands.md +++ b/docs/getting-started/console-commands.md @@ -116,6 +116,11 @@ php hyde build:rss Generate the RSS feed +This command compiles the registered RSS page. The page is normally registered when RSS generation is enabled, a site +URL is configured, at least one Markdown post exists, and the SimpleXML extension is available. The command exits with +an error if no RSS page is registered. A custom page registered at the configured `hyde.rss.filename` route is compiled +instead of Hyde's generated page. + ## Generate the `docs/search.json` file @@ -136,6 +141,11 @@ php hyde build:sitemap Generate the `sitemap.xml` file +This command compiles the registered `sitemap.xml` page. The page is normally registered when sitemap generation is +enabled, a site URL is configured, and the SimpleXML extension is available. The command exits with an error if no +sitemap page is registered. A custom page registered at the `sitemap.xml` route is compiled instead of Hyde's generated +page. + ## Scaffold a new Markdown, Blade, or documentation page file diff --git a/packages/framework/config/hyde.php b/packages/framework/config/hyde.php index 0214fd7d1d8..2c0f6380926 100644 --- a/packages/framework/config/hyde.php +++ b/packages/framework/config/hyde.php @@ -134,6 +134,57 @@ 'description' => env('SITE_NAME', 'HydePHP').' RSS Feed', ], + /* + |-------------------------------------------------------------------------- + | Robots.txt Generation + |-------------------------------------------------------------------------- + | + | When enabled, a robots.txt file allowing all crawlers will be generated + | when you compile your static site. A link to your sitemap is included + | when the sitemap feature is enabled. + | + | Values added to the disallow array are written verbatim as Disallow + | rule values for all crawlers, so wildcard patterns are supported. + | + */ + + 'robots' => [ + // Should the robots.txt file be generated? + 'enabled' => true, + + // Disallow rule values asking crawlers not to access matching paths. + 'disallow' => [ + // '/private', + // '/*.pdf$', + ], + ], + + /* + |-------------------------------------------------------------------------- + | Llms.txt Generation + |-------------------------------------------------------------------------- + | + | When enabled, an llms.txt file listing your pages will be generated when + | you compile your static site, helping AI services and agents find your + | content. Set this to true if you would like to publish one. + | + | This feature requires that a site base URL has been set. + | + | Unlike robots.txt, llms.txt is not an established web convention: it's + | an emerging proposal that may still change in minor and patch releases, + | and publishing it is a deliberate invitation for AI services to read + | your site, so it defaults to off until you decide you want that. + | + */ + + 'llms' => [ + // Should the llms.txt file be generated? + 'enabled' => false, + + // An optional summary of your site, added as the introductory blockquote. + 'description' => null, + ], + /* |-------------------------------------------------------------------------- | Source Root Directory diff --git a/packages/framework/src/Console/Commands/BuildRssFeedCommand.php b/packages/framework/src/Console/Commands/BuildRssFeedCommand.php index 6880c50b6b6..29cfda584f4 100644 --- a/packages/framework/src/Console/Commands/BuildRssFeedCommand.php +++ b/packages/framework/src/Console/Commands/BuildRssFeedCommand.php @@ -4,8 +4,13 @@ namespace Hyde\Console\Commands; -use Hyde\Framework\Actions\PostBuildTasks\GenerateRssFeed; -use LaravelZero\Framework\Commands\Command; +use Hyde\Hyde; +use Hyde\Console\Concerns\Command; +use Hyde\Foundation\Facades\Routes; +use Hyde\Framework\Actions\StaticPageBuilder; +use Hyde\Framework\Features\XmlGenerators\RssFeedGenerator; + +use function sprintf; /** * Run the build process for the RSS feed. @@ -20,6 +25,18 @@ class BuildRssFeedCommand extends Command public function handle(): int { - return (new GenerateRssFeed())->run($this->output); + $page = Routes::find(RssFeedGenerator::getFilename())?->getPage(); + + if ($page === null) { + $this->error('Cannot generate the RSS feed as the feature is not enabled'); + + return Command::FAILURE; + } + + $path = StaticPageBuilder::handle($page); + + $this->infoComment(sprintf('Created [%s]', Hyde::pathToRelative($path))); + + return Command::SUCCESS; } } diff --git a/packages/framework/src/Console/Commands/BuildSitemapCommand.php b/packages/framework/src/Console/Commands/BuildSitemapCommand.php index e6eaacf1212..b1c8abc6afc 100644 --- a/packages/framework/src/Console/Commands/BuildSitemapCommand.php +++ b/packages/framework/src/Console/Commands/BuildSitemapCommand.php @@ -4,8 +4,12 @@ namespace Hyde\Console\Commands; -use Hyde\Framework\Actions\PostBuildTasks\GenerateSitemap; -use LaravelZero\Framework\Commands\Command; +use Hyde\Hyde; +use Hyde\Console\Concerns\Command; +use Hyde\Foundation\Facades\Routes; +use Hyde\Framework\Actions\StaticPageBuilder; + +use function sprintf; /** * Run the build process for the sitemap. @@ -20,6 +24,18 @@ class BuildSitemapCommand extends Command public function handle(): int { - return (new GenerateSitemap())->run($this->output); + $page = Routes::find('sitemap.xml')?->getPage(); + + if ($page === null) { + $this->error('Cannot generate the sitemap as the feature is not enabled'); + + return Command::FAILURE; + } + + $path = StaticPageBuilder::handle($page); + + $this->infoComment(sprintf('Created [%s]', Hyde::pathToRelative($path))); + + return Command::SUCCESS; } } diff --git a/packages/framework/src/Facades/Features.php b/packages/framework/src/Facades/Features.php index 4a9a6383eee..b46519c5ccc 100644 --- a/packages/framework/src/Facades/Features.php +++ b/packages/framework/src/Facades/Features.php @@ -114,6 +114,23 @@ public static function hasRss(): bool && count(MarkdownPost::files()) > 0; } + /** + * Can a robots.txt file be generated? + */ + public static function hasRobotsTxt(): bool + { + return Config::getBool('hyde.robots.enabled', true); + } + + /** + * Can an llms.txt file be generated? + */ + public static function hasLlmsTxt(): bool + { + return Hyde::hasSiteUrl() + && Config::getBool('hyde.llms.enabled', false); + } + /** * Should documentation search be enabled? */ diff --git a/packages/framework/src/Foundation/HydeCoreExtension.php b/packages/framework/src/Foundation/HydeCoreExtension.php index 01ffab65695..a8d6716567a 100644 --- a/packages/framework/src/Foundation/HydeCoreExtension.php +++ b/packages/framework/src/Foundation/HydeCoreExtension.php @@ -4,12 +4,14 @@ namespace Hyde\Foundation; +use Closure; use Hyde\Hyde; use Hyde\Pages\HtmlPage; use Hyde\Pages\BladePage; use Hyde\Pages\MarkdownPage; use Hyde\Pages\MarkdownPost; use Hyde\Pages\DocumentationPage; +use Hyde\Pages\InMemoryPage; use Hyde\Pages\Concerns\HydePage; use Hyde\Support\BuildWarnings; use Hyde\Support\Models\Redirect; @@ -21,10 +23,15 @@ use Hyde\Facades\Config; use Hyde\Framework\Features\Documentation\DocumentationSearchPage; use Hyde\Framework\Features\Documentation\DocumentationSearchIndex; +use Hyde\Framework\Features\TextGenerators\LlmsTxtGenerator; +use Hyde\Framework\Features\TextGenerators\RobotsTxtGenerator; +use Hyde\Framework\Features\XmlGenerators\RssFeedGenerator; +use Hyde\Framework\Features\XmlGenerators\SitemapGenerator; use Hyde\Framework\Features\Documentation\Versioning\DocumentationVersion; use Hyde\Framework\Features\Documentation\Versioning\DocumentationVersions; use function Hyde\unslash; +use function app; use function array_filter; use function array_keys; use function sprintf; @@ -80,6 +87,68 @@ public function discoverPages(PageCollection $collection): void } } } + + if (Features::hasSitemap()) { + $this->discoverSitemapPage($collection); + } + + if (Features::hasRss()) { + $this->discoverRssFeedPage($collection); + } + + if (Features::hasRobotsTxt()) { + $this->discoverRobotsTxtPage($collection); + } + + if (Features::hasLlmsTxt()) { + $this->discoverLlmsTxtPage($collection); + } + } + + protected function discoverSitemapPage(PageCollection $collection): void + { + $this->addGeneratedPage( + $collection, + 'sitemap.xml', + fn (): string => app(SitemapGenerator::class)->generate()->getXml(), + ); + } + + protected function discoverRssFeedPage(PageCollection $collection): void + { + $this->addGeneratedPage( + $collection, + RssFeedGenerator::getFilename(), + fn (): string => app(RssFeedGenerator::class)->generate()->getXml(), + ); + } + + protected function discoverRobotsTxtPage(PageCollection $collection): void + { + $this->addGeneratedPage( + $collection, + 'robots.txt', + fn (): string => app(RobotsTxtGenerator::class)->generate(), + ); + } + + protected function discoverLlmsTxtPage(PageCollection $collection): void + { + $this->addGeneratedPage( + $collection, + 'llms.txt', + fn (): string => app(LlmsTxtGenerator::class)->generate(), + ); + } + + protected function addGeneratedPage(PageCollection $collection, string $routeKey, Closure $contents): void + { + if (! $this->hasPageWithRouteKey($collection, $routeKey)) { + $collection->addPage(new InMemoryPage( + $routeKey, + contents: $contents, + )); + } } /** Discard documentation source files stored outside the version directories. */ diff --git a/packages/framework/src/Foundation/Kernel/FileCollection.php b/packages/framework/src/Foundation/Kernel/FileCollection.php index 49c67e49aa3..7659cbbd9df 100644 --- a/packages/framework/src/Foundation/Kernel/FileCollection.php +++ b/packages/framework/src/Foundation/Kernel/FileCollection.php @@ -59,7 +59,7 @@ protected function runExtensionHandlers(): void protected function discoverFilesFor(string $pageClass): void { // Scan the source directory, and directories therein, for files that match the model's file extension. - foreach (Filesystem::findFiles($pageClass::sourceDirectory(), $pageClass::fileExtension(), true) as $path) { + foreach (Filesystem::findFiles($pageClass::sourceDirectory(), $pageClass::sourceExtension(), true) as $path) { if (! str_starts_with(basename((string) $path), '_')) { $this->addFile(SourceFile::make($path, $pageClass)); } diff --git a/packages/framework/src/Framework/Actions/PostBuildTasks/GenerateRssFeed.php b/packages/framework/src/Framework/Actions/PostBuildTasks/GenerateRssFeed.php deleted file mode 100644 index e974d12906b..00000000000 --- a/packages/framework/src/Framework/Actions/PostBuildTasks/GenerateRssFeed.php +++ /dev/null @@ -1,35 +0,0 @@ -path = Hyde::sitePath(RssFeedGenerator::getFilename()); - - $this->needsParentDirectory($this->path); - - file_put_contents($this->path, RssFeedGenerator::make()); - } - - public function printFinishMessage(): void - { - $this->createdSiteFile($this->path)->withExecutionTime(); - } -} diff --git a/packages/framework/src/Framework/Actions/PostBuildTasks/GenerateSitemap.php b/packages/framework/src/Framework/Actions/PostBuildTasks/GenerateSitemap.php deleted file mode 100644 index 4409ede2934..00000000000 --- a/packages/framework/src/Framework/Actions/PostBuildTasks/GenerateSitemap.php +++ /dev/null @@ -1,39 +0,0 @@ -skip('Cannot generate sitemap without a valid base URL'); - } - - $this->path = Hyde::sitePath('sitemap.xml'); - - $this->needsParentDirectory($this->path); - - file_put_contents($this->path, SitemapGenerator::make()); - } - - public function printFinishMessage(): void - { - $this->createdSiteFile($this->path)->withExecutionTime(); - } -} diff --git a/packages/framework/src/Framework/Factories/NavigationDataFactory.php b/packages/framework/src/Framework/Factories/NavigationDataFactory.php index 8ba85db9461..3cecc8ec52b 100644 --- a/packages/framework/src/Framework/Factories/NavigationDataFactory.php +++ b/packages/framework/src/Framework/Factories/NavigationDataFactory.php @@ -21,6 +21,7 @@ use function array_flip; use function array_intersect; use function array_key_exists; +use function str_ends_with; /** * Discover data used for navigation menus and the documentation sidebar. @@ -40,6 +41,7 @@ class NavigationDataFactory extends Concerns\PageDataFactory implements Navigati protected readonly ?int $priority; private readonly string $title; private readonly string $routeKey; + private readonly string $outputPath; private readonly string $pageClass; private readonly string $identifier; private readonly FrontMatter $matter; @@ -57,6 +59,7 @@ public function __construct(CoreDataObject $pageData, string $title) $this->identifier = $pageData->identifier; $this->pageClass = $pageData->pageClass; $this->routeKey = $pageData->routeKey; + $this->outputPath = $pageData->outputPath; $this->title = $title; $this->configurationKeys = $this->isInstanceOf(DocumentationPage::class) @@ -103,10 +106,18 @@ protected function makeGroup(): ?string protected function makeHidden(): bool { + $frontMatterHidden = $this->searchForHiddenInFrontMatter(); + return $this->isInstanceOf(MarkdownPost::class) - || $this->searchForHiddenInFrontMatter() + || $frontMatterHidden === true || $this->searchForHiddenInConfigs() - || $this->isNonDocumentationPageInHiddenSubdirectory(); + || $this->isNonDocumentationPageInHiddenSubdirectory() + || ($frontMatterHidden === null && $this->hasNonHtmlOutput()); + } + + private function hasNonHtmlOutput(): bool + { + return ! str_ends_with($this->outputPath, '.html'); } protected function makePriority(): int diff --git a/packages/framework/src/Framework/Features/Documentation/DocumentationSearchIndex.php b/packages/framework/src/Framework/Features/Documentation/DocumentationSearchIndex.php index b6cff21ceeb..bde991b7e74 100644 --- a/packages/framework/src/Framework/Features/Documentation/DocumentationSearchIndex.php +++ b/packages/framework/src/Framework/Features/Documentation/DocumentationSearchIndex.php @@ -49,9 +49,4 @@ public static function routeKey(?DocumentationVersion $version = null): string { return RouteKey::fromPage(DocumentationPage::class, $version === null ? 'search' : "$version->name/search").'.json'; } - - public function getOutputPath(): string - { - return static::routeKey($this->version); - } } diff --git a/packages/framework/src/Framework/Features/TextGenerators/LlmsTxtGenerator.php b/packages/framework/src/Framework/Features/TextGenerators/LlmsTxtGenerator.php new file mode 100644 index 00000000000..ffc70f80f16 --- /dev/null +++ b/packages/framework/src/Framework/Features/TextGenerators/LlmsTxtGenerator.php @@ -0,0 +1,162 @@ +getLines())."\n"; + } + + /** + * The page types listed in the file, and the section heading each is listed under. + * + * Pages of a type not listed here, like the virtual pages Hyde generates, are not + * added to the file. Override this to group your own page types into sections. + * + * @return array, string> + */ + protected function sections(): array + { + return [ + HtmlPage::class => 'Pages', + BladePage::class => 'Pages', + MarkdownPage::class => 'Pages', + DocumentationPage::class => 'Documentation', + MarkdownPost::class => 'Blog Posts', + ]; + } + + /** @return array */ + protected function getLines(): array + { + $lines = ['# '.Config::getString('hyde.name', 'HydePHP')]; + + $description = Config::getNullableString('hyde.llms.description'); + + if (filled($description)) { + $lines[] = ''; + $lines[] = '> '.$this->normalizeText($description); + } + + foreach ($this->getSections() as $heading => $routes) { + $lines[] = ''; + $lines[] = "## $heading"; + $lines[] = ''; + + foreach ($routes as $route) { + $lines[] = $this->makeLink($route); + } + } + + return $lines; + } + + protected function getSections(): array + { + $sections = $this->sections(); + + $grouped = array_fill_keys(array_values($sections), []); + + Routes::all()->each(function (Route $route) use ($sections, &$grouped): void { + $page = $route->getPage(); + + if (! $this->shouldListPage($page)) { + return; + } + + foreach ($sections as $pageClass => $heading) { + if ($page instanceof $pageClass) { + $grouped[$heading][] = $route; + + return; + } + } + }); + + return array_filter($grouped); + } + + protected function shouldListPage(HydePage $page): bool + { + return $page->showInSitemap() && $page->getIdentifier() !== '404'; + } + + protected function makeLink(Route $route): string + { + $page = $route->getPage(); + + $link = sprintf('- [%s](%s)', $this->escapeLinkLabel($page->title), Hyde::url($route->getOutputPath())); + + $description = $this->getPageDescription($page); + + return $description === null ? $link : "$link: $description"; + } + + protected function getPageDescription(HydePage $page): ?string + { + $description = $page->matter('abstract') ?? $page->matter('description'); + + if (! is_string($description) || ! filled($description)) { + return null; + } + + return $this->normalizeText($description); + } + + protected function escapeLinkLabel(string $label): string + { + return addcslashes($this->normalizeText($label), '[]\\'); + } + + protected function normalizeText(string $text): string + { + return preg_replace('/\s+/', ' ', trim($text)); + } +} diff --git a/packages/framework/src/Framework/Features/TextGenerators/RobotsTxtGenerator.php b/packages/framework/src/Framework/Features/TextGenerators/RobotsTxtGenerator.php new file mode 100644 index 00000000000..e9fd579182a --- /dev/null +++ b/packages/framework/src/Framework/Features/TextGenerators/RobotsTxtGenerator.php @@ -0,0 +1,58 @@ +getLines())."\n"; + } + + /** @return array */ + protected function getLines(): array + { + $lines = array_merge(['User-agent: *'], $this->getRuleLines()); + + if (Features::hasSitemap()) { + $lines = array_merge($lines, ['', 'Sitemap: '.Hyde::url('sitemap.xml')]); + } + + return $lines; + } + + /** @return array */ + protected function getRuleLines(): array + { + $rules = Config::getArray('hyde.robots.disallow', []); + + if ($rules === []) { + return ['Allow: /']; + } + + $lines = []; + + foreach ($rules as $rule) { + $lines[] = 'Disallow: '.(string) $rule; + } + + return $lines; + } +} diff --git a/packages/framework/src/Framework/Features/XmlGenerators/SitemapGenerator.php b/packages/framework/src/Framework/Features/XmlGenerators/SitemapGenerator.php index f21f18404de..abb862fb216 100644 --- a/packages/framework/src/Framework/Features/XmlGenerators/SitemapGenerator.php +++ b/packages/framework/src/Framework/Features/XmlGenerators/SitemapGenerator.php @@ -15,7 +15,6 @@ use Hyde\Facades\Filesystem; use Hyde\Pages\InMemoryPage; use Hyde\Support\Models\Route; -use Hyde\Support\Models\Redirect; use Illuminate\Support\Carbon; use Hyde\Pages\DocumentationPage; use Hyde\Foundation\Facades\Routes; @@ -31,7 +30,7 @@ class SitemapGenerator extends BaseXmlGenerator public function generate(): static { Routes::all()->each(function (Route $route): void { - if (! $route->getPage() instanceof Redirect) { + if ($route->getPage()->showInSitemap()) { $this->addRoute($route); } }); diff --git a/packages/framework/src/Framework/Services/BuildTaskService.php b/packages/framework/src/Framework/Services/BuildTaskService.php index d71d646f7a0..9b3589892cf 100644 --- a/packages/framework/src/Framework/Services/BuildTaskService.php +++ b/packages/framework/src/Framework/Services/BuildTaskService.php @@ -5,14 +5,11 @@ namespace Hyde\Framework\Services; use Hyde\Facades\Config; -use Hyde\Facades\Features; use Hyde\Facades\Filesystem; use Hyde\Framework\Features\BuildTasks\BuildTask; use Hyde\Framework\Features\BuildTasks\PreBuildTask; use Hyde\Framework\Features\BuildTasks\PostBuildTask; use Hyde\Framework\Actions\PreBuildTasks\CleanSiteDirectory; -use Hyde\Framework\Actions\PostBuildTasks\GenerateRssFeed; -use Hyde\Framework\Actions\PostBuildTasks\GenerateSitemap; use Hyde\Framework\Actions\PreBuildTasks\TransferMediaAssets; use Hyde\Framework\Actions\PostBuildTasks\GenerateBuildManifest; use Illuminate\Console\OutputStyle; @@ -135,8 +132,6 @@ private function registerFrameworkTasks(): void $this->registerIf(CleanSiteDirectory::class, $this->canCleanSiteDirectory()); $this->registerIf(TransferMediaAssets::class, $this->canTransferMediaAssets()); $this->registerIf(GenerateBuildManifest::class, $this->canGenerateManifest()); - $this->registerIf(GenerateSitemap::class, $this->canGenerateSitemap()); - $this->registerIf(GenerateRssFeed::class, $this->canGenerateFeed()); } private function canCleanSiteDirectory(): bool @@ -153,14 +148,4 @@ private function canGenerateManifest(): bool { return Config::getBool('hyde.generate_build_manifest', true); } - - private function canGenerateSitemap(): bool - { - return Features::hasSitemap(); - } - - private function canGenerateFeed(): bool - { - return Features::hasRss(); - } } diff --git a/packages/framework/src/Pages/BladePage.php b/packages/framework/src/Pages/BladePage.php index 5dd6bb8084b..22384dd26c9 100644 --- a/packages/framework/src/Pages/BladePage.php +++ b/packages/framework/src/Pages/BladePage.php @@ -21,7 +21,7 @@ class BladePage extends HydePage { public static string $sourceDirectory = '_pages'; public static string $outputDirectory = ''; - public static string $fileExtension = '.blade.php'; + public static string $sourceExtension = '.blade.php'; /** @param string $identifier The identifier, which also serves as the view key. */ public function __construct(string $identifier = '', FrontMatter|array $matter = []) diff --git a/packages/framework/src/Pages/Concerns/BaseMarkdownPage.php b/packages/framework/src/Pages/Concerns/BaseMarkdownPage.php index ba738c6a422..d9f32e95d08 100644 --- a/packages/framework/src/Pages/Concerns/BaseMarkdownPage.php +++ b/packages/framework/src/Pages/Concerns/BaseMarkdownPage.php @@ -25,7 +25,7 @@ abstract class BaseMarkdownPage extends HydePage implements MarkdownDocumentCont { public Markdown $markdown; - public static string $fileExtension = '.md'; + public static string $sourceExtension = '.md'; /** @inheritDoc */ public static function make(string $identifier = '', FrontMatter|array $matter = [], Markdown|string $markdown = ''): static diff --git a/packages/framework/src/Pages/Concerns/HydePage.php b/packages/framework/src/Pages/Concerns/HydePage.php index e60e44dc187..f78ea430194 100644 --- a/packages/framework/src/Pages/Concerns/HydePage.php +++ b/packages/framework/src/Pages/Concerns/HydePage.php @@ -29,6 +29,7 @@ use function filled; use function ltrim; use function rtrim; +use function str_ends_with; /** * The base class for all Hyde pages. @@ -56,7 +57,8 @@ abstract class HydePage implements PageSchema, SerializableContract public static string $sourceDirectory; public static string $outputDirectory; - public static string $fileExtension; + public static string $sourceExtension; + public static string $outputExtension = '.html'; public static string $template; public readonly string $identifier; @@ -95,7 +97,7 @@ public function __construct(string $identifier = '', FrontMatter|array $matter = */ public static function isDiscoverable(): bool { - return isset(static::$sourceDirectory, static::$outputDirectory, static::$fileExtension) && filled(static::$sourceDirectory); + return isset(static::$sourceDirectory, static::$outputDirectory, static::$sourceExtension) && filled(static::$sourceDirectory); } // Section: Query @@ -158,7 +160,7 @@ public static function sourceDirectory(): string } /** - * Get the output subdirectory to store compiled HTML files for the page type. + * Get the output subdirectory where compiled files are stored for the page type. */ public static function outputDirectory(): string { @@ -166,15 +168,25 @@ public static function outputDirectory(): string } /** - * Get the file extension of the source files for the page type. + * Get the file extension of the source files for the page type, such as `.md` or `.blade.php`. */ - public static function fileExtension(): string + public static function sourceExtension(): string { - return static::$fileExtension ?? ''; + return static::$sourceExtension ?? ''; } /** - * Set the output directory for the page type. + * Get the output file extension for the page type, such as `.html` or `.txt`. + * + * The value includes the leading dot, so it can be used directly as a file name suffix. + */ + public static function outputExtension(): string + { + return static::$outputExtension; + } + + /** + * Set the source directory for the page type. */ public static function setSourceDirectory(string $sourceDirectory): void { @@ -182,7 +194,7 @@ public static function setSourceDirectory(string $sourceDirectory): void } /** - * Set the source directory for the page type. + * Set the output directory for the page type. */ public static function setOutputDirectory(string $outputDirectory): void { @@ -190,11 +202,11 @@ public static function setOutputDirectory(string $outputDirectory): void } /** - * Set the file extension for the page type. + * Set the source file extension for the page type. */ - public static function setFileExtension(string $fileExtension): void + public static function setSourceExtension(string $sourceExtension): void { - static::$fileExtension = rtrim('.'.ltrim($fileExtension, '.'), '.'); + static::$sourceExtension = rtrim('.'.ltrim($sourceExtension, '.'), '.'); } /** @@ -202,7 +214,7 @@ public static function setFileExtension(string $fileExtension): void */ public static function sourcePath(string $identifier): string { - return unslash(static::sourceDirectory().'/'.unslash($identifier).static::fileExtension()); + return unslash(static::sourceDirectory().'/'.unslash($identifier).static::sourceExtension()); } /** @@ -210,7 +222,13 @@ public static function sourcePath(string $identifier): string */ public static function outputPath(string $identifier): string { - return RouteKey::fromPage(static::class, $identifier).'.html'; + $routeKey = RouteKey::fromPage(static::class, $identifier); + + if (static::outputExtension() === '.html') { + return "$routeKey.html"; + } + + return (string) $routeKey; } /** @@ -232,7 +250,7 @@ public static function pathToIdentifier(string $path): string { return unslash(Str::between(Hyde::pathToRelative($path), static::sourceDirectory().'/', - static::fileExtension()) + static::sourceExtension()) ); } @@ -292,12 +310,15 @@ public function getOutputPath(): string /** * Get the route key for the page. * - * The route key is the page URL path, relative to the site root, but without any file extensions. + * The route key is the page URL path, relative to the site root, but without the HTML file extension. * For example, if the page will be saved to `_site/docs/index.html`, the key is `docs/index`. + * Pages compiled to non-HTML files keep their extension in the route key, so a page + * saved to `_site/docs/search.json` has the route key `docs/search.json`. * * Route keys are used to identify page routes, similar to how named routes work in Laravel, * only that here the name is not just arbitrary, but also defines the output location, - * as the route key is used to determine the output path which is `$routeKey.html`. + * as the route key is used to determine the output path which is `$routeKey.html`, + * or the route key as-is for pages compiled to non-HTML files. */ public function getRouteKey(): string { @@ -373,6 +394,18 @@ public function showInNavigation(): bool return ! $this->navigation->hidden; } + /** + * Can the page be shown in the sitemap? + * + * It can be explicitly set in the front matter using the `sitemap` key, + * otherwise it defaults to true for pages compiled to HTML files, and + * false for pages compiled to non-HTML files like `robots.txt`. + */ + public function showInSitemap(): bool + { + return filter_var($this->matter('sitemap', str_ends_with($this->getOutputPath(), '.html')), FILTER_VALIDATE_BOOLEAN); + } + /** * Get the priority of the page in the navigation menu. */ diff --git a/packages/framework/src/Pages/HtmlPage.php b/packages/framework/src/Pages/HtmlPage.php index eef1f7a9cce..24870704505 100644 --- a/packages/framework/src/Pages/HtmlPage.php +++ b/packages/framework/src/Pages/HtmlPage.php @@ -18,7 +18,7 @@ class HtmlPage extends HydePage { public static string $sourceDirectory = '_pages'; public static string $outputDirectory = ''; - public static string $fileExtension = '.html'; + public static string $sourceExtension = '.html'; public function contents(): string { diff --git a/packages/framework/src/Pages/InMemoryPage.php b/packages/framework/src/Pages/InMemoryPage.php index 0a62949f60b..5e5441b990c 100644 --- a/packages/framework/src/Pages/InMemoryPage.php +++ b/packages/framework/src/Pages/InMemoryPage.php @@ -11,6 +11,10 @@ use Illuminate\Support\Facades\View; use InvalidArgumentException; +use function Hyde\unslash; +use function pathinfo; +use function str_ends_with; + /** * Extendable class for in-memory (or virtual) Hyde pages that are not based on source files. * @@ -31,7 +35,7 @@ class InMemoryPage extends HydePage { public static string $sourceDirectory; public static string $outputDirectory; - public static string $fileExtension; + public static string $sourceExtension; /** * The literal page contents, or a closure that generates them at compile time. @@ -74,6 +78,7 @@ public static function make( * * View values ending in `.blade.php` are treated as Blade file paths. Other values are treated * as registered Laravel view keys. + * Identifiers that already have an extension use it as the output path unchanged. * * @param string $identifier * @param FrontMatter|array $matter @@ -102,6 +107,19 @@ public function __construct( $this->view = $view ?? ''; } + /** + * Qualify a page identifier into a target output file path. + * + * Identifiers with a file extension are used verbatim, while identifiers + * without an extension are compiled to HTML files. + */ + public static function outputPath(string $identifier): string + { + $identifier = unslash($identifier); + + return $identifier.(pathinfo($identifier, PATHINFO_EXTENSION) === '' ? '.html' : ''); + } + /** * Get the literal contents or invoke the configured content closure. */ diff --git a/packages/framework/src/Support/Models/Redirect.php b/packages/framework/src/Support/Models/Redirect.php index 8a9cfbe95d3..94aaa5b1ed7 100644 --- a/packages/framework/src/Support/Models/Redirect.php +++ b/packages/framework/src/Support/Models/Redirect.php @@ -44,6 +44,11 @@ public function showInNavigation(): bool return false; } + public function showInSitemap(): bool + { + return false; + } + protected function normalizePath(string $path): string { if (str_ends_with($path, '.html')) { diff --git a/packages/framework/src/Support/Models/RouteKey.php b/packages/framework/src/Support/Models/RouteKey.php index a97916c8263..a371b327af9 100644 --- a/packages/framework/src/Support/Models/RouteKey.php +++ b/packages/framework/src/Support/Models/RouteKey.php @@ -11,16 +11,20 @@ use Hyde\Framework\Features\Blogging\BlogPostDatePrefixHelper; use function Hyde\unslash; +use function str_ends_with; /** * Route keys provide the core bindings of the HydePHP routing system as they are what canonically identifies a page. * This class both provides a data object for normalized type-hintable values, and general related helper methods. * - * In short, the route key is the URL path relative to the site webroot, without the file extension. + * In short, the route key is the URL path relative to the site webroot, without the HTML file extension. * * For example, `_pages/index.blade.php` would be compiled to `_site/index.html` and thus has the route key of `index`. * As another example, `_posts/welcome.md` would be compiled to `_site/posts/welcome.html` and thus has the route key of `posts/welcome`. * + * Only the HTML extension is implicit: pages compiled to non-HTML files keep their extension in the route key, + * so the documentation search index saved to `_site/docs/search.json` has the route key `docs/search.json`. + * * Note that if the source page's output directory is changed, the route key will change accordingly. * This can potentially cause links to break when changing the output directory for a page class. */ @@ -52,8 +56,14 @@ public function get(): string public static function fromPage(string $pageClass, string $identifier): self { $identifier = self::stripPrefixIfNeeded($pageClass, $identifier); + $key = unslash("{$pageClass::baseRouteKey()}/$identifier"); + $extension = $pageClass::outputExtension(); + + if ($extension !== '.html' && ! str_ends_with($key, $extension)) { + $key .= $extension; + } - return new self(unslash("{$pageClass::baseRouteKey()}/$identifier")); + return new self($key); } /** diff --git a/packages/framework/tests/Feature/AutomaticNavigationConfigurationsTest.php b/packages/framework/tests/Feature/AutomaticNavigationConfigurationsTest.php index 9caffbcf199..33d8fa43991 100644 --- a/packages/framework/tests/Feature/AutomaticNavigationConfigurationsTest.php +++ b/packages/framework/tests/Feature/AutomaticNavigationConfigurationsTest.php @@ -97,6 +97,28 @@ public function testInMemoryPagesAreAddedToNavigationMenu() ]); } + public function testNonHtmlInMemoryPagesAreNotAddedToNavigationMenuByDefault() + { + $this->assertMenuEquals([], [ + new InMemoryPage('feed.xml'), + new InMemoryPage('robots.txt'), + ]); + } + + public function testNonHtmlInMemoryPagesCanBeExplicitlyAddedToNavigationMenu() + { + $this->assertMenuEquals(['Feed', 'Humans'], [ + new InMemoryPage('feed.xml', [ + 'navigation.visible' => true, + 'navigation.label' => 'Feed', + ]), + new InMemoryPage('humans.txt', [ + 'navigation.hidden' => false, + 'navigation.label' => 'Humans', + ]), + ]); + } + public function testMainNavigationDoesNotInclude404Page() { $this->assertMenuEquals([], [new MarkdownPage('404')]); diff --git a/packages/framework/tests/Feature/Commands/BuildRssFeedCommandTest.php b/packages/framework/tests/Feature/Commands/BuildRssFeedCommandTest.php index 56b891b4740..21547b40dd9 100644 --- a/packages/framework/tests/Feature/Commands/BuildRssFeedCommandTest.php +++ b/packages/framework/tests/Feature/Commands/BuildRssFeedCommandTest.php @@ -6,10 +6,11 @@ use Hyde\Facades\Filesystem; use Hyde\Hyde; +use Hyde\Pages\InMemoryPage; +use Hyde\Foundation\HydeKernel; use Hyde\Testing\TestCase; #[\PHPUnit\Framework\Attributes\CoversClass(\Hyde\Console\Commands\BuildRssFeedCommand::class)] -#[\PHPUnit\Framework\Attributes\CoversClass(\Hyde\Framework\Actions\PostBuildTasks\GenerateRssFeed::class)] class BuildRssFeedCommandTest extends TestCase { public function testRssFeedIsGeneratedWhenConditionsAreMet() @@ -41,4 +42,56 @@ public function testRssFilenameCanBeChanged() $this->assertFileExists(Hyde::path('_site/blog.xml')); Filesystem::unlink('_site/blog.xml'); } + + public function testRssFeedIsNotGeneratedWithoutSiteUrl() + { + config(['hyde.url' => '']); + $this->file('_posts/foo.md'); + + $this->artisan('build:rss') + ->expectsOutput('Cannot generate the RSS feed as the feature is not enabled') + ->assertExitCode(1); + + $this->assertFileDoesNotExist(Hyde::path('_site/feed.xml')); + } + + public function testRssFeedIsNotGeneratedWhenFeedIsDisabledInConfig() + { + $this->withSiteUrl(); + config(['hyde.rss.enabled' => false]); + $this->file('_posts/foo.md'); + + $this->artisan('build:rss') + ->expectsOutput('Cannot generate the RSS feed as the feature is not enabled') + ->assertExitCode(1); + + $this->assertFileDoesNotExist(Hyde::path('_site/feed.xml')); + } + + public function testRssFeedIsNotGeneratedWhenThereAreNoPosts() + { + $this->withSiteUrl(); + + $this->artisan('build:rss') + ->expectsOutput('Cannot generate the RSS feed as the feature is not enabled') + ->assertExitCode(1); + + $this->assertFileDoesNotExist(Hyde::path('_site/feed.xml')); + } + + public function testCommandBuildsUserDefinedFeedPageEvenWhenRssFeatureConditionsAreNotMet() + { + $this->withSiteUrl(); + config(['hyde.rss.enabled' => false]); + + $this->cleanUpWhenDone('_site/feed.xml'); + + Hyde::kernel()->booting(function (HydeKernel $kernel): void { + $kernel->pages()->addPage(InMemoryPage::make('feed.xml', contents: '')); + }); + + $this->artisan('build:rss')->assertExitCode(0); + + $this->assertSame('', file_get_contents(Hyde::path('_site/feed.xml'))); + } } diff --git a/packages/framework/tests/Feature/Commands/BuildSitemapCommandTest.php b/packages/framework/tests/Feature/Commands/BuildSitemapCommandTest.php index 193517fca59..02d8358654f 100644 --- a/packages/framework/tests/Feature/Commands/BuildSitemapCommandTest.php +++ b/packages/framework/tests/Feature/Commands/BuildSitemapCommandTest.php @@ -5,10 +5,11 @@ namespace Hyde\Framework\Testing\Feature\Commands; use Hyde\Hyde; +use Hyde\Pages\InMemoryPage; +use Hyde\Foundation\HydeKernel; use Hyde\Testing\TestCase; #[\PHPUnit\Framework\Attributes\CoversClass(\Hyde\Console\Commands\BuildSitemapCommand::class)] -#[\PHPUnit\Framework\Attributes\CoversClass(\Hyde\Framework\Actions\PostBuildTasks\GenerateSitemap::class)] class BuildSitemapCommandTest extends TestCase { public function testSitemapIsGeneratedWhenConditionsAreMet() @@ -20,9 +21,7 @@ public function testSitemapIsGeneratedWhenConditionsAreMet() $this->assertFileDoesNotExist(Hyde::path('_site/sitemap.xml')); $this->artisan('build:sitemap') - ->expectsOutputToContain('Generating sitemap...') - ->doesntExpectOutputToContain('Skipped') - ->expectsOutputToContain(' > Created _site/sitemap.xml') + ->expectsOutputToContain('Created [_site/sitemap.xml]') ->assertExitCode(0); $this->assertFileExists(Hyde::path('_site/sitemap.xml')); @@ -35,11 +34,52 @@ public function testSitemapIsNotGeneratedWhenConditionsAreNotMet() $this->assertFileDoesNotExist(Hyde::path('_site/sitemap.xml')); $this->artisan('build:sitemap') - ->expectsOutputToContain('Generating sitemap...') - ->expectsOutputToContain('Skipped') - ->expectsOutput(' > Cannot generate sitemap without a valid base URL') - ->assertExitCode(3); + ->expectsOutput('Cannot generate the sitemap as the feature is not enabled') + ->assertExitCode(1); $this->assertFileDoesNotExist(Hyde::path('_site/sitemap.xml')); } + + public function testSitemapIsNotGeneratedWhenSitemapGenerationIsDisabledInConfig() + { + config(['hyde.url' => 'https://example.com']); + config(['hyde.generate_sitemap' => false]); + + $this->artisan('build:sitemap') + ->expectsOutput('Cannot generate the sitemap as the feature is not enabled') + ->assertExitCode(1); + + $this->assertFileDoesNotExist(Hyde::path('_site/sitemap.xml')); + } + + public function testCommandBuildsUserDefinedSitemapPageWhenOneIsRegistered() + { + config(['hyde.url' => 'https://example.com']); + + $this->cleanUpWhenDone('_site/sitemap.xml'); + + Hyde::kernel()->booting(function (HydeKernel $kernel): void { + $kernel->pages()->addPage(InMemoryPage::make('sitemap.xml', contents: '')); + }); + + $this->artisan('build:sitemap')->assertExitCode(0); + + $this->assertSame('', file_get_contents(Hyde::path('_site/sitemap.xml'))); + } + + public function testCommandBuildsUserDefinedSitemapPageEvenWhenSitemapFeatureIsDisabled() + { + config(['hyde.url' => 'https://example.com']); + config(['hyde.generate_sitemap' => false]); + + $this->cleanUpWhenDone('_site/sitemap.xml'); + + Hyde::kernel()->booting(function (HydeKernel $kernel): void { + $kernel->pages()->addPage(InMemoryPage::make('sitemap.xml', contents: '')); + }); + + $this->artisan('build:sitemap')->assertExitCode(0); + + $this->assertSame('', file_get_contents(Hyde::path('_site/sitemap.xml'))); + } } diff --git a/packages/framework/tests/Feature/Commands/RouteListCommandTest.php b/packages/framework/tests/Feature/Commands/RouteListCommandTest.php index 021ceced56b..15864f2705f 100644 --- a/packages/framework/tests/Feature/Commands/RouteListCommandTest.php +++ b/packages/framework/tests/Feature/Commands/RouteListCommandTest.php @@ -16,6 +16,13 @@ #[\PHPUnit\Framework\Attributes\CoversClass(\Hyde\Support\Internal\RouteListItem::class)] class RouteListCommandTest extends TestCase { + protected function setUp(): void + { + parent::setUp(); + + config(['hyde.generate_sitemap' => false, 'hyde.robots.enabled' => false, 'hyde.llms.enabled' => false]); + } + public function testRouteListCommand() { $this->artisan('route:list') diff --git a/packages/framework/tests/Feature/ConfigurableFeaturesTest.php b/packages/framework/tests/Feature/ConfigurableFeaturesTest.php index fb04fe0a978..9c2f98386d5 100644 --- a/packages/framework/tests/Feature/ConfigurableFeaturesTest.php +++ b/packages/framework/tests/Feature/ConfigurableFeaturesTest.php @@ -64,6 +64,37 @@ public function testCanGenerateSitemapHelperReturnsFalseIfSitemapsAreDisabledInC $this->assertFalse(Features::hasSitemap()); } + public function testHasRobotsTxtReturnsTrueByDefault() + { + $this->assertTrue(Features::hasRobotsTxt()); + } + + public function testHasRobotsTxtReturnsFalseWhenDisabledInConfig() + { + config(['hyde.robots.enabled' => false]); + $this->assertFalse(Features::hasRobotsTxt()); + } + + public function testHasLlmsTxtReturnsFalseByDefaultEvenWhenHydeHasBaseUrl() + { + $this->withSiteUrl(); + $this->assertFalse(Features::hasLlmsTxt()); + } + + public function testHasLlmsTxtReturnsFalseIfHydeDoesNotHaveBaseUrl() + { + $this->withSiteUrl(); + config(['hyde.llms.enabled' => true, 'hyde.url' => '']); + $this->assertFalse(Features::hasLlmsTxt()); + } + + public function testHasLlmsTxtReturnsTrueWhenEnabledInConfigAndHydeHasBaseUrl() + { + $this->withSiteUrl(); + config(['hyde.llms.enabled' => true]); + $this->assertTrue(Features::hasLlmsTxt()); + } + public function testHasThemeToggleButtonsReturnsTrueWhenDarkmodeEnabledAndConfigTrue() { // Enable dark mode and set hyde.theme_toggle_buttons config option to true diff --git a/packages/framework/tests/Feature/DiscoveryServiceTest.php b/packages/framework/tests/Feature/DiscoveryServiceTest.php index 814bc0a437e..de5de419a09 100644 --- a/packages/framework/tests/Feature/DiscoveryServiceTest.php +++ b/packages/framework/tests/Feature/DiscoveryServiceTest.php @@ -68,17 +68,17 @@ public function testGetSourceFileListForModelMethodFindsCustomizedSourceDirector MarkdownPage::setSourceDirectory('_pages'); } - public function testGetSourceFileListForModelMethodFindsCustomizedFileExtension() + public function testGetSourceFileListForModelMethodFindsCustomizedSourceExtension() { $this->directory('foo'); MarkdownPage::setSourceDirectory('foo'); - MarkdownPage::setFileExtension('.foo'); + MarkdownPage::setSourceExtension('.foo'); $this->unitTestMarkdownBasedPageList(MarkdownPage::class, 'foo/foo.foo', 'foo'); MarkdownPage::setSourceDirectory('_pages'); - MarkdownPage::setFileExtension('.md'); + MarkdownPage::setSourceExtension('.md'); } public function testGetMediaAssetFiles() diff --git a/packages/framework/tests/Feature/DocumentationSearchIndexTest.php b/packages/framework/tests/Feature/DocumentationSearchIndexTest.php index 5c8a3d4b373..bc48fb20013 100644 --- a/packages/framework/tests/Feature/DocumentationSearchIndexTest.php +++ b/packages/framework/tests/Feature/DocumentationSearchIndexTest.php @@ -45,6 +45,7 @@ public function testRouteKeyIsSetToVersionedDocumentationOutputDirectory() $this->assertSame('docs/1.x/search.json', $page->routeKey); $this->assertSame('docs/1.x/search.json', $page->getOutputPath()); + $this->assertSame($page::outputPath($page->getIdentifier()), $page->getOutputPath()); $this->assertSame('1.x', $page->getDocumentationVersion()->name); } diff --git a/packages/framework/tests/Feature/HydeExtensionFeatureTest.php b/packages/framework/tests/Feature/HydeExtensionFeatureTest.php index d48aa5d2e69..4fe121218cf 100644 --- a/packages/framework/tests/Feature/HydeExtensionFeatureTest.php +++ b/packages/framework/tests/Feature/HydeExtensionFeatureTest.php @@ -199,7 +199,7 @@ class HydeExtensionTestPage extends HydePage { public static string $sourceDirectory = 'foo'; public static string $outputDirectory = 'foo'; - public static string $fileExtension = '.txt'; + public static string $sourceExtension = '.txt'; public function compile(): string { @@ -211,7 +211,7 @@ class TestPageClass extends HydePage { public static string $sourceDirectory = 'foo'; public static string $outputDirectory = 'foo'; - public static string $fileExtension = '.txt'; + public static string $sourceExtension = '.txt'; public function compile(): string { diff --git a/packages/framework/tests/Feature/HydePageTest.php b/packages/framework/tests/Feature/HydePageTest.php index 9ca64d1495f..36fb949272e 100644 --- a/packages/framework/tests/Feature/HydePageTest.php +++ b/packages/framework/tests/Feature/HydePageTest.php @@ -46,9 +46,14 @@ public function testBaseOutputDirectory() $this->assertSame('', HydePage::outputDirectory()); } - public function testBaseFileExtension() + public function testBaseSourceExtension() { - $this->assertSame('', HydePage::fileExtension()); + $this->assertSame('', HydePage::sourceExtension()); + } + + public function testBaseOutputExtension() + { + $this->assertSame('.html', HydePage::outputExtension()); } public function testBaseSourcePath() @@ -93,9 +98,14 @@ public function testOutputDirectory() $this->assertSame('output', TestPage::outputDirectory()); } - public function testFileExtension() + public function testSourceExtension() { - $this->assertSame('.md', TestPage::fileExtension()); + $this->assertSame('.md', TestPage::sourceExtension()); + } + + public function testOutputExtension() + { + $this->assertSame('.html', TestPage::outputExtension()); } public function testSourcePath() @@ -108,6 +118,48 @@ public function testOutputPath() $this->assertSame('output/hello-world.html', TestPage::outputPath('hello-world')); } + public function testOutputExtensionCanBeOverriddenByChildClasses() + { + $this->assertSame('.txt', NonHtmlOutputTestPage::outputExtension()); + } + + public function testOutputPathUsesTheOutputExtensionOfThePageClass() + { + $this->assertSame('output/hello-world.txt', NonHtmlOutputTestPage::outputPath('hello-world')); + } + + public function testOutputExtensionWithoutLeadingDotIsReturnedUnchanged() + { + $this->assertSame('txt', MissingDotOutputExtensionTestPage::outputExtension()); + } + + public function testOutputExtensionWithPathSeparatorIsReturnedUnchanged() + { + $this->assertSame('.txt/../evil', PathSeparatorOutputExtensionTestPage::outputExtension()); + } + + public function testGetRouteKeyForPageWithNonHtmlOutputExtensionIncludesExtension() + { + $this->assertSame('output/hello-world.txt', (new NonHtmlOutputTestPage('hello-world'))->getRouteKey()); + } + + public function testGetOutputPathUsesTheOutputExtensionOfThePageClass() + { + $this->assertSame('output/hello-world.txt', (new NonHtmlOutputTestPage('hello-world'))->getOutputPath()); + } + + public function testGetLinkForPageWithNonHtmlOutputExtension() + { + $this->assertSame('output/hello-world.txt', (new NonHtmlOutputTestPage('hello-world'))->getLink()); + } + + public function testGetLinkForPageWithNonHtmlOutputExtensionIsNotAffectedByPrettyUrls() + { + config(['hyde.pretty_urls' => true]); + + $this->assertSame('output/hello-world.txt', (new NonHtmlOutputTestPage('hello-world'))->getLink()); + } + public function testPath() { $this->assertSame(Hyde::path('source/hello-world'), TestPage::path('hello-world')); @@ -246,27 +298,27 @@ public function testSetOutputDirectoryTrimsTrailingSlashes() $this->resetDirectoryConfiguration(); } - public function testGetFileExtensionReturnsStaticProperty() + public function testGetSourceExtensionReturnsStaticProperty() { - MarkdownPage::setFileExtension('.foo'); + MarkdownPage::setSourceExtension('.foo'); - $this->assertSame('.foo', MarkdownPage::fileExtension()); + $this->assertSame('.foo', MarkdownPage::sourceExtension()); $this->resetDirectoryConfiguration(); } - public function testSetFileExtensionForcesLeadingPeriod() + public function testSetSourceExtensionForcesLeadingPeriod() { - MarkdownPage::setFileExtension('foo'); + MarkdownPage::setSourceExtension('foo'); - $this->assertSame('.foo', MarkdownPage::fileExtension()); + $this->assertSame('.foo', MarkdownPage::sourceExtension()); $this->resetDirectoryConfiguration(); } - public function testSetFileExtensionRemovesTrailingPeriod() + public function testSetSourceExtensionRemovesTrailingPeriod() { - MarkdownPage::setFileExtension('foo.'); + MarkdownPage::setSourceExtension('foo.'); - $this->assertSame('.foo', MarkdownPage::fileExtension()); + $this->assertSame('.foo', MarkdownPage::sourceExtension()); $this->resetDirectoryConfiguration(); } @@ -288,10 +340,10 @@ public function testSetOutputDirectory() $this->assertSame('foo', ConfigurableSourcesTestPage::outputDirectory()); } - public function testSetFileExtension() + public function testSetSourceExtension() { - ConfigurableSourcesTestPage::setFileExtension('.foo'); - $this->assertSame('.foo', ConfigurableSourcesTestPage::fileExtension()); + ConfigurableSourcesTestPage::setSourceExtension('.foo'); + $this->assertSame('.foo', ConfigurableSourcesTestPage::sourceExtension()); } public function testStaticGetMethodReturnsDiscoveredPage() @@ -354,7 +406,7 @@ public function testQualifyBasenameTrimsSlashesFromInput() public function testQualifyBasenameUsesTheStaticProperties() { MarkdownPage::setSourceDirectory('foo'); - MarkdownPage::setFileExtension('txt'); + MarkdownPage::setSourceExtension('txt'); $this->assertSame('foo/bar.txt', MarkdownPage::sourcePath('bar')); @@ -497,7 +549,7 @@ public function testAllPageModelsHaveConfiguredOutputDirectory() } } - public function testAllPageModelsHaveConfiguredFileExtension() + public function testAllPageModelsHaveConfiguredSourceExtension() { $pages = [ BladePage::class => '.blade.php', @@ -508,7 +560,7 @@ public function testAllPageModelsHaveConfiguredFileExtension() foreach ($pages as $page => $expected) { assert(is_a($page, HydePage::class, true)); - $this->assertSame($expected, $page::fileExtension()); + $this->assertSame($expected, $page::sourceExtension()); } } @@ -527,14 +579,14 @@ public function testAbstractMarkdownPageHasMarkdownDocumentProperty() $this->assertTrue(property_exists(BaseMarkdownPage::class, 'markdown')); } - public function testAbstractMarkdownPageHasFileExtensionProperty() + public function testAbstractMarkdownPageHasSourceExtensionProperty() { - $this->assertTrue(property_exists(BaseMarkdownPage::class, 'fileExtension')); + $this->assertTrue(property_exists(BaseMarkdownPage::class, 'sourceExtension')); } - public function testAbstractMarkdownPageFileExtensionPropertyIsSetToMd() + public function testAbstractMarkdownPageSourceExtensionPropertyIsSetToMd() { - $this->assertSame('.md', BaseMarkdownPage::fileExtension()); + $this->assertSame('.md', BaseMarkdownPage::sourceExtension()); } public function testAbstractMarkdownPageConstructorArgumentsAreOptional() @@ -1253,7 +1305,7 @@ protected function resetDirectoryConfiguration(): void MarkdownPage::setSourceDirectory('_pages'); MarkdownPost::setSourceDirectory('_posts'); DocumentationPage::setSourceDirectory('_docs'); - MarkdownPage::setFileExtension('.md'); + MarkdownPage::setSourceExtension('.md'); } } @@ -1263,17 +1315,42 @@ class TestPage extends HydePage public static string $sourceDirectory = 'source'; public static string $outputDirectory = 'output'; - public static string $fileExtension = '.md'; + public static string $sourceExtension = '.md'; public static string $template = 'template'; } +class NonHtmlOutputTestPage extends HydePage +{ + use VoidCompiler; + + public static string $sourceDirectory = 'source'; + public static string $outputDirectory = 'output'; + public static string $sourceExtension = '.txt'; + public static string $outputExtension = '.txt'; + public static string $template = 'template'; +} + +class MissingDotOutputExtensionTestPage extends HydePage +{ + use VoidCompiler; + + public static string $outputExtension = 'txt'; +} + +class PathSeparatorOutputExtensionTestPage extends HydePage +{ + use VoidCompiler; + + public static string $outputExtension = '.txt/../evil'; +} + class ConfigurableSourcesTestPage extends HydePage { use VoidCompiler; public static string $sourceDirectory; public static string $outputDirectory; - public static string $fileExtension; + public static string $sourceExtension; public static string $template; } @@ -1283,7 +1360,7 @@ class DiscoverableTestPage extends HydePage public static string $sourceDirectory = 'foo'; public static string $outputDirectory = 'bar'; - public static string $fileExtension = 'baz'; + public static string $sourceExtension = 'baz'; public static string $template; } @@ -1293,7 +1370,7 @@ class NonDiscoverableTestPage extends HydePage public static string $sourceDirectory; public static string $outputDirectory; - public static string $fileExtension; + public static string $sourceExtension; } class PartiallyDiscoverablePage extends HydePage @@ -1302,7 +1379,7 @@ class PartiallyDiscoverablePage extends HydePage public static string $sourceDirectory = 'foo'; public static string $outputDirectory; - public static string $fileExtension; + public static string $sourceExtension; } class DiscoverablePageWithInvalidSourceDirectory extends HydePage @@ -1311,7 +1388,7 @@ class DiscoverablePageWithInvalidSourceDirectory extends HydePage public static string $sourceDirectory = ''; public static string $outputDirectory = ''; - public static string $fileExtension = ''; + public static string $sourceExtension = ''; } class MissingSourceDirectoryMarkdownPage extends BaseMarkdownPage diff --git a/packages/framework/tests/Feature/LlmsTxtGeneratorTest.php b/packages/framework/tests/Feature/LlmsTxtGeneratorTest.php new file mode 100644 index 00000000000..364dc059eb5 --- /dev/null +++ b/packages/framework/tests/Feature/LlmsTxtGeneratorTest.php @@ -0,0 +1,216 @@ +withSiteUrl(); + } + + public function testGeneratesFileWithSiteNameAsHeading() + { + config(['hyde.name' => 'My Site']); + + $this->assertSame(<<<'TXT' + # My Site + + ## Pages + + - [Index](https://example.com/index.html) + + TXT, $this->generate()); + } + + public function testGeneratesSummaryBlockquoteWhenDescriptionIsConfigured() + { + config(['hyde.llms.description' => 'Everything about the example project.']); + + $this->assertStringContainsString("# HydePHP\n\n> Everything about the example project.\n\n## Pages", $this->generate()); + } + + public function testOmitsSummaryBlockquoteWhenNoDescriptionIsConfigured() + { + $this->assertStringStartsWith("# HydePHP\n\n## Pages", $this->generate()); + } + + public function testGroupsPagesIntoSectionsByPageType() + { + $this->file('_pages/about.md', '# About'); + $this->file('_docs/installation.md', '# Installation'); + $this->file('_posts/hello-world.md', '# Hello World'); + + $this->assertSame(<<<'TXT' + # HydePHP + + ## Pages + + - [Index](https://example.com/index.html) + - [About](https://example.com/about.html) + + ## Documentation + + - [Installation](https://example.com/docs/installation.html) + + ## Blog Posts + + - [Hello World](https://example.com/posts/hello-world.html) + + TXT, $this->generate()); + } + + public function testEmptySectionsAreOmitted() + { + $this->assertStringNotContainsString('## Documentation', $this->generate()); + $this->assertStringNotContainsString('## Blog Posts', $this->generate()); + } + + public function testUsesPageAbstractAsLinkDescription() + { + $this->markdown('_docs/installation.md', '# Installation', ['abstract' => 'How to install the project.']); + + $this->assertStringContainsString('- [Installation](https://example.com/docs/installation.html): How to install the project.', $this->generate()); + } + + public function testFallsBackToPageDescriptionWhenThereIsNoAbstract() + { + $this->markdown('_docs/installation.md', '# Installation', ['description' => 'The meta description.']); + + $this->assertStringContainsString('- [Installation](https://example.com/docs/installation.html): The meta description.', $this->generate()); + } + + public function testPrefersTheAbstractOverTheDescription() + { + $this->markdown('_docs/installation.md', '# Installation', [ + 'abstract' => 'The abstract.', + 'description' => 'The meta description.', + ]); + + $this->assertStringContainsString('- [Installation](https://example.com/docs/installation.html): The abstract.', $this->generate()); + } + + public function testMultiLineDescriptionsAreCollapsedToASingleLine() + { + $this->file('_docs/installation.md', "---\nabstract: |\n First line.\n Second line.\n---\n\n# Installation"); + + $this->assertStringContainsString('- [Installation](https://example.com/docs/installation.html): First line. Second line.', $this->generate()); + } + + public function testPagesExcludedFromTheSitemapAreNotListed() + { + $this->markdown('_pages/private.md', '# Private', ['sitemap' => false]); + + $this->assertStringNotContainsString('Private', $this->generate()); + } + + public function testErrorPagesAreNotListed() + { + $this->assertStringNotContainsString('404', $this->generate()); + } + + public function testGeneratedNonHtmlPagesAreNotListed() + { + $contents = $this->generate(); + + $this->assertStringNotContainsString('llms.txt', $contents); + $this->assertStringNotContainsString('robots.txt', $contents); + $this->assertStringNotContainsString('sitemap.xml', $contents); + } + + public function testVirtualPagesLikeTheDocumentationSearchPageAreNotListed() + { + $this->file('_docs/installation.md', '# Installation'); + + $contents = $this->generate(); + + $this->assertStringContainsString('- [Installation](https://example.com/docs/installation.html)', $contents); + $this->assertStringNotContainsString('docs/search', $contents); + } + + public function testRedirectsAreNotListed() + { + Routes::addRoute(new Route(new Redirect('old-page', 'new-page'))); + + $this->assertStringNotContainsString('old-page', $this->generate()); + } + + public function testCustomPageClassesAreListedUnderTheirParentClassSection() + { + Routes::addRoute(new Route(new LlmsTxtGeneratorTestPage('custom'))); + + $this->assertStringContainsString("## Pages\n\n- [Index](https://example.com/index.html)\n- [Custom](https://example.com/custom.html)", $this->generate()); + } + + public function testPagesAreListedInRouteOrderSoNumericPrefixesKeepTheirReadingOrder() + { + $this->file('_docs/02-usage.md', '# Usage'); + $this->file('_docs/01-installation.md', '# Installation'); + $this->file('_docs/03-advanced.md', '# Advanced'); + + $this->assertStringContainsString(<<<'TXT' + ## Documentation + + - [Installation](https://example.com/docs/installation.html) + - [Usage](https://example.com/docs/usage.html) + - [Advanced](https://example.com/docs/advanced.html) + TXT, $this->generate()); + } + + public function testMarkdownSyntaxInPageTitlesIsEscapedInLinkLabels() + { + $this->markdown('_docs/arrays.md', '# Arrays', ['title' => 'Arrays [Advanced]']); + + $this->assertStringContainsString('- [Arrays \[Advanced\]](https://example.com/docs/arrays.html)', $this->generate()); + } + + public function testPrettyUrlsAreUsedWhenEnabled() + { + config(['hyde.pretty_urls' => true]); + + $this->file('_docs/installation.md', '# Installation'); + + $this->assertStringContainsString('- [Installation](https://example.com/docs/installation)', $this->generate()); + } + + public function testGeneratorOverrideCanListAPageThatIsExcludedFromTheSitemap() + { + $this->markdown('_pages/thin.md', '# Thin Page', ['sitemap' => false]); + $this->markdown('_pages/private.md', '# Private Page', ['sitemap' => false]); + + $generator = new class extends LlmsTxtGenerator + { + protected function shouldListPage(HydePage $page): bool + { + return parent::shouldListPage($page) || $page->getIdentifier() === 'thin'; + } + }; + + $contents = $generator->generate(); + + $this->assertStringContainsString('- [Thin Page](https://example.com/thin.html)', $contents); + $this->assertStringNotContainsString('Private Page', $contents); + } + + protected function generate(): string + { + return (new LlmsTxtGenerator())->generate(); + } +} + +class LlmsTxtGeneratorTestPage extends MarkdownPage +{ +} diff --git a/packages/framework/tests/Feature/LlmsTxtPageTest.php b/packages/framework/tests/Feature/LlmsTxtPageTest.php new file mode 100644 index 00000000000..eb489535d5b --- /dev/null +++ b/packages/framework/tests/Feature/LlmsTxtPageTest.php @@ -0,0 +1,166 @@ +withSiteUrl(); + + config(['hyde.llms.enabled' => true]); + } + + protected function tearDown(): void + { + File::cleanDirectory(Hyde::path('_site')); + + parent::tearDown(); + } + + public function testLlmsTxtPageIsRegisteredAsRouteWhenEnabled() + { + $this->assertTrue(Routes::exists('llms.txt')); + + $page = Routes::get('llms.txt')->getPage(); + + $this->assertSame(InMemoryPage::class, $page::class); + $this->assertSame('llms.txt', $page->getOutputPath()); + $this->assertSame($page::outputPath($page->getIdentifier()), $page->getOutputPath()); + $this->assertSame('llms.txt', $page->getRouteKey()); + } + + public function testLlmsTxtPageIsNotRegisteredWithoutSiteUrl() + { + $this->withoutSiteUrl(); + + $this->assertFalse(Routes::exists('llms.txt')); + } + + public function testLlmsTxtPageIsNotRegisteredWhenDisabledInConfig() + { + config(['hyde.llms.enabled' => false]); + + $this->assertFalse(Routes::exists('llms.txt')); + } + + public function testLlmsTxtPageIsHiddenFromNavigationAndExcludedFromTheSitemap() + { + $page = Routes::get('llms.txt')->getPage(); + + $this->assertFalse($page->showInNavigation()); + $this->assertFalse($page->showInSitemap()); + } + + public function testLlmsTxtPageCompilesUsingTheLlmsTxtGenerator() + { + $this->assertSame((new LlmsTxtGenerator())->generate(), Routes::get('llms.txt')->getPage()->compile()); + } + + public function testLlmsTxtGeneratorCanBeSwappedThroughTheServiceContainer() + { + app()->bind(LlmsTxtGenerator::class, fn (): LlmsTxtGenerator => new class extends LlmsTxtGenerator + { + public function generate(): string + { + return 'custom generator output'; + } + }); + + $this->assertSame('custom generator output', Routes::get('llms.txt')->getPage()->compile()); + } + + public function testBuildCommandCompilesLlmsTxtPageAsDynamicPage() + { + $this->artisan('build') + ->expectsOutput('Creating Dynamic Pages...') + ->assertExitCode(0); + + $this->assertStringStartsWith('# HydePHP', file_get_contents(Hyde::path('_site/llms.txt'))); + } + + public function testBuiltLlmsTxtIsExcludedFromTheSitemapAndItself() + { + $this->artisan('build')->assertExitCode(0); + + $this->assertStringNotContainsString('llms.txt', file_get_contents(Hyde::path('_site/sitemap.xml'))); + $this->assertStringNotContainsString('llms.txt', file_get_contents(Hyde::path('_site/llms.txt'))); + } + + public function testLlmsTxtPageIsIncludedInTheBuildManifest() + { + $this->artisan('build')->assertExitCode(0); + + $manifest = json_decode(file_get_contents(Hyde::path('app/storage/framework/cache/build-manifest.json')), true); + + $this->assertArrayHasKey('llms.txt', $manifest['pages']); + $this->assertSame('llms.txt', $manifest['pages']['llms.txt']['output_path']); + } + + public function testLlmsTxtRouteIsIncludedInTheRouteList() + { + $this->artisan('route:list') + ->expectsOutputToContain('llms.txt') + ->assertExitCode(0); + } + + public function testUserPageRegisteredInBootingCallbackSuppressesTheGeneratedLlmsTxtPage() + { + Hyde::kernel()->booting(function (HydeKernel $kernel): void { + $kernel->pages()->addPage(InMemoryPage::make('llms.txt', contents: 'user defined llms')); + }); + + $page = Routes::get('llms.txt')->getPage(); + + $this->assertSame('user defined llms', $page->compile()); + $this->assertSame(1, Hyde::pages()->filter(fn ($page) => $page->getRouteKey() === 'llms.txt')->count()); + + $this->artisan('build')->assertExitCode(0); + + $this->assertSame('user defined llms', file_get_contents(Hyde::path('_site/llms.txt'))); + } + + public function testUserPageRegisteredThroughExtensionSuppressesTheGeneratedLlmsTxtPage() + { + Hyde::kernel()->registerExtension(LlmsTxtPageTestExtension::class); + + $page = Routes::get('llms.txt')->getPage(); + + $this->assertSame('extension defined llms', $page->compile()); + $this->assertSame(1, Hyde::pages()->filter(fn ($page) => $page->getRouteKey() === 'llms.txt')->count()); + + $this->artisan('build')->assertExitCode(0); + + $this->assertSame('extension defined llms', file_get_contents(Hyde::path('_site/llms.txt'))); + } +} + +class LlmsTxtPageTestExtension extends HydeExtension +{ + public function discoverPages(PageCollection $collection): void + { + $collection->addPage(InMemoryPage::make('llms.txt', contents: 'extension defined llms')); + } +} diff --git a/packages/framework/tests/Feature/NonHtmlPageOutputTest.php b/packages/framework/tests/Feature/NonHtmlPageOutputTest.php new file mode 100644 index 00000000000..de720d66926 --- /dev/null +++ b/packages/framework/tests/Feature/NonHtmlPageOutputTest.php @@ -0,0 +1,140 @@ +booting(function (HydeKernel $kernel): void { + $kernel->pages()->addPage(InMemoryPage::make('robots.txt', contents: "User-agent: *\nAllow: /")); + }); + + $this->artisan('build')->assertExitCode(0); + + $this->assertFileExists(Hyde::path('_site/robots.txt')); + $this->assertSame("User-agent: *\nAllow: /", file_get_contents(Hyde::path('_site/robots.txt'))); + $this->assertFileDoesNotExist(Hyde::path('_site/robots.txt.html')); + } + + public function testBuildCommandCompilesNestedNonHtmlOutputPath() + { + Hyde::kernel()->booting(function (HydeKernel $kernel): void { + $kernel->pages()->addPage(InMemoryPage::make('foo/bar.txt', contents: 'baz')); + }); + + $this->artisan('build')->assertExitCode(0); + + $this->assertFileExists(Hyde::path('_site/foo/bar.txt')); + $this->assertSame('baz', file_get_contents(Hyde::path('_site/foo/bar.txt'))); + } + + public function testNonHtmlInMemoryPageIsRegisteredAsRoute() + { + Hyde::kernel()->booting(function (HydeKernel $kernel): void { + $kernel->pages()->addPage(InMemoryPage::make('robots.txt', contents: 'User-agent: *')); + }); + + $this->assertTrue(Routes::exists('robots.txt')); + $this->assertSame('robots.txt', Routes::get('robots.txt')->getOutputPath()); + } + + public function testStaticPageBuilderCompilesNonHtmlInMemoryPage() + { + StaticPageBuilder::handle(InMemoryPage::make('llms.txt', contents: '# Hello World')); + + $this->assertFileExists(Hyde::path('_site/llms.txt')); + $this->assertSame('# Hello World', file_get_contents(Hyde::path('_site/llms.txt'))); + } + + public function testNonHtmlInMemoryPageCanCompileUsingView() + { + $this->file('_pages/robots.blade.php', 'User-agent: {{ $agent }}'); + + StaticPageBuilder::handle(InMemoryPage::make('robots.txt', ['agent' => '*'], view: 'robots')); + + $this->assertFileExists(Hyde::path('_site/robots.txt')); + $this->assertSame('User-agent: *', file_get_contents(Hyde::path('_site/robots.txt'))); + } + + public function testBuildCommandExcludesNonHtmlInMemoryPageFromSitemap() + { + $this->withSiteUrl(); + + Hyde::kernel()->booting(function (HydeKernel $kernel): void { + $kernel->pages()->addPage(InMemoryPage::make('robots.txt', contents: 'User-agent: *')); + }); + + $this->artisan('build')->assertExitCode(0); + + $this->assertFileExists(Hyde::path('_site/robots.txt')); + $this->assertFileExists(Hyde::path('_site/sitemap.xml')); + $this->assertStringNotContainsString('robots.txt', file_get_contents(Hyde::path('_site/sitemap.xml'))); + } + + public function testBuildCommandCompilesDiscoverableCustomPageClassWithNonHtmlOutputExtension() + { + $this->directory('_leaves'); + $this->file('_leaves/hello.md', 'Hello World'); + + Hyde::kernel()->registerExtension(NonHtmlPageTestExtension::class); + + $this->assertSame(['hello'], DiscoverableNonHtmlTestPage::files()); + $this->assertTrue(Routes::exists('hello.txt')); + + $this->artisan('build')->assertExitCode(0); + + $this->assertFileExists(Hyde::path('_site/hello.txt')); + $this->assertSame('Hello World', file_get_contents(Hyde::path('_site/hello.txt'))); + $this->assertFileDoesNotExist(Hyde::path('_site/hello.txt.html')); + $this->assertFileDoesNotExist(Hyde::path('_site/hello.html')); + $this->assertFileDoesNotExist(Hyde::path('_site/hello.md.html')); + } +} + +class DiscoverableNonHtmlTestPage extends HydePage +{ + public static string $sourceDirectory = '_leaves'; + public static string $outputDirectory = ''; + public static string $sourceExtension = '.md'; + public static string $outputExtension = '.txt'; + + public function compile(): string + { + return file_get_contents(Hyde::path($this->getSourcePath())); + } +} + +class NonHtmlPageTestExtension extends HydeExtension +{ + public static function getPageClasses(): array + { + return [DiscoverableNonHtmlTestPage::class]; + } +} diff --git a/packages/framework/tests/Feature/PageCollectionTest.php b/packages/framework/tests/Feature/PageCollectionTest.php index 38ce661fd6f..4d08f92ee04 100644 --- a/packages/framework/tests/Feature/PageCollectionTest.php +++ b/packages/framework/tests/Feature/PageCollectionTest.php @@ -22,6 +22,13 @@ #[\PHPUnit\Framework\Attributes\CoversClass(\Hyde\Foundation\Facades\Pages::class)] class PageCollectionTest extends TestCase { + protected function setUp(): void + { + parent::setUp(); + + config(['hyde.generate_sitemap' => false, 'hyde.rss.enabled' => false, 'hyde.robots.enabled' => false, 'hyde.llms.enabled' => false]); + } + public function testBootMethodCreatesNewPageCollectionAndDiscoversPagesAutomatically() { $collection = PageCollection::init(Hyde::getInstance())->boot(); diff --git a/packages/framework/tests/Feature/RedirectTest.php b/packages/framework/tests/Feature/RedirectTest.php index 1ce30f25c27..ee051044bb8 100644 --- a/packages/framework/tests/Feature/RedirectTest.php +++ b/packages/framework/tests/Feature/RedirectTest.php @@ -64,6 +64,14 @@ public function testConfiguredRedirectsAreRegisteredWithTheKernelAndBuiltWithThe Filesystem::unlink('_site/foo.html'); } + public function testDottedRedirectPathKeepsItsExtension() + { + $redirect = new Redirect('legacy.json', 'new-location'); + + $this->assertSame('legacy.json', $redirect->getRouteKey()); + $this->assertSame('legacy.json', $redirect->getOutputPath()); + } + public function testRedirectsCannotWriteOutsideTheBuildPipeline() { $this->assertFalse(method_exists(Redirect::class, 'create')); @@ -74,4 +82,9 @@ public function testRedirectsAreHiddenFromNavigation() { $this->assertFalse((new Redirect('foo', 'bar'))->showInNavigation()); } + + public function testRedirectsAreHiddenFromSitemaps() + { + $this->assertFalse((new Redirect('foo', 'bar'))->showInSitemap()); + } } diff --git a/packages/framework/tests/Feature/RobotsTxtGeneratorTest.php b/packages/framework/tests/Feature/RobotsTxtGeneratorTest.php new file mode 100644 index 00000000000..5f73afb21c1 --- /dev/null +++ b/packages/framework/tests/Feature/RobotsTxtGeneratorTest.php @@ -0,0 +1,71 @@ +withoutSiteUrl(); + + $this->assertSame("User-agent: *\nAllow: /\n", $this->generate()); + } + + public function testGeneratesSitemapLineWhenSitemapFeatureIsEnabled() + { + $this->withSiteUrl(); + + $this->assertSame("User-agent: *\nAllow: /\n\nSitemap: https://example.com/sitemap.xml\n", $this->generate()); + } + + public function testOmitsSitemapLineWhenSitemapIsDisabledInConfig() + { + $this->withSiteUrl(); + config(['hyde.generate_sitemap' => false]); + + $this->assertSame("User-agent: *\nAllow: /\n", $this->generate()); + } + + public function testGeneratesDisallowRulesFromConfig() + { + $this->withoutSiteUrl(); + config(['hyde.robots.disallow' => ['/private', '/admin']]); + + $this->assertSame("User-agent: *\nDisallow: /private\nDisallow: /admin\n", $this->generate()); + } + + public function testDisallowRulesAreWrittenVerbatim() + { + $this->withoutSiteUrl(); + config(['hyde.robots.disallow' => ['/*.pdf$', '']]); + + $this->assertSame("User-agent: *\nDisallow: /*.pdf$\nDisallow: \n", $this->generate()); + } + + public function testNumericDisallowRulesAreCastToStrings() + { + $this->withoutSiteUrl(); + config(['hyde.robots.disallow' => [123, 3.14]]); + + $this->assertSame("User-agent: *\nDisallow: 123\nDisallow: 3.14\n", $this->generate()); + } + + public function testGeneratesDisallowRulesAndSitemapLineTogether() + { + $this->withSiteUrl(); + config(['hyde.robots.disallow' => ['/private']]); + + $this->assertSame("User-agent: *\nDisallow: /private\n\nSitemap: https://example.com/sitemap.xml\n", $this->generate()); + } + + protected function generate(): string + { + return (new RobotsTxtGenerator())->generate(); + } +} diff --git a/packages/framework/tests/Feature/RobotsTxtPageTest.php b/packages/framework/tests/Feature/RobotsTxtPageTest.php new file mode 100644 index 00000000000..2e99f5ff310 --- /dev/null +++ b/packages/framework/tests/Feature/RobotsTxtPageTest.php @@ -0,0 +1,163 @@ +assertTrue(Routes::exists('robots.txt')); + + $page = Routes::get('robots.txt')->getPage(); + + $this->assertSame(InMemoryPage::class, $page::class); + $this->assertSame('robots.txt', $page->getOutputPath()); + $this->assertSame($page::outputPath($page->getIdentifier()), $page->getOutputPath()); + $this->assertSame('robots.txt', $page->getRouteKey()); + } + + public function testRobotsTxtPageIsRegisteredWithoutSiteUrl() + { + $this->withoutSiteUrl(); + + $this->assertTrue(Routes::exists('robots.txt')); + } + + public function testRobotsTxtPageIsNotRegisteredWhenDisabledInConfig() + { + config(['hyde.robots.enabled' => false]); + + $this->assertFalse(Routes::exists('robots.txt')); + } + + public function testRobotsTxtPageIsHiddenFromNavigationAndExcludedFromTheSitemap() + { + $page = Routes::get('robots.txt')->getPage(); + + $this->assertFalse($page->showInNavigation()); + $this->assertFalse($page->showInSitemap()); + } + + public function testRobotsTxtPageCompilesUsingTheRobotsTxtGenerator() + { + $this->withoutSiteUrl(); + + $this->assertSame("User-agent: *\nAllow: /\n", Routes::get('robots.txt')->getPage()->compile()); + } + + public function testRobotsTxtGeneratorCanBeSwappedThroughTheServiceContainer() + { + app()->bind(RobotsTxtGenerator::class, fn (): RobotsTxtGenerator => new class extends RobotsTxtGenerator + { + public function generate(): string + { + return 'custom generator output'; + } + }); + + $this->assertSame('custom generator output', Routes::get('robots.txt')->getPage()->compile()); + } + + public function testBuildCommandCompilesRobotsTxtPageAsDynamicPage() + { + $this->withoutSiteUrl(); + + $this->artisan('build') + ->expectsOutput('Creating Dynamic Pages...') + ->assertExitCode(0); + + $this->assertSame("User-agent: *\nAllow: /\n", file_get_contents(Hyde::path('_site/robots.txt'))); + } + + public function testBuiltRobotsTxtLinksToSitemapAndIsExcludedFromIt() + { + $this->withSiteUrl(); + + $this->artisan('build')->assertExitCode(0); + + $this->assertStringContainsString('Sitemap: https://example.com/sitemap.xml', file_get_contents(Hyde::path('_site/robots.txt'))); + $this->assertStringNotContainsString('robots.txt', file_get_contents(Hyde::path('_site/sitemap.xml'))); + } + + public function testRobotsTxtPageIsIncludedInTheBuildManifest() + { + $this->artisan('build')->assertExitCode(0); + + $manifest = json_decode(file_get_contents(Hyde::path('app/storage/framework/cache/build-manifest.json')), true); + + $this->assertArrayHasKey('robots.txt', $manifest['pages']); + $this->assertSame('robots.txt', $manifest['pages']['robots.txt']['output_path']); + } + + public function testRobotsTxtRouteIsIncludedInTheRouteList() + { + $this->artisan('route:list') + ->expectsOutputToContain('robots.txt') + ->assertExitCode(0); + } + + public function testUserPageRegisteredInBootingCallbackSuppressesTheGeneratedRobotsTxtPage() + { + Hyde::kernel()->booting(function (HydeKernel $kernel): void { + $kernel->pages()->addPage(InMemoryPage::make('robots.txt', contents: 'user defined robots')); + }); + + $page = Routes::get('robots.txt')->getPage(); + + $this->assertSame('user defined robots', $page->compile()); + $this->assertSame(1, Hyde::pages()->filter(fn ($page) => $page->getRouteKey() === 'robots.txt')->count()); + + $this->artisan('build')->assertExitCode(0); + + $this->assertSame('user defined robots', file_get_contents(Hyde::path('_site/robots.txt'))); + } + + public function testUserPageRegisteredThroughExtensionSuppressesTheGeneratedRobotsTxtPage() + { + Hyde::kernel()->registerExtension(RobotsTxtPageTestExtension::class); + + $page = Routes::get('robots.txt')->getPage(); + + $this->assertSame('extension defined robots', $page->compile()); + $this->assertSame(1, Hyde::pages()->filter(fn ($page) => $page->getRouteKey() === 'robots.txt')->count()); + + $this->artisan('build')->assertExitCode(0); + + $this->assertSame('extension defined robots', file_get_contents(Hyde::path('_site/robots.txt'))); + } +} + +class RobotsTxtPageTestExtension extends HydeExtension +{ + public function discoverPages(PageCollection $collection): void + { + $collection->addPage(InMemoryPage::make('robots.txt', contents: 'extension defined robots')); + } +} diff --git a/packages/framework/tests/Feature/RouteCollectionTest.php b/packages/framework/tests/Feature/RouteCollectionTest.php index d36be1209b5..0e75ad34344 100644 --- a/packages/framework/tests/Feature/RouteCollectionTest.php +++ b/packages/framework/tests/Feature/RouteCollectionTest.php @@ -22,6 +22,13 @@ #[\PHPUnit\Framework\Attributes\CoversClass(\Hyde\Foundation\Facades\Routes::class)] class RouteCollectionTest extends TestCase { + protected function setUp(): void + { + parent::setUp(); + + config(['hyde.generate_sitemap' => false, 'hyde.rss.enabled' => false, 'hyde.robots.enabled' => false, 'hyde.llms.enabled' => false]); + } + public function testBootMethodDiscoversAllPages() { $collection = RouteCollection::init(Hyde::getInstance())->boot(); diff --git a/packages/framework/tests/Feature/RssFeedPageTest.php b/packages/framework/tests/Feature/RssFeedPageTest.php new file mode 100644 index 00000000000..9931e624509 --- /dev/null +++ b/packages/framework/tests/Feature/RssFeedPageTest.php @@ -0,0 +1,200 @@ +withSiteUrl(); + $this->file('_posts/hello-world.md', "# Hello, World!\n\nThis is the first post."); + } + + protected function tearDown(): void + { + File::cleanDirectory(Hyde::path('_site')); + + parent::tearDown(); + } + + public function testFeedPageIsRegisteredAsRouteWhenRssFeatureIsEnabled() + { + $this->assertTrue(Routes::exists('feed.xml')); + + $page = Routes::get('feed.xml')->getPage(); + + $this->assertSame(InMemoryPage::class, $page::class); + $this->assertSame('feed.xml', $page->getOutputPath()); + $this->assertSame($page::outputPath($page->getIdentifier()), $page->getOutputPath()); + $this->assertSame('feed.xml', $page->getRouteKey()); + } + + public function testFeedPageIsNotRegisteredWithoutSiteUrl() + { + $this->withoutSiteUrl(); + + $this->assertFalse(Routes::exists('feed.xml')); + } + + public function testFeedPageIsNotRegisteredWhenThereAreNoPosts() + { + Filesystem::unlink('_posts/hello-world.md'); + + $this->assertFalse(Routes::exists('feed.xml')); + } + + public function testFeedPageIsNotRegisteredWhenRssIsDisabledInConfig() + { + config(['hyde.rss.enabled' => false]); + + $this->assertFalse(Routes::exists('feed.xml')); + } + + public function testFeedPageUsesConfiguredFilenameAsRouteKey() + { + config(['hyde.rss.filename' => 'blog.xml']); + + $this->assertFalse(Routes::exists('feed.xml')); + $this->assertTrue(Routes::exists('blog.xml')); + $this->assertSame('blog.xml', Routes::get('blog.xml')->getPage()->getOutputPath()); + } + + public function testFeedPageUsesConfiguredFilenameVerbatimForAnyExtension() + { + config(['hyde.rss.filename' => 'feed.rss']); + + $this->assertTrue(Routes::exists('feed.rss')); + $this->assertSame('feed.rss', Routes::get('feed.rss')->getPage()->getOutputPath()); + $this->assertSame('feed.rss', InMemoryPage::outputPath('feed.rss')); + } + + public function testFeedPageIsHiddenFromNavigationAndExcludesItselfFromTheSitemap() + { + $page = Routes::get('feed.xml')->getPage(); + + $this->assertFalse($page->showInNavigation()); + $this->assertFalse($page->showInSitemap()); + } + + public function testFeedPageCompilesUsingTheRssFeedGenerator() + { + $contents = Routes::get('feed.xml')->getPage()->compile(); + + $this->assertStringStartsWith('', $contents); + $this->assertStringContainsString('assertStringContainsString('version="2.0"', $contents); + $this->assertStringContainsString('Hello, World!', $contents); + } + + public function testRssFeedGeneratorCanBeSwappedThroughTheServiceContainer() + { + app()->bind(RssFeedGenerator::class, fn (): RssFeedGenerator => new class extends RssFeedGenerator + { + public function generate(): static + { + return $this; + } + + public function getXml(): string + { + return 'custom generator output'; + } + }); + + $this->assertSame('custom generator output', Routes::get('feed.xml')->getPage()->compile()); + } + + public function testBuildCommandCompilesFeedPageAsDynamicPage() + { + $this->artisan('build') + ->expectsOutput('Creating Dynamic Pages...') + ->assertExitCode(0); + + $contents = file_get_contents(Hyde::path('_site/feed.xml')); + + $this->assertStringStartsWith('', $contents); + $this->assertStringContainsString('assertStringContainsString('version="2.0"', $contents); + + $this->assertStringNotContainsString('feed.xml', file_get_contents(Hyde::path('_site/sitemap.xml'))); + } + + public function testFeedPageIsIncludedInTheBuildManifest() + { + $this->artisan('build')->assertExitCode(0); + + $manifest = json_decode(file_get_contents(Hyde::path('app/storage/framework/cache/build-manifest.json')), true); + + $this->assertArrayHasKey('feed.xml', $manifest['pages']); + $this->assertSame('feed.xml', $manifest['pages']['feed.xml']['output_path']); + } + + public function testFeedRouteIsIncludedInTheRouteList() + { + $this->artisan('route:list') + ->expectsOutputToContain('feed.xml') + ->assertExitCode(0); + } + + public function testUserPageRegisteredInBootingCallbackSuppressesTheGeneratedFeedPage() + { + Hyde::kernel()->booting(function (HydeKernel $kernel): void { + $kernel->pages()->addPage(InMemoryPage::make('feed.xml', contents: 'user defined feed')); + }); + + $page = Routes::get('feed.xml')->getPage(); + + $this->assertSame('user defined feed', $page->compile()); + $this->assertSame(1, Hyde::pages()->filter(fn ($page) => $page->getRouteKey() === 'feed.xml')->count()); + + $this->artisan('build')->assertExitCode(0); + + $this->assertSame('user defined feed', file_get_contents(Hyde::path('_site/feed.xml'))); + } + + public function testUserPageRegisteredThroughExtensionSuppressesTheGeneratedFeedPage() + { + Hyde::kernel()->registerExtension(RssFeedPageTestExtension::class); + + $page = Routes::get('feed.xml')->getPage(); + + $this->assertSame('extension defined feed', $page->compile()); + $this->assertSame(1, Hyde::pages()->filter(fn ($page) => $page->getRouteKey() === 'feed.xml')->count()); + + $this->artisan('build')->assertExitCode(0); + + $this->assertSame('extension defined feed', file_get_contents(Hyde::path('_site/feed.xml'))); + } +} + +class RssFeedPageTestExtension extends HydeExtension +{ + public function discoverPages(PageCollection $collection): void + { + $collection->addPage(InMemoryPage::make('feed.xml', contents: 'extension defined feed')); + } +} diff --git a/packages/framework/tests/Feature/Services/BuildTaskServiceTest.php b/packages/framework/tests/Feature/Services/BuildTaskServiceTest.php index 2026767dcbb..9dc0f97e423 100644 --- a/packages/framework/tests/Feature/Services/BuildTaskServiceTest.php +++ b/packages/framework/tests/Feature/Services/BuildTaskServiceTest.php @@ -19,8 +19,6 @@ #[\PHPUnit\Framework\Attributes\CoversClass(\Hyde\Framework\Features\BuildTasks\BuildTask::class)] #[\PHPUnit\Framework\Attributes\CoversClass(\Hyde\Framework\Features\BuildTasks\PreBuildTask::class)] #[\PHPUnit\Framework\Attributes\CoversClass(\Hyde\Framework\Features\BuildTasks\PostBuildTask::class)] -#[\PHPUnit\Framework\Attributes\CoversClass(\Hyde\Framework\Actions\PostBuildTasks\GenerateSitemap::class)] -#[\PHPUnit\Framework\Attributes\CoversClass(\Hyde\Framework\Actions\PostBuildTasks\GenerateRssFeed::class)] #[\PHPUnit\Framework\Attributes\CoversClass(\Hyde\Console\Commands\BuildSiteCommand::class)] class BuildTaskServiceTest extends TestCase { @@ -30,10 +28,10 @@ public function testBuildCommandCanRunBuildTasks() $this->artisan('build') ->expectsOutputToContain('Removing all files from build directory') - ->expectsOutputToContain('Generating sitemap') - ->expectsOutputToContain('Created _site/sitemap.xml') ->assertExitCode(0); + $this->assertFileExists(Hyde::path('_site/sitemap.xml')); + File::cleanDirectory(Hyde::path('_site')); } diff --git a/packages/framework/tests/Feature/Services/SitemapServiceTest.php b/packages/framework/tests/Feature/Services/SitemapServiceTest.php index 276cfee5d22..8c02c5afe1d 100644 --- a/packages/framework/tests/Feature/Services/SitemapServiceTest.php +++ b/packages/framework/tests/Feature/Services/SitemapServiceTest.php @@ -10,8 +10,11 @@ use Hyde\Facades\Filesystem; use Hyde\Framework\Features\XmlGenerators\SitemapGenerator; use Hyde\Hyde; +use Hyde\Pages\InMemoryPage; +use Hyde\Support\Models\Route; use Hyde\Testing\TestCase; use Hyde\Foundation\HydeKernel; +use Hyde\Foundation\Facades\Routes; use Illuminate\Support\Facades\File; #[\PHPUnit\Framework\Attributes\CoversClass(\Hyde\Framework\Features\XmlGenerators\SitemapGenerator::class)] @@ -94,18 +97,64 @@ public function testGenerateAddsDocumentationPagesToXml() $this->restoreDocumentationSearch(); } - public function test_generate_adds_documentation_search_pages_to_xml() + public function testGenerateAddsDocumentationSearchPageButNotSearchIndexToXml() { Filesystem::touch('_docs/foo.md'); $service = new SitemapGenerator(); $service->generate(); - $this->assertCount(5, $service->getXmlElement()->url); + $this->assertCount(4, $service->getXmlElement()->url); + $this->assertStringContainsString('docs/search.html', $service->getXml()); + $this->assertStringNotContainsString('search.json', $service->getXml()); Filesystem::unlink('_docs/foo.md'); } + public function testGenerateDoesNotAddPagesWithSitemapFrontMatterSetToFalse() + { + $this->file('_pages/foo.md', "---\nsitemap: false\n---\n\n# Foo"); + + $service = new SitemapGenerator(); + $service->generate(); + + $this->assertCount(2, $service->getXmlElement()->url); + $this->assertStringNotContainsString('foo', $service->getXml()); + } + + public function testGenerateDoesNotAddPagesWithSitemapFrontMatterSetToQuotedFalseString() + { + $this->file('_pages/foo.md', "---\nsitemap: \"false\"\n---\n\n# Foo"); + + $service = new SitemapGenerator(); + $service->generate(); + + $this->assertCount(2, $service->getXmlElement()->url); + $this->assertStringNotContainsString('foo', $service->getXml()); + } + + public function testGenerateDoesNotAddPagesWithNonHtmlOutputPaths() + { + Routes::addRoute(new Route(InMemoryPage::make('robots.txt'))); + + $service = new SitemapGenerator(); + $service->generate(); + + $this->assertCount(2, $service->getXmlElement()->url); + $this->assertStringNotContainsString('robots.txt', $service->getXml()); + } + + public function testGenerateAddsNonHtmlPagesWithSitemapFrontMatterSetToTrue() + { + Routes::addRoute(new Route(InMemoryPage::make('robots.txt', ['sitemap' => true]))); + + $service = new SitemapGenerator(); + $service->generate(); + + $this->assertCount(3, $service->getXmlElement()->url); + $this->assertStringContainsString('robots.txt', $service->getXml()); + } + public function testGetXmlReturnsXmlString() { $service = new SitemapGenerator(); diff --git a/packages/framework/tests/Feature/SitemapFeatureTest.php b/packages/framework/tests/Feature/SitemapFeatureTest.php index f2a55e04aca..2792a979f58 100644 --- a/packages/framework/tests/Feature/SitemapFeatureTest.php +++ b/packages/framework/tests/Feature/SitemapFeatureTest.php @@ -20,7 +20,6 @@ * @see \Hyde\Framework\Testing\Feature\Commands\BuildSitemapCommandTest */ #[\PHPUnit\Framework\Attributes\CoversClass(\Hyde\Framework\Features\XmlGenerators\SitemapGenerator::class)] -#[\PHPUnit\Framework\Attributes\CoversClass(\Hyde\Framework\Actions\PostBuildTasks\GenerateSitemap::class)] #[\PHPUnit\Framework\Attributes\CoversClass(\Hyde\Console\Commands\BuildSitemapCommand::class)] class SitemapFeatureTest extends TestCase { @@ -36,7 +35,7 @@ public function testTheSitemapFeature() $this->withSiteUrl(); $this->artisan('build:sitemap') - ->expectsOutputToContain('Created _site/sitemap.xml') + ->expectsOutputToContain('Created [_site/sitemap.xml]') ->assertExitCode(0); $this->assertFileExists('_site/sitemap.xml'); @@ -50,6 +49,7 @@ public function testTheSitemapFeature() protected function setUpBroadSiteStructure(): void { $this->file('_pages/about.md', "# About\n\nThis is the about page."); + $this->file('_pages/secret.md', "---\nsitemap: false\n---\n\n# Secret\n\nThis page is excluded from the sitemap."); $this->file('_pages/contact.html', '

Contact

This is the contact page.

'); $this->file('_posts/hello-world.md', "# Hello, World!\n\nThis is the first post."); $this->file('_posts/second-post.md', "# Second Post\n\nThis is the second post."); @@ -123,12 +123,6 @@ protected function expected(string $version): string daily 0.9 - - https://example.com/docs/search.json - 2024-01-01T12:00:00+00:00 - weekly - 0.5 - https://example.com/docs/search.html 2024-01-01T12:00:00+00:00 diff --git a/packages/framework/tests/Feature/SitemapPageTest.php b/packages/framework/tests/Feature/SitemapPageTest.php new file mode 100644 index 00000000000..29cfe3ee1ad --- /dev/null +++ b/packages/framework/tests/Feature/SitemapPageTest.php @@ -0,0 +1,181 @@ +withSiteUrl(); + + $this->assertTrue(Routes::exists('sitemap.xml')); + + $page = Routes::get('sitemap.xml')->getPage(); + + $this->assertSame(InMemoryPage::class, $page::class); + $this->assertSame('sitemap.xml', $page->getOutputPath()); + $this->assertSame($page::outputPath($page->getIdentifier()), $page->getOutputPath()); + $this->assertSame('sitemap.xml', $page->getRouteKey()); + } + + public function testSitemapPageIsNotRegisteredWithoutSiteUrl() + { + $this->withoutSiteUrl(); + + $this->assertFalse(Routes::exists('sitemap.xml')); + } + + public function testSitemapPageIsNotRegisteredWhenSitemapIsDisabledInConfig() + { + $this->withSiteUrl(); + config(['hyde.generate_sitemap' => false]); + + $this->assertFalse(Routes::exists('sitemap.xml')); + } + + public function testSitemapPageIsHiddenFromNavigationAndExcludesItselfFromTheSitemap() + { + $this->withSiteUrl(); + + $page = Routes::get('sitemap.xml')->getPage(); + + $this->assertFalse($page->showInNavigation()); + $this->assertFalse($page->showInSitemap()); + } + + public function testSitemapPageCompilesUsingTheSitemapGenerator() + { + $this->withSiteUrl(); + + $contents = Routes::get('sitemap.xml')->getPage()->compile(); + + $this->assertStringStartsWith('', $contents); + $this->assertStringContainsString('withSiteUrl(); + + app()->bind(SitemapGenerator::class, fn (): SitemapGenerator => new class extends SitemapGenerator + { + public function generate(): static + { + return $this; + } + + public function getXml(): string + { + return 'custom generator output'; + } + }); + + $this->assertSame('custom generator output', Routes::get('sitemap.xml')->getPage()->compile()); + } + + public function testBuildCommandCompilesSitemapPageAsDynamicPage() + { + $this->withSiteUrl(); + + $this->artisan('build') + ->expectsOutput('Creating Dynamic Pages...') + ->assertExitCode(0); + + $contents = file_get_contents(Hyde::path('_site/sitemap.xml')); + + $this->assertStringStartsWith('', $contents); + $this->assertStringContainsString('assertStringNotContainsString('sitemap.xml', $contents); + } + + public function testSitemapPageIsIncludedInTheBuildManifest() + { + $this->withSiteUrl(); + + $this->artisan('build')->assertExitCode(0); + + $manifest = json_decode(file_get_contents(Hyde::path('app/storage/framework/cache/build-manifest.json')), true); + + $this->assertArrayHasKey('sitemap.xml', $manifest['pages']); + $this->assertSame('sitemap.xml', $manifest['pages']['sitemap.xml']['output_path']); + } + + public function testSitemapRouteIsIncludedInTheRouteList() + { + $this->withSiteUrl(); + + $this->artisan('route:list') + ->expectsOutputToContain('sitemap.xml') + ->assertExitCode(0); + } + + public function testUserPageRegisteredInBootingCallbackSuppressesTheGeneratedSitemapPage() + { + $this->withSiteUrl(); + + Hyde::kernel()->booting(function (HydeKernel $kernel): void { + $kernel->pages()->addPage(InMemoryPage::make('sitemap.xml', contents: 'user defined sitemap')); + }); + + $page = Routes::get('sitemap.xml')->getPage(); + + $this->assertSame('user defined sitemap', $page->compile()); + $this->assertSame(1, Hyde::pages()->filter(fn ($page) => $page->getRouteKey() === 'sitemap.xml')->count()); + + $this->artisan('build')->assertExitCode(0); + + $this->assertSame('user defined sitemap', file_get_contents(Hyde::path('_site/sitemap.xml'))); + } + + public function testUserPageRegisteredThroughExtensionSuppressesTheGeneratedSitemapPage() + { + $this->withSiteUrl(); + + Hyde::kernel()->registerExtension(SitemapPageTestExtension::class); + + $page = Routes::get('sitemap.xml')->getPage(); + + $this->assertSame('extension defined sitemap', $page->compile()); + $this->assertSame(1, Hyde::pages()->filter(fn ($page) => $page->getRouteKey() === 'sitemap.xml')->count()); + + $this->artisan('build')->assertExitCode(0); + + $this->assertSame('extension defined sitemap', file_get_contents(Hyde::path('_site/sitemap.xml'))); + } +} + +class SitemapPageTestExtension extends HydeExtension +{ + public function discoverPages(PageCollection $collection): void + { + $collection->addPage(InMemoryPage::make('sitemap.xml', contents: 'extension defined sitemap')); + } +} diff --git a/packages/framework/tests/Feature/StaticSiteServiceTest.php b/packages/framework/tests/Feature/StaticSiteServiceTest.php index b510423ddec..b4065fbe67c 100644 --- a/packages/framework/tests/Feature/StaticSiteServiceTest.php +++ b/packages/framework/tests/Feature/StaticSiteServiceTest.php @@ -145,6 +145,8 @@ public function testAllPageTypesCanBeCompiled() public function testOnlyProgressBarsForTypesWithPagesAreShown() { + config(['hyde.generate_sitemap' => false, 'hyde.robots.enabled' => false, 'hyde.llms.enabled' => false]); + $this->file('_pages/blade.blade.php'); $this->file('_pages/markdown.md'); @@ -191,9 +193,9 @@ public function testSitemapIsNotGeneratedWhenConditionsAreNotMet() $this->withoutSiteUrl(); config(['hyde.generate_sitemap' => false]); - $this->artisan('build') - ->doesntExpectOutput('Generating sitemap...') - ->assertExitCode(0); + $this->artisan('build')->assertExitCode(0); + + $this->assertFileDoesNotExist(Hyde::path('_site/sitemap.xml')); } public function testSitemapIsGeneratedWhenConditionsAreMet() @@ -201,9 +203,10 @@ public function testSitemapIsGeneratedWhenConditionsAreMet() $this->withSiteUrl(); config(['hyde.generate_sitemap' => true]); - $this->artisan('build') - // ->expectsOutput('Generating sitemap...') - ->assertExitCode(0); + $this->artisan('build')->assertExitCode(0); + + $this->assertFileExists(Hyde::path('_site/sitemap.xml')); + Filesystem::unlink('_site/sitemap.xml'); } @@ -212,9 +215,9 @@ public function testRssFeedIsNotGeneratedWhenConditionsAreNotMet() $this->withoutSiteUrl(); config(['hyde.rss.enabled' => false]); - $this->artisan('build') - ->doesntExpectOutput('Generating RSS feed...') - ->assertExitCode(0); + $this->artisan('build')->assertExitCode(0); + + $this->assertFileDoesNotExist(Hyde::path('_site/feed.xml')); } public function testRssFeedIsGeneratedWhenConditionsAreMet() @@ -224,9 +227,9 @@ public function testRssFeedIsGeneratedWhenConditionsAreMet() Filesystem::touch('_posts/foo.md'); - $this->artisan('build') - // ->expectsOutput('Generating RSS feed...') - ->assertExitCode(0); + $this->artisan('build')->assertExitCode(0); + + $this->assertFileExists(Hyde::path('_site/feed.xml')); Filesystem::unlink('_posts/foo.md'); Filesystem::unlink('_site/feed.xml'); diff --git a/packages/framework/tests/Unit/BuildTaskServiceUnitTest.php b/packages/framework/tests/Unit/BuildTaskServiceUnitTest.php index a5fbfb79d7d..5ef62223993 100644 --- a/packages/framework/tests/Unit/BuildTaskServiceUnitTest.php +++ b/packages/framework/tests/Unit/BuildTaskServiceUnitTest.php @@ -7,9 +7,8 @@ use Closure; use Hyde\Foundation\HydeKernel; use Hyde\Foundation\Kernel\Filesystem; -use Hyde\Framework\Actions\PostBuildTasks\GenerateBuildManifest; -use Hyde\Framework\Actions\PostBuildTasks\GenerateRssFeed; -use Hyde\Framework\Actions\PostBuildTasks\GenerateSitemap as FrameworkGenerateSitemap; +use Hyde\Framework\Actions\PostBuildTasks\GenerateBuildManifest as FrameworkGenerateBuildManifest; +use Hyde\Framework\Actions\PreBuildTasks\TransferMediaAssets; use Hyde\Framework\Features\BuildTasks\BuildTask; use Hyde\Framework\Features\BuildTasks\PostBuildTask; use Hyde\Framework\Features\BuildTasks\PreBuildTask; @@ -136,16 +135,16 @@ public function testRegisterTaskWithTaskAlreadyRegisteredInConfig() public function testCanRegisterFrameworkTasks() { - $this->service->registerTask(FrameworkGenerateSitemap::class); - $this->assertSame([FrameworkGenerateSitemap::class], $this->service->getRegisteredTasks()); + $this->service->registerTask(FrameworkGenerateBuildManifest::class); + $this->assertSame([FrameworkGenerateBuildManifest::class], $this->service->getRegisteredTasks()); } public function testCanOverloadFrameworkTasks() { - $this->service->registerTask(FrameworkGenerateSitemap::class); - $this->service->registerTask(GenerateSitemap::class); + $this->service->registerTask(FrameworkGenerateBuildManifest::class); + $this->service->registerTask(GenerateBuildManifest::class); - $this->assertSame([GenerateSitemap::class], $this->service->getRegisteredTasks()); + $this->assertSame([GenerateBuildManifest::class], $this->service->getRegisteredTasks()); } public function testCanSetOutputWithNull() @@ -160,17 +159,7 @@ public function testCanSetOutputWithOutputStyle() public function testGenerateBuildManifestExtendsPostBuildTask() { - $this->assertInstanceOf(PostBuildTask::class, new GenerateBuildManifest()); - } - - public function testGenerateRssFeedExtendsPostBuildTask() - { - $this->assertInstanceOf(PostBuildTask::class, new GenerateRssFeed()); - } - - public function testGenerateSitemapExtendsPostBuildTask() - { - $this->assertInstanceOf(PostBuildTask::class, new FrameworkGenerateSitemap()); + $this->assertInstanceOf(PostBuildTask::class, new FrameworkGenerateBuildManifest()); } public function testCanRunPreBuildTasks() @@ -281,8 +270,8 @@ public function testServiceSearchesForTasksInAppDirectory() public function testServiceFindsTasksInAppDirectory() { $files = [ - 'app/Actions/GenerateBuildManifestBuildTask.php' => GenerateBuildManifest::class, - 'app/Actions/GenerateRssFeedBuildTask.php' => GenerateRssFeed::class, + 'app/Actions/GenerateBuildManifestBuildTask.php' => FrameworkGenerateBuildManifest::class, + 'app/Actions/TransferMediaAssetsBuildTask.php' => TransferMediaAssets::class, ]; $this->mockKernelFilesystem($files); @@ -291,7 +280,7 @@ public function testServiceFindsTasksInAppDirectory() $this->assertSame([ 'Hyde\Framework\Actions\PostBuildTasks\GenerateBuildManifest', - 'Hyde\Framework\Actions\PostBuildTasks\GenerateRssFeed', + 'Hyde\Framework\Actions\PreBuildTasks\TransferMediaAssets', ], $this->service->getRegisteredTasks()); $this->resetKernelInstance(); @@ -362,7 +351,7 @@ class TestBuildTaskNotExtendingChildren extends BuildTask } /** Test class to test overloading */ -class GenerateSitemap extends FrameworkGenerateSitemap +class GenerateBuildManifest extends FrameworkGenerateBuildManifest { use VoidHandleMethod; } diff --git a/packages/framework/tests/Unit/ExtensionsUnitTest.php b/packages/framework/tests/Unit/ExtensionsUnitTest.php index d1666295977..966612fd4d1 100644 --- a/packages/framework/tests/Unit/ExtensionsUnitTest.php +++ b/packages/framework/tests/Unit/ExtensionsUnitTest.php @@ -275,7 +275,7 @@ class HydeExtensionTestPage extends HydePage { public static string $sourceDirectory = 'foo'; public static string $outputDirectory = 'foo'; - public static string $fileExtension = '.txt'; + public static string $sourceExtension = '.txt'; public function compile(): string { diff --git a/packages/framework/tests/Unit/GenerateBuildManifestTest.php b/packages/framework/tests/Unit/GenerateBuildManifestTest.php index d8410dcc853..bc1b3b4e5ea 100644 --- a/packages/framework/tests/Unit/GenerateBuildManifestTest.php +++ b/packages/framework/tests/Unit/GenerateBuildManifestTest.php @@ -20,6 +20,8 @@ class GenerateBuildManifestTest extends UnitTestCase public function testActionGeneratesBuildManifest() { + self::mockConfig(['hyde.robots.enabled' => false, 'hyde.llms.enabled' => false]); + Hyde::pages()->addPage(new DocumentationSearchIndex()); (new GenerateBuildManifest())->handle(); diff --git a/packages/framework/tests/Unit/NavigationDataFactoryUnitTest.php b/packages/framework/tests/Unit/NavigationDataFactoryUnitTest.php index 5c6e8a5da94..d04d78c170d 100644 --- a/packages/framework/tests/Unit/NavigationDataFactoryUnitTest.php +++ b/packages/framework/tests/Unit/NavigationDataFactoryUnitTest.php @@ -247,6 +247,32 @@ public function testPageIsHiddenBasedOnNavigationConfiguration() $this->assertFalse($factory->makeHidden()); } + public function testNonHtmlPagesAreHiddenFromNavigationByDefault() + { + $factory = new NavigationConfigTestClass($this->makeCoreDataObject( + routeKey: 'feed.xml', + outputPath: 'feed.xml', + )); + + $this->assertTrue($factory->makeHidden()); + } + + public function testNonHtmlPagesCanBeExplicitlyShownInNavigation() + { + foreach ([ + ['navigation.visible' => true], + ['navigation.hidden' => false], + ] as $matter) { + $factory = new NavigationConfigTestClass($this->makeCoreDataObject( + routeKey: 'feed.xml', + outputPath: 'feed.xml', + matter: $matter, + )); + + $this->assertFalse($factory->makeHidden()); + } + } + public function testPageIsHiddenBasedOnSidebarConfigurationForDocumentationPage() { self::mockConfig(['docs.sidebar.exclude' => ['hiddenDocPage']]); @@ -374,9 +400,14 @@ public function testFrontMatterValueOverridesFilenamePrefixPriority() $this->assertSame(10, $factory->makePriority()); } - protected function makeCoreDataObject(string $identifier = '', string $routeKey = '', string $pageClass = MarkdownPage::class): CoreDataObject - { - return new CoreDataObject(new FrontMatter(), new Markdown(), $pageClass, $identifier, '', '', $routeKey); + protected function makeCoreDataObject( + string $identifier = '', + string $routeKey = '', + string $pageClass = MarkdownPage::class, + string $outputPath = 'page.html', + array $matter = [], + ): CoreDataObject { + return new CoreDataObject(new FrontMatter($matter), new Markdown(), $pageClass, $identifier, '', $outputPath, $routeKey); } } diff --git a/packages/framework/tests/Unit/NumericalPageOrderingHelperUnitTest.php b/packages/framework/tests/Unit/NumericalPageOrderingHelperUnitTest.php index f317b106524..28f113c77aa 100644 --- a/packages/framework/tests/Unit/NumericalPageOrderingHelperUnitTest.php +++ b/packages/framework/tests/Unit/NumericalPageOrderingHelperUnitTest.php @@ -127,9 +127,9 @@ public function testIdentifiersForDeeplyNestedPagesWithoutNumericalPrefixesAreNo #[\PHPUnit\Framework\Attributes\DataProvider('pageTypeProvider')] public function testIdentifiersWithNumericalPrefixesAreDetectedForPageType(string $type) { - $this->assertTrue(NumericalPageOrderingHelper::hasNumericalPrefix('01-home.'.$type::$fileExtension)); - $this->assertTrue(NumericalPageOrderingHelper::hasNumericalPrefix('02-about.'.$type::$fileExtension)); - $this->assertTrue(NumericalPageOrderingHelper::hasNumericalPrefix('03-contact.'.$type::$fileExtension)); + $this->assertTrue(NumericalPageOrderingHelper::hasNumericalPrefix('01-home.'.$type::$sourceExtension)); + $this->assertTrue(NumericalPageOrderingHelper::hasNumericalPrefix('02-about.'.$type::$sourceExtension)); + $this->assertTrue(NumericalPageOrderingHelper::hasNumericalPrefix('03-contact.'.$type::$sourceExtension)); } /** @@ -138,9 +138,9 @@ public function testIdentifiersWithNumericalPrefixesAreDetectedForPageType(strin #[\PHPUnit\Framework\Attributes\DataProvider('pageTypeProvider')] public function testIdentifiersWithoutNumericalPrefixesAreNotDetectedForPageType(string $type) { - $this->assertFalse(NumericalPageOrderingHelper::hasNumericalPrefix('home.'.$type::$fileExtension)); - $this->assertFalse(NumericalPageOrderingHelper::hasNumericalPrefix('about.'.$type::$fileExtension)); - $this->assertFalse(NumericalPageOrderingHelper::hasNumericalPrefix('contact.'.$type::$fileExtension)); + $this->assertFalse(NumericalPageOrderingHelper::hasNumericalPrefix('home.'.$type::$sourceExtension)); + $this->assertFalse(NumericalPageOrderingHelper::hasNumericalPrefix('about.'.$type::$sourceExtension)); + $this->assertFalse(NumericalPageOrderingHelper::hasNumericalPrefix('contact.'.$type::$sourceExtension)); } /** @@ -149,9 +149,9 @@ public function testIdentifiersWithoutNumericalPrefixesAreNotDetectedForPageType #[\PHPUnit\Framework\Attributes\DataProvider('pageTypeProvider')] public function testIdentifiersWithNumericalPrefixesAreDetectedWhenUsingSnakeCaseDelimitersForPageType(string $type) { - $this->assertTrue(NumericalPageOrderingHelper::hasNumericalPrefix('01_home.'.$type::$fileExtension)); - $this->assertTrue(NumericalPageOrderingHelper::hasNumericalPrefix('02_about.'.$type::$fileExtension)); - $this->assertTrue(NumericalPageOrderingHelper::hasNumericalPrefix('03_contact.'.$type::$fileExtension)); + $this->assertTrue(NumericalPageOrderingHelper::hasNumericalPrefix('01_home.'.$type::$sourceExtension)); + $this->assertTrue(NumericalPageOrderingHelper::hasNumericalPrefix('02_about.'.$type::$sourceExtension)); + $this->assertTrue(NumericalPageOrderingHelper::hasNumericalPrefix('03_contact.'.$type::$sourceExtension)); } /** @@ -160,9 +160,9 @@ public function testIdentifiersWithNumericalPrefixesAreDetectedWhenUsingSnakeCas #[\PHPUnit\Framework\Attributes\DataProvider('pageTypeProvider')] public function testSplitNumericPrefixForDeeplyNestedPagesForPageType(string $type) { - $this->assertSame([1, 'foo/bar/home.'.$type::$fileExtension], NumericalPageOrderingHelper::splitNumericPrefix('foo/bar/01-home.'.$type::$fileExtension)); - $this->assertSame([2, 'foo/bar/about.'.$type::$fileExtension], NumericalPageOrderingHelper::splitNumericPrefix('foo/bar/02-about.'.$type::$fileExtension)); - $this->assertSame([3, 'foo/bar/contact.'.$type::$fileExtension], NumericalPageOrderingHelper::splitNumericPrefix('foo/bar/03-contact.'.$type::$fileExtension)); + $this->assertSame([1, 'foo/bar/home.'.$type::$sourceExtension], NumericalPageOrderingHelper::splitNumericPrefix('foo/bar/01-home.'.$type::$sourceExtension)); + $this->assertSame([2, 'foo/bar/about.'.$type::$sourceExtension], NumericalPageOrderingHelper::splitNumericPrefix('foo/bar/02-about.'.$type::$sourceExtension)); + $this->assertSame([3, 'foo/bar/contact.'.$type::$sourceExtension], NumericalPageOrderingHelper::splitNumericPrefix('foo/bar/03-contact.'.$type::$sourceExtension)); } public function testSplitNumericPrefixForDeeplyNestedPages() diff --git a/packages/framework/tests/Unit/Pages/BladePageUnitTest.php b/packages/framework/tests/Unit/Pages/BladePageUnitTest.php index 0e8ea4b2884..742b9d763dd 100644 --- a/packages/framework/tests/Unit/Pages/BladePageUnitTest.php +++ b/packages/framework/tests/Unit/Pages/BladePageUnitTest.php @@ -31,9 +31,14 @@ public function testBaseRouteKey() $this->assertSame('', BladePage::baseRouteKey()); } - public function testFileExtension() + public function testSourceExtension() { - $this->assertSame('.blade.php', BladePage::fileExtension()); + $this->assertSame('.blade.php', BladePage::sourceExtension()); + } + + public function testOutputExtension() + { + $this->assertSame('.html', BladePage::outputExtension()); } public function testSourcePath() @@ -81,6 +86,12 @@ public function testShowInNavigation() $this->assertTrue((new BladePage())->showInNavigation()); } + public function testShowInSitemap() + { + $this->assertTrue((new BladePage())->showInSitemap()); + $this->assertFalse((new BladePage('foo', ['sitemap' => false]))->showInSitemap()); + } + public function testNavigationMenuPriority() { $this->assertSame(999, (new BladePage())->navigationMenuPriority()); diff --git a/packages/framework/tests/Unit/Pages/DocumentationPageUnitTest.php b/packages/framework/tests/Unit/Pages/DocumentationPageUnitTest.php index 3e37ce76cb4..791ef32b70d 100644 --- a/packages/framework/tests/Unit/Pages/DocumentationPageUnitTest.php +++ b/packages/framework/tests/Unit/Pages/DocumentationPageUnitTest.php @@ -33,9 +33,14 @@ public function testBaseRouteKey() $this->assertSame('docs', DocumentationPage::baseRouteKey()); } - public function testFileExtension() + public function testSourceExtension() { - $this->assertSame('.md', DocumentationPage::fileExtension()); + $this->assertSame('.md', DocumentationPage::sourceExtension()); + } + + public function testOutputExtension() + { + $this->assertSame('.html', DocumentationPage::outputExtension()); } public function testSourcePath() @@ -83,6 +88,12 @@ public function testShowInNavigation() $this->assertTrue((new DocumentationPage())->showInNavigation()); } + public function testShowInSitemap() + { + $this->assertTrue((new DocumentationPage())->showInSitemap()); + $this->assertFalse((new DocumentationPage('foo', ['sitemap' => false]))->showInSitemap()); + } + public function testNavigationMenuPriority() { $this->assertSame(999, (new DocumentationPage())->navigationMenuPriority()); diff --git a/packages/framework/tests/Unit/Pages/HtmlPageUnitTest.php b/packages/framework/tests/Unit/Pages/HtmlPageUnitTest.php index 0838e934a41..d8bab24b266 100644 --- a/packages/framework/tests/Unit/Pages/HtmlPageUnitTest.php +++ b/packages/framework/tests/Unit/Pages/HtmlPageUnitTest.php @@ -40,11 +40,19 @@ public function testBaseRouteKey() ); } - public function testFileExtension() + public function testSourceExtension() { $this->assertSame( '.html', - HtmlPage::fileExtension() + HtmlPage::sourceExtension() + ); + } + + public function testOutputExtension() + { + $this->assertSame( + '.html', + HtmlPage::outputExtension() ); } @@ -114,6 +122,12 @@ public function testShowInNavigation() $this->assertTrue((new HtmlPage())->showInNavigation()); } + public function testShowInSitemap() + { + $this->assertTrue((new HtmlPage())->showInSitemap()); + $this->assertFalse((new HtmlPage('foo', ['sitemap' => false]))->showInSitemap()); + } + public function testNavigationMenuPriority() { $this->assertSame(999, (new HtmlPage())->navigationMenuPriority()); diff --git a/packages/framework/tests/Unit/Pages/InMemoryPageTest.php b/packages/framework/tests/Unit/Pages/InMemoryPageTest.php index 2e6c2312cc2..d27fd1f8578 100644 --- a/packages/framework/tests/Unit/Pages/InMemoryPageTest.php +++ b/packages/framework/tests/Unit/Pages/InMemoryPageTest.php @@ -397,6 +397,29 @@ public function getContents(): string $this->assertSame('view', $page->compile()); $this->assertSame(0, $page->invocations); } + + public function testOutputPathInfersFormatFromIdentifier() + { + $this->assertSame('foo.html', InMemoryPage::outputPath('foo')); + $this->assertSame('robots.txt', InMemoryPage::outputPath('robots.txt')); + $this->assertSame('data.json', InMemoryPage::outputPath('data.json')); + $this->assertSame('sitemap.xml', InMemoryPage::outputPath('sitemap.xml')); + $this->assertSame('docs/search.json', InMemoryPage::outputPath('docs/search.json')); + $this->assertSame('foo.md', InMemoryPage::outputPath('foo.md')); + $this->assertSame('foo.html', InMemoryPage::outputPath('foo.html')); + $this->assertSame('docs/1.x', InMemoryPage::outputPath('docs/1.x')); + $this->assertSame('docs/1.x/index.html', InMemoryPage::outputPath('docs/1.x/index')); + } + + public function testStaticAndInstanceOutputPathsUseTheSameSemantics() + { + foreach (['foo', 'robots.txt', 'docs/search.json', 'docs/1.x/index'] as $identifier) { + $this->assertSame( + InMemoryPage::outputPath($identifier), + (new InMemoryPage($identifier))->getOutputPath() + ); + } + } } class InMemoryPageContentTestPage extends InMemoryPage diff --git a/packages/framework/tests/Unit/Pages/InMemoryPageUnitTest.php b/packages/framework/tests/Unit/Pages/InMemoryPageUnitTest.php index ed76a546fca..505663f37cc 100644 --- a/packages/framework/tests/Unit/Pages/InMemoryPageUnitTest.php +++ b/packages/framework/tests/Unit/Pages/InMemoryPageUnitTest.php @@ -44,11 +44,19 @@ public function testBaseRouteKey() ); } - public function testFileExtension() + public function testSourceExtension() { $this->assertSame( '', - InMemoryPage::fileExtension() + InMemoryPage::sourceExtension() + ); + } + + public function testOutputExtension() + { + $this->assertSame( + '.html', + InMemoryPage::outputExtension() ); } @@ -118,6 +126,12 @@ public function testShowInNavigation() $this->assertTrue((new InMemoryPage('foo'))->showInNavigation()); } + public function testShowInSitemap() + { + $this->assertTrue((new InMemoryPage('foo'))->showInSitemap()); + $this->assertFalse((new InMemoryPage('foo', ['sitemap' => false]))->showInSitemap()); + } + public function testNavigationMenuPriority() { $this->assertSame(999, (new InMemoryPage('foo'))->navigationMenuPriority()); diff --git a/packages/framework/tests/Unit/Pages/MarkdownPageUnitTest.php b/packages/framework/tests/Unit/Pages/MarkdownPageUnitTest.php index 2401f58e84f..8b335b43845 100644 --- a/packages/framework/tests/Unit/Pages/MarkdownPageUnitTest.php +++ b/packages/framework/tests/Unit/Pages/MarkdownPageUnitTest.php @@ -42,11 +42,19 @@ public function testBaseRouteKey() ); } - public function testFileExtension() + public function testSourceExtension() { $this->assertSame( '.md', - MarkdownPage::fileExtension() + MarkdownPage::sourceExtension() + ); + } + + public function testOutputExtension() + { + $this->assertSame( + '.html', + MarkdownPage::outputExtension() ); } @@ -116,6 +124,12 @@ public function testShowInNavigation() $this->assertTrue((new MarkdownPage())->showInNavigation()); } + public function testShowInSitemap() + { + $this->assertTrue((new MarkdownPage())->showInSitemap()); + $this->assertFalse((new MarkdownPage('foo', ['sitemap' => false]))->showInSitemap()); + } + public function testNavigationMenuPriority() { $this->assertSame(999, (new MarkdownPage())->navigationMenuPriority()); diff --git a/packages/framework/tests/Unit/Pages/MarkdownPostUnitTest.php b/packages/framework/tests/Unit/Pages/MarkdownPostUnitTest.php index bd72a040fa8..c8c30195dbe 100644 --- a/packages/framework/tests/Unit/Pages/MarkdownPostUnitTest.php +++ b/packages/framework/tests/Unit/Pages/MarkdownPostUnitTest.php @@ -42,11 +42,19 @@ public function testBaseRouteKey() ); } - public function testFileExtension() + public function testSourceExtension() { $this->assertSame( '.md', - MarkdownPost::fileExtension() + MarkdownPost::sourceExtension() + ); + } + + public function testOutputExtension() + { + $this->assertSame( + '.html', + MarkdownPost::outputExtension() ); } @@ -116,6 +124,12 @@ public function testShowInNavigation() $this->assertFalse((new MarkdownPost())->showInNavigation()); } + public function testShowInSitemap() + { + $this->assertTrue((new MarkdownPost())->showInSitemap()); + $this->assertFalse((new MarkdownPost('foo', ['sitemap' => false]))->showInSitemap()); + } + public function testNavigationMenuPriority() { $this->assertSame(10, (new MarkdownPost())->navigationMenuPriority()); diff --git a/packages/framework/tests/Unit/RouteKeyTest.php b/packages/framework/tests/Unit/RouteKeyTest.php index 0eb00c91316..906224ea9c7 100644 --- a/packages/framework/tests/Unit/RouteKeyTest.php +++ b/packages/framework/tests/Unit/RouteKeyTest.php @@ -9,6 +9,7 @@ use Hyde\Pages\InMemoryPage; use Hyde\Pages\MarkdownPage; use Hyde\Pages\MarkdownPost; +use Hyde\Pages\Concerns\HydePage; use Hyde\Pages\DocumentationPage; use Hyde\Support\Models\RouteKey; use Hyde\Testing\UnitTestCase; @@ -81,6 +82,29 @@ public function testFromPageWithInMemoryPage() $this->assertEquals(new RouteKey('foo/bar'), RouteKey::fromPage(InMemoryPage::class, 'foo/bar')); } + public function testFromPageWithDottedInMemoryPageIdentifier() + { + $this->assertEquals(new RouteKey('robots.txt'), RouteKey::fromPage(InMemoryPage::class, 'robots.txt')); + $this->assertEquals(new RouteKey('docs/search.json'), RouteKey::fromPage(InMemoryPage::class, 'docs/search.json')); + } + + public function testFromPageWithNonHtmlOutputExtensionIncludesExtensionInRouteKey() + { + $this->assertEquals(new RouteKey('foo.txt'), RouteKey::fromPage(NonHtmlOutputPageStub::class, 'foo')); + $this->assertEquals(new RouteKey('foo/bar.txt'), RouteKey::fromPage(NonHtmlOutputPageStub::class, 'foo/bar')); + } + + public function testFromPageWithNonHtmlOutputExtensionDoesNotDuplicateExtensionAlreadyInIdentifier() + { + $this->assertEquals(new RouteKey('foo.txt'), RouteKey::fromPage(NonHtmlOutputPageStub::class, 'foo.txt')); + } + + public function testFromPageWithNonHtmlOutputExtensionAndEmptyIdentifierAppendsExtensionToOutputDirectory() + { + $this->assertEquals(new RouteKey('feed.xml'), RouteKey::fromPage(NonHtmlOutputDirectoryPageStub::class, '')); + $this->assertEquals(new RouteKey('feed/episode.xml'), RouteKey::fromPage(NonHtmlOutputDirectoryPageStub::class, 'episode')); + } + public function testFromPageWithCustomOutputDirectory() { MarkdownPage::setOutputDirectory('foo'); @@ -136,3 +160,24 @@ public function testItDoesNotExtractNonNumericalFilenamePrefixes() $this->assertSame('docs/abc-bar', RouteKey::fromPage(DocumentationPage::class, 'abc-bar')->get()); } } + +class NonHtmlOutputPageStub extends HydePage +{ + public static string $outputExtension = '.txt'; + + public function compile(): string + { + return ''; + } +} + +class NonHtmlOutputDirectoryPageStub extends HydePage +{ + public static string $outputDirectory = 'feed'; + public static string $outputExtension = '.xml'; + + public function compile(): string + { + return ''; + } +} diff --git a/packages/realtime-compiler/src/Http/DashboardController.php b/packages/realtime-compiler/src/Http/DashboardController.php index 5a46b46bcf1..64f5ce2c55b 100644 --- a/packages/realtime-compiler/src/Http/DashboardController.php +++ b/packages/realtime-compiler/src/Http/DashboardController.php @@ -467,8 +467,8 @@ protected function formatPageIdentifier(string $title): string { $title = trim($title, '/\\'); - if (str_ends_with(strtolower($title), HtmlPage::fileExtension())) { - $title = substr($title, 0, -strlen(HtmlPage::fileExtension())); + if (str_ends_with(strtolower($title), HtmlPage::sourceExtension())) { + $title = substr($title, 0, -strlen(HtmlPage::sourceExtension())); } $directory = str_contains($title, '/') diff --git a/packages/realtime-compiler/src/Routing/Router.php b/packages/realtime-compiler/src/Routing/Router.php index 315e6bf3eaf..4638194660b 100644 --- a/packages/realtime-compiler/src/Routing/Router.php +++ b/packages/realtime-compiler/src/Routing/Router.php @@ -19,9 +19,6 @@ class Router protected Request $request; - protected bool $assetPathResolved = false; - protected ?string $resolvedAssetPath = null; - public function __construct(Request $request) { $this->request = $request; @@ -29,7 +26,9 @@ public function __construct(Request $request) public function handle(): Response { - if ($this->shouldProxy()) { + // Media files are always static assets, so we proxy them + // directly without paying for booting the application. + if (str_starts_with($this->request->path, '/media/')) { return $this->proxyStatic(); } @@ -43,41 +42,16 @@ public function handle(): Response return $virtualRoutes[$this->request->path]($this->request); } - // A path with a file extension that matches neither a static file nor a page (like a - // missing stylesheet or source map) is a missing asset, and not a missing web page, - // so we send a normal 404 response instead of the pretty page not found error. + // A path with a file extension that isn't a web page is a static asset request, + // unless a page route is registered for the path (like `docs/search.json`), + // as pages take precedence over the on-disk files the proxy serves. if ($this->hasAssetLikeExtension() && ! PageRouter::hasRoute($this->request)) { - return $this->notFound(); + return $this->proxyStatic(); } return PageRouter::handle($this->request); } - /** - * If the request is not for a web page, we assume it's - * a static asset, which we instead want to proxy. - */ - protected function shouldProxy(): bool - { - // Always proxy media files. This condition is just to improve performance - // without having to check the file extension. - if (str_starts_with($this->request->path, '/media/')) { - return true; - } - - if (! $this->hasAssetLikeExtension()) { - return false; - } - - // Don't proxy the search.json file, as it's generated on the fly. - if (str_ends_with($this->request->path, 'search.json')) { - return false; - } - - // Dotted page routes (like documentation version folders) are proxied only when a matching asset exists. - return $this->resolveAssetPath() !== null; - } - /** * Does the request path have an extension that isn't a web page? * @@ -90,16 +64,6 @@ protected function hasAssetLikeExtension(): bool return $extension !== null && $extension !== 'html'; } - protected function resolveAssetPath(): ?string - { - if (! $this->assetPathResolved) { - $this->resolvedAssetPath = AssetFileLocator::find($this->request->path); - $this->assetPathResolved = true; - } - - return $this->resolvedAssetPath; - } - /** * Override the configured site URL so compiled pages reference the local * server instead of the production URL. Without this, assets such as @@ -182,7 +146,7 @@ protected function getConfiguredServerHost(string $scheme): string */ protected function proxyStatic(): Response { - $path = $this->resolveAssetPath(); + $path = AssetFileLocator::find($this->request->path); if ($path === null) { return $this->notFound(); diff --git a/packages/realtime-compiler/tests/RealtimeCompilerTest.php b/packages/realtime-compiler/tests/RealtimeCompilerTest.php index 20822b305cd..af1f320f7e9 100644 --- a/packages/realtime-compiler/tests/RealtimeCompilerTest.php +++ b/packages/realtime-compiler/tests/RealtimeCompilerTest.php @@ -216,6 +216,65 @@ public function testErrorThrownWhileCompilingExistingDottedPageIsNotSentAsA404() } } + public function testServesRegisteredPageRouteEvenWhenMatchingAssetExists() + { + $this->mockCompilerRoute('9.x'); + + Filesystem::ensureDirectoryExists('_pages/9.x'); + Filesystem::put('_pages/9.x/index.md', '# Hello World!'); + Filesystem::put('_media/9.x', 'static decoy'); + + try { + $kernel = new HttpKernel(); + $response = $kernel->handle(new Request()); + + $this->assertSame(200, $response->statusCode); + $this->assertStringContainsString('Hello World!', $response->body); + $this->assertStringNotContainsString('static decoy', $response->body); + } finally { + Filesystem::deleteDirectory('_pages/9.x'); + Filesystem::unlink('_media/9.x'); + } + } + + public function testDocsSearchJsonRouteWinsOverMatchingAssetFile() + { + $this->mockCompilerRoute('docs/search.json'); + + Filesystem::put('_docs/index.md', '# Hello World!'); + Filesystem::ensureDirectoryExists('_media/docs'); + Filesystem::put('_media/docs/search.json', '"static decoy"'); + + try { + $kernel = new HttpKernel(); + $response = $kernel->handle(new Request()); + + $this->assertSame(200, $response->statusCode); + $this->assertNotSame('"static decoy"', $response->body); + $this->assertIsArray(json_decode($response->body, true)); + } finally { + Filesystem::unlink('_docs/index.md'); + Filesystem::deleteDirectory('_media/docs'); + } + } + + public function testProxiesRootLevelAssetWhenNoRouteMatchesThePath() + { + $this->mockCompilerRoute('data.json'); + + Filesystem::put('_media/data.json', '{"static": true}'); + + try { + $kernel = new HttpKernel(); + $response = $kernel->handle(new Request()); + + $this->assertSame(200, $response->statusCode); + $this->assertSame('{"static": true}', $response->body); + } finally { + Filesystem::unlink('_media/data.json'); + } + } + public function testTrailingSlashesAreNormalizedFromRoute() { $this->mockCompilerRoute('foo/'); @@ -288,6 +347,121 @@ public function testDocsSearchJsonRendersSearchIndexWithJsonContentType() Filesystem::unlink('_docs/index.md'); } + public function testVersionedDocsSearchJsonRendersSearchIndexWithJsonContentType() + { + $this->mockCompilerRoute('docs/1.x/search.json'); + Filesystem::ensureDirectoryExists('_docs/1.x'); + Filesystem::put('_docs/1.x/index.md', '# Hello World!'); + + try { + $router = new Router(new Request()); + + $this->bootRouterApplication($router); + config(['docs.versions' => ['1.x']]); + Hyde::boot(); + + $response = $router->handle(); + + $this->assertInstanceOf(Response::class, $response); + $this->assertNotInstanceOf(HtmlResponse::class, $response); + $this->assertSame(200, $response->statusCode); + $this->assertSame('OK', $response->statusMessage); + + $headers = $this->getResponseHeaders($response); + $this->assertSame('application/json', $headers['Content-Type']); + $this->assertSame((string) strlen($response->body), $headers['Content-Length']); + + $this->assertIsArray(json_decode($response->body, true)); + } finally { + Filesystem::deleteDirectory('_docs/1.x'); + } + } + + public function testSitemapXmlRouteIsServedWithXmlContentType() + { + config(['hyde.url' => 'https://example.com']); + + $this->mockCompilerRoute('sitemap.xml'); + + $kernel = new HttpKernel(); + $response = $kernel->handle(new Request()); + + $this->assertInstanceOf(Response::class, $response); + $this->assertNotInstanceOf(HtmlResponse::class, $response); + $this->assertSame(200, $response->statusCode); + $this->assertSame('OK', $response->statusMessage); + + $headers = $this->getResponseHeaders($response); + $this->assertSame('application/xml', $headers['Content-Type']); + + $this->assertStringStartsWith('', $response->body); + $this->assertStringContainsString('body); + } + + public function testRssFeedRouteIsServedWithXmlContentType() + { + config(['hyde.url' => 'https://example.com']); + + $this->mockCompilerRoute('feed.xml'); + + Filesystem::put('_posts/rc-test-post.md', '# Hello World!'); + + try { + $kernel = new HttpKernel(); + $response = $kernel->handle(new Request()); + + $this->assertInstanceOf(Response::class, $response); + $this->assertNotInstanceOf(HtmlResponse::class, $response); + $this->assertSame(200, $response->statusCode); + $this->assertSame('OK', $response->statusMessage); + + $headers = $this->getResponseHeaders($response); + $this->assertSame('application/xml', $headers['Content-Type']); + + $this->assertStringStartsWith('', $response->body); + $this->assertStringContainsString('body); + } finally { + Filesystem::unlink('_posts/rc-test-post.md'); + } + } + + public function testRobotsTxtRouteIsServedWithPlainTextContentType() + { + $this->mockCompilerRoute('robots.txt'); + + $kernel = new HttpKernel(); + $response = $kernel->handle(new Request()); + + $this->assertInstanceOf(Response::class, $response); + $this->assertNotInstanceOf(HtmlResponse::class, $response); + $this->assertSame(200, $response->statusCode); + $this->assertSame('OK', $response->statusMessage); + + $headers = $this->getResponseHeaders($response); + $this->assertSame('text/plain', $headers['Content-Type']); + + $this->assertSame("User-agent: *\nAllow: /\n\nSitemap: http://localhost:8080/sitemap.xml\n", $response->body); + } + + public function testLlmsTxtRouteIsServedWithPlainTextContentType() + { + $this->mockCompilerRoute('llms.txt'); + + $kernel = new HttpKernel(); + $response = $kernel->handle(new Request()); + + $this->assertInstanceOf(Response::class, $response); + $this->assertNotInstanceOf(HtmlResponse::class, $response); + $this->assertSame(200, $response->statusCode); + $this->assertSame('OK', $response->statusMessage); + + $headers = $this->getResponseHeaders($response); + $this->assertSame('text/plain', $headers['Content-Type']); + + $this->assertStringStartsWith('# HydePHP', $response->body); + $this->assertStringContainsString('http://localhost:8080/', $response->body); + } + public function testGetContentTypeReturnsApplicationJsonForJsonOutputPath() { $page = $this->makePageWithOutputPath('foo.json'); @@ -590,6 +764,13 @@ protected function invokeOverrideSiteUrl(): void $method->invoke(new Router(new Request())); } + protected function bootRouterApplication(Router $router): void + { + $method = new ReflectionMethod(Router::class, 'bootApplication'); + $method->setAccessible(true); + $method->invoke($router); + } + protected function invokeGetContentType(InMemoryPage $page): string { $this->mockCompilerRoute('foo'); diff --git a/packages/testing/src/Common/BaseHydePageUnitTest.php b/packages/testing/src/Common/BaseHydePageUnitTest.php index a02b84be84f..47b3791e329 100644 --- a/packages/testing/src/Common/BaseHydePageUnitTest.php +++ b/packages/testing/src/Common/BaseHydePageUnitTest.php @@ -92,6 +92,8 @@ abstract public function testGetRoute(); abstract public function testShowInNavigation(); + abstract public function testShowInSitemap(); + abstract public function testGetSourcePath(); abstract public function testGetLink(); @@ -102,7 +104,9 @@ abstract public function testHas(); abstract public function testToCoreDataObject(); - abstract public function testFileExtension(); + abstract public function testSourceExtension(); + + abstract public function testOutputExtension(); abstract public function testSourceDirectory();