diff --git a/components/engine/engine-intent/CLAUDE.md b/components/engine/engine-intent/CLAUDE.md index 055e696ff02..6794e85a5d7 100644 --- a/components/engine/engine-intent/CLAUDE.md +++ b/components/engine/engine-intent/CLAUDE.md @@ -423,7 +423,7 @@ Semantics worth knowing: - **`trigger: { onCreate|onUpdate|onDelete: , when: "" }` starts the process on the named `` lifecycle event** - fully wired (Java). Any of the three events is supported: `onCreate` binds the entity's base topic, `onUpdate`/`onDelete` the `-updated`/`-deleted` topics the Java DAO publishes (`TriggerSupport` + `EventBinding`); an optional `when` guard (a single `field ==|!= literal`, via `NotificationSupport.guard`) gates `Process.start`. Three parts: (1) the parser validates at most one event kind and that the target is a declared entity; (2) the EDM generator adds a `ProcessId` back-reference property (VARCHAR) plus the per-process `ProcessIds` stamps column (VARCHAR, `Process=instanceId` pairs) to that entity and a `triggers` collection to the `.model` (`TriggerSupport` + `EdmIntentGenerator.buildTriggers`); (3) the **`template-application-events-java`** template (intent-driven, like the other language templates) reads that `triggers` collection and emits one **`gen/events//Trigger.java`** per trigger - a client-Java self-describing `MessageHandler` (a `@Component` whose `destination()` is the entity's per-operation topic via `topicSuffix` and whose `kind()` is `TOPIC`) that loads the entity, applies the `when` guard, calls `Process.start(, businessKey, )`, and writes the instance id back to `ProcessIds` against its own process name plus `ProcessId` (so THIS process starts at most once - one `ProcessId` cannot say WHICH process ran, and reading it as "some process ran" silently skipped every follow-up flow on an already-stamped record, #6862). **The write-back is crash-safe by construction (#6815):** the per-process stamp in `ProcessIds` IS the at-most-once guard while the start and the write-back commit independently, so everything that can precede the start does — the minted business key is persisted first, and every process variable (the `__entityUrl`/`__entityId` locators, the FK locators, `__personalUser`) rides the start payload instead of a post-start `setVariable` (all are known up front, and a wait-state-less process finishes inside `start`, where a `setVariable` would then throw). The one remaining post-start step is the targeted `updateProperties` of both process columns at once (a record carrying one without the other is either invisible to the task UI or blocked from ever starting the flow again), which a `checks:` gate can no longer refuse (the generated repository runs `enforceChecks` only for a write that touches an **authored** column - a gate has no opinion about which process handles the document, and by then the instance is running), a swallowed start (`null` id) is logged rather than written, and if the write still does not land — the row was deleted meanwhile, or it threw — the instance is **cancelled** (`Process.cancel`) and the failure re-thrown, rather than left running with nothing pointing at it. The Java DAO template (`template-application-dao-java`) now publishes the create event (`Producer.sendToTopic('${projectName}-${perspectiveName}-${name}', json)`) the way the TS DAO does - that's the topic the handler binds to. `gen/events/` (the `` segment = the sanitized intent name, `IntentNaming.javaModule`) is a sibling of `gen/`, so it survives the per-model regeneration wipe. The events template iterates the model's `triggers` via a new **`triggers` collection case in the generation pipeline's `ModelGenerator`** (the engine's collection switch is hardcoded; the case has its own loop because triggers are not entity-shaped). The BPM **business key** defaults to the entity's primary key but is **configurable**: `trigger: { ..., businessKey: }` names which trigger-entity field becomes the started instance's business key (the listener still loads the entity by its PK via `findById`; only the business key differs — a separate `businessKeyProperty` in `.glue`). An optional `businessKeyStrategy: timestamp` mints a `yyyyMMddHHmmss` value into that field when it is blank and persists it via the listener's existing update — the simple "for now" generator and the **extension point** for richer pluggable number generators later (sequential, zero-padded, config-prefixed invoice numbers); the parser validates the field exists, the strategy is supported, and (for `timestamp`) the field is `string`/`text`. `TriggerSupport.triggerBusinessKey`/`triggerBusinessKeyStrategy` read them; `GlueIntentGenerator` emits `businessKeyProperty` + `generateBusinessKey`; `Trigger.java.template` renders the mint-if-blank block. `onSchedule` is still unmodelled. **Casing subtlety in the generated handler:** its `import gen..data..{Entity,Repository}` must use the **lowercased** Java package segment (`javaPerspective` = `sanitizeJavaIdentifier(perspective)`, matching the DAO/entity templates' `javaPerspectiveName` folder), while the `destination()` topic (`"--"`) keeps the **raw** perspective so it matches the topic the DAO publishes to (`${projectName}-${perspectiveName}-${name}`). The `triggers` collection case in the pipeline supplies both (`javaPerspective` for the import, `perspective` for the topic). Using the raw perspective in the import compiled on macOS (case-insensitive FS) but failed `javac` with "package gen.x.data.Member does not exist" because the entity files declare the lowercased package. - **A trigger's stamp is trusted only while the engine knows the instance it names, and a stuck record is restarted without SQL (#7599).** `ProcessStamps.has` alone left a record stuck for good once its stamped instance was gone from the engine (cancelled with its deployment under the old cascade - #7597 -, lost with an ephemeral SystemDB): no Inbox task, the guard still saying "started", and `ProcessId`/`ProcessIds` system-owned so no UI or REST write could clear them. The generated trigger now reads `ProcessStamps.idFor` and asks `Process.exists(id)` (running OR in the history; with `DIRIGIBLE_FLOWABLE_HISTORY_LEVEL=none` a finished instance cannot be told from a lost one, so the stamp is trusted as before) - a dangling stamp logs a WARN and the record counts as unstarted, the new instance overwriting the stamp. Because a create event never fires twice, the trigger also implements the SDK `ProcessTrigger` (`process()`, `entity()`, `restart(String id)`), and the platform's `POST /services/bpm/bpm-processes/restart?process=&id=` (`ProcessTriggerRestartEndpoint` in `engine-java`, monitoring roles) locates the trigger bean through `ClientBeansHolder` and re-runs it: 409 while the stamped instance still runs, 404 for an unknown process or record, otherwise the new instance id with the record re-stamped. The event path and the restart share ONE `startFor(entity)` - business key, locators, `__subjectFields`, `__personalUser`, start, stamp write-back - so the two cannot drift; the trigger's `when` guard applies to both. The restart's id arrives as text, so the `.glue` trigger entry carries `keyType` and `GlueGenerator.bindTrigger` renders `keyParse` (`Integer.valueOf(id)` by default). `ProcessTriggerRestartIT` drives all of it live over an `onUpdate` trigger (trusted while running, distrusted once deleted from runtime + history, 409 / 200 / 404 through the endpoint). - **`dependsOn` on a to-one relation or a field = the EDM Depends-On feature (cascading dropdowns + auto-populated fields).** `dependsOn: { relation: , valueFrom?: , filterBy?: }` — the widget reacts to the sibling trigger: the generated form loads the trigger's selected record, reads `valueFrom` (default: the trigger target's PK), then a **relation** re-filters its dropdown options where its own target's `filterBy` (default: that target's PK) equals the value (`POST /search` with an EQ condition; a single remaining option auto-selects), while a **field** copies the value (auto-population; `valueFrom` mandatory, `filterBy` rejected). Emitted by `EdmIntentGenerator.putDependsOn` as the four scalar `widgetDependsOn*` property attributes the AngularJS stacks already consume (so those work for free); the Harmonia runtime was added in the same pass (`form-page.js.template` per-property watcher + `applyDependsOn` methods covering manage/master-detail/allocation forms; `document-page.js.template` header watchers + a generic metadata-driven `applyDraftDependsOn` for the line-item dialog off `detail-register.js.template`'s `editColumns[].dependsOn`, with filtered options in a separate `draftOptions` store so the items table's label resolution keeps the full set; `ModelParameterProcessor` precomputes `widgetDependsOnControllerUrl` from the trigger sibling). `valueFrom`/`filterBy` use the target's **authored** property names (field lower-camel / relation as declared); same-model references are parse-validated, cross-model ones generation-validated against the resolved owner model (`CrossModelSupport.TargetInfo.propertyNames`). A `documentStatus` relation can neither declare nor trigger a dependsOn. Canonical cases (the `codbex-sample-model-depends-on` set): Country→City cascade (`filterBy` only), Product→UoM narrow-to-referenced (`valueFrom` only), Product→price auto-populate (field). **Conditional auto-populate (#6358):** a FIELD's `valueFrom` may be `{ by: , cases: { : }, default?: }` — the copied trigger-target property is picked by a classifier resolved from the `by` path (own property / one-hop `.` / a path starting at the composition parent relation = the open document header). Parser `validateConditionalValueFrom` (shape, path segments, case/default properties against the trigger target); EDM emits `widgetDependsOnValueBy` (+`ByHeader`/`ByHeaderEntity`/`ByEntity` for the hop fetch), `widgetDependsOnValueCases` (JSON string, PascalCased properties), `widgetDependsOnValueDefault`, and NO `widgetDependsOnValueFrom`; `ModelParameterProcessor` derives `widgetDependsOnValueByUrl` (the hop record's controller); Harmonia consumes it via `resolveDependsOnSource` (document page: dialog + header form; `resolveDependsOnSource` on the manage form) — Harmonia-only, the AngularJS `#if` guards skip it (no `ValueFrom`). Editor round-trip: the six attrs are in `model.js`/`serializer.js` (no dialog UI - intent is the source). **Header-mediated trigger (#6358, the issue's other half):** `relation: .
` on a document ITEM field (`relation: SalesInvoice.Customer, valueFrom: standardDiscount`) makes the line default from a record the open DOCUMENT points at instead of one of the line's own relations - the canonical case being a line discount defaulting from the header partner's terms. Parser `validateHeaderMediatedDependsOn` (fields only - a header selection has no option list to cascade, so `valueFrom` is mandatory and `filterBy` rejected; the first segment must be the composition parent, the second a to-one of the header, and `valueFrom` resolves against THAT relation's target). `putDependsOn` resolves the trigger through the header and adds `widgetDependsOnHeader` + `widgetDependsOnHeaderEntity`; `ModelParameterProcessor` resolves `widgetDependsOnControllerUrl` on the HEADER entity (the trigger is not a property of the item). Harmonia: `detail-register` emits `dependsOn.header`, and `document-page` gains `applyHeaderDependsOnToDraft` - called on a CREATE draft open and from a `form.` watcher while the dialog is open, never on an edit draft (the stored value may be a deliberate override); `applyDraftDependsOn`/the draft watchers explicitly skip header columns so a same-named row column cannot drive them. **Every sibling-assuming stack is guarded** (`&& !$property.widgetDependsOnHeader` in the four `-java`/`-v2`/legacy AngularJS controller templates and the Harmonia `form-page`) - without it they emit a watcher on `entity.` / `this.form.` that does not exist on the item. Composable with the conditional `valueFrom`. -- **`postings:` (top-level) = declarative posting (source-document status → generated local document + computed items).** The accounting "documents → ledger" capability, generalized (spike-derived; see the driving suite's spike findings). `PostingIntent` + parser `validatePostings` (creates = local document owning a composition items child; backReference = its to-one to the source, the at-most-once guard; event trigger `onTransition` with a mandatory `when: " == "` status guard, or `onCreate` for a source with NO status lifecycle - a booked payment - binding the `-created` topic with the `when` guard optional (#6421); item cells = `rule()` refs into a single-selector rule entity or Calc arithmetic over the source; row `when: ==|!= `). `GlueIntentGenerator.buildPostings` pre-renders EVERYTHING as Java expressions (the expansions convention — the template stays shape-only): topic + re-load coordinates via `CrossModelSupport`, guard, header assignments (copy / literal / `{placeholder}` concat), `ruleRow.` refs, `Calc.eval("", source, )` amounts with the scale from the LOCAL item field, null-safe Calc row guards. `postings` glue collection → the pipeline's collection case (source gen folder = sanitized model alias, topic keeps the RAW perspective) → `Posting.java.template`: a `MessageHandler` on `---transitioned` (#6220's channel) that re-loads the source by id (the payload lacks later-step data — the stamped number), guards, resolves the rule row (missing row / null referenced column → SKIP, the unposted worklist), and writes target + items through the repositories — so numbering / status `init:` / `checks:` fire on the created document. **Idempotent + resumable + amendable, and the post itself is ONE transaction** — the handler's own writes (the stale rows a rewrite replaces, the header, every derived line) share a `UnitOfWork`, so a line the item repository refuses leaves the previous post standing instead of a header with a partial line set (#7132: unlike the half-post case there is no second event to self-heal from, so a partial rewrite ends up worse than the stale but balanced post it set out to fix). Across STEPS the model is unchanged and deliberately not transactional — the source's own commit, this handler's post and a reversal are separate events, and a bad post is unwound by a correcting entry, not a rollback: the handler derives the WHOLE content first and compares it with the post the back-reference finds — identical is a redelivery (no-op), different is either a HALF-post (an item write failed after the target was saved) to complete or an AMENDED source to rewrite from. The amendment half is #7071: the amend path (Confirm → Reject → edit the lines → Issue again) raises the SAME moment a second time, and the old `item count ≥ expectedItems` test read that as "already posted", so the entry kept the amounts of the previous issue while the document it references had moved on — no second entry (right) and a ledger 60.00 short (wrong), silently. Now the existing post is REWRITTEN in place (header assignments re-applied through `update`, items replaced) — but only while nobody has acted on the created document, which the posting itself defines: its `function: EntityStatus` relation still holds the `init:` the posting's own create wrote (a target with no status lifecycle is always rewritable, one whose status is declared without an `init:` is rewritable while still empty). Once it has moved, the divergence is LOGGED naming both documents and the entry is left alone — unwinding a posted entry is a correcting entry's job (`reverses:`), not a silent overwrite. The comparison is order-insensitive (row order is not a query guarantee) over the union of every cell the item rows assign (`itemComparedProps`), and numbers compare by VALUE so a rescaled amount is not a change. **It is over the values as they will be STORED, not as the derived rows stand**: `save()` fills a column before the insert, so each compared property carries the default its own derived side will end up with (#7131), and a column the WRITE fills is compared accordingly (#7177, #7234) - a column filled UNCONDITIONALLY is dropped from the comparison entirely and the discarded assignment reported at generation (its value never stays in the column, so left in it is a difference no redelivery can ever clear): a `calculatedOnCreate`/`calculatedActionOnCreate` one; an `aggregate: true` header column the lines also declare, which the document master's `recalculate()` sets to the SUM over the lines on every write (#7234 - the item sum the write stored against the source value the `map:` computed, off by a rounding, a sign convention or a partially posted line set; the master is resolved through `IntentEntities.documentMasters`, the SAME rule the `MANAGE_DOCUMENT` layout and so the DAO's `documentMaster` are emitted by, never through the broader `documentItemsChild`); and, on the created document only - it is rewritten IN PLACE through `update()`, where its lines are deleted and re-inserted - a `calculatedOnUpdate`/`calculatedActionOnUpdate` column (recomputed on every rewrite) or any `aggregate`/`readOnly` one (`update()` preserves it from the stored row), whose mapped value survives the create and is discarded by every rewrite after it, so after one legitimate amendment the compared cell mismatched forever. A `uuid` or `number:` one - filled only when the row leaves it empty, like a `date`/`timestamp` default only the DATABASE can apply - is compared only for the rows that do derive it. Header `map:` expressions are hoisted into numbered locals so the comparison and the assignment read ONE evaluation, after the back-reference lookup so a return that writes nothing never pays for them; `amendableGuard`/`itemComparedProps` are pre-rendered into the glue like everything else, and both default to the pre-#7071 behaviour when a `.glue` predates them - and `bindPosting` reads the "compare only when derived" flag under its #7163 spelling `expressionDefault` wherever the `compareOnlyWhenDerived` #7188 renamed it to is absent - through ONE rule, in the header-assignment normaliser #7256 added and in `GlueGenerator.comparedCells` for the item cells - so a `.glue` generated between the two keeps its CURRENT_DATE-default treatment instead of silently falling to a plain `same()` until the intent is re-generated (#7234). Concurrent-redelivery de-duplication is best-effort (a check-then-act on the back-reference) until a real UNIQUE key on the back-reference lands with schema constraint emission. Storno/negation mode LANDED as **`reverses:`** (paired with the `transitions:` void primitive - the "void-document event" is a transition into the void status): a reversal posting inherits creates/backReference/rule/map/items from the reversed sibling, negates every item amount expression on the SAME side (`Calc.eval("-()", ...)` - red storno), locates the original through the empty `storno:` self-link (none -> fail-soft skip), stamps the link on its creation, and both handlers' idempotency guards discriminate by that link (reversal counts linked rows, the sibling counts unlinked ones - `stornoProperty`/`stornoFilterProperty` in the glue). The explicit manual Reverse action (no source void) remains a follow-up. Compensation, not a transaction, is how a bad post is unwound. **The flattened sibling `posts:` (`PostIntent`, `Posts.java.template`) is one transaction per source event too (#7179)** - every row one event derives is written inside a `UnitOfWork`, which is what makes its idempotency guard exact: the guard is the mere EXISTENCE of a back-referencing row, so written a save per transaction a refused row left the earlier ones durable, the guard read that half-post as finished, and no redelivery ever wrote the rest. **A unit of work covers the rows and their outbox events, and nothing else** - here, in the rewrite (#7132), in the scheduled tick (#7133) and in the create-from (#7069) alike: a target carrying `number:` allocates its document number before the insert and a `history: true` one writes its trail after it, each on its own connection (`JavaEntityStore.inUnitOfWork` is explicit about it), so a rolled-back unit can consume a number and record the attempt. Say "no rows", not "nothing". +- **`postings:` (top-level) = declarative posting (source-document status → generated local document + computed items).** The accounting "documents → ledger" capability, generalized (spike-derived; see the driving suite's spike findings). `PostingIntent` + parser `validatePostings` (creates = local document owning a composition items child; backReference = its to-one to the source, the at-most-once guard; event trigger `onTransition` with a mandatory `when: " == "` status guard, or `onCreate` for a source with NO status lifecycle - a booked payment - binding the `-created` topic with the `when` guard optional (#6421); item cells = `rule()` refs into a single-selector rule entity or Calc arithmetic over the source; row `when: ==|!= `). `GlueIntentGenerator.buildPostings` pre-renders EVERYTHING as Java expressions (the expansions convention — the template stays shape-only): topic + re-load coordinates via `CrossModelSupport`, guard, header assignments (copy / literal / `{placeholder}` concat), `ruleRow.` refs, `Calc.eval("", source, )` amounts with the scale from the LOCAL item field, null-safe Calc row guards. **A rule column's null gate follows the line that reads it (#7649).** `usedRuleColumns` collected every `rule()` reference of every row, so one column read only by a `when:`-guarded line gated the WHOLE posting: adding an optional line stopped every document of the type from posting on every tenant whose rule row predated the new column, the ones the line does not apply to included, and nothing was logged. `buildPostings` now accumulates per row - an unguarded row's columns join `usedRuleColumns` (it books on every document, so a null there genuinely stops the posting), a guarded row's ride on the row as `ruleColumns` and are checked inside its own guard. `GlueGenerator.conditionalRuleGuards` splits the same way for the classifier ternaries (`ruleCaseGuards` on the row), and every skip path in `Posting.java.template` now WARNs naming the rule entity, the match value and the source key. `postings` glue collection → the pipeline's collection case (source gen folder = sanitized model alias, topic keeps the RAW perspective) → `Posting.java.template`: a `MessageHandler` on `---transitioned` (#6220's channel) that re-loads the source by id (the payload lacks later-step data — the stamped number), guards, resolves the rule row (missing row / null referenced column → SKIP, the unposted worklist - **logged**, since the handler fires only on the event and a silent skip leaves nothing anywhere to explain the gap, #7649), and writes target + items through the repositories — so numbering / status `init:` / `checks:` fire on the created document. **Idempotent + resumable + amendable, and the post itself is ONE transaction** — the handler's own writes (the stale rows a rewrite replaces, the header, every derived line) share a `UnitOfWork`, so a line the item repository refuses leaves the previous post standing instead of a header with a partial line set (#7132: unlike the half-post case there is no second event to self-heal from, so a partial rewrite ends up worse than the stale but balanced post it set out to fix). Across STEPS the model is unchanged and deliberately not transactional — the source's own commit, this handler's post and a reversal are separate events, and a bad post is unwound by a correcting entry, not a rollback: the handler derives the WHOLE content first and compares it with the post the back-reference finds — identical is a redelivery (no-op), different is either a HALF-post (an item write failed after the target was saved) to complete or an AMENDED source to rewrite from. The amendment half is #7071: the amend path (Confirm → Reject → edit the lines → Issue again) raises the SAME moment a second time, and the old `item count ≥ expectedItems` test read that as "already posted", so the entry kept the amounts of the previous issue while the document it references had moved on — no second entry (right) and a ledger 60.00 short (wrong), silently. Now the existing post is REWRITTEN in place (header assignments re-applied through `update`, items replaced) — but only while nobody has acted on the created document, which the posting itself defines: its `function: EntityStatus` relation still holds the `init:` the posting's own create wrote (a target with no status lifecycle is always rewritable, one whose status is declared without an `init:` is rewritable while still empty). Once it has moved, the divergence is LOGGED naming both documents and the entry is left alone — unwinding a posted entry is a correcting entry's job (`reverses:`), not a silent overwrite. The comparison is order-insensitive (row order is not a query guarantee) over the union of every cell the item rows assign (`itemComparedProps`), and numbers compare by VALUE so a rescaled amount is not a change. **It is over the values as they will be STORED, not as the derived rows stand**: `save()` fills a column before the insert, so each compared property carries the default its own derived side will end up with (#7131), and a column the WRITE fills is compared accordingly (#7177, #7234) - a column filled UNCONDITIONALLY is dropped from the comparison entirely and the discarded assignment reported at generation (its value never stays in the column, so left in it is a difference no redelivery can ever clear): a `calculatedOnCreate`/`calculatedActionOnCreate` one; an `aggregate: true` header column the lines also declare, which the document master's `recalculate()` sets to the SUM over the lines on every write (#7234 - the item sum the write stored against the source value the `map:` computed, off by a rounding, a sign convention or a partially posted line set; the master is resolved through `IntentEntities.documentMasters`, the SAME rule the `MANAGE_DOCUMENT` layout and so the DAO's `documentMaster` are emitted by, never through the broader `documentItemsChild`); and, on the created document only - it is rewritten IN PLACE through `update()`, where its lines are deleted and re-inserted - a `calculatedOnUpdate`/`calculatedActionOnUpdate` column (recomputed on every rewrite) or any `aggregate`/`readOnly` one (`update()` preserves it from the stored row), whose mapped value survives the create and is discarded by every rewrite after it, so after one legitimate amendment the compared cell mismatched forever. A `uuid` or `number:` one - filled only when the row leaves it empty, like a `date`/`timestamp` default only the DATABASE can apply - is compared only for the rows that do derive it. Header `map:` expressions are hoisted into numbered locals so the comparison and the assignment read ONE evaluation, after the back-reference lookup so a return that writes nothing never pays for them; `amendableGuard`/`itemComparedProps` are pre-rendered into the glue like everything else, and both default to the pre-#7071 behaviour when a `.glue` predates them - and `bindPosting` reads the "compare only when derived" flag under its #7163 spelling `expressionDefault` wherever the `compareOnlyWhenDerived` #7188 renamed it to is absent - through ONE rule, in the header-assignment normaliser #7256 added and in `GlueGenerator.comparedCells` for the item cells - so a `.glue` generated between the two keeps its CURRENT_DATE-default treatment instead of silently falling to a plain `same()` until the intent is re-generated (#7234). Concurrent-redelivery de-duplication is best-effort (a check-then-act on the back-reference) until a real UNIQUE key on the back-reference lands with schema constraint emission. Storno/negation mode LANDED as **`reverses:`** (paired with the `transitions:` void primitive - the "void-document event" is a transition into the void status): a reversal posting inherits creates/backReference/rule/map/items from the reversed sibling, negates every item amount expression on the SAME side (`Calc.eval("-()", ...)` - red storno), locates the original through the empty `storno:` self-link (none -> fail-soft skip), stamps the link on its creation, and both handlers' idempotency guards discriminate by that link (reversal counts linked rows, the sibling counts unlinked ones - `stornoProperty`/`stornoFilterProperty` in the glue). The explicit manual Reverse action (no source void) remains a follow-up. Compensation, not a transaction, is how a bad post is unwound. **The flattened sibling `posts:` (`PostIntent`, `Posts.java.template`) is one transaction per source event too (#7179)** - every row one event derives is written inside a `UnitOfWork`, which is what makes its idempotency guard exact: the guard is the mere EXISTENCE of a back-referencing row, so written a save per transaction a refused row left the earlier ones durable, the guard read that half-post as finished, and no redelivery ever wrote the rest. **A unit of work covers the rows and their outbox events, and nothing else** - here, in the rewrite (#7132), in the scheduled tick (#7133) and in the create-from (#7069) alike: a target carrying `number:` allocates its document number before the insert and a `history: true` one writes its trail after it, each on its own connection (`JavaEntityStore.inUnitOfWork` is explicit about it), so a rolled-back unit can consume a number and record the attempt. Say "no rows", not "nothing". - **Lifecycle-aware aggregates: seed-row `stage:` + report `scope:` + symbolic status names (#6645).** An aggregate over an entity carrying a `function: EntityStatus` was **wrong by default** - drafts nobody had issued, cancelled and voided (анулиране) rows all landed in the sum unless the author remembered a magic-number status predicate in `filter:`, and nothing said so (the motivating case: a voided invoice kept its 2000 in "Revenue this month" because the report declared dimensions + measures and no `filter`, so the emitted query had no `WHERE` at all). Four coordinated pieces, all in `LifecycleStages` + `ReportIntentGenerator.scopePredicate` + `StatusSymbolResolver`: (1) a status **seed row** classifies what the status MEANS with a closed-vocabulary `stage: draft|live|cancelled|void` - metadata, never a column (the CSV generator only emits declared fields + referenced FKs, and `CsvimIntentGeneratorTest` pins that); (2) a report declares `scope: all` or a stage name, emitted as `."" IN ()` ANDed onto the filter; (3) with the nomenclature classified, an **aggregating** report **defaults to `live`** - but only when its dimensions/`filter` do not already reference the status (a breakdown BY status must keep its draft rows, and an authored predicate is authoritative), so an existing model is byte-identical until it adopts `stage:`; (4) every site that names a status accepts the **seeded name** (`from: [ISSUED]`, `setStatus: VOIDED`, `init: DRAFT`, `setRelationField` `value:`, `abortOn.status`, a check's `status`/`setStatus`, `immutableWhen`, a posting's `event.when`, the `event.when` of a `notifications`/`integrations`/`outbound` entry ([#7289](https://github.com/eclipse-dirigible/dirigible/issues/7289)), a report's `filter`, and the status condition of a `where` row query - a `schedules[]` one ([#7251](https://github.com/eclipse-dirigible/dirigible/issues/7251)) or a create-from's `items:` rule ([#7091](https://github.com/eclipse-dirigible/dirigible/issues/7091)), both through the shared `StatusSymbolResolver.rewriteConditions`, each on the QUERIED entity's own nomenclature) - resolved on the **raw YAML tree before the typed Gson mapping** (the `rejectRemovedNumberKeys` precedent), so every validator, generator and template keeps seeing plain integers. **Why names matter more than they look:** an id is positional, so inserting a status mid-nomenclature shifts every later id and silently retargets every guard authored against the old numbering - that is how a `reverses:` posting guarded `when: "Status == 8"` stopped matching a Void that now writes 9, leaving the ledger with a receivable for a document that no longer existed, with well-formed Java emitted throughout. **Boundaries, deliberate:** the nomenclature must be seeded IN THIS MODEL - the parser holds one file and no repository, so a **cross-model** status can neither be stage-scoped nor named (both fail loudly naming the numeric-id fallback; cross-model symbols need the name→id map on the generated `.model` and are follow-up work). A cross-model **row query** is the one site where the parser cannot even say so — which of its `{ field, op, value }` triples names the status is knowable only from the owner's `.model` — so the refusal is made where that model is read, at generation: a condition on the owner's `DOCUMENT_STATUS` property whose value is not an integer is a 422 naming the relation, the name, the owner model and the id-only rule, for a create-from's `items:` rule ([#7225](https://github.com/eclipse-dirigible/dirigible/issues/7225)) and for `schedules[].where` ([#7288](https://github.com/eclipse-dirigible/dirigible/issues/7288)) alike — both through the shared `GlueIntentGenerator.crossModelStatusName`. Left silent, the schedule one was #7251's own failure mode one `model:` key away: `.eq("Status", "OVERDUE")` against an integer FK, matching nothing forever. A symbolic **ordering** comparison (`Status >= ISSUED`) is rejected - names have no order, that is what `scope:` is for. A nomenclature that declares its own `stage` property collides with the marker and is rejected rather than guessed. Nothing is emitted into the `.model` for `stage` - no consumer needs it yet (the Harmonia badge's `statusVariant` keyword guess is the obvious future one). **Part 3, the cheap half that catches everything the other three cannot:** when a report aggregates over a lifecycle entity and neither declares `scope:` nor filters on the status AND the nomenclature is unclassified, generation records a `context.addIssue` warning - surfaced in the generate response's `warnings` and now in the **Intent Editor**'s own amber strip (it used to discard them on success; the Builder shell already showed them). That warning, not the default, is what turns an invisible modelling omission into a visible one. **And the invariant is checked at the consuming site too, independently of the resolver's site list:** a `where` condition on the queried entity's `function: EntityStatus` relation must carry an integer by the time validation runs (`IntentParser.validateWhereStatusValue`), so a value no status can equal is refused instead of rendering `.eq("Status", "OVERDUE")` into a query that matches nothing for as long as the job keeps ticking. `schedules[].where` was left behind for exactly that reason - #7091 taught the resolver the items rule and not the construct it was modelled on, and nothing anywhere failed. - **`lifecycle:` on an entity = the declarative state machine (#6714).** The whole set of legal status edges, declared once over the entity's `function: EntityStatus` nomenclature (`edges: [{ from: DRAFT, to: [ISSUED, CANCELLED] }, ...]`, either side a seeded name or an id) and **enforced on every status write**. The gap it closes: the status machinery was a set of point constructs - `init:` names the start, a `transitions:` button guards the flips that go through THAT button, a workflow `setRelationField` writes one unguarded, a `checks:` rejection files another - and nothing declared which edges were legal at all, so any other writer (a workflow branch, a glue action, a plain REST call) could jump a document from any status to any other and nothing noticed. **Enforcement lives in the generated REPOSITORY, deliberately** (`Repository.java.template`: `LIFECYCLE_EDGES` + `enforceLifecycle` / `enforceLifecycleMove` / `enforceLifecycleStart`, `ValidationException` -> 400) - it is the ONE choke point every writer passes through: `update` (the REST payload), `updateWithoutEvent` (system writes), and `updateProperties` (which `updateProperty`, and therefore the transition controller, the workflow setters and `updateDerived`, all route through - so the targeted-write overrides are now emitted for a lifecycle entity too, not only for `documentChecks`/`hasLabel`). Guarding the transition endpoints instead would have left every other writer free, which is the whole defect. `enforceLifecycleStart` (emitted only when the status relation declares `init:`) additionally refuses a CREATE filed anywhere but at the start - entering the lifecycle mid-graph skips it rather than travelling it - and is placed BEFORE the aggregate-guard macros in `save()` so an `outcome: reject` can still file the record where the model says. Emission is three scalars on the entity map (`lifecycleStatusProperty`, `lifecycleEdges` as `1>2,1>9` pairs, `lifecycleStatusNames` as `1=DRAFT,...` so a rejection reads "cannot move from ISSUED to DRAFT" instead of quoting positional ids, plus `lifecycleInitialStatus`) - scalars, so they reach the `.edm` twin like `immutableStatusValues`. **Parse-time is where the other status sites are made to agree** (`validateLifecycles`): every `from` of a `transitions:` entry must reach its `setStatus` along an edge (a button is presentation over the graph), and a status written by a `setRelationField` step or forced by a check's rejection must be one some edge reaches - which is what catches a reject path transiting through an approved status when the file is read. **Deliberate boundaries:** no `on:` key - the graph is always over the EntityStatus relation, so naming it would be redundant, and YAML 1.1 reads a bare `on` as the boolean `true` (it would arrive as the key `true` and bind to nothing), so `rejectLifecycleOn` refuses it in the raw-tree preprocessing rather than dropping it silently; a cross-model nomenclature is seeded in its owner model and so is its lifecycle (refused, naming that); the nomenclature must be seeded here (the ids are validated against the seeds); no reachability check - one nomenclature may serve two entities with different graphs, so "unreachable here" is not an error. - **A status a `processes:` flow writes is the FLOW's column, not a payload field (#7339).** An entity whose `function: EntityStatus` relation is moved by a `setRelationField` step carries `workflowStatusProperty` (the FK) and `workflowStatusInitial` (the relation's `init:`) on its entity map (`EdmIntentGenerator.putWorkflowStatus` / `writesStatus`, scalars reaching the `.edm` twin like `immutableStatusValues`), and all three generated REST controllers refuse a create/update that sets or changes it - **409** `'Status' changes through the workflow, not a direct edit`. The hole it closes is the whole point of having a flow at all: a plain `PUT {"Status": 3}` put a vacation request into APPROVED with the capacity check never run, no manager task ever raised and the leave account never charged - the document read approved and the accounts did not know. **`immutableWhen:` cannot close it** (it locks the way OUT of a final status; a DRAFT is mutable by definition, which is what the jump starts from) and neither can `transitions:` (a guarded EXTRA endpoint beside the plain PUT, not instead of it). Two deliberate non-refusals, each because refusing would be a different feature: an **absent** value is not a change - it is taken from the stored row, which is also what stops a partial payload from erasing the status - and a create carrying exactly the declared `init:` starts the record where the model says it starts (with no `init:`, any create value is refused). **A capacity roll-up's `status:` claims the whole column too (#7553)** - it is derived state exactly like a flow's. **A `transitions:` button claims only the VALUES it sets (#7553), emitted as `workflowStatusValues` (`2,3`) when no process or roll-up owns the column:** a plain `PUT` of the button's own target seed id bypasses the button's `from:`/`when:` guards, so that value is refused on update and on create - but a Cancel button says nothing about who moves the record to POSTED, and claiming the whole column would make every other status reachable from nowhere (the first cut of #7553 did, and `IntentEmissionCoverageIT`'s Entry - one `CancelEntry` button, POSTED reached by hand - could no longer be posted at all). Every other hand move stays with `lifecycle:`, enforced in the repository. **A system writer's status re-routes the checks conditioned on it (#7595):** an UNGATED `requiredWhen` whose `when` has a `Status == X` term, X being a value a `transitions:` button OR a `processes:` `setRelationField` step writes, takes X as its gate - `CheckSupport.effectiveGate` over `CheckSupport.systemStatusTargets`, the ONE rule both generators read. Neither writer reaches a controller (both go through `updateProperties`), so a rule left in `validate()` never ran at the moment it is about - base-sales-invoices found it live (an Issue with no reason went through) and fixed it by hand with an explicit `status:`. The EDM generator routes the check into the repository's `enforceChecks`; `BpmnIntentGenerator.gatesACheck` reads the same gate, so the step writing X runs in the completing transaction and the refusal is the 400 the approver sees, not a dead-lettered incident (#7014). `IntentEmissionCoverageIT` posts its Doc through `PostDocTransition` (counterparty rule + gated `compare` refuse there, 400); `IntentWorkflowStatusIT` refuses an approval without an approver at task completion; `CheckGateBpmnTest` pins the synchronous step. The flow's own writers are untouched: a `setRelationField` step and a `transitions[]` endpoint reach the repository through the targeted `updateProperty`/`updateProperties` primitives, never through a controller. Unit: `EdmIntentGeneratorTest`; end-to-end: `IntentWorkflowStatusIT` (the refused jump, the refused create, the ordinary edit that still saves, the omitted status that is not erased, and the flow's own write still landing). diff --git a/components/engine/engine-intent/src/main/java/org/eclipse/dirigible/components/intent/generator/GlueIntentGenerator.java b/components/engine/engine-intent/src/main/java/org/eclipse/dirigible/components/intent/generator/GlueIntentGenerator.java index d00e65d0e3d..9522a5f39f1 100644 --- a/components/engine/engine-intent/src/main/java/org/eclipse/dirigible/components/intent/generator/GlueIntentGenerator.java +++ b/components/engine/engine-intent/src/main/java/org/eclipse/dirigible/components/intent/generator/GlueIntentGenerator.java @@ -2623,6 +2623,10 @@ private static List> buildPostings(IntentModel model, Map rendered = new LinkedHashMap<>(); List> assigns = new ArrayList<>(); Map rowGuard = Map.of(); + // The rule columns THIS row reads. Where they are required is decided once the row's + // own `when:` is known (#7649): a column only a guarded row reads must not gate a + // document the guard excludes. + java.util.Set rowRuleColumns = new java.util.LinkedHashSet<>(); for (Map.Entry cell : row.entrySet()) { String value = cell.getValue() == null ? "" : cell.getValue() @@ -2648,7 +2652,7 @@ private static List> buildPostings(IntentModel model, Map> buildPostings(IntentModel model, Map(rowRuleColumns)); + } rendered.put("assigns", assigns); itemRows.add(rendered); } diff --git a/components/engine/engine-intent/src/main/resources/intent-assistant-guide.md b/components/engine/engine-intent/src/main/resources/intent-assistant-guide.md index da39c04adcf..a0f21503911 100644 --- a/components/engine/engine-intent/src/main/resources/intent-assistant-guide.md +++ b/components/engine/engine-intent/src/main/resources/intent-assistant-guide.md @@ -768,7 +768,14 @@ field may declare: references, arithmetic over the SOURCE's fields, or - for a to-one relation cell - a bare SOURCE relation name whose FK is copied onto the line; a row `when` is ` ==|!= `. A missing rule row or null referenced column SKIPS the posting (the unposted worklist = final-status - documents with no back-referencing target), never throws. `rule.match` is a single + documents with no back-referencing target), never throws - and says so in the log naming the rule + entity, the match value and the source, because nothing else in the system marks the gap (#7649). + **Where a column is required follows the line that reads it.** A column read by an UNGUARDED line is + required of every document of this type and is checked once, up front. A column only a + `when:`-guarded line reads is checked INSIDE that line's own guard: a document the guard excludes + books nothing on the column, so an optional line added later - a promotion account on the few + invoices that carry one - must not stop every document of the type from posting on every tenant + whose rule row predates it. `rule.match` is a single `column: literal` selector and the literal must be there - an empty one is refused at parse, because it is rendered into the handler as the authored literal and would select no rule row at all, leaving every source document on the worklist with nothing failing anywhere. diff --git a/components/engine/engine-intent/src/test/java/org/eclipse/dirigible/components/intent/generator/GluePostingsTest.java b/components/engine/engine-intent/src/test/java/org/eclipse/dirigible/components/intent/generator/GluePostingsTest.java index 51dbebb1a68..eae724708f0 100644 --- a/components/engine/engine-intent/src/test/java/org/eclipse/dirigible/components/intent/generator/GluePostingsTest.java +++ b/components/engine/engine-intent/src/test/java/org/eclipse/dirigible/components/intent/generator/GluePostingsTest.java @@ -114,7 +114,15 @@ void postingGlueIsFullyPreRendered() { "a to-one relation item cell must pre-render as a source-FK copy"); // the third row carries a null-safe Calc guard assertEquals("Calc.eval(\"Vat\", source, 6).compareTo(new java.math.BigDecimal(\"0\")) != 0", GlueRendering.guard(rows.get(2))); - assertEquals(List.of("ReceivableAccount", "RevenueAccount", "VatAccount"), p.get("usedRuleColumns")); + // Only the columns the UNGUARDED rows read are required of every document of this type + // (#7649): the VAT account is read by the `when: "Vat != 0"` row alone, so it is required where + // that row books and nowhere else - a rule row without one still posts a zero-rated invoice. + assertEquals(List.of("ReceivableAccount", "RevenueAccount"), p.get("usedRuleColumns")); + assertEquals(null, rows.get(0) + .get("ruleColumns"), + "an unguarded row's columns are gated up front, not on the row"); + assertEquals(List.of("VatAccount"), rows.get(2) + .get("ruleColumns")); } /** diff --git a/components/ide/ide-template/src/main/java/org/eclipse/dirigible/components/ide/template/service/model/GlueGenerator.java b/components/ide/ide-template/src/main/java/org/eclipse/dirigible/components/ide/template/service/model/GlueGenerator.java index 5d5d2336aa7..9782e18a134 100644 --- a/components/ide/ide-template/src/main/java/org/eclipse/dirigible/components/ide/template/service/model/GlueGenerator.java +++ b/components/ide/ide-template/src/main/java/org/eclipse/dirigible/components/ide/template/service/model/GlueGenerator.java @@ -1648,6 +1648,9 @@ static List> rows(Object raw) { String guard = declared.containsKey("guardReading") ? JavaExpressions.expression(declared.get("guardReading")) : null; row.put("guard", guard != null ? guard : strOr(declared, "guard", "")); row.put("assigns", assignments(declared.get("assigns"))); + // The classifier ternaries this row's own cells carry, for the null-check that belongs + // inside its guard (#7649). Empty for an unguarded row - those are gated up front. + row.put("ruleCaseGuards", str(row, "guard").isEmpty() ? List.of() : ruleCaseExpressions(row)); rows.add(row); } return rows; @@ -1664,15 +1667,28 @@ static List> rows(Object raw) { static List conditionalRuleGuards(List> rows) { List guards = new ArrayList<>(); for (Map row : rows) { - for (Map assignment : asMaps(row.get("assigns"))) { - if (assignment.get("reading") instanceof Map reading && "ruleCase".equals(reading.get("kind"))) { - guards.add(str(assignment, "expr")); - } + // A GUARDED row's classifier ternaries are required only where that row books (#7649), so + // they are carried on the row and null-checked inside its own guard; only an unguarded + // row's reach the up-front gate, which every document of this type passes through. + if (!str(row, "guard").isEmpty()) { + continue; } + guards.addAll(ruleCaseExpressions(row)); } return guards; } + /** The classifier ternaries one row's cells carry, already rendered. */ + static List ruleCaseExpressions(Map row) { + List expressions = new ArrayList<>(); + for (Map assignment : asMaps(row.get("assigns"))) { + if (assignment.get("reading") instanceof Map reading && "ruleCase".equals(reading.get("kind"))) { + expressions.add(str(assignment, "expr")); + } + } + return expressions; + } + /** * Sanitizes a descriptor's value into a Java identifier. * diff --git a/components/ide/ide-template/src/test/java/org/eclipse/dirigible/components/ide/template/service/model/GlueGeneratorTest.java b/components/ide/ide-template/src/test/java/org/eclipse/dirigible/components/ide/template/service/model/GlueGeneratorTest.java index d9f45ced59b..58892cc52c6 100644 --- a/components/ide/ide-template/src/test/java/org/eclipse/dirigible/components/ide/template/service/model/GlueGeneratorTest.java +++ b/components/ide/ide-template/src/test/java/org/eclipse/dirigible/components/ide/template/service/model/GlueGeneratorTest.java @@ -383,6 +383,29 @@ void theConditionalRuleGuardsAreTheRuleCaseCells() { "(Calc.eval(\"Method\", source, 6).compareTo(new java.math.BigDecimal(\"1\")) == 0 ? ruleRow.CashAccount : ruleRow.BankAccount)"); } + /** + * ...and a GUARDED row's ternaries are not among them (#7649): a classifier the row needs only + * where it books must not stop a document the row's own {@code when:} excludes. They ride on the + * row instead, for a null check inside that guard. + */ + @Test + void aGuardedRowsRuleCaseCellsGuardThatRowAlone() { + Map ruleCase = new LinkedHashMap<>(); + ruleCase.put("targetProp", "Account"); + ruleCase.put("reading", Map.of("kind", "ruleCase", "by", "Method", "owner", "source", "cases", + List.of(Map.of("value", "1", "column", "CashAccount")), "otherwise", "BankAccount")); + Map guarded = new LinkedHashMap<>(); + guarded.put("guardReading", Map.of("kind", "calcCompare", "owner", "source", "property", "Vat", "equal", false, "text", "0")); + guarded.put("assigns", List.of(ruleCase)); + + List> rows = GlueGenerator.rows(List.of(guarded)); + + assertThat(GlueGenerator.conditionalRuleGuards(rows)).isEmpty(); + assertThat(rows.get(0) + .get("ruleCaseGuards")).asInstanceOf(org.assertj.core.api.InstanceOfAssertFactories.LIST) + .hasSize(1); + } + /** * An event binding's guard is rendered from its neutral terms - {@code true} for none - and a * descriptor written before the split keeps the rendered expression it carries. diff --git a/components/template/template-application-events-java/src/main/resources/META-INF/dirigible/template-application-events-java/events/Posting.java.template b/components/template/template-application-events-java/src/main/resources/META-INF/dirigible/template-application-events-java/events/Posting.java.template index 148664cfa1b..26df45400df 100644 --- a/components/template/template-application-events-java/src/main/resources/META-INF/dirigible/template-application-events-java/events/Posting.java.template +++ b/components/template/template-application-events-java/src/main/resources/META-INF/dirigible/template-application-events-java/events/Posting.java.template @@ -83,16 +83,27 @@ public class ${className}Posting implements MessageHandler { new gen.${javaGenFolderName}.data.${ruleJavaPerspective}.${ruleEntity}Repository().findAll( Criteria.create().eq("${ruleMatchProperty}", ${ruleMatchValueJava})); if (ruleRows.isEmpty()) { + // Said out loud (#7649): the unposted worklist is otherwise invisible - the handler only + // fires on the event, so a silent skip leaves nothing anywhere to explain the gap. + LOG.warn("Posting ${name}: no ${ruleEntity} matches [{}] - ${sourceEntity} [{}] stays unposted", + ${ruleMatchValueJava}, source.${sourceKeyField}); return; // no determination rule -> the document stays on the unposted worklist } gen.${javaGenFolderName}.data.${ruleJavaPerspective}.${ruleEntity}Entity ruleRow = ruleRows.get(0); +## The columns every line of this posting reads - the UNGUARDED rows'. A null here stops a document +## that would genuinely have booked on the column, so it is checked once, up front. A column only a +## `when:`-guarded line reads is checked inside that line's own guard below (#7649). #foreach($column in $usedRuleColumns) if (ruleRow.${column} == null) { + LOG.warn("Posting ${name}: ${ruleEntity} matching [{}] has no ${column} - ${sourceEntity} [{}] stays unposted", + ${ruleMatchValueJava}, source.${sourceKeyField}); return; // incomplete determination -> unposted worklist } #end #foreach($guard in $conditionalRuleGuards) if (${guard} == null) { + LOG.warn("Posting ${name}: the ${ruleEntity} matching [{}] determines no account for ${sourceEntity} [{}]" + + " - it stays unposted", ${ruleMatchValueJava}, source.${sourceKeyField}); return; // conditional rule(by: ...) selected no account (unmatched or null) -> unposted worklist } #end @@ -116,6 +127,23 @@ public class ${className}Posting implements MessageHandler { #foreach($row in $itemRows) #if($row.guard != "") if (${row.guard}) { +## The rule columns only THIS line reads (#7649). Required where the line books, not before: a +## document whose `when:` excludes the line books nothing on the column, so a rule row that predates +## it - every tenant's, the day an optional line is added - must not stop that document from posting. +#foreach($column in $row.ruleColumns) + if (ruleRow.${column} == null) { + LOG.warn("Posting ${name}: ${ruleEntity} matching [{}] has no ${column}, which this ${sourceEntity} [{}] books on" + + " - it stays unposted", ${ruleMatchValueJava}, source.${sourceKeyField}); + return; + } +#end +#foreach($guard in $row.ruleCaseGuards) + if (${guard} == null) { + LOG.warn("Posting ${name}: the ${ruleEntity} matching [{}] determines no account for the line this ${sourceEntity}" + + " [{}] books - it stays unposted", ${ruleMatchValueJava}, source.${sourceKeyField}); + return; + } +#end #end { gen.${javaGenFolderName}.data.${itemsJavaPerspective}.${itemsEntity}Entity item =