Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 45 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,14 @@ router. `civiccore.search` now ships normalization and
fusion helpers, but not a full search engine or indexer.
`civiccore.notifications` now ships notice deadline and compliance
helpers, but not delivery queues or outbound notification orchestration.
`civiccore.platform` now ships the Windows-local desktop contracts for module
manifests, install-profile validation, operator health summaries, durable local
task envelopes, backup/restore manifests, and installer/runtime action results.
It also ships the PostgreSQL-backed `civiccore_local_tasks` migration, async
queue helpers, and `python -m civiccore.tasks.worker` entry point for the local
desktop worker. Product modules still own task handlers and UI, while CivicCore
owns shared validation, queue state, retry semantics, and plain-English
operator state.
`civiccore.verification` now ships the first release-evidence helper
surface, while sovereignty verification remains future work.
`civiccore.connectors` now also ships shared local-payload import
Expand Down Expand Up @@ -83,7 +91,10 @@ on top of shared vendor delta request planning plus reusable no-network
mock-city proof contracts for agenda vendors, municipal OIDC, and backup
retention/off-host storage, on top of shared live connector sync retry/circuit-breaker primitives,
including run-result normalization, operator health copy, retry delay policy, and async HTTP retry,
on top of shared persisted audit-log hash and verification helpers for
on top of shared Windows-local module/runtime contracts for the desktop shell,
including no-Docker/no-WSL manifest validation, plain-English health summaries,
PostgreSQL-backed local task queue helpers, and backup/restore checksum
manifests, on top of shared persisted audit-log hash and verification helpers for
database-backed module audit rows on top of shared trusted-header auth config
loading and proxy-source enforcement helpers on top of shipped
trusted-header auth helpers on top of shipped
Expand Down Expand Up @@ -345,7 +356,35 @@ assert verify_persisted_audit_chain([

These APIs are deliberately offline-first. They do not provide JWT
issuance, SSO, user directories, credential storage, vendor-specific network
adapters, worker scheduling, legal determinations, or vendor write-back.
adapters, worker execution, legal determinations, or vendor write-back.

## Windows-local platform contracts

`civiccore.platform` exposes the shared contracts and queue helpers the
CivicSuite Windows desktop shell uses to keep future modules pluggable without
making clerks learn infrastructure:

```python
from civiccore.platform import (
ModuleManifest,
build_module_registry,
PlatformHealthCheck,
summarize_platform_health,
LocalTaskEnvelope,
enqueue_local_task,
claim_next_local_task,
record_task_attempt,
run_one_local_task,
build_backup_manifest,
plan_restore,
)
```

For the `windows_local` install profile, module manifests cannot require
Docker, WSL, Linux shell setup, or a terminal-only operator path. CivicCore also
ships task envelopes, a durable PostgreSQL task table, async queue helpers, and
a worker CLI so downstream modules share retry, health, checksum, and
restore-safety semantics while registering their own task handlers.

## Document ingestion

Expand Down Expand Up @@ -534,9 +573,10 @@ router integration, and persistence orchestration remain future work.
## Scheduling helper

`civiccore.scheduling` exposes the shared cron expression contract used by
module background jobs. Modules keep their own Celery/worker/runtime wiring,
but should reuse this validation so one-minute accidental or adversarial
schedules are blocked consistently across CivicSuite.
module background jobs. `civiccore.platform` exposes the local task envelope,
PostgreSQL queue helpers, and retry contract. Modules keep their own task
handlers, but should reuse these helpers so one-minute accidental or adversarial
schedules and task retry behavior are handled consistently across CivicSuite.

```python
from civiccore.scheduling import compute_next_sync_at, validate_cron_expression
Expand Down
92 changes: 92 additions & 0 deletions civiccore/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,53 @@
next_profile_prompt,
parse_profile_answer,
)
from civiccore.platform import (
WINDOWS_LOCAL_BLOCKED_RUNTIME_KINDS,
BackupItem,
BackupManifest,
BackupRestoreAction,
BackupRestorePlan,
BackupValidationResult,
LocalRuntimeProfile,
LocalTask,
LocalTaskEnvelope,
LocalTaskResult,
ModuleBackupHook,
ModuleDependency,
ModuleHealthCheck,
ModuleManifest,
ModuleMigration,
ModuleModelRequirement,
ModulePermission,
ModuleRegistryEntry,
ModuleRegistryState,
ModuleRoute,
ModuleRuntimeRequirement,
ModuleService,
PlatformHealthCheck,
PlatformHealthStatus,
PlatformHealthSummary,
RuntimeActionResult,
TaskQueueSummary,
TaskHandler,
TaskRetryPolicy,
build_backup_manifest,
build_module_registry,
can_run_task,
claim_next_local_task,
complete_local_task,
enqueue_local_task,
fail_local_task,
next_retry_at,
plan_restore,
record_task_attempt,
run_one_local_task,
summarize_platform_health,
summarize_task_queue,
task_row_to_envelope,
validate_backup_manifest,
validate_windows_local_manifest,
)
from civiccore.provenance import (
CitationTarget,
DocumentMetadata,
Expand Down Expand Up @@ -245,6 +292,40 @@
"ModuleEnablement",
"OnboardingField",
"OnboardingProgress",
"BackupItem",
"BackupManifest",
"BackupRestoreAction",
"BackupRestorePlan",
"BackupValidationResult",
"LocalRuntimeProfile",
"LocalTask",
"LocalTaskEnvelope",
"LocalTaskResult",
"ModuleBackupHook",
"ModuleDependency",
"ModuleHealthCheck",
"ModuleManifest",
"ModuleMigration",
"ModuleModelRequirement",
"ModulePermission",
"ModuleRegistryEntry",
"ModuleRegistryState",
"ModuleRoute",
"ModuleRuntimeRequirement",
"ModuleService",
"PlatformHealthCheck",
"PlatformHealthStatus",
"PlatformHealthSummary",
"RuntimeActionResult",
"TaskQueueSummary",
"TaskHandler",
"TaskRetryPolicy",
"WINDOWS_LOCAL_BLOCKED_RUNTIME_KINDS",
"build_backup_manifest",
"build_module_registry",
"can_run_task",
"claim_next_local_task",
"complete_local_task",
"access_level_allows",
"completed_profile_fields",
"compute_onboarding_status",
Expand All @@ -269,7 +350,13 @@
"normalize_search_query",
"normalize_search_text",
"min_interval_minutes",
"enqueue_local_task",
"fail_local_task",
"next_retry_at",
"parse_profile_answer",
"plan_restore",
"record_task_attempt",
"run_one_local_task",
"roles_grant_access",
"search_text_matches_query",
"reciprocal_rank_fusion",
Expand All @@ -282,8 +369,13 @@
"ingest_structured_record",
"register_handler",
"validate_cited_sentences",
"summarize_platform_health",
"summarize_task_queue",
"task_row_to_envelope",
"validate_backup_manifest",
"validate_cron_expression",
"validate_fernet_key_setting",
"validate_windows_local_manifest",
"validate_odbc_connection_string",
"validate_password_setting",
"validate_secret_setting",
Expand Down
44 changes: 44 additions & 0 deletions civiccore/migrations/versions/civiccore_0003_local_task_queue.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
"""CivicCore migration 0003 - local task queue."""

from __future__ import annotations

import sqlalchemy as sa
from sqlalchemy.dialects import postgresql

from civiccore.migrations.guards import idempotent_create_index, idempotent_create_table


revision = "civiccore_0003_local_task_queue"
down_revision = "civiccore_0002_llm"
branch_labels = None
depends_on = None


def upgrade() -> None:
idempotent_create_table(
"civiccore_local_tasks",
sa.Column("task_id", sa.String(128), primary_key=True),
sa.Column("module_id", sa.String(64), nullable=False),
sa.Column("task_type", sa.String(100), nullable=False),
sa.Column("status", sa.String(20), nullable=False, server_default="queued"),
sa.Column(
"payload",
postgresql.JSONB(astext_type=sa.Text()),
nullable=False,
server_default=sa.text("'{}'::jsonb"),
),
sa.Column("attempt_count", sa.Integer(), nullable=False, server_default="0"),
sa.Column("queued_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.text("now()")),
sa.Column("available_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("last_error", sa.Text(), nullable=True),
sa.Column("idempotency_key", sa.String(255), nullable=True, unique=True),
sa.Column("audit_subject_id", sa.String(255), nullable=True),
)
idempotent_create_index("ix_civiccore_local_tasks_module", "civiccore_local_tasks", ["module_id"])
idempotent_create_index("ix_civiccore_local_tasks_status", "civiccore_local_tasks", ["status"])
idempotent_create_index("ix_civiccore_local_tasks_type", "civiccore_local_tasks", ["task_type"])


def downgrade() -> None:
"""No-op; local task queue data is preserved for point-in-time restore."""
return None
2 changes: 2 additions & 0 deletions civiccore/models/__init__.py
Original file line number Diff line number Diff line change
@@ -1,11 +1,13 @@
"""CivicCore shared SQLAlchemy ORM model exports."""

from civiccore.ingest.models import DataSource, Document, DocumentChunk, IngestionStatus, SourceType
from civiccore.platform.task_queue import LocalTask

__all__ = [
"DataSource",
"Document",
"DocumentChunk",
"IngestionStatus",
"LocalTask",
"SourceType",
]
106 changes: 106 additions & 0 deletions civiccore/platform/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
"""Windows-local platform contracts for CivicSuite desktop deployments."""

from __future__ import annotations

from civiccore.platform.backup import (
BackupItem,
BackupManifest,
BackupRestoreAction,
BackupRestorePlan,
BackupValidationResult,
build_backup_manifest,
plan_restore,
validate_backup_manifest,
)
from civiccore.platform.health import (
PlatformHealthCheck,
PlatformHealthStatus,
PlatformHealthSummary,
summarize_platform_health,
)
from civiccore.platform.modules import (
WINDOWS_LOCAL_BLOCKED_RUNTIME_KINDS,
ModuleBackupHook,
ModuleDependency,
ModuleHealthCheck,
ModuleManifest,
ModuleMigration,
ModuleModelRequirement,
ModulePermission,
ModuleRegistryEntry,
ModuleRegistryState,
ModuleRoute,
ModuleRuntimeRequirement,
ModuleService,
build_module_registry,
validate_windows_local_manifest,
)
from civiccore.platform.runtime import LocalRuntimeProfile, RuntimeActionResult
from civiccore.platform.task_queue import (
LocalTask,
TaskHandler,
claim_next_local_task,
complete_local_task,
enqueue_local_task,
fail_local_task,
run_one_local_task,
task_row_to_envelope,
)
from civiccore.platform.tasks import (
LocalTaskEnvelope,
LocalTaskResult,
TaskQueueSummary,
TaskRetryPolicy,
can_run_task,
next_retry_at,
record_task_attempt,
summarize_task_queue,
)

__all__ = [
"BackupItem",
"BackupManifest",
"BackupRestoreAction",
"BackupRestorePlan",
"BackupValidationResult",
"LocalRuntimeProfile",
"LocalTask",
"LocalTaskEnvelope",
"LocalTaskResult",
"ModuleBackupHook",
"ModuleDependency",
"ModuleHealthCheck",
"ModuleManifest",
"ModuleMigration",
"ModuleModelRequirement",
"ModulePermission",
"ModuleRegistryEntry",
"ModuleRegistryState",
"ModuleRoute",
"ModuleRuntimeRequirement",
"ModuleService",
"PlatformHealthCheck",
"PlatformHealthStatus",
"PlatformHealthSummary",
"RuntimeActionResult",
"TaskQueueSummary",
"TaskHandler",
"TaskRetryPolicy",
"WINDOWS_LOCAL_BLOCKED_RUNTIME_KINDS",
"build_backup_manifest",
"build_module_registry",
"can_run_task",
"claim_next_local_task",
"complete_local_task",
"enqueue_local_task",
"fail_local_task",
"next_retry_at",
"plan_restore",
"record_task_attempt",
"run_one_local_task",
"summarize_platform_health",
"summarize_task_queue",
"task_row_to_envelope",
"validate_backup_manifest",
"validate_windows_local_manifest",
]
Loading