Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .changes/+heroku-self-hosting.added.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
支持通过 GitHub Actions 将自托管实例部署到 Heroku 与 Neon。
1 change: 1 addition & 0 deletions .changes/+web-extension-install.fixed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
修复 Web 无法通过所选 Core Host 安装 Python-only Extension 的接入缺口,复用既有精确版本安装与兼容性校验,不改变 Extension Host SDK 契约。
1 change: 1 addition & 0 deletions .changes/+worktree-ssh-socket.fixed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
修复深层 worktree 路径下开发数据库 SSH control socket 超长导致启动失败的问题,保留实例独立的 tunnel 与清理归属。
120 changes: 120 additions & 0 deletions .github/workflows/self-host-heroku-neon.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
name: Deploy self-hosted InKCre to Heroku

on:
workflow_dispatch:

permissions:
contents: read

concurrency:
group: self-host-heroku-neon-${{ github.repository }}
cancel-in-progress: false

env:
CORE_DATABASE_PASSWORD: ${{ secrets.CORE_DATABASE_PASSWORD }}
HEAD_SHA: ${{ github.sha }}
HEROKU_API_KEY: ${{ secrets.HEROKU_API_KEY }}
IMAGE_TAG: inkcre-self-host-schema:${{ github.sha }}
JWT_SECRET: ${{ secrets.JWT_SECRET }}
NEON_API_KEY: ${{ secrets.NEON_API_KEY }}
NEON_PROJECT_ID: ${{ vars.NEON_PROJECT_ID }}
POSTGREST_DATABASE_PASSWORD: ${{ secrets.POSTGREST_DATABASE_PASSWORD }}
SOURCE_REVISION: ${{ github.sha }}

jobs:
deploy:
name: Converge Heroku and Neon deployment
runs-on: ubuntu-latest
services:
postgres:
image: >-
pgvector/pgvector:pg17@sha256:d2ef61f42ef767baa5a1475393303cc235bcd92febd9d7014eddb48b41f3bad0
env:
POSTGRES_DB: inkcre
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
ports:
- 5432:5432
options: >-
--health-cmd "pg_isready -U postgres -d inkcre"
--health-interval 5s
--health-timeout 5s
--health-retries 12
steps:
- name: Checkout exact selected commit
uses: actions/checkout@fbc6f3992d24b796d5a048ff273f7fcc4a7b6c09 # v5
with:
ref: ${{ github.sha }}
persist-credentials: false

- name: Validate required fork settings
id: settings
env:
HEROKU_APP_PREFIX: ${{ vars.HEROKU_APP_PREFIX }}
run: bash scripts/automation/self_host.sh validate-heroku

- name: Build exact service and transport images
run: |
bash scripts/automation/runtime_artifact.sh build-schema-source
bash scripts/automation/runtime_contract.sh export
bash scripts/automation/runtime_artifact.sh stage-schema-evidence
bash scripts/automation/runtime_contract.sh restore
docker build --platform linux/amd64 --provenance=false \
--build-arg "SOURCE_REVISION=$HEAD_SHA" --target service \
--tag "inkcre-production-web:$HEAD_SHA" .
bash scripts/automation/runtime_artifact.sh build-production-transports

- name: Install pinned Heroku CLI
run: npm install --global heroku@11.8.1

- name: Resolve the Neon default branch owner coordinates
id: neon
run: bash scripts/automation/self_host.sh resolve-neon

- name: Resolve self-hosted Heroku apps
id: heroku
env:
EXPECTED_DEPLOYMENT_PROFILE: ${{ steps.settings.outputs.deployment_profile }}
HEROKU_APP_NAME: ${{ steps.settings.outputs.app_name }}
POSTGREST_APP_NAME: ${{ steps.settings.outputs.postgrest_app_name }}
run: bash scripts/automation/production_delivery.sh resolve-apps

- name: Converge the self-hosted database contract
id: database
env:
DATABASE_ENVIRONMENT: runtime
MIGRATION_DATABASE_URL: ${{ steps.neon.outputs.migration_database_url }}
POOLED_OWNER_DATABASE_URL: ${{ steps.neon.outputs.source_database_url }}
run: bash scripts/automation/production_delivery.sh converge-database

- name: Push and release self-hosted peer processes
id: release
env:
APP_NAME: ${{ steps.heroku.outputs.app_name }}
DATABASE_URL: ${{ steps.database.outputs.core_database_url }}
INKCRE_DEPLOYMENT_PROFILE: ${{ steps.settings.outputs.deployment_profile }}
PEER_NAME: ${{ steps.heroku.outputs.app_name }}
POSTGREST_APP_NAME: ${{ steps.heroku.outputs.postgrest_app_name }}
POSTGREST_DATABASE_URL: ${{ steps.database.outputs.postgrest_database_url }}
run: bash scripts/automation/production_delivery.sh release

- name: Converge self-hosted Peer advertisement
env:
APP_NAME: ${{ steps.heroku.outputs.app_name }}
DATABASE_URL: ${{ steps.database.outputs.core_database_url }}
WEB_URL: ${{ steps.heroku.outputs.web_url }}
run: bash scripts/automation/production_delivery.sh advertise

- name: Probe self-hosted peers
env:
APP_NAME: ${{ steps.heroku.outputs.app_name }}
CORE_IMAGE_DIGEST: ${{ steps.release.outputs.core_local_image_id }}
CORE_IMAGE_LABEL: Exact local image ID
CORE_LOCAL_IMAGE_ID: ${{ steps.release.outputs.core_local_image_id }}
DEPLOYMENT_SUMMARY_TITLE: Self-hosted InKCre on Heroku + Neon
POSTGREST_APP_NAME: ${{ steps.heroku.outputs.postgrest_app_name }}
POSTGREST_RELEASE: ${{ steps.release.outputs.postgrest_release }}
POSTGREST_URL: ${{ steps.heroku.outputs.postgrest_url }}
WEB_RELEASE: ${{ steps.release.outputs.web_release }}
WEB_URL: ${{ steps.heroku.outputs.web_url }}
run: bash scripts/automation/production_delivery.sh probe
13 changes: 7 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,14 +21,15 @@ Developer setup and shared-skill notes: [CONTRIBUTING.md](CONTRIBUTING.md)

## Browser-Only Self-Hosting

A repository owner can deploy their own InKCre instance to Neon and two Render Docker
services without cloning this repository. Forking is the onboarding mechanism; the deployed
instance is independent from InKCre's canonical production environment. Configure the six
documented GitHub Secrets/Variables, then run the checked-in `Deploy self-hosted InKCre`
workflow. The JWT signing secret remains private and grants full Peer authority.
A repository owner can deploy their own InKCre instance to Neon and either Render or Heroku
without cloning this repository. Forking is the onboarding mechanism; the deployed instance is
independent from InKCre's canonical production environment. Configure the provider profile's
documented GitHub Secrets/Variables, then run its checked-in self-host workflow. The JWT signing
secret remains private and grants full Peer authority.

Exact onboarding steps, runtime limits, and cleanup:
[Self-Hosting On Render And Neon](docs/40-deployment/render-neon-self-host.md).
[Render and Neon](docs/40-deployment/render-neon-self-host.md) or
[Heroku and Neon](docs/40-deployment/heroku-neon-self-host.md).

## Security

Expand Down
3 changes: 3 additions & 0 deletions app/business/extension/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
DisableExtensionCommand,
EnableExtensionCommand,
ExtensionManagementCommand,
InstallExtensionCommand,
PatchExtensionConfigCommand,
)
from app.schemas.peer import PeerProtocolRequest, PeerProtocolResponse, PeerRef
Expand Down Expand Up @@ -218,6 +219,8 @@ async def manage_local(
command: ExtensionManagementCommand,
) -> InstalledExtension:
"""Execute one already-validated command without entering delegation."""
if isinstance(command, InstallExtensionCommand):
return await self.install(command.extension, command.version)
if isinstance(command, EnableExtensionCommand):
return await self.enable(command.extension)
if isinstance(command, DisableExtensionCommand):
Expand Down
2 changes: 2 additions & 0 deletions app/schemas/extension/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
ExtensionManagementCommand,
ExtensionModel,
ExtensionName,
InstallExtensionCommand,
PatchExtensionConfigCommand,
)

Expand All @@ -17,5 +18,6 @@
"ExtensionManagementCommand",
"ExtensionModel",
"ExtensionName",
"InstallExtensionCommand",
"PatchExtensionConfigCommand",
]
13 changes: 12 additions & 1 deletion app/schemas/extension/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,14 @@
)


class InstallExtensionCommand(pydantic.BaseModel):
model_config = pydantic.ConfigDict(extra="forbid", frozen=True)

action: typing.Literal["install"]
extension: ExtensionName
version: str = pydantic.Field(pattern=EXTENSION_SEMVER_PATTERN)


class EnableExtensionCommand(pydantic.BaseModel):
model_config = pydantic.ConfigDict(extra="forbid", frozen=True)

Expand All @@ -42,7 +50,10 @@ class PatchExtensionConfigCommand(pydantic.BaseModel):


ExtensionManagementCommand: typing.TypeAlias = typing.Annotated[
EnableExtensionCommand | DisableExtensionCommand | PatchExtensionConfigCommand,
InstallExtensionCommand
| EnableExtensionCommand
| DisableExtensionCommand
| PatchExtensionConfigCommand,
pydantic.Field(discriminator="action"),
]

Expand Down
3 changes: 3 additions & 0 deletions docs/30-unit-tdd/business-pipeline-and-authority.md
Original file line number Diff line number Diff line change
Expand Up @@ -206,6 +206,9 @@ runtime 路径禁止同步 session、scoped session 和直接驱动连接;`scr
- Business owners retain typed codecs and non-delegating local seams。Current exact inbounds are
`core.semantic_retrieval.v1`、`core.feature_retrieval.lexical.v1`、`core.organization.rumination.v1` and exact-target
`core.extension.management.v1`。
- Extension management accepts exact-version `install` alongside enable, disable and config patch. It invokes the same
Core Host install boundary as REST, including compatibility and version-change guards; it neither enables the
Extension nor acquires a browser distribution. Older Peers reject the unknown action rather than routing elsewhere.
- `core.peer.protocol.http.v1` owns normalized query/headers/body envelopes、Peer JWT and HTTP response projection。
Generic failover occurs only after pre-dispatch failure or exact `InkCre-Peer-Execution: not-executed`;a normal domain
response or outcome-unknown stops。
Expand Down
2 changes: 2 additions & 0 deletions docs/40-deployment/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ GitHub workflow and composite-action YAML owns only GitHub event selection, perm
- [runtime-orchestration.md](runtime-orchestration.md)
- [mcp-sink.md](mcp-sink.md)
- [render-neon-self-host.md](render-neon-self-host.md)
- [heroku-neon-self-host.md](heroku-neon-self-host.md)

## Checked-In Runtime Anchors

Expand All @@ -40,6 +41,7 @@ GitHub workflow and composite-action YAML owns only GitHub event selection, perm
- `.github/workflows/preview-deploy.yml`
- `.github/workflows/production-deploy.yml`
- `.github/workflows/self-host-render-neon.yml`
- `.github/workflows/self-host-heroku-neon.yml`
- `scripts/deploy_render_neon.py`
- `scripts/generate-openapi.py`
- `docs/openapi.json`
2 changes: 2 additions & 0 deletions docs/40-deployment/development-environment.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,8 @@ tracked files never own a hostname, user, key, or machine path.
The SSH provider sends an allowlisted build context and one bounded Compose payload to the
remote host. Remote services publish dynamic remote-loopback ports and an instance-owned
OpenSSH control tunnel maps independent local-loopback ports to them.
The descriptor keeps the absolute control-socket path, while SSH runs from its parent directory
with the short socket name so long worktree paths do not exceed Unix-domain socket limits.

Runtime state is written to `.runtime/database/<core-svc-instance>/`. Its `runtime.json`,
`profile.json`, and `readiness.json` record:
Expand Down
70 changes: 70 additions & 0 deletions docs/40-deployment/heroku-neon-self-host.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
# Self-Hosting On Heroku And Neon

## Purpose And Boundary

The checked-in `Deploy self-hosted InKCre to Heroku` workflow gives a repository owner a
browser-only path from a public `core-py` fork to two Heroku Container Stack apps backed by
the owner's Neon default branch:

- `HEROKU_APP_PREFIX-core` runs the complete `core-py` Peer;
- `HEROKU_APP_PREFIX-postgrest` exposes the executable peer database contract.

The workflow builds the selected commit as `linux/amd64` images, creates missing apps in the
Heroku US region, and runs one Eco web dyno for each app. Heroku charges and Eco sleep behavior
remain the deployment owner's responsibility. This deployment is independent from InKCre's
canonical production environment.

The `JWT_SECRET` grants full admitted-Peer authority. Keep it private. The two database-role
passwords are also persistent deployment credentials: reruns must use the same values unless the
database roles and both app configurations are rotated together.

## Browser-Only Onboarding

1. Fork this repository and enable GitHub Actions for the fork.
2. Create a Neon project and an API key that can access it. Its default branch must retain the
standard `neondb` database and `neondb_owner` role. Copy the project ID.
3. Create a Heroku account with billing enabled and an API key.
4. In **Settings → Secrets and variables → Actions**, add these repository settings.

| Kind | Exact name | Meaning |
| --- | --- | --- |
| Secret | `NEON_API_KEY` | Can resolve and mutate `NEON_PROJECT_ID` |
| Secret | `HEROKU_API_KEY` | Can create, configure, release, and scale the two apps |
| Secret | `JWT_SECRET` | At least 32 bytes; owner-only Peer signing authority |
| Secret | `CORE_DATABASE_PASSWORD` | At least 32 bytes; persistent `inkcre_core` role password |
| Secret | `POSTGREST_DATABASE_PASSWORD` | At least 32 bytes; persistent `authenticator` role password |
| Variable | `NEON_PROJECT_ID` | Target Neon project identity |
| Variable | `HEROKU_APP_PREFIX` | Unique 3–18 character lowercase app prefix |

Use independently generated values for all three credential secrets. Do not reuse a provider API
key or another password as the JWT secret.

5. Open **Actions → Deploy self-hosted InKCre to Heroku → Run workflow** and select the branch
to deploy.
6. Read the workflow summary for the public app URLs, Peer identity, exact commit, and Heroku
releases. Keep using the repository's private `JWT_SECRET` when an admitted client asks for
the signing key.

## Convergence And Safety

The workflow validates all settings before provider mutation, builds the exact selected commit,
resolves the Neon default branch owner coordinates, creates missing Heroku apps, and refuses an
existing app unless its deployment profile matches the same Neon project. It then initializes the
database contract, stores only role-specific URLs in Heroku config, releases both images, converges
the Core Peer's public address, and verifies Core readiness plus the authenticated PostgREST
read/write/deny contract.

The Neon owner URL exists only in the GitHub Actions job. It is never placed in either Heroku app.
Workflow output contains no JWT, database password, or database URL. Rerunning the workflow is a
convergence operation and does not infer credential rotation or cleanup authority.

Heroku's [Container Registry documentation](https://devcenter.heroku.com/articles/container-registry-and-runtime)
defines the image release and `linux/amd64` requirements. Its
[config-var documentation](https://devcenter.heroku.com/articles/config-vars) describes the
persistent runtime configuration used by this profile.

## Cleanup

Delete the two exact Heroku apps when the deployment is no longer wanted. Delete the Neon project
only if it is dedicated to this deployment and its data is disposable. Both actions are explicitly
owner-controlled and intentionally absent from the deployment workflow.
2 changes: 1 addition & 1 deletion docs/_shared
Submodule _shared updated 38 files
+37 −0 .github/workflows/pages-preview.yml
+8 −0 20-product-tdd/cross-unit-contracts.md
+103 −0 20-product-tdd/extension-documentation-hosting.md
+51 −0 website/.vitepress/config.mts
+54 −0 website/.vitepress/theme/InterfaceGuide.vue
+10 −1 website/.vitepress/theme/index.ts
+26 −6 website/README.md
+4 −2 website/content/en/developer/architecture.md
+4 −0 website/content/en/developer/contributing.md
+45 −0 website/content/en/developer/ecosystem/index.md
+286 −0 website/content/en/developer/ecosystem/source-extension.md
+8 −4 website/content/en/developer/index.md
+84 −0 website/content/en/getting-started.md
+67 −0 website/content/en/guide/collect.md
+63 −0 website/content/en/guide/connect-cli.md
+52 −0 website/content/en/guide/connect.md
+62 −0 website/content/en/guide/daily-use.md
+67 −0 website/content/en/guide/extensions.md
+39 −0 website/content/en/guide/first-source.md
+87 −0 website/content/en/guide/schedules.md
+82 −0 website/content/en/guide/search.md
+196 −0 website/content/en/guide/sinks/chatgpt.md
+47 −0 website/content/en/guide/sources.md
+79 −0 website/content/en/guide/sources/github.md
+118 −0 website/content/en/guide/sources/mail.md
+108 −0 website/content/en/guide/sources/memos.md
+104 −0 website/content/en/guide/sources/rss.md
+101 −0 website/content/en/guide/sources/telegram.md
+221 −0 website/content/en/guide/sources/twitter.md
+25 −0 website/content/en/guide/troubleshooting.md
+7 −3 website/content/en/index.md
+80 −0 website/content/en/self-hosted/advanced.md
+51 −0 website/content/en/self-hosted/getting-started.md
+56 −0 website/content/en/self-hosted/heroku-neon.md
+36 −0 website/content/en/self-hosted/index.md
+58 −0 website/content/en/self-hosted/render-neon.md
+37 −0 website/extension-docs/README.md
+56 −0 website/extension-docs/preset.mts
7 changes: 7 additions & 0 deletions extensions/memos/docs/global/.vitepress/config.mts
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
import { extensionDocs } from '../../../../../docs/_shared/website/extension-docs/preset.mts'

export default extensionDocs({
name: 'inkcre/memos', title: 'Memos for InKCre', scope: 'global',
description: 'Capture notes in InKCre with a Memos-compatible app.',
sidebar: [{ text: 'Overview', link: '/' }, { text: 'Connect Your App', link: '/connect' }],
})
1 change: 1 addition & 0 deletions extensions/memos/docs/global/.vitepress/theme/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
export { default } from '../../../../../../docs/_shared/website/.vitepress/theme/index'
70 changes: 70 additions & 0 deletions extensions/memos/docs/global/connect.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
---
description: Prepare a server URL and personal access token, then connect your Memos client.
outline: false
---

# Connect your app

Prepare `inkcre/memos` on your Core. If you use the Web setup interface, install its matching Web
Distribution and enable it in your browser as well. Installation, Core enablement, and browser
enablement are separate actions; both Hosts share one installed Extension version.

Your app needs two values: the Core server URL ending in `/memos`, and the Memos personal access
token (PAT). Neither is your PostgREST connection setting, and the PAT is not the instance JWT.

<InterfaceGuide>
<template #web>

## Prepare in the Web app

1. Open **Extensions**, locate **Memos**, and open **Setup** from its enabled browser extension.
2. Select the Core that should provide the service if more than one is available. If the public
address is missing, configure that Core's Public HTTP Base URL in **Clients → Config**, then
refresh the setup page. Use an address reachable from your phone or other client device.
3. If no PAT is saved, select **Generate PAT**, then **Save PAT and enable Memos**. A generated value
is only a draft until saved. If a PAT already exists, the wizard keeps it; enable Core if needed.
4. Copy the displayed **Server URL** and **Personal Access Token**. Keep the token private. The page
confirms that connection information is ready, not that an external app has connected.

</template>
<template #cli>

## Prepare through an Agent or operator

Ask your Agent or operator to configure `personal_access_token` using the Core Extension management
capability, then enable `inkcre/memos` on the intended Core. The value must be `memos_pat_` followed
by 32 ASCII letters or digits, generated with a cryptographic random generator. Do not reuse a JWT
or another service's password.

The configuration belongs to the deployment's existing Extension record. Do not write directly to
the database and assume a running Core has reloaded it. Obtain the Core's public base address and
append `/memos`, preserving any existing base path. Do not append `/api/v1`.

</template>
</InterfaceGuide>

## Sign in from the client

1. Open the client application's server sign-in screen.
2. Paste the exact Server URL, including `/memos`.
3. Paste the PAT into its personal-access-token field and sign in.
4. Confirm that the app opens normally. Preparing or copying credentials alone does not test the
phone's network access or the client's protocol compatibility.

You can now deliberately create a note in the client. The setup interface itself does not create
test notes or collect information. Connecting here does not upload notes kept in an unrelated
account automatically.

## Recover or disconnect

If saving fails, correct the connection problem and retry with the retained draft. If saving
succeeds but Core enablement fails, retry enabling with the same saved PAT. After a timeout, refresh
the actual saved state before trying again.

If client sign-in fails, first check that the device can reach the Core address and that Memos is
enabled there. A browser and a phone may have different network access. Also check the client's
version against the supported baseline; do not add API paths to the URL to guess around an error.

To replace or revoke the PAT, explicitly change Extension Config through Core. Replacing it affects
every client using the old PAT; update those clients manually. Disabling Memos on Core removes its
API routes. Closing the setup page does not disable the service or revoke its token.
Loading
Loading