925b6051a3
abi-capability-surface.md is the canonical plugin-need → ABI-mechanism map (manifest / host_call / hook), with WZ-020 rows marked. wasm-abi.md: content family drift fixed (list_posts), new families and hooks documented, open items closed except slot/media/embed render resolvers. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
603 lines
36 KiB
Markdown
603 lines
36 KiB
Markdown
# Wasm Plugin ABI (v1)
|
||
|
||
The host↔guest wire contract for wazero-loaded BlockNinja plugins.
|
||
Schema: [`abi/proto/v1/`](../abi/proto/v1/) (buf module [`abi/`](../abi/)) —
|
||
generated Go: `git.dev.alexdunmow.com/block/core/abi/v1` (`abiv1`).
|
||
Design rationale: the wasm plugin migration design spec in the cms repo
|
||
(`docs/superpowers/specs/2026-07-03-wasm-plugin-migration-design.md`).
|
||
|
||
Regenerate with `make abi` (runs `buf lint` + `buf generate` in `abi/`).
|
||
The `abi/` buf module is deliberately separate from the repo-root buf config:
|
||
`proto/` is the shared block/proto git submodule (service API contracts),
|
||
while this ABI is SDK-internal and versions in lockstep with the guest shim,
|
||
so it lives repo-local.
|
||
|
||
## Versioning — `abi_version`
|
||
|
||
`PluginManifest.abi_version` (field 1, `manifest.proto`) carries the ABI
|
||
**major** version the plugin was built against. Current value: **1**.
|
||
|
||
- The host **rejects** any manifest whose major version it does not support —
|
||
at install/publish time (manifest read) and again at `DESCRIBE`
|
||
(`DescribeRequest.host_abi_version` tells the guest who is calling, so a
|
||
newer guest shim can refuse an older host symmetrically).
|
||
- Within a major version, evolution is protobuf-additive only: new fields,
|
||
new `Hook` values, new capability methods. Removing or renaming anything
|
||
wire-visible requires a major bump. `buf breaking` (FILE rules, configured
|
||
in `abi/buf.yaml`) enforces this against the previous commit.
|
||
|
||
## Module lifecycle — REACTOR mode (`_initialize`, no `_start`)
|
||
|
||
Plugins compile as WASI **reactors** (Go ≥ 1.24 toolchain for
|
||
`go:wasmexport`; this repo's floor is higher — check `go.mod`):
|
||
|
||
```
|
||
GOOS=wasip1 GOARCH=wasm go build -buildmode=c-shared -o plugin.wasm .
|
||
```
|
||
|
||
with one boilerplate main file next to the untouched Registration:
|
||
|
||
```go
|
||
//go:build wasip1
|
||
|
||
package main
|
||
|
||
import "git.dev.alexdunmow.com/block/core/plugin/wasmguest"
|
||
|
||
func init() { wasmguest.Serve(Registration) }
|
||
func main() {} // never called — reactor mode
|
||
```
|
||
|
||
A reactor module exports `_initialize` instead of `_start`. The host MUST
|
||
run `_initialize` exactly once per instance **before any `bn_invoke`**
|
||
(wazero: `ModuleConfig.WithStartFunctions("_initialize")`); it runs package
|
||
init funcs — hence `Serve`, which stores the registration and returns.
|
||
`main` exists only to satisfy the linker and is never called.
|
||
|
||
> Command mode (plain `go build`) does NOT work and must not be used
|
||
> (empirically verified, Go 1.26 + wazero v1.12.0, 2026-07-03): `_start`
|
||
> runs `main` synchronously, so a blocking `main` (`select{}`) trips the Go
|
||
> deadlock detector and traps, while a returning `main` exits and closes
|
||
> the module — either way the exports are never callable. The original
|
||
> blocking-main design was amended to reactor mode for this reason.
|
||
|
||
## Building & packing (`ninja plugin build`)
|
||
|
||
`ninja plugin build` turns a plugin repo into a `.bnp` artifact in one command —
|
||
no Docker/podman, just the local Go toolchain (**≥ 1.24**, enforced with a clear
|
||
error) plus wazero. It replaces the in-container `.so` compile and the old
|
||
`make build-so`. Steps:
|
||
|
||
1. Parse `plugin.mod` for `name`/`version` (both required).
|
||
2. `GOOS=wasip1 GOARCH=wasm go build -buildmode=c-shared -o plugin.wasm .`
|
||
(reactor mode).
|
||
3. Extract `manifest.pb`: instantiate `plugin.wasm` with wazero and drive one
|
||
`HOOK_DESCRIBE`. The host functions are **stubbed to fail** — DESCRIBE must
|
||
not need a live host, so a plugin that reaches a capability at
|
||
register/describe time (e.g. a `db.*` call from `Register`) gets an
|
||
actionable error **naming the offending method** instead of a hang.
|
||
4. Stamp `manifest.data_dir` from `plugin.mod` (see below).
|
||
5. Pack a `tar.zst`: `plugin.wasm`, `plugin.mod`, `manifest.pb`, plus
|
||
`migrations/`, `schemas/`, `assets/`, and `web/dist` (Module Federation
|
||
output, flattened under `web/`) when present.
|
||
|
||
```
|
||
ninja plugin build [--dir .] [-o <name>-<version>.bnp]
|
||
ninja plugin verify <file.bnp>
|
||
```
|
||
|
||
`ninja plugin verify` re-runs the CMS `.bnp` reader's checks standalone (layout,
|
||
required members, path-safety + size caps on extraction, `abi_version` support,
|
||
and manifest name == `plugin.mod` name) so CI and the registry can gate uploads.
|
||
These rules are a **deliberate duplication** of the reader
|
||
(`cms backend/plugin/bnp/reader.go`); core cannot import the CMS module, and
|
||
WO-WZ-010's integration suite keeps the two in lockstep.
|
||
|
||
### Makefile convention — `make build-wasm`
|
||
|
||
Plugin repos expose a `build-wasm` target (replacing `build-so`) that just calls
|
||
the CLI:
|
||
|
||
```make
|
||
.PHONY: build-wasm
|
||
build-wasm:
|
||
ninja plugin build
|
||
```
|
||
|
||
### `plugin.mod` `data_dir`
|
||
|
||
`plugin.mod` may set an optional first-class boolean:
|
||
|
||
```toml
|
||
[plugin]
|
||
name = "my-plugin"
|
||
version = "0.1.0"
|
||
data_dir = true # request a persistent per-plugin /data preopen at load
|
||
```
|
||
|
||
`data_dir` is a real field on the mod parser (not an arbitrary key) precisely so
|
||
the CLI's mod round-trip cannot silently drop it — `writeMod` reconstructs
|
||
`plugin.mod` from known struct fields only. `ninja plugin build` copies it into
|
||
`PluginManifest.data_dir`; OFF by default.
|
||
|
||
## Calling convention (ptr+len, packed u64)
|
||
|
||
wasm exports can only pass `i32/i64/f32/f64`, so all payloads cross as
|
||
protobuf bytes in guest linear memory. The guest exports (via
|
||
`go:wasmexport`):
|
||
|
||
| Export | Signature | Purpose |
|
||
|---|---|---|
|
||
| `bn_alloc` | `(size: u32) → ptr: u32` | Host asks the guest to allocate `size` bytes in guest memory. The returned region stays valid until the current `bn_invoke` call returns. |
|
||
| `bn_invoke` | `(hook_id: u32, ptr: u32, len: u32) → packed: u64` | Host writes a serialized `InvokeRequest` at `(ptr, len)` (memory from `bn_alloc`) and calls with `hook_id = Hook` enum value (duplicated in the envelope for decode sanity). The return packs the response location: `packed = (ptr << 32) | len`, framing a serialized `InvokeResponse` in guest memory, valid until the next `bn_invoke` on this instance. |
|
||
|
||
Host functions (guest→host capability calls) live in wasm import module
|
||
**`blockninja`** and use the same shape in reverse. There is exactly ONE
|
||
generic import rather than one symbol per capability family:
|
||
|
||
```
|
||
(blockninja) host_call(ptr: u32, len: u32) → packed: u64
|
||
```
|
||
|
||
The guest passes `(ptr, len)` framing a serialized `HostCallRequest` (whose
|
||
`method` string — `"<family>.<snake_method>"`, including the `db.*` driver
|
||
methods — already selects the family); the host returns a packed `u64`
|
||
framing a `HostCallResponse` that it wrote into guest memory via `bn_alloc`.
|
||
The guest shim releases that buffer after decoding, so the host must not
|
||
reuse it. Per-family import symbols were considered and rejected (decision,
|
||
WO-WZ-002): they would add ~40 declarations on both sides for zero type
|
||
safety, since the payloads are opaque protobuf bytes either way.
|
||
|
||
Buffers MUST come from `bn_alloc` on both paths — the guest rejects a
|
||
`bn_invoke` request pointer it did not hand out (`ABI_ERROR_CODE_DECODE`),
|
||
and treats an unknown host-call response pointer the same way.
|
||
|
||
A `packed` value of `0` means the callee could not even produce an envelope
|
||
(allocation failure / trap); the caller treats it as
|
||
`ABI_ERROR_CODE_INTERNAL` and discards the instance.
|
||
|
||
> A logically-empty response is **not** packed `0`. A successful hook whose
|
||
> response message has no set fields (e.g. `LoadResponse`/`UnloadResponse`)
|
||
> proto-marshals to zero bytes; the guest still frames it as `(ptr, 0)` with a
|
||
> real pointer so the host reads a valid empty envelope. `bn_invoke` never
|
||
> returns packed `0` for a successful call. (Fixed in WO-WZ-003: the earlier
|
||
> `len==0 → return 0` shortcut made every successful empty-response hook —
|
||
> notably `HOOK_LOAD` — look like an INTERNAL failure and discard the
|
||
> instance. The cms host in WO-WZ-006 must likewise not conflate a
|
||
> zero-length payload with a missing envelope.)
|
||
|
||
Instances are single-threaded: one `bn_invoke` at a time per instance;
|
||
concurrency comes from the per-plugin instance pool.
|
||
|
||
### Per-instance state: block pools & `HostServices`
|
||
|
||
Because concurrency is per-instance, guest state a render path depends on must be
|
||
established on **every** pooled instance, not just the one the host runs the
|
||
`HOOK_LOAD` hook on. `HOOK_LOAD` fires exactly once per plugin (on one acquired
|
||
instance); the deps-receiving entry points (`HTTPHandler`/`JobHandlers` init)
|
||
init lazily and only on instances that serve those hooks. A block render
|
||
(`HOOK_RENDER_BLOCK`) receives **no** services — only `ctx` + content — so a
|
||
DB-backed block cannot get its pool from Load.
|
||
|
||
The same rule governs the WO-WZ-019 callback hooks: AI tool handlers
|
||
(`ai.tools.register`) and bridge service values (`Bridge.RegisterService`)
|
||
are recorded per instance, and `HOOK_AI_TOOL_CALL` / `HOOK_BRIDGE_CALL` may
|
||
land on ANY pooled instance — so register them in `Register`, not `Load`.
|
||
|
||
The escape hatch is `wasmguest.HostServices()`: it returns the `CoreServices`
|
||
bound at `_initialize` on **every** instance (live `db.*` `Pool` + capability
|
||
stubs). A DB-backed block registers its pool from `HostServices().Pool` inside
|
||
`Register` (which `newGuest` runs on every instance, after the Pool is bound) —
|
||
see symposium's `register.go`. In native/DESCRIBE builds `HostServices()` returns
|
||
the zero value (Serve never ran), so callers nil-check `Pool`.
|
||
|
||
### Instance pool sizing & memory budget (WO-WZ-012)
|
||
|
||
Defaults (`wasmhost.DefaultConfig`, overridable via `WASM_POOL_MAX_SIZE` /
|
||
`WASM_MEMORY_LIMIT_MB`): **pool of 4 live instances per plugin, 512 MiB linear
|
||
memory cap per instance**, 30 s call deadline, 5 m idle TTL. The compiled module
|
||
(machine code + data segments) is compiled **once** per plugin and shared across
|
||
its pool; only each instance's linear memory + Go heap is per-instance.
|
||
|
||
Measured against the ported **symposium** plugin (45 MiB wasm — the fleet's
|
||
largest; representative public block mix = wiki index + course index + community
|
||
feed, one sqlc list query each over the `db.*` bridge, real wazero + Postgres):
|
||
|
||
| Metric | Value |
|
||
|---|---|
|
||
| Page render (3-block mix) p50 / p95 / p99 | **5.5 ms / 9.4 ms / 13.9 ms** |
|
||
| Per-block render (incl. DB round-trip) | ~1.8–3 ms |
|
||
| Process RSS, pool=1 (compile + 1 instance) | ~400–500 MiB |
|
||
| Process RSS, pool=2 under concurrent load | ~540 MiB |
|
||
| Process RSS, pool=4 under concurrent load | ~628 MiB |
|
||
| Marginal cost per extra pooled instance | ~50–130 MiB |
|
||
|
||
The ~400 MiB fixed cost is dominated by wazero compiling the 45 MiB module (once,
|
||
shared); marginal instances are cheap. Against the default orchestrator container
|
||
limit (`CONTAINER_MEMORY_MB` = **4096 MiB**), symposium at pool=4 (~628 MiB) plus
|
||
two smaller site-plugin pools + the CMS base fits comfortably. **Decision: keep
|
||
pool=4 / 512 MiB.** Memory-constrained deployments (≤1 GiB containers running the
|
||
largest plugins) should lower `WASM_POOL_MAX_SIZE` to 2.
|
||
|
||
Latency gate: the `.so` baseline for the identical block mix is unavailable on
|
||
`cms` main (the `.so` loader/builder was deleted in the big-bang cutover,
|
||
a04277ee2/da6177e1e), so a direct ≤25% p95 regression comparison cannot be run.
|
||
Absolute wasm numbers are recorded above; the wasm boundary overhead is small
|
||
relative to the per-block DB round-trip (~sub-ms of the ~3 ms), so a material
|
||
regression is not expected — recorded here for **explicit sign-off** per the
|
||
adjusted gate. Bench harness: `cms/backend/plugin/wasmintegration/symbench`.
|
||
|
||
## Hook catalog (`invoke.proto`)
|
||
|
||
Host→guest calls. `InvokeRequest{hook, payload, deadline_ms}` →
|
||
`InvokeResponse{payload, error}`; `payload` holds the hook-specific message:
|
||
|
||
| Hook | Request / Response | Fires |
|
||
|---|---|---|
|
||
| `HOOK_RENDER_BLOCK` | `RenderBlockRequest` / `RenderBlockResponse` | Public render of one plugin block (`blocks.BlockFunc`). The response carries **either** final `html` **or** a `powered` `{template, data_json}` result — see [Powered blocks](#powered-blocks--render-as-a-host-capability). |
|
||
| `HOOK_RENDER_TEMPLATE` | `RenderTemplateRequest` / `RenderTemplateResponse` | Render of one plugin template (`templates.TemplateFunc`). |
|
||
| `HOOK_RENDER_TAG` | `RenderTagRequest` / `RenderTagResponse` | Run a plugin-declared template tag (`manifest.declared_tags`) the host engine hit while rendering a powered block (`blocks.RegisterTag`). Re-entrancy-free: the originating RENDER_BLOCK has returned. |
|
||
| `HOOK_APPLY_FILTER` | `ApplyFilterRequest` / `ApplyFilterResponse` | Apply a plugin-declared template filter (`manifest.declared_filters`, `blocks.RegisterFilter`). |
|
||
| `HOOK_HANDLE_HTTP` | `HttpRequest` / `HttpResponse` | Buffered HTTP/ConnectRPC request forwarded to the guest's internal mux (only when `manifest.has_http_handler`). No streaming/SSE/WebSocket in v1. |
|
||
| `HOOK_JOB` | `JobRequest` / `JobResponse` | Background job dispatch for a `manifest.job_types` entry (`plugin.JobHandlerFunc`). |
|
||
| `HOOK_LOAD` | `LoadRequest` / `LoadResponse` | Plugin load (`PluginRegistration.Load`); `LoadRequest.host_config` delivers `AppURL`/`MediaPath`. |
|
||
| `HOOK_UNLOAD` | `UnloadRequest` / `UnloadResponse` | Plugin unload (`PluginRegistration.Unload`). |
|
||
| `HOOK_RAG_FETCH` | `RagFetchRequest` / `RagFetchResponse` | RAG re-index callback for a `manifest.rag_content_fetcher_types` entry (`plugin.ContentFetcher`). |
|
||
| `HOOK_MEDIA_HOOK` | `MediaHookRequest` / `MediaHookResponse` | Media lifecycle event (`plugin.MediaHooksProvider`), only when `manifest.has_media_hooks`. |
|
||
| `HOOK_DESCRIBE` | `DescribeRequest` / `DescribeResponse` | Publish-time manifest capture; the result is stored as `manifest.pb` in the `.bnp` artifact. Never called on a live instance. |
|
||
| `HOOK_AI_TOOL_CALL` | `AiToolCallRequest` / `AiToolCallResponse` | Execute the guest handler of a tool registered via `ai.tools.register`. Register tools in `Register` (every pooled instance runs it), not `Load`. |
|
||
| `HOOK_BRIDGE_CALL` | `BridgeCallRequest` / `BridgeCallResponse` | Provider side of `bridge.invoke`: run a method on one of THIS plugin's registered bridge services (`plugin.BridgeInvokable`). Register services in `Register`. |
|
||
| `HOOK_DIRECTORY_PANEL_SECTION` | `DirectoryPanelSectionRequest` / `DirectoryPanelSectionResponse` | Render the index-th `DirectoryExtensions.PanelSections` callback. |
|
||
| `HOOK_DIRECTORY_PIN_DECORATOR` | `DirectoryPinDecoratorRequest` / `DirectoryPinDecoratorResponse` | Run the index-th pin decorator; the mutated pin map is returned. |
|
||
|
||
## Capability calls (`capability.proto`, `db.proto`)
|
||
|
||
Guest→host. Envelope: `HostCallRequest{method, payload}` →
|
||
`HostCallResponse{payload, error}`. `method` is `"<family>.<snake_method>"`;
|
||
each pair mirrors one Go interface method from `CoreServices` 1:1
|
||
(UUIDs as canonical strings, `map[string]any`/JSON as bytes):
|
||
|
||
| Family | Methods | Go surface |
|
||
|---|---|---|
|
||
| `content` | `get_author_profile`, `get_page`, `get_post`, `list_posts`, `slugify`, `block_note_to_html`, `generate_excerpt`, `strip_html` — plus the WO-WZ-019 writes: `create_page`, `set_page_blocks`, `publish_page`, `set_page_seo`, `upsert_post` | `content.Content`, `content.Author` |
|
||
| `settings` | `get_site_settings`, `get_plugin_settings`, `update_site_setting`, `update_plugin_settings` (host-pinned to the caller's own plugin) | `settings.Settings`, `settings.Updater` |
|
||
| `gating` | `get_subscriber_tier_level`, `evaluate_access` | `gating.Gating` |
|
||
| `crypto` | `encrypt_secret`, `decrypt_secret` | `crypto.Crypto` |
|
||
| `menus` | `get_menu_by_name`, `get_menu_items` | `menus.Menus` |
|
||
| `datasources` | `resolve_bucket`, `resolve_bucket_by_key` | `datasources.Datasources` |
|
||
| `users` | `get_by_username`, `get_by_id` | `auth.PublicUsers` |
|
||
| `subscriptions` | `get_user_tier_level`, `get_tier_by_slug`, `list_tiers`, `list_active_plans` | `subscriptions.Subscriptions` |
|
||
| `media` | `deposit` | `plugin.Media` |
|
||
| `email` | `send` | `plugin.EmailSender` |
|
||
| `ai` | `text_call`, `tools.register` | `CoreServices.AITextCall`, `ai.ToolRegistry` |
|
||
| `bridge` | `register_service`, `get_service`, `invoke` (opaque-payload cross-plugin calls; provider answers via `HOOK_BRIDGE_CALL` or an in-process `plugin.BridgeInvokable`) | `plugin.PluginBridge` |
|
||
| `jobs` | `submit`, `progress` (correlated to the running job via the HOOK_JOB call context) | `plugin.JobRunner` + `JobHandlerFunc`'s progress callback |
|
||
| `provisioner` | `ensure_data_table`, `merge_site_settings`, `ensure_setting`, `ensure_page`, `override_site_settings`, `ensure_menu_item`, `register_embedding_config`, `ensure_embed`, `ensure_job_schedule`, `update_data_table_row_field`, `disable_orphaned_job_schedules`, `ensure_plugin`, `ensure_custom_color`, `ensure_media` — **Load-time**: call `deps.Provisioner` from `Load`; `RegisterWithProvisioner` is inert under wasm (DESCRIBE stubs host functions). `EnsureEmbed` rejects `RenderFunc`-only embeds (functions cannot cross). | `plugin.Provisioner` |
|
||
| `embeddings` | `generate_embedding`, `embed_content`, `is_available` | `plugin.EmbeddingService` |
|
||
| `rag` | `query`, `on_content_changed` | `plugin.RAGService` |
|
||
| `reviews` | `submit_review` | `plugin.ReviewSubmitter` |
|
||
| `badges` | `refresh_badges` | `plugin.BadgeRefresher` |
|
||
| `db` | `query`, `exec`, `tx_begin`, `tx_commit`, `tx_rollback` | `CoreServices.Pool` via the guest `database/sql` driver (`db.proto`) |
|
||
|
||
The guest half of the SDK implements the existing Go interfaces as stubs
|
||
marshaling to these calls (`core/plugin/wasmguest/caps/`, WO-WZ-003), so
|
||
plugin code compiles unchanged. `caps.NewCoreServices(call)` assembles them;
|
||
the wasm shim binds `call` to the real `host_call` transport, tests inject a
|
||
fake, and a nil transport (DESCRIBE probes) fails every capability cleanly
|
||
instead of nil-panicking.
|
||
|
||
### Method disposition (every `CoreServices` member)
|
||
|
||
No silent gaps: each member is either a guest stub or served host-side.
|
||
|
||
| Member | Disposition |
|
||
|---|---|
|
||
| `Content` (8 methods) | **stub** — `caps/content.go` |
|
||
| `ContentAuthor` (5 write methods, WO-WZ-019) | **stub** — `caps/author.go` |
|
||
| `Provisioner` (14 methods, WO-WZ-019) | **stub** — `caps/provisioner.go`; Load-time only (see the `provisioner` family row) |
|
||
| `Settings` / `SettingsUpdater` | **stub** — `caps/settings.go` (one value, both fields) |
|
||
| `Gating` | **stub** — `caps/gating.go`; `EvaluateAccess` crosses but falls back to the pure `gating.EvaluateAccess` on transport error |
|
||
| `Crypto` | **stub** — `caps/crypto.go` |
|
||
| `Menus` | **stub** — `caps/menus.go` |
|
||
| `Datasources` | **stub** — `caps/datasources.go` |
|
||
| `PublicUsers` | **stub** — `caps/users.go` |
|
||
| `Subscriptions` | **stub** — `caps/subscriptions.go` |
|
||
| `Media` | **stub** — `caps/media.go` |
|
||
| `ToolRegistry` + `AITextCall` | **stub** — `caps/ai.go` (`ai.tools.register` + `ai.text_call`); the tool `Handler` stays guest-side, recorded per instance and executed via `HOOK_AI_TOOL_CALL` — register tools in `Register`, not `Load` |
|
||
| `EmailSender` | **stub** — `caps/email.go` |
|
||
| `Bridge` | **stub** — `caps/bridge.go`; `RegisterService` forwards the name AND records the value locally for `HOOK_BRIDGE_CALL` dispatch; `GetService` reports availability but returns `nil` (a typed value cannot cross — by design); cross-plugin calls use `Invoke` (`bridge.invoke`, opaque payloads) |
|
||
| `ReviewSubmitter` | **stub** — `caps/reviews.go` |
|
||
| `BadgeRefresher` | **stub** — `caps/badges.go` |
|
||
| `JobRunner` | **stub** — `caps/jobs.go` |
|
||
| `EmbeddingService` | **stub** — `caps/embeddings.go` |
|
||
| `RAGService` | **stub** — `caps/rag.go`; `Query`/`OnContentChanged` cross, `RegisterContentFetcher` records guest-side for `HOOK_RAG_FETCH` |
|
||
| `Pool` | **host-side** — the `db.*` driver (db.proto), per-plugin Postgres role |
|
||
| `Interceptors` | **host-side** — the host builds the connect option chain; RBAC merges from `manifest.rbac_method_roles`. The caller's **verified** identity reaches the guest via **trusted identity headers** (see "Trusted identity headers" below) — never by decoding a client token. |
|
||
| `AppURL` / `MediaPath` | **host-side** — delivered once in `LoadRequest.host_config` |
|
||
| `CoreServiceBindings` | **host-side** — static `manifest.core_service_bindings`; the host constructs and mounts the `http.Handler` (cannot cross the sandbox), so `caps` provides no stub |
|
||
|
||
Interface satisfaction is proven at compile time by a `var _ <iface> =
|
||
(*stub)(nil)` line per family; a wasip1 build of `testdata/fixture` (whose
|
||
`Load` hook calls `deps.Content`/`deps.Settings`/`deps.Bridge` unchanged) plus
|
||
the `TestWasmFixtureCapabilityRoundTrip` end-to-end wazero test prove the path
|
||
crosses the ABI for real.
|
||
|
||
Error mapping (`caps/caps.go`): a transport `AbiError` surfaces as a Go error
|
||
wrapped with `<family>.<method>` context; an `ABI_ERROR_CODE_DEADLINE_EXCEEDED`
|
||
reply is mapped onto `context.DeadlineExceeded` so `errors.Is` keeps working.
|
||
Methods without an error channel (`Slugify`, `IsAvailable`, `EvaluateAccess`,
|
||
`ToolRegistry.Register`, `Bridge.*`, `RAG.OnContentChanged`, …) degrade to the
|
||
zero value / best-effort on transport failure.
|
||
|
||
`CoreServices` members that do **not** cross as capability calls:
|
||
|
||
- `Pool` → the `db.*` driver messages (`db.proto`); the host executes under
|
||
the per-plugin Postgres role. `DbError.code` carries the SQLSTATE.
|
||
Transactions: `tx_begin` returns an opaque `tx_handle` (never 0); `query`/
|
||
`exec` with `tx_handle = 0` run autocommit. Handles die with the call
|
||
chain's deadline so a guest can never pin a connection. **Guest side
|
||
(WO-WZ-004):** `core/plugin/wasmguest/bnwasm` implements this over the
|
||
transport as two surfaces — a `database/sql` driver registered as `"bnwasm"`,
|
||
and a `plugin.Pool` handing out a `pgx.Tx`-shaped value. The latter is the
|
||
primary path: current plugins' sqlc configs use `sql_package: "pgx/v5"`, so
|
||
their generated `DBTX` needs `pgconn.CommandTag`/`pgx.Rows`/`pgx.Row` (which
|
||
`database/sql` cannot produce), and the `Pool`/`Tx` satisfy it with no source
|
||
edits. `DbError` surfaces as `*pgconn.PgError` (SQLSTATE preserved for
|
||
`errors.As`). Named args (`pgx.NamedArgs`/`QueryRewriter`) and nested
|
||
transactions/savepoints are rejected with clear errors — no fleet plugin uses
|
||
either. The DbValue↔Go scan mapping is pinned in the exported
|
||
`bnwasm.DbValueFixtures` table, which the WO-WZ-007 host executor mirrors.
|
||
**`text[]` NULL-element limit:** `DbValue.text_array` (`abiv1.TextArray`) is a
|
||
repeated string with no per-element NULL, so a Postgres `text[]` like
|
||
`{a,NULL,b}` cannot round-trip — a NULL element collapses to `""`. The array as
|
||
a whole can still be SQL NULL (nil `[]string` → `DbValue_Null`); only a NULL
|
||
*inside* the array is unrepresentable. The host executor must honor this same
|
||
limit (encode a NULL element as `""` or reject it), not invent a sentinel.
|
||
**`uuid[]` (`DbValue.uuid_array_value`, `abiv1.UuidArray`):** a first-class
|
||
variant distinct from `text[]`, so a query keeps native `ANY($1::uuid[])`
|
||
(no `::text[]::uuid[]` cast workaround) and a `uuid[]` column scans straight
|
||
into `[]uuid.UUID`. Each element is a canonical UUID string (like the scalar
|
||
`uuid_value`); nil `[]uuid.UUID` → `DbValue_Null`, empty stays a non-NULL
|
||
empty `uuid[]`. The host binds a native `[]uuid.UUID` parameter and reads a
|
||
`UUIDArrayOID` column back into this variant. Pinned by the `uuid_array`
|
||
entry in `bnwasm.DbValueFixtures`.
|
||
- `Interceptors` (`connect.Option`) → host-side only; RBAC merges from
|
||
`manifest.rbac_method_roles`.
|
||
- `AppURL` / `MediaPath` → delivered once in `LoadRequest.host_config`.
|
||
- `CoreServiceBindings.Bind` → static `manifest.core_service_bindings`
|
||
declaration; the host constructs and mounts the handlers.
|
||
- `RAGService.RegisterContentFetcher` → static
|
||
`manifest.rag_content_fetcher_types` declaration + `HOOK_RAG_FETCH`
|
||
callback inversion.
|
||
|
||
## Trusted identity headers
|
||
|
||
Context values do **not** cross the ABI, so a guest cannot see the
|
||
`auth.Claims` / `auth.PublicClaims` the host's middleware built. A guest that
|
||
needs the caller's identity (its Connect RPCs call
|
||
`auth.GetUserFromContext` / `GetPublicUserFromContext`) reads it from
|
||
**host-set trusted headers**, and **only** from those.
|
||
|
||
**Trust model (host-enforced, guest-trusting):**
|
||
|
||
1. Upstream CMS auth middleware verifies the JWT signature and populates
|
||
`r.Context()` with the principal. The wasm mount's RBAC guard then runs
|
||
`rbac.Authorize` against that **verified** principal (deny-by-default,
|
||
live-user validator) *before* the request is forwarded.
|
||
2. When forwarding over `HOOK_HANDLE_HTTP`, the host **strips any
|
||
client-supplied copy** of the trusted headers from the request and **sets
|
||
them itself** from the verified context principal. A guest therefore trusts
|
||
them unconditionally — they are not attacker-controllable.
|
||
3. The guest reconstructs its context with `auth.TrustedHeaderMiddleware`
|
||
(wrap it around your `HTTPHandler`) or `auth.ContextFromTrustedHeaders`.
|
||
No token parsing, no signature check — the guest holds no signing secret by
|
||
design.
|
||
|
||
The headers (`core/auth/trustedheaders.go`, canonical MIME form):
|
||
|
||
| Header | Source |
|
||
|---|---|
|
||
| `X-Bn-Verified-User-Id` / `X-Bn-Verified-Role` / `X-Bn-Verified-Email` | admin `auth.Claims` |
|
||
| `X-Bn-Verified-Public-User-Id` / `X-Bn-Verified-Public-Username` / `X-Bn-Verified-Public-Email` | public `auth.PublicClaims` |
|
||
|
||
> **SECURITY — do NOT decode a client token in the guest.** The host forwards
|
||
> the raw request, so its `Cookie` / `Authorization` headers are
|
||
> attacker-controlled across the boundary. Decoding an unsigned `access_token`
|
||
> cookie (or Bearer JWT) as identity in the guest is a **privilege-escalation
|
||
> bug** (a verified public user can forge an admin JWT the guest would then
|
||
> honour on a `RolePublic` method). Identity comes from the trusted headers
|
||
> above and nowhere else.
|
||
|
||
## Error semantics
|
||
|
||
`AbiError{code, message}` travels in `InvokeResponse.error` and
|
||
`HostCallResponse.error`:
|
||
|
||
| Code | Meaning | Instance consequence |
|
||
|---|---|---|
|
||
| `ABI_ERROR_CODE_INTERNAL` | Handler ran and failed; `message` is the Go error text. | None (normal error). Guest traps / packed `0` returns are *treated as* INTERNAL by the host and **do** discard the instance. |
|
||
| `ABI_ERROR_CODE_DECODE` | Envelope or payload failed to decode. | Instance discarded (protocol desync). |
|
||
| `ABI_ERROR_CODE_UNIMPLEMENTED` | Callee does not implement the hook/capability (e.g. host too old for a new capability method). | None. |
|
||
| `ABI_ERROR_CODE_DEADLINE_EXCEEDED` | `deadline_ms` elapsed. | Instance considered poisoned, discarded. |
|
||
| `ABI_ERROR_CODE_PERMISSION_DENIED` | Caller not entitled to the capability. | None. |
|
||
| `ABI_ERROR_CODE_TX_EXPIRED` | A `db.*` call named a transaction handle the host already expired (dropped at the call-chain deadline, WO-WZ-007). | None — retryable. The guest maps it to `bnwasm.ErrTxExpired`; plugin code re-runs the unit of work in a fresh transaction. |
|
||
|
||
`ABI_ERROR_CODE_TX_EXPIRED` is emitted by the cms `dbexec` side (adopted
|
||
separately from this WO) in place of the earlier INTERNAL+message-marker
|
||
workaround, so guests can distinguish a retryable tx expiry from a real fault
|
||
via `errors.Is(err, bnwasm.ErrTxExpired)`.
|
||
|
||
Host-function errors surface to plugin code as ordinary Go errors via the
|
||
guest SDK. Repeated instance failures trip the existing
|
||
`PluginStatusFailed` path + admin notification. DB failures use `DbError`
|
||
(SQLSTATE-carrying) inside the `db.*` responses instead of `AbiError`, so
|
||
sqlc/pgx error handling keeps working.
|
||
|
||
## Manifest ↔ `PluginRegistration` mapping
|
||
|
||
`manifest.pb` (a serialized `PluginManifest`) is produced at publish time via
|
||
`HOOK_DESCRIBE` and read by the loader without instantiating the module.
|
||
Field-by-field:
|
||
|
||
| `PluginRegistration` field | Wire counterpart |
|
||
|---|---|
|
||
| `Name` | `PluginManifest.name` |
|
||
| `Version` | `PluginManifest.version` |
|
||
| `Dependencies` | `dependencies` (`Dependency`) |
|
||
| `Register` | Static effects captured by DESCRIBE: `blocks` (`BlockMeta`), `block_template_overrides`, `template_keys`, `system_templates`, `page_templates`, `email_wrapper_system_keys`, `declared_tags`, `declared_filters` |
|
||
| `blocks.RegisterTag` / `blocks.RegisterFilter` (called in `Register`) | `declared_tags` / `declared_filters` + `HOOK_RENDER_TAG` / `HOOK_APPLY_FILTER` |
|
||
| `RegisterWithProvisioner` | Same captures + `has_provisioner` (provisioning runs at load, host-side) |
|
||
| `Assets` | `.bnp` artifact `assets/` directory (host serves directly; never crosses the boundary) |
|
||
| `Schemas` | `.bnp` artifact `schemas/` directory (host loads into the block registry) |
|
||
| `SettingsSchema` | `settings_schema` (JSON bytes) |
|
||
| `ThemePresets` | `theme_presets` (JSON bytes) |
|
||
| `BundledFonts` | `bundled_fonts` (JSON bytes) |
|
||
| `MasterPages` | `master_pages` (`MasterPageDefinition`/`MasterPageBlock`) |
|
||
| `HTTPHandler` | `has_http_handler` + `HOOK_HANDLE_HTTP` |
|
||
| `SettingsPanel` | `settings_panel` |
|
||
| `AdminPages` | `admin_pages` (`AdminPage`) |
|
||
| `CSSManifest` | `css_manifest` (`CssManifest`) |
|
||
| `ServiceHandlers` | `rbac_method_roles` (method → role; the services themselves answer via `HOOK_HANDLE_HTTP`) + `core_service_bindings` |
|
||
| `JobHandlers` | `job_types` + `HOOK_JOB` |
|
||
| `AIActions` | `ai_actions` (`AiAction`) |
|
||
| `DirectoryExtensions` | `directory_extensions` (static fields + callback counts) |
|
||
| `MediaHooks` | `has_media_hooks` + `HOOK_MEDIA_HOOK` |
|
||
| `Load` | `has_load_hook` + `HOOK_LOAD` |
|
||
| `Unload` | `has_unload_hook` + `HOOK_UNLOAD` |
|
||
| `Migrations` | `.bnp` artifact `migrations/` directory (Goose runs host-side; never crosses) |
|
||
| `RequiredIconPacks` | `required_icon_packs` |
|
||
| *(none — `plugin.mod` `data_dir`)* | `data_dir` (bool). Not a `PluginRegistration` field: the grant lives in `plugin.mod`, which the guest code cannot see, so `ninja plugin build` stamps `PluginManifest.data_dir` from `plugin.mod` after DESCRIBE. The `.bnp` reader also reads it straight from `plugin.mod` as a fallback. |
|
||
|
||
## Render context
|
||
|
||
`RenderContext` (`render.proto`) is the explicit envelope of every value
|
||
blocks read from `ctx` today (`core/blocks/context.go`): request info,
|
||
`BlockContext` (the pongo2 data struct, 1:1), current page/post/author/
|
||
category/master-page, requested path, injected/expected slots, editor flag,
|
||
block + page IDs, human-proof banner, detail row, theme variables JSON, and
|
||
locale.
|
||
|
||
Function-valued context entries cannot serialize; their v1 mapping:
|
||
|
||
- `GetQueries` → the `db.*` driver (plugin sqlc code, per-plugin role).
|
||
- `SlotRenderer` / `MediaResolver` / `EmbedResolver` → **open items** (below).
|
||
|
||
## Powered blocks — render as a host capability
|
||
|
||
pongo2/ninjatpl is a **host capability**: the template engine stays host-side
|
||
(cms) and is **never** compiled into a guest. A guest-reachable core package
|
||
(anything under `blocks/` or `plugin/wasmguest/`) MUST NOT import
|
||
`github.com/flosch/pongo2/*` — verified by `go list -deps` on the compiled
|
||
guest plugin showing zero `flosch/pongo2`. `core/templates/pongo` (the engine)
|
||
is core-resident but host-only; it is not in any guest's dependency graph.
|
||
|
||
Because the guest can't render a template itself, a template-backed block
|
||
**defers** rendering to the host. A `blocks.BlockFunc` returns EITHER:
|
||
|
||
- **plain HTML** — a non-template block returns its final string as always; or
|
||
- **a powered result** — `blocks.PoweredBlock(template, data)`: the block
|
||
builds its data (via capabilities like `Content.ListPosts`) and hands the
|
||
host a `{template, data}` pair instead of final HTML.
|
||
|
||
`PoweredBlock` encodes the `{template, data}` behind a NUL-delimited sentinel
|
||
in the `BlockFunc`'s `string` return, so **`BlockFunc`'s signature is
|
||
unchanged** (the smallest additive change — no ripple to existing blocks or the
|
||
host guest-side). The guest's RENDER_BLOCK handler decodes the sentinel
|
||
(`blocks.DecodePoweredBlock`) and fills `RenderBlockResponse.powered`
|
||
(`PoweredBlock{template, data_json}`); a plain HTML block fills
|
||
`RenderBlockResponse.html` and leaves `powered` nil.
|
||
|
||
### Flow (re-entrancy-free by construction)
|
||
|
||
1. Host calls guest `HOOK_RENDER_BLOCK`. A template-backed block returns a
|
||
**powered** result `{template, data_json}`; a plain block returns `html`.
|
||
2. **After the block-invoke returns**, the host renders the powered `template`
|
||
with `data_json` using pongo2, host-side. The guest instance is now free.
|
||
3. When the host engine hits a plugin-declared tag (`{% mytag %}`) or filter
|
||
(`{{ x|myfilter }}`), it calls back into the *free* guest instance:
|
||
`HOOK_RENDER_TAG {tag_name, args_json, render_context} → {html, error}` or
|
||
`HOOK_APPLY_FILTER {filter_name, input, args_json} → {output, error}`. Each
|
||
callback is a **fresh** `bn_invoke` — never nested inside the still-running
|
||
RENDER_BLOCK — so there is no guest re-entrancy.
|
||
|
||
### Plugin API (`core/blocks`)
|
||
|
||
Registered in the plugin's `Register` func (alongside blocks), guest-safe (no
|
||
pongo2):
|
||
|
||
```go
|
||
// A block returns a powered result instead of final HTML:
|
||
func MyBlock(ctx context.Context, content map[string]any) string {
|
||
posts := loadPosts(ctx) // via deps/HostServices capabilities
|
||
return blocks.PoweredBlock("{% for p in posts %}<li>{{ p.title }}</li>{% endfor %}",
|
||
map[string]any{"posts": posts})
|
||
}
|
||
|
||
// Custom tag: args are the parsed tag arguments; rctx is the render context
|
||
// (blocks.RenderContext == context.Context — read state via blocks.Get*).
|
||
blocks.RegisterTag("mytag", func(args map[string]any, rctx blocks.RenderContext) (string, error) {
|
||
return "<span>…</span>", nil
|
||
})
|
||
|
||
// Custom filter: input is the piped value; args are the filter arguments.
|
||
blocks.RegisterFilter("myfilter", func(input string, args map[string]any) (string, error) {
|
||
return strings.ToUpper(input), nil
|
||
})
|
||
```
|
||
|
||
`RegisterTag`/`RegisterFilter` write a package-level registry (one plugin owns
|
||
a wasm module). The guest resets it at the start of each registration's
|
||
`Register` pass (`runRegister`), so DESCRIBE captures exactly this plugin's
|
||
names into `declared_tags`/`declared_filters` and HOOK dispatch resolves them
|
||
via `blocks.LookupTag`/`LookupFilter`. Register tags/filters **in `Register`**
|
||
(not package `init`), so DESCRIBE — which runs `Register` — sees them.
|
||
|
||
A tag/filter fn error surfaces in `RenderTagResponse.error` /
|
||
`ApplyFilterResponse.error` (a normal logic failure the host decides how to
|
||
render). `AbiError` stays reserved for transport/decode/panic failures; a panic
|
||
in a tag/filter recovers to `ABI_ERROR_CODE_INTERNAL` and the instance stays
|
||
callable, matching every other hook.
|
||
|
||
### Host-side contract (the cms phase implements)
|
||
|
||
The core half (this SDK) defines the wire + guest dispatch. The cms host must:
|
||
|
||
1. **Render powered blocks host-side.** After `HOOK_RENDER_BLOCK` returns, if
|
||
`RenderBlockResponse.powered` is set, render `powered.template` with
|
||
`powered.data_json` (JSON → `map[string]any`) through the existing pongo2
|
||
engine (`core/templates/pongo`) and use that as the block's HTML. If `html`
|
||
is set instead, use it directly. Do the pongo2 render **outside** the
|
||
guest-invoke call so the instance is free for callbacks.
|
||
2. **Wire per-plugin callback tags/filters.** At load, read
|
||
`manifest.declared_tags` / `manifest.declared_filters` and register, in the
|
||
pongo2 environment used to render that plugin's powered blocks, one tag per
|
||
declared name that invokes `HOOK_RENDER_TAG` (packing the tag args +
|
||
current `RenderContext`) and one filter per declared name that invokes
|
||
`HOOK_APPLY_FILTER`. Surface a non-empty response `error` as a render error.
|
||
3. **Rely on the re-entrancy guarantee.** Because step 1 renders only after the
|
||
block-invoke returned, the callbacks in step 2 are fresh invokes on a free
|
||
instance — no special re-entrancy handling is needed.
|
||
|
||
> Migration note: existing blocks that call `blocks.RenderTemplate` inside the
|
||
> block (host-side .so world) switch to returning `blocks.PoweredBlock` in the
|
||
> wasm world. This SDK ships the mechanism only; per-plugin migration (e.g.
|
||
> assumechaos) is a later phase.
|
||
|
||
## Open items
|
||
|
||
WO-WZ-019 closed the original set: AI tool execution (`HOOK_AI_TOOL_CALL`),
|
||
job progress (`jobs.progress`), cross-plugin bridge invocation
|
||
(`bridge.invoke` + `HOOK_BRIDGE_CALL` + `plugin.BridgeInvokable`), directory
|
||
extension callbacks (`HOOK_DIRECTORY_PANEL_SECTION` /
|
||
`HOOK_DIRECTORY_PIN_DECORATOR`), plugin provisioning (the `provisioner.*`
|
||
family), content authoring (`content.*` writes), and the plugin-own settings
|
||
write. The full capability→mechanism map lives in
|
||
[abi-capability-surface.md](abi-capability-surface.md).
|
||
|
||
Still open:
|
||
|
||
- **Slot/media/embed resolvers in render**: container-slot rendering and
|
||
media/embed resolution during a guest render need host functions (or host-
|
||
side pre-rendering into `RenderContext`). The powered-block path sidesteps
|
||
this — host-side rendering has the real resolvers.
|