core/docs/wasm-abi.md
Alex Dunmow c35199f0bc feat(wasmguest): guest runtime shim for the wasm plugin ABI (WO-WZ-002)
The wasip1 half of the ABI: bn_alloc/bn_invoke/bn_free exports with a
live-pin map so the GC never frees host-visible buffers, the generic
`blockninja.host_call` import (single import decided over per-family
symbols; recorded in wasm-abi.md), and a dispatch table adapting an
unmodified plugin.PluginRegistration to all nine v1 hooks. Panics inside
plugin hooks come back as ABI_ERROR_CODE_INTERNAL — the instance stays
callable; traps stay reserved for runtime corruption.

DESCRIBE builds the PluginManifest from the registration's static funcs
plus a capture-only Register pass (block metas via the same
PluginBlockRegistry prefixing the .so loader applies, template/system/
page-template/email-wrapper keys), probes JobHandlers/ServiceHandlers/
Load with capture-only services for job types, RBAC roles, core-service
bindings, and RAG fetcher types. RenderContext values are rehydrated
through the exact core/blocks context keys, so existing block code
reading from ctx works unchanged.

Plugins build in REACTOR mode (go build -buildmode=c-shared): init()
calls wasmguest.Serve (non-blocking), main is never called, and the host
runs _initialize before any bn_invoke. Command mode deadlocks or exits
(verified against wazero v1.12.0) — documented prominently in
wasm-abi.md, which also now reconciles the import module namespace to
`blockninja` and requires bn_alloc'd buffers on both directions.

Dispatch/describe/context logic is buildable on every GOOS; only
exports.go and hostcalls.go carry the wasip1 tag. dispatch_test.go
covers describe, hook routing, envelope mismatch, decode failures,
template-override resolution, panic recovery, and lifecycle hooks
natively.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 13:53:24 +08:00

249 lines
14 KiB
Markdown

# Wasm Plugin ABI (v1)
The host↔guest wire contract for wazero-loaded BlockNinja plugins.
Schema: [`abi/proto/v1/`](../abi/proto/v1/) (buf module [`abi/`](../abi/)) —
generated Go: `git.dev.alexdunmow.com/block/core/abi/v1` (`abiv1`).
Design rationale: the wasm plugin migration design spec in the cms repo
(`docs/superpowers/specs/2026-07-03-wasm-plugin-migration-design.md`).
Regenerate with `make abi` (runs `buf lint` + `buf generate` in `abi/`).
The `abi/` buf module is deliberately separate from the repo-root buf config:
`proto/` is the shared block/proto git submodule (service API contracts),
while this ABI is SDK-internal and versions in lockstep with the guest shim,
so it lives repo-local.
## Versioning — `abi_version`
`PluginManifest.abi_version` (field 1, `manifest.proto`) carries the ABI
**major** version the plugin was built against. Current value: **1**.
- The host **rejects** any manifest whose major version it does not support —
at install/publish time (manifest read) and again at `DESCRIBE`
(`DescribeRequest.host_abi_version` tells the guest who is calling, so a
newer guest shim can refuse an older host symmetrically).
- Within a major version, evolution is protobuf-additive only: new fields,
new `Hook` values, new capability methods. Removing or renaming anything
wire-visible requires a major bump. `buf breaking` (FILE rules, configured
in `abi/buf.yaml`) enforces this against the previous commit.
## Module lifecycle — REACTOR mode (`_initialize`, no `_start`)
Plugins compile as WASI **reactors** (Go ≥ 1.24 toolchain for
`go:wasmexport`; this repo's floor is higher — check `go.mod`):
```
GOOS=wasip1 GOARCH=wasm go build -buildmode=c-shared -o plugin.wasm .
```
with one boilerplate main file next to the untouched Registration:
```go
//go:build wasip1
package main
import "git.dev.alexdunmow.com/block/core/plugin/wasmguest"
func init() { wasmguest.Serve(Registration) }
func main() {} // never called — reactor mode
```
A reactor module exports `_initialize` instead of `_start`. The host MUST
run `_initialize` exactly once per instance **before any `bn_invoke`**
(wazero: `ModuleConfig.WithStartFunctions("_initialize")`); it runs package
init funcs — hence `Serve`, which stores the registration and returns.
`main` exists only to satisfy the linker and is never called.
> Command mode (plain `go build`) does NOT work and must not be used
> (empirically verified, Go 1.26 + wazero v1.12.0, 2026-07-03): `_start`
> runs `main` synchronously, so a blocking `main` (`select{}`) trips the Go
> deadlock detector and traps, while a returning `main` exits and closes
> the module — either way the exports are never callable. The original
> blocking-main design was amended to reactor mode for this reason.
## Calling convention (ptr+len, packed u64)
wasm exports can only pass `i32/i64/f32/f64`, so all payloads cross as
protobuf bytes in guest linear memory. The guest exports (via
`go:wasmexport`):
| Export | Signature | Purpose |
|---|---|---|
| `bn_alloc` | `(size: u32) → ptr: u32` | Host asks the guest to allocate `size` bytes in guest memory. The returned region stays valid until the current `bn_invoke` call returns. |
| `bn_invoke` | `(hook_id: u32, ptr: u32, len: u32) → packed: u64` | Host writes a serialized `InvokeRequest` at `(ptr, len)` (memory from `bn_alloc`) and calls with `hook_id = Hook` enum value (duplicated in the envelope for decode sanity). The return packs the response location: `packed = (ptr << 32) | len`, framing a serialized `InvokeResponse` in guest memory, valid until the next `bn_invoke` on this instance. |
Host functions (guest→host capability calls) live in wasm import module
**`blockninja`** and use the same shape in reverse. There is exactly ONE
generic import rather than one symbol per capability family:
```
(blockninja) host_call(ptr: u32, len: u32) → packed: u64
```
The guest passes `(ptr, len)` framing a serialized `HostCallRequest` (whose
`method` string — `"<family>.<snake_method>"`, including the `db.*` driver
methods — already selects the family); the host returns a packed `u64`
framing a `HostCallResponse` that it wrote into guest memory via `bn_alloc`.
The guest shim releases that buffer after decoding, so the host must not
reuse it. Per-family import symbols were considered and rejected (decision,
WO-WZ-002): they would add ~40 declarations on both sides for zero type
safety, since the payloads are opaque protobuf bytes either way.
Buffers MUST come from `bn_alloc` on both paths — the guest rejects a
`bn_invoke` request pointer it did not hand out (`ABI_ERROR_CODE_DECODE`),
and treats an unknown host-call response pointer the same way.
A `packed` value of `0` means the callee could not even produce an envelope
(allocation failure / trap); the caller treats it as
`ABI_ERROR_CODE_INTERNAL` and discards the instance.
Instances are single-threaded: one `bn_invoke` at a time per instance;
concurrency comes from the per-plugin instance pool.
## Hook catalog (`invoke.proto`)
Host→guest calls. `InvokeRequest{hook, payload, deadline_ms}`
`InvokeResponse{payload, error}`; `payload` holds the hook-specific message:
| Hook | Request / Response | Fires |
|---|---|---|
| `HOOK_RENDER_BLOCK` | `RenderBlockRequest` / `RenderBlockResponse` | Public render of one plugin block (`blocks.BlockFunc`). |
| `HOOK_RENDER_TEMPLATE` | `RenderTemplateRequest` / `RenderTemplateResponse` | Render of one plugin template (`templates.TemplateFunc`). |
| `HOOK_HANDLE_HTTP` | `HttpRequest` / `HttpResponse` | Buffered HTTP/ConnectRPC request forwarded to the guest's internal mux (only when `manifest.has_http_handler`). No streaming/SSE/WebSocket in v1. |
| `HOOK_JOB` | `JobRequest` / `JobResponse` | Background job dispatch for a `manifest.job_types` entry (`plugin.JobHandlerFunc`). |
| `HOOK_LOAD` | `LoadRequest` / `LoadResponse` | Plugin load (`PluginRegistration.Load`); `LoadRequest.host_config` delivers `AppURL`/`MediaPath`. |
| `HOOK_UNLOAD` | `UnloadRequest` / `UnloadResponse` | Plugin unload (`PluginRegistration.Unload`). |
| `HOOK_RAG_FETCH` | `RagFetchRequest` / `RagFetchResponse` | RAG re-index callback for a `manifest.rag_content_fetcher_types` entry (`plugin.ContentFetcher`). |
| `HOOK_MEDIA_HOOK` | `MediaHookRequest` / `MediaHookResponse` | Media lifecycle event (`plugin.MediaHooksProvider`), only when `manifest.has_media_hooks`. |
| `HOOK_DESCRIBE` | `DescribeRequest` / `DescribeResponse` | Publish-time manifest capture; the result is stored as `manifest.pb` in the `.bnp` artifact. Never called on a live instance. |
## Capability calls (`capability.proto`, `db.proto`)
Guest→host. Envelope: `HostCallRequest{method, payload}`
`HostCallResponse{payload, error}`. `method` is `"<family>.<snake_method>"`;
each pair mirrors one Go interface method from `CoreServices` 1:1
(UUIDs as canonical strings, `map[string]any`/JSON as bytes):
| Family | Methods | Go surface |
|---|---|---|
| `content` | `get_author_profile`, `get_page`, `get_post`, `slugify`, `block_note_to_html`, `generate_excerpt`, `strip_html` | `content.Content` |
| `settings` | `get_site_settings`, `get_plugin_settings`, `update_site_setting` | `settings.Settings`, `settings.Updater` |
| `gating` | `get_subscriber_tier_level`, `evaluate_access` | `gating.Gating` |
| `crypto` | `encrypt_secret`, `decrypt_secret` | `crypto.Crypto` |
| `menus` | `get_menu_by_name`, `get_menu_items` | `menus.Menus` |
| `datasources` | `resolve_bucket`, `resolve_bucket_by_key` | `datasources.Datasources` |
| `users` | `get_by_username`, `get_by_id` | `auth.PublicUsers` |
| `subscriptions` | `get_user_tier_level`, `get_tier_by_slug`, `list_tiers`, `list_active_plans` | `subscriptions.Subscriptions` |
| `media` | `deposit` | `plugin.Media` |
| `email` | `send` | `plugin.EmailSender` |
| `ai` | `text_call`, `tools.register` | `CoreServices.AITextCall`, `ai.ToolRegistry` |
| `bridge` | `register_service`, `get_service` | `plugin.PluginBridge` |
| `jobs` | `submit` | `plugin.JobRunner` |
| `embeddings` | `generate_embedding`, `embed_content`, `is_available` | `plugin.EmbeddingService` |
| `rag` | `query`, `on_content_changed` | `plugin.RAGService` |
| `reviews` | `submit_review` | `plugin.ReviewSubmitter` |
| `badges` | `refresh_badges` | `plugin.BadgeRefresher` |
| `db` | `query`, `exec`, `tx_begin`, `tx_commit`, `tx_rollback` | `CoreServices.Pool` via the guest `database/sql` driver (`db.proto`) |
The guest half of the SDK implements the existing Go interfaces as stubs
marshaling to these calls, so plugin code compiles unchanged.
`CoreServices` members that do **not** cross as capability calls:
- `Pool` → the `db.*` driver messages (`db.proto`); the host executes under
the per-plugin Postgres role. `DbError.code` carries the SQLSTATE.
Transactions: `tx_begin` returns an opaque `tx_handle` (never 0); `query`/
`exec` with `tx_handle = 0` run autocommit. Handles die with the call
chain's deadline so a guest can never pin a connection.
- `Interceptors` (`connect.Option`) → host-side only; RBAC merges from
`manifest.rbac_method_roles`.
- `AppURL` / `MediaPath` → delivered once in `LoadRequest.host_config`.
- `CoreServiceBindings.Bind` → static `manifest.core_service_bindings`
declaration; the host constructs and mounts the handlers.
- `RAGService.RegisterContentFetcher` → static
`manifest.rag_content_fetcher_types` declaration + `HOOK_RAG_FETCH`
callback inversion.
## Error semantics
`AbiError{code, message}` travels in `InvokeResponse.error` and
`HostCallResponse.error`:
| Code | Meaning | Instance consequence |
|---|---|---|
| `ABI_ERROR_CODE_INTERNAL` | Handler ran and failed; `message` is the Go error text. | None (normal error). Guest traps / packed `0` returns are *treated as* INTERNAL by the host and **do** discard the instance. |
| `ABI_ERROR_CODE_DECODE` | Envelope or payload failed to decode. | Instance discarded (protocol desync). |
| `ABI_ERROR_CODE_UNIMPLEMENTED` | Callee does not implement the hook/capability (e.g. host too old for a new capability method). | None. |
| `ABI_ERROR_CODE_DEADLINE_EXCEEDED` | `deadline_ms` elapsed. | Instance considered poisoned, discarded. |
| `ABI_ERROR_CODE_PERMISSION_DENIED` | Caller not entitled to the capability. | None. |
Host-function errors surface to plugin code as ordinary Go errors via the
guest SDK. Repeated instance failures trip the existing
`PluginStatusFailed` path + admin notification. DB failures use `DbError`
(SQLSTATE-carrying) inside the `db.*` responses instead of `AbiError`, so
sqlc/pgx error handling keeps working.
## Manifest ↔ `PluginRegistration` mapping
`manifest.pb` (a serialized `PluginManifest`) is produced at publish time via
`HOOK_DESCRIBE` and read by the loader without instantiating the module.
Field-by-field:
| `PluginRegistration` field | Wire counterpart |
|---|---|
| `Name` | `PluginManifest.name` |
| `Version` | `PluginManifest.version` |
| `Dependencies` | `dependencies` (`Dependency`) |
| `Register` | Static effects captured by DESCRIBE: `blocks` (`BlockMeta`), `block_template_overrides`, `template_keys`, `system_templates`, `page_templates`, `email_wrapper_system_keys` |
| `RegisterWithProvisioner` | Same captures + `has_provisioner` (provisioning runs at load, host-side) |
| `Assets` | `.bnp` artifact `assets/` directory (host serves directly; never crosses the boundary) |
| `Schemas` | `.bnp` artifact `schemas/` directory (host loads into the block registry) |
| `SettingsSchema` | `settings_schema` (JSON bytes) |
| `ThemePresets` | `theme_presets` (JSON bytes) |
| `BundledFonts` | `bundled_fonts` (JSON bytes) |
| `MasterPages` | `master_pages` (`MasterPageDefinition`/`MasterPageBlock`) |
| `HTTPHandler` | `has_http_handler` + `HOOK_HANDLE_HTTP` |
| `SettingsPanel` | `settings_panel` |
| `AdminPages` | `admin_pages` (`AdminPage`) |
| `CSSManifest` | `css_manifest` (`CssManifest`) |
| `ServiceHandlers` | `rbac_method_roles` (method → role; the services themselves answer via `HOOK_HANDLE_HTTP`) + `core_service_bindings` |
| `JobHandlers` | `job_types` + `HOOK_JOB` |
| `AIActions` | `ai_actions` (`AiAction`) |
| `DirectoryExtensions` | `directory_extensions` (static fields + callback counts) |
| `MediaHooks` | `has_media_hooks` + `HOOK_MEDIA_HOOK` |
| `Load` | `has_load_hook` + `HOOK_LOAD` |
| `Unload` | `has_unload_hook` + `HOOK_UNLOAD` |
| `Migrations` | `.bnp` artifact `migrations/` directory (Goose runs host-side; never crosses) |
| `RequiredIconPacks` | `required_icon_packs` |
## Render context
`RenderContext` (`render.proto`) is the explicit envelope of every value
blocks read from `ctx` today (`core/blocks/context.go`): request info,
`BlockContext` (the pongo2 data struct, 1:1), current page/post/author/
category/master-page, requested path, injected/expected slots, editor flag,
block + page IDs, human-proof banner, detail row, theme variables JSON, and
locale.
Function-valued context entries cannot serialize; their v1 mapping:
- `GetQueries` → the `db.*` driver (plugin sqlc code, per-plugin role).
- `SlotRenderer` / `MediaResolver` / `EmbedResolver`**open items** (below).
## Open items (flagged for runtime WOs — additive, no major bump needed)
- **AI tool execution**: `AiToolRegisterRequest` registers a tool; executing
its guest-side `Handler` needs a host→guest hook (e.g. `HOOK_AI_TOOL_CALL`).
Deliberately not added here — the WO fixes the v1 hook catalog.
- **Job progress**: `plugin.JobHandlerFunc`'s `progress(current, total,
message)` callback needs a `jobs.progress` host function.
- **Bridge calls**: `bridge.register_service`/`get_service` cover
registration/lookup only; typed cross-plugin *invocation* (today: shared
in-process Go values) needs a runtime design.
- **Directory extension callbacks**: `panel_section_count` /
`pin_decorator_count` declare the guest callbacks; invoking them needs a
hook.
- **Slot/media/embed resolvers in render**: container-slot rendering and
media/embed resolution during a guest render need host functions (or host-
side pre-rendering into `RenderContext`).