core/docs/wasm-abi.md
Alex Dunmow 081bacf2ab feat(wasmguest): wire bnwasm Pool into guest CoreServices (WO-WZ-004)
Bind the db.* driver over the same transport as the capability stubs so
deps.Pool keeps working for plugin sqlc code:

- dispatch.go: g.services.Pool = bnwasm.NewPool(capTransport) (nil on
  native/DESCRIBE → fails cleanly, matching the caps stubs).
- hostcalls.go (wasip1): also bind the "bnwasm" database/sql driver's
  process-global transport to CallHost.
- docs/wasm-abi.md: document the implemented guest driver + pgx-primary
  rationale under the Pool disposition.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 14:41:56 +08:00

18 KiB

Wasm Plugin ABI (v1)

The host↔guest wire contract for wazero-loaded BlockNinja plugins. Schema: abi/proto/v1/ (buf module abi/) — generated Go: git.dev.alexdunmow.com/block/core/abi/v1 (abiv1). Design rationale: the wasm plugin migration design spec in the cms repo (docs/superpowers/specs/2026-07-03-wasm-plugin-migration-design.md).

Regenerate with make abi (runs buf lint + buf generate in abi/). The abi/ buf module is deliberately separate from the repo-root buf config: proto/ is the shared block/proto git submodule (service API contracts), while this ABI is SDK-internal and versions in lockstep with the guest shim, so it lives repo-local.

Versioning — abi_version

PluginManifest.abi_version (field 1, manifest.proto) carries the ABI major version the plugin was built against. Current value: 1.

  • The host rejects any manifest whose major version it does not support — at install/publish time (manifest read) and again at DESCRIBE (DescribeRequest.host_abi_version tells the guest who is calling, so a newer guest shim can refuse an older host symmetrically).
  • Within a major version, evolution is protobuf-additive only: new fields, new Hook values, new capability methods. Removing or renaming anything wire-visible requires a major bump. buf breaking (FILE rules, configured in abi/buf.yaml) enforces this against the previous commit.

Module lifecycle — REACTOR mode (_initialize, no _start)

Plugins compile as WASI reactors (Go ≥ 1.24 toolchain for go:wasmexport; this repo's floor is higher — check go.mod):

GOOS=wasip1 GOARCH=wasm go build -buildmode=c-shared -o plugin.wasm .

with one boilerplate main file next to the untouched Registration:

//go:build wasip1

package main

import "git.dev.alexdunmow.com/block/core/plugin/wasmguest"

func init() { wasmguest.Serve(Registration) }
func main() {} // never called — reactor mode

A reactor module exports _initialize instead of _start. The host MUST run _initialize exactly once per instance before any bn_invoke (wazero: ModuleConfig.WithStartFunctions("_initialize")); it runs package init funcs — hence Serve, which stores the registration and returns. main exists only to satisfy the linker and is never called.

Command mode (plain go build) does NOT work and must not be used (empirically verified, Go 1.26 + wazero v1.12.0, 2026-07-03): _start runs main synchronously, so a blocking main (select{}) trips the Go deadlock detector and traps, while a returning main exits and closes the module — either way the exports are never callable. The original blocking-main design was amended to reactor mode for this reason.

Calling convention (ptr+len, packed u64)

wasm exports can only pass i32/i64/f32/f64, so all payloads cross as protobuf bytes in guest linear memory. The guest exports (via go:wasmexport):

Export Signature Purpose
bn_alloc (size: u32) → ptr: u32 Host asks the guest to allocate size bytes in guest memory. The returned region stays valid until the current bn_invoke call returns.
bn_invoke (hook_id: u32, ptr: u32, len: u32) → packed: u64 Host writes a serialized InvokeRequest at (ptr, len) (memory from bn_alloc) and calls with hook_id = Hook enum value (duplicated in the envelope for decode sanity). The return packs the response location: `packed = (ptr << 32)

Host functions (guest→host capability calls) live in wasm import module blockninja and use the same shape in reverse. There is exactly ONE generic import rather than one symbol per capability family:

(blockninja) host_call(ptr: u32, len: u32) → packed: u64

The guest passes (ptr, len) framing a serialized HostCallRequest (whose method string — "<family>.<snake_method>", including the db.* driver methods — already selects the family); the host returns a packed u64 framing a HostCallResponse that it wrote into guest memory via bn_alloc. The guest shim releases that buffer after decoding, so the host must not reuse it. Per-family import symbols were considered and rejected (decision, WO-WZ-002): they would add ~40 declarations on both sides for zero type safety, since the payloads are opaque protobuf bytes either way.

Buffers MUST come from bn_alloc on both paths — the guest rejects a bn_invoke request pointer it did not hand out (ABI_ERROR_CODE_DECODE), and treats an unknown host-call response pointer the same way.

A packed value of 0 means the callee could not even produce an envelope (allocation failure / trap); the caller treats it as ABI_ERROR_CODE_INTERNAL and discards the instance.

A logically-empty response is not packed 0. A successful hook whose response message has no set fields (e.g. LoadResponse/UnloadResponse) proto-marshals to zero bytes; the guest still frames it as (ptr, 0) with a real pointer so the host reads a valid empty envelope. bn_invoke never returns packed 0 for a successful call. (Fixed in WO-WZ-003: the earlier len==0 → return 0 shortcut made every successful empty-response hook — notably HOOK_LOAD — look like an INTERNAL failure and discard the instance. The cms host in WO-WZ-006 must likewise not conflate a zero-length payload with a missing envelope.)

Instances are single-threaded: one bn_invoke at a time per instance; concurrency comes from the per-plugin instance pool.

Hook catalog (invoke.proto)

Host→guest calls. InvokeRequest{hook, payload, deadline_ms}InvokeResponse{payload, error}; payload holds the hook-specific message:

Hook Request / Response Fires
HOOK_RENDER_BLOCK RenderBlockRequest / RenderBlockResponse Public render of one plugin block (blocks.BlockFunc).
HOOK_RENDER_TEMPLATE RenderTemplateRequest / RenderTemplateResponse Render of one plugin template (templates.TemplateFunc).
HOOK_HANDLE_HTTP HttpRequest / HttpResponse Buffered HTTP/ConnectRPC request forwarded to the guest's internal mux (only when manifest.has_http_handler). No streaming/SSE/WebSocket in v1.
HOOK_JOB JobRequest / JobResponse Background job dispatch for a manifest.job_types entry (plugin.JobHandlerFunc).
HOOK_LOAD LoadRequest / LoadResponse Plugin load (PluginRegistration.Load); LoadRequest.host_config delivers AppURL/MediaPath.
HOOK_UNLOAD UnloadRequest / UnloadResponse Plugin unload (PluginRegistration.Unload).
HOOK_RAG_FETCH RagFetchRequest / RagFetchResponse RAG re-index callback for a manifest.rag_content_fetcher_types entry (plugin.ContentFetcher).
HOOK_MEDIA_HOOK MediaHookRequest / MediaHookResponse Media lifecycle event (plugin.MediaHooksProvider), only when manifest.has_media_hooks.
HOOK_DESCRIBE DescribeRequest / DescribeResponse Publish-time manifest capture; the result is stored as manifest.pb in the .bnp artifact. Never called on a live instance.

Capability calls (capability.proto, db.proto)

Guest→host. Envelope: HostCallRequest{method, payload}HostCallResponse{payload, error}. method is "<family>.<snake_method>"; each pair mirrors one Go interface method from CoreServices 1:1 (UUIDs as canonical strings, map[string]any/JSON as bytes):

Family Methods Go surface
content get_author_profile, get_page, get_post, slugify, block_note_to_html, generate_excerpt, strip_html content.Content
settings get_site_settings, get_plugin_settings, update_site_setting settings.Settings, settings.Updater
gating get_subscriber_tier_level, evaluate_access gating.Gating
crypto encrypt_secret, decrypt_secret crypto.Crypto
menus get_menu_by_name, get_menu_items menus.Menus
datasources resolve_bucket, resolve_bucket_by_key datasources.Datasources
users get_by_username, get_by_id auth.PublicUsers
subscriptions get_user_tier_level, get_tier_by_slug, list_tiers, list_active_plans subscriptions.Subscriptions
media deposit plugin.Media
email send plugin.EmailSender
ai text_call, tools.register CoreServices.AITextCall, ai.ToolRegistry
bridge register_service, get_service plugin.PluginBridge
jobs submit plugin.JobRunner
embeddings generate_embedding, embed_content, is_available plugin.EmbeddingService
rag query, on_content_changed plugin.RAGService
reviews submit_review plugin.ReviewSubmitter
badges refresh_badges plugin.BadgeRefresher
db query, exec, tx_begin, tx_commit, tx_rollback CoreServices.Pool via the guest database/sql driver (db.proto)

The guest half of the SDK implements the existing Go interfaces as stubs marshaling to these calls (core/plugin/wasmguest/caps/, WO-WZ-003), so plugin code compiles unchanged. caps.NewCoreServices(call) assembles them; the wasm shim binds call to the real host_call transport, tests inject a fake, and a nil transport (DESCRIBE probes) fails every capability cleanly instead of nil-panicking.

Method disposition (every CoreServices member)

No silent gaps: each member is either a guest stub or served host-side.

Member Disposition
Content (7 methods) stubcaps/content.go
Settings / SettingsUpdater stubcaps/settings.go (one value, both fields)
Gating stubcaps/gating.go; EvaluateAccess crosses but falls back to the pure gating.EvaluateAccess on transport error
Crypto stubcaps/crypto.go
Menus stubcaps/menus.go
Datasources stubcaps/datasources.go
PublicUsers stubcaps/users.go
Subscriptions stubcaps/subscriptions.go
Media stubcaps/media.go
ToolRegistry + AITextCall stubcaps/ai.go (ai.tools.register + ai.text_call; tool Handler stays guest-side)
EmailSender stubcaps/email.go
Bridge stubcaps/bridge.go; RegisterService forwards names only (value dropped), GetService reports availability but returns nil (a typed value cannot cross — open item)
ReviewSubmitter stubcaps/reviews.go
BadgeRefresher stubcaps/badges.go
JobRunner stubcaps/jobs.go
EmbeddingService stubcaps/embeddings.go
RAGService stubcaps/rag.go; Query/OnContentChanged cross, RegisterContentFetcher records guest-side for HOOK_RAG_FETCH
Pool host-side — the db.* driver (db.proto), per-plugin Postgres role
Interceptors host-side — the host builds the connect option chain; RBAC merges from manifest.rbac_method_roles. Auth context reaches the guest via HttpRequest headers (host-side interceptors already ran).
AppURL / MediaPath host-side — delivered once in LoadRequest.host_config
CoreServiceBindings host-side — static manifest.core_service_bindings; the host constructs and mounts the http.Handler (cannot cross the sandbox), so caps provides no stub

Interface satisfaction is proven at compile time by a var _ <iface> = (*stub)(nil) line per family; a wasip1 build of testdata/fixture (whose Load hook calls deps.Content/deps.Settings/deps.Bridge unchanged) plus the TestWasmFixtureCapabilityRoundTrip end-to-end wazero test prove the path crosses the ABI for real.

Error mapping (caps/caps.go): a transport AbiError surfaces as a Go error wrapped with <family>.<method> context; an ABI_ERROR_CODE_DEADLINE_EXCEEDED reply is mapped onto context.DeadlineExceeded so errors.Is keeps working. Methods without an error channel (Slugify, IsAvailable, EvaluateAccess, ToolRegistry.Register, Bridge.*, RAG.OnContentChanged, …) degrade to the zero value / best-effort on transport failure.

CoreServices members that do not cross as capability calls:

  • Pool → the db.* driver messages (db.proto); the host executes under the per-plugin Postgres role. DbError.code carries the SQLSTATE. Transactions: tx_begin returns an opaque tx_handle (never 0); query/ exec with tx_handle = 0 run autocommit. Handles die with the call chain's deadline so a guest can never pin a connection. Guest side (WO-WZ-004): core/plugin/wasmguest/bnwasm implements this over the transport as two surfaces — a database/sql driver registered as "bnwasm", and a plugin.Pool handing out a pgx.Tx-shaped value. The latter is the primary path: current plugins' sqlc configs use sql_package: "pgx/v5", so their generated DBTX needs pgconn.CommandTag/pgx.Rows/pgx.Row (which database/sql cannot produce), and the Pool/Tx satisfy it with no source edits. DbError surfaces as *pgconn.PgError (SQLSTATE preserved for errors.As). Named args (pgx.NamedArgs/QueryRewriter) and nested transactions/savepoints are rejected with clear errors — no fleet plugin uses either. The DbValue↔Go scan mapping is pinned in the exported bnwasm.DbValueFixtures table, which the WO-WZ-007 host executor mirrors.
  • Interceptors (connect.Option) → host-side only; RBAC merges from manifest.rbac_method_roles.
  • AppURL / MediaPath → delivered once in LoadRequest.host_config.
  • CoreServiceBindings.Bind → static manifest.core_service_bindings declaration; the host constructs and mounts the handlers.
  • RAGService.RegisterContentFetcher → static manifest.rag_content_fetcher_types declaration + HOOK_RAG_FETCH callback inversion.

Error semantics

AbiError{code, message} travels in InvokeResponse.error and HostCallResponse.error:

Code Meaning Instance consequence
ABI_ERROR_CODE_INTERNAL Handler ran and failed; message is the Go error text. None (normal error). Guest traps / packed 0 returns are treated as INTERNAL by the host and do discard the instance.
ABI_ERROR_CODE_DECODE Envelope or payload failed to decode. Instance discarded (protocol desync).
ABI_ERROR_CODE_UNIMPLEMENTED Callee does not implement the hook/capability (e.g. host too old for a new capability method). None.
ABI_ERROR_CODE_DEADLINE_EXCEEDED deadline_ms elapsed. Instance considered poisoned, discarded.
ABI_ERROR_CODE_PERMISSION_DENIED Caller not entitled to the capability. None.

Host-function errors surface to plugin code as ordinary Go errors via the guest SDK. Repeated instance failures trip the existing PluginStatusFailed path + admin notification. DB failures use DbError (SQLSTATE-carrying) inside the db.* responses instead of AbiError, so sqlc/pgx error handling keeps working.

Manifest ↔ PluginRegistration mapping

manifest.pb (a serialized PluginManifest) is produced at publish time via HOOK_DESCRIBE and read by the loader without instantiating the module. Field-by-field:

PluginRegistration field Wire counterpart
Name PluginManifest.name
Version PluginManifest.version
Dependencies dependencies (Dependency)
Register Static effects captured by DESCRIBE: blocks (BlockMeta), block_template_overrides, template_keys, system_templates, page_templates, email_wrapper_system_keys
RegisterWithProvisioner Same captures + has_provisioner (provisioning runs at load, host-side)
Assets .bnp artifact assets/ directory (host serves directly; never crosses the boundary)
Schemas .bnp artifact schemas/ directory (host loads into the block registry)
SettingsSchema settings_schema (JSON bytes)
ThemePresets theme_presets (JSON bytes)
BundledFonts bundled_fonts (JSON bytes)
MasterPages master_pages (MasterPageDefinition/MasterPageBlock)
HTTPHandler has_http_handler + HOOK_HANDLE_HTTP
SettingsPanel settings_panel
AdminPages admin_pages (AdminPage)
CSSManifest css_manifest (CssManifest)
ServiceHandlers rbac_method_roles (method → role; the services themselves answer via HOOK_HANDLE_HTTP) + core_service_bindings
JobHandlers job_types + HOOK_JOB
AIActions ai_actions (AiAction)
DirectoryExtensions directory_extensions (static fields + callback counts)
MediaHooks has_media_hooks + HOOK_MEDIA_HOOK
Load has_load_hook + HOOK_LOAD
Unload has_unload_hook + HOOK_UNLOAD
Migrations .bnp artifact migrations/ directory (Goose runs host-side; never crosses)
RequiredIconPacks required_icon_packs

Render context

RenderContext (render.proto) is the explicit envelope of every value blocks read from ctx today (core/blocks/context.go): request info, BlockContext (the pongo2 data struct, 1:1), current page/post/author/ category/master-page, requested path, injected/expected slots, editor flag, block + page IDs, human-proof banner, detail row, theme variables JSON, and locale.

Function-valued context entries cannot serialize; their v1 mapping:

  • GetQueries → the db.* driver (plugin sqlc code, per-plugin role).
  • SlotRenderer / MediaResolver / EmbedResolveropen items (below).

Open items (flagged for runtime WOs — additive, no major bump needed)

  • AI tool execution: AiToolRegisterRequest registers a tool; executing its guest-side Handler needs a host→guest hook (e.g. HOOK_AI_TOOL_CALL). Deliberately not added here — the WO fixes the v1 hook catalog.
  • Job progress: plugin.JobHandlerFunc's progress(current, total, message) callback needs a jobs.progress host function.
  • Bridge calls: bridge.register_service/get_service cover registration/lookup only; typed cross-plugin invocation (today: shared in-process Go values) needs a runtime design.
  • Directory extension callbacks: panel_section_count / pin_decorator_count declare the guest callbacks; invoking them needs a hook.
  • Slot/media/embed resolvers in render: container-slot rendering and media/embed resolution during a guest render need host functions (or host- side pre-rendering into RenderContext).