alex/skills
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

developing-blockninja-plugins: rewrite for the wasm-plugin era

Browse Source 
.so pipeline is dead (cms a04277ee2): standalone plugins are reactor-mode
wasip1 wasm packed into .bnp artifacts. Skill now teaches ninja plugin
build/verify/publish --bnp, bnwasm sqlc DB (per-plugin role, own schema
only), capability dispositions, per-instance-state gotchas (HostServices,
deps stashing), sandbox constraints (data_dir, buffered HTTP, hot-swap),
with plugins/testplugin as the worked example. html-blocks.md: reword the
one .so compat line.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Alex Dunmow 2026-07-03 22:09:55 +08:00
parent b23e0219b7
commit cb1acbab6b
2 changed files with 207 additions and 38 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

243
developing-blockninja-plugins/SKILL.md
View File

@ -7,31 +7,69 @@ description: Use when creating, modifying, building, or publishing BlockNinja CM
## Overview ## Overview
> **⚠️ If you remember `.so` instructions, they are obsolete.** Standalone plugins are now
> **wasm** (wazero, WASI reactor mode) — the big-bang migration landed 2026-07-03 and the
> `.so` loader/builder were deleted from cms (`so_loader.go`, `internal/builder/`, cms
> commit a04277ee2). There is NO `make build-so`, NO `plugin.Open`, NO CGO, NO in-container
> compile, NO host/plugin Go-version lock-step, NO `copy-plugin-source` deploy, and NO
> go:embed'd web bundle. Instances **download** a prebuilt `.bnp` artifact — they never
> compile. A registry version published as a source archive is **rejected** at install
> ("legacy source archive; republish as .bnp").
Never write plugin code from memory — every SDK symbol is locally verifiable. Truth lives at: Never write plugin code from memory — every SDK symbol is locally verifiable. Truth lives at:
- **SDK source:** `~/src/blockninja/core` — import prefix `git.dev.alexdunmow.com/block/core/...` is the ONLY one allowed; never `block/cms/...` - **SDK source:** `~/src/blockninja/core` — import prefix `git.dev.alexdunmow.com/block/core/...` is the ONLY one allowed; never `block/cms/...`
- **Canonical guide:** `~/src/blockninja/cms/docs/PLUGIN_DEVELOPMENT.md` - **ABI contract (authoritative):** `~/src/blockninja/core/docs/wasm-abi.md` — reactor mode, hook catalog, capability dispositions, error codes, `plugin.mod` `data_dir`
- **Publish workflow + hard rules:** `~/src/blockninja/plugins/CLAUDE.md` - **Architecture spec:** `~/src/blockninja/cms/docs/superpowers/specs/2026-07-03-wasm-plugin-migration-design.md`
- **Exemplars:** `plugins/messenger` (compact), `plugins/symposium` (service-heavy: RPC, jobs, AI, embeddings) - **Canonical guide (registration concepts, blocks, provisioning):** `~/src/blockninja/cms/docs/PLUGIN_DEVELOPMENT.md`
- **Exemplars:** `plugins/testplugin` — THE reference wasm plugin, every surface + failure endpoints (see its README surface table); `plugins/symposium` — the largest real port (service-heavy: RPC, jobs, AI, templ blocks; its Makefile is the build/publish convention)
Core-vs-plugin: platform-wide behavior → core; domain-specific or owns its own data/UI → plugin (decision table in PLUGIN_DEVELOPMENT.md). Core-vs-plugin: platform-wide behavior → core; domain-specific or owns its own data/UI → plugin (decision table in PLUGIN_DEVELOPMENT.md).
## Doc routing ## Anatomy of a wasm plugin
| Working on | Read first (under `cms/docs/`) | `PluginRegistration` is **unchanged** from the .so era — same fields, same `Register`,
same `CoreServices`. What changed is how it's served. Boilerplate `main.go`
(testplugin/main.go verbatim):
```go
//go:build wasip1
package main
import "git.dev.alexdunmow.com/block/core/plugin/wasmguest"
func init() { wasmguest.Serve(Registration) }
func main() {} // never called — reactor mode
```
Compile: `GOOS=wasip1 GOARCH=wasm go build -buildmode=c-shared -o plugin.wasm .`
(Go ≥ 1.24; reactor mode — plain `go build` command mode does NOT work, the exports are
never callable). You never run this by hand: `ninja plugin build` does it.
At publish time a DESCRIBE probe instantiates the module once and captures every static
registration surface into `manifest.pb`; the CMS loader reads that without instantiating
the module. Dynamic work (block render, HTTP, jobs, Load/Unload, RAG fetch, media hooks)
crosses the ABI as protobuf hooks into a **pool of instances** (default 4, 512 MiB memory
cap each, 30 s per-call deadline). Full hook catalog + manifest↔Registration field map:
`wasm-abi.md`.
**DESCRIBE constraint:** host functions are stubbed to fail during the probe, so
`Register` (and anything reached from it) must not call capabilities or the DB — it gets a
named error at build time. Load-time work belongs in `Load`.
## Worked example — plugins/testplugin
| Surface | File |
|---|---| |---|---|
| Scaffold, registration, custom block types, `CoreServices` | PLUGIN_DEVELOPMENT.md | | Blocks (`rendercontext`, `content`) + template (`testplugin_page`) | `registration.go` |
| **Section/content blocks — html block vs custom; menus; live vs pre-render** | **[html-blocks.md](html-blocks.md)** (in this skill) — read BEFORE writing a custom block type | | Connect service via `plugin.NewConnectServiceBinding` (RBAC roles in manifest) | `service.go` |
| Depositing media into the library (seed `EnsureMedia` / runtime `CoreServices.Media`) | PLUGIN_DEVELOPMENT.md §Depositing Media | | Raw chi routes + failure endpoints (`/panic`, `/slow`, `/forbidden-sql`, `/alloc`) | `httphandler.go` |
| Plugin migrations / DB tables — each plugin owns a Postgres schema (named after it), never `public` | PLUGIN_DEVELOPMENT.md §Migrations | | Job handler, media hooks, Load/Unload, RAG fetcher, deps stashing | `registration.go` |
| **Platform data tables** — provision via `Provisioner.EnsureDataTable` (`RegisterWithProvisioner`); the row-as-JSONB store the admin Data Platform / buckets / **data Views** read, idempotent + **View-rootable**. NOT the plugin's own Postgres schema above | PLUGIN_DEVELOPMENT.md §Provisioning → Data tables | | Goose migration + sqlc (pgx/v5) queries | `migrations/`, `sql/queries/`, `db/` |
| Public HTTP routes (webhooks, widgets) | PLUGIN_HTTP_HANDLERS.md | | Settings schema, theme preset (go:embed'd JSON) | `assets/` |
| Load/Unload, runtime state, goroutines | PLUGIN_LIFECYCLE_HOOKS.md | | `data_dir = true` grant + `/data` round-trip | `plugin.mod`, `onLoad` |
| Themes, templates, master pages, CSS | TEMPLATE_PLUGINS.md |
| Theme preview screenshots (showcase preset, gallery instances, `ninja theme screenshot`, registry `previewImageUrl`) | theme-previews.md |
| Block editor / settings UI (Module Federation) | PLUGIN_EDITOR_SDK.md |
| .so build pipeline, loader internals | compiled-plugin-architecture.md |
| Release, registry, install | `plugins/CLAUDE.md` (not cms/docs) |
## Blocks: prefer the built-in `html` block ## Blocks: prefer the built-in `html` block
@ -44,35 +82,166 @@ custom block only when you need a structured field-form editor or render logic a
express. Custom keys also risk red `block-fallback` boxes when a key isn't registered; built-in express. Custom keys also risk red `block-fallback` boxes when a key isn't registered; built-in
`html` never does. Decision guide, render modes, menu wiring, and gotchas: **[html-blocks.md](html-blocks.md)**. `html` never does. Decision guide, render modes, menu wiring, and gotchas: **[html-blocks.md](html-blocks.md)**.
## Verifying SDK symbols Custom blocks that hit the DB have a wasm-specific gotcha — see "Per-instance state" below.
Before using an unfamiliar SDK call, check: (1) the pinned SDK the build compiles against — `go doc git.dev.alexdunmow.com/block/core/plugin CoreServices` from the plugin dir; (2) nearest exemplar usage in messenger/symposium; (3) the CMS-side implementation in `cms/backend` for semantics. ## Database — sqlc unchanged, sandboxed role
**Missing capability** ⇒ two sanctioned paths only: extend `block/core` (long-term; needs SDK release + re-pin + image rebuild), or vendor the cms-internal package into the plugin's `internal/` with a provenance header (established convention — check-safety vendors cms `internal/theme` this way). NEVER `replace` directives; NEVER `block/cms` imports. sqlc-generated code (`sql_package: "pgx/v5"`) works **without source edits**: at runtime
`deps.Pool` is a `bnwasm.Pool` (pgx-shaped `Pool`/`Tx` over the ABI's `db.*` messages).
Autocommit: type-assert the pool to your `db.DBTX` (`registration.go` `openQueries`).
Transactions: `deps.Pool.Begin(ctx)` → `db.New(tx)` → `Commit/Rollback` (`httphandler.go`
`/db-tx`). Errors surface as `*pgconn.PgError` with real SQLSTATEs.
**Version pinning:** `go.mod` pins `block/core` to exactly what the CMS uses: Hard rules and limits:
`grep 'block/core ' ~/src/blockninja/cms/backend/go.mod`
## Gates — run before commit / bump / publish - **Own schema ONLY.** Queries run under a per-plugin Postgres role (`plugin_<name>`)
granted only on schema `<name>`. `public.*` is **permission denied (SQLSTATE 42501) by
design** — core data goes through `CoreServices` capability interfaces, never SQL.
- **No raw SQL** — sqlc for static, Bob for dynamic (workspace rule, applies to plugins).
- Savepoints/nested transactions and named args (`pgx.NamedArgs`) are **rejected** with
clear errors.
- `text[]` values cannot carry NULL **elements** (collapse to `""`); the whole array can
still be NULL.
- A tx handle dies with the call chain's deadline: `errors.Is(err, bnwasm.ErrTxExpired)`
is retryable — re-run the unit of work in a fresh transaction.
- Migrations are plain Goose SQL in `migrations/` — run by the **host** (never cross the
boundary). `db/*.sql.go` is committed; regenerate with `sqlc generate`.
## Capabilities — CoreServices works as before, know the dispositions
Plugin code compiles against the same Go interfaces (`deps.Content`, `deps.Settings`,
`deps.Crypto`, `deps.Menus`, `deps.RAGService`, …) — the guest SDK ships them as stubs
marshaling to host functions. Full family/method table + disposition of every
`CoreServices` member: **`wasm-abi.md`**. The ones that behave differently:
- `Interceptors`, `CoreServiceBindings` — **host-side**; RBAC method roles merge from the
manifest, auth context arrives via forwarded HTTP headers (host interceptors already ran).
- `AppURL`/`MediaPath` — delivered once in `LoadRequest.host_config`.
- `Bridge.GetService` reports availability but returns `nil` (typed values can't cross —
open item); `RegisterService` forwards names only.
- `RAGService.RegisterContentFetcher` — records guest-side + a manifest declaration; the
host calls back via `HOOK_RAG_FETCH`.
- Methods without an error channel (`Slugify`, `EvaluateAccess`, `Bridge.*`, …) degrade to
zero values on transport failure.
### Per-instance state (the #1 wasm gotcha)
Concurrency = an instance pool; each instance is a separate Go runtime with its own
globals. `Load` fires **once, on one instance** — globals it sets do NOT exist on the
others. Two sanctioned patterns (both in testplugin/symposium):
1. **Stash deps in every deps-receiving entry point** (`Load`, `HTTPHandler`,
`JobHandlers`) so deps-less hooks (media hooks, `Unload`) can reach the DB — testplugin's
`pkgDeps`/`stashDeps`. Instances are single-threaded, so a plain global is safe.
2. **DB-backed blocks:** `HOOK_RENDER_BLOCK` receives no services. Get the pool inside
`Register` (which runs on every instance) via `wasmguest.HostServices().Pool` —
**nil-check it**: in native/DESCRIBE builds it's the zero value. See symposium
`register.go`.
## Sandbox constraints — write reload-safe code
- **No filesystem, no network, no env** in the guest. One opt-in exception: `data_dir = true`
in `plugin.mod` grants a read-write preopen at guest `/data` — a per-plugin directory on
a persistent volume (`instance-<slug>-plugin-data`), shared across the plugin's pool
instances, deleted on uninstall. Off by default; the grant is visible at publish time.
Absent at DESCRIBE time — tolerate the write failure (testplugin `onLoad`).
- **Buffered HTTP only** — no SSE/WebSocket/streaming from plugin endpoints. Realtime
features use the core realtime system (`@block-ninja/ui` hooks / core topics).
- **Panics recover to errors**: an in-hook panic → INTERNAL error → 5xx; the host and other
plugins survive. Deadline overruns and OOM discard the instance; the pool refills.
Error-code table (INTERNAL/DECODE/UNIMPLEMENTED/DEADLINE/PERMISSION/TX_EXPIRED):
`wasm-abi.md`.
- **Hot swap**: plugin updates swap live (`SwapPlugin`) with no restart — never assume an
instance lives long, or that in-memory state survives an update. Persist in the DB or
`/data`.
- No `os.Getenv` for secrets (nothing is there anyway) — plugin settings or `CoreServices`.
## Build → verify → publish
`plugin.mod` is still the manifest (name/version/kind/scope/categories/tags, plus the new
first-class `data_dir` bool). `kind` is still **frozen at first publish**. Pin a recent
`block/core`; exact-version lock-step with the CMS is dead (testplugin builds on v0.15.1
while cms runs v0.15.2) — compatibility is gated by ABI major (`abi_version`, currently 1).
```bash ```bash
make # CGO build; templ/sqlc drift surfaces here ninja plugin build --dir . # wasip1 compile + DESCRIBE probe → <name>-<version>.bnp
cd ~/src/blockninja/check-safety && go run . <plugin-path> # MUST exit 0 ninja plugin verify <name>-<ver>.bnp # loader-identical checks: layout, abi_version, name match
make archive-check # proves `git archive HEAD` (= what publish ships) compiles cd ~/src/blockninja/check-safety && go run . <plugin-path> # MUST exit 0 (unchanged mandate)
ninja plugin bump patch # commits plugin.mod — does NOT git-tag
git tag vX.Y.Z && git push origin main vX.Y.Z # --follow-tags skips lightweight tags
ninja plugin build --dir . # rebuild at the bumped version
ninja plugin publish --bnp <name>-<ver>.bnp # --bnp is REQUIRED: without it publish
# ships a source archive, which instances REJECT
``` ```
check-safety traps: plugin `web/` lint extends `../../../cms/web/eslint.config.js`, so it only runs from the canonical sibling layout; raw `<button>` in plugin UI fails (use `@block-ninja/ui` Button); `any` usage warns. The `.bnp` (tar.zst) packs `plugin.wasm`, `plugin.mod`, `manifest.pb`, plus `migrations/`,
`schemas/`, `assets/`, and `web/dist` (flattened under `web/`) when present. Statics are
consumed host-side: Goose runs migrations, assets are served directly, and the Module
Federation admin bundle is served from the artifact at `/plugins/<name>/` — **no longer
go:embed'd**, but `web/dist` must be built (`cd web && pnpm run build`) before
`ninja plugin build`. Makefile convention: copy `plugins/symposium/Makefile`
(`build` / `build-frontend` / `verify` / `archive-check` / `publish` — archive-check now
proves `git archive HEAD` wasm-compiles). Generated Go (`*_templ.go`, `db/*.sql.go`,
`*connect.go`) and `web/dist` stay committed; `.gitignore` adds `*.bnp` and `*.wasm`.
**Dev loop:** publish to the DEV orchestrator (`ninja --host
https://my.localdev.blockninjacms.com plugin publish --bnp …` — `my.blockninjacms.com`
without `localdev` is PRODUCTION). Install via Admin → Plugins → Browse Registry
(`InstallFromRegistry`): the instance downloads the `.bnp`, checksum-verifies, and
**hot-loads without restart**; updates hot-swap the same way. First publish lands
`private`; review via the orchestrator dashboard. Private account-scoped plugins:
`--private` on init/publish.
## Verifying SDK symbols
Before using an unfamiliar SDK call, check: (1) the pinned SDK — `go doc
git.dev.alexdunmow.com/block/core/plugin CoreServices` from the plugin dir; (2) the
capability disposition table in `core/docs/wasm-abi.md` (does it cross the ABI, and how?);
(3) nearest exemplar usage in testplugin/symposium; (4) the CMS-side implementation in
`cms/backend` for semantics.
**Missing capability** ⇒ two sanctioned paths only: extend `block/core` (guest stub + host
function + ABI proto — additive within ABI major 1), or vendor the cms-internal package
into the plugin's `internal/` with a provenance header. NEVER `replace` directives; NEVER
`block/cms` imports. Remember the wasm boundary: a new capability that returns
functions/handlers can't cross — it needs a hook or a manifest declaration instead.
## Doc routing
| Working on | Read first |
|---|---|
| ABI, hooks, capabilities, error codes, `data_dir`, build/pack internals | `core/docs/wasm-abi.md` |
| Scaffold, registration, custom block types, `CoreServices` concepts | `cms/docs/PLUGIN_DEVELOPMENT.md` |
| **Section/content blocks — html block vs custom; menus; live vs pre-render** | **[html-blocks.md](html-blocks.md)** (in this skill) — read BEFORE writing a custom block type |
| Depositing media (seed `EnsureMedia` / runtime `CoreServices.Media`) | PLUGIN_DEVELOPMENT.md §Depositing Media |
| Plugin migrations / DB tables — own schema, per-plugin role | PLUGIN_DEVELOPMENT.md §Migrations + this skill §Database |
| Platform data tables (`Provisioner.EnsureDataTable`, View-rootable JSONB store) | PLUGIN_DEVELOPMENT.md §Provisioning → Data tables |
| Public HTTP routes (webhooks, widgets) — buffered only | `cms/docs/PLUGIN_HTTP_HANDLERS.md` |
| Load/Unload, runtime state | `cms/docs/PLUGIN_LIFECYCLE_HOOKS.md` + this skill §Per-instance state |
| Themes, templates, master pages, CSS | `cms/docs/TEMPLATE_PLUGINS.md` |
| Theme preview screenshots (`ninja theme screenshot`, registry `previewImageUrl`) | `cms/docs/theme-previews.md` |
| Block editor / settings UI (Module Federation) | `cms/docs/PLUGIN_EDITOR_SDK.md` |
| Migration rationale, what stayed/died | the 2026-07-03 wasm migration spec (cms/docs/superpowers/specs) |
Stale docs — do NOT follow: `cms/docs/compiled-plugin-architecture.md` (.so pipeline,
dead) and any `plugins/CLAUDE.md` / memory notes describing `make build-so`, in-container
compiles, go:embed'd web bundles, or core-version skew.
## Release traps ## Release traps
```bash - `ninja plugin publish` **without `--bnp` ships a source archive** — the registry accepts
ninja plugin bump patch # commits plugin.mod — does NOT git-tag it but every instance install fails with "legacy source archive; republish as .bnp".
git tag vX.Y.Z # manual; must equal plugin.mod version (hard rule 6) Always build first and pass `--bnp` (or `make publish`).
git push origin main vX.Y.Z # explicit — --follow-tags skips lightweight tags - **`kind` is frozen at first publish.** The registry stores `kind` (`plugin`|`theme`) on
ninja plugin publish # ships `git archive HEAD`: untracked files DON'T ship; first `CreatePlugin` and never updates it; publish only compares. A theme MUST declare
# web/dist and ALL generated Go must be committed `kind = "theme"` BEFORE its first publish. Dev-only remedy for a misregistered row:
``` `UPDATE registry_plugins SET kind='theme'` on `orchestrator-db`, then republish.
- `ninja plugin publish` ships metadata from `git HEAD` (README/CHANGELOG, warnings on
dirty tree) and the artifact from `--bnp` — keep the tree committed at the bumped
version so the two agree; `make archive-check` proves HEAD compiles.
- `ninja plugin init` is interactive and DROPS unknown `plugin.mod` keys on rewrite —
re-check `plugin.mod` after running it (`data_dir` is a known key and survives).
**`kind` is frozen at first publish.** The registry stores `kind` (`plugin`|`theme`) on the first `CreatePlugin` and NEVER updates it; `publish` only *compares* plugin.mod `kind` against that frozen DB column and rejects a mismatch with `plugin.mod kind does not match registered kind`. A theme MUST declare `kind = "theme"` BEFORE its first publish — editing plugin.mod afterwards is not enough (no RPC re-aligns it). Remedy for an already-misregistered theme (dev only, no supported RPC): `UPDATE registry_plugins SET kind='theme' WHERE id=<row>` on `orchestrator-db` (preserves version history), then republish. gotham/lcars first shipped `kind=plugin`; lcars corrected this way 2026-06. New plugin: `ninja plugin init`, copy `main.go` + layout from testplugin and the Makefile
from symposium, minimal registration per PLUGIN_DEVELOPMENT.md.
New plugin: `ninja plugin init`, minimal files per PLUGIN_DEVELOPMENT.md §Minimal Layout, copy the Makefile from messenger. First publish lands `private`; review via orchestrator dashboard.

2
developing-blockninja-plugins/html-blocks.md
View File

@ -114,7 +114,7 @@ re-seeding.
> The `menus`-in-html-block capability is a CMS-core feature (`cms/backend/blocks/builtin/html.go` > The `menus`-in-html-block capability is a CMS-core feature (`cms/backend/blocks/builtin/html.go`
> injects it). If a target CMS predates it, that's a core change (rebuild the backend), not an SDK > injects it). If a target CMS predates it, that's a core change (rebuild the backend), not an SDK
> bump — the plugin imports `block/core` only, so a cms-internal change keeps the `.so` compatible. > bump — the plugin imports `block/core` only, so a cms-internal change keeps the plugin's wasm artifact compatible.
## Gotchas ## Gotchas