From 1d9ca44f55c5b238ef9774173d6912db7fe83577 Mon Sep 17 00:00:00 2001 From: Alex Dunmow Date: Wed, 3 Jun 2026 00:58:21 +0800 Subject: [PATCH] docs(spec): plugin publish (tarball) + categories design Design for two coupled changes: drop git as the publish transport in favour of tar.zst uploads, and add a first-class plugin kind plus a configurable, validated category list. Co-Authored-By: Claude Opus 4.7 --- ...03-plugin-publish-and-categories-design.md | 253 ++++++++++++++++++ 1 file changed, 253 insertions(+) create mode 100644 docs/superpowers/specs/2026-06-03-plugin-publish-and-categories-design.md diff --git a/docs/superpowers/specs/2026-06-03-plugin-publish-and-categories-design.md b/docs/superpowers/specs/2026-06-03-plugin-publish-and-categories-design.md new file mode 100644 index 0000000..efe8c5f --- /dev/null +++ b/docs/superpowers/specs/2026-06-03-plugin-publish-and-categories-design.md @@ -0,0 +1,253 @@ +# Plugin publish (tarball) + categories — design + +Date: 2026-06-03 +Status: Draft + +## Motivation + +Two related changes to the plugin registry: + +1. **Drop git as the publish transport.** The current flow has `ninja plugin + publish` push a git tag to a bare repo served over smart-HTTP by the + orchestrator. This couples the client to git state (the `ninja` remote + stored in `.git/config` just bit us with a URL-shape drift bug), forces + the orchestrator to maintain bare repos plus an archive store, and gives + the developer a remote they didn't ask for. Move to a single artifact: + the CLI builds a `tar.zst` from the source tree and posts it. + +2. **First-class plugin kind, configurable category list.** Today + `registry_plugins.categories` (`text[]`) and `Plugin.categories` (proto) + exist but nothing writes to them, there is no canonical list, and themes + are not distinguished from plugins. Add a `kind` field + (`plugin` | `theme`), validate categories against a server-side canonical + list, and surface both in the CLI. + +Greenfield — no consumers depend on the existing model. We can change shapes +and wipe existing rows. + +## Plugin kind & categories + +### Model + +`registry_plugins` gains: +- `kind text NOT NULL DEFAULT 'plugin' CHECK (kind IN ('plugin', 'theme'))` + +`registry_categories` is a new table: +- `slug text PRIMARY KEY` — e.g. `analytics`, `social`, `import-export` +- `display_name text NOT NULL` +- `description text NOT NULL DEFAULT ''` +- `sort_order int NOT NULL DEFAULT 0` + +Seeded at migration time with a starter list (configurable later via an +admin endpoint; out of scope for this spec). `registry_plugins.categories` +values must be subsets of `registry_categories.slug`. Validation is +enforced by the API layer, not by FK — categories may be removed later +without orphaning plugins. + +Themes do not carry categories in v1. Categories on theme rows are rejected +with `CodeInvalidArgument`. + +### plugin.mod + +`[plugin]` gains two fields: + +```toml +[plugin] +name = "lcars" +scope = "@themes" +version = "0.2.0" +kind = "theme" # plugin | theme — defaults to "plugin" if absent +categories = ["analytics"] # only allowed when kind = "plugin" +``` + +The CLI keeps `plugin.mod` in sync with the registry. On `init` it writes +both fields. On `publish` it sends them through and the orchestrator +verifies the tarball's `plugin.mod` matches the request. + +### Proto + +`Plugin` gains: +- `string kind = 10;` + +`Plugin.categories` (existing) stays. `CreatePluginRequest` gains `kind` +and `categories`. Listing endpoints gain a `kind` filter: + +```proto +message ListPluginsRequest { + int32 limit = 1; + int32 offset = 2; + string query = 3; + string kind = 4; // "" = any, "plugin", "theme" + repeated string categories = 5; +} +``` + +A new `ListCategoriesRequest/Response` exposes the canonical list to the +CLI for interactive picking. + +## Tarball publish flow + +### Format + +`tar.zst` — `archive/tar` from stdlib for tarring, `github.com/klauspost/compress/zstd` +for compression. Same dep on both ends. Default zstd level 3 (fast, +~gzip-9 ratio). + +A hard size cap of 25 MiB on the *compressed* (posted) bytes, enforced +both sides. Configurable via `REGISTRY_PUBLISH_MAX_BYTES`. The orchestrator +additionally caps decompressed size at 4× the configured value to limit +zip-bomb exposure during validation. + +### Archive contents + +`git archive --format=tar HEAD` piped through zstd. This gives: +- Only tracked files. Anything matching `.gitignore` is excluded so long + as it isn't already tracked — i.e. `.gitignore` is honoured by virtue of + those files never having been added. The one edge case is a file that + was committed *before* being added to `.gitignore`: `git archive` will + still ship it. We do not paper over this — the developer must + `git rm --cached ` to drop it from the index. Documented in the + CLI publish output if we detect tracked files matching the current + `.gitignore`. +- A clean snapshot at HEAD. +- Deterministic enough that re-running publish with no changes produces + byte-identical archives modulo timestamps. + +The CLI shells out to `git archive HEAD` (no in-process git library +dependency). It does not produce a tag, push, or modify `.git/config`. + +### Proto change + +`PublishVersionRequest` becomes: + +```proto +message PublishVersionRequest { + string plugin_id = 1; + string version = 2; // was: git_ref; now plain "0.2.0" + string channel = 3; + bytes archive = 4; // tar.zst, max 25 MiB + string readme_md = 5; + string changelog_md = 6; +} +``` + +`git_ref` is removed. + +### Server-side validation + +The orchestrator: +1. Verifies the caller is a scope member. +2. Decompresses the archive in-memory, walks the tar, finds `plugin.mod`. +3. Parses `plugin.mod` and checks `scope`, `name`, `version`, `kind`, and + `categories` match the request and the plugin row. +4. Re-validates categories against `registry_categories.slug`. +5. Computes sha256 and size of the *compressed* archive bytes (what gets + stored). +6. Stores via `regstorage.Put` with key + `///source.tar.zst` and content-type + `application/zstd`. +7. Records the `registry_versions` row. +8. Returns the existing `PublishVersionResponse` shape (a signed + `archive_url` for immediate download verification). + +`git_commit` / `git_tag` columns on `registry_versions` are dropped. + +### Install flow + +`ResolveInstall` still returns a signed URL. The only change is the object +extension (`.tar.zst` instead of `.tar.gz`) and content-type. + +## Init flow + +`ninja plugin init`: + +1. Prompt scope (existing behavior). +2. Prompt plugin name (existing). +3. Prompt kind (`plugin` | `theme`). Default: `plugin`. +4. If `kind == plugin`: fetch the canonical category list via + `ListCategories` and let the user multi-select (zero or more). +5. Write `plugin.mod` with all fields. +6. Call `CreatePlugin` with kind + categories. +7. If in a git repo and `plugin.mod` is new or modified: `git add plugin.mod`, + `git commit -m "Add plugin.mod"`. If the file is unchanged from HEAD, + skip the commit. +8. **Does not** add a `ninja` git remote. Does not touch `.git/config`. + +If not in a git repo, init still writes `plugin.mod` and registers the +plugin; it prints a warning that the user will need to `git init` before +they can `publish` (`git archive` requires a repo). + +## Publish flow + +`ninja plugin publish`: + +1. Read `plugin.mod`. +2. Working-tree-clean check unless `--allow-dirty`. +3. Lookup `Plugin` via `GetPlugin(scope, name)` to verify it exists and to + get its `id`. +4. `git archive --format=tar HEAD` → zstd compress → bytes in memory. +5. Optional README/CHANGELOG bytes. +6. Call `PublishVersion(plugin_id, version, channel, archive, readme, + changelog)`. +7. Print version + warnings. + +No tag is created. No git push. No `.git/config` writes anywhere in the +flow. + +## What gets deleted + +Orchestrator: +- `backend/internal/registry/git/` — the entire package (server, repo + manager, smart-HTTP, Provision, ResolveTag, ReadFileAtCommit, tests). +- `r.Route("/git", …)` in `registry_wiring.go`. +- `RegistryGitPath` config. +- `git_commit`, `git_tag` columns on `registry_versions`. +- `git_remote_url` field from `CreatePluginResponse` (the URL no longer + exists — there is nothing for the developer to push to manually). + +CLI: +- `git remote remove ninja` / `git remote add ninja …` block in + `cmd/ninja/cmd/plugin.go`. +- `git tag -a vX.Y.Z` / `git push ninja vX.Y.Z` block. + +Database: +- `git_commit`, `git_tag` columns dropped. +- `registry_plugins` rows can be truncated for dev environments since the + schema is changing; production is greenfield so this is moot. + +## What gets added + +Orchestrator: +- `registry_categories` table + a migration that seeds it with a starter + list. Slugs only; copy can be refined later. +- A `kind` column on `registry_plugins`. +- `ListCategories` RPC added to the existing `PluginRegistryService` + (one service is fine at this size). +- Archive validation helper (open zstd, walk tar, extract `plugin.mod`). +- `REGISTRY_PUBLISH_MAX_BYTES` config (default `25 * 1024 * 1024`). + +CLI: +- `bump`, `version` (just shipped, no change). +- Init prompts for kind + categories; calls `ListCategories`. +- Publish builds `tar.zst` from `git archive HEAD`. + +Proto: +- `Plugin.kind`, `CreatePluginRequest.kind/categories`, + `ListPluginsRequest.kind/categories`, new `ListCategories*` messages, + `PublishVersionRequest.archive` (bytes), removal of `git_ref` and the + top-level `git_remote_url` from `CreatePluginResponse`. + +## Open questions + +- What is the starter category list? Seed something reasonable; admins + can curate it later. Suggested starting set: `analytics`, `seo`, + `social`, `commerce`, `forms`, `import-export`, `media`, `developer`. + +## Out of scope + +- Admin UI for managing categories (use a migration for now). +- Category translations / i18n. +- Plugin search by category in the install UI (data is there once the + field is populated; consumers can use it when ready). +- Auth for the publish endpoint — already in place via existing bearer + token interceptor.