docs(spec): plugin publish (tarball) + categories design
Design for two coupled changes: drop git as the publish transport in favour of tar.zst uploads, and add a first-class plugin kind plus a configurable, validated category list. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
Alex Dunmow
parent
32c6528162
commit
1d9ca44f55
@ -0,0 +1,253 @@
|
||||
# Plugin publish (tarball) + categories — design
|
||||
|
||||
Date: 2026-06-03
|
||||
Status: Draft
|
||||
|
||||
## Motivation
|
||||
|
||||
Two related changes to the plugin registry:
|
||||
|
||||
1. **Drop git as the publish transport.** The current flow has `ninja plugin
|
||||
publish` push a git tag to a bare repo served over smart-HTTP by the
|
||||
orchestrator. This couples the client to git state (the `ninja` remote
|
||||
stored in `.git/config` just bit us with a URL-shape drift bug), forces
|
||||
the orchestrator to maintain bare repos plus an archive store, and gives
|
||||
the developer a remote they didn't ask for. Move to a single artifact:
|
||||
the CLI builds a `tar.zst` from the source tree and posts it.
|
||||
|
||||
2. **First-class plugin kind, configurable category list.** Today
|
||||
`registry_plugins.categories` (`text[]`) and `Plugin.categories` (proto)
|
||||
exist but nothing writes to them, there is no canonical list, and themes
|
||||
are not distinguished from plugins. Add a `kind` field
|
||||
(`plugin` | `theme`), validate categories against a server-side canonical
|
||||
list, and surface both in the CLI.
|
||||
|
||||
Greenfield — no consumers depend on the existing model. We can change shapes
|
||||
and wipe existing rows.
|
||||
|
||||
## Plugin kind & categories
|
||||
|
||||
### Model
|
||||
|
||||
`registry_plugins` gains:
|
||||
- `kind text NOT NULL DEFAULT 'plugin' CHECK (kind IN ('plugin', 'theme'))`
|
||||
|
||||
`registry_categories` is a new table:
|
||||
- `slug text PRIMARY KEY` — e.g. `analytics`, `social`, `import-export`
|
||||
- `display_name text NOT NULL`
|
||||
- `description text NOT NULL DEFAULT ''`
|
||||
- `sort_order int NOT NULL DEFAULT 0`
|
||||
|
||||
Seeded at migration time with a starter list (configurable later via an
|
||||
admin endpoint; out of scope for this spec). `registry_plugins.categories`
|
||||
values must be subsets of `registry_categories.slug`. Validation is
|
||||
enforced by the API layer, not by FK — categories may be removed later
|
||||
without orphaning plugins.
|
||||
|
||||
Themes do not carry categories in v1. Categories on theme rows are rejected
|
||||
with `CodeInvalidArgument`.
|
||||
|
||||
### plugin.mod
|
||||
|
||||
`[plugin]` gains two fields:
|
||||
|
||||
```toml
|
||||
[plugin]
|
||||
name = "lcars"
|
||||
scope = "@themes"
|
||||
version = "0.2.0"
|
||||
kind = "theme" # plugin | theme — defaults to "plugin" if absent
|
||||
categories = ["analytics"] # only allowed when kind = "plugin"
|
||||
```
|
||||
|
||||
The CLI keeps `plugin.mod` in sync with the registry. On `init` it writes
|
||||
both fields. On `publish` it sends them through and the orchestrator
|
||||
verifies the tarball's `plugin.mod` matches the request.
|
||||
|
||||
### Proto
|
||||
|
||||
`Plugin` gains:
|
||||
- `string kind = 10;`
|
||||
|
||||
`Plugin.categories` (existing) stays. `CreatePluginRequest` gains `kind`
|
||||
and `categories`. Listing endpoints gain a `kind` filter:
|
||||
|
||||
```proto
|
||||
message ListPluginsRequest {
|
||||
int32 limit = 1;
|
||||
int32 offset = 2;
|
||||
string query = 3;
|
||||
string kind = 4; // "" = any, "plugin", "theme"
|
||||
repeated string categories = 5;
|
||||
}
|
||||
```
|
||||
|
||||
A new `ListCategoriesRequest/Response` exposes the canonical list to the
|
||||
CLI for interactive picking.
|
||||
|
||||
## Tarball publish flow
|
||||
|
||||
### Format
|
||||
|
||||
`tar.zst` — `archive/tar` from stdlib for tarring, `github.com/klauspost/compress/zstd`
|
||||
for compression. Same dep on both ends. Default zstd level 3 (fast,
|
||||
~gzip-9 ratio).
|
||||
|
||||
A hard size cap of 25 MiB on the *compressed* (posted) bytes, enforced
|
||||
both sides. Configurable via `REGISTRY_PUBLISH_MAX_BYTES`. The orchestrator
|
||||
additionally caps decompressed size at 4× the configured value to limit
|
||||
zip-bomb exposure during validation.
|
||||
|
||||
### Archive contents
|
||||
|
||||
`git archive --format=tar HEAD` piped through zstd. This gives:
|
||||
- Only tracked files. Anything matching `.gitignore` is excluded so long
|
||||
as it isn't already tracked — i.e. `.gitignore` is honoured by virtue of
|
||||
those files never having been added. The one edge case is a file that
|
||||
was committed *before* being added to `.gitignore`: `git archive` will
|
||||
still ship it. We do not paper over this — the developer must
|
||||
`git rm --cached <file>` to drop it from the index. Documented in the
|
||||
CLI publish output if we detect tracked files matching the current
|
||||
`.gitignore`.
|
||||
- A clean snapshot at HEAD.
|
||||
- Deterministic enough that re-running publish with no changes produces
|
||||
byte-identical archives modulo timestamps.
|
||||
|
||||
The CLI shells out to `git archive HEAD` (no in-process git library
|
||||
dependency). It does not produce a tag, push, or modify `.git/config`.
|
||||
|
||||
### Proto change
|
||||
|
||||
`PublishVersionRequest` becomes:
|
||||
|
||||
```proto
|
||||
message PublishVersionRequest {
|
||||
string plugin_id = 1;
|
||||
string version = 2; // was: git_ref; now plain "0.2.0"
|
||||
string channel = 3;
|
||||
bytes archive = 4; // tar.zst, max 25 MiB
|
||||
string readme_md = 5;
|
||||
string changelog_md = 6;
|
||||
}
|
||||
```
|
||||
|
||||
`git_ref` is removed.
|
||||
|
||||
### Server-side validation
|
||||
|
||||
The orchestrator:
|
||||
1. Verifies the caller is a scope member.
|
||||
2. Decompresses the archive in-memory, walks the tar, finds `plugin.mod`.
|
||||
3. Parses `plugin.mod` and checks `scope`, `name`, `version`, `kind`, and
|
||||
`categories` match the request and the plugin row.
|
||||
4. Re-validates categories against `registry_categories.slug`.
|
||||
5. Computes sha256 and size of the *compressed* archive bytes (what gets
|
||||
stored).
|
||||
6. Stores via `regstorage.Put` with key
|
||||
`<scope>/<name>/<version>/source.tar.zst` and content-type
|
||||
`application/zstd`.
|
||||
7. Records the `registry_versions` row.
|
||||
8. Returns the existing `PublishVersionResponse` shape (a signed
|
||||
`archive_url` for immediate download verification).
|
||||
|
||||
`git_commit` / `git_tag` columns on `registry_versions` are dropped.
|
||||
|
||||
### Install flow
|
||||
|
||||
`ResolveInstall` still returns a signed URL. The only change is the object
|
||||
extension (`.tar.zst` instead of `.tar.gz`) and content-type.
|
||||
|
||||
## Init flow
|
||||
|
||||
`ninja plugin init`:
|
||||
|
||||
1. Prompt scope (existing behavior).
|
||||
2. Prompt plugin name (existing).
|
||||
3. Prompt kind (`plugin` | `theme`). Default: `plugin`.
|
||||
4. If `kind == plugin`: fetch the canonical category list via
|
||||
`ListCategories` and let the user multi-select (zero or more).
|
||||
5. Write `plugin.mod` with all fields.
|
||||
6. Call `CreatePlugin` with kind + categories.
|
||||
7. If in a git repo and `plugin.mod` is new or modified: `git add plugin.mod`,
|
||||
`git commit -m "Add plugin.mod"`. If the file is unchanged from HEAD,
|
||||
skip the commit.
|
||||
8. **Does not** add a `ninja` git remote. Does not touch `.git/config`.
|
||||
|
||||
If not in a git repo, init still writes `plugin.mod` and registers the
|
||||
plugin; it prints a warning that the user will need to `git init` before
|
||||
they can `publish` (`git archive` requires a repo).
|
||||
|
||||
## Publish flow
|
||||
|
||||
`ninja plugin publish`:
|
||||
|
||||
1. Read `plugin.mod`.
|
||||
2. Working-tree-clean check unless `--allow-dirty`.
|
||||
3. Lookup `Plugin` via `GetPlugin(scope, name)` to verify it exists and to
|
||||
get its `id`.
|
||||
4. `git archive --format=tar HEAD` → zstd compress → bytes in memory.
|
||||
5. Optional README/CHANGELOG bytes.
|
||||
6. Call `PublishVersion(plugin_id, version, channel, archive, readme,
|
||||
changelog)`.
|
||||
7. Print version + warnings.
|
||||
|
||||
No tag is created. No git push. No `.git/config` writes anywhere in the
|
||||
flow.
|
||||
|
||||
## What gets deleted
|
||||
|
||||
Orchestrator:
|
||||
- `backend/internal/registry/git/` — the entire package (server, repo
|
||||
manager, smart-HTTP, Provision, ResolveTag, ReadFileAtCommit, tests).
|
||||
- `r.Route("/git", …)` in `registry_wiring.go`.
|
||||
- `RegistryGitPath` config.
|
||||
- `git_commit`, `git_tag` columns on `registry_versions`.
|
||||
- `git_remote_url` field from `CreatePluginResponse` (the URL no longer
|
||||
exists — there is nothing for the developer to push to manually).
|
||||
|
||||
CLI:
|
||||
- `git remote remove ninja` / `git remote add ninja …` block in
|
||||
`cmd/ninja/cmd/plugin.go`.
|
||||
- `git tag -a vX.Y.Z` / `git push ninja vX.Y.Z` block.
|
||||
|
||||
Database:
|
||||
- `git_commit`, `git_tag` columns dropped.
|
||||
- `registry_plugins` rows can be truncated for dev environments since the
|
||||
schema is changing; production is greenfield so this is moot.
|
||||
|
||||
## What gets added
|
||||
|
||||
Orchestrator:
|
||||
- `registry_categories` table + a migration that seeds it with a starter
|
||||
list. Slugs only; copy can be refined later.
|
||||
- A `kind` column on `registry_plugins`.
|
||||
- `ListCategories` RPC added to the existing `PluginRegistryService`
|
||||
(one service is fine at this size).
|
||||
- Archive validation helper (open zstd, walk tar, extract `plugin.mod`).
|
||||
- `REGISTRY_PUBLISH_MAX_BYTES` config (default `25 * 1024 * 1024`).
|
||||
|
||||
CLI:
|
||||
- `bump`, `version` (just shipped, no change).
|
||||
- Init prompts for kind + categories; calls `ListCategories`.
|
||||
- Publish builds `tar.zst` from `git archive HEAD`.
|
||||
|
||||
Proto:
|
||||
- `Plugin.kind`, `CreatePluginRequest.kind/categories`,
|
||||
`ListPluginsRequest.kind/categories`, new `ListCategories*` messages,
|
||||
`PublishVersionRequest.archive` (bytes), removal of `git_ref` and the
|
||||
top-level `git_remote_url` from `CreatePluginResponse`.
|
||||
|
||||
## Open questions
|
||||
|
||||
- What is the starter category list? Seed something reasonable; admins
|
||||
can curate it later. Suggested starting set: `analytics`, `seo`,
|
||||
`social`, `commerce`, `forms`, `import-export`, `media`, `developer`.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- Admin UI for managing categories (use a migration for now).
|
||||
- Category translations / i18n.
|
||||
- Plugin search by category in the install UI (data is there once the
|
||||
field is populated; consumers can use it when ready).
|
||||
- Auth for the publish endpoint — already in place via existing bearer
|
||||
token interceptor.
|
||||
Loading…
x
Reference in New Issue
Block a user