Develop helixCore

A practical guide to developing Helix and its client extensions from a Honeywick workstation. The everyday loop is bare-metal, no container; the container is for testing and shipping deployments.

1. What you’re building

helixCore — the client-agnostic platform: a React SPA + a TypeScript backend (crud, dispatch) that wraps the proprietary Spiral optimiser (spiral.l64), backed by MongoDB.
Client extensions — per-client repos (e.g. AA) that plug into helixCore via the published @helix/client-sdk: domain/map config + React config/components. helixCore ships without any client baked in.
Spiral — the optimisation engine, built separately.

The split: anything client-specific (wording, behaviour, templates, icons, custom components) lives in the client repo / domain file — never hardcoded in helixCore.

2. Repo layout on a Honeywick box

~/Honeywick/
  Helix/                         # the helixCore platform repo
    helixCore/
      frontend/                  # React SPA (Vite)
      *.ts                       # backend: crud, dispatch, spiral, queue, crudObject, …
      client-sdk/                # @helix/client-sdk source
      startHelix                 # bare-metal + container launcher
      config                     # env contract (ports, paths, domain/map)
    release/helixctl             # container launcher (host-side)
    Dockerfile.release           # client-agnostic runtime image
    publishHelixCore.sh          # build + push the image / SDK
    AA -> ../Clients/AA/helix/custom   # symlink to the active client's customisation (per client)
  Clients/<CLIENT>/              # each client's repo (git)
    helix/
      custom/                    # client extension SOURCE (= Helix/<CLIENT> via the symlink)
        <CLIENT>.domain.json     # Spiral/domain config (stopParams, resourceParams, helix.* blocks)
        <CLIENT>.map.json        # Spiral map
        <CLIENT>.ui.json         # UI params
        config                   # client env overrides (sourced by helixCore/config)
        frontend/clientConfig.tsx# the client extension (templates, helixOverrides, logo, editors)
        vite.client.config.ts    # builds the extension -> ../runtime/client
        package.json             # depends on @helix/client-sdk
      runtime/                   # launcher stub + version pins + built bundle (client/, served as /client)
      scenarios/                 # Helix-only client scenarios
    tests/                       # client tests + results (ROOT — shared with the Spiral-only world)
    release/                     # legacy Honeywick/Spiral artifacts
  Spiral/                        # the Spiral optimiser source (builds spiral.l64)

Each client provides Helix/<CLIENT> as a symlink to Clients/<CLIENT>/helix/custom — that’s what startHelix <CLIENT> resolves.

3. Prerequisites (one-time)

On PATH: node, npm, mongod, mongosh.
Environment (typically in ~/.bashrc):
- SPIRAL_LICENCE — a Spiral command, e.g. {"set":{"licence":"<hex>"}} (not the raw key; Spiral reads this env var via an include environment directive). The app refuses to start without it.
- GOOGLE_MAPS_API_KEY — for the map.
- Optional: SPIRAL_JSON, SPIRAL_TRAVEL_CALCULATOR, AA_VEHICLE_API_KEY.
Install deps: (cd helixCore/frontend && npm install) and (cd Clients/<CLIENT>/helix/custom && npm install).

4. The everyday loop — bare-metal dev (no container)

From the Helix repo:

./helixCore/startHelix <CLIENT> --dev        # e.g. ./helixCore/startHelix AA --dev

This brings up, with dynamic ports (printed) and opens the browser:

mongod (data under ~/helix/<CLIENT>/logs/dev/mongoData),
dispatch + crud (the backend, wrapping Spiral),
a Vite dev server with HMR.

Live reload covers both sides:

editing the helixCore frontend (helixCore/frontend/src/**) hot-reloads,
editing the client extension (Clients/<CLIENT>/helix/custom/frontend/**, resolved via the @client alias) hot-reloads too.

Domain/map are read from Clients/<CLIENT>/helix/custom/<CLIENT>.domain.json + .map.json.

Logs (everything you need to debug): ~/helix/<CLIENT>/logs/dev/ — crud.log, dispatch.log, spiral.log, crud2spiral.log, vite.log, mongo.

Other bare-metal modes (same script)

--solo — single node, no persistence, no Vite HMR (logs in …/logs/solo-<pid>).
--spiral <runfile> — read-only Spiral streaming: feed a run file straight to Spiral; no entity edits.
--master / --replicate <primaryHost> / --readOnly <primaryHost> — cluster nodes (mongo replica set).

This is the primary Honeywick workflow — every run mode is testable here with no container.

5. The client-extension model

clientConfig.tsx exports config: ClientConfig — templates (entity defaults via createTemplates), helixOverrides (custom helix.* functions), about, clientLogo, and optional data-extensions / full-control editors.
@helix/client-sdk provides the types + helpers (ClientConfig, createTemplates, StopState, colors, …). In Honeywick dev it resolves to the helixCore source via a Vite alias; a standalone client build resolves the published package from the GitLab npm registry.
Domain config drives the UI. stopParams entries carry a helix block that the frontend reads — icons, tag, warning templates, the stop templates (option generator), add/morph-stop menus, state colours, etc. Change behaviour/wording there, not in the frontend.
helix.* parameters are string expressions (e.g. "helixColorDelay()") bound to clientModule functions and client helixOverrides, evaluated with this = the stop/resource (StopContext, with typed getInStop()/getOutStop() accessors). {prop|default} placeholders interpolate values. Resolution chains client helixOverrides first, then core clientModule, and a function returning undefined defers to the previous override — so overrides can handle only the cases they care about.
Icons live in helixCore/frontend/public/icons/*.png (core) plus client icons, named via the icon registry (icons.tsx).

Trip hazard: a Vite define cannot carry per-client config. frontend/vite.config.mjs bakes several __HELIX_*__ values from process.env at build time, and Dockerfile.release builds the SPA client-agnostically — with no client’s config sourced. So any define fed by a client-supplied env var is empty in every released image, and the behaviour it gates silently never happens in a container, while working perfectly in bare-metal dev (which does source the client config before Vite runs).

This bit us once already: client helixOverrides were registered only if the module was named in HELIX_CLIENT_MODULES, so no client override resolved in any containerised deployment — the fault surfaced as wrong colours on unresourced stops, because those states were the only ones whose value came from an override. The fix was to stop asking a build-time define a runtime question: the overrides arrive as a field of the runtime-loaded ClientConfig, so they are now loaded whenever they are present.

The rule: if a value can differ per client, it must reach the frontend at runtime — via /api/config, the reference/domain, or the loaded ClientConfig — never through a define. Defines are for facts about the build (core version, git state). Bare-metal dev will not catch the difference.

6. Backend development

The backend is the root *.ts in helixCore/ (crud.ts, dispatch.ts, spiral.ts, queue.ts, crudObject.ts, crudHandlers.ts, …).
crud spawns spiral.l64 and talks to it over stdin/stdout, persisting to MongoDB; dispatch tails Mongo change streams and broadcasts to the SPA over WebSockets.
Tags correlate a command with Spiral’s per-command “done” echo (used for REST replies, progress, and run-file completion). Input and output run on two independent async queues so command delivery isn’t throttled by output processing.
Client backend hooks (crudClientModule.ts core + the client’s crudClientModule.js, merged in crud.ts): the crud calls module functions named by the domain — stopParams[type].helix.check (validation, input path, crudObject.ts), resourceParams[type].helix.onCreate/onDelete (entity lifecycle, input path, live mutations only), and stopParams[type].helix.onState[from][to] (stop transitions, Spiral-output path, crudHandlers.ts updateStopTable). Each gets this = entity/stop and a CrudHookInfo first arg; info.now = ctx.now() (wall-clock + Spiral’s timeDelta). null matrix keys mark create/delete edges. Only entity types with a params block (currently resource) can carry entity hooks. The contract types are exported from @helix/client-sdk.
Functional tests are run files: Clients/<CLIENT>/tests/regression/*.run.json (repo root; shared with the Spiral-only world; replayed through the CRUD pipeline). There is no unit-test framework.

7. Type-check & quality gate

(cd helixCore/frontend && npx tsc --noEmit)   # frontend — aim for 0 errors
(cd helixCore         && npx tsc --noEmit)    # backend  — aim for 0 errors

Run both before committing. The regression .run.json files are the functional safety net.

8. Container development & deployment (`helixctl`)

The container is for deployment (and testing it) — not the daily loop. The launcher helixctl is client-agnostic; the client name is always explicit (mirrors startHelix <CLIENT>).

Build/refresh the image locally (when testing the container path):

./publishHelixCore.sh <ver>                 # build + push;  HELIX_OFFLINE_TAR=1 to save a local tar instead

Runtime: helixctl and the image run on podman or Docker (podman first when both are present; HELIX_CONTAINER_RUNTIME picks). Building the image is podman-only — publishHelixCore.sh uses podman’s --ignorefile, which docker build has no equivalent for. The saved air-gapped archive is a docker-archive, so helixctl load works under either runtime.

Fetch the launcher from the image (registry-only):

podman run --rm --entrypoint cat <image> /app/helixctl > helixctl && chmod +x helixctl

Container client dev (live rebuild, browser refresh — no HMR). To develop a client extension the way a client does (no helixCore source, no bare metal), use the launcher that ships in the client repo’s helix/runtime/ and run from that repo’s root:

cd Clients/<CLIENT>          # the client repo root (holds helix/ + tests/)
./helix/runtime/helixctl pull
./helix/runtime/helixctl <CLIENT> dev       # mounts helix/ (+ root tests/), runs `vite build --watch`
# edit client source -> ~1s rebuild (helix/runtime/logs/dev/clientbuild.log) -> refresh the browser

See Develop a client extension for the full client-side flow.

Deploy to a node:

(cd Clients/<CLIENT> && ./helix/runtime/helixctl release <ver> --as bundle)   # -> a bundle tar
helixctl deploy <bundle.tar>                          # unpacks client/ + helix-config/
helixctl <CLIENT> run | spiral | master | replicant | readOnly

run = writable single node; cluster modes need HELIX_PUBLIC_HOST (and HELIX_PRIMARY for replicant/readOnly).
HELIX_LOG_DIR_HOST=<dir> bind-mounts the container’s /app/logs to the host so logs survive --rm (defaulted for dev).

9. Testing client modes from Honeywick (no registry needed)

Bare-metal already exercises every mode: startHelix <CLIENT> --dev | --solo | --spiral <file> | --master | --replicate | --readOnly. Start here.
Local container loop to validate the deployed path: build the image locally (HELIX_OFFLINE_TAR=1 ./publishHelixCore.sh <ver>), build the client bundle, then point helixctl at the local image (HELIX_IMAGE=<local-ref> helixctl <CLIENT> dev … or … deploy && … run). No publish required.

10. Release flow

There are two release models, and they meet at the pin. The pin itself — floating versus exact, what a node reports, and how to upgrade one or a fleet — is set out in Versions and upgrades; this section is the release side of it. The core release publishes an engine; a client (or demo) release ships a tested pairing of that engine with an extension.

10.1 The core release model — `publishHelixCore.sh`

./publishHelixCore.sh <ver>                 # push (the default when neither flag is given)
./publishHelixCore.sh <ver> --set           # cut an air-gapped core set instead, no push
./publishHelixCore.sh <ver> --push --set    # both — the registry AND the airgap in one run

--push and --set are independent outputs, not alternatives, which is the point of the pair: a release commonly wants both. --set cuts helixcore-release-<ver>.tar by calling helixctl release --as core-set — one assembler for every release set, living in the launcher rather than duplicated here. The far side installs it with helixctl install (load + pin; no bundle, so no deploy step). HELIX_OFFLINE_TAR=1 still means exactly what it always did — --set and no push — because an offline build box sets it precisely because it cannot reach a registry.

Three tags per build: :<ver>, :latest, and the floating :sdk-<MAJOR.MINOR> (e.g. :sdk-0.1) computed from helixCore/client-sdk/package.json.
The image carries its own identity: /app/VERSION (the true core) and /app/SDK_VERSION (the contract), plus the helix.version / helix.sdk labels. Everything downstream reads these rather than trusting a tag, because a floating tag by definition cannot say which core it is.
helixCore/shared/version.ts is stamped automatically — commit it with the release.
@helix/client-sdk → GitLab npm registry; bump it when the SDK contract changes. A breaking bump to 0.2.x starts a fresh :sdk-0.2 line and leaves :sdk-0.1 frozen at the last 0.1 core.

:sdk-0.1 is what a client repo develops against: helixctl pull then fetches the newest core on that SDK line without touching the extension. It is not what a client releases on — see below.

10.2 The client release model — `helixctl release`

One command, four delivery forms. They start from the same two facts — an exact core, and a bundle built by that very core inside a container — so they differ only in how the result travels:

`--as`	artifact	the node runs	who it is for
`bundle`	`helix-<client>-<ver>.tar`	`helixctl deploy`	a node already on the right core
`set` (default)	one tar: core image + bundle + launcher + pin	`helixctl install`	air-gapped or controlled production
`image`	one image, client baked in, pushed	`helixctl pull`	registry-based deployment
`core-set`	the core alone — no client	`helixctl install`, then `deploy`	bootstrapping a machine with no core, or an air-gapped estate (this is what `publishHelixCore.sh --set` cuts)

cd Clients/<CLIENT>
./helix/runtime/helixctl release <ver>              # a set
./helix/runtime/helixctl release <ver> --as image   # a full client container

release resolves the current pin to an exact core, builds the extension in a container of that image (helixctl __bundle — the launcher is baked in, so there is nothing to mirror into client repos), and refuses if the bundle’s stamped coreVersion is not the core being shipped.

The host needs no node, npm or client toolchain — only the container runtime it already needed. That is what helixctl check has always claimed for client development, and until 0.0.5 the release step quietly broke it: publishHelixClient.sh ran npm install on the host, and lived as a hand-synced copy in every client repo. It is gone; --as bundle is what it used to produce.

A client image is the exact core with the sealed bundle baked at /client.tar, labelled helix.client / helix.client.version / helix.client.sdk, inheriting the core’s own helix.version and /app/VERSION. It publishes to <CLIENT_REGISTRY>/<client>:<ver> (a CLIENT_REGISTRY file beside helix/runtime/VERSION, or --image-ref), and pushes by default (--no-push holds it back). helixctl reads that label at launch: with a baked client it mounts nothing, and it refuses if the node also has a deployed bundle, or if the image is baked for a different client — one node, one answer.

10.3 Consuming a release

# a set
tar xf helix-<client>-release-<ver>.tar && cd helix-<client>-release-<ver>
./helixctl install          # load the image, pin VERSION, deploy the bundle
./helixctl <CLIENT> run

# an image — REGISTRY/VERSION point at the client image; there is nothing else to install
./helixctl pull && ./helixctl <CLIENT> run

Live modes refuse a floating pin. run, spiral, spiralOnly, master, replicant, readOnly and snapshot will not start on latest or sdk-*, because such a node cannot say which core it is running. helixctl pin resolves the float and writes the exact tag. dev, tuning and demo are exempt by design — they mount the full repo and switching core is part of the job.

Every container states its own identity at boot, from the image rather than the tag it was pulled under:

[startHelix] family=cluster concurrency=cluster mode=master client=AA core=0.0.5 bundle=AA-1.4.0 sdk=0.1.6 | mongo:durable(ext4)@…

helixctl version reports the same pairing from the host side, and helixctl deploy prints a line (never refuses) when a bundle’s coreVersion differs from the node’s core. SDK compatibility is still the gate: a 0.1.x bundle is refused on a 0.2 core.

10.4 Demo releases — the 100% container model

Each demo (Demos/BlueLight, Demos/HRA, Demos/HRM) is a standalone client-style repo, released exactly like a client and, on the demo server, always --as image:

cd Demos/<DEMO> && ./helix/runtime/helixctl release <ver> --as image

The demo server deploys nothing, unpacks nothing and mounts nothing. Which demos it serves, and at which version, is one line in /srv/helix-demo/secrets.env:

DEMO_SANDBOXES=breakdown:HRA@1.4.0,bluelight:BlueLight@0.9.2,rail:HRM@0.1.0

That line is the upgrade control. Change a version, systemctl restart helix-demo, and the next visitor gets the new build while sessions already running finish on the old image — a rolling upgrade for free, because a session container is per-visitor and ephemeral. Two demos may sit on different cores. curl /versions on the orchestrator reports exactly what is being served. See demoOrchestrator/README.md for the server layout, the Cloudflare tunnel and the session lifecycle.

11. Conventions

Domain-driven: client wording/behaviour lives in <CLIENT>.domain.json (stopParams.helix.*), not the frontend.
spiral.l64 is a built artifact (from ~/Honeywick/Spiral) — it is not maintained/committed from the Helix repo.
Logs: bare-metal under ~/helix/<CLIENT>/logs/{dev,solo-<pid>}; container via the HELIX_LOG_DIR_HOST bind mount.
Repos are separate: Helix (platform) and each Clients/<CLIENT> are independent git repos — commit each on its own; don’t cross-commit.
Commit only when asked; finish commit messages with the agreed Co-Authored-By trailer.