Manage Hikyo with Hikyo

Hikyo owns a protected configuration project on each logical instance. Open Instance settings → Manage Hikyo configuration to find the project, the running revision and the state of every admitted node.

:::note[Apply scope] Ordinary owner and node settings reload live. Startup settings require explicit deployment enrollment. The Kubernetes provider supports a singleton Recreate Deployment; it does not move data, install a new image or change operator custody. See Configuration rollouts on Kubernetes. :::

Set up or adoptCopy Anchor Link

Fresh administrator setup provisions a protected Hikyo organization, an instance project and its Production environment. Existing installations use Preview adoption. The preview lists configured key names and any import warnings without displaying secret values. Review it, then adopt with fresh reauthentication. Adoption imports the server’s effective settings once.

Adoption grants the initiating administrator normal scoped project access. Sign in again when requested so the session reflects those grants. Instance administration alone does not grant secret reveal or history disclosure.

Once adopted, the applied revision owns the supported settings. Stale startup variables and imported file paths cannot override it. Owner and node settings use the catalogue below. Database and root sources remain in external custody; an enrolled deployment provider can apply selected source aliases through a controlled rollout. Enrolled singleton HA mode and node identity use the topology workflow below. Multi-member transitions and other unsupported startup transitions remain externally managed. See Configuration.

Change mail settings without restartingCopy Anchor Link

1. Open Edit configuration project. Change values in the normal matrix.
2. Publish the drafts. Publishing creates a revision; running settings stay unchanged.
3. Return to Hikyo configuration, select the exact published revision and optionally send a test email to a named recipient.
4. Select Apply selected revision. Hikyo prepares it first; review Reload live or Controlled rollout, then authorize the exact ready plan with a fresh authenticator code or passkey.
5. Check the node list. A pending or partial apply is still converging; only completed node acknowledgements establish that all admitted nodes use the target.

A test uses the selected revision and does not activate it. Apply validates the mail settings locally and does not contact the mail relay. A relay connection failure does not roll back the applied configuration. In-flight operations can finish using the bundle they started with; subsequent operations use the new bundle.

Original mail and notification keysCopy Anchor Link

The project schema belongs to the runtime. Values use ordinary drafts, publication, encryption and history. Users cannot change the catalogue’s types, secret classification or source mode. Certificate and password files are imported as contents once during adoption. Remote apply never reads arbitrary server filesystem paths.

Owner and node settingsCopy Anchor Link

The catalogue includes the nine values above, shared owner settings, HIKYO_NODE_OVERRIDES and HIKYO_BOOTSTRAP_SOURCES.

Change the project key limitCopy Anchor Link

1. Open Instance settings → Manage Hikyo configuration → Edit configuration project.
2. In the configuration project’s Matrix, set HIKYO_DEFAULT_KEYS_PER_PROJECT to the desired inherited limit, for example 1000, and HIKYO_MAX_KEYS_PER_PROJECT to the maximum project administrators may choose, for example 10000. The default must not exceed the maximum.
3. Publish, return to Hikyo configuration, select that revision and choose Apply selected revision with fresh administrator authentication.
4. Check that every admitted node acknowledges the applied revision. Subsequent key creation and definitions plan/apply use the new limit.

These settings apply to every project owned by this logical instance. They count declared keys once, regardless of how many environments contain their values. The 1000 default is a conservative starting point for small installations; it is not a measured Raspberry Pi capacity threshold. Larger values increase catalogue, Matrix, validation and publish work. Choose a value appropriate to your hardware and verify your workload before relying on the extra capacity.

Projects also have a 100,000-cell budget: declared keys × environments. For example, 2500 keys fit in 40 environments; adding a key or a 41st environment at that point is refused. At 10000 keys the budget permits at most 10 environments. The setting leaves value sizes, publish deadlines, render-size limits, storage limits and operation-specific batch limits intact. Definitions bundles retain their separate limit of 10,000 entries counting keys, environments and groups together, plus their 1 MiB size limit.

To give one project more capacity, open Project settings > Policy > Key capacity, select a custom limit such as 5000, and save. Other projects can continue inheriting 1000. This requires project-settings authority and actual policy changes are audited. Select Inherit instance default to clear the override. The panel shows current key and environment counts, instance default and maximum, and the effective project limit.

The effective limit is min(saved override or instance default, instance maximum). Lowering the instance maximum retains a larger saved override and caps the effective limit. Raising the maximum restores the saved override up to the new maximum. A project administrator cannot save a new override above the active maximum. If project or instance policy changes while settings are open, review the refreshed policy before saving again; stale writes are refused.

Lowering either policy preserves existing keys and delivery. Existing keys remain editable and deletable, and definitions plans can keep or reduce the current count. Adding keys above the new limit is refused. A plan created before a limit reduction is checked against the active setting again when applied. Removing either managed value restores its catalogue default after Publish and Apply, provided the default/maximum pair remains valid.

The same project policy is available through the CLI:

hikyo project capacity get --org example --project backend
hikyo project capacity set --org example --project backend --max-keys 5000
hikyo project capacity set --org example --project backend --inherit

Use -o json for the full policy and usage response. A CLI update reads the current policy and submits its revision and instance bounds with the write; conflicts are reported without automatically retrying the change.

HIKYO_NODE_OVERRIDES is an encrypted secret JSON document. Its version-1 nodes map keys are the stable IDs shown in status. Each admitted node needs its own entry; never copy another node’s identity or assume independent remotes share values. Edit listener addresses, operational listener, PostgreSQL pool limit, admission memory budget, backup directory and trusted proxy CIDRs in that node’s entry. Trusted proxy policy is exclusively node-local.

TLS uses HIKYO_TLS_CERT_PEM and HIKYO_TLS_KEY_PEM contents in the same entry. Adapter, OIDC and dynamic egress use their *_EGRESS_POLICY_JSON contents. Initial file aliases import once; private TLS files must have mode 0400 or 0600. After adoption, edit and Apply the managed contents instead of expecting an old file path or SIGHUP to override them. A mismatched certificate/key pair is refused before activation.

Live Apply prepares resources, drains old requests and workers, installs the new service graph and acknowledges actual installation. Changing a listener inside Hikyo does not configure an external reverse proxy, firewall, Service or Ingress. Make the new address reachable through the existing deployment. On nodes already started with --dev, the node entry may also contain HIKYO_DEV_ADMISSION_PER_IP_PER_MINUTE, HIKYO_DEV_SERVICE_BUDGETS_DISABLED and HIKYO_DEV_ADAPTER_FAKE_PROVIDER. They reload live after the same administrator authentication. Production nodes refuse these fields, even when their value is false; changing configuration cannot turn a production node into a development node. Removing an override restores its default while preserving existing admission and budget counters.

Real/fake adapter provider changes require a standalone development node with no retained adapter configuration, pending configuration work or simulated provider contents. Prepare and activation both check this condition. Ordinary reloads preserve the simulated provider state; changing providers never redirects an existing adapter’s work to a different provider.

A missing HA node remains unready and permits configuration repair only until its explicit entry is published and applied.

Changing the external origin’s RP hostname requires a fresh authenticator code and a current password plus confirmed TOTP on the same initiating admin. An old-origin passkey alone cannot authorize that transition. Same-host port changes can use a passkey. This check preserves local login recovery; it does not prove target DNS, TLS or that you have successfully logged in there.

Controlled bootstrap rolloutCopy Anchor Link

HIKYO_BOOTSTRAP_SOURCES selects enrolled external database_source, root_source and upgrade_source aliases in a version-1 document. Explicit singleton topology enrollment also exposes HA mode and node identity. Never put raw root keys, database credentials or arbitrary filesystem paths in these cells. The application needs independently available database/root custody before it can decrypt its own project.

The concrete provider currently targets an explicitly enrolled, singleton, Recreate Deployment on Kubernetes 1.36. The deployment must grant the constrained controller and admission integration authority over its declared resources. Enrollment is an operator installation task. Host, GitOps and multi-replica HA bootstrap providers are not implemented. Unsupported transitions are refused; a prepared plan is not evidence that a provider is installed.

Database aliases must reconnect to the same verified datastore; this is not a data migration facility. Root source changes retain the old recovery wrapper. There is no automatic root finalization.

Preparation remains reviewable for five minutes. Each synchronous request waits at most 30 seconds; stale participant evidence must be refreshed. Final MFA binds the exact plan digest. Only verified external source/template observations and actual runtime acknowledgements establish convergence after replacement.

Independent remotes and HACopy Anchor Link

The root management view groups project references. Each remote owns its organization, project, schema, secrets and generation. Open an owner reference to connect through that instance’s human workspace session. Requests go directly from the browser to the owner. A local administrator or directory credential has no remote configuration authority.

Only HA replicas of the same logical instance share one project and apply. Two independent remotes can configure different values and optional mail settings; their revision numbers are unrelated. Disconnecting a remote from the viewing interface does not remove its project or stop its application. See High availability.

A passkey must run on its owner’s origin. From a remote workspace, use the fresh authenticator code or the Use a passkey on owner link to complete the same workflow directly on the owner.

Status, conflict and rollbackCopy Anchor Link

An apply records one durable target generation. Before that commit, validation failure leaves the old target intact. After commit, a failed node swap remains pending or partial and reconciles to the committed target. Refresh status before retrying an uncertain response. The API’s expected generation refuses concurrent stale decisions; the idempotency key identifies retries of one apply.

For a partial controlled rollout, use Restore deployment inputs. This needs a separate fresh MFA decision bound to the original job and plan. Wait for the verified controller restoration. The desired revision stays unchanged and business operations remain fenced. Publish a repaired revision and apply it with another fresh MFA decision to resume. Restoration never means runtime Applied, and an unresolved external handoff cannot be bypassed by publishing a new target.

To roll back ordinary configuration, open History and rollback, restore the chosen revision into drafts, publish and apply that new revision. Existing unrelated drafts remain visible. History permissions and fresh reauthentication still apply. Required runtime snapshots are retained while active or involved in an apply.

After backup restore, outbound use stays fenced until access reconciliation and explicit configuration resumption. Recovery runs on the affected owner, not the viewing instance. See Backup and restore and Troubleshooting.

Human CLICopy Anchor Link

The existing human CLI supports status, adoption, Apply and test-email:

hikyo instance-config status -o json
hikyo instance-config adopt
hikyo instance-config adopt --yes
hikyo instance-config apply --revision 3 --expected-generation 2
hikyo instance-config test-email --revision 3 --expected-generation 2 --to admin@example.com

Use the UI for the expanded prepare/review and deployment Restore workflow; complete CLI parity for controlled rollouts is not claimed here.

Mutations open the existing browser authorization handoff. It shows the exact owner, action, revision and generation, then returns a single-use decision to the initiating CLI session. Machine credentials cannot adopt, apply or test. Use --idempotency-key to retain the same apply identity when retrying an uncertain result. The --instance flag selects the owning instance.

After a restore, Apply requires confirmation that the restored credentials have been reviewed and access grants reconciled. This confirmation is part of the exact reauthentication decision. The CLI equivalent is --confirm-restored-credentials on instance-config apply; use it only while the owner reports recovery_required.

Recover an invalid applied revision on the hostCopy Anchor Link

If an invalid stored revision prevents normal startup, use local host authority with the same database and root-key configuration as the affected instance:

hikyo admin config status
hikyo admin config recover --revision 3

Stop every replica and wait at least 30 seconds before recover. Recovery refuses live consumers, validates the exact retained published revision against the current catalogue, cancels any stalled apply and records the new target in one transaction. It never accepts arbitrary environment overrides or reads configuration files named in project values. Restart the replicas afterward.

Recovery preserves the outbound fence after a backup restore. Reconcile access and confirm the restored credentials through the normal Apply ceremony before resuming outbound use. These commands have no network endpoint and require access to the server host and its existing root-key custody.

Candidate root sourceCopy Anchor Link

A node override can select HIKYO_NEW_ROOT_SOURCE, the alias of an already enrolled root source. Apply reloads the candidate used by the separate root-rotation operation. Selection does not change the primary root or create or retire wrappers. Clearing the alias disables the candidate and does not restore an old environment file path.

To clear it, remove the optional HIKYO_NEW_ROOT_SOURCE field from that node’s override entry, then publish and apply. An empty string is not a valid alias.

For initial import, HIKYO_NEW_ROOT_KEY_FILE must exactly name the enrolled mounted candidate. An unavailable, malformed or unenrolled candidate refuses preparation and startup. Install enrollment before selecting it, or leave the candidate unset during initial setup.

Enrolled singleton HA mode and node identityCopy Anchor Link

An enrolled Kubernetes Recreate Deployment can opt into topology changes with rollout.topologyNodeIDs. Include its current <fullname>-server identity and the explicitly permitted replacement identities. The Deployment remains at one replica. HA mode on that singleton enables coordination; it does not add redundancy. Ordinary multi-replica HA deployments retain their separate membership setup.

The protected HIKYO_BOOTSTRAP_SOURCES value then includes the actual startup mode and identity, alongside any installed source aliases:

{"version":1,"topology":{"ha":true,"node_id":"replacement-server"}}

Changing topology.ha or topology.node_id requires the existing administrator passkey/TOTP Apply confirmation and a controlled replacement. HA requires the same PostgreSQL datastore and explicit shared root-key authority. For an identity change, publish one complete HIKYO_NODE_OVERRIDES entry under the replacement identity; no other node’s values are inherited. Database, root and upgrade source changes must be applied separately from an actual topology change.

The committed plan retains the old/new identity correspondence and installed template stamp. Old processes cannot refresh heartbeats, claim or renew scheduler work, or serve a newly assigned generation. Ordinary configuration applies retain this fence. Restore only restores external inputs; business and scheduler work remain fenced until a separately reviewed repair is applied. A topology repair uses a fresh controlled replacement even when it returns to the original identity.