HOWTO – Repair an EVE-NG IOL Licence Binding¶

Recover an authorised IOL licence after an existing EVE-NG host is repaired or reconstructed with a different host identity. This path is not an initial licence-provisioning mechanism.

Difficulty: Advanced

Prerequisites: A previously successful IOL image deployment, an existing EVENG_IOL_LICENSE in the selected runtime vault, permission to update the selected secret authority, and either access to the authorised repair broker or approval to use the manual break-glass bundle described below.

Security and scope¶

HybridOps.Core does not contain, download, cache, or execute an IOL licence generator. An operator-controlled HTTPS broker generates authorised repair material server-side and returns a signed, host-bound response.

The repair path runs only when all of these conditions are true:

hyops blueprint deploy was explicitly given --repair-iol-license;
the failing module is platform/linux/eve-ng-images;
the module reports the structured hostname and eight-digit host ID mismatch;
its existing state still contains eveng_images_iol_license_ready: true from a previous successful deployment;
the runtime vault already contains EVENG_IOL_LICENSE; and
a broker URL, bearer token, and pinned response-verification key are present.

The broker must independently enforce tenant and environment authorization. Core's client-side gates prevent accidental use but cannot constrain a modified client.

Configure the operator environment¶

Store HYOPS_IOL_REPAIR_TOKEN in the environment runtime vault or export it only for the repair session. Configure the HTTPS endpoint and the broker's Ed25519 public key:

export HYOPS_IOL_REPAIR_BROKER_URL=https://repair.example.net/v1/iol
export HYOPS_IOL_REPAIR_PUBLIC_KEY=/etc/hybridops/iol-repair-ed25519.pub.pem

For a broker using a private certificate authority, also set:

export HYOPS_IOL_REPAIR_CA_BUNDLE=/etc/hybridops/repair-broker-ca.pem

Do not place the bearer token, response-verification key, iourc content, or licence value in blueprint YAML or shell history.

Confirm repair eligibility¶

Inspect the image module state for the selected environment:

hyops show module \
  platform/linux/eve-ng-images#images \
  --env <env>

Confirm that the preserved outputs contain:

eveng_images_iol_license_ready: true

Confirm the existing secret key is present without printing its value:

hyops secrets keys --env <env> | grep '^EVENG_IOL_LICENSE$'

If either check fails, stop. Use the normal authorised initial provisioning procedure instead of repair recovery.

Run the controlled repair¶

Run the same blueprint deployment with repair explicitly enabled:

hyops blueprint deploy \
  --env <env> \
  --ref gcp/eve-ng@v1 \
  --execute \
  --repair-iol-license

For a Proxmox-hosted environment, replace the reference with onprem/eve-ng@v1.

The repaired value is always written to the encrypted runtime vault. To update an external authority in the same operation, add one of:

--iol-repair-persist gsm
--iol-repair-persist vault

Core validates the failure type, prior readiness, original secret presence, HTTPS endpoint, Ed25519 response signature, request nonce, hostname, host ID, five-minute maximum expiry, and exact iourc binding. It then writes through the existing secrets API, removes the private temporary plaintext directory, and retries the failed image module exactly once.

Manual break-glass alternative¶

Use this path only when the automated broker is unavailable and the operator is authorised to repair the existing IOL deployment. It runs a standalone utility outside HybridOps.Core. It must not be used for initial licence provisioning or for software and images the operator is not authorised to run.

The download URL and its decryption fragment act as a bearer capability: anyone who receives the complete URL can retain the bundle. Download-and-delete is therefore an exposure-reduction measure, not an access-control mechanism. The broker path remains preferred because it can enforce identity, single-use authorization, expiry, rate limits, and audit logging server-side.

Download and verify the bundle¶

Download ioukeygen.tar.gz through the browser into the operator's download directory. Before extraction, verify the archive against the Academy-published digest:

SHA-256: 8cb8beb5f39b73c2dc61433fcdc56beb60af901ab3e1cf3285e0b0df5353e36d

Linux:

sha256sum ~/Downloads/ioukeygen.tar.gz

macOS:

shasum -a 256 ~/Downloads/ioukeygen.tar.gz

Stop if the digest does not match exactly. Do not extract or execute an unverified copy.

Extract into a private working directory¶

umask 077
mkdir ./ioukeygen-repair
tar -xzf ~/Downloads/ioukeygen.tar.gz -C ./ioukeygen-repair
cd ./ioukeygen-repair

The extracted directory must contain only:

README.md
ioukeygen.py

Review the bundled README before execution. The utility does not update ~/.iourc, modify /etc/hosts, or persist its inputs. Its default output is iourc.txt in this extracted directory.

Obtain the repaired host identity¶

Use the hostname and eight-digit host ID reported by the failed HybridOps run. If direct verification is required, run these commands on the repaired EVE-NG host:

hostname
hostid

The hostname must exactly match the name HybridOps uses for the target host. Do not substitute the previous host identity.

Generate and import the repaired iourc¶

Run the prompted mode and enter the repaired host ID followed by its hostname:

python3 ioukeygen.py --prompt

Confirm that iourc.txt was created beside the script, then import it without printing its contents:

hyops secrets set \
  --env <env> \
  --from-file EVENG_IOL_LICENSE=./iourc.txt

When an external secret authority is configured, persist it in the same operation with either --persist gsm or --persist vault. Do not paste the licence into YAML, shell history, tickets, chat, or run-record notes.

Rerun the original blueprint normally, without --repair-iol-license:

hyops blueprint deploy \
  --env <env> \
  --ref gcp/eve-ng@v1 \
  --execute

Use onprem/eve-ng@v1 for a Proxmox-hosted environment.

Clean up the manual bundle¶

After the encrypted secret write and successful deployment verification, remove the generated plaintext, extracted utility, and downloaded archive:

cd ..
rm -f ./ioukeygen-repair/iourc.txt
rm -f ./ioukeygen-repair/ioukeygen.py ./ioukeygen-repair/README.md
rmdir ./ioukeygen-repair
rm -f ~/Downloads/ioukeygen.tar.gz

Do not move these files to a synchronized folder or a desktop trash location. On SSDs and copy-on-write filesystems, deletion is not guaranteed secure erase; use an encrypted operator workstation and a private working directory from the start.

Verify the result¶

The deployment output should record an IOL repair with status: ok. In JSON mode the non-secret evidence appears under iol_license_repairs and includes the step ID, host identity, broker request ID, and persistence target.

Verify the module state:

hyops show module \
  platform/linux/eve-ng-images#images \
  --env <env>

Confirm:

module status is ok;
eveng_images_iol_license_ready is true;
the requested IOL image count is correct; and
an authorised IOL node starts successfully on the repaired host.

Do not print EVENG_IOL_LICENSE during verification. Retain the Core run record and broker request ID as the audit trail.

Failure handling¶

A missing readiness marker or original secret is an intentional hard stop.
A broker rejection must be resolved at the broker; do not bypass the client eligibility checks.
A signature, nonce, host, or expiry error indicates an invalid response and must not be retried until investigated.
If external persistence fails after the encrypted runtime vault was updated, reconcile the external authority before another deployment.
If the single module retry fails, inspect the new run record. Core will not enter a repair loop.

Broker contract¶

Core sends an authenticated JSON POST using schema hybridops/iol-license-repair/v1. The request contains the environment, module state reference, hostname, host ID, a random nonce, the previous readiness assertion, and only a SHA-256 digest of the old iourc. The old licence is never sent.

The broker returns HTTP 200 with a JSON body shaped as follows:

{
  "schema": "hybridops/iol-license-repair/v1",
  "request_id": "auditable-unique-id",
  "nonce": "the-request-nonce",
  "hostname": "eve-ng-01",
  "host_id": "500a0232",
  "expires_at": "2026-08-22T12:02:00Z",
  "iourc": "[license]\neve-ng-01 = 0123456789abcdef;\n"
}

Sign the exact response bytes with Ed25519 and place the base64 signature in X-HybridOps-Signature. The response expiry must be no more than five minutes in the future.

The broker must enforce single-use authorization, rate limits, audit logging, tenant/environment binding, and its own proof of repair eligibility. It should never distribute the generator or expose a reusable generation endpoint.