search

Private / Encrypted Inference v0 Spec

Status: Design for Testnet Phase #3 (v0 only – dedicated sessions). Scope: How private / encrypted inference works today on dedicated-node sessions, including env config, key issuance, offchain payload v2 format, router tool support, and end-to-end execution flow.

hashtag
1. High-Level Summary

v0 private inference is router-centric and session-scoped, and is only supported on the v2 completion surface:

  • Applies to:

    • POST /api/v2/completion (offchain payload–aware completion endpoint).

    • v1 completion endpoints remain non-encrypted / non-offchain for now.

  • Only dedicated sessions are supported (a router is wired directly to a specific miner or miner group).

  • The router:

    • Owns a secret keyring (ENCRYPTION_SEED + versions).

    • Issues scoped encryption keys via auth endpoints.

    • Encrypts all prompts/results for private sessions.

    • Stores encrypted blobs in an offchain payload v2 store (e.g., S3).

    • Stores only URN references in the Session / SessionQueue records.

  • Miners:

    • See offchain payload v2 objects with explicit encryption metadata.

    • Ask the router for the right key if they are allowed.

    • Decrypt, run inference, re-encrypt the result, and push it back offchain.

  • Apps (router owner / dev):

    • Call /api/v2/completion against private sessions.

    • Receive decrypted final results from the router.

    • Do not see keys; encryption/decryption is handled inside the router.

Because entire payloads are encrypted, streaming is not supported for encrypted v0 workloads.

hashtag
2. Components & Roles

  • Router Node

    • Hosts:

      • Completion endpoint: POST /api/v2/completion.

      • Auth endpoints:

        • POST /api/v1/auth/payload_enc_key/session

        • POST /api/v1/auth/payload_enc_key/task

    • Owns:

      • Encryption keyring (ENCRYPTION_SEED + versioning).

      • ENCRYPTION_ALLOWED_LIST policy.

      • Offchain payload v2 integration (e.g., S3 bucket, URN scheme).

      • Tooling to:

        • Generate strong secrets for env.

        • Rotate keys and backfill encrypted payloads.

  • Dedicated Miner Node

    • Assigned to one or more private sessions.

    • Holds an address (EOA) used to authenticate when requesting keys.

    • Has logic to:

      • Detect encrypted payloads in offchain v2.

      • Call router auth endpoints to get keys.

      • Decrypt, run models, re-encrypt outputs.

  • App / User

    • Talks to router’s /api/v2/completion.

    • Selects which sessions are private (dedicated, encrypted).

    • Treats encrypted sessions as non-streaming.

hashtag
3. v0 Env Configuration

hashtag
3.1 Encryption Seed (Keyring Root)

Basic v0 uses a single env seed as the root of a versioned keyring:

  • ENCRYPTION_SEED – strong, random, secret seed used to derive scoped payload keys.

Example (conceptual):

Keyring model (high-level):

  • Internally, the router should behave as if it has:

    • active_version = vN

    • legacy_versions = [vN-1, vN-2, ...]

  • For v0:

    • Encryption uses active_version.

    • Decryption checks key_version inside the encrypted payload metadata and selects the right seed/version.

    • This requires the router to retain old seed material (decrypt-only) until rotation and backfill are complete.

Key derivation (conceptual):

  • Deterministic, scoped, versioned:

    K(scope, key_version) = HKDF(seed_for_version, info = "cts:v0:" + scope)

Current SHA-256 derivation is acceptable for early v0, but HKDF-style derivation is the target.

hashtag
3.2 Allowed List – v0 Policy

ENCRYPTION_ALLOWED_LIST controls who can request keys and for which scopes.

Supported forms:

  • Global address – can access keys for all sessions/tasks:

  • Session-scoped – address allowed for a specific session:

  • Task-scoped – address allowed for a specific session + task:

  • Combined example – commas inside a scope, semicolons between scopes:

Semantics:

  • 0xAAA...AAA → Address can request any scoped key (any session/task).

  • 101:0xBBB...BBB → Address can request keys for session 101 (session-level scope).

  • 101-9001:0xCCC...CCC → Address can request keys for session 101, task 9001 only.

Env format must have:

  • No spaces.

  • Addresses as 0x + 40 hex characters.

hashtag
4. v0 Auth Endpoints (Key Issuance)

These endpoints are internal to the Router’s private inference design and are called by trusted operators (router owner, miners). They are not general public APIs.

hashtag
4.1 Session-Level Key Issuance

  • Endpoint: POST /api/v1/auth/payload_enc_key/session

Request body (conceptual):

  • address – caller EOA.

  • signature – signature over the canonical scope string.

  • session_id – integer session ID.

Canonical signed message:

  • For session scope:

Example for session_id = 101:

  • Signed string: "101"

Router behavior:

  1. Verify signature matches address over "101".

  2. Check ENCRYPTION_ALLOWED_LIST to see if address is allowed for:

    • Global, or

    • 101, or

    • Any compatible pattern (depending on policy rules).

  3. If authorized, derive payload_enc_key from:

    • Current keyring seed and

    • Scope string ("101"),

    • Associated key_version.

  4. Return:

    • payload_enc_key

    • key_version (e.g., "v3").

hashtag
4.2 Task-Level Key Issuance

  • Endpoint: POST /api/v1/auth/payload_enc_key/task

Request body (conceptual):

  • address

  • signature

  • session_id

  • task_id

Canonical signed message:

  • For task scope:

Example for session_id = 101, task_id = 9001:

  • Signed string: "101:9001"

Router behavior is the same as session-level, but scope is the pair (session_id, task_id) and may use a different policy branch in ENCRYPTION_ALLOWED_LIST.

hashtag
5. Offchain Payload v2 Format (Encrypted vs Plain)

hashtag
5.1 Plain Offchain Payload v2 (Baseline)

For non-encrypted workloads, offchain v2 is conceptually:

  • A JSON document containing:

    • The prompt / input (in plaintext).

    • Optional helper metadata (timestamps, tags, etc.).

The Session / SessionQueue objects store only a URN pointing to this JSON blob in S3 (or another blob store). When a miner resolves the URN, it sees a normal JSON payload with plaintext fields.

hashtag
5.2 Encrypted Offchain v2 (v0 Private Inference)

For encrypted workloads, offchain v2 wraps payloads in an encryption envelope with explicit metadata.

Example envelope (conceptual):

Key points:

  • payload_type = "encrypted" tells miners:

    • This is not plaintext; do not parse ciphertext directly.

  • data.alg and related fields describe:

    • Encryption algorithm.

    • Scope type (session vs task).

    • Scope identifiers.

    • key_version used for derivation.

    • Nonce, tag, ciphertext, and timestamp.

  • The ciphertext contains the entire original prompt JSON (offchain v2 payload body) encrypted under the scoped key.

For plain offchain v2 (non-encrypted), the payload would instead have:

  • payload_type = "plain" (or omitted).

  • A data object containing plaintext fields (e.g., prompt, metadata).

hashtag
6. End-to-End Flow (v0 Dedicated Session)

This describes the full flow for one private completion call on a dedicated session using /api/v2/completion.

hashtag
6.1 One-Time Setup

  1. Router + Miner Pairing

    • Operator configures a dedicated session (e.g., session_id = 101) and assigns it to a specific miner or miner group.

    • Session is marked as:

      • private = true

      • offchain_v2 = true

  2. Router Env

    • Set ENCRYPTION_SEED to a strong random value.

    • Configure ENCRYPTION_ALLOWED_LIST so that:

      • The miner’s EOA is allowed globally or for session 101.

      • The router owner/operator address is allowed as needed.

  3. Miner Config

    • Miner has:

      • A configured EOA used as its identity.

      • Logic to:

        • Detect payload_type = "encrypted" in offchain v2.

        • Call /api/v1/auth/payload_enc_key/* with signed scope strings.

        • Perform AES-GCM decrypt/encrypt and model execution.

hashtag
6.2 App → Router: Private Completion Call

  1. App calls:

    with:

    • session_id = 101

    • Prompt body in plaintext.

    • Implicitly or explicitly flagged as a private session (based on session config).

  2. Router receives plaintext request:

    • Checks session 101 config:

      • Private + encrypted.

      • Offchain v2 enabled.

  3. Router obtains a session-level payload key:

    • Derives from keyring (ENCRYPTION_SEED + active version) and scope "101".

    • Internally equivalent to calling the session-level auth logic.

  4. Router encrypts the prompt:

    • Uses alg = aes-256-gcm.

    • Generates nonce, computes tag, and produces ciphertext.

    • Wraps everything into an offchain v2 envelope with:

      • payload_type = "encrypted".

      • scope_type = "session".

      • session_id = 101.

      • key_version = current key version.

  5. Router uploads encrypted payload to offchain storage (e.g., S3):

    • Stores object keyed by URN, such as:

  6. Router updates Session / SessionQueue:

    • Stores only the URN (not the plaintext prompt) in its internal records.

  7. Router enqueues work for the dedicated miner:

    • Job contains:

      • URN.

      • Session ID / task metadata.

    • No plaintext leaves the router.

hashtag
6.3 Miner: Decrypt, Run, Re-Encrypt

  1. Miner sees a new job for session_id = 101 with a URN.

  2. Miner resolves URN → fetches the offchain v2 blob:

    • Sees:

      • version = "v2"

      • payload_type = "encrypted"

      • Encryption metadata inside data.

  3. Miner decides it needs a scoped key:

    • Scope for session-level:

  4. Miner signs the scope string "101" with its EOA to generate signature.

  5. Miner calls:

    with:

    • address (miner EOA).

    • session_id = 101.

    • signature (over "101").

  6. Router verifies:

    • Signature is valid for address.

    • address is authorized by ENCRYPTION_ALLOWED_LIST.

  7. Router derives payload_enc_key and returns it plus key_version.

  8. Miner uses:

    • payload_enc_key

    • nonce and tag from offchain v2 envelope to decrypt ciphertext and recover the original prompt JSON.

  9. Miner runs inference locally:

    • Uses whatever model/config is attached to session 101.

  10. Miner prepares encrypted result envelope:

    • Encrypts the model output JSON using the same scoped key and key_version.

    • Produces a new offchain v2 object with:

      • payload_type = "encrypted".

      • Updated ciphertext, nonce, tag, created_at.

  11. Miner uploads encrypted result to offchain storage:

    • Gets a result URN for the encrypted output.

  12. Miner reports completion back to the router:

    • Updates Session / job record with the result URN.

hashtag
6.4 Router → App: Decrypt and Return Result

  1. Router sees that the job for session 101 is complete:

    • Reads result URN from Session / SessionQueue.

  2. Router resolves URN → fetches encrypted result envelope.

  3. Router derives or fetches the scoped key:

    • Uses session_id = 101.

    • Reads key_version from the envelope.

    • Uses keyring to get the correct seed/version.

  4. Router decrypts the result:

    • Recovers the plaintext model output JSON.

  5. Router returns plaintext result to the app:

    • /api/v2/completion response is a normal completion payload (no encryption envelope).

To the app, this looks like a normal completion; the privacy path between router and miner is invisible.

hashtag
7. Streaming & Behavior Differences in v0

Because v0 encrypts entire payloads:

  • /api/v2/completion for private sessions is non-streaming:

    • Router buffers full prompt → encrypts → offchain.

    • Router buffers full result → decrypts → returns to caller.

  • v1 completion endpoints are unaffected and behave as before (no offchain v2 + encryption).

  • Logs / telemetry for private sessions must:

    • Avoid logging plaintext prompts/results.

    • Prefer URNs and high-level metadata.

hashtag
8. Quick Reference – v0 Checklist

To run v0 private dedicated sessions safely:

  1. Router

    • Has ENCRYPTION_SEED configured and stored securely.

    • Has ENCRYPTION_ALLOWED_LIST configured:

      • Global dev/operator and/or specific miner addresses.

    • Supports:

      • /api/v2/completion with offchain v2.

      • /api/v1/auth/payload_enc_key/session and /api/v1/auth/payload_enc_key/task.

    • Integrates:

      • Offchain payload v2 store (e.g., S3).

      • URN generation and resolution.

  2. Sessions

    • Are explicitly marked as:

      • Dedicated → bound to specific miner(s).

      • Private → require encryption.

      • Offchain v2 → store payloads in blob store, URNs in DB.

  3. Miners

    • Have an EOA identity.

    • Implement:

      • URN resolution.

      • Encrypted envelope detection and AES-GCM decrypt/encrypt.

      • Auth calls to key issuance endpoints.

    • Never store plaintext payloads in logs.

  4. Apps

    • Use /api/v2/completion against private sessions.

    • Treat them as non-streaming calls.

    • Handle standard plaintext completion responses.

hashtag
9. v0 Tooling for Secrets & Key Rotation

To make v0 operational, the router needs two main tooling paths:

  1. Secret generation tooling (one-time or periodic).

  2. Key rotation + backfill tooling (ongoing operational safety).

All of this applies specifically to /api/v2/completion and offchain v2 encrypted workloads.

hashtag
9.1 Secret Generation Tool (Router Operator Utility)

Goal:

  • Easily generate a strong ENCRYPTION_SEED and write it into router env/config.

Expected behavior:

  • CLI (example shape):

  • Responsibilities:

    • Generate cryptographically strong random bytes for ENCRYPTION_SEED.

    • Encode as safe ASCII (e.g., hex or base64).

    • Write or update env/config file (or secrets backend).

    • Optionally print:

      • Seed fingerprint (e.g., first 8 bytes of hash) for ops verification.

    • Never log the full seed to stdout in production mode.

Notes:

  • Only run on fresh environments or during a controlled rotation flow.

  • Should verify that:

    • The file permissions for config are appropriate.

    • The router will reload or be restarted after the change.

hashtag
9.2 Key Rotation & Backfill Tool (Router-Side)

Goal:

  • Rotate ENCRYPTION_SEED and re-encrypt all router-owned, dedicated private sessions without breaking references (URNs stay valid).

Constraints:

  • Offchain v2 objects must be re-encrypted in place or with new blobs under the same URN (or under a new URN plus a pointer update).

  • Old keys must remain available in decrypt-only mode until migration is complete.

Conceptual flow:

  1. Preparation

    • Generate a new seed (via the Secret Generation Tool).

    • Add it to the keyring as:

      • key_version = vN+1 (active).

    • Mark old seeds (vN, vN-1, etc.) as decrypt-only.

  2. Session Enumeration

    • Tool scans router storage to find:

      • All sessions configured as:

        • Private = true.

        • Offchain v2 = true.

        • Dedicated node sessions owned by this router.

    • For each such session, tool enumerates:

      • Prompt payload URNs.

      • Result payload URNs (if needed).

  3. Backfill Pass (Re-encrypt)

    • For each URN:

      • Fetch current offchain v2 envelope.

      • Read key_version (old version).

      • Use keyring to get the right old key.

      • Decrypt ciphertext into plaintext JSON.

      • Re-encrypt plaintext JSON with new active key version.

      • Produce a new envelope with:

        • Same structure.

        • Updated key_version = active version.

        • New nonce and tag.

      • Write back:

        • Either:

          • Overwrite existing blob at the same URN, or

          • Write a new blob and update router’s internal references to point to the new URN.

    • Maintain audit logs:

      • URN processed.

      • Old key_version.

      • New key_version.

      • Success/failure status.

  4. Verification

    • For a sample of URNs:

      • Run a test decrypt path using the active key version.

      • Confirm that:

        • Decryption succeeds.

        • The payload is structurally valid (basic JSON checks).

    • Optionally:

      • Run a dry-run mode that:

        • Decrypts and re-encrypts in memory only.

        • Does not write changes, just validates key readiness.

  5. Finalize

    • Once all dedicated private session payloads are re-encrypted:

      • Mark old key versions as:

        • “decrypt-only allowed for grace period” or

        • Fully retired (if no old payloads remain).

    • Remove old seeds from keyring when safe.

CLI sketch (example):

Behavior notes:

  • Scope:

    • In v0, this tool focuses only on:

      • Dedicated node sessions.

      • Router-owned offchain v2 URNs.

    • Shared pools and ephemeral sessions come later (v1+).

  • Safety:

    • Never delete old blobs or seeds until:

      • Backfill has completed,

      • Verification passed,

      • A grace window has elapsed.

  • Performance:

    • Should be resumable:

      • Track progress per URN.

      • Allow stopping and continuing without re-processing successfully migrated payloads.

hashtag
9.3 Router Behavior During Rotation

While rotation/backfill is in progress:

  • Decryption path:

    • Must look at key_version inside the encrypted envelope.

    • Select the appropriate seed from keyring.

    • Support:

      • Both old and new versions for the migration period.

  • Encryption path (for new calls):

    • Must always use the active key version only.

This ensures that:

  • New traffic is always protected under the newest key version.

  • Legacy data is gradually migrated without downtime.

This v0 detail document, plus the separate privacy roadmap (v0 → v3/v4 + TEEs), together define how Cortensor introduces practical, operational private inference on dedicated sessions:

  • Router and miners share scoped, deterministic keys via auth endpoints.

  • Payloads move offchain in encrypted v2 envelopes.

  • /api/v2/completion becomes the primary encrypted workload entry.

  • Router tools manage:

    • Strong secret generation.

    • Safe key rotation and backfill over time, without breaking URNs or session history.

PreviousPrivate / Encrypted InferenceNextv0.5 – Contract-Backed ACL for Dedicated Private Sessions (Session-Based Only)

Last updated