Testing

How do I test an Orkestra operator?

Two tools. One for fast iteration, one for production confidence:

ork simulate — runs the reconciler against a fake in-memory cluster. No Kubernetes. No Docker. Sub-second. Declare what your operator should produce per cycle, get a pass or fail. The same reconciler code that runs in production runs here.

ork e2e — runs the operator against a real kind cluster. Creates the cluster, applies CRDs and the operator, applies your CR, asserts Kubernetes resource state, tears down. Full integration truth.

The pattern: simulate in your inner loop, e2e as your pre-deploy gate.

What is the difference between ork simulate and ork e2e?

They are not alternatives. Simulate is your inner loop. E2E is your pre-deploy gate. Use both.

How do I generate a simulate.yaml?

ork simulate init runs the reconciler once, captures the cycle-1 create operations, and writes a simulate.yaml pre-filled with expect: rules:

ork simulate init           # writes simulate.yaml in the current directory
ork simulate init --dry-run # preview: prints to stdout, nothing written
ork simulate init --force   # overwrite existing simulate.yaml

The generated file includes a commented absent: block seeded with the first observed resource type — a starting point for failure-path assertions you can fill in.

→ Scaffolding tests with simulate init

What does skipExternal: true do?

It stubs all external: HTTP calls with an empty 200 response instead of hitting the real network. Useful when the external service is not available in your simulate environment and you only care about the structural output — which resources are created — not the content shaped by the external response.

spec:
  katalog: ./katalog.yaml
  cr: ./cr.yaml
  skipExternal: true

If you need the external calls to return meaningful responses — for example, a feature flag that controls whether a resource is created — use --dev-server instead.

What is ork simulate --dev-server?

ork simulate --dev-server starts the Orkestra mock HTTP server at localhost:9999 before running simulate. The dev server responds to the same endpoints used in the learning examples:

• GET /health — 200 healthy
• GET /config/:name — static JSON config blob
• GET /flags/:name/:flag — feature flag value
• GET /status/200, GET /status/503 — explicit status codes

This lets examples that call external services (config injection, feature flags, health gates) get realistic responses during simulate — no real network, no cluster required.

--skip-external stubs calls with empty 200s. --dev-server gives them real responses. Use --dev-server when the response content affects what resources your operator creates.

→ simulate CLI reference

Can I run simulate for multiple operators at once?

Yes — with an aggregator. A simulate.yaml with imports: and no spec: runs each imported file independently:

apiVersion: orkestra.orkspace.io/v1
kind: Simulate
metadata:
  name: platform-sim

imports:
  - ./database/simulate.yaml
  - ./cache/simulate.yaml
  - ./api/simulate.yaml

Each file runs independently. The suite fails if any file fails.

To run all simulate files in a directory tree:

ork simulate ./...

→ Aggregator mode

Does simulate run the real reconciler?

Yes. ork simulate uses the same reconciler pipeline that runs against a live cluster — the same template evaluation, the same when: conditions, the same external: calls, the same hooks. The only difference is the backing store: instead of etcd and the Kubernetes API, simulate uses an in-memory fake client.

If a test passes in simulate, the reconciler logic is correct. What you cannot verify in simulate: admission webhook behavior end-to-end, network policies, actual pod scheduling, and anything that depends on Kubernetes controllers running (e.g. ReplicaSet → Pod creation). That is what ork e2e is for.