simple-private-codecollection/docs/07-validate-and-observe.md

07 — Validate and observe

What you'll do

Confirm end-to-end that:

1. The generation rule is schema-valid.
2. Workspace-builder clones the private CC from Gitea.
3. One SLX is rendered per Crossplane Bucket in the cluster.
4. The RunWhen Platform receives the SLXs and the SLI + TaskSet run.

Step 1 — Local schema validation

If you have runwhen-local checked out, you can validate before pushing:

cd /path/to/runwhen-local/src
python3 validate_generation_rules.py /path/to/simple-private-codecollection -v

Expected:

No validation errors found.

Or a one-liner without cloning runwhen-local:

python3 -c "
import json, yaml, jsonschema, urllib.request
schema = json.loads(urllib.request.urlopen(
  'https://raw.githubusercontent.com/runwhen-contrib/runwhen-local/main/src/generation-rule-schema.json'
).read())
with open('codebundles/gcp-bucket-crossplane-health/.runwhen/generation-rules/gcp-bucket-crossplane-health.yaml') as f:
  jsonschema.validate(yaml.safe_load(f), schema)
print('OK')
"

Screenshot placeholder: images/07-schema-validation.png.

Step 2 — Push to Gitea

git add .
git commit -m "Initial codecollection: Crossplane GCP Bucket rules-only example"
git push -u origin main

Refresh the Gitea repo URL in a browser — you should see the file tree.

Screenshot placeholder: images/07-gitea-repo.png — Gitea file browser showing codebundles/ and docs/.

Step 3 — Reconcile Flux

flux reconcile hr runwhen-local -n runwhen-env-airgap-runner --with-source

Step 4 — Watch workspace-builder logs

kubectl -n runwhen-env-airgap-runner logs \
  -l runwhen.com/component=workspace-builder \
  -c workspace-builder --tail=200 -f

Look for lines like:

• Cloning from git source: https://.../simple-private-codecollection.git
• Loading generation rules from ...gcp-bucket-crossplane-health.yaml
• Rendered N SLXs (where N is the number of Buckets)

Common errors and their meaning:

Error	Cause
fatal: Authentication failed on clone	Token in URL missing, expired, or lacks read scope
KeyError: 'match_resource' in a template	Jinja variable typo, or trying to use a variable that isn't populated for the platform
Validation failed for ... generation rule ... required "outputItems"	Schema drift — check the JSON schema
No matches for resource type buckets.storage.gcp.upbound.io/v1beta1	The CRD isn't installed on this cluster, or the plural is wrong
Forbidden ... cannot list buckets.storage.gcp.upbound.io	Missing RBAC for workspace-builder SA

Screenshot placeholder: images/07-workspace-builder-logs.png.

Step 5 — Inspect rendered SLXs in the runner filesystem

workspace-builder writes the rendered artifacts to disk before uploading them. Peek at one:

kubectl -n runwhen-env-airgap-runner exec deploy/runwhen-local \
  -c workspace-builder -- \
  find /shared/output -type f | grep xp-bkt-hlth | head -5

Then read one:

kubectl -n runwhen-env-airgap-runner exec deploy/runwhen-local \
  -c workspace-builder -- \
  cat /shared/output/output/slx/<name>/sli.yaml

Confirm the codeBundle.repoUrl points at the internal rw-generic-codecollection proxy and the KUBECTL_COMMAND includes the bucket name.

Step 6 — See it in the RunWhen UI

Sign into the RunWhen workspace UI: https://app.airgap.shared.runwhen.com/map/self

You should see one SLX per Bucket under the shared-cluster hierarchy. Click into one — the SLI panel should already be graphing a value near 1.0 (assuming buckets are healthy). Trigger the TaskSet from the SLX detail page and confirm the report includes Command stdout showing the JSON envelope, and any non-True conditions become issues.

Screenshot placeholders:

• images/07-runwhen-map.png — the map view with the new SLXs visible.
• images/07-sli-graph.png — the SLI panel graphing 1.0.
• images/07-taskset-report.png — the TaskSet execution report.
• images/07-issue-detail.png — a rendered issue (deliberately break a Bucket by editing the Crossplane spec to force a Synced=False condition, so we can capture what an unhealthy report looks like).

Iteration checklist

• Iteration 1: private CC scaffolded, cloned by workspace-builder, generation rule loaded, CRD discovery attempted. Blocked on upstream runwhen-local bugs — see 08 — Current limitations.
• Iteration 2: fix upstream runwhen-local bugs (import + scope-aware CRD listing), rebuild image, re-run and capture screenshots.
• Iteration 3: prose polish, .test/ harness for local runwhen-local container validation, migration to Option A (cc-catalog proxy) for production hygiene.

Known limitations discovered during validation

• The workspace-builder image 0.11.0 cannot list cluster-scoped CRDs. Our Crossplane Bucket CRD is cluster-scoped, so no SLXs render yet. Full analysis and proposed patches are in 08 — Current limitations.
• The pattern (rules + templates only, delegate runtime to rw-generic-codecollection) is otherwise fully proven end-to-end.

Where to file feedback

If something in these docs is confusing or wrong, edit the markdown directly and push. This is a living training guide — improvements are expected.