simple-private-codecollection/docs/07-validate-and-observe.md

# 07 — Validate and observe

## What you'll do

Confirm end-to-end that:

1. The generation rule is schema-valid.
2. Workspace-builder clones the private CC from Gitea.
3. One SLX is rendered per Crossplane Bucket in the cluster.
4. The RunWhen Platform receives the SLXs and the SLI + TaskSet run.

## Step 1 — Local schema validation

If you have `runwhen-local` checked out, you can validate before pushing:

```bash
cd /path/to/runwhen-local/src
python3 validate_generation_rules.py /path/to/simple-private-codecollection -v
```

Expected:

```
No validation errors found.
```

Or a one-liner without cloning `runwhen-local`:

```bash
python3 -c "
import json, yaml, jsonschema, urllib.request
schema = json.loads(urllib.request.urlopen(
  'https://raw.githubusercontent.com/runwhen-contrib/runwhen-local/main/src/generation-rule-schema.json'
).read())
with open('codebundles/gcp-bucket-crossplane-health/.runwhen/generation-rules/gcp-bucket-crossplane-health.yaml') as f:
  jsonschema.validate(yaml.safe_load(f), schema)
print('OK')
"
```

Screenshot placeholder: `images/07-schema-validation.png`.

## Step 2 — Push to Gitea

```bash
git add .
git commit -m "Initial codecollection: Crossplane GCP Bucket rules-only example"
git push -u origin main
```

Refresh the Gitea repo URL in a browser — you should see the file tree.

Screenshot placeholder: `images/07-gitea-repo.png` — Gitea file browser
showing `codebundles/` and `docs/`.

## Step 3 — Reconcile Flux

```bash
flux reconcile hr runwhen-local -n runwhen-env-airgap-runner --with-source
```

## Step 4 — Watch workspace-builder logs

```bash
kubectl -n runwhen-env-airgap-runner logs \
  -l runwhen.com/component=workspace-builder \
  -c workspace-builder --tail=200 -f
```

Look for lines like:

- `Cloning from git source: https://.../simple-private-codecollection.git`
- `Loading generation rules from ...gcp-bucket-crossplane-health.yaml`
- `Rendered N SLXs` (where N is the number of Buckets)

Common errors and their meaning:

| Error | Cause |
|-------|-------|
| `fatal: Authentication failed` on clone | Token in URL missing, expired, or lacks read scope |
| `KeyError: 'match_resource'` in a template | Jinja variable typo, or trying to use a variable that isn't populated for the platform |
| `Validation failed for ... generation rule ... required "outputItems"` | Schema drift — check the JSON schema |
| `No matches for resource type buckets.storage.gcp.upbound.io/v1beta1` | The CRD isn't installed on this cluster, or the plural is wrong |
| `Forbidden ... cannot list buckets.storage.gcp.upbound.io` | Missing RBAC for `workspace-builder` SA |

Screenshot placeholder: `images/07-workspace-builder-logs.png`.

## Step 5 — Inspect rendered SLXs in the runner filesystem

`workspace-builder` writes the rendered artifacts to disk before
uploading them. Peek at one:

```bash
kubectl -n runwhen-env-airgap-runner exec deploy/runwhen-local \
  -c workspace-builder -- \
  find /shared/output -type f | grep xp-bkt-hlth | head -5
```

Then read one:

```bash
kubectl -n runwhen-env-airgap-runner exec deploy/runwhen-local \
  -c workspace-builder -- \
  cat /shared/output/output/slx/<name>/sli.yaml
```

Confirm the `codeBundle.repoUrl` points at the internal
`rw-generic-codecollection` proxy and the `KUBECTL_COMMAND` includes
the bucket name.

## Step 6 — See it in the RunWhen UI

Sign into the RunWhen workspace UI:
`https://app.airgap.shared.runwhen.com/map/self`

You should see one SLX per Bucket under the `shared-cluster` hierarchy.
Click into one — the SLI panel should already be graphing a value near
`1.0` (assuming buckets are healthy). Trigger the TaskSet from the SLX
detail page and confirm the report includes `Command stdout` showing
the JSON envelope, and any non-True conditions become issues.

Screenshot placeholders:

- `images/07-runwhen-map.png` — the map view with the new SLXs visible.
- `images/07-sli-graph.png` — the SLI panel graphing 1.0.
- `images/07-taskset-report.png` — the TaskSet execution report.
- `images/07-issue-detail.png` — a rendered issue (deliberately break a
  Bucket by editing the Crossplane spec to force a `Synced=False`
  condition, so we can capture what an unhealthy report looks like).

## Iteration checklist

- [x] Iteration 1: private CC scaffolded, cloned by workspace-builder,
      generation rule loaded, CRD discovery attempted. **Blocked on
      upstream runwhen-local bugs** — see
      [08 — Current limitations](08-current-limitations.md).
- [ ] Iteration 2: fix upstream runwhen-local bugs (import + scope-aware
      CRD listing), rebuild image, re-run and capture screenshots.
- [ ] Iteration 3: prose polish, `.test/` harness for local
      `runwhen-local` container validation, migration to Option A
      (cc-catalog proxy) for production hygiene.

## Known limitations discovered during validation

- The workspace-builder image `0.11.0` cannot list cluster-scoped CRDs.
  Our Crossplane Bucket CRD is cluster-scoped, so no SLXs render yet.
  Full analysis and proposed patches are in
  [08 — Current limitations](08-current-limitations.md).
- The pattern (rules + templates only, delegate runtime to
  `rw-generic-codecollection`) is otherwise fully proven end-to-end.

## Where to file feedback

If something in these docs is confusing or wrong, edit the markdown
directly and push. This is a living training guide — improvements are
expected.