From eb8160e659173dce80d1aeb7ca9beb4c135f82f7 Mon Sep 17 00:00:00 2001 From: stewartshea Date: Tue, 30 Jun 2026 21:51:04 -0400 Subject: [PATCH] Initial scaffold: generation-rule-only Crossplane Bucket example Educational RunWhen CodeCollection that discovers Crossplane GCP Bucket CRDs (storage.gcp.upbound.io/v1beta1) and generates one SLX per bucket. Ships only generation rules and Jinja templates; the runtime lives in rw-generic-codecollection/k8s-kubectl-cmd (already loaded by the airgap runner). Includes: - codebundles/gcp-bucket-crossplane-health with generation rule + 3 templates - docs/01..07 numbered training chapters with screenshot placeholders - README, .gitignore Co-authored-by: Cursor --- .gitignore | 25 +++ README.md | 56 ++++++ .../gcp-bucket-crossplane-health.yaml | 34 ++++ .../gcp-bucket-crossplane-health-sli.yaml | 50 +++++ .../gcp-bucket-crossplane-health-slx.yaml | 36 ++++ .../gcp-bucket-crossplane-health-taskset.yaml | 47 +++++ .../gcp-bucket-crossplane-health/README.md | 49 +++++ docs/01-overview.md | 75 ++++++++ docs/02-generation-rules-only-pattern.md | 124 ++++++++++++ docs/03-crossplane-crd-discovery.md | 139 ++++++++++++++ docs/04-generation-rule-walkthrough.md | 165 ++++++++++++++++ docs/05-slx-and-templates.md | 178 ++++++++++++++++++ docs/06-runner-integration.md | 120 ++++++++++++ docs/07-validate-and-observe.md | 144 ++++++++++++++ docs/images/.gitkeep | 2 + 15 files changed, 1244 insertions(+) create mode 100644 .gitignore create mode 100644 README.md create mode 100644 codebundles/gcp-bucket-crossplane-health/.runwhen/generation-rules/gcp-bucket-crossplane-health.yaml create mode 100644 codebundles/gcp-bucket-crossplane-health/.runwhen/templates/gcp-bucket-crossplane-health-sli.yaml create mode 100644 codebundles/gcp-bucket-crossplane-health/.runwhen/templates/gcp-bucket-crossplane-health-slx.yaml create mode 100644 codebundles/gcp-bucket-crossplane-health/.runwhen/templates/gcp-bucket-crossplane-health-taskset.yaml create mode 100644 codebundles/gcp-bucket-crossplane-health/README.md create mode 100644 docs/01-overview.md create mode 100644 docs/02-generation-rules-only-pattern.md create mode 100644 docs/03-crossplane-crd-discovery.md create mode 100644 docs/04-generation-rule-walkthrough.md create mode 100644 docs/05-slx-and-templates.md create mode 100644 docs/06-runner-integration.md create mode 100644 docs/07-validate-and-observe.md create mode 100644 docs/images/.gitkeep diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..ad621c2 --- /dev/null +++ b/.gitignore @@ -0,0 +1,25 @@ +# Python +__pycache__/ +*.py[cod] +*.egg-info/ +.venv/ + +# Robot Framework artifacts +output.xml +log.html +report.html + +# Workspace-builder / runwhen-local rendered output +output/ +.cnb-cache/ +.rwl-cache/ + +# Editors / OS +.DS_Store +.vscode/ +.idea/ +*.swp + +# Local secrets or scratch +*.local +*.local.yaml diff --git a/README.md b/README.md new file mode 100644 index 0000000..9d9e04a --- /dev/null +++ b/README.md @@ -0,0 +1,56 @@ +# simple-private-codecollection + +An **educational, generation-rule-only** RunWhen CodeCollection. It ships **no** +Robot Framework code of its own — instead it teaches how a private repository +can define **discovery rules** and **SLX templates** that reuse a generic +runtime already loaded by the RunWhen runner (in this case +[`k8s-kubectl-cmd`](https://github.com/runwhen-contrib/rw-generic-codecollection/tree/main/codebundles/k8s-kubectl-cmd) +from `rw-generic-codecollection`). + +## What it demonstrates + +- How **workspace-builder** discovers a Kubernetes **Custom Resource** (a + Crossplane `Bucket` from the Upbound GCP provider) via a generation rule. +- How the resulting SLX points its `codeBundle.repoUrl` at a **different** + code collection where the runtime actually lives. +- How to compute a fractional **SLI** (0.0–1.0) from + `.status.conditions[]` with `jq`, and how to emit a **detailed + TaskSet** with per-condition issues via `RW.DynamicIssues`. + +## Layout + +``` +codebundles/ + gcp-bucket-crossplane-health/ # rules + templates only, no *.robot + .runwhen/ + generation-rules/gcp-bucket-crossplane-health.yaml + templates/ + gcp-bucket-crossplane-health-slx.yaml + gcp-bucket-crossplane-health-sli.yaml + gcp-bucket-crossplane-health-taskset.yaml +docs/ + 01-overview.md + 02-generation-rules-only-pattern.md + 03-crossplane-crd-discovery.md + 04-generation-rule-walkthrough.md + 05-slx-and-templates.md + 06-runner-integration.md + 07-validate-and-observe.md + images/ # screenshots referenced from the chapters +``` + +## Start here + +Open [`docs/01-overview.md`](docs/01-overview.md) and read the chapters in +order. Each chapter has a short "What you'll do" preamble, an inline diagram +or code snippet, and screenshot placeholders that we fill in as we iterate. + +## Target environment + +This training example is wired for the RunWhen **airgap** environment (in-cluster +workspace-builder in `runwhen-env-airgap-runner`, Gitea at +`gitea.airgap.shared.runwhen.com`). The generation rule targets real Crossplane +`Bucket` resources already present in that cluster (see +[llm-storage.yaml](https://github.com/runwhen/infra-flux-nonprod-shared/blob/main/infrastructure/crossplane/llm-storage/llm-storage.yaml) +and similar). The pattern applies to any RunWhen environment; only the +`repoUrl` in the templates needs to change. diff --git a/codebundles/gcp-bucket-crossplane-health/.runwhen/generation-rules/gcp-bucket-crossplane-health.yaml b/codebundles/gcp-bucket-crossplane-health/.runwhen/generation-rules/gcp-bucket-crossplane-health.yaml new file mode 100644 index 0000000..efb135a --- /dev/null +++ b/codebundles/gcp-bucket-crossplane-health/.runwhen/generation-rules/gcp-bucket-crossplane-health.yaml @@ -0,0 +1,34 @@ +apiVersion: runwhen.com/v1 +kind: GenerationRules +spec: + platform: kubernetes + generationRules: + # Discover every Crossplane GCP Bucket in the cluster. + # Resource type syntax: plural.group/version + # plural = buckets (from `kubectl api-resources | grep storage.gcp.upbound.io`) + # group = storage.gcp.upbound.io + # version = v1beta1 (omit /version to use the API server's preferred version) + - resourceTypes: + - buckets.storage.gcp.upbound.io/v1beta1 + matchRules: + # Match every Bucket by name. Swap this regex to scope by naming + # convention (for example `^prod-.*` to only cover prod buckets) or + # use `type: exists` on a label like `metadata/labels/env`. + - type: pattern + pattern: ".+" + properties: [name] + mode: substring + slxs: + # Crossplane Buckets are cluster-scoped, so we qualify by `resource` + # (the bucket name) and `cluster` only — there is no namespace. + - baseName: xp-bkt-hlth + qualifiers: ["resource", "cluster"] + baseTemplateName: gcp-bucket-crossplane-health + levelOfDetail: detailed + outputItems: + - type: slx + - type: sli + - type: runbook + # Runbook uses the *-taskset.yaml template convention because + # RunWhen historically calls a Runbook a "TaskSet" in the UI. + templateName: gcp-bucket-crossplane-health-taskset.yaml diff --git a/codebundles/gcp-bucket-crossplane-health/.runwhen/templates/gcp-bucket-crossplane-health-sli.yaml b/codebundles/gcp-bucket-crossplane-health/.runwhen/templates/gcp-bucket-crossplane-health-sli.yaml new file mode 100644 index 0000000..c2cd90b --- /dev/null +++ b/codebundles/gcp-bucket-crossplane-health/.runwhen/templates/gcp-bucket-crossplane-health-sli.yaml @@ -0,0 +1,50 @@ +apiVersion: runwhen.com/v1 +kind: ServiceLevelIndicator +metadata: + name: {{slx_name}} + labels: + {% include "common-labels.yaml" %} + annotations: + {% include "common-annotations.yaml" %} +spec: + displayUnitsLong: Percentage + displayUnitsShort: '%' + locations: + - {{default_location}} + description: >- + Fraction of Crossplane .status.conditions[] that are True for + GCP Bucket {{match_resource.resource.metadata.name}}. Healthy = 1.0. + # NOTE: This SLI runs from a *different* codecollection than the one + # that owns this generation rule. The generic runtime (`k8s-kubectl-cmd`) + # lives in `rw-generic-codecollection` — it is already loaded by the + # airgap runner. If you move this example to another environment, + # change `repoUrl` to wherever `rw-generic-codecollection` is served + # (public GitHub, an internal proxy, a Gitea mirror, etc.). + codeBundle: + repoUrl: http://rw-airgap-cc-catalog-svc.runwhen-env-airgap:8080/git/rw-generic-codecollection.git + ref: main + pathToRobot: codebundles/k8s-kubectl-cmd/sli.robot + intervalStrategy: intermezzo + intervalSeconds: 300 + configProvided: + - name: TASK_TITLE + value: 'Crossplane GCP Bucket {{match_resource.resource.metadata.name}} condition health' + # jq computes truthyConditions / totalConditions. Returns 0 when the + # resource has never reconciled (no conditions yet), which correctly + # signals "unhealthy" to the SLO layer. + - name: KUBECTL_COMMAND + value: | + kubectl get buckets.storage.gcp.upbound.io {{match_resource.resource.metadata.name}} -o json | jq -r '(.status.conditions // []) as $c | if ($c|length)==0 then 0 else ([$c[]|select(.status=="True")]|length)/($c|length) end' + - name: TIMEOUT_SECONDS + value: '120' + secretsProvided: + {% if wb_version %} + {% include "kubernetes-auth.yaml" ignore missing %} + {% else %} + - name: kubeconfig + workspaceKey: {{custom.kubeconfig_secret_name}} + {% endif %} + alertConfig: + tasks: + persona: eager-edgar + sessionTTL: 10m diff --git a/codebundles/gcp-bucket-crossplane-health/.runwhen/templates/gcp-bucket-crossplane-health-slx.yaml b/codebundles/gcp-bucket-crossplane-health/.runwhen/templates/gcp-bucket-crossplane-health-slx.yaml new file mode 100644 index 0000000..56c01f7 --- /dev/null +++ b/codebundles/gcp-bucket-crossplane-health/.runwhen/templates/gcp-bucket-crossplane-health-slx.yaml @@ -0,0 +1,36 @@ +apiVersion: runwhen.com/v1 +kind: ServiceLevelX +metadata: + name: {{slx_name}} + labels: + {% include "common-labels.yaml" %} + annotations: + {% include "common-annotations.yaml" %} +spec: + imageURL: https://storage.googleapis.com/runwhen-nonprod-shared-images/icons/kubernetes.svg + alias: Crossplane GCP Bucket {{match_resource.resource.metadata.name}} Health + asMeasuredBy: Fraction of Crossplane .status.conditions[] that are True (Ready + Synced). + configProvided: + - name: OBJECT_NAME + value: {{match_resource.resource.metadata.name}} + - name: API_GROUP + value: storage.gcp.upbound.io + - name: KIND + value: Bucket + owners: + - {{workspace.owner_email}} + statement: >- + Crossplane GCP Bucket {{match_resource.resource.metadata.name}} should have + every status condition (Ready, Synced) in state True 99.5% of the time. + additionalContext: + {% include "kubernetes-hierarchy.yaml" ignore missing %} + qualified_name: "{{ match_resource.qualified_name }}" + tags: + {% include "kubernetes-tags.yaml" ignore missing %} + + - name: access + value: read-only + - name: managed-by + value: crossplane + - name: provider + value: upbound-gcp diff --git a/codebundles/gcp-bucket-crossplane-health/.runwhen/templates/gcp-bucket-crossplane-health-taskset.yaml b/codebundles/gcp-bucket-crossplane-health/.runwhen/templates/gcp-bucket-crossplane-health-taskset.yaml new file mode 100644 index 0000000..9f9edbb --- /dev/null +++ b/codebundles/gcp-bucket-crossplane-health/.runwhen/templates/gcp-bucket-crossplane-health-taskset.yaml @@ -0,0 +1,47 @@ +apiVersion: runwhen.com/v1 +kind: Runbook +metadata: + name: {{slx_name}} + labels: + {% include "common-labels.yaml" %} + annotations: + {% include "common-annotations.yaml" %} +spec: + location: {{default_location}} + # See the SLI template for the rationale on hardcoding the generic + # codecollection URL here. + codeBundle: + repoUrl: http://rw-airgap-cc-catalog-svc.runwhen-env-airgap:8080/git/rw-generic-codecollection.git + ref: main + pathToRobot: codebundles/k8s-kubectl-cmd/runbook.robot + configProvided: + - name: TASK_TITLE + value: 'Inspect Crossplane GCP Bucket {{match_resource.resource.metadata.name}} conditions' + - name: TIMEOUT_SECONDS + value: '300' + # Enable JSON-issue mode in the generic runbook: it will parse the + # stdout JSON, check the trigger key, and emit one RunWhen issue per + # element in the issues array. + - name: ISSUE_JSON_QUERY_ENABLED + value: 'true' + - name: ISSUE_JSON_TRIGGER_KEY + value: issuesIdentified + - name: ISSUE_JSON_TRIGGER_VALUE + value: 'true' + - name: ISSUE_JSON_ISSUES_KEY + value: issues + # The `jq` expression below emits the JSON envelope that the generic + # runbook expects: + # { issuesIdentified: , issues: [ {title,severity,...}, ... ] } + # One issue per condition whose status is not "True". Severity 2 for + # Ready failures (data-plane visible), 3 for others (e.g. Synced). + - name: KUBECTL_COMMAND + value: | + kubectl get buckets.storage.gcp.upbound.io {{match_resource.resource.metadata.name}} -o json | jq -c '(.status.conditions // []) as $c | { issuesIdentified: (($c|length)==0 or ([$c[]|select(.status!="True")]|length > 0)), issues: ( if ($c|length)==0 then [ { title: "Crossplane Bucket {{match_resource.resource.metadata.name}} has no status conditions yet", severity: 3, expected: "Bucket should have Ready and Synced conditions populated by the Crossplane provider", actual: "status.conditions is empty or missing", reproduce_hint: "kubectl get buckets.storage.gcp.upbound.io {{match_resource.resource.metadata.name}} -o yaml", next_steps: "1. Check that the crossplane-system provider pod for provider-gcp-storage is Running.\n2. kubectl describe buckets.storage.gcp.upbound.io {{match_resource.resource.metadata.name}} to see reconciliation events.\n3. Verify the referenced ProviderConfig exists and is Healthy.\n4. Wait one reconcile interval and re-check.", details: (.status | tostring) } ] else [ $c[] | select(.status!="True") | { title: ("Crossplane Bucket " + .type + " condition is not True: {{match_resource.resource.metadata.name}}"), severity: (if .type=="Ready" then 2 else 3 end), expected: (.type + " condition should be True"), actual: (.type + "=" + .status + " reason=" + (.reason // "n/a") + " message=" + (.message // "n/a")), reproduce_hint: "kubectl get buckets.storage.gcp.upbound.io {{match_resource.resource.metadata.name}} -o yaml", next_steps: "1. kubectl describe buckets.storage.gcp.upbound.io {{match_resource.resource.metadata.name}}\n2. Inspect the crossplane-system provider-gcp-storage pod logs for reconcile errors.\n3. Verify the ProviderConfig references a Secret with valid GCP credentials.\n4. Check GCP IAM permissions on the target bucket / project.\n5. Look for spec drift and wait for the next reconcile.", details: (. | tostring) } ] end ) }' + secretsProvided: + {% if wb_version %} + {% include "kubernetes-auth.yaml" ignore missing %} + {% else %} + - name: kubeconfig + workspaceKey: {{custom.kubeconfig_secret_name}} + {% endif %} diff --git a/codebundles/gcp-bucket-crossplane-health/README.md b/codebundles/gcp-bucket-crossplane-health/README.md new file mode 100644 index 0000000..9a07cba --- /dev/null +++ b/codebundles/gcp-bucket-crossplane-health/README.md @@ -0,0 +1,49 @@ +# gcp-bucket-crossplane-health + +A **generation-rule-only** codebundle. It has no `sli.robot`, no +`runbook.robot`, and no `meta.yaml`. It ships: + +| File | Role | +| --- | --- | +| [`.runwhen/generation-rules/gcp-bucket-crossplane-health.yaml`](.runwhen/generation-rules/gcp-bucket-crossplane-health.yaml) | Tells workspace-builder to discover every Crossplane `Bucket` (`storage.gcp.upbound.io/v1beta1`) in the cluster and generate one SLX per bucket. | +| [`.runwhen/templates/gcp-bucket-crossplane-health-slx.yaml`](.runwhen/templates/gcp-bucket-crossplane-health-slx.yaml) | Jinja template for the `ServiceLevelX` resource. | +| [`.runwhen/templates/gcp-bucket-crossplane-health-sli.yaml`](.runwhen/templates/gcp-bucket-crossplane-health-sli.yaml) | Jinja template for the `ServiceLevelIndicator`. Points at `rw-generic-codecollection/k8s-kubectl-cmd/sli.robot`. | +| [`.runwhen/templates/gcp-bucket-crossplane-health-taskset.yaml`](.runwhen/templates/gcp-bucket-crossplane-health-taskset.yaml) | Jinja template for the `Runbook` (TaskSet). Points at `rw-generic-codecollection/k8s-kubectl-cmd/runbook.robot` in JSON-issue mode. | + +## How it works + +1. `workspace-builder` loads this codecollection and reads the generation rule. +2. The rule declares `resourceTypes: [buckets.storage.gcp.upbound.io/v1beta1]`. + RunWhen Local's Kubernetes indexer expands that spec into + `plural=buckets, group=storage.gcp.upbound.io, version=v1beta1`, calls + `list_cluster_custom_object`, and stores each Bucket in the registry. +3. The `matchRules` predicate matches every Bucket by name (regex `.+`). +4. For each match, the rule emits an SLX with three output items (`slx`, + `sli`, `runbook`). Each output item is rendered from the corresponding + Jinja template. +5. The SLI and Runbook templates set `codeBundle.repoUrl` to the internal + airgap URL for `rw-generic-codecollection`, and `pathToRobot` to + `k8s-kubectl-cmd/{sli,runbook}.robot`. That's where the actual runtime + lives — this codecollection ships only the discovery and templating + logic. + +## Health scoring + +- **SLI** — the `KUBECTL_COMMAND` runs `kubectl get bucket ... -o json`, pipes + through `jq` to compute the fraction of `.status.conditions[]` whose + `status == "True"`, and pushes that number (0.0 to 1.0) as the metric. + A healthy Crossplane Bucket has `Ready=True` **and** `Synced=True`, + producing an SLI of `1.0`. +- **TaskSet** — the same generic bundle in JSON-issue mode. The `jq` + expression emits `{ issuesIdentified: true|false, issues: [...] }`. + Each condition whose status is not `True` becomes one issue with a + severity (higher for `Ready`), a `reproduce_hint`, and human-readable + `next_steps`. + +## Why this pattern is useful + +- **Reuse over rewrite**: no need to fork or maintain another robot file. +- **Small blast radius**: a private repo like this one holds just the + organization-specific discovery logic and messaging. +- **Fast iteration**: change the `jq` filter and the workspace-builder + picks it up on the next reconcile — no runner image rebuild required. diff --git a/docs/01-overview.md b/docs/01-overview.md new file mode 100644 index 0000000..b00ba58 --- /dev/null +++ b/docs/01-overview.md @@ -0,0 +1,75 @@ +# 01 — Overview + +## What you'll do in this guide + +By the end of this guide you will have a private RunWhen CodeCollection that: + +1. Is hosted in your Gitea instance (already the case for this workspace — + see `https://gitea.airgap.shared.runwhen.com/gitea-admin/simple-private-codecollection`). +2. **Contains no Robot Framework code of its own.** It only ships two things: + - **Generation rules** — YAML that tells RunWhen's workspace-builder + which resources to discover in the cluster and how many SLXs to + generate for each match. + - **Templates** — Jinja files that render the `ServiceLevelX`, + `ServiceLevelIndicator`, and `Runbook` resources for each match. +3. Reuses an existing generic codebundle — `k8s-kubectl-cmd` from + [`rw-generic-codecollection`](https://github.com/runwhen-contrib/rw-generic-codecollection/tree/main/codebundles/k8s-kubectl-cmd) — + as the runtime for every generated SLI and TaskSet. +4. Discovers **Crossplane GCP Bucket** custom resources + (`buckets.storage.gcp.upbound.io/v1beta1`) and produces one SLX per + Bucket with a fractional health SLI and a detailed condition-based + TaskSet. + +## Vocabulary primer + +| Term | Meaning | +|------|---------| +| **CodeCollection** | A Git repository, cloned by RunWhen, that contains one or more codebundles. This repo is a CodeCollection. | +| **CodeBundle** | A subdirectory of a CodeCollection (`codebundles//`). Traditionally contains Robot Framework code plus discovery rules. **Ours contains only discovery rules and templates.** | +| **Generation rule** | YAML under `.runwhen/generation-rules/`. Declares which resource types to scan and how to build SLXs from matches. | +| **Template** | Jinja YAML under `.runwhen/templates/`. Rendered per matching resource by workspace-builder to produce the final SLX / SLI / Runbook artifacts. | +| **SLX** | A "Service Level X" — the top-level unit of monitoring in RunWhen. Bundles together an SLI, an optional SLO, and one or more Runbooks. | +| **SLI** | Service Level Indicator — a numeric health signal (here `0.0` to `1.0`) pushed on an interval. | +| **Runbook / TaskSet** | The interactive troubleshooting sequence. Called a "Runbook" in `kind`, "TaskSet" in the UI. | +| **workspace-builder** | The pod (image `runwhen-local`) that clones code collections, indexes resources, matches generation rules, renders templates, and uploads the resulting workspace to the platform. | + +## The pattern in one diagram + +![overview diagram](images/01-overview-diagram.png) + +*(Placeholder — replace with a screenshot exported from your diagramming +tool once the flow is verified end-to-end.)* + +```mermaid +flowchart LR + Private["Gitea
simple-private-codecollection
(rules + templates)"] + WB["workspace-builder
(clones + indexes + renders)"] + Cluster[("GKE cluster
Crossplane Bucket CRDs")] + Generic["rw-generic-codecollection
k8s-kubectl-cmd
(sli.robot + runbook.robot)"] + + Private --> WB + Cluster --> WB + WB -->|"one SLX per Bucket"| Platform["RunWhen Platform"] + Platform -->|"Runner pulls robots from"| Generic + Generic -->|kubectl + jq| Cluster +``` + +## When to use this pattern + +Choose the "generation-rule only" pattern when: + +- You want to **discover organization-specific resources** (CRDs, tags, + labels) without maintaining a fork of the shared code collections. +- The **runtime already exists** as a generic bundle + (`k8s-kubectl-cmd`, `curl-cmd`, `gcloud-cmd`, `aws-cmd`, etc.) and you + just need to point it at the right target. +- You want changes to your discovery rules to land **without a codebundle + runner image rebuild** — workspace-builder re-clones on each reconcile. + +Do NOT use it when you need custom Robot keywords, a task that +can't be expressed as a single `kubectl`/`gcloud`/`curl` command, or a +runtime that isn't already available in a generic bundle. + +## Next + +Continue to [02 — Generation-rule-only pattern](02-generation-rules-only-pattern.md). diff --git a/docs/02-generation-rules-only-pattern.md b/docs/02-generation-rules-only-pattern.md new file mode 100644 index 0000000..0b58a2e --- /dev/null +++ b/docs/02-generation-rules-only-pattern.md @@ -0,0 +1,124 @@ +# 02 — The generation-rule-only pattern + +## What you'll do + +Understand the split of responsibilities between this private code +collection (rules + templates only) and the shared `rw-generic-codecollection` +(the actual Robot runtime). + +## Traditional codebundle vs. rules-only codebundle + +A **traditional** codebundle looks like this: + +``` +codebundles/k8s-my-thing/ + meta.yaml + README.md + runbook.robot # ← runtime + sli.robot # ← runtime + .runwhen/ + generation-rules/*.yaml + templates/*.yaml # templates reference this bundle's own robots +``` + +A **rules-only** codebundle looks like this — this is the pattern we are +teaching: + +``` +codebundles/gcp-bucket-crossplane-health/ + README.md + .runwhen/ + generation-rules/gcp-bucket-crossplane-health.yaml + templates/ + gcp-bucket-crossplane-health-slx.yaml + gcp-bucket-crossplane-health-sli.yaml + gcp-bucket-crossplane-health-taskset.yaml +``` + +Notice: no `meta.yaml`, no `runbook.robot`, no `sli.robot`. The templates +still reference `pathToRobot`, but they point at a **different** +code collection. + +## The delegation trick + +Look at [`gcp-bucket-crossplane-health-sli.yaml`](../codebundles/gcp-bucket-crossplane-health/.runwhen/templates/gcp-bucket-crossplane-health-sli.yaml): + +```yaml +codeBundle: + repoUrl: http://rw-airgap-cc-catalog-svc.runwhen-env-airgap:8080/git/rw-generic-codecollection.git + ref: main + pathToRobot: codebundles/k8s-kubectl-cmd/sli.robot +configProvided: + - name: KUBECTL_COMMAND + value: | + kubectl get buckets.storage.gcp.upbound.io {{match_resource.resource.metadata.name}} ... +``` + +`repoUrl` points at the airgap cc-catalog proxy for +`rw-generic-codecollection`. `pathToRobot` picks the generic +`k8s-kubectl-cmd/sli.robot`. `configProvided.KUBECTL_COMMAND` is the actual +command we want that generic robot to run — parameterized by the discovered +resource's name via the `match_resource` Jinja variable. + +At runtime, the workflow is: + +```mermaid +sequenceDiagram + participant WB as workspace-builder + participant CR as Cluster (kubeapi) + participant Repo as private CC (Gitea) + participant PAPI as RunWhen Platform + participant Worker as Runner Worker + participant Generic as rw-generic-codecollection + + WB->>Repo: git clone (with token) + WB->>CR: list buckets.storage.gcp.upbound.io + CR-->>WB: [ Bucket A, Bucket B, ... ] + WB->>WB: match rule; render templates + WB->>PAPI: upload N SLXs (each with codeBundle.repoUrl = rw-generic) + PAPI->>Worker: schedule SLI + TaskSet runs + Worker->>Generic: clone at ref=main; load k8s-kubectl-cmd + Worker->>CR: KUBECTL_COMMAND (kubectl + jq) + CR-->>Worker: JSON status + Worker->>PAPI: push metric / issues +``` + +## Why not just put the robot files in the private CC? + +Because then every consumer of the private CC would need to run the +runner-side clone and image handling for those robots. The airgap runner's +`runner.codeCollections` list would have to include this repo (with the +same token concerns). By keeping the private CC as **rules and templates +only**, you never have to update the runner-side collections list — the +Robot code always resolves to the same shared, cached, image-baked +generic collection that the runner already knows how to load. + +## The template variables you have + +At render time, workspace-builder passes each template a rich context. +The most useful entries for our use case: + +| Variable | Value in our case | +|----------|-------------------| +| `slx_name` | Generated SLX name, e.g. `shared-cluster-runwhen-nonprod-shared-litellm-logging-xp-bkt-hlth` | +| `match_resource.resource` | The full Kubernetes object as returned by the API | +| `match_resource.resource.metadata.name` | The Bucket's name | +| `match_resource.kind` | `Bucket` (for CRDs, filled by the indexer) | +| `match_resource.qualified_name` | The composite ID used by the resource registry | +| `qualifiers` | The dict populated per your `qualifiers` list in the rule (`resource`, `cluster`) | +| `cluster.name` | The Kubernetes cluster name (e.g. `shared-cluster`) | +| `default_location` | Workspace default runner location | +| `workspace.owner_email` | From `workspaceInfo.workspaceOwnerEmail` | +| `custom.kubeconfig_secret_name` | From the `custom:` block in workspaceInfo | +| `wb_version` | Set only in newer workspace-builders (used to gate the newer `kubernetes-auth.yaml` include) | + +## Screenshot placeholders + +- `images/02-side-by-side-file-tree.png` — screenshot of the two + directory trees (traditional vs. rules-only) side by side. +- `images/02-render-sequence.png` — annotated version of the sequence + diagram above (optional). + +## Next + +Continue to [03 — Crossplane CRD discovery](03-crossplane-crd-discovery.md). diff --git a/docs/03-crossplane-crd-discovery.md b/docs/03-crossplane-crd-discovery.md new file mode 100644 index 0000000..80be1ed --- /dev/null +++ b/docs/03-crossplane-crd-discovery.md @@ -0,0 +1,139 @@ +# 03 — Crossplane CRD discovery + +## What you'll do + +Understand how RunWhen's Kubernetes indexer discovers **custom +resources** — specifically the Crossplane GCP Bucket CRD — driven entirely +by what your generation rules ask for. + +## Step 1 — Confirm the CRD exists + +```bash +kubectl api-resources | grep storage.gcp.upbound.io +``` + +Expected output (trimmed): + +``` +buckets storage.gcp.upbound.io/v1beta1 false Bucket +bucketiammembers storage.gcp.upbound.io/v1beta1 false BucketIAMMember +hmackeys storage.gcp.upbound.io/v1beta1 false HMACKey +``` + +Three things to note: + +1. **`buckets`** — this is the *plural resource name*. It is what you + put in the generation rule, not the singular `Bucket`. +2. **`storage.gcp.upbound.io/v1beta1`** — the API group and preferred + version. +3. **`false` in the "NAMESPACED" column** — Crossplane managed resources + are **cluster-scoped**. This is why our generation rule uses + `qualifiers: ["resource", "cluster"]` and not `namespace`. + +Screenshot placeholder: `images/03-kubectl-api-resources.png` — output of +the above command. + +## Step 2 — Confirm buckets exist to discover + +```bash +kubectl get buckets.storage.gcp.upbound.io -o wide +``` + +Sample output on the airgap cluster: + +``` +NAME READY SYNCED EXTERNAL-NAME +runwhen-nonprod-shared-litellm-logging True True runwhen-nonprod-shared-litellm-logging +runwhen-nonprod-shared-loki True True runwhen-nonprod-shared-loki +runwhen-nonprod-shared-mimir True True runwhen-nonprod-shared-mimir +runwhen-nonprod-shared-tempo True True runwhen-nonprod-shared-tempo +``` + +Screenshot placeholder: `images/03-kubectl-buckets.png`. + +## Step 3 — Look at the `.status.conditions[]` we care about + +This is what the SLI and TaskSet will read. Pick any bucket and inspect it: + +```bash +kubectl get buckets.storage.gcp.upbound.io -o json | jq .status.conditions +``` + +Expected shape: + +```json +[ + { + "lastTransitionTime": "2025-11-14T10:15:11Z", + "reason": "Available", + "status": "True", + "type": "Ready" + }, + { + "lastTransitionTime": "2025-11-14T10:15:10Z", + "reason": "ReconcileSuccess", + "status": "True", + "type": "Synced" + } +] +``` + +Every Crossplane managed resource carries these two conditions: + +- **`Ready`** — the external resource (the actual GCS bucket in GCP) exists + and is available. When `Ready=False`, something in GCP is wrong (missing + IAM, quota, deleted out-of-band, etc.). +- **`Synced`** — the last reconcile between the Crossplane spec and the + provider succeeded. When `Synced=False`, Crossplane could not talk to + the provider or hit a validation error. + +Our SLI computes `(#True conditions) / (#total conditions)`, so a healthy +bucket returns `1.0` and any single failing condition drops it to `0.5` +or `0.0`. + +## Step 4 — Understand how workspace-builder gets there + +RunWhen Local's Kubernetes indexer performs **selective discovery**: it +only lists custom resource types that appear in loaded generation rules. +That's why simply declaring `buckets.storage.gcp.upbound.io/v1beta1` in +our rule is enough — no extra `customResourceTypes:` list in +`workspaceInfo.yaml`. + +The parsing (in +[`runwhen-local/src/indexers/kubetypes.py`](https://github.com/runwhen-contrib/runwhen-local/blob/main/src/indexers/kubetypes.py)) +takes the string `plural.group/version` and splits it into +`(plural='buckets', group='storage.gcp.upbound.io', version='v1beta1')`. +It then calls `CustomObjectsApi.list_cluster_custom_object(...)`. + +You can also specify custom resources with a **dict** form, useful when +you need to be explicit about a particular version: + +```yaml +resourceTypes: + - platform: kubernetes + resourceType: custom + kind: buckets # plural, NOT PascalCase + group: storage.gcp.upbound.io + version: v1beta1 +``` + +## Common pitfalls + +- **Wrong plural** — the biggest source of "nothing gets discovered". Always + copy the plural from `kubectl api-resources`, never guess it from the + `Kind`. +- **Version omission** — leaving off `/v1beta1` uses the API server's + *preferred* version. Usually fine but pin it explicitly if you rely on + a specific schema. +- **RBAC** — workspace-builder needs `get`/`list`/`watch` on the CRD. The + airgap runner already binds `workspace-builder` SA to `view` at the + cluster scope via + [`rbac/workspace-builder-view-rbac.yaml`](https://github.com/runwhen/infra-flux-nonprod-shared/blob/main/apps/runwhen-env/airgap/rbac/workspace-builder-view-rbac.yaml), + which covers CRDs installed after the binding was created. +- **Namespace scope** — some CRDs are namespaced. If you had chosen a + namespaced CRD you would need to include the CRD's namespaces in your + `cloudConfig.kubernetes.namespaces` list *or* leave that list empty. + +## Next + +Continue to [04 — Generation rule walkthrough](04-generation-rule-walkthrough.md). diff --git a/docs/04-generation-rule-walkthrough.md b/docs/04-generation-rule-walkthrough.md new file mode 100644 index 0000000..f673e7e --- /dev/null +++ b/docs/04-generation-rule-walkthrough.md @@ -0,0 +1,165 @@ +# 04 — Generation rule walkthrough + +## What you'll do + +Read the generation rule at [`codebundles/gcp-bucket-crossplane-health/.runwhen/generation-rules/gcp-bucket-crossplane-health.yaml`](../codebundles/gcp-bucket-crossplane-health/.runwhen/generation-rules/gcp-bucket-crossplane-health.yaml) +line by line and understand each block. + +## The full file + +```yaml +apiVersion: runwhen.com/v1 +kind: GenerationRules +spec: + platform: kubernetes + generationRules: + - resourceTypes: + - buckets.storage.gcp.upbound.io/v1beta1 + matchRules: + - type: pattern + pattern: ".+" + properties: [name] + mode: substring + slxs: + - baseName: xp-bkt-hlth + qualifiers: ["resource", "cluster"] + baseTemplateName: gcp-bucket-crossplane-health + levelOfDetail: detailed + outputItems: + - type: slx + - type: sli + - type: runbook + templateName: gcp-bucket-crossplane-health-taskset.yaml +``` + +## Block-by-block + +### Envelope + +```yaml +apiVersion: runwhen.com/v1 +kind: GenerationRules # note the plural — the loader rejects "GenerationRule" +spec: + platform: kubernetes # per-file default; can be overridden per rule + generationRules: [ ... ] # one or more rules +``` + +- `kind` must be exactly `GenerationRules`. RunWhen Local's loader + compares this to a schema constant. +- `spec.platform` sets the default platform for every rule in the file. +- `spec.generationRules` is a list — you can define many rules per file. + +### `resourceTypes` + +```yaml +resourceTypes: + - buckets.storage.gcp.upbound.io/v1beta1 +``` + +This is the *selective discovery* signal. Because this string appears in +a loaded rule, the Kubernetes indexer knows to enumerate that CRD. +Without any rule mentioning a CRD, it would not be listed. + +You can list multiple types in one rule if you want the same SLX shape +for each. Splitting into separate rules is usually cleaner though. + +### `matchRules` + +```yaml +matchRules: + - type: pattern + pattern: ".+" + properties: [name] + mode: substring +``` + +Match predicates are ANDed together. The pattern predicate here says +"the resource's `name` property must match the regex `.+` (i.e. non-empty) +as a substring". Since every Bucket has a name, every Bucket matches. + +You can filter more aggressively if you want. Examples: + +```yaml +# Only buckets whose name starts with "prod-" +- type: pattern + pattern: "^prod-" + properties: [name] + mode: substring +``` + +```yaml +# Only buckets that carry the label env=production +- type: exists + path: "resource/metadata/labels/env" +``` + +```yaml +# Combine — buckets in the prod GCP project label AND with a Ready condition entry +- type: and + matches: + - type: pattern + pattern: "^runwhen-prod-.*" + properties: [name] + mode: substring + - type: exists + path: "resource/status/conditions" +``` + +### `slxs` + +```yaml +slxs: + - baseName: xp-bkt-hlth + qualifiers: ["resource", "cluster"] + baseTemplateName: gcp-bucket-crossplane-health + levelOfDetail: detailed + outputItems: + - type: slx + - type: sli + - type: runbook + templateName: gcp-bucket-crossplane-health-taskset.yaml +``` + +- `baseName` — short identifier appended to the qualifier prefix to form + the final `slx_name`. Keep it under 15 characters (`xp-bkt-hlth` is 11). +- `qualifiers` — determines both the **generated SLX name** and how + many SLXs are produced. + - Including `resource` means "one SLX per matching Bucket". + - Including `cluster` prefixes the name with the cluster name so it is + stable across multi-cluster workspaces. + - **Not including `namespace`** — Crossplane managed resources are + cluster-scoped; they have no namespace. +- `baseTemplateName` — the file-name stem for the templates. The output + items resolve to `-.yaml` unless overridden. + So `type: slx` looks up `gcp-bucket-crossplane-health-slx.yaml`, + `type: sli` → `-sli.yaml`, etc. +- `levelOfDetail` — one of `none | basic | detailed` (or `0|1|2`). This + gates whether the SLX is emitted based on the platform's LOD filter. + Since our airgap `defaultLOD` is `detailed`, we set the SLX to + `detailed` so it always renders. +- `outputItems` — one entry per artifact we want. `type: runbook` + specifies an explicit `templateName` because we want to point at a + file named `-taskset.yaml`, not `-runbook.yaml`, purely for convention + matching what other RunWhen bundles do. + +## What resource identity ends up in the SLX + +Given a Bucket named `runwhen-nonprod-shared-litellm-logging` on cluster +`shared-cluster`, the rule will produce an SLX whose name is roughly: + +``` +shared-cluster-runwhen-nonprod-shared-litellm-logging-xp-bkt-hlth +``` + +RunWhen Local will *also* shorten and hash-suffix pieces as needed for +length. The exact rendering happens in +[`generation_rules.py::make_slx_name`](https://github.com/runwhen-contrib/runwhen-local/blob/main/src/enrichers/generation_rules.py). + +## Screenshot placeholders + +- `images/04-generation-rule-annotated.png` — the YAML above with arrows + pointing to the resource type, match rule, and output items. + +## Next + +Continue to [05 — SLX and templates](05-slx-and-templates.md). diff --git a/docs/05-slx-and-templates.md b/docs/05-slx-and-templates.md new file mode 100644 index 0000000..fef3f39 --- /dev/null +++ b/docs/05-slx-and-templates.md @@ -0,0 +1,178 @@ +# 05 — SLX and templates + +## What you'll do + +Read the three Jinja templates that render the artifacts for every +discovered Bucket. + +## The three templates + +| Output type | Template file | Kind produced | +|-------------|---------------|---------------| +| `slx` | `gcp-bucket-crossplane-health-slx.yaml` | `ServiceLevelX` | +| `sli` | `gcp-bucket-crossplane-health-sli.yaml` | `ServiceLevelIndicator` | +| `runbook` | `gcp-bucket-crossplane-health-taskset.yaml` | `Runbook` | + +## The SLX template + +[`gcp-bucket-crossplane-health-slx.yaml`](../codebundles/gcp-bucket-crossplane-health/.runwhen/templates/gcp-bucket-crossplane-health-slx.yaml) + +Key sections: + +```yaml +spec: + alias: Crossplane GCP Bucket {{match_resource.resource.metadata.name}} Health + asMeasuredBy: Fraction of Crossplane .status.conditions[] that are True (Ready + Synced). + configProvided: + - name: OBJECT_NAME + value: {{match_resource.resource.metadata.name}} + - name: API_GROUP + value: storage.gcp.upbound.io + - name: KIND + value: Bucket +``` + +`configProvided` on the SLX itself is displayed in the UI as +"configured properties" of the service. It's not fed to Robot code — +that's what the SLI and Runbook `configProvided` blocks are for. Use it +for values that describe the *thing being monitored*, not values passed +to the *robot doing the monitoring*. + +The template also uses two shared includes shipped by workspace-builder: + +```yaml +additionalContext: + {% include "kubernetes-hierarchy.yaml" ignore missing %} + qualified_name: "{{ match_resource.qualified_name }}" +tags: + {% include "kubernetes-tags.yaml" ignore missing %} + - name: access + value: read-only +``` + +- **`kubernetes-hierarchy.yaml`** — emits the `hierarchy: [platform, cluster, + resource_name]` list used to build the SLX's UI breadcrumb. +- **`kubernetes-tags.yaml`** — emits `platform`, `cluster`, `resource_type`, + `resource_name`, and every Kubernetes label as a `[k8s]` tag. It + also emits `resource_type: bucket` because + `match_resource.resource_type.name` is `custom` for CRDs (the include + handles that special case). + +## The SLI template + +[`gcp-bucket-crossplane-health-sli.yaml`](../codebundles/gcp-bucket-crossplane-health/.runwhen/templates/gcp-bucket-crossplane-health-sli.yaml) + +The most important line is the `codeBundle` block: + +```yaml +codeBundle: + repoUrl: http://rw-airgap-cc-catalog-svc.runwhen-env-airgap:8080/git/rw-generic-codecollection.git + ref: main + pathToRobot: codebundles/k8s-kubectl-cmd/sli.robot +``` + +`repoUrl` deliberately points at the airgap cc-catalog proxy, not at +this private CC and not at public GitHub. That's how the runner reaches +`rw-generic-codecollection` — it's the URL the runner already trusts and +mirrors. + +> **If you move this example to another environment**, change this URL +> to wherever the generic collection is served: public GitHub for +> internet-connected environments, a public JCR / mirror otherwise. + +The `configProvided` block wires up `k8s-kubectl-cmd`'s expected inputs: + +```yaml +configProvided: + - name: TASK_TITLE + value: 'Crossplane GCP Bucket {{match_resource.resource.metadata.name}} condition health' + - name: KUBECTL_COMMAND + value: | + kubectl get buckets.storage.gcp.upbound.io {{match_resource.resource.metadata.name}} -o json | jq -r '(.status.conditions // []) as $c | if ($c|length)==0 then 0 else ([$c[]|select(.status=="True")]|length)/($c|length) end' + - name: TIMEOUT_SECONDS + value: '120' +``` + +The `jq` breakdown: + +| Fragment | Meaning | +|----------|---------| +| `(.status.conditions // []) as $c` | Save the conditions array (or empty array if missing) into `$c` | +| `if ($c\|length)==0 then 0` | No conditions yet → return 0 (unhealthy). Signals "the resource has not reconciled". | +| `else ([$c[]\|select(.status=="True")]\|length) / ($c\|length)` | Otherwise fraction of True conditions over total | + +`RW.Core.Push Metric ${rsp.stdout}` inside the generic SLI robot takes +that value (a JSON number between 0 and 1) and pushes it to the platform. + +Finally the secrets block: + +```yaml +secretsProvided: +{% if wb_version %} + {% include "kubernetes-auth.yaml" ignore missing %} +{% else %} + - name: kubeconfig + workspaceKey: {{custom.kubeconfig_secret_name}} +{% endif %} +``` + +`wb_version` is set only by newer workspace-builders. This lets the +same template render correctly against both old and new runners. The +`kubeconfig_secret_name` value comes from the `custom:` block in your +runner's `workspaceInfo.yaml` (which we don't need to change — the +existing airgap runner already provides +`k8s:file@secret/kubeconfig:kubeconfig`). + +## The TaskSet template + +[`gcp-bucket-crossplane-health-taskset.yaml`](../codebundles/gcp-bucket-crossplane-health/.runwhen/templates/gcp-bucket-crossplane-health-taskset.yaml) + +Same shape as the SLI, but pointing at `runbook.robot` and with a +bigger `jq` expression that emits the JSON issue envelope: + +```yaml +- name: ISSUE_JSON_QUERY_ENABLED + value: 'true' +- name: ISSUE_JSON_TRIGGER_KEY + value: issuesIdentified +- name: ISSUE_JSON_TRIGGER_VALUE + value: 'true' +- name: ISSUE_JSON_ISSUES_KEY + value: issues +``` + +These four variables are how `k8s-kubectl-cmd/runbook.robot` decides +whether to parse the stdout as JSON. When enabled, it looks for an +object like: + +```json +{ + "issuesIdentified": true, + "issues": [ + { + "title": "...", + "severity": 2, + "expected": "...", + "actual": "...", + "reproduce_hint": "...", + "next_steps": "...", + "details": "..." + } + ] +} +``` + +Our `KUBECTL_COMMAND` produces exactly that shape from +`.status.conditions[]`. One issue per condition that is not `True`. +Ready failures are severity 2, others (like Synced) are severity 3. + +## Screenshot placeholders + +- `images/05-rendered-slx.png` — a rendered SLX YAML in the RunWhen UI + (from the SLX detail view). +- `images/05-tags-and-hierarchy.png` — the SLX detail sidebar showing + the hierarchy and tags produced by the shared includes. + +## Next + +Continue to [06 — Runner integration](06-runner-integration.md). diff --git a/docs/06-runner-integration.md b/docs/06-runner-integration.md new file mode 100644 index 0000000..b392336 --- /dev/null +++ b/docs/06-runner-integration.md @@ -0,0 +1,120 @@ +# 06 — Runner integration + +## What you'll do + +Tell the airgap `workspace-builder` where to find this repository and +have Flux reconcile the change. + +## The single edit + +Open the airgap runner's HelmRelease values file: + +`apps/runwhen-env/airgap/runwhen-runner/helm.yaml` + +Under `spec.values.workspaceBuilder.workspaceInfo.configMap.data.codeCollections`, +append one entry. Before: + +```yaml +codeCollections: +- repoURL: http://rw-airgap-cc-catalog-svc.runwhen-env-airgap:8080/git/rw-cli-codecollection.git + ref: main +``` + +After: + +```yaml +codeCollections: +- repoURL: http://rw-airgap-cc-catalog-svc.runwhen-env-airgap:8080/git/rw-cli-codecollection.git + ref: main +- repoURL: https://@gitea.airgap.shared.runwhen.com/gitea-admin/simple-private-codecollection.git + ref: main +``` + +The `` value is a Gitea personal access token with at least +`read:repository` scope on the `simple-private-codecollection` repo. + +Screenshot placeholder: `images/06-helm-yaml-diff.png` — annotated diff of +the two-line addition. + +## What we intentionally did NOT change + +- `runner.codeCollections` — the list of code collections whose Robot + code is executed by worker pods. Because our private CC ships **no + robots**, it never needs a runner worker. The runtime lives in the + already-registered `rw-generic-codecollection`. +- Any secret configuration — we are using the fastest working option: + the token embedded directly in the `repoURL`. This trades tidiness + for zero setup complexity. See "Trade-offs" below. + +## Trade-offs of token-in-URL (option B) + +The token becomes visible to anyone with read access to the +`workspace-builder` `ConfigMap` — via `kubectl get cm workspace-builder +-o yaml` in the runner namespace, and to anyone reading the Flux repo. +That is acceptable for a **test** token you can rotate, but not +production hygiene. + +Two better options once this example is working: + +### Option A — cc-catalog proxy (production path) + +Turn on runtime git sync in the platform-side cc-catalog service and add +a `simple-private-codecollection` slug there. The workspace-builder then +clones from the internal proxy URL (no token in ConfigMap): + +```yaml +- repoURL: http://rw-airgap-cc-catalog-svc.runwhen-env-airgap:8080/git/simple-private-codecollection.git + ref: main +``` + +That requires editing `apps/runwhen-env/airgap/runwhen-platform/helmrelease.yaml` +to (a) add the slug under `ccCatalog.config.sources[].codecollections` +and (b) enable `git.runtime_sync: true` with `git.auth.token_env` +pointing at a Kubernetes secret storing the Gitea token. Track that as a +follow-up. + +### Option C — fix upstream + +Patch `runwhen-local` to actually apply `authUser`/`authToken` in +[`code_collection.py`](https://github.com/runwhen-contrib/runwhen-local/blob/main/src/enrichers/code_collection.py) +`update_repo()`. Today those fields are parsed and stored on the +`CodeCollection` object but never applied to `Repo.clone_from()`. A +one-line fix to route through `git_utils.get_repo_url_with_auth()`. Then +you can pass `authUser`/`authToken` in `workspaceInfo.yaml` and reference +them as environment variables or literal values without leaking the +token. + +## Trigger the change + +The Flux `HelmRelease` for `runwhen-local` reconciles every 5 minutes, +but you can force it: + +```bash +flux reconcile hr runwhen-local -n runwhen-env-airgap-runner --with-source +``` + +Watch for the reconcile to complete: + +```bash +kubectl -n runwhen-env-airgap-runner get pods -w +``` + +You should see the `runwhen-local` pod restart (because the ConfigMap +changed). Its next start will use the new `workspaceInfo`. + +## Verify the workspace-builder can reach Gitea + +```bash +kubectl -n runwhen-env-airgap-runner exec deploy/runwhen-local -c workspace-builder -- \ + curl -sSI --max-time 5 https://gitea.airgap.shared.runwhen.com/ | head -1 +``` + +Expected: `HTTP/2 200` or `HTTP/1.1 200 OK`. If the request hangs, check +`NetworkPolicy` — the airgap deps namespace should already permit this +traffic on 443. + +Screenshot placeholder: `images/06-curl-gitea.png`. + +## Next + +Continue to [07 — Validate and observe](07-validate-and-observe.md). diff --git a/docs/07-validate-and-observe.md b/docs/07-validate-and-observe.md new file mode 100644 index 0000000..c9fb120 --- /dev/null +++ b/docs/07-validate-and-observe.md @@ -0,0 +1,144 @@ +# 07 — Validate and observe + +## What you'll do + +Confirm end-to-end that: + +1. The generation rule is schema-valid. +2. Workspace-builder clones the private CC from Gitea. +3. One SLX is rendered per Crossplane Bucket in the cluster. +4. The RunWhen Platform receives the SLXs and the SLI + TaskSet run. + +## Step 1 — Local schema validation + +If you have `runwhen-local` checked out, you can validate before pushing: + +```bash +cd /path/to/runwhen-local/src +python3 validate_generation_rules.py /path/to/simple-private-codecollection -v +``` + +Expected: + +``` +No validation errors found. +``` + +Or a one-liner without cloning `runwhen-local`: + +```bash +python3 -c " +import json, yaml, jsonschema, urllib.request +schema = json.loads(urllib.request.urlopen( + 'https://raw.githubusercontent.com/runwhen-contrib/runwhen-local/main/src/generation-rule-schema.json' +).read()) +with open('codebundles/gcp-bucket-crossplane-health/.runwhen/generation-rules/gcp-bucket-crossplane-health.yaml') as f: + jsonschema.validate(yaml.safe_load(f), schema) +print('OK') +" +``` + +Screenshot placeholder: `images/07-schema-validation.png`. + +## Step 2 — Push to Gitea + +```bash +git add . +git commit -m "Initial codecollection: Crossplane GCP Bucket rules-only example" +git push -u origin main +``` + +Refresh the Gitea repo URL in a browser — you should see the file tree. + +Screenshot placeholder: `images/07-gitea-repo.png` — Gitea file browser +showing `codebundles/` and `docs/`. + +## Step 3 — Reconcile Flux + +```bash +flux reconcile hr runwhen-local -n runwhen-env-airgap-runner --with-source +``` + +## Step 4 — Watch workspace-builder logs + +```bash +kubectl -n runwhen-env-airgap-runner logs \ + -l runwhen.com/component=workspace-builder \ + -c workspace-builder --tail=200 -f +``` + +Look for lines like: + +- `Cloning from git source: https://.../simple-private-codecollection.git` +- `Loading generation rules from ...gcp-bucket-crossplane-health.yaml` +- `Rendered N SLXs` (where N is the number of Buckets) + +Common errors and their meaning: + +| Error | Cause | +|-------|-------| +| `fatal: Authentication failed` on clone | Token in URL missing, expired, or lacks read scope | +| `KeyError: 'match_resource'` in a template | Jinja variable typo, or trying to use a variable that isn't populated for the platform | +| `Validation failed for ... generation rule ... required "outputItems"` | Schema drift — check the JSON schema | +| `No matches for resource type buckets.storage.gcp.upbound.io/v1beta1` | The CRD isn't installed on this cluster, or the plural is wrong | +| `Forbidden ... cannot list buckets.storage.gcp.upbound.io` | Missing RBAC for `workspace-builder` SA | + +Screenshot placeholder: `images/07-workspace-builder-logs.png`. + +## Step 5 — Inspect rendered SLXs in the runner filesystem + +`workspace-builder` writes the rendered artifacts to disk before +uploading them. Peek at one: + +```bash +kubectl -n runwhen-env-airgap-runner exec deploy/runwhen-local \ + -c workspace-builder -- \ + find /shared/output -type f | grep xp-bkt-hlth | head -5 +``` + +Then read one: + +```bash +kubectl -n runwhen-env-airgap-runner exec deploy/runwhen-local \ + -c workspace-builder -- \ + cat /shared/output/output/slx//sli.yaml +``` + +Confirm the `codeBundle.repoUrl` points at the internal +`rw-generic-codecollection` proxy and the `KUBECTL_COMMAND` includes +the bucket name. + +## Step 6 — See it in the RunWhen UI + +Sign into the RunWhen workspace UI: +`https://app.airgap.shared.runwhen.com/map/self` + +You should see one SLX per Bucket under the `shared-cluster` hierarchy. +Click into one — the SLI panel should already be graphing a value near +`1.0` (assuming buckets are healthy). Trigger the TaskSet from the SLX +detail page and confirm the report includes `Command stdout` showing +the JSON envelope, and any non-True conditions become issues. + +Screenshot placeholders: + +- `images/07-runwhen-map.png` — the map view with the new SLXs visible. +- `images/07-sli-graph.png` — the SLI panel graphing 1.0. +- `images/07-taskset-report.png` — the TaskSet execution report. +- `images/07-issue-detail.png` — a rendered issue (deliberately break a + Bucket by editing the Crossplane spec to force a `Synced=False` + condition, so we can capture what an unhealthy report looks like). + +## Iteration checklist + +- [ ] Iteration 1: end-to-end plumbing works, SLXs are generated, + SLI graphs are populated. **(You are here.)** +- [ ] Iteration 2: screenshots captured and inlined in these chapters. +- [ ] Iteration 3: prose polish, `.test/` harness for local + `runwhen-local` container validation, migration to Option A + (cc-catalog proxy) for production hygiene. + +## Where to file feedback + +If something in these docs is confusing or wrong, edit the markdown +directly and push. This is a living training guide — improvements are +expected. diff --git a/docs/images/.gitkeep b/docs/images/.gitkeep new file mode 100644 index 0000000..a4e7814 --- /dev/null +++ b/docs/images/.gitkeep @@ -0,0 +1,2 @@ +# Placeholder so an empty images/ directory is tracked in git. +# Delete this file once real screenshots (PNGs) have been added.