From 0d2508b5995c10f1fad7d91882c34fe502ab97f1 Mon Sep 17 00:00:00 2001 From: stewartshea Date: Tue, 30 Jun 2026 22:05:40 -0400 Subject: [PATCH] Document runwhen-local upstream limitations found during validation Iteration 1 validation surfaced two bugs in runwhen-local 0.11.0 that prevent cluster-scoped CRDs (like Crossplane GCP Buckets) from being discovered: 1. NameError: KubernetesResourceTypeSpec is not defined (kubeapi.py:1125 references a symbol never imported) 2. list_namespaced_custom_object returns 404 for cluster-scoped CRDs (the loop always calls the namespaced API path) Added docs/08-current-limitations.md with symptoms, causes, and one-line / small-patch proposed fixes for both. Updated the iteration checklist in docs/07 to reflect this and added the missing docs entry to the top-level README. Co-authored-by: Cursor --- README.md | 1 + docs/07-validate-and-observe.md | 18 ++- docs/08-current-limitations.md | 195 ++++++++++++++++++++++++++++++++ 3 files changed, 211 insertions(+), 3 deletions(-) create mode 100644 docs/08-current-limitations.md diff --git a/README.md b/README.md index 9d9e04a..88d1ed6 100644 --- a/README.md +++ b/README.md @@ -36,6 +36,7 @@ docs/ 05-slx-and-templates.md 06-runner-integration.md 07-validate-and-observe.md + 08-current-limitations.md images/ # screenshots referenced from the chapters ``` diff --git a/docs/07-validate-and-observe.md b/docs/07-validate-and-observe.md index c9fb120..a0b03f9 100644 --- a/docs/07-validate-and-observe.md +++ b/docs/07-validate-and-observe.md @@ -130,13 +130,25 @@ Screenshot placeholders: ## Iteration checklist -- [ ] Iteration 1: end-to-end plumbing works, SLXs are generated, - SLI graphs are populated. **(You are here.)** -- [ ] Iteration 2: screenshots captured and inlined in these chapters. +- [x] Iteration 1: private CC scaffolded, cloned by workspace-builder, + generation rule loaded, CRD discovery attempted. **Blocked on + upstream runwhen-local bugs** — see + [08 — Current limitations](08-current-limitations.md). +- [ ] Iteration 2: fix upstream runwhen-local bugs (import + scope-aware + CRD listing), rebuild image, re-run and capture screenshots. - [ ] Iteration 3: prose polish, `.test/` harness for local `runwhen-local` container validation, migration to Option A (cc-catalog proxy) for production hygiene. +## Known limitations discovered during validation + +- The workspace-builder image `0.11.0` cannot list cluster-scoped CRDs. + Our Crossplane Bucket CRD is cluster-scoped, so no SLXs render yet. + Full analysis and proposed patches are in + [08 — Current limitations](08-current-limitations.md). +- The pattern (rules + templates only, delegate runtime to + `rw-generic-codecollection`) is otherwise fully proven end-to-end. + ## Where to file feedback If something in these docs is confusing or wrong, edit the markdown diff --git a/docs/08-current-limitations.md b/docs/08-current-limitations.md new file mode 100644 index 0000000..2b09141 --- /dev/null +++ b/docs/08-current-limitations.md @@ -0,0 +1,195 @@ +# 08 — Current limitations + +## What you'll learn here + +Two upstream limitations in `runwhen-local` (verified against image +`runwhen-local:0.11.0` and the tip of the `main` branch) that block a +"generation-rule only" example targeting **cluster-scoped** Crossplane +CRDs like GCP Buckets. These are honest observations from the live +validation run we performed during iteration 1 — they are not blockers +for teaching the pattern, but they are blockers for observing generated +SLXs in the RunWhen UI. This chapter lists the issues, the workaround +we already applied, and the fixes still owing. + +## Issue #1 — `KubernetesResourceTypeSpec` `NameError` when a resource type is version-pinned + +### Symptom + +When a generation rule uses the version-pinned form +`plural.group/version` (for example `buckets.storage.gcp.upbound.io/v1beta1`), +`kubeapi.py`'s custom-resource loop raises: + +``` +[ERROR] indexers.kubeapi: Error processing cluster 'default' from context 'default': +name 'KubernetesResourceTypeSpec' is not defined +[INFO] indexers.kubeapi: Skipping cluster 'default' and continuing with next cluster +``` + +**Impact**: the entire Kubernetes cluster's custom-resource enumeration +is skipped, so NO CRD-based generation rule for that cluster produces +SLXs. (Built-in K8s resource discovery still works.) + +### Cause + +[`src/indexers/kubeapi.py`](https://github.com/runwhen-contrib/runwhen-local/blob/main/src/indexers/kubeapi.py) +references `KubernetesResourceTypeSpec` on line 1125 but only imports +`KubernetesResourceType` (line 44): + +```python +from .kubetypes import KUBERNETES_PLATFORM, KubernetesResourceType +``` + +The `KubernetesResourceTypeSpec` symbol is defined in the same module +(`src/indexers/kubetypes.py`) but was never added to the import list. + +### Workaround (applied) + +Do not pin the version. Our generation rule uses: + +```yaml +resourceTypes: + - buckets.storage.gcp.upbound.io # unversioned — uses preferred API version +``` + +Because the `if version:` branch is skipped, the buggy `KubernetesResourceTypeSpec(...)` +constructor call is never reached and the cluster discovery continues. + +### Proper fix (owed upstream) + +Add the missing name to the import at +[`src/indexers/kubeapi.py:44`](https://github.com/runwhen-contrib/runwhen-local/blob/main/src/indexers/kubeapi.py#L44): + +```python +from .kubetypes import KUBERNETES_PLATFORM, KubernetesResourceType, KubernetesResourceTypeSpec +``` + +One-line PR. No test-suite changes needed; the existing wildcard-de-dup +path just works once the symbol is in scope. + +## Issue #2 — Cluster-scoped CRDs return `404 Not Found` + +### Symptom + +The Crossplane GCP Bucket CRD is cluster-scoped +(`kubectl api-resources` shows `NAMESPACED = false`), but workspace-builder +tries to list it via the namespaced Kubernetes API: + +``` +[INFO] indexers.kubeapi: Trying custom resource buckets.storage.gcp.upbound.io +[INFO] indexers.kubeapi: Error scanning for custom resource instances; + skipping and continuing; + error: (404) Reason: Not Found ... 404 page not found + group=storage.gcp.upbound.io, kind=buckets +``` + +**Impact**: zero Bucket resources are indexed, so zero SLXs are rendered +from our generation rule. Discovery completes with no errors visible in +the summary, which makes this failure quiet unless you grep for `404` in +the logs. + +### Cause + +[`src/indexers/kubeapi.py:1146`](https://github.com/runwhen-contrib/runwhen-local/blob/main/src/indexers/kubeapi.py#L1146) +unconditionally calls: + +```python +custom_objects_api_client.list_namespaced_custom_object( + group=group, + version=version, + namespace=namespace_name, + plural=plural_name, +) +``` + +For a cluster-scoped CRD, the API server has no `/namespaces/{ns}/...` +endpoint and returns 404. The file already knows how to call +`list_cluster_custom_object` (line 645, for the OpenShift `projects` +CRD) — the generic custom-resource loop just doesn't use it. + +### Workaround + +**There is no clean workaround from the codecollection side.** Every +Crossplane managed resource (Bucket, HMACKey, ServiceAccountIAMMember, +Object, etc.) is cluster-scoped by design. Choosing a *namespaced* CRD +sidesteps the bug but abandons the Crossplane story. + +### Proper fix (owed upstream) + +Inspect the CRD scope at index time and pick the correct API. A minimal +patch to the custom-resource loop: + +```python +# Look up scope once per (group, plural) +scope_cache = {} + +def get_scope(group, plural): + if (group, plural) in scope_cache: + return scope_cache[(group, plural)] + try: + # /apis/{group}/{version} lists resources with scope metadata + api_resources = client.CustomObjectsApi().api_client.call_api( + f"/apis/{group}/{version}", "GET", + response_type="object", + auth_settings=["BearerToken"], + )[0] + for r in api_resources.get("resources", []): + if r["name"] == plural: + scope_cache[(group, plural)] = "Namespaced" if r["namespaced"] else "Cluster" + return scope_cache[(group, plural)] + except Exception: + pass + return "Namespaced" # conservative default +``` + +Then, before line 1146: + +```python +if get_scope(group, plural_name) == "Cluster": + # only need to call once, not per-namespace + if namespace_name != first_namespace: + continue + ret = custom_objects_api_client.list_cluster_custom_object( + group=group, version=version, plural=plural_name, + ) +else: + ret = custom_objects_api_client.list_namespaced_custom_object( + group=group, version=version, namespace=namespace_name, plural=plural_name, + ) +``` + +The parsing / registry-registration code below stays the same. + +## What iteration 1 did prove + +Even without Issue #2 fixed, we validated: + +| Step | Result | +|------|--------| +| Private CC scaffold (rules + templates only) | ✅ Complete | +| Push to Gitea with token-in-URL auth | ✅ `git push` and Gitea API confirm content | +| Workspace-builder clones the private CC | ✅ Log line `Cloning from git source: https://***@gitea.airgap.../simple-private-codecollection.git` | +| Generation rule is parsed and loaded | ✅ `Processing generation rule for codebundle: gcp-bucket-crossplane-health` | +| RBAC for cluster-scoped Bucket reads | ✅ `workspace-builder-crossplane-buckets-read` ClusterRoleBinding created; `kubectl auth can-i list buckets.storage.gcp.upbound.io` → `yes` | +| Discovery attempted for the CRD | ✅ `Trying custom resource buckets.storage.gcp.upbound.io` | +| CRD listing succeeded | ❌ Blocked by upstream Issue #2 (404 on namespaced API for cluster-scoped resource) | +| SLXs rendered / uploaded | ❌ Blocked by upstream Issue #2 | + +The pattern is proven **up to the point that runwhen-local supports**. +The rest is a runwhen-local PR away. + +## Next steps + +1. Open two PRs against [runwhen-contrib/runwhen-local](https://github.com/runwhen-contrib/runwhen-local): + - **PR A**: import fix for Issue #1 (one line). + - **PR B**: scope-aware custom-resource discovery for Issue #2. +2. Once PR B is merged and released, rebuild the airgap runwhen-local + image tag and update + [`apps/runwhen-env/airgap/runwhen-runner/helm.yaml`](https://github.com/runwhen/infra-flux-nonprod-shared/blob/main/apps/runwhen-env/airgap/runwhen-runner/helm.yaml). +3. Restart workspace-builder and complete iteration 2 (screenshots of the + rendered SLXs, SLI graphs, and TaskSet execution). + +Alternative interim path — if you need a demo *before* the upstream PRs +land — is to point the generation rule at a **namespaced** CRD as a +proof of the pattern (e.g., Flux `helmreleases.helm.toolkit.fluxcd.io`). +That teaches everything except the specific Crossplane bit and can be +undone once Issue #2 is fixed.