Document runwhen-local upstream limitations found during validation
Iteration 1 validation surfaced two bugs in runwhen-local 0.11.0 that
prevent cluster-scoped CRDs (like Crossplane GCP Buckets) from being
discovered:
1. NameError: KubernetesResourceTypeSpec is not defined
(kubeapi.py:1125 references a symbol never imported)
2. list_namespaced_custom_object returns 404 for cluster-scoped CRDs
(the loop always calls the namespaced API path)
Added docs/08-current-limitations.md with symptoms, causes, and
one-line / small-patch proposed fixes for both. Updated the iteration
checklist in docs/07 to reflect this and added the missing docs entry
to the top-level README.
Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
stewartshea
@@ -0,0 +1,195 @@
|
||||
# 08 — Current limitations
|
||||
|
||||
## What you'll learn here
|
||||
|
||||
Two upstream limitations in `runwhen-local` (verified against image
|
||||
`runwhen-local:0.11.0` and the tip of the `main` branch) that block a
|
||||
"generation-rule only" example targeting **cluster-scoped** Crossplane
|
||||
CRDs like GCP Buckets. These are honest observations from the live
|
||||
validation run we performed during iteration 1 — they are not blockers
|
||||
for teaching the pattern, but they are blockers for observing generated
|
||||
SLXs in the RunWhen UI. This chapter lists the issues, the workaround
|
||||
we already applied, and the fixes still owing.
|
||||
|
||||
## Issue #1 — `KubernetesResourceTypeSpec` `NameError` when a resource type is version-pinned
|
||||
|
||||
### Symptom
|
||||
|
||||
When a generation rule uses the version-pinned form
|
||||
`plural.group/version` (for example `buckets.storage.gcp.upbound.io/v1beta1`),
|
||||
`kubeapi.py`'s custom-resource loop raises:
|
||||
|
||||
```
|
||||
[ERROR] indexers.kubeapi: Error processing cluster 'default' from context 'default':
|
||||
name 'KubernetesResourceTypeSpec' is not defined
|
||||
[INFO] indexers.kubeapi: Skipping cluster 'default' and continuing with next cluster
|
||||
```
|
||||
|
||||
**Impact**: the entire Kubernetes cluster's custom-resource enumeration
|
||||
is skipped, so NO CRD-based generation rule for that cluster produces
|
||||
SLXs. (Built-in K8s resource discovery still works.)
|
||||
|
||||
### Cause
|
||||
|
||||
[`src/indexers/kubeapi.py`](https://github.com/runwhen-contrib/runwhen-local/blob/main/src/indexers/kubeapi.py)
|
||||
references `KubernetesResourceTypeSpec` on line 1125 but only imports
|
||||
`KubernetesResourceType` (line 44):
|
||||
|
||||
```python
|
||||
from .kubetypes import KUBERNETES_PLATFORM, KubernetesResourceType
|
||||
```
|
||||
|
||||
The `KubernetesResourceTypeSpec` symbol is defined in the same module
|
||||
(`src/indexers/kubetypes.py`) but was never added to the import list.
|
||||
|
||||
### Workaround (applied)
|
||||
|
||||
Do not pin the version. Our generation rule uses:
|
||||
|
||||
```yaml
|
||||
resourceTypes:
|
||||
- buckets.storage.gcp.upbound.io # unversioned — uses preferred API version
|
||||
```
|
||||
|
||||
Because the `if version:` branch is skipped, the buggy `KubernetesResourceTypeSpec(...)`
|
||||
constructor call is never reached and the cluster discovery continues.
|
||||
|
||||
### Proper fix (owed upstream)
|
||||
|
||||
Add the missing name to the import at
|
||||
[`src/indexers/kubeapi.py:44`](https://github.com/runwhen-contrib/runwhen-local/blob/main/src/indexers/kubeapi.py#L44):
|
||||
|
||||
```python
|
||||
from .kubetypes import KUBERNETES_PLATFORM, KubernetesResourceType, KubernetesResourceTypeSpec
|
||||
```
|
||||
|
||||
One-line PR. No test-suite changes needed; the existing wildcard-de-dup
|
||||
path just works once the symbol is in scope.
|
||||
|
||||
## Issue #2 — Cluster-scoped CRDs return `404 Not Found`
|
||||
|
||||
### Symptom
|
||||
|
||||
The Crossplane GCP Bucket CRD is cluster-scoped
|
||||
(`kubectl api-resources` shows `NAMESPACED = false`), but workspace-builder
|
||||
tries to list it via the namespaced Kubernetes API:
|
||||
|
||||
```
|
||||
[INFO] indexers.kubeapi: Trying custom resource buckets.storage.gcp.upbound.io
|
||||
[INFO] indexers.kubeapi: Error scanning for custom resource instances;
|
||||
skipping and continuing;
|
||||
error: (404) Reason: Not Found ... 404 page not found
|
||||
group=storage.gcp.upbound.io, kind=buckets
|
||||
```
|
||||
|
||||
**Impact**: zero Bucket resources are indexed, so zero SLXs are rendered
|
||||
from our generation rule. Discovery completes with no errors visible in
|
||||
the summary, which makes this failure quiet unless you grep for `404` in
|
||||
the logs.
|
||||
|
||||
### Cause
|
||||
|
||||
[`src/indexers/kubeapi.py:1146`](https://github.com/runwhen-contrib/runwhen-local/blob/main/src/indexers/kubeapi.py#L1146)
|
||||
unconditionally calls:
|
||||
|
||||
```python
|
||||
custom_objects_api_client.list_namespaced_custom_object(
|
||||
group=group,
|
||||
version=version,
|
||||
namespace=namespace_name,
|
||||
plural=plural_name,
|
||||
)
|
||||
```
|
||||
|
||||
For a cluster-scoped CRD, the API server has no `/namespaces/{ns}/...`
|
||||
endpoint and returns 404. The file already knows how to call
|
||||
`list_cluster_custom_object` (line 645, for the OpenShift `projects`
|
||||
CRD) — the generic custom-resource loop just doesn't use it.
|
||||
|
||||
### Workaround
|
||||
|
||||
**There is no clean workaround from the codecollection side.** Every
|
||||
Crossplane managed resource (Bucket, HMACKey, ServiceAccountIAMMember,
|
||||
Object, etc.) is cluster-scoped by design. Choosing a *namespaced* CRD
|
||||
sidesteps the bug but abandons the Crossplane story.
|
||||
|
||||
### Proper fix (owed upstream)
|
||||
|
||||
Inspect the CRD scope at index time and pick the correct API. A minimal
|
||||
patch to the custom-resource loop:
|
||||
|
||||
```python
|
||||
# Look up scope once per (group, plural)
|
||||
scope_cache = {}
|
||||
|
||||
def get_scope(group, plural):
|
||||
if (group, plural) in scope_cache:
|
||||
return scope_cache[(group, plural)]
|
||||
try:
|
||||
# /apis/{group}/{version} lists resources with scope metadata
|
||||
api_resources = client.CustomObjectsApi().api_client.call_api(
|
||||
f"/apis/{group}/{version}", "GET",
|
||||
response_type="object",
|
||||
auth_settings=["BearerToken"],
|
||||
)[0]
|
||||
for r in api_resources.get("resources", []):
|
||||
if r["name"] == plural:
|
||||
scope_cache[(group, plural)] = "Namespaced" if r["namespaced"] else "Cluster"
|
||||
return scope_cache[(group, plural)]
|
||||
except Exception:
|
||||
pass
|
||||
return "Namespaced" # conservative default
|
||||
```
|
||||
|
||||
Then, before line 1146:
|
||||
|
||||
```python
|
||||
if get_scope(group, plural_name) == "Cluster":
|
||||
# only need to call once, not per-namespace
|
||||
if namespace_name != first_namespace:
|
||||
continue
|
||||
ret = custom_objects_api_client.list_cluster_custom_object(
|
||||
group=group, version=version, plural=plural_name,
|
||||
)
|
||||
else:
|
||||
ret = custom_objects_api_client.list_namespaced_custom_object(
|
||||
group=group, version=version, namespace=namespace_name, plural=plural_name,
|
||||
)
|
||||
```
|
||||
|
||||
The parsing / registry-registration code below stays the same.
|
||||
|
||||
## What iteration 1 did prove
|
||||
|
||||
Even without Issue #2 fixed, we validated:
|
||||
|
||||
| Step | Result |
|
||||
|------|--------|
|
||||
| Private CC scaffold (rules + templates only) | ✅ Complete |
|
||||
| Push to Gitea with token-in-URL auth | ✅ `git push` and Gitea API confirm content |
|
||||
| Workspace-builder clones the private CC | ✅ Log line `Cloning from git source: https://***@gitea.airgap.../simple-private-codecollection.git` |
|
||||
| Generation rule is parsed and loaded | ✅ `Processing generation rule for codebundle: gcp-bucket-crossplane-health` |
|
||||
| RBAC for cluster-scoped Bucket reads | ✅ `workspace-builder-crossplane-buckets-read` ClusterRoleBinding created; `kubectl auth can-i list buckets.storage.gcp.upbound.io` → `yes` |
|
||||
| Discovery attempted for the CRD | ✅ `Trying custom resource buckets.storage.gcp.upbound.io` |
|
||||
| CRD listing succeeded | ❌ Blocked by upstream Issue #2 (404 on namespaced API for cluster-scoped resource) |
|
||||
| SLXs rendered / uploaded | ❌ Blocked by upstream Issue #2 |
|
||||
|
||||
The pattern is proven **up to the point that runwhen-local supports**.
|
||||
The rest is a runwhen-local PR away.
|
||||
|
||||
## Next steps
|
||||
|
||||
1. Open two PRs against [runwhen-contrib/runwhen-local](https://github.com/runwhen-contrib/runwhen-local):
|
||||
- **PR A**: import fix for Issue #1 (one line).
|
||||
- **PR B**: scope-aware custom-resource discovery for Issue #2.
|
||||
2. Once PR B is merged and released, rebuild the airgap runwhen-local
|
||||
image tag and update
|
||||
[`apps/runwhen-env/airgap/runwhen-runner/helm.yaml`](https://github.com/runwhen/infra-flux-nonprod-shared/blob/main/apps/runwhen-env/airgap/runwhen-runner/helm.yaml).
|
||||
3. Restart workspace-builder and complete iteration 2 (screenshots of the
|
||||
rendered SLXs, SLI graphs, and TaskSet execution).
|
||||
|
||||
Alternative interim path — if you need a demo *before* the upstream PRs
|
||||
land — is to point the generation rule at a **namespaced** CRD as a
|
||||
proof of the pattern (e.g., Flux `helmreleases.helm.toolkit.fluxcd.io`).
|
||||
That teaches everything except the specific Crossplane bit and can be
|
||||
undone once Issue #2 is fixed.
|
||||
Reference in New Issue
Block a user