simple-private-codecollection/docs/08-current-limitations.md

08 — Current limitations

What you'll learn here

Custom-resource discovery in runwhen-local works today for the common case — every existing generation rule in rw-cli-codecollection that targets a CRD (Flux HelmRelease, cert-manager Certificate, Prometheus, ArgoCD Application, Crunchy/Zalando Postgres, etc.) discovers and renders SLXs correctly. Iteration 1's validation, however, surfaced two narrow, conditional issues in the current code path (verified against image runwhen-local:0.11.0 and the tip of main) that together prevent this specific example from rendering SLXs today:

1. A NameError triggered only when a generation rule pins a version (e.g. plural.group/v1beta1). Existing rules that omit the version are unaffected — that's why nothing broke before.
2. A namespaced-only listing call for CRDs, which returns 404 for cluster-scoped CRDs. Every existing CRD referenced by rw-cli-codecollection is namespaced, so this had no prior surface. Crossplane managed resources like buckets.storage.gcp.upbound.io are the first cluster-scoped CRDs the pattern has been used against.

Both are small, contained fixes. The larger "generation-rule only" pattern this codecollection is teaching is otherwise fully working — the workspace-builder clones the private CC, loads the generation rule, and attempts discovery correctly.

Issue #1 — KubernetesResourceTypeSpec NameError when a resource type is version-pinned

Symptom

When a generation rule uses the version-pinned form plural.group/version (for example buckets.storage.gcp.upbound.io/v1beta1), kubeapi.py's custom-resource loop raises:

[ERROR] indexers.kubeapi: Error processing cluster 'default' from context 'default':
name 'KubernetesResourceTypeSpec' is not defined
[INFO]  indexers.kubeapi: Skipping cluster 'default' and continuing with next cluster

Impact: the entire Kubernetes cluster's custom-resource enumeration is skipped, so NO CRD-based generation rule for that cluster produces SLXs. (Built-in K8s resource discovery still works.)

Cause

src/indexers/kubeapi.py references KubernetesResourceTypeSpec on line 1125 but only imports KubernetesResourceType (line 44):

from .kubetypes import KUBERNETES_PLATFORM, KubernetesResourceType

The KubernetesResourceTypeSpec symbol is defined in the same module (src/indexers/kubetypes.py) but was never added to the import list.

Workaround (applied)

Do not pin the version. Our generation rule uses:

resourceTypes:
  - buckets.storage.gcp.upbound.io       # unversioned — uses preferred API version

Because the if version: branch is skipped, the buggy KubernetesResourceTypeSpec(...) constructor call is never reached and the cluster discovery continues.

Proper fix (owed upstream)

Add the missing name to the import at src/indexers/kubeapi.py:44:

from .kubetypes import KUBERNETES_PLATFORM, KubernetesResourceType, KubernetesResourceTypeSpec

One-line PR. No test-suite changes needed; the existing wildcard-de-dup path just works once the symbol is in scope.

Issue #2 — Cluster-scoped CRDs return 404 Not Found

Symptom

The Crossplane GCP Bucket CRD is cluster-scoped (kubectl api-resources shows NAMESPACED = false), but workspace-builder tries to list it via the namespaced Kubernetes API:

[INFO] indexers.kubeapi: Trying custom resource buckets.storage.gcp.upbound.io
[INFO] indexers.kubeapi: Error scanning for custom resource instances;
    skipping and continuing;
    error: (404) Reason: Not Found ... 404 page not found
    group=storage.gcp.upbound.io, kind=buckets

Impact: zero Bucket resources are indexed, so zero SLXs are rendered from our generation rule. Discovery completes with no errors visible in the summary, which makes this failure quiet unless you grep for 404 in the logs.

Cause

src/indexers/kubeapi.py:1146 unconditionally calls:

custom_objects_api_client.list_namespaced_custom_object(
    group=group,
    version=version,
    namespace=namespace_name,
    plural=plural_name,
)

For a cluster-scoped CRD, the API server has no /namespaces/{ns}/... endpoint and returns 404. The file already knows how to call list_cluster_custom_object (line 645, for the OpenShift projects CRD) — the generic custom-resource loop just doesn't use it.

Workaround

There is no clean workaround from the codecollection side. Every Crossplane managed resource (Bucket, HMACKey, ServiceAccountIAMMember, Object, etc.) is cluster-scoped by design. Choosing a namespaced CRD sidesteps the bug but abandons the Crossplane story.

Proper fix (owed upstream)

Inspect the CRD scope at index time and pick the correct API. A minimal patch to the custom-resource loop:

# Look up scope once per (group, plural)
scope_cache = {}

def get_scope(group, plural):
    if (group, plural) in scope_cache:
        return scope_cache[(group, plural)]
    try:
        # /apis/{group}/{version} lists resources with scope metadata
        api_resources = client.CustomObjectsApi().api_client.call_api(
            f"/apis/{group}/{version}", "GET",
            response_type="object",
            auth_settings=["BearerToken"],
        )[0]
        for r in api_resources.get("resources", []):
            if r["name"] == plural:
                scope_cache[(group, plural)] = "Namespaced" if r["namespaced"] else "Cluster"
                return scope_cache[(group, plural)]
    except Exception:
        pass
    return "Namespaced"  # conservative default

Then, before line 1146:

if get_scope(group, plural_name) == "Cluster":
    # only need to call once, not per-namespace
    if namespace_name != first_namespace:
        continue
    ret = custom_objects_api_client.list_cluster_custom_object(
        group=group, version=version, plural=plural_name,
    )
else:
    ret = custom_objects_api_client.list_namespaced_custom_object(
        group=group, version=version, namespace=namespace_name, plural=plural_name,
    )

The parsing / registry-registration code below stays the same.

What iteration 1 did prove

Even without Issue #2 fixed, we validated:

Step	Result
Private CC scaffold (rules + templates only)	✅ Complete
Push to Gitea with token-in-URL auth	✅ git push and Gitea API confirm content
Workspace-builder clones the private CC	✅ Log line Cloning from git source: https://***@gitea.airgap.../simple-private-codecollection.git
Generation rule is parsed and loaded	✅ Processing generation rule for codebundle: gcp-bucket-crossplane-health
RBAC for cluster-scoped Bucket reads	✅ workspace-builder-crossplane-buckets-read ClusterRoleBinding created; kubectl auth can-i list buckets.storage.gcp.upbound.io → yes
Discovery attempted for the CRD	✅ Trying custom resource buckets.storage.gcp.upbound.io
CRD listing succeeded	❌ Blocked by upstream Issue #2 (404 on namespaced API for cluster-scoped resource)
SLXs rendered / uploaded	❌ Blocked by upstream Issue #2

The pattern is proven up to the point that runwhen-local supports. The rest is a runwhen-local PR away.

Next steps

1. Open two PRs against runwhen-contrib/runwhen-local:
   • PR A: import fix for Issue #1 (one line).
   • PR B: scope-aware custom-resource discovery for Issue #2.
2. Once PR B is merged and released, rebuild the airgap runwhen-local image tag and update apps/runwhen-env/airgap/runwhen-runner/helm.yaml.
3. Restart workspace-builder and complete iteration 2 (screenshots of the rendered SLXs, SLI graphs, and TaskSet execution).

Alternative interim path — if you need a demo before the upstream PRs land — is to point the generation rule at a namespaced CRD as a proof of the pattern (e.g., Flux helmreleases.helm.toolkit.fluxcd.io). That teaches everything except the specific Crossplane bit and can be undone once Issue #2 is fixed.