template.kicker
Operator Installation Guide
How to install and configure nebua-operator, the optional in-cluster operator for self-healing, policy enforcement, and add-on lifecycle
Nebua Operator Installation Guide
nebua-operator is the optional in-cluster Kubernetes operator for advanced NebuaCloud platform features. Unlike nebua-agent, which runs on the host and connects outbound to NebuaCloud, the operator runs entirely inside the cluster using an in-cluster service account and reconciles Kubernetes-native resources and NebuaCloud CRDs directly. See Nebua Runtime Components for how it fits alongside the agent and gateway.
Install nebua-operator when you need any of:
- self-healing and bounded workload remediation (crash loops, image pull failures, unavailable replicas, node pressure, stuck PVCs, networking symptoms)
- policy enforcement through
NebuaPolicyand admission policy providers - backup orchestration through
NebuaBackupPlan - GitOps enforcement and drift control
- add-on install, upgrade, health, drift detection, and repair
The platform still supports basic cluster management through nebua-agent alone — the operator is optional.
Prerequisites
- A working
kubectlcontext with cluster-admin (or equivalent) permissions on the target cluster - Cluster reachable from wherever you run the installer (the installer applies manifests through your local
kubectl) - Outbound HTTPS access from inside the cluster to
github.com/objects.githubusercontent.com— the operator Deployment'sbundle-fetchinitContainer downloads the compiled operator bundle at pod startup, same asnebua-agent/nebua-gatewaydo on their hosts - A namespace for the operator, default
nebua-system(created automatically)
Install
There are three ways to install nebua-operator, in order of convenience:
1. Hosted bundle installer
curl -fsSL https://nebuacloud.com/install/nebua-operator.sh | bash -s -- \
--namespace nebua-system
This downloads the same Nebua agent bundle used by nebua-agent, then runs the bundled nebua-operator-install.sh from inside it. No sudo is required — the script only calls kubectl, it does not touch the host. There is no separate nebua-operator container image to pull: the Deployment runs a stock node:20-alpine, and an initContainer fetches the same public bundle tarball directly inside the cluster and unpacks the compiled operator into a shared volume the main container runs from.
2. Local source installer
If you already have the monorepo checked out:
./nebua-operator-install.sh \
--namespace nebua-system
3. Manual manifest install
For full control, apply the manifests directly from agents/nebua-operator/manifests:
kubectl apply -f manifests/crds/
kubectl apply -f manifests/rbac.yaml
kubectl apply -f manifests/deployment.yaml
Installer options
--namespace NAME Namespace for nebua-operator, default nebua-system
--bundle-platform PLATFORM Bundle platform, default linux-x64
--bundle-version VERSION Bundle release tag, default latest
--bundle-repo OWNER/REPO Bundle release repo, default nebua/agent-releases
--bundle-url URL Exact bundle tarball URL, overrides version/repo/platform
--watch-namespace NAME Optional namespace scope; omit for cluster-wide watch
--healing-enabled true|false Enable operator self-healing, default true
--skip-crds Do not apply platform.nebua.io CRDs
--no-wait Do not wait for deployment rollout
--kubeconfig PATH Kubeconfig path for kubectl
--context NAME Kubeconfig context for kubectl
The installer applies the platform.nebua.io CRDs, RBAC, and Deployment, points the Deployment's bundle-fetch initContainer at the requested bundle version/repo/URL and sets the OPERATOR_HEALING_ENABLED environment variable on the main container, and (unless --no-wait) waits for the rollout to finish.
Runtime Configuration
The operator Deployment reads these environment variables:
OPERATOR_HEALING_ENABLED:trueorfalse, defaulttrueOPERATOR_HEALING_COOLDOWN_MS: minimum time between remediation attempts on the same resource, default300000(5 minutes)OPERATOR_HEALING_MAX_BACKOFF_MS: maximum backoff between attempts, default3600000(1 hour)OPERATOR_HEALING_MAX_ATTEMPTS: attempts before falling back to a recommendation instead of another action, default3OPERATOR_HEALING_STUCK_RESOURCE_MS: how long a resource must be stuck before a strategy considers it, default1200000(20 minutes)HEALING_EVENT_NAMESPACE: namespace healing Events are emitted into, defaultnebua-systemWATCH_NAMESPACE: set by--watch-namespace; omit to watch the whole cluster
Change these after install with kubectl -n nebua-system set env deployment/nebua-operator KEY=VALUE, then let the rollout restart the pod.
Verify Installation
kubectl -n nebua-system get deployment nebua-operator
kubectl -n nebua-system rollout status deployment/nebua-operator
kubectl -n nebua-system logs deployment/nebua-operator -f
Confirm the CRDs registered:
kubectl get crd | grep platform.nebua.io
You should see nebuapolicies.platform.nebua.io, nebuabackupplans.platform.nebua.io, and nebuaremediations.platform.nebua.io.
Self-Healing Framework
Once running, the operator watches native Kubernetes resources and reconciles them through independent strategies: CrashLoopBackOffStrategy, ImagePullBackOffStrategy, WorkloadReplicaStrategy, NodeHealthStrategy, StorageStrategy, NetworkingStrategy, AutoscalingStrategy, and PolicyStrategy. Each strategy watches, detects, evaluates a safety window, decides whether to act or only recommend, executes a bounded remediation, and reports a Kubernetes Event for the NebuaCloud dashboard to ingest.
Safety state is persisted as annotations on the affected resource (healing.nebua.io/attempts, healing.nebua.io/last-strategy, healing.nebua.io/last-action-at, healing.nebua.io/next-action-at, healing.nebua.io/last-result), so the operator will not loop endlessly — once attempt/cooldown limits are hit it reports a recommendation or a manual-intervention Event instead of retrying.
Custom Resources
The initial API group is platform.nebua.io/v1alpha1:
NebuaRemediation: declarative remediation requests such asrestartDeployment,cordonNode,uncordonNode, andannotateNamespaceNebuaPolicy: policy/compliance intent inauditorenforcemode, enforced natively throughValidatingAdmissionPolicyNebuaBackupPlan: backup orchestration intent for provider-specific backup controllers
Troubleshooting
Operator pod not starting
kubectl -n nebua-system describe deployment nebua-operator
kubectl -n nebua-system get events --sort-by=.lastTimestamp
Common causes:
- RBAC not applied (
manifests/rbac.yaml) — the operator's service account lacks permission to watch/patch resources - CRDs not applied — the operator crashes on start if
platform.nebua.ioCRDs are missing and--skip-crdswas used unintentionally bundle-fetchinitContainer failing — checkkubectl -n nebua-system logs deployment/nebua-operator -c bundle-fetch; usually a bad--bundle-url/--bundle-version/--bundle-repovalue or no outbound HTTPS access from the cluster to GitHub
Healing is not happening
- Confirm
OPERATOR_HEALING_ENABLED=trueon the Deployment - Check whether a resource is inside its
OPERATOR_HEALING_COOLDOWN_MSwindow or has exhaustedOPERATOR_HEALING_MAX_ATTEMPTS— look at the resource'shealing.nebua.io/*annotations - Check
kubectl -n nebua-system logs deployment/nebua-operatorfor the specific strategy's decision
Uninstall
Remove the Deployment and RBAC
kubectl -n nebua-system delete deployment nebua-operator
kubectl delete clusterrolebinding nebua-operator
kubectl delete clusterrole nebua-operator
kubectl -n nebua-system delete serviceaccount nebua-operator
Replace nebua-system with whatever --namespace you installed into. Stopping the Deployment first (before removing CRDs below) avoids the operator logging reconcile errors against resources it can no longer find.
Remove the CRDs
kubectl delete crd nebuapolicies.platform.nebua.io nebuabackupplans.platform.nebua.io nebuaremediations.platform.nebua.io
This deletes every NebuaPolicy, NebuaBackupPlan, and NebuaRemediation object in the cluster along with the CRDs — Kubernetes always garbage-collects custom resources when their CRD is removed. Check what exists first if you're not sure:
kubectl get nebuapolicies,nebuabackupplans,nebuaremediations -A
Skip this step if you plan to reinstall the operator later and want to keep existing policy/backup-plan definitions.
Remove the namespace
Only do this if nebua-system (or your chosen --namespace) was created solely for the operator and holds nothing else:
kubectl delete namespace nebua-system
Uninstalling the operator disables self-healing, policy enforcement, and add-on reconciliation for the cluster. nebua-agent keeps working independently — basic cluster management, k3s bootstrap, and health reporting are unaffected.
Security Considerations
- The operator uses in-cluster service account credentials only; it does not need outbound internet access to function.
- Scope
--watch-namespaceif you want to limit the operator to a subset of the cluster instead of cluster-wide. - Review the RBAC manifest (
manifests/rbac.yaml) before applying in a shared or regulated cluster — it grants the permissions the healing strategies and policy enforcement need. - Healing actions are bounded and reported as Kubernetes Events; review
HEALING_EVENT_NAMESPACEevents regularly if healing is enabled.