# RegScale Orchestration Hub (ROH) Helm Chart Deploys the RegScale Orchestration Hub — the app tier, per-queue Celery workers, Celery Beat, and a migration hook — on AKS, EKS, or GKE. ROH talks to a single core RegScale instance (one-way) and needs its own SQL Server database and Redis instance; it does not share the core chart's database or Redis. This chart is a sibling of the root `regscale` chart in this repository and does not modify it. It reuses the same conventions: `_helpers.tpl` naming/labels (helper prefix `roh.`), the `existingSecret` pattern, the `build-ca-trust` init container, a restricted Pod Security Context, and a render-time `fail`-based `validateValues` guard. ## Prerequisites - Kubernetes 1.23+ (the chart's binding APIs are `autoscaling/v2` and `policy/v1`), Helm 3.0+. - A reachable SQL Server database (managed, e.g. Azure SQL/RDS SQL Server/Cloud SQL for SQL Server, or the chart's dev/test in-cluster StatefulSet — see [Database](#database)). - A reachable Redis instance (managed, or the chart's in-cluster Bitnami `redis` subchart — see [Redis](#redis)). - The core RegScale application already installed and reachable (see [Install order](#install-order-r11)). - **If `workers.scaling.mode: keda` (the default): the KEDA operator is a cluster prerequisite.** This chart does not install it. Install it once per cluster before installing ROH: ```bash helm repo add kedacore https://kedacore.github.io/charts helm repo update helm install keda kedacore/keda --namespace keda --create-namespace ``` ## Installing ```bash helm repo add bitnami https://charts.bitnami.com/bitnami helm repo add synqly https://helm.synqly.com/stable cd regscale-orchestration-hub helm dependency build . # Without Synqly Embedded; for it, add -f values-synqly.yaml (see "Synqly Embedded"). # coreProxy puts the chart's proxy in front of core's Service: core 6.34 fails every # ROH login it does not see as HTTPS (see "Core wiring"). helm install roh . \ --namespace regscale --create-namespace \ --set regscaleUrl=http://regscale-svc.regscale.svc.cluster.local \ --set coreProxy.enabled=true \ --set synqly.enabled=false \ --set prefixNamespace=acme-prod \ --set secrets.secretKey=$(openssl rand -hex 32) \ --set secrets.encryptionKey=$(openssl rand -hex 16) \ --set secrets.databaseUrl='mssql+pymssql://user:pass@sql-host:1433/db' \ --set secrets.metricsToken=$(openssl rand -hex 32) \ --set redis.auth.password=$(openssl rand -hex 32) ``` For a real deployment, prefer one of the per-cloud overlays and an `existingSecret` rather than passing secret material on the command line (see [Per-cloud overlays](#per-cloud-overlays) and [Secrets](#secrets--existingsecret)): ```bash helm install roh . -f values-aks.yaml --set synqly.enabled=false --namespace regscale --create-namespace ``` `secrets.encryptionKey` must be **exactly 32 characters**, which is what `openssl rand -hex 16` produces. `secrets.secretKey` has a 32-character minimum, so `-hex 32` is valid there. Supplying a 64-character encryption key renders and installs cleanly but fails inside the migration hook; the chart now rejects it at render time. **The application tier requires a relaxed security context.** See [Application tier security context](#application-tier-security-context) below — without it both app replicas enter `CrashLoopBackOff`. `helm template`/`helm install`/`helm upgrade` will `fail` at render time with a descriptive error if required values are missing — see [Render-time validation](#render-time-validation). A step-by-step installation guide, including per-cloud prerequisites and a worked example of running this chart alongside the core RegScale chart, is published at [regscale.github.io/helm/roh-setup.html](https://regscale.github.io/helm/roh-setup.html). ## Values schema This section documents the keys you are most likely to change. See `values.yaml` for the full annotated default file, and `values-aks.yaml` / `values-eks.yaml` / `values-gke.yaml` for complete per-cloud examples. ### Image ```yaml image: registry: docker.io appRepository: regscale/orchestration-hub-app workerRepository: regscale/orchestration-hub-worker tag: "" # defaults to .Chart.AppVersion pullPolicy: IfNotPresent pullSecrets: [] global: imageRegistry: "" # override registry host for air-gapped / ECR / Artifact Registry mirrors imagePullSecrets: [] ``` `global.imageRegistry`, when set, replaces `image.registry` for both the app and worker images (R9) — use it to point at a private mirror without editing `image.appRepository`/`image.workerRepository`. Both images are published to public Docker Hub ([`regscale/orchestration-hub-app`](https://hub.docker.com/r/regscale/orchestration-hub-app), [`regscale/orchestration-hub-worker`](https://hub.docker.com/r/regscale/orchestration-hub-worker)), so no `imagePullSecrets` are required for a default install. `global.imageRegistry` also applies to the core proxy image, but not to Synqly Embedded: its images come from `quay.io/synqly`. For a mirror, set `synqly.embedded.registry` and `synqly.managementUI.registry`. `image.tag` defaults to `.Chart.AppVersion`; pin it explicitly if you need a version other than the one this chart ships against. ### Core wiring ```yaml regscaleUrl: "" # REQUIRED. ROH -> core RegScale, one-way. In-cluster svc DNS or external URL. prefixNamespace: "" # REQUIRED. Unique, non-default. Also drives the KEDA listName. logLevel: INFO debug: false # DEBUG=False activates ROH's production validators ``` Core 6.34 fails every ROH login that it does not see as HTTPS. With an in-cluster `http://` `regscaleUrl`, turn on the core proxy: ```yaml coreProxy: enabled: false # the cloud overlays set true replicas: 2 # plus a PodDisruptionBudget (pdb.minAvailable: 1) dnsResolver: kube-dns.kube-system.svc.cluster.local clientMaxBodySize: 100m # core's default upload limit readTimeout: 600s ``` It runs nginx pods that set `X-Forwarded-Proto: https` and forward to `regscaleUrl`, and ROH calls them instead of core. nginx resolves `regscaleUrl` per request without search domains, so it must be core's full Service name (`-svc..svc.cluster.local`); the chart stops otherwise. The proxy does nothing for an `https://` `regscaleUrl`. Set `ForwardedHeaders` on core too: see [roh-setup.md, "Make core see ROH logins as HTTPS"](../roh-setup.md#make-core-see-roh-logins-as-https). `prefixNamespace` becomes `ROH_PREFIX_NAMESPACE` and is embedded verbatim in the Redis key prefix Celery/KEDA use to address queues (`{roh:}.` — see [KEDA scaling](#keda-scaling-default)). Two ROH releases sharing a Redis instance/db **must** use different `prefixNamespace` values or their queues will collide. ### Secrets / `existingSecret` ```yaml secrets: existingSecret: "" # name of a pre-created Secret; overrides the fields below secretKey: "" # SECRET_KEY (chart-created secret only) encryptionKey: "" # SECRETS_ENCRYPTION_KEY (chart-created secret only) — IMMUTABLE, see below databaseUrl: "" # DATABASE_URL, pymssql form (chart-created secret only) metricsToken: "" # METRICS_TOKEN (chart-created secret only) ``` If `secrets.existingSecret` is set, the chart does not create a Secret and instead references yours by name for `envFrom`. **Your secret must use these exact keys**, since every pod consumes them by canonical env-var name: `SECRET_KEY`, `SECRETS_ENCRYPTION_KEY`, `DATABASE_URL`, `REDIS_URL`, `METRICS_TOKEN`. Do **not** put `ROH_BROKER_URL` or `ROH_RESULT_BACKEND` in your secret — the chart derives them from `REDIS_URL` on every pod (see [Single Redis endpoint (R14)](#single-redis-endpoint-wiring-r14)). **`SECRETS_ENCRYPTION_KEY` is immutable (R3).** It decrypts AES-256 data already at rest. The chart never generates it — you always supply the value, and once anything has been encrypted with it, it cannot be rotated by re-running `helm upgrade` with a new value. Back it up outside the cluster. Treat losing it as unrecoverable data loss for encrypted fields. ### Render-time validation `helm template`/`install`/`upgrade` fails fast (via the chart's `validateValues` helper) when, and only when, the chart itself would create a Secret and required fields are missing: - `prefixNamespace` is always required, even with `existingSecret`. - Without `existingSecret`: `secrets.secretKey`, `secrets.encryptionKey` are required; `secrets.databaseUrl` is required unless `database.incluster.enabled`; `redis.auth.password` is required when `redis.mode: incluster` and `redis.auth.enabled` (the default). - `redis.managed.url` is required whenever `redis.mode: managed`. - With `synqly.enabled` (the default, and also when the key is missing, as in an upgrade from 0.2.0 with `--reuse-values`): `global.organizationAdmin.default.secretRef` is required (or `inviteMembers` with SMTP); `global.organizationID`, `organizationFullName` and `apiAddress` must be set and not `REPLACE-`; Vault AppRole or token auth must use `secretRef`; and no Synqly credential may be set as a plain value, the onboarding Member's `synqlyIntegration.onboardingMember.secret` and `.password` included. See [Synqly Embedded](#synqly-embedded). - With `coreProxy.enabled` and an `http://` `regscaleUrl`, `regscaleUrl` must be a full Service name with no path. ### App ```yaml app: replicaCount: 2 database: {poolSize: 10, maxOverflow: 5} hpa: {enabled: true, minReplicas: 2, maxReplicas: 8, targetCPUUtilizationPercentage: 70} service: {type: ClusterIP, port: 80} ingress: {enabled: false, className: nginx, websocket: true, hosts: [], tls: []} pdb: {enabled: true, minAvailable: 1} podSecurityContext: {runAsNonRoot: true, runAsUser: 1001, runAsGroup: 1001, fsGroup: 1001} securityContext: allowPrivilegeEscalation: false readOnlyRootFilesystem: true runAsNonRoot: true capabilities: {drop: [ALL]} probes: {liveness: {...}, readiness: {...}, startup: {enabled: false}} ``` #### Application tier security context The defaults above are hardened, and the application image cannot run under them. Its entrypoint is `supervisord`, which manages nginx and gunicorn as separate processes and must start as root in order to change to the service account user. Installing with the defaults leaves both app replicas in `CrashLoopBackOff` reporting `Error: Can't drop privilege as nonroot user`. Override the security context for the application tier only: ```yaml app: podSecurityContext: runAsNonRoot: false runAsUser: 0 runAsGroup: 0 fsGroup: 0 securityContext: runAsNonRoot: false runAsUser: 0 allowPrivilegeEscalation: true readOnlyRootFilesystem: false capabilities: {drop: []} ``` `capabilities: {drop: []}` is required. Helm replaces list values rather than merging them, but only where the key is supplied; omitting it leaves `drop: [ALL]` in effect, which removes the `SETUID`/`SETGID` capabilities `supervisord` needs and the `CHOWN` capability nginx needs. The worker and beat tiers are unaffected and retain the hardened defaults. They run Celery directly as an unprivileged user. A permanent resolution requires a rootless application image; the chart defaults are left hardened so that no chart change is needed when that ships. `app.ingress.websocket: true` adds long (`86400`s) nginx-specific proxy read/send-timeout annotations for the long-lived `/api/v1/executions` connections. This flag always emits the same `nginx.ingress.kubernetes.io/*` annotations regardless of `app.ingress.className` — on the ALB/GCE overlays (`values-eks.yaml`, `values-gke.yaml`), which also leave `websocket: true`, those nginx annotations are simply ignored as no-ops by the ALB/GCE controllers. On non-nginx controllers, set the equivalent long-timeout behavior yourself via `app.ingress.annotations` (e.g. ALB's `alb.ingress.kubernetes.io/idle-timeout.timeout_seconds`, or the GCE BackendConfig's `timeoutSec`) rather than relying on `websocket: true`. `app.database.{poolSize,maxOverflow}` sets the app tier's SQLAlchemy pool — see [DB pool sizing vs. SQL connection cap (R2)](#db-pool-sizing-vs-sql-connection-cap-r2). `app.probes.readiness` currently points at the open `/api/v1/health` endpoint — see [App-repo dependencies](#app-repo-dependencies-not-shipped-by-this-chart). `app.pdb` protects the app Deployment from **voluntary node drains only**. It does not protect against `helm upgrade` rollouts — see [Drain / upgrade behavior (R4, §7)](#drain--upgrade-behavior-r4-7). ### Workers / scaling ```yaml workers: scaling: {mode: keda} # keda | hpa | static database: {poolSize: 3, maxOverflow: 2} defaults: {terminationGracePeriodSeconds: 15300, nodeSelector: {}, tolerations: [], affinity: {}} queues: cli_tasks: {min: 0, max: 20, queueLength: 4, gracePeriod: 15300, resources: {...}} coordinator: {min: 0, max: 10, queueLength: 2, gracePeriod: 15300, resources: {...}} priority: {min: 1, max: 3, queueLength: 5, gracePeriod: 120, resources: {...}} celery: {min: 0, max: 2, queueLength: 5, gracePeriod: 120, resources: {...}} keda: cooldownPeriod: 300 pollingInterval: 30 ``` One Deployment is rendered per entry in `workers.queues`, running `celery ... worker -Q --concurrency=1`. `workers.scaling.mode` selects the scaling engine for **all** queues simultaneously: - **`keda`** (default, recommended) — one KEDA `ScaledObject` per queue, scaling on Redis list length (`workers.queues..queueLength`), `min`/`max` replica bounds. The Deployment omits `replicas` entirely so `helm upgrade` does not reset live-scaled replica counts back to the floor. - **`static`** — no autoscaler; each Deployment is pinned at `replicas: .max`. - **`hpa`** (discouraged fallback) — one CPU-based `HorizontalPodAutoscaler` per queue. CPU is a poor proxy for I/O-bound Celery workers and HPA cannot scale to 0 (`minReplicas` is floored at 1 even if `min: 0`). Prefer `keda` or `static`. `workers.database.{poolSize,maxOverflow}` is deliberately small — see [R2](#db-pool-sizing-vs-sql-connection-cap-r2). `gracePeriod` per queue feeds `terminationGracePeriodSeconds` — see [R4](#drain--upgrade-behavior-r4-7). #### KEDA scaling (default) The KEDA `listName` trigger metadata is generated as exactly `{roh:}.` (braces + trailing dot before the queue name, matching the app's Redis key prefix scheme). Do not override this by hand; it is derived entirely from `prefixNamespace` and the queue name. The chart renders a KEDA `TriggerAuthentication` and wires every `ScaledObject` to it whenever KEDA needs a Redis password. This covers both Redis modes. For in-cluster Redis with `redis.auth.enabled: true`, it points at the Bitnami subchart's password Secret. For `redis.mode: managed`, it is rendered when the password is available from either source: embedded in `redis.managed.url`, or supplied under the `REDIS_PASSWORD` key of `secrets.existingSecret`. The `TriggerAuthentication` reads that key in both cases, so an operator who keeps credentials in their own Secret — rather than in a Helm value — does not need to create one by hand. The rendered `TriggerAuthentication` follows `redis.auth.existingSecret`: if you point the Bitnami subchart at a Kubernetes Secret you manage yourself via `redis.auth.existingSecret` / `redis.auth.existingSecretPasswordKey` (for example, to reuse the same secret as `secrets.existingSecret` — this is what `values-aks.yaml` does), the `TriggerAuthentication` references that same secret/key instead of the Bitnami subchart's default (`{{ .Release.Name }}-redis` / `redis-password`). ### Beat ```yaml beat: resources: {limits: {cpu: 250m, memory: 512Mi}, requests: {cpu: 100m, memory: 128Mi}} ``` Exactly one Beat Deployment is rendered, `replicas: 1`, `strategy: Recreate`. Beat is a scheduler singleton — running two instances would double-schedule every periodic task. Do not scale this Deployment. ### Redis ```yaml redis: mode: incluster # incluster (Bitnami subchart) | managed enabled: true # subchart toggle; set false when mode: managed db: 0 # db index shared by REDIS_URL/ROH_BROKER_URL/ROH_RESULT_BACKEND managed: {url: ""} # redis(s)://[:pass@]host:port — required when mode: managed auth: {enabled: true, password: ""} architecture: replication master: persistence: {enabled: true, size: 8Gi} configuration: |- maxmemory-policy noeviction ``` `mode`, `enabled`, `db`, and `managed` are parent-chart-only keys read by ROH's own templates. Everything else under `redis:` (`auth`, `architecture`, `master`, etc.) is passed straight through to the Bitnami `redis` subchart, because Helm subcharts read values nested under their dependency name — there is no `redis.incluster.*` nesting; Bitnami's own keys live directly under `redis:` alongside the parent-only keys (Bitnami ignores keys it doesn't recognize). - **`mode: incluster`** — the Bitnami `redis` subchart is installed as part of this release (`redis.enabled: true`, the `Chart.yaml` dependency's `condition`). Set `redis.auth.password` (or `redis.auth.existingSecret` / `redis.auth.existingSecretPasswordKey` to point at a secret you manage) so the subchart and the chart-created `REDIS_URL` agree on credentials. - **`mode: managed`** — set `redis.enabled: false` (skip installing the subchart) and `redis.managed.url` to the full `redis(s)://[:pass@]host:port` connection string for your managed Redis. Use the `rediss://` scheme wherever the endpoint requires TLS: the chart derives KEDA's `enableTLS` from it, so `redis://` against a TLS-only endpoint both attempts a cleartext connection and disables TLS for KEDA. ElastiCache AUTH requires transit encryption, so AWS deployments always use `rediss://`. Supported services include Azure Cache, ElastiCache, and Memorystore. **R1 — eviction policy.** The in-cluster subchart is configured with `maxmemory-policy noeviction`. If you use a managed Redis, set the same policy yourself — Azure Cache and Memorystore both default to `volatile-lru`/similar eviction policies that can silently drop in-flight Celery broker/result/lock keys under memory pressure. #### Single Redis endpoint wiring (R14) The app code reads three separate Redis URLs (`REDIS_URL`, `ROH_BROKER_URL`, `ROH_RESULT_BACKEND`), which default to db 0/1/2 respectively if left unset, and `ROH_BROKER_URL` has **no fallback** to `REDIS_URL` — leaving it unset breaks locking, task dispatch, the dead-letter queue, and internal auth/monitoring in every pod. This chart wires all three to the **same** endpoint (same host, port, and db): only `REDIS_URL` is ever placed in the Secret (chart-created or `existingSecret`), and every pod (app, worker, beat, migration Job) sets `ROH_BROKER_URL` and `ROH_RESULT_BACKEND` as explicit pod `env` entries with a `secretKeyRef` back to that same `REDIS_URL` key (explicit `env` takes precedence over `envFrom`). **Practical consequence: if you bring your own `existingSecret`, you only need to get `REDIS_URL` right** (with its password embedded, e.g. `redis://:pw@host:6379/0`). Do not also set `ROH_BROKER_URL`/`ROH_RESULT_BACKEND` in your secret — the chart's explicit per-pod `env` entries always win over `envFrom`, so any value you put there would be silently overridden anyway; simplest is to omit them from your secret entirely. ### Database ```yaml database: external: {connectionString: ""} # informational; DATABASE_URL is actually sourced from secrets.databaseUrl incluster: enabled: false # dev/test/air-gapped ONLY image: mcr.microsoft.com/mssql/server:2022-latest saPassword: "" persistence: {enabled: true, size: 20Gi, storageClass: ""} resources: {limits: {cpu: 2, memory: 4Gi}, requests: {cpu: 1, memory: 2Gi}} ``` - **External (default, `database.incluster.enabled: false`)** — supply the full pymssql `DATABASE_URL` connection string via `secrets.databaseUrl` (chart-created secret) or in your `secrets.existingSecret`'s `DATABASE_URL` key. This is the only supported path for production: managed SQL Server (Azure SQL MI/VM, RDS SQL Server, Cloud SQL for SQL Server). - **In-cluster (`database.incluster.enabled: true`)** — the chart deploys a single-replica SQL Server `StatefulSet` + PVC and derives `DATABASE_URL` for you from `database.incluster.saPassword`. **Dev/test/ air-gapped only** — no HA, no managed backups, single replica, not FedRAMP-eligible. Do not use this for production. **R10 — SQL authentication only.** ROH's database connection is always SQL-auth (username/password in the connection string), never cloud IAM auth, even when `serviceAccount.annotations` wires up Azure Workload Identity / IRSA / GKE Workload Identity. Those annotations are only useful for other cloud-native integrations (e.g. Key Vault, Secrets Manager, object storage) — they do not change how ROH authenticates to SQL Server. **DB pool sizing vs. SQL connection cap (R2).** Each app pod defaults to `app.database.poolSize: 10` + `app.database.maxOverflow: 5` = 15 connections/pod; each worker pod defaults to `workers.database.poolSize: 3` + `workers.database.maxOverflow: 2` = 5 connections/pod (workers run `--concurrency=1`, so they need far fewer connections than the app tier). ROH's SQL Server is shared with core RegScale. Before raising `app.hpa.maxReplicas`, any `workers.queues..max`, or the pool sizes themselves, compute the worst case (`app.hpa.maxReplicas * (poolSize+maxOverflow)` + `sum(workers.queues.*.max * (workers.database.poolSize+maxOverflow))` + Beat + core's own pool) against your SQL Server's actual connection limit. Exceeding it causes connection-pool exhaustion failures across both ROH and core. ### Monitoring ```yaml monitoring: metrics: {enabled: true} # ENABLE_METRICS serviceMonitor: {enabled: false, interval: 30s, labels: {}} # Prometheus Operator: scrapes the app Service (AKS/EKS-style clusters) podMonitor: {enabled: false, interval: 30s, labels: {}} # Prometheus Operator: scrapes app pods directly (bypasses the Service) gcpPodMonitoring: {enabled: false, interval: 30s} # GKE Managed Prometheus dashboards: {enabled: false, labels: {grafana_dashboard: "1"}} # 12 Grafana dashboard JSON ConfigMaps alertRules: {enabled: false, labels: {}} # PrometheusRule from alerts.yml ``` All monitoring resources are off by default and additive — enabling them never changes app behavior. `monitoring.serviceMonitor` and `monitoring.podMonitor` are both Prometheus-Operator CRDs targeting the same app-tier `/metrics` endpoint — `serviceMonitor` scrapes via the app Service's `metrics` port (which targets container port `gunicorn`, 8000), `podMonitor` scrapes the pod's `gunicorn` port directly, bypassing the Service. Pick whichever your cluster's Prometheus Operator convention prefers (typically just `serviceMonitor`); use `monitoring.gcpPodMonitoring` on GKE instead of either, since GKE Managed Prometheus does not consume Prometheus-Operator CRDs. **R15 — `METRICS_TOKEN` secures `/metrics`.** `secrets.metricsToken` (or your `existingSecret`'s `METRICS_TOKEN` key) is required for the `/metrics` endpoint to be meaningfully protected; the rendered `ServiceMonitor` authenticates its scrape using this same token (`bearerTokenSecret` → your secret's `METRICS_TOKEN` key). Always set a real, non-guessable token — an empty or default token leaves metrics (including internal operational detail) exposed to anything in-cluster that can reach the Service. ### NetworkPolicy ```yaml networkPolicy: enabled: false # off by default for self-host coreNamespace: regscale # for cross-namespace rules when core lives in a dedicated namespace ``` Off by default. When enabled, it allows ingress to the app's port `4200` from any namespace and egress to DNS, SQL Server (1433), Redis (6379), HTTP and HTTPS (80, 443), the core proxy pods (8080), and, with `synqly.enabled`, the Synqly Embedded API (8000). With `coreProxy.enabled`, it also allows ingress on 8080, since the proxy pods carry the chart's labels. HTTPS is intentionally broad because `regscale-cli`-driven scan/integration tasks make outbound HTTPS calls to arbitrary scan targets and registries. The `podSelector` excludes the optional in-cluster SQL Server StatefulSet (`app.kubernetes.io/component: mssql`) via a `NotIn` match expression, so this policy governs only the ROH app-tier workloads (app/worker/beat/ migrate) and never default-denies the mssql pod's own ingress. If you enable **both** `networkPolicy.enabled` and `database.incluster.enabled`, the mssql pod remains ungoverned by this policy (open ingress/egress); add your own supplemental `NetworkPolicy` for the mssql pod if you need to restrict it too. ### Synqly Embedded ```yaml synqly: enabled: true # subchart synqly/embedded 0.1.166, alias synqly ``` On by default. The chart installs Synqly Embedded in the release namespace, sets `SYNQLY_BASE_URL` to `http://embedded..svc.cluster.local:8000` in the ConfigMap, and opens NetworkPolicy egress on 8000. With `synqly.enabled: false`, the release holds no Synqly resource and no `SYNQLY_BASE_URL`. The subchart reads its settings under `synqly:` and the top-level `global:`. Start from [`values-synqly.yaml`](values-synqly.yaml), which holds no credential: every Synqly credential, the Vault AppRole included, comes from a Secret you create (`secretRef`). Helm stores every value in the release record, so the chart stops if Vault AppRole or token auth is on without `secretRef`, or if any of these is set as a plain value, under `global:` or `synqly.global:`: `dbPassword`, `root.credentials.*`, `smtpPassword`, `organizationAdmin.default.credentials.password`, the Vault `roleId`, `secretId` and `tokenValue`, `aws.secretAccessKey`, the Azure Key Vault `clientSecret`, the AWS KMS user `secretAccessKey`, the GCP KMS `keyJSON`, `synqly.pods.synqlyRegistryCredentials.password`, and `synqly.managementUI.uiCookieEncryptionKey`. Subchart placeholders such as `` pass. - One Synqly Embedded per namespace: the subchart fixes the Service name `embedded` and the Secret name `synqly-vault-access`. - Each Synqly image stops working 90 days after release unless `global.synqlyLicense` holds a Synqly-issued license. - `global.organizationAdmin.default.secretRef` is required, or `inviteMembers` with SMTP (`global.smtpServer` or `global.smtpSecretRef`), since the invitation goes by email. Synqly creates its default admin only when it first creates its organization. - `organizationID` and `organizationFullName` cannot change after the first start. `values-synqly.yaml` ships them as `REPLACE-` values, which the chart rejects. With Synqly on, the chart also sets ROH's Synqly settings, as the SaaS configuration does: `SYNQLY_ORG_ID` (from `global.organizationID`), `SYNQLY_ALLOW_INSECURE_HTTP=true` (for the in-cluster `http` address), and from `synqlyIntegration`: `ISL_ENABLED`, `ISL_SERVE_AUTOSTART`, `ISL_MINT_WORKER_CONCURRENCY`, `SYNQLY_ONBOARD_ON_LOGIN`, `SYNQLY_TIER` (`prod` | `nonprod`), `SYNQLY_ENVIRONMENT` (`test` | `prod`) and `SYNQLY_CATALOG_MEMBER_NAME`. Only the app container gets `SYNQLY_ADMIN_MEMBER_NAME` and `SYNQLY_ADMIN_SECRET` (from the default admin's Secret, or from `synqlyIntegration.onboardingMember.secretRef` when it is set, see below) and `SYNQLY_CATALOG_MEMBER_SECRET` (from `synqlyIntegration.catalogMember.secretRef`). `ROH_PUBLIC_URL` comes from `publicUrl` or the first `app.ingress` host, and `GUNICORN_WORKERS` from `app.gunicornWorkers`. See [roh-setup.md, "ROH settings for Synqly"](../roh-setup.md#roh-settings-for-synqly). To onboard tenants as a narrow Member instead of the default admin (REG-38470), set `synqlyIntegration.onboardingMember.secretRef` to a Secret that holds the Member's secret (and, with `nameKey`, its name). The app container then logs on as the `service` Member `isl-onboarder`, which holds only the `account-provisioner` permission set, and a hook Job makes that Role and Member in Synqly Embedded if they do not exist. It needs an ISL release that has REG-33941 (rs-isl 1.1.0.0 or later). Workers and Beat get no Member. ```yaml synqlyIntegration: onboardingMember: name: isl-onboarder # used when secretRef.nameKey is empty roleName: isl-onboarder secretRef: {secretName: synqly-onboarder, secretKey: secret, nameKey: username} job: {enabled: true, readyTimeoutSeconds: 600, backoffLimit: 2} # needs global.organizationAdmin.default.secretRef ``` After each install and upgrade, a hook Job applies the allow lists in [`files/synqly-allow-lists.json`](files/synqly-allow-lists.json) (a copy of rs-isl `.github/synqly/allow-lists.json`) to the templates `regscale-`, as `isl manage allow-lists` does: ```yaml synqlyAllowLists: enabled: true # needs global.organizationAdmin.default.secretRef lists: {} # empty: the chart's file; otherwise the lists, in the same shape readyTimeoutSeconds: 600 backoffLimit: 2 ``` Helm waits for the Job within `--timeout` (5 minutes by default), and the Job can run for `readyTimeoutSeconds` plus 300 seconds. Pass `--timeout 20m` to installs and upgrades with Synqly. The full procedure is in [roh-setup.md, "Synqly Embedded"](../roh-setup.md#synqly-embedded). ### Service account ```yaml serviceAccount: create: true name: "" annotations: {} # azure.workload.identity/client-id | eks.amazonaws.com/role-arn | iam.gke.io/gcp-service-account ``` Used for cloud-native workload identity to reach non-database resources (see [R10](#database) above) — Key Vault/Secrets Manager, object storage, etc., depending on cloud and integration. ## Per-cloud overlays `values-aks.yaml`, `values-eks.yaml`, and `values-gke.yaml` are complete example overlays for the three supported clouds. Each assumes an `existingSecret` named `roh-secrets` that you create out-of-band (see [Secrets](#secrets--existingsecret)) and a unique `prefixNamespace` you must replace. Each sets `regscaleUrl` to core's Service and turns on `coreProxy`. Install with `helm install roh . -f values-.yaml`, plus `-f values-synqly.yaml` or `--set synqly.enabled=false`. | Overlay | Ingress | Redis | Monitoring | |---|---|---|---| | `values-aks.yaml` | nginx | in-cluster (Bitnami), `existingSecret`-backed password | ServiceMonitor + PrometheusRule + dashboards | | `values-eks.yaml` | ALB | managed (ElastiCache) | ServiceMonitor + PrometheusRule + dashboards | | `values-gke.yaml` | GCE | managed (Memorystore) | GCP PodMonitoring + dashboards | ## Supported combination matrix ROH supports **2 SQL modes × 2 Redis modes × {`keda`, `static`} scaling × 3 clouds = 24 combinations**. `hpa` scaling is a discouraged fallback and is not part of the supported 24 — use it only where KEDA cannot be installed and `static` sizing is unacceptable. | Axis | Values | |---|---| | SQL | external/managed (`database.incluster.enabled: false`) | in-cluster (`database.incluster.enabled: true`, dev/test only) | | Redis | in-cluster (`redis.mode: incluster`) | managed (`redis.mode: managed`, `redis.enabled: false`) | | Worker scaling | `keda` (recommended) | `static` | | Cloud | AKS | EKS | GKE | CI (`roh-static-checks` in `.github/workflows/ci.yml`) renders every overlay (`values.yaml`, `values-aks.yaml`, `values-eks.yaml`, `values-gke.yaml`) and every scaling mode (`keda`, `hpa`, `static`) on every push, plus a server-side `kubectl apply --dry-run=server` against a kind cluster (`roh-apiserver-validate` in `.github/workflows/integration.yml`, `static` scaling + monitoring off, since kind lacks the KEDA/Prometheus-Operator CRDs). GKE end-to-end validation against a live cluster is a fast-follow, not yet performed. ## Operational caveats ### Drain / upgrade behavior (R4, §7) Worker Deployments run with `terminationGracePeriodSeconds: 15300` (the app's hard task-execution limit of 4 hours plus a buffer) and `strategy: RollingUpdate` with `maxUnavailable: 0`. On `helm upgrade`, old worker pods are not killed — they stay `Terminating` until in-flight tasks finish (**up to ~4h15m by design**) or the grace period elapses. **Do not run `helm upgrade --wait` with a short `--timeout` against this chart** — a short timeout will report a false failure while old workers are still correctly draining in the background; the upgrade itself has already succeeded from Kubernetes' perspective. Either omit `--wait`/use a very long timeout (`--timeout 4h30m` or more), or watch `kubectl rollout status` asynchronously. `app.pdb` (`PodDisruptionBudget`) only protects the app tier from voluntary node evictions/drains (e.g. cluster autoscaler node removal, `kubectl drain`) — it does **not** apply to `helm upgrade` rolling updates. Do not rely on it as an upgrade-safety mechanism. Celery Beat is a scheduler singleton (`replicas: 1`, `strategy: Recreate`) — briefly no periodic tasks fire during a Beat rollout; this is expected and intentional (two Beats running concurrently would double-schedule every periodic task). ### Immutable `SECRETS_ENCRYPTION_KEY` (R3) See [Secrets](#secrets--existingsecret) above — generate it once, store it outside the cluster, and never rotate it by changing the value in a `helm upgrade`. ### Single Redis endpoint wiring (R14) See [Single Redis endpoint wiring](#single-redis-endpoint-wiring-r14) above. ### DB pool sizing vs. SQL connection cap (R2) See [Database](#database) above. ### `METRICS_TOKEN` required to secure `/metrics` (R15) See [Monitoring](#monitoring) above. ### Install order (R11) Install in this order: **SQL Server reachable → core RegScale application installed and healthy → ROH**. ROH's migration hook and app pods assume the core RegScale application is already reachable at `regscaleUrl` and that the database is reachable; installing ROH first (or against a not-yet-migrated/available core deployment) can leave the pre-install migration Job failing or the app pods crash-looping on startup. ### App-repo dependencies (not shipped by this chart) Two behaviors depend on changes in the ROH application image itself, not on this chart, and are not yet shipped as of this chart's writing: 1. **`init.sh` `RUN_MIGRATIONS` gate.** The chart's ConfigMap sets `RUN_MIGRATIONS: "false"` and runs migrations exactly once via a `pre-install,pre-upgrade` hook Job (`alembic upgrade head` then `python -m app.cli.main commands init`). Until the application image's `init.sh` actually honors `RUN_MIGRATIONS=false` to skip its own startup migration step, **every** app/worker/beat pod also attempts `alembic upgrade head` on start, redundant with the hook Job. This is currently harmless (Alembic is a no-op when the schema is already current) but is wasted work per pod start and is not defense against concurrent migration races if the image's behavior changes. 2. **Unauthenticated readiness route.** The app image does not yet ship a readiness endpoint that can be safely called without authentication and that verifies database connectivity. Until it does, `app.probes.readiness` falls back to the open `GET /api/v1/health` liveness-style endpoint, which does **not** gate readiness on database connectivity. A pod can be marked Ready by Kubernetes (and receive traffic) even if its database connection is broken. ## Uninstalling ```bash helm uninstall roh --namespace regscale ``` This does not delete PVCs created by the in-cluster Redis subchart or the in-cluster SQL Server StatefulSet (`database.incluster.enabled: true`); remove them manually if you are decommissioning the release entirely.