Temporal

Koldan requires a reachable Temporal service. Use either the optional chart dependency or an operator-managed Temporal deployment, and keep the selected namespace and persistence available throughout normal operation.

Prerequisites

Record the Temporal frontend endpoint, namespace, TLS and authentication requirements, retention policy, persistence databases, availability design, and recovery owner. Permit pod egress to the frontend gRPC endpoint, normally TCP 7233. Synchronize clocks on the cluster and identity provider.

Connect to an external Temporal service

Disable the dependency and configure the endpoint and namespace through supported Spring Temporal properties:

temporal:
  install: false

config: |
  spring:
    temporal:
      connection:
        target: temporal.example.net:7233
      namespace: koldan

Create the namespace before starting Koldan. Configure retention in Temporal according to operational and compliance requirements; do not assume a Koldan Helm upgrade creates or changes an external namespace.

If the service requires TLS, mTLS, or authentication, use the spring.temporal.* interface supported by the Temporal Spring Boot dependency packaged with the installed Koldan release. Mount any required client key material as a Kubernetes Secret and add a private issuing CA through TLS and custom CAs. Validate those settings against the exact release before rollout because the Temporal dependency owns their names and behavior.

Install the chart dependency

Enable bundled Temporal with temporal.install: true. Review at least:

• temporal.server.config.namespaces
• temporal.server.config.persistence.datastores.default.sql
• temporal.server.config.persistence.datastores.visibility.sql

Provision the temporal and temporal_visibility databases and separate credentials before installation. Use existingSecret and secretKey in each datastore instead of plaintext passwords where supported by the dependency. Confirm the required Secret key format against the packaged chart version.

The chart dependency is disabled by default. Its sample settings are not a production availability or backup design. Size and monitor the Temporal service and PostgreSQL persistence independently. Do not publish the Temporal UI unless it has an explicit authentication, authorization, and network-access policy.

Understand worker pools and task queues

The chart deploys three independently scalable worker pools. Each pool polls a fixed, version-matched set of Temporal task queues:

Use the task-queue name in Temporal monitoring and incident response. Use the worker name-not the task-queue name-as the key under capacityOverrides.

Every applicable executor and poller limit is release-managed and explicit. The table shows the common chart values before a supported capacity profile is applied. A dash means that the worker does not register that task type and therefore does not create that executor or poller.

The single-node profile changes the three compute activity limits to 4, 6, and 4; the medium-cluster profile changes them to 8, 12, and 8 per replica. Inference and orchestration limits remain as shown. See Scaling and capacity for effective pod resources, replicas, and site aggregates.

For each applicable task type, poller autoscaling is explicitly bounded per replica: minimum 1, initial min(2, executors), and maximum min(5, executors). Temporal Server 1.28 and later can adjust the poller count within those bounds. When an older external server does not support the feedback protocol, the Java SDK holds the explicitly configured initial count instead of using an unbounded or version-dependent value.

The orchestration pool also explicitly sets the worker-factory workflow cache to 200 instances and 400 workflow threads, with virtual workflow threads disabled for Java 17. The current 164 aggregate workflow-executor slots fit below the cache, the cache does not exceed the thread ceiling, and the thread ceiling is at least twice the executor total.

Koldan currently registers no local activities or Nexus services. It therefore does not configure irrelevant local-activity or Nexus slots. Activity-per-second limits remain unset until a measured provider or destination quota exists; concurrency remains the hard protection for long-running work. Virtual threads remain disabled on Java 17.

The packaged chart is the source of truth for this mapping. Review the installed chart's workerPools.<pool>.workers list after every upgrade because workers and queue assignments can change between releases. Do not copy, replace, add to, rename, or remove that list in an operator values file. Helm replaces lists instead of safely merging them, which can leave a task queue without a poller.

Override worker capacity

The supported capacity controls are:

• max-concurrent-activity-executors for activities performed by a worker replica.
• max-concurrent-workflow-task-executors for workflow tasks processed by a worker replica.
• activity-task-pollers-configuration.poller-behavior-autoscaling.* for bounded activity pollers.
• workflow-task-pollers-configuration.poller-behavior-autoscaling.* for bounded workflow pollers.

Set only the fields that need to change in workerPools.<pool>.capacityOverrides.<worker-name>. The map is merged over the chart default for that worker; an omitted worker or field keeps its packaged default.

workerPools:
  inference:
    replicaCount: 2
    capacityOverrides:
      transcription-decoding-processing:
        max-concurrent-activity-executors: 4
        activity-task-pollers-configuration:
          poller-behavior-autoscaling:
            max-concurrent-task-pollers: 4
  orchestration:
    capacityOverrides:
      webhook-delivery:
        max-concurrent-activity-executors: 12

An override key must exactly match a worker name in the same pool to have an effect. The chart deep-merges and renders these values but does not enforce Temporal capacity relationships, so review the effective ConfigMap after changing executors, pollers, or cache settings. Do not set spring.temporal.workers under config; the chart generates a pool-specific worker configuration so each Deployment polls only its assigned queues.

Treat replica and concurrency changes as one capacity decision. Increasing workerPools.inference.replicaCount, for example, increases pollers for every inference queue. For each constrained provider, ensure:

pool replicas × per-replica activity concurrency
  ≤ tested provider capacity - reserved capacity - safety headroom

Also recalculate PostgreSQL connections, CPU, memory, ephemeral storage, and termination time. See Scaling and capacity for the packaged capacity posture and Define the site capacity contract for the site procedure.

Apply and verify an override

1. Pin the chart version and confirm the worker name, pool, queue, and packaged default in that chart's values.yaml.
2. Calculate the capacity ceiling for every queue in the affected pool, including provider reservations and failure headroom.
3. Add only the selected capacityOverrides entries to the protected operator values file.
4. Run helm template and inspect the affected pool ConfigMap. Confirm every packaged worker remains present and the selected capacity fields have the intended values.
5. Apply the Helm upgrade.
6. Wait for the affected worker-pool Deployment rollout. A capacity change automatically rolls to the pods of the affected pool.

kubectl --namespace <namespace> get deployment \
  --selector dixilang.com/worker-pool=inference
kubectl --namespace <namespace> rollout status deployment/<deployment-name>

1. Confirm the expected number of pollers appears for every queue in the pool. Run a representative operation and monitor queue backlog, schedule-to-start latency, completion rate, retries, dependency latency, and throttling.

For rollback, restore the previous replica and override values together, run the Helm rollback or upgrade, wait for the automatic affected-pool rollout, and repeat the queue and dependency checks. Reducing capacity does not remove already queued work; keep monitoring until backlog and latency return to their normal range.

Verify

After installation or a connection change:

1. Confirm DNS and TCP 7233 access from the Koldan namespace.
2. Confirm TLS hostname validation and client authentication, when enabled.
3. Use Temporal administrative tooling to verify frontend health and that the configured namespace exists.
4. Confirm every affected Koldan workload becomes Ready without namespace, authorization, or connection errors.
5. Exercise a representative Koldan operation that uses background processing and confirm it completes.
6. Monitor Temporal service errors, persistence errors, task backlog, schedule-to-start latency, and failed operations during the rollout.

Maintenance, backup, and recovery

Before Temporal or persistence maintenance, control new Koldan traffic as appropriate and allow active work to settle. Back up both Temporal persistence databases using a database-consistent method. A backup of only the Temporal UI or Kubernetes objects cannot restore workflow state.

For complete disaster recovery, restore Temporal persistence consistently with the Koldan database and object storage. Keep the namespace name and retention configuration aligned with the restored release. After restore, verify frontend health first, then start Koldan and check representative in-flight and new operations. Do not replay or delete Temporal state manually unless directed by an established recovery procedure.