Skip to content

Deploy the observability service

Composes the E1 baseline + a sink resource per cloud so telemetry lands somewhere managed:

  • Azure — Application Insights (backed by a Log Analytics workspace)
  • GCP — Cloud Trace + Cloud Monitoring APIs enabled on the project
  • AWS — CloudWatch Log Group (ADOT Collector's default target)

Verified 2026-08-08

Layout

examples/observability/
  service/                        # OTel-instrumented chat wrapper
    telemetry.py                  # per-provider exporter picker + span helpers
    redact.py                     # PII redactor (SpanProcessor)
    main.py                       # FastAPI /chat + /healthz (fails closed)
    Dockerfile
    requirements.txt
  chart/                          # Helm chart
    Chart.yaml
    values.yaml
    templates/{deployment,service,serviceaccount}.yaml
  terraform/
    azure/  (module "baseline" + Log Analytics + Application Insights)
    gcp/    (module "baseline" + enable cloudtrace + monitoring services)
    aws/    (module "baseline" + CloudWatch Log Group for ADOT + IAM)

Terraform — compose + one sink

module "baseline" {
  source  = "../../../foundations/azure"
  name    = var.name
  region  = var.region
  compute = var.compute
}

resource "azurerm_log_analytics_workspace" "chiron" {
  name                = "${var.name}-law"
  location            = var.region
  resource_group_name = module.baseline.resource_group_name
  sku                 = "PerGB2018"
  retention_in_days   = var.retention_days
}

resource "azurerm_application_insights" "chiron" {
  name                = "${var.name}-ai"
  location            = var.region
  resource_group_name = module.baseline.resource_group_name
  application_type    = "web"
  workspace_id        = azurerm_log_analytics_workspace.chiron.id
}

Feed azurerm_application_insights.chiron.connection_string into the chart via env.APPLICATIONINSIGHTS_CONNECTION_STRING.

module "baseline" {
  source     = "../../../foundations/gcp"
  project_id = var.project_id
  name       = var.name
  region     = var.region
  compute    = var.compute
}

resource "google_project_service" "cloudtrace" {
  project                    = var.project_id
  service                    = "cloudtrace.googleapis.com"
  disable_on_destroy         = false
}

resource "google_project_service" "monitoring" {
  project                    = var.project_id
  service                    = "monitoring.googleapis.com"
  disable_on_destroy         = false
}

# Give the workload's SA rights to write traces + metrics.
resource "google_service_account" "observability" {
  account_id   = "${var.name}-obs"
  display_name = "Chiron observability writer"
}
resource "google_project_iam_member" "trace_agent" {
  project = var.project_id
  role    = "roles/cloudtrace.agent"
  member  = "serviceAccount:${google_service_account.observability.email}"
}
resource "google_project_iam_member" "metric_writer" {
  project = var.project_id
  role    = "roles/monitoring.metricWriter"
  member  = "serviceAccount:${google_service_account.observability.email}"
}

module "baseline" {
  source  = "../../../foundations/aws"
  name    = var.name
  region  = var.region
  compute = var.compute
}

# Where the ADOT Collector writes its logs; also where you'd point
# Bedrock model-invocation logging if you enable it.
resource "aws_cloudwatch_log_group" "chiron" {
  name              = "/chiron/${var.name}"
  retention_in_days = var.retention_days
}

# Policy for the workload role to write traces to X-Ray + metrics to CloudWatch.
data "aws_iam_policy_document" "otel_write" {
  statement {
    effect = "Allow"
    actions = [
      "xray:PutTraceSegments",
      "xray:PutTelemetryRecords",
      "cloudwatch:PutMetricData",
      "logs:CreateLogStream",
      "logs:PutLogEvents",
    ]
    resources = ["*"]
  }
}

resource "aws_iam_policy" "otel_write" {
  name   = "${var.name}-otel-write"
  policy = data.aws_iam_policy_document.otel_write.json
}

Fail-closed health check

The chart's readiness probe hits /healthz — which returns 503 if CHIRON_LOG_PROMPT_TEXT=true but the redactor isn't active. See Privacy for the invariant. This means an unsafe pod never receives traffic — Kubernetes stops rolling if the check fails.

Verify (validate-only)

for p in azure gcp aws; do
  ( cd examples/observability/terraform/$p && terraform init -backend=false && terraform validate )
done

helm lint examples/observability/chart
helm template obs examples/observability/chart > /dev/null

python -m compileall examples/observability/service

Live apply

Wire the connection-string (Azure) / workload-identity SA email (GCP) / IAM role (AWS) from the module outputs into the chart at install time. Phase 3 covers billed apply against real accounts.