Backends — per-cloud exporter setup¶

The OTel API in your code doesn't change per cloud. The exporter does. This page shows the one-line (Azure) or few-line (GCP, AWS) setup that connects your TracerProvider to the vendor backend.

Verified 2026-08-08

azure-monitor-opentelemetry — pypi.org/project/azure-monitor-opentelemetry
opentelemetry-exporter-gcp-trace — pypi.org/project/opentelemetry-exporter-gcp-trace
aws-opentelemetry-distro — pypi.org/project/aws-opentelemetry-distro
Azure OpenAI diagnostic settings — learn.microsoft.com/…/openai/how-to/monitor-openai
Vertex AI audit logs — cloud.google.com/vertex-ai/docs/general/audit-logging
Bedrock model invocation logging — docs.aws.amazon.com/bedrock/latest/userguide/model-invocation-logging

Application-level (OTel)¶

AzureGCPAWS

One line at process start; the rest of your code uses the vanilla OTel API.

import os, logging
from azure.monitor.opentelemetry import configure_azure_monitor

# Reads APPLICATIONINSIGHTS_CONNECTION_STRING from env.
configure_azure_monitor(logger_name="chiron")
logger = logging.getLogger("chiron")

# From here on: normal OTel + logging.
from opentelemetry import trace
tracer = trace.get_tracer("chiron.observability")
with tracer.start_as_current_span("chat"):
    ...

Set APPLICATIONINSIGHTS_CONNECTION_STRING via the chart — never in code (per the Microsoft docs' recommendation for production).

Two SDK options — pick one:

Option A — In-process (opentelemetry-exporter-gcp-trace):

from opentelemetry import trace
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.cloud_trace import CloudTraceSpanExporter

resource = Resource.create({"service.name": "chiron-observability"})
provider = TracerProvider(resource=resource)
provider.add_span_processor(BatchSpanProcessor(CloudTraceSpanExporter()))
trace.set_tracer_provider(provider)

Metrics: install opentelemetry-exporter-gcp-monitoring and repeat the pattern with CloudMonitoringMetricsExporter.

Option B — OTLP → OpenTelemetry Collector (recommended by GCP):

from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter

provider.add_span_processor(BatchSpanProcessor(OTLPSpanExporter()))

The Collector (deployed as a sidecar or DaemonSet — see OpenTelemetry Collector on GKE) forwards to Cloud Trace + Cloud Monitoring using its googlecloud exporter. This is the pattern the GCP setup guide recommends.

Auto-instrumentation via ADOT — one command wraps your app:

pip install aws-opentelemetry-distro
OTEL_PYTHON_DISTRO="aws_distro" \
OTEL_PYTHON_CONFIGURATOR="aws_configurator" \
OTEL_EXPORTER_OTLP_ENDPOINT="http://localhost:4317" \
opentelemetry-instrument python3 main.py

Ships to the ADOT Collector on localhost:4317; the collector forwards traces to AWS X-Ray and metrics to CloudWatch EMF. Standard @tracer.start_as_current_span and metric counters keep working.

Manual wiring is also documented at aws-otel.github.io/docs/getting-started/python-sdk/manual-instr for cases where you don't want the auto agent.

Native model-invocation logging¶

The provider records every request/response server-side. Chiron treats these as the audit trail — separate from your OTel spans, which are the operational view.

AzureGCPAWS

Turn on diagnostic settings on the Azure OpenAI Cognitive account and route to a Log Analytics workspace. Relevant categories:

Audit — administrative operations on the resource
RequestResponse — every model call, prompt + completion bodies
Trace — model-internal traces
AllMetrics — token counts + latency

Once enabled, KQL these tables in Log Analytics: AzureDiagnostics (routed logs) and AzureMetrics (routed metrics).

Vertex AI writes Data Access audit logs for aiplatform.googleapis.com when you enable them at the org / folder / project level (cloud.google.com/vertex-ai/docs/general/audit-logging). The predict / streamGenerateContent / embedContent methods land in the data_access log stream. Redact prompt+response bodies with a log exclusion filter or a Cloud Logging field-level redaction rule before writes leave the log sink (see Privacy).

Bedrock has a first-class model invocation logging feature — one API call, two sinks:

import boto3
bedrock = boto3.client("bedrock")
bedrock.put_model_invocation_logging_configuration(
    loggingConfig={
        "cloudWatchConfig": {
            "logGroupName": "chiron-bedrock-invocations",
            "roleArn":      "arn:aws:iam::…:role/chiron-bedrock-logging",
        },
        "s3Config": {"bucketName": "chiron-bedrock-invocation-archive"},
        "textDataDeliveryEnabled":  True,   # ⚠ writes prompts+responses
        "imageDataDeliveryEnabled": True,
        "embeddingDataDeliveryEnabled": False,
    },
)

⚠ The *DataDeliveryEnabled flags control whether the actual prompt / image / embedding bytes are written — set to False unless you've applied redaction upstream (see Privacy).

Chart wiring¶

The observability chart adds one env var per backend:

# Azure
helm upgrade --install obs ./examples/observability/chart \
  --set env.APPLICATIONINSIGHTS_CONNECTION_STRING="InstrumentationKey=...;IngestionEndpoint=..."

# GCP (in-process option)
helm upgrade --install obs ./examples/observability/chart \
  --set env.OTEL_EXPORTER=cloud_trace

# AWS (ADOT auto)
helm upgrade --install obs ./examples/observability/chart \
  --set env.OTEL_PYTHON_DISTRO="aws_distro" \
  --set env.OTEL_PYTHON_CONFIGURATOR="aws_configurator" \
  --set env.OTEL_EXPORTER_OTLP_ENDPOINT="http://otel-collector.observability.svc.cluster.local:4317"

The single source of truth for which backend is wired is CHIRON_PROVIDER — the service module picks the exporter at process-start.