1. What is AI Observability for? #
Monitor generative AI applications: LLMs, agents, prompts, tokens and evaluations.
The principle #
AI Observability gives end-to-end visibility into AI workloads: services, large language models (LLMs), agents and protocols. The app tracks model calls, their token cost, latency and answer quality.
2. Key features #
What the app offers.
Overview #
| Feature | Role |
|---|---|
| Flux de prompts | Agent and LLM interactions: model version, token count, prompt and response |
| Vue trace | Agent span hierarchy and tool calls, for debugging |
| Évaluation | LLM-as-judge: verdicts and scores on agent runs |
| Technologies | More than 20, including OpenAI, Amazon Bedrock, Gemini, Vertex AI, Anthropic, LangChain |
3. Getting started #
First steps.
Step by step #
- Instrument the application with OneAgent, which automatically covers many AI frameworks.
- Open the app and spot the most expensive or slowest models.
- Drill into an agent trace to see tool calls.
- Add evaluations to track quality.
Prerequisites #
- OneAgent, and for cloud AI services the provider's cloud monitoring.
- A Dynatrace Platform Subscription (metrics, traces and logs on Grail).
Sensitive data. Prompts and answers can contain personal data: decide what you capture and mask. OpenLLMetry support is end of life: follow the documentation to migrate.
Sources #
From the official documentation consulted in November 2026: labels and availability change with platform versions.
