Using the Dynatrace API v2: a complete guide to automation

1. Why use the Dynatrace API? #

The API turns Dynatrace from a visualization tool into an automation engine.

Automation and monitoring as code #

Using the Dynatrace API addresses three major needs:

Industrialization: configure hundreds of alerts or dashboards with tools such as Monaco or Terraform.
CI/CD integration: integrate Dynatrace into your pipelines (Jenkins, GitLab CI, Azure DevOps, etc.) to automate actions such as setting maintenance windows, ingesting deployment events (custom deployment) so that Davis AI can correlate changes with possible regressions, or validating SLOs.
Data extraction: feed third-party tools (Power BI, Splunk) or run complex analytical queries on Grail.

2. Security: the notion of scopes #

Understanding how to restrict access for API tokens.

The principle of least privilege #

A scope defines the permissions granted to an API token. Dynatrace v2 splits rights granularly:

Read scopes: allow reading data (e.g. entities.read, metrics.read, logs.read).
Write scopes: allow modifying the configuration (e.g. settings.write).
Ingest scopes: allow sending data (e.g. events.ingest, logs.ingest).

Best practice: Never create an 'admin' token. A CI/CD deployment token should only have the scopes it actually uses, for example for events: events.ingest.

3. Architecture: where should you send your requests? #

Choosing the API entry point is strategic to optimize your network flows and comply with your infrastructure's security policies, and it depends on your Dynatrace deployment (SaaS or Managed).

Dynatrace server (SaaS) or Dynatrace cluster (Managed): direct access #

Dynatrace API access points by deployment

For SaaS environments, this is the default public URL (https://{env-id}.live.dynatrace.com/api/v2/ for the environment API, https://{env-id}.apps.dynatrace.com/platform/ for the platform API (Grail, DQL)) for tools with direct Internet access. For Managed deployments, it is the URL of your Dynatrace cluster (https://{votre-cluster-managed}/e/{env-id}/api/v2/), which is usually an **internal** address on your network. It is the direct entry point for API requests that do not go through an ActiveGate.

Environment ActiveGate (SaaS or Managed) #

The environment ActiveGate acts as a secure gateway for communication between your internal network and the Dynatrace cluster (SaaS or Managed). It is essential to centralize outbound flows (single point of egress) from your internal servers, drastically limiting firewall openings to the Internet by using a single pivot IP. The API is then available on port 9999 of the ActiveGate.

Cluster ActiveGate (Managed) #

In a Dynatrace Managed (on-premises) installation, the cluster ActiveGate intelligently routes API requests to the different environments (tenants) managed by the cluster, while keeping the data within your private network. It is the main, **internal-only** entry point for API interactions with a Managed deployment.

Summary table of API URLs #

Understanding how to build the base URL of your API requests depending on your deployment type and the targeted service.

Building the URLs #

Deployment type	Access point	URL format	Example (SaaS)	Example (Managed)
SaaS	Direct (environment API)	`https://{env-id}.live.dynatrace.com/api/v2/...`	`https://abc12345.live.dynatrace.com/api/v2/entities`	N/A
SaaS	Direct (platform API / Grail)	`https://{env-id}.apps.dynatrace.com/platform/...`	`https://abc12345.apps.dynatrace.com/platform/storage/query/v1/query:execute`	N/A
SaaS or Managed	Environment ActiveGate	`https://{ActiveGate-IP}:9999/e/{env-id}/api/v2/...`	`https://10.0.0.10:9999/e/abc12345/api/v2/metrics`	`https://10.0.0.10:9999/e/def67890/api/v2/metrics`
Managed	Dynatrace cluster	`https://{votre-cluster-managed}/e/{env-id}/api/v2/...`	N/A	`https://moncluster.local/e/def67890/api/v2/problems`

Note: {env-id} is the ID of your Dynatrace environment. {ActiveGate-IP} is the IP address or host name of your ActiveGate. {votre-cluster-managed} is the base URL of your Dynatrace Managed deployment.

4. SaaS: apps.dynatrace.com vs live.dynatrace.com #

On Dynatrace SaaS, the domain changes depending on whether you target the classic environment APIs or the new platform services.

The difference between these two domains reflects the major evolution of the Dynatrace architecture in recent years. It separates the classic environment (Gen2) from the new modern platform based on the Grail data lakehouse and the new user experience (Gen3). Here are the details to understand the distinction:

live.dynatrace.com (the classic architecture / Gen2) #

This domain is associated with standard Dynatrace SaaS environments. It is the historical URL used to access the console and interact with the platform's classic APIs.

Underlying storage: uses the older storage architecture (Elasticsearch/Cassandra for classic metrics, traces and logs).
APIs: hosts the standard v1 and v2 APIs (such as /api/v2/events/ingest, /api/v2/metrics, etc.).
Interface: gives access to classic dashboards and traditional diagnostic screens.

URL format: https://{env-id}.live.dynatrace.com/api/v2/...

apps.dynatrace.com (the new platform / Gen3 & Grail) #

This domain represents the modern Dynatrace. It centralizes the platform's new apps, the next-generation user experience and the execution of automated workflows.

Underlying storage: fully powered by Grail, the massive, index-free data lakehouse that centralizes logs, metrics, traces and events in a unified way.
Key features: this is where new features such as the Dashboards app, Notebooks, Workflows (automation) and the DQL (Dynatrace Query Language) run.
APIs: hosts endpoints specific to apps, AppEngine and Grail queries.

URL format: https://{env-id}.apps.dynatrace.com/platform/...

In short: who does what? #

Characteristic	live.dynatrace.com	apps.dynatrace.com
Generation	Dynatrace Gen2 (classic)	Dynatrace Gen3 (modern)
Data engine	Classic storage (indexed)	Grail (data lakehouse)
Query language	USQL / standard selectors	DQL (Dynatrace Query Language)
Main use	Classic ingestion, configuration APIs, legacy monitoring	Advanced visualization, Notebooks, automation, modern log management

5. Practical examples #

Examples based on the https://demo.dynatrace.rousseltm.fr environment. To explore and test these APIs, Dynatrace provides a Swagger UI (OpenAPI) interface. It lets you browse the technical documentation and run requests directly from the browser.

Accessing Swagger:

SaaS: https://{env-id}.apps.dynatrace.com/platform/swagger-ui/index.html
Managed: https://{votre-cluster-managed}/e/{env-id}/swagger-ui/index.html

1. Ingesting a deployment event #

Lets Davis AI know that a change has occurred. This example shows how to use shell variables to define the start and end timestamps.

START_TIME=$(date +%s%3N)
END_TIME=$(($(date +%s) + 63))000
curl -X POST "https://{env-id}.live.dynatrace.com/api/v2/events/ingest" \
-H "Authorization: Api-Token {TOKEN}" \
-H "Content-Type: application/json" \
-d '{
  "eventType": "CUSTOM_DEPLOYMENT",
  "title": "Deploy v1.2.0 - E-learning Platform",
  "entitySelector": "type(SERVICE),entityName(AuthenticationService)",
  "startTime": '"$START_TIME"',
  "endTime": '"$END_TIME"',
  "properties": {
    "git.commit": "a1b2c3d",
    "ci.user": "martin.roussel",
    "release.url": "https://gitlab.rousseltm.fr/test/repo/-/releases/v1.2.0",
    "build.version": "1.2.0.45",
    "stage": "production",
    "product": "E-learning Platform"
  }
}'

2. Creating a maintenance window #

Disable alerts during an update through the Settings API.

curl -X POST "https://demo.dynatrace.rousseltm.fr/api/v2/settings/objects" \
-H "Authorization: Api-Token {TOKEN}" \
-d '[{
  "schemaId": "builtin:maintenance-window",
  "value": {
    "enabled": true,
    "generalProperties": { "name": "Patching OS", "type": "PLANNED" },
    "schedule": { "type": "ONCE", "start": "2024-12-01T01:00:00", "end": "2024-12-01T02:00:00" }
  }
}]'

3. Running a DQL query (Grail) #

DQL queries go through the platform API (apps.dynatrace.com), with a platform token or an OAuth client: a classic API token (Api-Token) is not accepted there.

curl -X POST "https://abc12345.apps.dynatrace.com/platform/storage/query/v1/query:execute" \
  -H "Authorization: Bearer {PLATFORM_TOKEN}" -H "Content-Type: application/json" \
  -d '{ "query": "fetch logs | filter loglevel == \"ERROR\" | limit 100", "defaultTimeframeStart": "now-15m" }'
# long-running query: a requestToken is returned, poll it with
# GET /platform/storage/query/v1/query:poll?request-token=...

4. Retrieving distributed traces (PurePaths) #

On the new platform, traces are spans stored in Grail: read them with a DQL query, through the same API.

curl -X POST "https://abc12345.apps.dynatrace.com/platform/storage/query/v1/query:execute" \
  -H "Authorization: Bearer {PLATFORM_TOKEN}" -H "Content-Type: application/json" \
  -d '{ "query": "fetch spans | filter toString(trace.id) == \"{traceId}\" | sort start_time asc" }'

5. Listing entities (entity selector) #

List all Linux hosts in production.

curl -G "https://demo.dynatrace.rousseltm.fr/api/v2/entities" \
--data-urlencode "entitySelector=type(HOST),osType(LINUX),tag(env:prod)" \
-H "Authorization: Api-Token {TOKEN}"

6. Extracting metrics #

Retrieve the average CPU usage of a service.

curl -G "https://demo.dynatrace.rousseltm.fr/api/v2/metrics/query" \
--data-urlencode "metricSelector=builtin:host.cpu.usage:avg" \
-H "Authorization: Api-Token {TOKEN}"

7. Listing active problems #

Retrieve the incidents detected by Davis AI.

curl -G "https://demo.dynatrace.rousseltm.fr/api/v2/problems" \
-H "Authorization: Api-Token {TOKEN}"

8. Synthetic Monitoring configuration #

Retrieve the JSON configuration of an HTTP monitor.

curl -G "https://demo.dynatrace.rousseltm.fr/api/v2/synthetic/monitors/{id}" \
-H "Authorization: Api-Token {TOKEN}"

9. Audit logs #

Check who changed the environment configuration.

curl -G "https://demo.dynatrace.rousseltm.fr/api/v2/auditlogs" \
-H "Authorization: Api-Token {TOKEN}"

10. Ingesting OpenTelemetry metrics #

Send custom metrics through the OTLP endpoint.

curl -X POST "https://demo.dynatrace.rousseltm.fr/api/v2/otlp/v1/metrics" \
-H "Authorization: Api-Token {TOKEN}" \
-H "Content-Type: application/x-protobuf" \
--data-binary @metrics.pb

6. Multidimensional analysis vs waterfall #

Two types of analysis, two different API approaches.

Multidimensional analysis (overview) #

The Metrics API is used for multidimensional analysis. It aggregates data over thousands of requests (e.g. average latency by country). It is the ideal tool for dashboarding.

Waterfall analysis (granular view) #

Waterfall analysis is available through the Sessions API. It returns the millisecond-by-millisecond details of loading a single page (DOM, images, scripts). It is used for deep debugging of a specific transaction.