Dynatrace Managed. Query multiple environments for alerts

Question

question

Irakli N. asked · Jul 01 at 09:13 AM

Dynatrace Managed. Query multiple environments for alerts

Hello,

Here is the situation - we have Dynatrace Managed with a number of (in dozens and hundreds) of different monitored environments.

I would like to make an API call to find out if all or subset of these environments (selected from a list) logged the specific type of alert during a specific timeframe.

For example - list all environments experienced 'slow disk' alerts on the specific day with a timestamp of start/stop of the alert.

How can I do the above using a single request?

I know how to query a specific environment for all/specific types of alerts using GET -

/api/v1/events

But as I said - I have hundreds of such environments and it would require to create access token for each environment and query each of them separately. This is not a practical solution.

Any alternatives?

Thanks

Dynatrace Managed alerting

10 |2000000 characters needed characters left characters exceeded

Attachments: Up to 10 attachments (including images) can be used with a maximum of 50.0 MiB each and 250.0 MiB total.

Answer 1 · 2020-07-01T09:46:57Z

Radoslaw S. answered · Jul 01 at 09:46 AM

Hi Irakli - take a look at my example here that I show how I update Frequent Issues Detection settings for many environments using Ansible:

https://github.com/rszulgo/providing-dynatrace-managed-as-a-service/tree/master/update-existing-environment

You cannot do that with a single call, but that's not bad when you automate that - you can call single... script.

2 Share

10 |2000000 characters needed characters left characters exceeded

Attachments: Up to 10 attachments (including images) can be used with a maximum of 50.0 MiB each and 250.0 MiB total.

Irakli N. · Jul 01 at 10:09 AM

I am kinda surprised that such a feature is not implemented...

Let me try your approach but in one of our clusters we have '2047 Environments' (not a joke) and I can imagine how much time will it take to go thru all of them.

The problem with the script that in it could be 15 from 2000 environments which experience such alert type but unless I query all of them, I won't be able to see which one was...

0 · ·

Radoslaw S. ♦♦ Irakli N. · Jul 01 at 10:14 AM

It's intentionally not implemented. The way we store separately data per environment, server would need to run the workload of iterating of hundreds or thousands of envs. You rather don't want to overload server with a single query, but rather run smaller chunks and throttle requests if it's too much...

BTW - what I guess you try to do is to implement some custom alerting on top of what Dynatrace does with AI. We work for better customization possibilities in that area - we'll see what next year brings.

Anyway... 2000+ envs it pretty too much, but you probably already know that ;)

1 · ·

Answer 2 · 2020-07-01T09:41:57Z

Julius L. answered · Jul 01 at 09:41 AM

There is no such option to query multiple environments. You will have to get Api Token for each one and query them individually.

Luckily, you should be able to automate this using the Cluster API v2. I did not try this yet on my own, but you should be able to create a token management token for each environment and then create the required API token for each environment.

5 Share

10 |2000000 characters needed characters left characters exceeded

Attachments: Up to 10 attachments (including images) can be used with a maximum of 50.0 MiB each and 250.0 MiB total.

Irakli N. · Jul 01 at 10:12 AM

I just looked at cluster API v2. I See

Environments
Manage environments

Migration to multi-datacenter deployment
Migration to multi-datacenter deployment

Remote access
Manage remote access requests

Synthetic - Locations and nodes
Manage synthetic locations and nodes

Tokens
Manage Dynatrace API authentication tokens

User management
Manage user sessions

How above is going to help me? One way or another, I need to create a token for each environment and then using that token query alerts. Nothing has changed, in terms of reducing time to get my answers, right?

Thanks

0 · ·

Radoslaw S. ♦♦ Irakli N. · Jul 01 at 10:16 AM

Julius and I meant "Tokens" API that can be used to create automatically tokens for each environment. See my github example.

For the specific operation you should look at the Environment's APIs.

0 · ·

Irakli N. Radoslaw S. ♦♦ · Jul 01 at 10:30 AM

OK, folks, thank you both.

Seems I need to take the path of Ansible/bash scripting way...

0 · ·

Irakli N. · Jul 01 at 10:13 AM

Sorry, I added my comment in the main thread.

0 · ·

Radoslaw S. ♦♦ Irakli N. · Jul 01 at 10:15 AM

OK - moved.

0 · ·

question