- last edited on by
MaciejNeumann
Background: As we move closer and closer to the Holiday season a common issue is suppressing certain alerts that don't reflect a problem. For example, Low User Traffic on a holiday. There have been multiple RFEs to address these type of problems. While we wait for a direct solution from Dynatrace I'd like to offer my Pro Tip on how to handle these unwanted alerts on certain days etc...
First step: Define what it is that you want to stop alerts on, it could be one thing, or a collection of things. But we need to know what you want to exclude. Lets use this Scenario - Your organization decided to leverage a federal holiday to roll out a bunch of patches. As a result, we fully expect 2 problems. First being 'Unexpected Low user traffic' and Second being 'High CPU Usage'.
Second Step: Understand how your organization alerts. Some organizations use privatized alerting, meaning that there is an alert profile and an alert integration that shoots out an email to the defined recipients. Other organizations use an event handler, such as Moogsoft, Servicenow etc.. Never the less, you'll need to understand where your alerts go and how they alert to your customers. For this case we will use an event handler as the method can be used for privatized alerting as well.
Third Step: Now that you know what you want to suppress, and you know what type of notification delivery method your organization uses, we need to validate what alert profiles these 'alerts' you want to suppress apply to. This is easier when you have an event handler because the average organization sends everything to it. So lets go into Dynatrace and look for the low user traffic:
Now we know what alert profiles qualify for this event. And since we are using an event handler in this scenario, I know that the alert profile starting with "N" goes to our event handler. That alert profile is now my target.
Fourth Step: Lets go into that Alert Profile so we can make the changes needed to suppress the qualification of the alerts. Looking at the Alert Profile we can see al the rules are set to immediately alert, but to remove an alert aspect we want to go into "Add Event Filter"
Fifth Step: The key to this is the "Negate" function. We can select a predefined event, basically anything out of the box with Dynatrace. Then we select that out of the box event we want to target. And the final aspect is to set the 'Negate' to ignore issues that fall under Low Traffic. Now our scenario stated that their is going to be high CPU as well, So ill just make two negate rules:
Now the system will qualify alerts that pertain to the defined Management Zone, AND any defined severity rule, AND validate that the alert does not contain Unexpected Low Traffic. We can add in another rule to negate the CPU as well.
Granted yes this is a manual effort, but if you know the dates and the scopes, you can automate it via the API as well.
I hope this helps everyone this holiday season 🙂
Featured Posts
