SUD Unavailable Incident Rule
We seem to have been getting Incident notifications on the Incident dashboard for an Incident rule called 'SUD Unavailable' whenever our application shuts down. The interesting thing to note is that there is no measure defined in the incident rule. I am not able to figure out the reason how can an incident be triggered without any conditions defined. We have some other incident rules as well that behave this way.
Any help would be greatly appreciated!
Amit
Hi Amit
dynaTrace has several Built-In Incidents - such as this one. They work on internal measures and are therefore special. So - you wont see an actual measure for some of these Built-In Incidents
Andi
Thank you for the very useful information, Andi! I have one more question which I think is perhaps on the same lines.
I see that there are two Incident rules defined with the same measure and threshold values but with different severity levels.
For example, If we open up the Incident rules 'MS GC Utilization' and 'MS GC Utilization-severe', we will find that the measures defined for both the incidents rules are 'MS GC Total Utilization' and 'MS Memory Utilization' and these measures have the same threshold values in the both the incident rules.
Is there a reason behind having two separate incident rules with same measures (and off course same thresholds) but different severity levels?
Also, the way I have seen this to be working so far is that whenever a violation actually occurs, it fires the incident with severity as warning or severe and not severe. I don't know why it works this way.
Amit
The idea with two Incidents on the same measure is that you can trigger different actions for Warning and Severe Threshold Violation.
Example: Free Memory with Warning to 40% and Severe with 10%
The Incident based on the Warning can send an email to the IT Ops Guy.
The Incident based on the Severe Threshold can send a Text Message, Email, Ticket, ...
This is how it is intended and how it should work
Makes sense?
Thanks Andi, but I think I am still not clear on this because what I see is that a measure say, Free Memory has threshold values for Upper Warning set to 40% and Upper Severe to 10%. Now, this same measure is used in two different incident rules say, 'Free Memory Rule' and 'Free Memory Rule-Severe'. Free Memory Rule uses Free Memory measure with threshold set to 'Warning or severe' while Free Memory Rule-Severe uses the same measure with threshold set to 'Severe' As the Free Memory reaches 40% (Upper Warning) would it not trigger both the incidents and thus the actions associated with them together?
Amit
In your example we need to talk about the "Lower" Warning and "Lower" Severe threshold because we are talking about a measure that represents a worse state when it reaches a lower value.
So - in your case - if you use the Lower Warning of 40 and Lower Severe of 10 the following should happen.
- If Free Memory reaches, e.g: 35 the Incident will trigger that is based on the warning threshold
- If Free Memory reaches, e.g.: 5 both incidents should trigger as 5 is below 40 (warning) as well as below 10 (severe)
In the warning action we should do something like sending an email. Additionally to that we trigger a severe action that is more noticable, e..g: Sending a Text Message. In this case we get both an email as well as a text message if our memory is below 10
I hope this makes sense.
