I've created a Business Transaction in our system profile to track all external consumers of our API, split by client ID, and chart how many requests to our API each client makes per minute. Based upon this specification: the created business transaction uses a method - argument value (a bool to represent an external user) to filter by, the client ID to split by and the API Request Count (a measure that's derived from the built in "Web Requests Count") to display as a Calculated Result.
When I add this measure to a chart ("Add Series" on new dashboard) it works as expected. However, when I create an Incident based on this measure, the incident raises for the wrong unit of measurement. The upper threshold on the measure is 2250 COUNT! As in 2250 hits per minute. Yet, the incident is raised on 2250ms or when an API call takes 2.25 seconds! The Incident is simply reacting to the wrong unit of measure. Why?
Answer by Anthony G. ·
I set the incident aggregation to Avg. because I want the incident to calculate the average count of web requests in the Evaluation Timeframe. I hope it's implicitly performing this behavior.
All in all it looks good! I'll mark this as resolved as soon as I run some more tests to make sure the data is copacetic.
Answer by Anthony G. ·
VIOLA! I think that worked. The alert email I receive now looks a lot the ones I was getting when it was working :
Answer by Andreas G. ·
Can you try setting the "Aggregation" in your Incident rule from Average to Count and verify if that makes a difference? It would then use the "number of PurePaths" that have this splitting instead of the Response Time of that PurePath
Answer by Anthony G. ·
Here's a copy of the email I received. It is not based on any breach of threshold. Once upon a time this alert worked. When it did, it would provide me with each violating client's partition ID in the Incident email received. The metric used to generate the email does not seem to be right. It appears as if the incident is triggered based off of the API request response time (in milliseconds) versus the number of API requests performed by a single client:
Answer by Andreas G. ·
Thanks. And if you put that same measure on a Chart using 15 Minute Average Aggregation - how does that chart look like?
JANUARY 15, 3:00 PM GMT / 10:00 AM ET