question

Julio M. avatar image
Julio M. asked ·

Automatic Thread Dumps are not created when incident is triggered

I have an incident rule for a custom metric, watching over Glassfish HTTP Threads listener queues.

The incident is created as expected when the conditions are met, but the action to trigger Thread Dumps on that incident is not working.

I'd expect 10 thread dumps to be created when the incident is triggered, but nothing is captured.

Am I missing something?

appmonconfigurationappmon 7java
1572027938359.png (176.1 KiB)
1572027950801.png (79.4 KiB)
10 |2000000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 50.0 MiB each and 250.0 MiB total.

1 Answer

Joe H. avatar image
Joe H. answered ·

I'm not familiar with the Glassfish HTTP Queue metric. Is this possibly a metric you're getting from an API (JMX?), and as such may not have an associated host where threads could be captured?

I would also suggest looking in the agent logs and seeing if there's any information which explains why thread dumps can't be captured.

I was able to reproduce your situation using the Current CPU Load metric (% value), by setting the threshold to 3%. Wasn't hard to trigger the incident and each time I got a thread dump. Perhaps as a test, setup a new incident on a different metric (CPU consumption) and see if you can trigger the dump. Be sure to use a metric that's directly coming from a JVM.

Let us know your results.

1 comment Share
10 |2000000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 50.0 MiB each and 250.0 MiB total.

Glassfish HTTP Queue is a custom metric we collect using the JMX Plugin. It's not a standard JVM metric, but I'd expect the incident rule to apply just the same.

I never found any error messages from the agents related to the thread dump.

Besides, I configured another incident based on a metric from PurePath response time. That one seems to work fine and Thread Dumps are triggered by incidents.

0 Likes 0 · ·