Timeouts - Async Path (threading)
We've started noticing that more and more of our requests are getting timeouts, I've begun digging into this but the only thing I've able to see is, in the purepath tree (on one of those requests) there was a sudden jump (of 30 sec) between async invocation and async path (threading). Further down in the tree our log4net reports that the timeout expired.
async-path-threading.png
Might be completely off with my "finding" but if anyone has any ideas of where to look I'd greatly appreciate it!
Hey Simon
I've seen this situation many times in the past. This typically indicates that your system is running low on worker threads. The PurePath you posted is trying to schedule a job that gets executed on a background thread. It takes 30s until the next thread becomes available. Now - 30s is a very "suspicous" time for me. I "assume" that the real problem is in some of your background thread processing where these threads are doing some work or waiting on something for up to 30s. After 30s they probably hit some type of timeout and then "finish/abort" their job. this means that the thread becomes available again which means they can pick up work from e.g: the PurePath you showed.
I would look into other PurePaths that show activity in these background threads. Try to figure out if they do some heavy processing, connect to external systems (or DB) and whether they run into any timeouts. I good way to start is by looking at the Exception Dashlet and try to find any "Timeout" exceptions. then drill to the PurePaths from there. Another option is to open the method hotspots dashlet -> this should show you if you have methods that wait a long time for a response from an external system
hope this helps
andi
