question

Julio M. avatar image
Julio M. asked ·

Exception during simple memory dump: out of message buffers

I'm having problems creating memory dumps for Java Agents. I can see the DynaTrace server times out wating for the dump:

2019-10-01 16:48:24 UTC WARNING [MemoryDumpTaskHandler] Dump Watchdog timeout: did not reach state 'readWriteData' after timeout of 600s. Waited for 600000ms, ending Dump
2019-10-01 16:48:24 UTC WARNING [MemoryDumpTaskHandler] Canceling Memory Dump since we did not receive any data from the agent
2019-10-01 16:48:24 UTC WARNING [MemoryDumpTaskHandler] New state 'finished' does not match expected state 'readWriteData'
2019-10-01 16:48:24 UTC INFO [MemoryDumpTaskHandler] Finished creating memory dump for agent Projudi-PRD_Producao@projudi10-prd:18962
2019-10-01 16:48:24 UTC WARNING [MemoryDumpTaskHandler] Unable to switch to new state 'finished' because dump task has already finished
2019-10-01 16:48:24 UTC WARNING [MemoryDumpTaskHandler] Memory dump canceled or failed for agent Projudi-PRD_Producao@projudi10-prd:18962
2019-10-01 16:48:24 UTC INFO [MemoryDumpTask] Creating Memory Dump for agentinfo[sessionReference=LiveSession[systemId=Producao], agentGroupReferenceKeyId=Projudi-PRD, agentMappingReferenceKeyId=Projudi-PRD_Producao, host='projudi10-prd' processId=18962 agentId=446006074, mode=direct]

... and at the agent log file, there's an Exception warning.

2019-10-01 16:43:43.503 UTC [13ff883f] warning [native] Exception during simple memory dump: out of message buffers
2019-10-01 16:48:25.557 UTC [13ff883f] info    [native] SimpleMemoryDump: Dumping ... (not forcing GC)
2019-10-01 16:48:41.454 UTC [13ff883f] info    [native] SimpleMemoryDump: Dump completed.
2019-10-01 16:48:44.489 UTC [205e083f] debug   [java  ] [httpclient]   - Failed to create artificial subpath
2019-10-01 16:53:45.990 UTC [13ff883f] warning [native] Exception during simple memory dump: out of message buffers

Would anyone be able to tell me what the error message means?

Using AppMon version 7.2.17.1032.

Best regards.

7.2errors
10 |2000000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 50.0 MiB each and 250.0 MiB total.

1 Answer

Sebastian K. avatar image
Sebastian K. answered ·

First question is if there are free resources to make such dump. If host is exhausted it may be problematic. I’m not sure if this is case here.

Sebastian

2 comments Share
10 |2000000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 50.0 MiB each and 250.0 MiB total.

At the moment, the host where the agent is running does not seem to be exhausted.

# free -h
             total       used       free     shared    buffers     cached
Mem:           62G        62G       352M       744K       283M        39G
-/+ buffers/cache:        23G        39G
Swap:         2.0G        16M       2.0G

From 64GB of RAM, the JVM is limited to 32GB. Gathered most of the used memory shown here is OS cached, the application is far from reaching the OS limit.

0 Likes 0 · ·

Allow me to add the fact that I'm trying to take a series of Memory Consumption Trend snapshots. These are not full dumps, so the ammount of memory required should not be that high.

0 Likes 0 · ·