Total and Selective Memory snapshots with DT – Error while capturing, Potential Native Memory Leak
I am evaluating DynaTrace tool to identify a potential memory problem in a java based web application.
Please see attached screenshot of taking memory snapshots with DT. First I found a error/warning message and then saw a ‘Native memory leak’ message in the actual snapshot. Is this expected?
Also I found a significant amount of memory used by Dynatrace specific objects in the heap memory. I am not sure why dynatrace specific objects present in the heap even after garbage collection (triggered during snapshot).
Hi
Thanks for the screenshots. Which JVM are you running your application on? Just curious becuase you have this warning in the beginning regarding the String value capturing.
As for the native memory leak warning. This is a warning to you as you have 427MB in native memory used. This can be because your app uses a lot of native libraries and that is expected. It can however also mean that your native libraries have in fact a native memory leak.
It is very odd that dynaTrace classes show in that quantity. It could be related to the "string value capturing may cause a problem" or it could be related to one of the Sensor PAcks you have placed. Can you attach your System Profile? I would be interested in having a look at it. In your cockpit (tree on the left) right click on the System Profile and select "Export System Profile".
thanks
Hi
I just received some of your memory dumps and other files such as the System Profile. Here are my observations that hopefully help you move forward with your evaluation
#1: The Memory Dump I received contained the Captured Strings as well as captured primitive data. So - it seems that this is working as expected. Could it be that the first dump you created as highlighted in your doc wasnt taken with that checkbox on?
#2: You have a very large list of Memory Sensor Rules. For instance you configured to selectively monitor all objects in com.bat*, com.it3.*, ... - and specifically you specified a rule for java.lang.String. Now - this is of course possible but it also means that dynaTrace instruments all these classes in order to keep count of current instances which you can query through a selective memory dump. Especially the rule for java.lang.String is however questionable if this is really what you want. Having that said. This explains the large number of dynatrace related objects on your heap because we need to keep track of instances. If you selective monitor a class like java.lang.String where millions of String objects are created it is no surprise to see dynaTrace objects on the heap as well. The question really is what you want to achieve with dynaTrace and why you defined that many MEmory Sensor Rules. If you want to find memory leaks it is better to use our Memory Snapshots for Leak Analysis. These are lightweight and give you insight into EVERY class and not just those where you defined a memory sensor rule. Memory Sensor Rules are meant to be used in Production Environments where you want to focus on a small set of your custom classes - NOT on java.lang.String though. If you want to learn more about Memory Leak Analysis please check out our recorded webinar: dynaLearn Webinar - Memory and Thread Diagnostics - February 15, 2012
#3: I also see that you have specified very generic Method Sensor rules for com.bat.*. That means that dynaTrace will instrument every method in that package. The question is if this is really what you want. dynaTrace has a feature called Auto Sensors that will record ANY method in case it is a potential performance problem. You dont need to define custom sensors for that. Custom Sensors are more targeted to instrument individual methods of interest - typically also to capture method arguments or return values as you want to get more context about an individual method call within a transaction. So - I would recommend removing that Sensor Rule but create individual rules for methods that you are really interested in
I hope this helps. To sum up - here are the next steps I propose
#1: Remove your Memory Sensor Rules and start with a Leak Analysis as described in the Webinar or on the following doc page: Memory Analysis
#2: Remove your generic Sensors and see what Auto Sensors give you. Also - learn more on Auto Sensors here: Auto Sensor
#3: Define Custom Method Sensors for methods that are of interest - also - check out our webinar for this: dynaLearn Webinar - Advanced Sensor Rules - September 5, 2012
#4: Define Custom Memory Sensor Rules for your own classes that you really need selective memory dumps in a production or high load environment. Otherwise stick with the Trending Memory Dumps accessible through the Total Memory Dashlet
I hope this helps
Andi
