• Forums
    • Public Forums
      • Community Connect
      • Dynatrace
        • Dynatrace Open Q&A
      • Application Monitoring & UEM
        • AppMon & UEM Open Q&A
      • Network Application Monitoring
        • NAM Open Q&A
  • Home /
  • Public Forums /
  • Application Monitoring & UEM /
  • AppMon & UEM Open Q&A /
avatar image
Question by Ankur S. · Aug 13, 2014 at 02:23 AM ·

Memory Leak Dump - No application specific class

Hi,

For a customer OOM event on one of the jvms (IBM Websphere) java 1.6 , created leak analysis thread dump.

While analyzing that memory dump cannot see any application related classes in different views. Hotspots tab showed close to 2 GB occupied by 'bytes' objects but cannot identify whether those are related to application or native server packages.

I have requested memory dump from customer to share, meanwhile want to know if any one came across this scenario, where application specific classes / methods not shown in memory dump ?

 

-Thanks

Ankur

Comment

People who like this

0 Show 0
10 |2000000 characters needed characters left characters exceeded
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 10 attachments (including images) can be used with a maximum of 50.0 MiB each and 250.0 MiB total.

3 Replies

  • Sort: 
  • Most voted
  • Newest
  • Oldest
avatar image

Answer by Ankur S. · Aug 21, 2014 at 10:57 PM

Thanks for help Rick.

Let me carry out these steps, will get back to you.

-Thanks, Ankur

Comment

People who like this

0 Show 0 · Share
10 |2000000 characters needed characters left characters exceeded
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 10 attachments (including images) can be used with a maximum of 50.0 MiB each and 250.0 MiB total.

avatar image

Answer by Ankur S. · Aug 21, 2014 at 03:13 AM

Hi Richard,

With 'Shortest path to root' I can see some classes related to application code (search with 'hdb') but still not able to relate which one is causing leak. (Application architecture is bit complex - 30-40 applications running in same jvm).

Memory dump show Byte classes creating Vector instances which occupying more and these vector instances point to ibm jars.

memory files are quite big, check if you are able to download from this link : ftp://ftp.compuware.com/pub/cso/singapore/outgoing/wasp FTP.zip

If not attached a ppt showing some screenshots. Memory Dump Screenshots.pptx

Please check if you can and let me know how to proceed.

Thanks, Ankur

Comment

People who like this

0 Show 1 · Share
10 |2000000 characters needed characters left characters exceeded
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 10 attachments (including images) can be used with a maximum of 50.0 MiB each and 250.0 MiB total.

avatar image Rick B. · Aug 21, 2014 at 04:44 AM 0
Share

Here's what I would do from here (can't download the session files but your screenshots contain everything I think I need to see).

  1. Put a memory sensor (through right-click -> Add Sensor Rules, or through the Sensors dialog in Edit System Profile) on com.ibm.mm.sdk.server.DKResultSetCursorICM class,
  2. configure the sensor properties to show the allocations on the PurePaths, then
  3. restart the app and take Selective Memory snapshots as the heap grows.  This will tell you what the allocating method is for the parent objects. 
  4. If that method is still not in their code, take a CPU sample (very high frequency for 30-60 seconds under load) and trace where that method is being called down the call stack to the customer's code.

Rick B

avatar image

Answer by Rick B. · Aug 13, 2014 at 05:48 AM

There are lots of fringe explanations for something like this:

  • Undersized Heap
  • Abusive Object allocation (so not a technical leak, just not enough memory to function)
  • An actual honest-to-goodness bug in the container or third-party APIs

Most often, though, the "hotspots" view is the starting place to find an answer, rather than the answer itself.  Do you mean that there is one byte[] listed in hotspots as having GC size 2 GB?  That's a lot.  Is there a similarly sized object in the Collections and Maps tab?  If you right-click the byte[] and follow the references to root, do you see application code there?  What sorts of classes are they?  How are they being implemented by the application?  Once you have more of these answers it should be more clear what is happening in the application.

HTH

Rick B

Comment

People who like this

0 Show 0 · Share
10 |2000000 characters needed characters left characters exceeded
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 10 attachments (including images) can be used with a maximum of 50.0 MiB each and 250.0 MiB total.

How to get started

First steps in the forum
Read Community User Guide
Best practices of using forum

NAM 2019 SP5 is available


Check the RHEL support added in the latest NAM service pack.

Learn more

LIVE WEBINAR

"Performance Clinic - Monitoring as a Self Service with Dynatrace"


JANUARY 15, 3:00 PM GMT / 10:00 AM ET

Register here

Follow this Question

Answers Answers and Comments

2 People are following this question.

avatar image avatar image

Forum Tags

dotnet mobile monitoring load iis 6.5 kubernetes mainframe rest api dashboard framework 7.0 appmon 7 health monitoring adk log monitoring services auto-detection uem webserver test automation license web performance monitoring ios nam probe collector migration mq web services knowledge sharing reports window java hybris javascript appmon sensors good to know extensions search 6.3+ server documentation easytravel web dashboard kibana system profile purelytics docker splunk 6.1 process groups account 7.2 rest dynatrace saas spa guardian appmon administration production user actions postgresql upgrade oneagent measures security Dynatrace Managed transactionflow technologies diagnostics user session monitoring unique users continuous delivery sharing configuration alerting NGINX splitting business transaction client 6.3 installation database scheduler apache mobileapp RUM php dashlet azure purepath agent 7.1 appmonsaas messagebroker nodejs 6.2 android sensor performance warehouse
  • Forums
  • Public Forums
    • Community Connect
    • Dynatrace
      • Dynatrace Open Q&A
    • Application Monitoring & UEM
      • AppMon & UEM Open Q&A
    • Network Application Monitoring
      • NAM Open Q&A