• Forums
    • Public Forums
      • Community Connect
      • Dynatrace
        • Dynatrace Open Q&A
      • Application Monitoring & UEM
        • AppMon & UEM Open Q&A
      • Network Application Monitoring
        • NAM Open Q&A
  • Home /
  • Public Forums /
  • Application Monitoring & UEM /
  • AppMon & UEM Open Q&A /
avatar image
Question by Eyal P. · Jan 29, 2015 at 08:35 PM ·

help needed - deep thread starvation and memory analysis on production

hello

we have an .net application (client/server) that have a serious memory leak problem causing them to stop responding and for the users to get 

the 4xx http messages.

the application is based on .net WCF services (on the servers) .

the servers are virtual based on VMWare - configured as shared use of the infrastracture capabilities.

each of them is designed to handle up to 16GB of RAM (to all the instances of the worker processes) and using a virtual 8 core CPU.

each of the servers is hosting a single application pool but with 5-10 instances of worker processes.

each of the worker processes is handling user requests by opening a paralel calls to an oracle database.

the worker processes internal threads are controled through .net 4.0 thread monitors and contacting the database through dbpool.

we use the dal of the enterprise library ... so we didnt overriden any of the basic db connection handling (no close() or dispose()).

 

we see in DT client that there are a lot of thread "sync" time and the memory is not cleaned (GC) till its way out of the worker process 1gb limit

(we can see up tp 2.5GB of memmory taken by the app)

we are monitoring the app for several days and while it keeps on crashing (especially on full load)... we can see that one thread is occuping

the CPU for 13% ... the rest of the threads are on WAIT status and never get CPU time ....

while this is hapaning ... we can see an increament on memory and than "crash".

 

so now for our question....

do any of you have any ideas how we can identify such an illusive application error ?

maybe you all have any suggestion on directing us upon monitoring this solution.

 

we have used already the threads deshlet and we are dumping memory just when they reach above 90% of memory...

but still.... no pinpoint for the exact cause.

 

thank you for your help.... the application is a very critical and couse us not just PR problems but mostly money transfer and handling ones.

 

therefore ... i'll very mush appriciate your help and cooperation

 

 

 

Comment

People who like this

0 Show 0
10 |2000000 characters needed characters left characters exceeded
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 10 attachments (including images) can be used with a maximum of 50.0 MiB each and 250.0 MiB total.

1 Reply

  • Sort: 
  • Most voted
  • Newest
  • Oldest
avatar image

Answer by Kristof R. · Jan 30, 2015 at 12:18 AM

Hi Eyal,

Some more thorough investigation is needed to analyse this problem.

This seems like something that the Dynatrace Enablement Services team can help you out with.

I have forwarded your query to your Customer Success Manager who will get in touch with you shortly.

KR,
Kristof 

Comment

People who like this

0 Show 1 · Share
10 |2000000 characters needed characters left characters exceeded
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 10 attachments (including images) can be used with a maximum of 50.0 MiB each and 250.0 MiB total.

avatar image Eyal P. · Jan 30, 2015 at 12:26 AM 0
Share

thank you very much

i'm appreciate your effort to help

 

 

How to get started

First steps in the forum
Read Community User Guide
Best practices of using forum

NAM 2019 SP5 is available


Check the RHEL support added in the latest NAM service pack.

Learn more

LIVE WEBINAR

"Performance Clinic - Monitoring as a Self Service with Dynatrace"


JANUARY 15, 3:00 PM GMT / 10:00 AM ET

Register here

Follow this Question

Answers Answers and Comments

2 People are following this question.

avatar image avatar image

Forum Tags

dotnet mobile monitoring load iis 6.5 kubernetes mainframe rest api dashboard framework 7.0 appmon 7 health monitoring adk log monitoring services auto-detection uem webserver test automation license web performance monitoring ios nam probe collector migration mq web services knowledge sharing reports window java hybris javascript appmon sensors good to know extensions search 6.3+ server documentation easytravel web dashboard kibana system profile purelytics docker splunk 6.1 process groups account 7.2 rest dynatrace saas spa guardian appmon administration production user actions postgresql upgrade oneagent measures security Dynatrace Managed transactionflow technologies diagnostics user session monitoring unique users continuous delivery sharing configuration alerting NGINX splitting business transaction client 6.3 installation database scheduler apache mobileapp RUM php dashlet azure purepath agent 7.1 appmonsaas messagebroker nodejs 6.2 android sensor performance warehouse
  • Forums
  • Public Forums
    • Community Connect
    • Dynatrace
      • Dynatrace Open Q&A
    • Application Monitoring & UEM
      • AppMon & UEM Open Q&A
    • Network Application Monitoring
      • NAM Open Q&A