PG detection rule creates 100 individual processes - what's the overhead?

[kalle_lahtinen]

Hi,

I have a case where a single Java process is launching batch jobs based on command line arguments. I've been able to define a process group detection rule, which separates these jobs into approx. 100 separate process groups. This helps identify the individual batch jobs better, and also define custom services with the scope being for a certain batch job only. If I would instead define the custom services for the main Java process only, the resulting data would be a mess because several of these methods are being called from various jobs, and I wouldn't know anymore which job is the call really originating from.

My only worry here is that since this single PG detection rule appears to be pretty powerful, am I causing extra overhead by having OneAgent monitor & report these as separate instances? I know there's some overhead for Deep Monitoring, which is not straightforward to analyze - you can only see the OS/NW/Log monitoring impact, but not really the Deep Monitoring part. Does anyone have insight on this; is the PG detection rule only changing the way data is sorted @ Dynatrace UI, or is there also an impact on the agent's footprint when we compare monitoring the single main Java process vs. the 100 individual processes?

[r_weber · Accepted solution]

Hi @Kalle L.,

I don't think that this will cause an issue. The instrumentation part is the same and a few hundred process groups shouldn't matter either on the Dynatrace side. I've been running installations with a few thousand process groups.
What you should consider though is that not for every INSTANCE of such a job a process group is created. e.g. if you start a batch job ABC today and then tomorrow again it would be the same process group "Job ABC" and not today the PG "Job ABC - <date>" and tomorrow the PG "Job ABC - <otherdate>".

Reinhard

[kalle_lahtinen]

Thanks for the response! Hands-on experience like that is always appreciated, good to hear then that my setup shouldn't likely result in any issues.

Regarding the last part; the PG detection rule I have doesn't indeed create a new PG every time a new batch job is launched. So the no. of process groups has actually remained stable, and I haven't seen any duplicates there. Meaning: for each batch job name, there's exactly one unique PG identifier.

One followup question on your comment "I've been running installations with a few thousand process groups." -> You mean split between several different servers though? In my case, this is a cluster of 2 servers where the batch jobs are running. So as a result, in the host view I have those ~100 Java processes listed per server. It does look a bit odd in the UI, when you're used to seeing somewhere around 5 to maybe 20 processes there.