Dynatrace PurePath From Agentless RUM Insertion

[mikerj83]

We have sites that are ReactJS applications. These applications are static in that they do not require an application server to be hosted. They interact with other services (on a different host). These applications are deployed to an Azure Static Application instance. This would also be similar to uploading the site to AWS S3 and then placing a LB in front of it for public access.

In order to get RUM for the sites we've enabled agentless monitoring and have placed the `script` tag as provided by Dynatrace in the deployed site. This works well. We're able to get the basic RUM information and use the client library to post custom errors, etc.

Where we're falling short with our monitoring is with PurePaths that map to the services traces. Our services run in an AKS Kubernetes cluster that is has OneAgent installed. We can see the requests enter the services and we can get a distributed trace. We do not, however, see a linkage between the client requests on session from the RUM monitoring and the services' distributed traces.

When we have a customer mention an error that they receive on the client we can often see that one was logged by the RUM agent running in the browser. However, we have to look through logs with somewhat of an idea of the error message in order to then correlate it to a distributed trace.

How can we get the PurePath to work from the time a user makes an action in the client web application and through to a database call? Are we missing some configuration? Is this because we have the client applications running on one domain, app1.ourDomain.com, and our services running on services.ourDomain.com (like a CORS issue)?

[Fin_Ubels]

Hey Mikerj,

This should be possible with a bit of extra configuration assuming that the technologies used are supported. The following documentation offers solutions, it just takes providing a little bit of extra configuration to tell the agentless JS where it can send Dynatrace headers to.

Link cross-origin XHR user actions and their distributed traces | Dynatrace Docs

Hope this helps!

[mikerj83]

Hi @Fin_Ubels,

I had previously read that page. We've been having some "fun" configuring the `withCredentials` portion in our API gateway. I decided to attempt to set up the x-dtc header (https://www.dynatrace.com/support/help/shortlink/link-cross-origin-xhrs#x-dtc-header). This is in the configuration section Link cross-origin XHR actions and their distributed traces.

As a note:
UI resides on: agents-stage.ourhostname.net
Services run on: services-stage.ourhostname.net

For the regex for the I've provided: services\-stage\.ourhostname\.net

Here's what gets generated for x-dtc.

sn="v_4_srv_-2D41_sn_8K1KCUSGKNRFQ837GVT385HV9OPB290H", v="1690815112867MPD55SR31FEF4KTAQ1DSHBR562JTG4V9", app="e1ef25463e3e38d8", r=""

Does this look correct? I'm still showing 0 service requests for the app in RUM monitoring.

I suppose that if this looks correct I need to verify that the header is getting through to the actual services running in Kubernetes. My assumption is that they are because we're not stripping or altering them. Is there any configuration that is necessary from the services side in Kubernetes that could be preventing this from working (like telling one agent to look for it)?

[mikerj83]

I was able to get a piece of middleware in our services that captures the headers and logs out if it finds the "x-dtc" header. It does:

The x-dtc header is present with: "sn=\"v_4_srv_3_sn_CCJ19ML2C5C334PT5LFR0SL27B25IFE6\", v=\"1691418046413UUERT5GT2P2DKMFE77EDH3GA61O60CDH\", app=\"41c958ff353186b7\", r=\"\""

Not quite sure what else could be the issue. Is there a way I can see if the OneAgent running in the cluster sees that?

[agrawal_shashan]

@mikerj83 Hey were you able to get this working? We have the exactly similar situation in GCP where we don't see any traces. We have done all the configuration for linking cross origin domain but still no luck. Let me know if you got it working.

[Romanenkov_Al3x]

@agrawal_shashan Hello, can you describe how looks like full path?

If your infra look like (User -> ServiceMesh component -> SomeFrontWebPod -> SomeBackPod -> DB) is expected behavior.

link

On the following web servers and applications, Dynatrace supports user action to distributed trace correlation for XHR requests

I have same problems and main reason was that unsupported tecnology of the web server.

If it possible or you have test\stage env you can try to exclude ServiceMesh component to check this.

In this case you will have link between user action and service traces.

Also I know that RD team works on improvement. Also I faced with this approximate 1 year, maybe now this scenario works.

Regards,

Alex

[mikerj83]

@Romanenkov_Al3xBoth @agrawal_shashan and I are attempting to get the cross origin fixes to work. My assumption is that as long as the cookie or header makes it to the instrumented service the trace should work. I've used another APM solution in the past where this exact scenario and issue were the similar. Solving it involved a solution close to what Dynatrace outlines.

One thing I'd like to understand is if there is a way to turn on a logging level to see if the OneAgent instrumentation of the service "sees" either of the fixes. My hope is that there is some log event we can look for and see if the agent is rejecting it for some reason. As mentioned I can get the value of the header from within my code, but perhaps it is mangled in some odd format that causes the agent to not parse it. As it stands it should work but we're not having luck.

[agrawal_shashan]

Hi @Roms Our flow looks like this -

User -> browser -> Angular App (hosted on GCP S3 & monitored by agentless RUM) -> APIGEE gateway (managed by Google) -> Kubernetes Cluster Services (Instrumented by One agent).

We have cross domain XHR calls which means my website is hosted on one domain and APIGEE XHR calls are hosted on another and because of that I have already applied that configuration for supporting cross domain XHRs.

Still after lot of investigations, we are not able to see the traces. And the frustrating part is Dynatrace documentation is not very plain and simple. There are lot of things which it doesn't support yet

Best Regards,
Shashank

[mikerj83]

@agrawal_shashanI have not. We've raised the issue with the Dynatrace team on a monthly call and I have not received a solution at this time. Did you attempt the with-credentials or x-dtc header method of cross origin domain fixes? Are you using an API gateway in front of your service layer? We're running Kong and I had long suspected that it was doing something with CORS or the header but a piece of middleware that I enabled as a test was able to log the x-dtc header out.