Impact of agents in one data center send data to collector in another data center
Hi,
We know that dynatrace agent is very light weight and there shouldn't be any latency between the agent and the collector.
We have a situation were we had to configure agent in app servers in different data centers and point those agents to collectors in another data center. And we have many location around the country where we have agents that talk to collectors in data center.
we are assessing the impact of such a configuration to all components involved in the dynatrace system like dynatrace server, collector, agents and applications being monitored. We are trying to see the impact because it would be too costly for us to place collectors in each locations across the nation to avoid this agent collector latency. Also these are .Net agents, this also might add some complexity.
-Sreerag
Could some help in identifying the impact due to such configuration.
Hi there
Obviously this is not a supported scenario as there should be a collector on the "local network"
I have used this previously though when we just could not get a collector into the remote data center. In my experience it can be done but keep the following in mind:
- At start up, the .NET process will take a much longer time to start-up. The instrumentation happen at start up between the collector and the agent. You will probably have to stagger the start-ups. Even then, a lot of the services will fail to start-up as the windows Service Manager will think the service takes too long to start-up and kill the process.. There is ways to hack the registry to increase the timeouts for the .NET process start-up.
- Depending on the type of application, there could be a lot of data that is being send to the remote collector. You loose the ability to compress the data and thus there could potentially be a LOT of data crossing your WAN. That might kill the network and impact your application performance.
- Managing the firewalls between the WAN nodes can also be a problem, as you have to open up many port-server combinations, instead of just the collector-to-server one. If you are using monitors on the remote servers, this can become quite tricky.
I hope this helps. Good luck!
Anton
One more important thing to note here, if the transformation takes too long the app will start without the instrumentation. This really only affects .NET and Java agents but it is significant because sometimes when it starts up it will be instrumented and sometimes it may not be. You can increase the wait timeout so it essentially forces the agent to instrument classes despite latency but this can cause VERY long startup times.
That is an interesting point. We have seen few agents without instrumentation but those were not the ones that send data too long over the wire to a different data center. But we will keep and eye on all agents that have this issue. Also on the application startup time being long should be ok for us as we usually start the application in the night.
Thanks Anton for the inputs,
-Sreerag
