Visit Coverage - Capture percentage of Visits
Hi,
How does Dynatrace capture percentage of visits when enabled under Visit Coverage–> Capture following percentage of visits. What is algorithms and on the basis of what it decide the percentage.
Please help to understand this.
Sachin
Hi Sachin
the "Visit Coverage" will impact our automatic injection of our JavaScript agent. It says that in the help and in the statusbar of that setting. So - for example. If you set it to 50% we will only inject our dtagent on web rquests for every other real visitor you have. It means we only capture UEM data for 50% of your users.
One thing to be careful with. This Coverage % ONLY impacts our automatic injection. That means - if you do manual injection we capture everything where our dynatrace agent is injected. Also - if you have CDNs or proxies and they cache a version with an injected agent we will capture them.
I hope this makes sense. more information here: System Profile - User Experience
Hi Andreas
I was wondering if you know which kind of algorithm Dynatrace uses for determining which visits to instrument and which not to. Also do you happen to know how long it takes to do this calculation at server side?
Thank you very much.
Axel
I think its a pretty simple algorithm. If you specify 50% coverage then we generate a random number between 1 and 100 and if the number is <= 50 we inject the agent. Also - please remember that this only impacts our automatic injection. if you have a CDN or proxies that cache your HTML page with an injected dtagent all of your users that get that HTML will be covered
With dynatrace 6.2 we introduced a new way for our manual dtagent injection which provides "bootstrapping" capabilities. So - you simply load a "dtagentbootstrap.js". this will then download the correctd version of your JS file - BUT - it will also check the Visitor Coverage setting
Andi
Thank you very much Andreas for your reply.
It makes sense this explanation, still the customer wants a "detailed" explanation of this algorithm. They would like to see a white paper or documentation on this but I don't think there is such document in the community. By any chance do you have a detailed explanation of this process?
Thank you really much, really appreciate your help.
Axel
Hi Axel,
for every new visit the injection state is evaluated once and persisted on the dtCookie. The algorithm works as follows:
- injectionPassCount and injectionBlockCount are calculated from visit percentage, e.g. for 30% injectionPassCount=3 and injectionBlockCount=7
- for each shouldBlockByPercentage call an index is incremented by 1
- shouldBlockByPercentage = ( ( index % (injectionPassCount + injectionBlockCount) ) >= injectionPassCount )
- index=11: 11 % 10 = 1 -> 1 < 3 -> not blocked
- index=17: 17 % 10 = 7 -> 7 > 3 -> blocked
hth,
Clemens
Hi Clemens
Thank you very much for your response. Let me see if I understood correctly how the algorithm works:
injectionPassCount equals the visit coverage and injectionBlockCount equals the difference to reach 100% (in 50% both PassCount and BlockCount equals 50%?).
I assume index is incremented every incoming visit? In this case index would be allowed from 10 to 13, blocked from 14 - 19 and would allow it from index = 20 - 23, blocking from 24 - 29 and so on.
Is that right? Again thank you very much.
Best
Axel
Note: as of AppMon 6.2, this statement is no longer true:
"This Coverage % ONLY impacts our automatic injection. That means - if you do manual injection we capture everything where our dynatrace agent is injected. Also - if you have CDNs or proxies and they cache a version with an injected agent we will capture them"
As of AppMon 6.2, the beacon processing has been upgraded so that it adheres to the Visit % Coverage configuration setting as well. So now only the proper amount of visits will be captured based on that desired setting.
Rob
Sachin,
Let's suppose you get 100 visits from branch A, 200 visits from branch B and 300 visits from branch C, for a total of 600 visits. At 15%, you want to trace 90 visits total.. Dynatrace will trace 15 visits from branch A, 30 visits from branch B and 45 visits from branch C.
If you want to trace an equal number of visits from each branch, in this example 30 visits from each branch, you'll need use manual injection and add JavaScript code to your web pages. Note that this can break down if the distribution of visits is too uneven. For example, if you get 190 visits from one branch and 10 visits from another, you can't trace 15 visits from each branch because one branch only got 10 visits.
The problem for Dynatrace is that even if the branch name was visible somewhere in the transaction, it would be discovered too late to affect JavaScript agent injection.
Just for completeness, although you can turn injection on and off by IP address range, the IP address is not taken into account when deciding, if the injection rate is less than 100%, whether to trace a particular visit.
-- Graeme
Sachin,
I'm not sure I understand the problem.
If you set the UEM injection rate to 15%, on average 15% of the visits from each branch will be traced. Depending on the number of branches and the number of visits from each branch, the statistical variation should be negligible. I guess I'd like you to build a "Visits split by branch" business transaction so you can see exactly what's happening in your environment.
If you need something more exact than the random method the automatic injection mechanism uses, you'll need to code something yourself in JavaScript. Since this will be simpler in version 6.2, my strong recommendation is to at least see what happens using the automatic mechanism for the next few weeks.
By the way, you're right that the automatic injection rate is set per application.
-- Graeme
HI Graeme,
In my case branch means we have group of IPs and users are not equally distributed across branches so we can get high visits from few branches and may get minimum visits from few branches so we cant get equal visits from all branches using percentage of visits. So we may not get 15% visits from all branches.
Please rectify if I have misinterpreted this.
Sachin
Hi Graeme,
Now it makes sense. Thanks for detailed information.
We have environment where application is accessed from various branches and we have set UEM to 15%, But client want to enable UEM 15% for all branches so that we can useful data for all branches as per my understanding we can only restrict this per application.
Pls suggest if it can be achieved.
Sachin
Sachin,
The injection of the JavaScript element is done either by a module loaded into the web server (e.g., for Apache or IIS), or by the agent in the front-end tier (e.g., for Java). I'll refer to either as an agent.
The agent recognizes a web request as representing a new visit because it doesn't include the Dynatrace session cookie. The agent then decides, based on the configured injection percentage, whether to instrument this visit. If the visit is going to be instrumented, the agent injects the JavaScript element into the response and attaches a new Dynatrace cookie. If the visit is not going to be instrumented, the agent just attaches the cookie.
The agent keeps track of which cookies represent instrumented visits and which represent uninstrumented visits. When a web request comes in with a cookie for an instrumented visit, the JavaScript element is injected into the response.
-- Graeme
Hi Andi,
Thanks for such sharing useful info. But still I have doubt that how DT controls injection of js and on the basis of what it decide to inject js to capture certain percentage of visits and to control injection it has maintain reference and how does it works.
Sachin.
Hi Sachin
The Visitor Coverage % defines for how many visitors the dtagent.js is actually injected. We are not injecting 100% and then throw away data. This % is really controlling the % of visitors that get that dtagent.js file injected. For those where it is injected we capture 100%. Thats also why I mentioned that this has no impact in case you do manual injection. If you manually put dtagent.js on every Visitor we will capture it for all of them - despite the % setting.
In the upcoming 6.2 we have a slight change where we actually provide a Boot Strap dtagent.js that you can use for manual injection. If you úse that boot strap agent the Visit Coverage % will also work as the boot strap agent also checks that setting
Andi
Hi Andreas,
Thanks for revert. I have also read link provided by you where its mentioned "Percentage of visits to be captured. For example, if you enter 10% then only each 10th visit of your web site will be captured." does that mean DT will inject dtagent.js for all users but it will capture every 10th user.
Sachin.
