We will possibly use Dynatrace in our unit testing framework as well.
to that end, the documentation is lacking on the definitions of the metrics and parameters in the Automation interface.
Some relevant questions are:
How is the "Corridor" calculated? Is that mean +/= Margin of error at 97% confidence interval?
what does the default value of 100% volatility mean? The impression is that it is a measure of variance, but how is volatility calculated?
Does the number of calibration runs apply only to the initial 5 measurements (runs)? or is it applied every time changes are accepted?
How can the system recognize a 'false positive'? It determines violations (negatives) by the actual result and the corridor definition. So then what does a false positive % mean and how is it calculated?
How do the "accept changes" and "accept changes from Here" functions affect the volatility and corridor calculations?
Thanks for your reply.
Answer by Saadat D. ·
Thanks, Andreas. I would like to get some clarification on the Hint Text. Specifically, once the minimum number of calibration runs is reached, are changes automatically accepted for every subsequent improved run, or after every “Improved Runs” lower limit? For instance, if Improved Runs lower limit is set to 2, are changes accepted after every 2 consecutive improved runs or simply after each improved run? Thanks.
The setting for Auto-accept changes is irrespective of the settings for improved runs. So changes are accepted for every subsequent improved/degraded run.
As the hint text for Improved Runs says this setting influences the classification of test performance state only. But there is a copy-paste-error in the hint text. The text must be
The minimum number of consecutive violations of the lower limit for a test until it is considered improved
of course.
Answer by Thomas B. ·
Hi John,
Hope this helps,
Thomas
One more question on the Corridor Settings (see System Profile > Test Automation)… How does the “Auto-accept changes” work? That is, what is the trigger when this flag is checked? Is it the “Calibration Runs” or the ‘Degraded Runs” & “Improved Runs”?. Thanks.
This is the Hint Text you get in the System Profile for this checkbox: "Automatically accept changes in improved runs as soon as the minimum number of calibration runs has been reached."
Does this help?
Answer by Guenter H. ·
Hello John,
sorry I can´t shoot out of the hip, but it´s Friday afternoon and I´ll need key people to answer your questions thoroughly.
May I ask for your patience to get back to you Monday (June 25th 2012 <g>)
Thanks
G.
Learn how Dynatrace Real User Monitoring automatically detects errors that impact your end users caused by erroneous 3rd party or CDNs.
December 12, 4:00 pm CET / 10:00 am ET
Register here
Learn how Dynatrace Real User Monitoring automatically detects errors that impact your end users caused by erroneous 3rd party or CDNs.
December 12, 4:00 pm CET / 10:00 am ET
Register here
Learn how Dynatrace Real User Monitoring automatically detects errors that impact your end users caused by erroneous 3rd party or CDNs.
December 12, 4:00 pm CET / 10:00 am ET
Register here
Learn how Dynatrace Real User Monitoring automatically detects errors that impact your end users caused by erroneous 3rd party or CDNs.
December 12, 4:00 pm CET / 10:00 am ET
Register here