• Forums
    • Public Forums
      • Community Connect
      • Dynatrace
        • Dynatrace Open Q&A
      • Application Monitoring & UEM
        • AppMon & UEM Open Q&A
      • Network Application Monitoring
        • NAM Open Q&A
        • Enterprise Synthetic Monitoring
      • Synthetic Classic
        • Synthetic Classic Open Q&A
  • Home /
  • Public Forums /
  • Application Monitoring & UEM /
  • AppMon & UEM Open Q&A /
avatar image
Question by John A. · Jun 22, 2012 at 08:24 AM · continuous delivery

Seeking details on Automation parameter definitions and metric calculations

We will possibly use Dynatrace in our unit testing framework as well.

to that end, the documentation is lacking on the definitions of the metrics and parameters in the Automation interface.

Some relevant questions are:

How is the "Corridor" calculated? Is that mean +/= Margin of error at 97% confidence interval?

what does the default value of 100% volatility mean? The impression is that it is a measure of variance, but how is volatility calculated?

Does the number of calibration runs apply only to the initial 5 measurements (runs)? or is it applied every time changes are accepted?

How can the system recognize a 'false positive'? It determines violations (negatives) by the actual result and the corridor definition. So then what does a false positive % mean and how is it calculated?

How do the "accept changes" and "accept changes from Here" functions affect the volatility and corridor calculations?

Thanks for your reply.

Comment

People who like this

0 Show 0
10 |2000000 characters needed characters left characters exceeded
â–¼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users

Up to 10 attachments (including images) can be used with a maximum of 52.4 MB each and 262.1 MB total.

3 Replies

  • Sort: 
  • Most voted
  • Newest
  • Oldest
avatar image

Answer by Saadat D. · Oct 12, 2012 at 05:42 AM

Thanks, Andreas. I would like to get some clarification on the Hint Text. Specifically, once the minimum number of calibration runs is reached, are changes automatically accepted for every subsequent improved run, or after every “Improved Runs” lower limit? For instance, if Improved Runs lower limit is set to 2, are changes accepted after every 2 consecutive improved runs or simply after each improved run? Thanks.

Comment

People who like this

0 Show 1 · Share
10 |2000000 characters needed characters left characters exceeded
â–¼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users

Up to 10 attachments (including images) can be used with a maximum of 52.4 MB each and 262.1 MB total.

avatar image Richard U. · Oct 18, 2012 at 04:56 PM 0
Share

The setting for Auto-accept changes is irrespective of the settings for improved runs. So changes are accepted for every subsequent improved/degraded run.

As the hint text for Improved Runs says this setting influences the classification of test performance state only. But there is a copy-paste-error in the hint text. The text must be

The minimum number of consecutive violations of the lower limit for a test until it is considered improved

of course.

avatar image

Answer by Thomas B. · Jun 25, 2012 at 03:24 PM

Hi John,

  1. The corridor is the (100-False Positive %)-% Confidence Interval of the Student's t-Distribution of a measure (see http://en.wikipedia.org/wiki/Student%27s_t-distribution) - since False Positive % is 1 by default, the default is the 99% confidence interval.
  2. Volatility is our name for the Coefficient of Variation: http://en.wikipedia.org/wiki/Coefficient_of_variation. By changing the Volatile % in the Test Automation Settings, you define how high this Coefficient of Variation has to be such that a Test is considered volatile.
  3. The calibration runs are applied every time changes are accepted.
  4. See 1 - the "False Positive %" is our term for setting the confidence interval.
  5. When accepting changes, the system behaves exactly as if the selected measurement was the first one ever observed; all existing values for standard deviation etc. are discarded and are only shown in the chart. When selecting "Accept Changes" instead of "Accept changes from here", the first measurement outside the corridor will be chosen for accepting changes.

Hope this helps,
Thomas

Comment

People who like this

0 Show 2 · Share
10 |2000000 characters needed characters left characters exceeded
â–¼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users

Up to 10 attachments (including images) can be used with a maximum of 52.4 MB each and 262.1 MB total.

avatar image Saadat D. · Oct 12, 2012 at 12:21 AM 0
Share

One more question on the Corridor Settings (see System Profile > Test Automation)… How does the “Auto-accept changes” work? That is, what is the trigger when this flag is checked? Is it the “Calibration Runs” or the ‘Degraded Runs” & “Improved Runs”?. Thanks.

avatar image Andreas G. &. Saadat D. · Oct 12, 2012 at 04:03 AM 0
Share

This is the Hint Text you get in the System Profile for this checkbox: "Automatically accept changes in improved runs as soon as the minimum number of calibration runs has been reached."

Does this help?

avatar image

Answer by Guenter H. · Jun 22, 2012 at 08:44 PM

Hello John,
sorry I can´t shoot out of the hip, but it´s Friday afternoon and I´ll need key people to answer your questions thoroughly.

May I ask for your patience to get back to you Monday (June 25th 2012 <g>)
Thanks
G.

Comment

People who like this

0 Show 1 · Share
10 |2000000 characters needed characters left characters exceeded
â–¼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users

Up to 10 attachments (including images) can be used with a maximum of 52.4 MB each and 262.1 MB total.

avatar image John A. · Jun 22, 2012 at 10:34 PM 0
Share

Thanks a lot Guenter.

I can certainly wait till Monday.

Welcome to the
Dynatrace Community Forums

Check out the Community User Guide and First steps in the forum to learn how to get started.

Community Member of the Month
December 2019

Announcing Dynatrace's Community Member of the Month for December 2019, Enrico F.! Click here to read more!

Employee Member of the Month
December 2019

Announcing Dynatrace's Employee Member of the Month for December 2019, Silvia M.! Click here to read more!

Live webinar: Ensuring Digital Business Availability with Dynatrace

Learn how Dynatrace Real User Monitoring automatically detects errors that impact your end users caused by erroneous 3rd party or CDNs.
December 12, 4:00 pm CET / 10:00 am ET
Register here

Live webinar: Ensuring Digital Business Availability with Dynatrace

Learn how Dynatrace Real User Monitoring automatically detects errors that impact your end users caused by erroneous 3rd party or CDNs.
December 12, 4:00 pm CET / 10:00 am ET
Register here

Live webinar: Ensuring Digital Business Availability with Dynatrace

Learn how Dynatrace Real User Monitoring automatically detects errors that impact your end users caused by erroneous 3rd party or CDNs.
December 12, 4:00 pm CET / 10:00 am ET
Register here

Live webinar: Ensuring Digital Business Availability with Dynatrace

Learn how Dynatrace Real User Monitoring automatically detects errors that impact your end users caused by erroneous 3rd party or CDNs.
December 12, 4:00 pm CET / 10:00 am ET
Register here

Follow this Question

Answers Answers and Comments

6 People are following this question.

avatar image avatar image avatar image avatar image avatar image avatar image

Related Questions

Correct way to export a saved session CSV? 4 Answers

Passing DT_TESTRUN_ID to a selenium run from Ant 5 Answers

dynaTrace + Selenium 2 + Python 1 Answer

Support for the Intern testing tool in Test Automation feature of DT 4 Answers

Multiple simultaneous test run registrations 1 Answer

Forum Tags

dotnet mobile monitoring load iis 6.5 kubernetes mainframe rest api errors dashboard framework 7.0 appmon 7 health monitoring adk log monitoring services auto-detection uem webserver test automation license web performance monitoring ios nam probe collector migration mq web services knowledge sharing reports window java browser agent community user guide hybris javascript appmon sensors good to know search 6.3+ server documentation easytravel web dashboard kibana system profile purelytics docker splunk 6.1 process groups account 7.2 rest dynatrace saas spa guardian appmon administration production user actions postgresql upgrade oneagent measures security Dynatrace Managed transactionflow diagnostics user session monitoring unique users continuous delivery configuration alerting NGINX splitting business transaction client 6.3 installation chart database scheduler apache mobileapp RUM php dashlet azure purepath plugins agent 7.1 appmonsaas messagebroker nodejs 6.2 incidents android sensor performance warehouse
  • Forums
  • Public Forums
    • Community Connect
    • Dynatrace
      • Dynatrace Open Q&A
    • Application Monitoring & UEM
      • AppMon & UEM Open Q&A
    • Network Application Monitoring
      • NAM Open Q&A
      • Enterprise Synthetic Monitoring
    • Synthetic Classic
      • Synthetic Classic Open Q&A