Retry on Error ... is for network errors only
This is a very important feature to understand. It's not a question unless I have misunderstood the RoE option. :)
Please see this feature enhancement idea RETRY ON ERROR IDEA and please up-vote, offer your comments etc.
99% of our synthetic tests are "multi step", crawling through apps, validating pages along the way etc.
When I first saw RoE I thought this was to help eliminate ALL false alarms, not just the "network" noise etc. as I now understand it is limited to the following:
"when Retry on Error (RoE) is enabled, network, socket and SSL errors (10xxx, 11xxx, 12xxx, 15000 and 18001 errors) will trigger immediate re-execution of the test."
Some of my tests run only once an hour and in the event of a step failure I would want the test to retry immediately and not wait an entire hour to alert me. I have Number of consecutive errors set to 2, and I was under the completely incorrect assumption that any failed step would rerun the test immediately and if there was an error on that second try, alert me immediately because it hit the 2 consecutive error limit.
I understand I can decrease the interval, but that is not always practical.
Your suggestion "...for multi-step transactions is the ability to 'ignore' certain errors (mostly timeouts) and try the next step regardless - like a new "wait for next step to be available" type command. This would solve the issue where third-party content hangs up the page (resulting in 31000 errors) but still allows the transaction to continue" would be a huge bonus and allow us the flexibility to focus on what's really important as the steps are being executed.
Thanks!
The KB article was great - thanks for linking to it!
I understand the retry on error is like saying, "error once, then retry to see if it really was an error, then count only ONE error, even if it fails the second time on the retry" but in some cases I would like the RoE to actually count as the SECOND error. It that were the case you would quickly be alerted to a problem instead of waiting for the next run interval on a node. Once the error is triggered, resume normal test interval, with no RoE until it passes at least once and resets everything to zero.
This would likely also need a "max retry" setting that would need to be set to <= the consecutive error number. It would retry up to "max retry" times, either upping the error count on error or, if successful, resetting the error count to zero and resuming with the next interval.
I would be happy with the ability to toggle a "RoE counts as consecutive error" type of setting. :)
That is a very interesting idea! PM needs to be aware of it. Do you mind pasting it to the RETRY ON ERROR IDEA page too? I can do it too if with your permit~
Thank you for highlighting that RoE is for network errors only.
Regarding to the consecutive errors, if it sets to 2, then the alert will be triggered when each node have failed twice in a row. Say you configured the alert to be triggered from 2 nodes, both nodes need fail twice in a row to trigger the alert. See this KB here about how consecutive alerts work.
Back to the RoE feature, if the test failed, it would retry; if the retry failed too, the node failure number recorded is still 1 not 2. Since the test results for the retry will replace the original run. With this, we can eliminate alerts being triggered if the retry is successful. Say first failure caused by a temporary internet route glitch.
