In dynatrace How can we efficiently retrieve historical data (few weeks data) for all well-known exceptions or known errors?

Hi Andi,

Thanks for the update.

Beeling

Hi. Sorry - must have missed your posting from last week. It is the engineering team in Austria that is working on extending dynatrace. I dont know exactly what their timeframe and level of extension is for the upcoming release. I will however forward your contact to the respective product manager so that they can include you in early access programs

Andi

Hi Andi,

Any update on whom should we contact?

Thanks,

Beeling

Hi Andi,

I am a colleague of Adam. I just spoke with him yesterday about this feature in DT. We are really interested in option B. Could you ask your colleague in US who is working on this feature to contact us? We like to know more in details.

Thanks,

Beeling

Andreas, thanks much. Option B is what we are looking for. (We are using Option A today already.)

Hi Adam

I understand your use case. There are two options that you have

a) you can use the dynaTrace Client to dig into PurePath data from the past as long as you have it either in the continuous sesssion storage or in a stored session. Simply open the exception dashlet and see whicih exceptions came in at a particular timeframe. Here I however suggest that you always look at smaller timeframes, e.g: 1 hour, 6 hours - otherwise the dT Server needs to load too much data at once to analyze this

b) I know that we plan on extending our export capabilities to make it easier to access all PurePaths to be used by external products. This is something for future releases. In the meantime I also know about one collague in the US that is working on an implementation that is going in a similar direction. I will forward him this thread and let him reach out to you.

Andi

Andreas, thanks. This completely makes sense as the right approach once we have identified exactly what exception form we are interested in.

However, our underlying motivation with this thread was to find a way to search historical data for an arbitrary, newly discovered exception in the past.

Historical, arbitrary full text search would be incredibly useful for the kinds of intermittent issues we need to identify and resolve to improve our software performance reliability.

FWIW, my organization is prepared to invest our own engineering effort if there are any feasible technical solutions. For example, can you suggest any appropriate interfaces in DT we may be able to use to create an automated process that exports in real-time all structured Pure Path data from DT into external tool such as Splunk for indexing, full text search, and longitudinal analysis? Our organization indexes dozens of custom source types at tens of gigabytes a day, so creating the right custom adapters or handling this volume of data are not anticipated issues.

Thanks for the use case explanation.

In this case I suggest to follow the path that Jalpesh and I proposed: Create a Business Transaction that uses an Exception Count Filter for your specific "Commit" transaction. With that BT all PurePAths that contain that Exception will be tagged. You can use that BT in a Chart for long term trending. Additional - once you know a certain timeframe of interest, e.g: "October 11th from 10 - 11AM" - you can open the PurePath Dashlet for that timeframe and filter by that Business Transaction. Now you see all PurePaths that contain that exception in that timeframe

Makes sense?

As an example of one use case, imagine we want to find all PurePaths over the last 30 days which happen to contain very intermittent exceptions of a very specific form. We want to locate these PurePaths in DT so we can perform for further analysis and drill-down using the native DT facilities.

For the very specific form, consider any exception containing the word "Commit" as part of any method name of the stack trace of the exception.

We have reason to believe we may be experiencing ~10-100 of these exceptions a day, out of 10K+ exceptions and 100K+ PurePaths.

Is there a way to do this analysis in DT retroactively? Unfortunately, we have only very recently realized these exceptions were occurring, and now need to establish the historical trend to explain some recent system failures.

Any help here appreciated and thanks in advance.

FWIW, we log all of our unhanded, outermost exceptions to Splunk today in raw text format. Normally, our workflow is to use Splunk when we need to do the kind of longitudinal search we are asking for here (and sometimes frequency analysis), and then after we identify interesting event(s) jump into DT for a deep-dive and explanation. Unfortunately, in this specific case, the exception we need to find is a handled exception never logged to Splunk, but is visible from within DT.

Hi Rajeev,

If I am not wrong, from your subject line it come to know that you want to analyse exception or error from your past data (historical data).

You can create a BT as per you requirement by creating measure and filter it but it will gives you the data after creating that BT but not for past data. 

Jalpesh