• Forums
    • Public Forums
      • Community Connect
      • Dynatrace
        • Dynatrace Open Q&A
      • Application Monitoring & UEM
        • AppMon & UEM Open Q&A
      • Network Application Monitoring
        • NAM Open Q&A
  • Home /
  • Public Forums /
  • Application Monitoring & UEM /
  • AppMon & UEM Open Q&A /
avatar image
Question by Luiz Q. · Mar 06, 2015 at 04:08 AM · continuous delivery

Unable to categorise tests as "Performance Tests" using .Net library

Hello,

I would like to know how can I categorise my unit test data as a "Performance Test" when I send it to the Dynatrace. I am currently using .Net library and nowadays my tests are only appearing in the "unit tests" section; I tried to look it up on the library documentation, though, I was not able to find this information. As far as I am concerned, Java library enables you to categorise your tests, but .Net library does not.

could someone help me ?

thanks!

 

Comment

People who like this

0 Show 0
10 |2000000 characters needed characters left characters exceeded
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 10 attachments (including images) can be used with a maximum of 50.0 MiB each and 250.0 MiB total.

4 Replies

  • Sort: 
  • Most voted
  • Newest
  • Oldest
avatar image

Answer by Daniel P. · Aug 27, 2015 at 03:40 PM

@Wiktor Bachnik Where in our visual studio project do we add the call to StartTest task?

Comment

People who like this

0 Show 3 · Share
10 |2000000 characters needed characters left characters exceeded
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 10 attachments (including images) can be used with a maximum of 50.0 MiB each and 250.0 MiB total.

avatar image Wiktor B. · Aug 27, 2015 at 06:57 PM 1
Share

Hello Daniel,

This call is meant to be used in your build scripts. Currently we support MSBuild and NAnt with our plugins. If you're using other build automation tool or want to register the test run manually, you should use our REST API.

Best regards, Wiktor Bachnik

avatar image Bill H. · Sep 04, 2015 at 03:22 PM 0
Share

@Wiktor Bachnik @Daniel Pohanka

I am working with Daniel on Categorizing my tests as performance. We have successfully

  • Added the DLLs to the Test Project
  • Updated the Test Project file to include the StartTestTask
  • We see the StartTestTask run and complete during a build.
  • We get errors if we leave out parameters such as VersionMajor. This is verification that the StartTestTask is running appropriately.

However, dynaTrace does not pick up any of the parameters we set including Category. The tests still show up as Unit Tests with an autogenerated version number. Below is what we have included in the Project File.

<UsingTask AssemblyFile="dynaTrace.MSBuild.Task.dll" TaskName="StartTestTask" />

<Target Name="BeforeBuild">

<StartTestTask VersionMajor="1" VersionMinor="1" VersionRevision="0" VersionMilestone="Final" VersionBuild="1" Marker="Final Build" Category="performance" Platform="Windows 8.1" SystemProfile="LocalProfile" CustomProperty="$(CustomProperties)" > </StartTestTask>

</Target>

avatar image Wiktor B. Bill H. · Sep 07, 2015 at 03:21 PM 0
Share

Hello Bill.

Do you see a line in your log file after running the StartTestTask that says:

StartTestTask: Setting property DT_TESTRUN_ID to value {UUID}

where {UUID} is some actual identifier?

If so, what do you see when you open the following URL in your browser:

http ://{YOUR_DYNATRACE_SERVER}:{PORT}/rest/management/profiles/{SYSTEM_PROFILE}/testruns/{UUID}

Where {YOUR_DYNATRACE_SERVER}, {PORT}, {SYSTEM_PROFILE} and {UUID} are replaced with proper values.

Do you see the same details of the test run you used when registering it? Is the category matching?

I suspect the test runner is executed from another process that doesn't inherit the DT_TESTRUN_ID environment variable set by the StartTestTask, that's why you see your tests in the default category (unit) with autogenerated version numbers.

Regards, Wiktor Bachnik

avatar image

Answer by Wiktor B. · Mar 11, 2015 at 12:49 AM

Hello Luiz,

I think I found the source of the problem. It looks like Dynatrace installation puts DynaTrace.Command20.dll library into Global Assembly Cache (GAC), and version that's being put there is wrong. It doesn't contain the IDTCommand61 interface and its implementation (verified that with ILSpy). The DynaTrace.Command20.dll included in the dynatrace-msbuild.plugin-6.1.0.7880.zip archive offers correct implementation.

Solution: Please try either replacing the DynaTrace.Command20.dll in GAC or removing it and using the local copy from our archive in your project.

Please let us know if that solved your problem.

Additional note: Remember that in order for the tests to be correctly reported, DT_TESTRUN_ID environment variable in the process executing your tests must be set to contain the test run UUID generated by the startTest() method. This is done automatically by the MSBuild and NAnt libraries, but you need to remember about it when using the command library directly.

Best regards,
Wiktor Bachnik

Comment

People who like this

0 Show 20 · Share
10 |2000000 characters needed characters left characters exceeded
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 10 attachments (including images) can be used with a maximum of 50.0 MiB each and 250.0 MiB total.

avatar image Luiz Q. · Mar 12, 2015 at 02:12 AM 0
Share

Hi Wiktor,

First of all, I appreciate your help.

Sencondly, I have tried the actions suggested by you, though It did not work for me.

The thing is, I had to remove all the previous versions of Dynatrace (Agent and Client) from my machine in order to get all DLLs from version below 6.1.0.7880 vanished from the GAC.

After that, I have got only the new version of the DLL installed in the GAC (Version 6.1.0.7880 ), however, I still receiving the same error message.

Could you help me ?

 

avatar image Wiktor B. Luiz Q. · Mar 13, 2015 at 01:09 AM 0
Share

Hi Luiz,

One important question: did you remove the original DynaTrace.Command20.dll with 6.1.0.7880 version from the GAC as well?

The problem is that the dll distributed with Dynatrace 6.1 installation has exactly the same version string (6.1.0.7880) as the one we provide in the download of the MSBuild plugin, but it's missing the implementation of the required methods. They will look the same in the GAC dll list.

You'd need to remove the dll installed by Dynatrace first and then add the dll found in msbuild.plugin-6.1.0.7880.zip zip file.

In order to confirm that the dll is correct, you can use the ILSpy too (http://ilspy.net), please see how it looks like in my system:

Best regards,
Wiktor Bachnik

avatar image Luiz Q. Wiktor B. · Mar 14, 2015 at 12:30 AM 0
Share

HI Wiktor, I did work for me!

Thank you very much!

 

avatar image Wiktor B. Luiz Q. · Mar 14, 2015 at 12:34 AM 0
Share

Great! I'm glad I could help you to solve the problem.

Best regards,
Wiktor Bachnik

avatar image Luiz Q. Wiktor B. · Mar 28, 2015 at 02:30 AM 0
Share

Hi Wiktor, I am currently able to use the new msbuild library in my projects (6.1), though I am not seeing my unit test categorised as a "performance test" in Dynatrace Client. Perhaps, due to the upgrade of MSBuild library or because my updagrade from Visual Studio 2010 to 2013 (Which forces you to use another MSBuild.exe and QTAgent.exe)  it is not working anymore. I still can send purepath data to the Dynatrace server, but the unit test data is missing to my view in client.


Just to clarify, I am showing the enviroment where things were used to work and now.

Enviroment Before(Where it was working) :

  • Visual Studio 2010 (VS2010)
  • MSTest.exe from VS2010 attached in Dynatrace Server Agent (5.6)
  • QTAgent.exe from VS2010 attached in Dynatrace Server Agent (5.6)
  • QTAgent32.exe from VS2010 attached in Dynatrace Server Agent (5.6)
  • MSBuild Tasks for Dynatrace Version 5.6 (Test Automation and MSBuild)
  • Dynatrace .NET Agent Configuration Version 5.6
  • Dynatrace Client Version 5.6

Enviroment Now(it is not working) :

  • Visual Studio 2013 (VS2013)
  • MSTest.exe from VS2013 attached in Dynatrace Server Agent (6.1)
  • QTAgent.exe from VS2013 attached in Dynatrace Server Agent (6.1)
  • QTAgent32.exe from VS2013 attached in Dynatrace Server Agent (6.1)
  • MSBuild Tasks for Dynatrace Version 6.1 (Test Automation and MSBuild)
  • Dynatrace .NET Agent Configuration Version 6.1
  • Dynatrace Client Version 6.1

avatar image Wiktor B. Luiz Q. · Mar 31, 2015 at 10:14 PM 0
Share

Hi Luiz,

I'm sorry for the late reply.

Please check sensor configuration for the appropriate Agent group in your system profile settings. In the "Sensor Placement" configuration the ".NET Tests" sensor should be enabled. In the "Sensor Configuration" tab, for the ".NET Tests" sensor the "Capture" column value should be set to "active and start PurePaths".

Please confirm that you're using the settings I mentioned. If you still can't see your tests after those changes, please let me know if you can see the PurePaths matching the tests you're running in the "PurePaths" dashet (System Profile->Diagnose Performance->PurePaths).

Best regards,
Wiktor Bachnik

avatar image Luiz Q. Wiktor B. · Apr 01, 2015 at 01:15 AM 0
Share

 

Hi Wiktor, I reached out the person who is responsible for this kind of configuration and he said it is all configured properly. Take a look at the images. Is something missing either way ?

avatar image Wiktor B. Luiz Q. · Apr 01, 2015 at 01:51 AM 0
Share

Hi Luiz,

That looks good. Can you please check for the PurePaths in the PurePaths dashlet? Do you see anything there?

Also, if possible, could you provide the log file for the agent configured for MSTest?

Best regards,
Wiktor Bachnik

avatar image Luiz Q. Wiktor B. · Apr 02, 2015 at 12:14 AM 0
Share

Hi, I can see the purepaths in Dynatrace Client. Where can i get the logs ? could you help me ?

 

 

avatar image Wiktor B. Luiz Q. · Apr 02, 2015 at 01:34 AM 0
Share

Hi Luiz,

You can set agent logging path for each .NET agent configuration in the "dynaTrace .NET Agent Configuration" tool.

What testing library you're using to write your tests? VisualStudio Unit Testing? NUnit? Something else?

Also, are you using continuous session recording or are you recording each session separately?

Best regards,
Wiktor Bachnik

avatar image Luiz Q. Wiktor B. · Apr 02, 2015 at 11:39 PM 0
Share

Hi,

I have written the unit tests on Visual Studio and they are processed by MSTest.exe (the same process used inside Visual Studio).

I am using continuous session recording, I mean, I start the recording, set up the information of the tests (like the category etc),  run all the unit tests and after that stop the recording.

I might activate the logs today and let you know the results.

Thank you!

avatar image Luiz Q. Wiktor B. · Apr 03, 2015 at 05:50 AM 0
Share

 

Hi Wiktor, another question...which would be the best configuration to use in Log ?  I am currently using log level "info", but there are plenty of them.

avatar image Wiktor B. Luiz Q. · Apr 04, 2015 at 12:59 AM 0
Share

Hi Luiz,

I'd set logging level to "Finest" - it gives most details.

I tried to recreate the problem today, but it seems it works correctly in my installation of Dynatrace 6.1. I instrumented MSTest and QTAgent32 processes and made sure that their agent names are used in agent mapping in my system profile, I also enabled the ".NET Tests" sensor for them. It seems you're at that stage as well since you can see the PurePaths correctly.

In order to have my tests reported in the "Performance" category, I registered a test run using REST API and got its UUID. Then I set the DT_TESTRUN_ID environment variable and executed MSTest. After a while, tests appeared in proper category. Here's how I did that in command line:

set DT_TESTRUN_ID=69d3284e-c3b5-4a0c-b938-0ba9603d45c1
mstest /testcontainer:TestProject1.dll

This can be easily automated using MSBuild, that would execute "StartTest" task before starting MSTest execution. Our plugin sets the DT_TESTRUN_ID environment variable in the context if MSBuild script so it gets automatically populated to MSTest.

So maybe the problem is that you're not passing the test run UUID in the environment variable? Please try doing that manually as in my example and let me know if that helped.

 Best regards,
Wiktor Bachnik 

 

 

 

avatar image Luiz Q. Wiktor B. · Apr 06, 2015 at 11:36 PM 0
Share

 

Hi Wiktor, can you tell me the address for "start test" restful services ? I am using soap UI for tests purposes but I could not find the "start test" address in the APM Website.

Thank you!

avatar image Wiktor B. Luiz Q. · Apr 16, 2015 at 07:33 PM 0
Share

Hi Luiz,

I'm very sorry for the late reply, I got sick and was not in the office last few days.

Our REST API endpoints are documented at: REST Interfaces for Test Automation

You're interested in sending a POST request to the /testruns endpoint to register your test run. Please note that our MSBuild plugin (and the DTCommand library we talked about previously) also uses this endpoint behind the scenes, it may be more convenient to you to use it instead of doing that manually.

Please let me know if you managed to register your test run successfully. If you still have problems, maybe I could set up a WebEx to assist you remotely.

Best regards,
Wiktor Bachnik

avatar image Luiz Q. Wiktor B. · Apr 07, 2015 at 12:13 AM 0
Share

Another doubt, I have turned the log on as you told me. Which information do you think is important to observe in the logs to know if it is everything ok ?

avatar image Wiktor B. Luiz Q. · Apr 16, 2015 at 07:34 PM 0
Share

The most important thing is checking if the agent reports receiving the test run identifier you set in the DT_TESTRUN_ID environment variable.

If in doubt, you can upload your log here.

Best regards,
Wiktor Bachnik

avatar image Luiz Q. Wiktor B. · Apr 17, 2015 at 03:21 AM 0
Share

I did not find this information here in the logs, I am thinking about attaching them here. Just to attach the right files, which Logs do you want to take a look, the logs generated by MSTest.exe or by QTAgent.exe ?

avatar image Wiktor B. Luiz Q. · Apr 17, 2015 at 08:22 PM 0
Share

In my environment, logs for QTAgent were relevant.

Regards,
Wiktor Bachnik

avatar image Luiz Q. Wiktor B. · Apr 27, 2015 at 07:52 PM 0
Share

Hi Wiktor, very good news!

It is working now, I believe it was the lack of right QTAgent.exe (since vs 2013 has some of them in different versions) that were the problem. Now I can see my unit tests on "Performance Tests" section. Thank you for your help!

avatar image

Answer by Luiz Q. · Mar 06, 2015 at 04:23 AM

 

Hi, you are totally right, It is related to the MSBuild library.

I took a look at the latest version(6.1) of the library and I saw that now I have an option to set the category of the tests.

I might test this today!!

Thank you very much!!

Comment

People who like this

0 Show 0 · Share
10 |2000000 characters needed characters left characters exceeded
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 10 attachments (including images) can be used with a maximum of 50.0 MiB each and 250.0 MiB total.

avatar image

Answer by Wolfgang G. · Mar 06, 2015 at 04:15 AM

Hi,

I assume you're referring to the MSBuild library (Test Automation and MSBuild)? The StartTest task should actually support a Category field where you can register a test run as a performance test. Which version of the library are you using?

-Wolfgang

Comment

People who like this

0 Show 1 · Share
10 |2000000 characters needed characters left characters exceeded
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 10 attachments (including images) can be used with a maximum of 50.0 MiB each and 250.0 MiB total.

avatar image Luiz Q. · Mar 07, 2015 at 01:46 AM 0
Share

Hi Wolfgang, I have tested it yesterday and I have got some problems while trying to use the new task called "StartTest".
I am currently receiving this error message :

Could not load type 'dynaTrace.Command.IDTCommand61' from assembly 'DynaTrace.Command20, Version=6.1.0.7880, Culture=neutral, PublicKeyToken=1e72a45839f59aeb'.

I am afraid I have to refer another dll beyond the two that comes with the zip file. What do you think ?

 

How to get started

First steps in the forum
Read Community User Guide
Best practices of using forum

NAM 2019 SP5 is available


Check the RHEL support added in the latest NAM service pack.

Learn more

LIVE WEBINAR

"Performance Clinic - Monitoring as a Self Service with Dynatrace"


JANUARY 15, 3:00 PM GMT / 10:00 AM ET

Register here

Follow this Question

Answers Answers and Comments

4 People are following this question.

avatar image avatar image avatar image avatar image

Related Questions

Jenkins -- Jmeter -- Automation with Maven scripts

Borland SilkPerformer Integration

How do we add timers to Selenium - Web Driver?

Instrumenting Groovy Rails (Tomcat) from Eclipse

Sending email on completion of test run

Forum Tags

dotnet mobile monitoring load iis 6.5 kubernetes mainframe rest api dashboard framework 7.0 appmon 7 health monitoring adk log monitoring services auto-detection uem webserver test automation license web performance monitoring ios nam probe collector migration mq web services knowledge sharing reports window java hybris javascript appmon sensors good to know extensions search 6.3+ server documentation easytravel web dashboard kibana system profile purelytics docker splunk 6.1 process groups account 7.2 rest dynatrace saas spa guardian appmon administration production user actions postgresql upgrade oneagent measures security Dynatrace Managed transactionflow technologies diagnostics user session monitoring unique users continuous delivery sharing configuration alerting NGINX splitting business transaction client 6.3 installation database scheduler apache mobileapp RUM php dashlet azure purepath agent 7.1 appmonsaas messagebroker nodejs 6.2 android sensor performance warehouse
  • Forums
  • Public Forums
    • Community Connect
    • Dynatrace
      • Dynatrace Open Q&A
    • Application Monitoring & UEM
      • AppMon & UEM Open Q&A
    • Network Application Monitoring
      • NAM Open Q&A