Synthetic Port and Ping test - add multiple tests

[ct_27]

I'm looking for some guidance on the easiest way to add a few hundred Port and Ping tests.  We have tested the extension for a while now against 50 or so items. We're interested in expanding usage across our Infrastructure Monitoring to hundreds of devices.   

Is there a way to update a configuration file or some other means other than the UI?  The UI will take us way too long to add so many devices.

We run the extension on multiple ActiveGates to spread the load so the same solution to setup via a configuration file would also allow us to put some on AG1, some on AG2, etc.

Thanks in advance for the help to whoever has some ideas.

[ChadTurner · Accepted solution]

I would imagine you could use either the API or something like postman to mass import the jobs and custom extension to a range of AG's. I would reach out to support on how to do this though as it is probably unique to the extension. 

[ct_27]

You're right....I had not seen the Extensions EarlyAdopter API until you mentioned something.

For those looking to do the same here are the steps taken:

1) Build an instance of the extension first using the UI
     note: take note of the unique name you give it

2) Obtain the id of the instance you just created
     a) Use API : Get /extensions/{id}/instances
     b) Provide the id of your extension (e.g.    custom.remote.python.thirdparty_ping)
          Note: You can find this by navigating in a browser to the extension in question. Click on it and within the URL you will find 'id='. Copy the text between the '=' and the ';'
    c) Run the API and you should get a Code 200.  
    d) Scroll down the response list until you find the "name" of the instance you created in Step 1
     e) copy the "id"

3) Obtain a copy of the instance configuration
    a) Use API: Get /extensions/{id}/instances/{configurationId}
    b) Provide the same ID of the extension used above (e.g.    custom.remote.python.thirdparty_ping) and the ID of the instance you found in Step 2e
    c) Run the API and you should get a Code 200
    d) Copy the Response body

4) Let's test creating a new instance with the API
     a) Use API: POST /extensions/{id}/instances/validator
     b) Provide the same ID of the extension used above (e.g. custom.remote.python.thirdparty_ping)
     c) Paste in the Response body copied in step 3d to the "Request body" for this API
     d) In the pasted JSON; Fill in your api_token and put double quotes around it
     e)  In the pasted JSON; Delete the line that starts with "endpointId" (this is the id of the existing one and you're trying to create a new one right now)
     f) Run the API, debug, fix, repeat until you get a Code 204
     g) Be sure to change attributes that uniquely identify the new instance you are trying to create.
     h) Copy the final JSON that works
  
5) Final Step - Create a new instance
       a) Use API: POST /extensions/{id}/instances
       b) Provide the same ID of the extension used above (e.g. custom.remote.python.thirdparty_ping)
       c) Paste in the JSON from Step 4h
       d) Run the API and you should get Code 201 with a new "id" in the Response body

That's it, you should be able to refresh the UI and see your new Extension Instance.

[bill_scheuernst]

How did this solution work for you when deploying "a few hundred" instances?  I'm doing the same thing for the LDAP Synthetic plugin for approximately 25-30 LDAP endpoints.

[ct_27]

I will provide a quick response.  The script to create the Extension Instances worked well, we brought it into a technology called NodeRed so we could repeat, delete, and update.  But we later abandoned the testing due to poor responsiveness of Dynatrace and too difficult to maintain.  

The details are fuzzy because it was a few months back but any PING automation, not just Dynatrace, can only do so many pings from a single machine before you impact the local network card and host machine itself.   We were running 1,000's and according to some online research you shouldn't even be able to do 1,000 from one machine.  Plus Dynatrace attempts to run all 1,000 in the same 1 minute interval, you can't stagger them or anything. We then split the ping checks across multiple ActiveGates.  It bought us some time but we were still having isssues with the Dyntrace 1 minute run cycles.

[NOTE: I've used two other monitoring technologies for Ping checks and they too require multiple host systems to run to handle the large quantity.  Nagios though had the best solution because you can put the nagios script on each host you want to check and have the host ping a single mothership and send the results to your monitoring aggregator.  This works well because there is no, one host to handle all ping checks]

The Dynatrace Extensions 1.0 screens made it impossible to manage the 1,000's we had.  If one had a problem you had to manually go through multiple pages, which isn't even in alphabetical order, to find the instance.  We built a custom Admin Interfaces in NodeRed to mange the CRUD via API calls.

We then hit the MAX of synthetics permitted by DT in a SaaS tenant.  They extended this for us once we brought it to their attention.

At the moment we have just 400 or so ping checks. We're no longer having unreliable monitoring and for the most part can use the clunky DT UI to manage them.  So, as long as you're just doing a few 100 you'll be ok. But a few 1,000, for us, wasn't manageable. 

[Mike_L]

We are looking to create out of the box functionality for ping and port tests which scale much better. Keep your ears open later this year or so 👍

[bill_scheuernst]

I completely understand the limits as this is encountered where I'm currently at.  I prefer using the API but creating a new endpoint for the extension either in the UI or by using the API could be painful if done manually one instance at a time.  The blog post about the Extensions API makes reference to a mass update but I didn't see it even after reading it a few times.  I'm thinking of some type of Postman script.