question

Babar Q. avatar image
Babar Q. asked ·

DCRUM HA & FO Technical Overview

Hello Everybody,

We designed a below proposed plan of DCRUM HA/FO.

Need your valuable advice to enhance and implement the plan.

Especially I am looking for the VIP to connect to the databases, whether it will work or not.

Seeking for your advices.

Regards,

Babar

nam servernamcss
10 |2000000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 50.0 MiB each and 250.0 MiB total.

Avner S. avatar image
Avner S. answered ·

Hi Babar, For the database HA I believe you can use the Windows/SQL cluster feature. In this case you will have your Primary CAS/ADS connected to one Database instance (DB_server1\Instance1\Primary_CAS_DB, for example) that is on one host and your secondary CAS/ADS connected to other Database instance (DB_server2\Instance2\Secondary_CAS_DB, for example), located in other host. Using the Windows/SQL Cluster feature, in case one of the hosts goes down, the SQL will be responsable for migrate the DB instance from one host to other. Your configuration will be something like:

Hope that helps,

Avner


hppma.png (6.2 KiB)
7 comments Share
10 |2000000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 50.0 MiB each and 250.0 MiB total.

Hello Avner,

Thank you for your reply.

I guess my question went on the wrong side. Basically I am not looking for the DB HA, actually my concern is that can I use a VIP to connect to two different DBs one for each CAS and in the same way one for each ADS.

I would love if you can modify my proposed design for the HA of CAS and ADS servers or if you already implemented the similar solution, so that will be very helpful for us.

Regards,

Babar

0 Likes 0 · ·

I don't think you can use a VIP in this case. I think the better choice is to conect each CAS / ADS to one DB instance, however, to have HA, your databases will have to be configured as a cluster, as mentioned before.

Your propose seems to be good for me. A long time ago (when we still have 12.3 as newest version) I searched about it and designed the solution below. I didn't mention the database in that case, but I searched and to have the HA solution as expected, I would have to configure the SQL cluster, as explained in the first comment.

Please, let me know if I misunderstood something again. :)

Best Regards,

Avner

0 Likes 0 · ·
meujy.png (98.6 KiB)

Hello Anver,

You did not misunderstood. I think my question was like that :)

As you suggested "I think the better choice is to connect each CAS / ADS to one DB instance, however, to have HA, your databases will have to be configured as a cluster, as mentioned before."

Can we use the same DB Server with two different databases (one for CAS and one for ADS) or it is not recommended? if YES

What will be the impact on performance?

How the CAS and ADS cluster will use the same databases or we will need two DB Servers for the HA of CAS and ADS?

I guess that I asked the correct question now :)

Regards,

Babar

0 Likes 0 · ·

Hello Babar,

Yes, you can.

In your scenario, you can have, for example, two physical machines for database server. Each machine will have one instance of DB. In one instance (let's say, instance PROD-A) you can have the Primary CAS and Secondary ADS and in the other instance (instance PROD-B) you can have the Secondary CAS and the Primary ADS.

The impact on performance will depends a lot of your environment. It is recommended to separate the devices that have more read/write access to database. In my customer for example, we have 4 DB instances (each instance in one physical machine), and we arrange the DB in the following way:

InstanceA - 2 CAS that monitor INTERNET (requires a lot of read/write and space) and 1 other CAS

InstanceB - 3 AppMon DBs (PROD, DEV and HML), 1 EP DB, and 3 DNA DB

InstanceC - 1 ADS (requires good performance in read/write), 1 CAS, 1 CSS and 1 SMPE

InstanceD - 1 ADS (again read/write), 2 CAS (one of it that have a lot of read/write) and 2 RC

However, this is in my customers environment. Sometimes you can have less CAS, but that requires much more read/write performance. In this case, would be good to have each CAS on one instance. On the other hand, you can have 5 CAS, but with not much data and read/write, then in this case, 2 instances could be enough for you, one with 3 CAS DB and other with 2.

When configuring cluster, each CAS and ADS will have their own databases. As far as I know, only CSS use the same database when configured as cluster.

Regards,

Avner

0 Likes 0 · ·

Hello Avner,

Thank you for sharing a real environment.

I would like to go with the below scenario.

Two physical machines for database servers. Each machine will have one instance of DB. In one instance (let's say, instance PROD-A) you can have the Primary CAS and Secondary ADS and in the other instance (instance PROD-B) you can have the Secondary CAS and the Primary ADS.

Our proposed database servers are virtual with the following specifications:

  • 64 GB RAM
  • Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz, 2500 Mhz, 12 Core(s), 12 Logical Processor(s)
  • 1-3 TB HDD (EMC/SAN)

Current database sizes:

  • CAS near about 400 GB
  • ADS near about 800 GB

Is this logical to go with my selected scenario without facing any capacity issues in future?

Regards,

Babar

0 Likes 0 · ·

Sorry Babar, but now this goes a bit beyond my knowledge. :/

Maybe someone from support might be able to give you a more acertive answer about the specifications for your database.

Considering that will be 2 machines with this specs, looks good for me, however, I'm not a big fan to virtualize some crucial services, specially the ones that have a lot of I/O, as databases. It will depend on the use of your server that will host the VMs. Be carefull not only with the CPU and Memory of your server, but also network...

Regards,

Avner

0 Likes 0 · ·

Hello Avner,

Thank you for all your assistance to reach at this level.

Our network is pretty fast and for database VMs we have a 10GB Ethernet connection.

I will raise this inquiry to the support and will share their feedback on the forum.

Once again, I am very thankful.

Regards,

Babar

0 Likes 0 · ·
Babar Q. avatar image
Babar Q. answered ·

Hello @Chris V. & @Avner S. & @Kris Z.

Once again we came back with the same requirements of HA/FO/LB for the Dynatrace Data Center RUM 2017 May SP3.

In the below screenshot only the RUM Console database is replicating with the Failover Zone and there is no such illustration for the CAS/ADS databases.

We will have independent databases virtual machines for the CASes/ADSes so how the data will be replicated between databases and what will be the status of the reporting services (e.g. Enabled/Disabled same like RUM Console)?

I am looking forward for your expert advises for the successful implementation.

Regards,

Babar


Share
10 |2000000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 50.0 MiB each and 250.0 MiB total.

Didier R. avatar image
Didier R. answered ·

Bonjour,

De retour le 03/01.

Pour ce qui concerne VAM et VISU, vous pouvez contacter Axel BACHELARD.

Share
10 |2000000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 50.0 MiB each and 250.0 MiB total.

Chris V. avatar image
Chris V. answered ·

Not SQL Clustering is not supported in DCRUM. See response to my query by Kris Z. here:

https://answers.dynatrace.com/questions/164370/hig...

CAS and ADS failover/HA is provided by the DCRUM clustering functionality, but it requires doubling up on databases.

1 comment Share
10 |2000000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 50.0 MiB each and 250.0 MiB total.

Hello Chris,

I am getting Access Denied message on opening the link.

Regards,

Baba

0 Likes 0 · ·
Babar Q. avatar image
Babar Q. answered ·

Hello Avner,

The below is the feedback from the support team.

"The VIP idea - I don't see it working.

As for the" Is this logical to go with my selected scenario without facing any capacity issues in future?

question - there's no possibility to tell that any hardware won't face capacity issues - it all depends on the users' configuration and traffic - it may be enough for next few years, but it also may not suffice for next week. But I assume you have a steady baseline of 400/800GB DBs. It's in the upper limits of what we usually see. I'd say that a standalone SQL server would be enough for each DB. Both DBs on the same server - I'm not sure. I'd focus on the I/O, especially that you want to virtualize the servers and use external storage. Both will introduce additional load/delay on the SQL. It's not a design we have tested and can share some numbers, unfortunately. For a standard solution (CAS+SQL on the same host) I've seen SQL hitting max 16000-17000 IOPS with an average of 1000-1200 IOPS. This also shows that CAS and ADS use DB in a rather uncommon way - they put a heavy load on it every 5 minutes (default interval) and then idle until next processing interval. So using average load to do any performance comparisons may be misleading.

After this information we are back to square one. It means I should go with my proposed design just to remove the VIP part, so there will be no IOPS issues with the dedicated SQL server for each CAS and ADS.

Regards,

Babar

2 comments Share
10 |2000000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 50.0 MiB each and 250.0 MiB total.

Hi Babar,

Good to know, thanks.

So that it's basically what we talked about: VIP does not work in DB cases and your DB servers configuration looks good, but it depends on the traffic of your environment.

I still think that, as you have more than one SQL server, you could configure cluster between them, just to make sure you have all your environment in HA.

If you proceed with your configuration having a dedicated SQL server for each CAS and ADS, you will have your tools in HA, however, if the one DB server goes down, the CAS or ADS will go with it. If you configure SQL Cluster, your CAS/ADS will still have a dedicated SQL server, but, if for some reason one of your SQL server goes down (problem or even maintenance), the instance will migrate automatically to the other server, and you will still have both CAS/ADS working.

In this case, your architecture would be something like:

Doing it will be even harder for you to have a downtime in your environment. Just my humble opinion :)

Best regards,

Avner

0 Likes 0 · ·
kt185.png (165.7 KiB)

Hello Avner,

Thank you for all your expert opinions.

Let me discuss your proposed plan with our DBA gurus :) and then share with you if they ask something new or have any concerns.

Regards,

Babar

0 Likes 0 · ·