Dynatrace HA Proxy

Hi Krzysztof,

many many thanks for your statement.
The load balancer appliance we uses HA Proxy and in the configuration with the WUI the STunnel for SSL.
I will test it with SSL over HA Proxy to see, what I need to in the manual configuration (the manual configuration overwrites the wizards configuration - which is great *yeah*).

The point with the processes and CPU cores is interesting and I will check that. Many many thanks for your information.

Best regards,
Jan

HAProxy is leveraging OpenSSL (same as stunnel), it's working very stable and configuration is fairly straightforward.

One thing to watch out for is performance - while HAProxy is very fast, it has single-threaded architecture optimized for high level of TCP traffic. SSL handling though is CPU bound, and especially full SSL handshake for new connections is quite costly. With one HAProxy process you could get into situation when one core is fully utilized and HAProxy cannot handle more incoming traffic while other cores are sitting idle.

So, depending on the profile of your service, you may need to test and tune the configuration. It is a good idea to adjust maxconn and maxsslrate params so that they match your hardware capabilities. For higher SSL loads you may need to switch haproxy into multi-process mode (with nbproc).

Hi Krzysztof,

a colleague of mine has experiences with STunnel, so we started with that. We use HA-Proxy 1.7, so I will give it a try. ,-)

What's your experience with HA-Proxy and SSL termination?

Best,
Jan

out of curiosity - why using stunnel instead of HAProxy native SSL support? Are you constrained to use HAProxy pre 1.5 version or do you see other benefits of using stunnel?

Hi Krzysztof,

it is. It seems, that the STunnel for the SSL termination breaks the connections from the client to the backend. We noticed disturbance in the whole platform. ,-(

Well, Dynatrace has problems in showing the traffic right, which comes from the STunnel and is sent to the HA-Proxy (both on the same host an listening on the same IP but different port) . Dynatrace showed us, that the outgoing traffic is going to another backend as we expected, because we know the platform configuration. ,-)

It would be nice to get an STunnel plugin for Dynatrace Managed/OneAgent, but it could be difficult, hm?

Best,
Jan

Good Morning,
thank you for the update. That was an interesting use case.

kind regards,
Kris

Good Morning,

we noticed, that our service will not work with the HA-Proxy and STunnel configuration we want. Perhaps it is a wrong idea.

So, thank you for you impressions and thought, I documented them here and we will use another approach.

Thank you for your help.

Best regards,

Jan

Hi,

thank you. I looked into the configuration of the HA-Proxy appliance (which comes from loadbalancer.org) and I see this

defaults
    mode http
    balance roundrobin
    timeout connect 4000
    timeout client 42000
    timeout server 43000
    log global
    option log-health-checks

and this

listen stats
    bind :7777 ssl crt $PATH/$to/$cert
    stats admin if TRUE
    stats enable
    stats hide-version
    stats uri /
    option httpclose
    stats auth $USER:$PASS

I think, your options are missing, right?
I look a bit deeper into it. Thanks.

Best,
Jan

Hi,

in your HAproxy configuration, please make sure that all the relevant frontents/backends are included in the statistics, this can with "stats scope" directive in "defaults" section. Not sure if it would address the issue with viewing the transactions, but at least you'd be able to see the overall statistics and continue troubleshooting from there.

Example configuration:

defaults
     # statistics settings
     stats enable
     stats show-legends
     stats scope servers
     stats scope http-in

Hi Krzysztof and other interested people. ,-)

I created some screen shots and published them here:
https://www.dropbox.com/sh/xy4tn8r950ib4ke/AACwx1x...

They are numbered in the right direction.

Well, starting from the point of the stunnel, we have a service here, terminating at port 5493 and working SSL. Ok, STunnel is a C program, we will not look into it with Dynatrace Managed. Check.

The STunnel runs on lb-n15-02, which is a loadbalancer, this is right.

At the second image one sees one service terminating at port 5493. That is right. I expect, that the service name stands here (because Dynatrace learned, that sageapp.questico.qintern.de:9493 exists and this is this point).

The third image shows the processes/services, which accept outgoing calls, which is the HA Proxy itselfs (LBINTERN), that is right and fapcas, which is not right, because, the STunnel does not know anything about fapcas, but fapcas is configured on the HA Proxy without SSL and works.

The fourth image shows the loadbalancer itselfs and shows great incoming calls from servers, which connect the loadbalancer on 5493. That is great.

The fifth image shows the outgoing traffic and the ssl backend is not seen (we configured HA Proxy to encrypt the traffic going to the backend).

So, our search/look terminates here.

Ok, starting to look from the backend point of view and sageapp-n15-01 is one of the backend servers. The IIS and the ASP.NET Application handle the incoming requests.
At the sixth image no one sees incoming traffic at all here, but this is not true, definitely not true.

The seventh image is slightly the same but we see outgoing traffic, its weird but ok with this application. .-/
The eighth image shows, that the IIS does not get incoming traffic. Hm ... and this ist also not true.

So, we do have some points here, what we like to fix. ,-)
I followed the documentation you pointed out to configure the OneAgent and the HA Plugin for Dynatrace and we do see here the the VIPs of the HA Proxy and their backend and have some statistics.
But now do we need to put the ends together here. How do we do that?

Thank you for your time.

Best regards,
Jan