running oneagent inside docker container

[SjoerdB]

In a customer environment, I'm facing a dockerized service that we want to monitor, but no way to install the oneagent...

Problem is that the docker host is "forbidden" to have any additional software installed on, and the containers are deployed via very strict deployment system, not allowing for privileged access.

In the old days (beginning of Ruxit), you could just run the agent installer inside a container, and get it represented as a separate host in Dynatrace, so then it could be added to the regular Dockerfile.

Is there a way around the required privileges and having this oneagent run as one host per container (we can fix the hostnaming etc by not using dynamic hosts, so we could reuse the hosts at container renewal).

[Bartek_Gatz]

Hi @Sjoerd B.

Currently in order to monitor dockerized services you need to deploy OneAgent on a "parent" host, which consequently requires write access for OneAgent binary and runtime files to be deployed. That rule applies in both cases: where OneAgent is deployed directly on a host, or via a Docker image.

That being said we are working on a solution to hopefully satisfy your needs. It will be based on a solution where OneAgent is deployed via a container, but actually stays within when it comes to its files, while still being able to monitor the parent host and all the other containers within.

Could you please share a few more details related to the hardening applies to the host operating system?

As of today (May 14th 2018) we have a working prototype tested internally to work with Google Container Optimized OS, which is a type of a hardened OS to serve particular needs of hosting containers and services within. I am not 100% sure if the prototype we have created will work in your case, so please share as many details as possible.

Thanks in advance.

CC: @Tomasz G.

[SjoerdB]

Hi @Bartosz G., thanks for the reply.

Currently the customer wants to limit the scope of Dynatrace monitoring to only the container that contains the service. So for this use case the old installer that wasn't aware of the world outside the container would suffice.

The hardening is mainly in the tooling around the coreos docker host. In the tooling all compose files are scanned and only the supported settings are allowed / deployed.

The department responsible for the docker PaaS, is very strict that no agent-based monitoring is allowed on the docker swarm nodes. So even if I would come around the deployment restrictions, I don't want to break the rules...

So some details (I can only get true versions when I'm in the customers office):

Large physical+vmware nodes running coreos. A dedicated docker swarm for every product team; no access to docker environment other than through deployment pipeline. Docker swarms are bound to a limited number of nodes, but when refreshing, nodes can be exchanged. Docker compose V2 is used, no plans to go to v3, as the next orchestration will be done with Kubernetes (near future).

Does the above make sense?

So basically I would like to have a Docker-PaaS agent... (also interesting for customers that actually use a docker-provider in the cloud).

[Tomasz_Gajger]

Hi Sjoerd,

Please check out the following question: https://community.dynatrace.com/spaces/482/dynatrace...

There is a possibility of baking PaaS agent into the application container and I think this should cover your use case.

As far as full-stack agent deployment via container goes, we are working on making it not deploy directly to the underlying filesystem, just as Bartek said. However, when it comes to making this container unprivileged, this is something that is not achievable due to the nature of OneAgent and the need to have access to all the other processes running on the system (hence the --privileged flag and shared PID, NET and IPC namespaces).

Regards,
Tomasz

[alois_mayr · Accepted solution]

Hi @Sjoerd B.

We still recommend running OneAgent on the cluster nodes and with the improvements mentioned by @Bartosz G. and @Tomasz G. we want to address the challenges with immutable infrastructure.

In your specific scenario I'd suggest you to follow the instructions on this help page. While the help topic is focused on Kubernetes, you will immediately notice that it can also be applied to your Docker Swarm scenario. The instructions described there basically guide you through the steps needed to add the OneAgent code-modules into your application Docker images.

Best,

Alois

[SjoerdB]

Hi Alois,

With the description on the webpage, I get the oneagent running perfectly in my environment.

Of course I had to set the hostname in the docker-compose.yml, and then I thought it would be a good idea to also add:

    environment:
     - DT_CUSTOM_PROP=Platform=DockerPAAS

Unfortunately I didn't see this custom property back at host-level (I tried a few ways of entering the env setting, but it didn't show up).