question

Alexander R. avatar image
Alexander R. asked ·

Netty - full PurePath visibility. (incoming request + outgoing requests/DB query in one PurePath)

Hello community.

I have Netty process that service incoming requests. (micro-services in docker containers)

When request come, one thread service this request, another one make outgoing http connection to another service (another netty services).There are 5-10 such outgoing requests.

I have 3 services for my main Netty process. (they are: requests executed in background threads - where i can see outgoing connections via okhttp3, service (fireChannelRead).

On back Netty services i also can see service maked by default rules (fireChannelRead method)


So i have such PurePaths (example)

1) Incoming request "/purchase" on the first Netty process on service (fireChannelRead)

2) Request executed in background threads on the first Netty process

*

-> /purchase_status on the back Netty (for example status.jar)

-> /purchase_create on the back Netty (for example create.jar)

3) Request executed in background threads on the back Netty "create.jar"

* -> outgoung request to another server or DB queries.

query...

There are 3 different PurePaths - we can see “pieces”( fragments) of PurePath, but can`t get it in normal view like:

"/purchase"

-> /purchase_status

-> /purchase_create

-> request to unmonitored hosts

->DB query

I have tried to add 10-20 classess/methods to get more visibility, but there is no result.Some classess i can`t instrument - they have "for security reasons these can not be instumented.

Maybe anyone have some experience with Netty or another Java service with multi-thread execution, please answer to me, what i have done wrong?

Maybe it is case to use SDK, but i have no idea will it help to me or not. (requests are executed in different threads)

Also maybe anyone try to install Dynatrace OneAgent on server with Spring Cloud Gateway (OpenSource).

I can see only incoming requests (it is simple gateway/proxy) I can`t see outgoing requests.


Regards,

Alexander

Dynatrace Managedpurepath
3 comments
10 |2000000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 50.0 MiB each and 250.0 MiB total.

Hi Alex,

Need help, We are also using Spring Cloud Gateway and that uses the Netty framework and we are using agent version 1.81 still no luck to see the outgoing requests, Even with Vanilla sprint cloud gateway app also not able to see the HTTP proxy outgoing requests.

Any idea, we placed the custom services which calls the downstream services method still no luck to see the outgoing request, its only showing within API gateway asyn cals.

Please share if you have any idea and how you able to fix your environment. Thanks!

Baskaran

0 Likes 0 · ·

Hello Baskaran.

We had support ticket about my question.

The main reason was that we can`t see request in other threads, because netty is multi thread application, with different methods of transfer execution to other threads.


Support recommended to us ask Customer change application:

  • Use "thenApply" instead of "thenApplyAsync". The code will still run asynchronously in one of many "OkHttp Dispatcher" threads.
  • rewrite the code and use retrofit2.Call::enqueue instead of CompletableFuture

Another way it was to use Java SDK.

Additionally R&D team added in 1.173 version "CompletableFuture" support out-of-the-box


I think it is better for you to create support ticket.


Regards,

Alexander

0 Likes 0 · ·

Hi Alex,

Thanks for the response, We couldn't change the application level also they suggested solution is more on spring internal changes, so we don't have an option until support provide the solutions. thanks!


Regards,

Baskaran

0 Likes 0 · ·
Harald B. avatar image
Harald B. answered ·

Hi,

Actually only the server side of reactor netty is supported. Spring cloud gateway uses the reactory netty http client for outgoing calls which is not detected up to now. We are working to include support for this scenario.

regards
Harry

1 comment Share
10 |2000000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 50.0 MiB each and 250.0 MiB total.

Thanks Harry for this information.

Hopefully it will be available as soon as possible.

Arnaud

0 Likes 0 · ·
Baskaran s. avatar image
Baskaran s. answered ·

Quick update, Support has the solution for spring cloud gateway to capture the outgoing requests, they enabled the Netty proxy flags on the Netty applications process group level. After the app restart, it starts detecting the downstream services calls. They told this fix will be released on the upcoming agent releases.

So if you have any issue in spring cloud gateway, Ask Support to apply the Netty proxy flag on the process group or node level, so it will detect the outgoing services calls.


I hope it will be helpful until the agent fix releases.


Regards,

Baskaran

Share
10 |2000000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 50.0 MiB each and 250.0 MiB total.

Sebastian K. avatar image
Sebastian K. answered ·

If requests are executed in separate threads, without direct option to pass DT tag between them it may be hard to connect them into one. But maybe somebody will have some extra idea here.

Sebastian

Share
10 |2000000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 50.0 MiB each and 250.0 MiB total.

Arnaud G. avatar image
Arnaud G. answered ·

Hello Alexander,

According this link, OneAgent 1.175 support Reactor-Netty.

https://www.dynatrace.com/support/help/whats-new/release-notes/oneagent/sprint-175/

Did you try with this version if there is full PurePath visibility (incoming request + outgoing requests/DB query in one PurePath) ?

Arnaud

Share
10 |2000000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 50.0 MiB each and 250.0 MiB total.