RtmMemoryPerfind

Jean-Louis,

The rtmMemoryPerfind metric shows the percentage of total memory that can be used by the AMD (data.mem.limit + SHM size) vs the amount actually used (normal memory allocation + SHM buffer usage). According to the chart you attached, the AMD is running out of SHM space - i.e. it is hitting the SHM buffer capacity and dropping packets - that's why you see a drop in monitored traffic during that time.

There can be two reasons for that:

• The AMD drops packets due to short bursts of increased traffic volume. In this case, you can solve the problem by increasing the SHM buffer size in rtminst.
• The AMD is overloaded (analyzes too much traffic) or there is some performance problem.

Regards,

Wojtek.

Hello Wojtek,

thank you for this explanation.

We have tested these 2 configurations :

The total memory is 146GB.

We increased the SHM buffer size during a test period. But with this new configuration, the AMD lost almost all the packets and there were only 5% of data collected in the CAS.

which value do you recommand for the SHM buffer size?

I don't think that the AMD is overloaded, there are 6 Gbps on the 10 Gb NIC received by the AMD, but there are only 300 Mbps filtered (and treated by the AMD).

It appears that the Max of the rtmMemoryPerflnd is reached with a high activity of the SOAP.

Regards,

JLL

Hi Harsal,

For this environment, the default of the SHM Buffer Size is 20642 MB (proposed automatically from the rtminst - I use the standard command of the documenation as you mentioned) and the max is 51606 MB.

I think the AMD change the default and max SHM buffer size depending of the total memory (146 MB here).

Regards,

JLL

Jean-Louis,

Increasing the SHM buffer space past some value won't have any effect on performance (it may only delay sampling a bit). The fact of the AMD being overloaded is not related to NIC capacities. The 300Mbps that is being filtered and sent to analysis equals to more or less 300k packets per second - which is roughly the top traffic volume for all decodes (you can find exact traffic processing capabilities in the documentation). The AMD is expected to drop packets and activate sampling when reaching this limit (the exact limit depends on the decodes being used and the traffic profile). 

Wojtek.

Hello Wotjek,

What is the max value for the SHM buffer size you recommend to configure in the rtminst ?

For this case, I don't think that the data collected has reached the limit. The AMD is a very large AMD (32 Proc, 146 GB Ram...), and a regular AMD can collect about 900 Mbps according to the documentation. When the limite of the rtmMemoryPerfInd is reached, there are max 10 MPkts by 5 minutes - the traffix does not exceed 33k packets per second!

regards,

JLL

Jean-Louis,

Usually one should use the value recommended by rtminst. Too large values may even increase the amount of dropped traffic (a large buffer allows a large backlog of packets to be accumulated, and when sampling starts the AMD must process all those buffered packets, further reducing performance and increasing sampling).

If you are experiencing drops/sampling at a traffic volume of 30kpps you should open a Support ticket. This apparently is a problem, but it is not memory-related (at least not directly). Determining the root cause would require anaylysis of AMD performance logs.

Regards,

Wojtek.

Very interesting. Thank you for this explanation.

The call is already opened. SUPDCRUM-5320

Regards.

JLL