Create custom alert to restart Pods, in my Saas account

[ayamaya]

I need help creating a custom alert which tells me when a pod has rebooted, but I can't find any metrics that show me that.

It is possible to carry out this alert, since the only one I found is one that counts the number of times the container has restarted, but what I'm looking for is that it only alerts me when it restarts and disappears when the pod is running again.

[skrystosik]

I think best option would be counting worker processes number. In general you have for example 10 containers this will result 10 worker processes. If threshold will go below static threshold, you should be alerted. Of course there is issue when you have autoscaler, you will not see situation when containers will scale down gracefully but this is only option I know. 

Sebastian 

[ayamaya]

If we currently have that, what we see is the limitation that when the Pods have automatic scaling, it will change and we can only change the group manually.

[dannemca · Accepted solution]

I am not a specialist on K8S/OC, but did you tried to use the Events and/or Prometheus custom metrics for it?

Also, if you find a Process Group that is pertinent to these pods you are interested about , you can try use the Process Group Availability monitoring, it will tell you when a process instance (pod) is down.