I have a relatively simple DQL query that returns a list of services that have at least one or more key requests as well as a list of the host id's these services run on (there's actually a bit more to the query but this first part is the part that is throwing the error).
When I run it, I get the results I expect, however it takes almost 8 seconds for this query to run even though it's only returning 45 records. The problem seems to be that, even though I'm limiting the runs_on relationship statement to only show runs_on[dt.entity.host] (just like their example shows in their documentation), it seems to also be processing all the other runs_on relationships (like process groups and process group instances).
Here is the query:
fetch dt.entity.service
| fieldsAdd svcMethd = contains[dt.entity.service_method]
| filter svcMethd != "undefined"
| fieldsAdd hr = runs_on[dt.entity.host]
And this is the error I get when I run it in Notebooks:
The number of entity IDs in the relationship fields has been limited. Please try filtering or narrowing your timeframe.
However, I get a bit more detail when I run this query against the API in Postman:
"executionTimeMilliseconds": 7990,
"notifications": [
{
"severity": "WARNING",
"messageFormat": "The number of entity IDs in the relationship fields has been limited. Please try filtering or narrowing your timeframe.",
"messageFormatSpecifierTypes": [],
"arguments": [
"The current limit is 100 IDs per relationship field. Additional IDs have been truncated. Affected fields are: runs_on[dt.entity.process_group_instance]"
],
"notificationType": "RELATIONSHIP_RESULT_SIZE_LIMIT",
"message": "The number of entity IDs in the relationship fields has been limited. Please try filtering or narrowing your timeframe."
}
And no, it's not the contains[dt.entity.service_method] section that causes the issue. If I just run the first 3 lines of the query, it runs fine. It's the runs_on[dt.entity.host] part that is tripping it up.
You should be able to copy paste that DQL query and get a similar result (although that will depend on if you have several services with one or more key request set and if one or more of the matching services have over 100 process_group_instance entities related to it).
So yeah, even though I'm telling it I'm only interested in retrieving data about the hosts that the Services run on, it is needlessly still processing all the runs_on relationships for these services, including the massive amount of process group instance entities associated with these services. Meaning, the 45 records it returns requires it to scan through over 6,800 records... Not very efficient at all. I hope that's a bug or an oversight that they plan on fixing.
Does anyone know a more efficient way this can be written in DQL to truly prevent it from scanning all these unrelated relationship records for services?
