nyanshak
03/08/2021, 10:18 PMsupport
block in this doc: https://github.com/fleetdm/fleet/blob/master/docs/1-Using-Fleet/2-fleetctl-CLI.md. Is that new? I've been using platform: darwin
(or windows, etc). Are there docs on what type of things can be in the support block? Looks like I can see osquery
, platforms
(list), and launcher
.
Might make sense to support some sort of per-query targeting (perhaps in the support
block?)Noah Talerman
03/09/2021, 10:45 PMnyanshak
03/09/2021, 10:45 PMNoah Talerman
03/09/2021, 10:45 PMis targeting always done at the pack levelYes, targeting is always done at the pack level.
support
block isn’t supported.support
block removedsupport
block but the option was never actually implemented.
That being said, I’d like to understand what you’re trying to accomplish by proposing the ability to target at the query level.nyanshak
03/09/2021, 10:54 PMmacOS monitoring
.
In this pack, I have a query called process_events
.
In osquery 4.8.0 (🤞), there will be a new table called es_process_events
that collects process events from endpoint security framework.
If I want to switch to es_process_events
, I need two things:
1. macOS is updated everywhere to 10.15+
2. macOS osquery clients are at 4.8.0+ (or whatever the version where this gets released).
Logically, this query still belongs in macOS monitoring
pack, but I don't want to run both queries on macOS
hosts, just the query that's supported.Noah Talerman
03/09/2021, 11:02 PMnyanshak
03/09/2021, 11:03 PMNoah Talerman
03/09/2021, 11:17 PMyou’ll need to move the original query into a new packWhy is this? Would you need to move the original query into a new pack because now the pack only targets a subset of the original set of hosts? In your example you can’t just leave
macOS monitoring
as is because it no longer targets all macOS
machines (instead a subset of them)nyanshak
03/09/2021, 11:19 PM---
pack definition
queries:
- queryA
- queryB
- query...Etcetera
- process_events
Imagine this is the original pack and it's targeted to macOS hosts (all macOS hosts)Noah Talerman
03/09/2021, 11:27 PMnyanshak
03/09/2021, 11:27 PMNoah Talerman
03/09/2021, 11:33 PMnyanshak
03/09/2021, 11:42 PMpack_a/
queries.yml
pack.yml
pack_b/
queries.yml
pack.yml
This isn't exactly what I do but it's similar and helpful for illustrating.
In this example, the queries for a given pack are stored in the directory for a pack, so it's easy to reference the list of queries for a pack.
---
In the 'just create a new identical pack' scenario, you can no longer manage the queries the same way, because applying the queries under one pack will overwrite the query definitions in another pack, and also you now have to maintain two copies of the pack.
You could just keep a comment in pack_b that says "hey, this query is defined in pack_a".
Basically, organizationally it could be problematic.
---
And also, if I have another query that needs targeting specifically, now instead of two packs, I need at minimum four packs. Next split results in eight, then sixteen, etc. Using packs to get around query targeting will create an exponential growth in number of packs used.Noah Talerman
03/09/2021, 11:46 PMyou now have to maintain two copies of the packRight and as you said this definitely doesn’t scale as you try to add more targeting by query.
I’m not sure what the right answer is exactly, but it’s been a persistent frustration.Totally understand. And my questioning / lack of understanding doesn’t help on the frustration front. Your explanations/examples are immense for my understanding. I too want to reach a good answer for this frustration. The method for arranging configurations you explained is very interesting. It seems logical to tie a specific pack to its set of queries. At a high level the issue we’ve been discussing seems to stem from new information (es_process_events_table) being only available for a specific criteria of hosts. The flexibility necessary to acquire this new info (new query) while still acquiring the rest of the info (the other queries) isn’t supported well by the pack level targeting. The new information is tied to a specific query. ^This is kind of a jumble and I’m typing as I think. Generally, I’m now curious if there are other ways to support the query level of flexibility without having targets at the query level.
Logically, this query still belongs inHi @nyanshak I’m revisiting this discussion. Why is it undesirable to run both queries onpack, but I don’t want to run both queries onmacOS monitoring
hosts, just the query that’s supported.macOS
macOS
hosts? Apologies if the answer to this question was already discussed.nyanshak
03/23/2021, 3:54 PMNoah Talerman
03/23/2021, 3:59 PMwe’d prefer one source over the other if it’s supported on that hostGot it. Am I right when thinking that the ultimate motivation for this is similar to the performance/cost motivation discussed in this thread?
nyanshak
03/23/2021, 4:03 PMNoah Talerman
03/23/2021, 5:24 PMnyanshak
03/23/2021, 5:26 PMosquery_schedule
table, which shows stats for each scheduled query execution.
We then make dashboards based on this and use it to help tune poorly-performing queries.Noah Talerman
03/23/2021, 5:49 PMnyanshak
03/23/2021, 5:53 PMNoah Talerman
03/23/2021, 6:02 PMnyanshak
03/23/2021, 6:03 PMNoah Talerman
03/23/2021, 6:17 PMosquery_schedule
table) allow you to answer this 2nd question:
• What’s the measurable amount we’ve reduced osquery overhead on a host?
Is this the answer to the 2nd question even valuable?nyanshak
03/23/2021, 6:21 PMWhat’s the measurable amount we’ve reduced osquery overhead on a host?In our scenarios, we're not looking at an individual host except in the problematic case. And the measure we'd use would be graphing osquery resource usage on the host over some time period to collect cpu / memory stats, for example, and comparing with one version of the query vs another version