https://github.com/osquery/osquery logo
Powered by
#general
Title
# general
h

HarlanF

01/25/2022, 12:42 AM
When I have a scheduled query that fails because of the watchdog process interrupting it for exceeding time or memory, where can I conveniently locate which query has failed? We've got numerous packs of scheduled queries, and it seems tricky to tease out which one's running afoul of the resource constraints.
z

zwass

01/25/2022, 12:46 AM
In Fleet we use the info in the 
osquery_schedule
table. I always recommend folks schedule a query to that table.
h

HarlanF

01/25/2022, 12:54 AM
@zwass as in having a periodic (hourly?) query there to see the current state of it at snapshots in time?
z

zwass

01/25/2022, 12:54 AM
Yes, just like that!
There's information about what is scheduled, the resources it's been using, watchdog settings, etc.
h

HarlanF

01/25/2022, 12:54 AM
Hmm, let me give that some thought and look at the schema. Thanks!
Certain classes of out-and-out errors log which query it was, but the scheduled queries interrupted by watchdog do not, which is a little frustrating.
@zwass, also odd regarding that osquery_schedule table is that our details are mostly zeroed out. I'll look up whether that's flag-controlled:
Copy code
name = pack_inci[...]
         query = select * [...]
      interval = 3600
    executions = 0
 last_executed = 0
    denylisted = 0
   output_size =
     wall_time = 0
     user_time = 0
   system_time = 0
average_memory = 0
z

zwass

01/25/2022, 1:07 AM
Hmmm, the info in the 
osquery_schedule
table is since the last startup. Persisting this across restarts is something @sharvil and I have been discussing putting on Fleet's osquery contribution roadmap.
h

HarlanF

01/25/2022, 1:08 AM
Ah, interesting, that's useful information, since we'd just rolled out updates today, and think osquery restarted most everywhere.
Was poking around the source code, and found that I can do 
--schedule_lognames=true
flag and get the system to output which query it's executing to the logs.
s

seph

01/25/2022, 11:44 AM
IIRC you could also turn on verbose logging
ty 1
h

HarlanF

01/25/2022, 9:33 PM
Yeah, I could, but across a fleet that'd be too much info. This'll get me what I need, the schedule_lognames thing above.
2 Views
Powered by