Hey folks i’m trying to get eventing properly but i always see these errors.

W0218 10:08:54.518049  1393 events.cpp:311] Expiring events for subscriber: user_events (overflowed limit 500000)
W0218 10:10:09.537274  1393 events.cpp:311] Expiring events for subscriber: process_events (overflowed limit 500000)

this is the eventing part in my flags file hopefully its correct

--audit_allow_config=true
--audit_allow_sockets=true
--audit_persist=true
--disable_audit=false
--events_max=500000
--events_expiry=86400
--disable_events=false
--audit_persist
--events_optimize=true

any thoughts, suggestions or help. Thanks a lot.

The event tables only store up to the

events_max

number of events before expiring them. To avoid losing events, you'll need to query the table more frequently , or change the

events_max

to a larger number. Check out the flag descriptions here (if you haven't already that is 🙂 : https://osquery.readthedocs.io/en/stable/installation/cli-flags/#events-control-flags

Thanks a lot blaedj, i actually understand that, and i set my queries to

600

second, and that what confused me because the expiration is

86400

and my queries were

600

i was expecting to be able query before expiry not sure if there is a troubleshooting tip i can use to see if the number of events is low or what should i do next.

Ah gotcha. As I understand it, you could potentially get > 5k events generated in < 10 minutes. I don't know if it's much help but I'd probably start by increasing the query frequency to see if the errors still occur, if not that indicates the jogs are just piling up really quickly.

https://github.com/osquery/osquery/pull/6954 coincidentally I opened this PR yesterday to try to explain these settings better