Channels
  • t

    Tal Kapon

    11 months ago
    hi team, I am trying to use osquery to get continuously bpf_socket_events. I am using osqueryi since it is already installed as part of our agent. My program starts osqueryi, and every minute sends query via stdin and grabs the result via stdout. However, I noticed the memory consumption of osqueryi starts at 200M and goes up very quickly even if I configure events_max = 2000 and events_expiry = 120. Can you help me understand why is that? Also, if there is a better way to get events from bpf_socket_events continuously with low impact on the hosting machine, I would like to learn about that. Thanks a lot
  • clong

    clong

    11 months ago
    @Tal Kapon any reason you wouldn’t just use osqueryd?
    osqueryi is geared more towards someone at the keyboard writing an interactive query and isn’t really designed to be queried on a regular basis
  • t

    Tal Kapon

    11 months ago
    thanks. the reason I used osqueryi is that it’s already installed with our agent. Also, it’s more complicated to create logging plugin than simply read stdout. Still, I can use osqueryd if there is a significant difference. Can you elaborate how come the memory footprint is so large given events_max = 2000 and events_expiry = 120? I would like to learn and understand better.
  • a

    alessandrogario

    11 months ago
    @Tal Kapon How does memory usage increase? is it after BPF is initialized? You can pass the
    --verbose
    option to know when the probes are installed
    You can estimate how much memory should be in use; scroll down to 'Memory usage' in the following link: https://osquery.readthedocs.io/en/stable/deployment/process-auditing/#linux-process-and-socket-auditing-using-bpf
    The short version is
    buffer_storage_bytes = memory_pool_count * (bpf_buffer_storage_size * 4096) * possible_cpu_count
    
    +
    
    perf_bytes = (2 ^ bpf_perf_event_array_exp) * online_cpu_count
    With memory_pool_count=6, bpf_buffer_storage_size set through command line options, and possible_cpu_count/online_cpu_count based on hardware
    You can see those two numbers here:
    possible_cpu_count: /sys/devices/system/cpu/possible
    online_cpu_count: /sys/devices/system/cpu/online
    If you are running on VMware Fusion, possible_cpu_count is set to 128 unless you disable CPU hotswapping in the .vmx file
View count: 11