I have a number of Launcher clients that are stuck running o

Question

I have a number of Launcher clients that are stuck running osquery 4 5 1 because the updated downloaded osqueryd is failing with a `segementation fault` ``` binary usr local launcher bin osqueryd updates 1652735666 osqueryd binaryName osqueryd caller findnew go 207 fullBinaryPath usr local launcher bin osqueryd level error msg not executable Skipping reason signal segmentation fault ts 2022 06 02T14 04 21 625141787Z updateDir usr local launcher bin osqueryd updates ``` I have removed the files in ` usr local launcher bin osqueryd updates ` and restarted launcher I see that the autoupdate process is kicked off ``` binaryName osqueryd caller autoupdate go 165 level debug msg Created Updater stagingPath var launcher securityonion osqueryd staging ts 2022 06 02T19 44 59 357491281Z updater osqueryd updatesDirectory usr local launcher bin osqueryd updates ``` But nothing appears to change Any thoughts on how to troubleshoot this further

seph · Answer

There s a lot to unpack here

seph · Answer

Why is ` usr local launcher bin osqueryd updates 1652735666 osqueryd` corrupt Did something happen on your client side

seph · Answer

Stuck is a bit odd launcher doesn t really get stuck per se What it does is download updates when they happen and then try to run them If it can run a file on local disk it ll skip that

seph · Answer

But launcher only runs updates when they happen It has no real concept of what version is should run just whether or not there has been a change One way to force it to notice and re update is to remove the local tuf store This is less scary than it sounds launcher will rebuild it from what s baked into the binary On your machines that s probably `var launcher securityonion osqueryd tuf` rm that and restart launcher

defensivedepth · Answer

Thanks for the thoughts still working on this Removing the local tuf store amp restarting launcher does not appear to force it to re update but it could be a timing issue Double checking some things

defensivedepth · Answer

``` binaryName osqueryd caller findnew go 230 fullBinaryPath usr local launcher bin osqueryd level debug msg no updates found ts 2022 06 03T20 26 44 163520332Z updateDir usr local launcher bin osqueryd updates ```

seph · Answer

How long did you wait after restarting launcher The updaters default to a 30 or 60 minut deay

defensivedepth · Answer

ya thats what I mean RE Timing Had an issue with the test system I was working with and it rebooted so not sure if it hit that 60min timer yet

defensivedepth · Answer

ah looks like osqueryd has an update delay of 90min

seph · Answer

60 and 90 I guess Seems believable

defensivedepth · Answer

Still not sure the root cause of the issue