#fleet - Hey I updated my fleet and osq

Home Docs

Join the conversation Join Slack

Channels

o
Ojas
5 months ago
Hey I updated my fleet and osquery recently but now: I am not able to find the osquery logs in /var/osquery/* not in C:\Program Files\osquery*
👀 1
there seems to be a new folder orbit. Inside this i did find osquery.db but the logs are not the sme way it used to be. its very badly formatted and just in one single file. Most of them just say what fleet tried to run on the machine and no results are logged
mac logs
ubuntu
Windows
All i need now is someone to show me how to configure the logs to be stored on the host itself. This is my config rn and it dosent seem to do the trick
Luke Heath
5 months ago
Hi Ojas, yes you can do this using the filesystem as the logger plugin: https://fleetdm.com/docs/using-fleet/osquery-logs#filesystem
Lucas Rodriguez
5 months ago
Hi Ojas!
```
logger_plugin
```
is for osquery
```
result
```
(scheduled queries results) and osquery
```
status
```
logs. If you are interested on osquery logs in general, you can check the following paths:https://github.com/fleetdm/fleet/tree/main/orbit#logs (Both Orbit and osquery logs should be in those locations.)
o
Ojas
5 months ago
@Luke Heath Thanks for you response. So i have configured correctly as given in above screenshot? Coz even after configuring this i dont see any proper logs on my hosts. I do see a file which contains data of what agent is connecting to and what query ran on the machine and everything. But not he proper logs like which contains the results of the query which ran on the machine.
@Lucas Rodriguez thanks for you response. I have configured the logger_plugin but still no logs on the endpoint. The orbit logs are not as proper as the older osquery logs which were is /var/log/osquery/* In the orbit logs file we have everything logged like what agent is doing what query ran and not in a standard format. any way i can configure it to goto old format
I have this err on mac os: E0304 05:47:53.505184 129021440 shutdown.cpp:75] Cannot activate filesystem logger plugin: Could not create file: /var/log/osquery/osqueryd.results.log
If i create the folder osquery there it starts working fine but when the folder is not there it throws err that cannot create it. Any fix for this? as i have to install it on alot of systems, manually creating those folders would be painfull
Same err on windows: Cannot activate filesystem logger plugin: Could not create file: \Program Files\osquery\
Lucas Rodriguez
5 months ago
Hi @Ojas! I believe you hit this bug: https://github.com/fleetdm/fleet/issues/4146 (Which will be fixed on our next release coming today/early-next-week, TBD)
You will have to remove
```
filesystem
```
from
```
logger_plugin
```
(and just leave
```
tls
```
or empty until the new release is out).
Though, once we release the new Orbit version, your instances should auto-update.
o
Ojas
5 months ago
alright. Thanks Lucas. I’ll wait for the update.
So after the update do i need to again generate the installer agent or do i need to install osquery on machine again to get updated config? or will that be auto-updated too
Lucas Rodriguez
5 months ago
Hi Ojas!, once we release fleet-osquery (aka orbit) - sometime early this week - it should auto-update automatically. (If things work as expected you won't need to re-generate the installers.)
o
Ojas
5 months ago
awesome thanks
Hey @Lucas Rodriguez I still dont see any logs created. Any update on patch? Do i need to generate a new installer?
Also now i see another older issue, my fleet_osquery service in windows keeps stopping.
Lucas Rodriguez
4 months ago
Hi @Ojas!
Do i need to generate a new installer? No, should auto-update.
Also now i see another older issue, my fleet_osquery service in windows keeps stopping. Could you check
```
C:\Windows\system32\config\systemprofile\AppData\Local\FleetDM\Orbit\Logs\orbit-osquery.log
```
? (You may be hitting a known issue we are trying to fix for next release.)
o
Ojas
4 months ago
it says ” Cannot activate filesystem logger plugin” :?
@User i still see the old error: 2022-03-30T11:19:42Z INF start osqueryd cmd=“C:\Program Files\Orbit\bin\osqueryd\windows\stable\osqueryd.exe --pidfile=C:\Program Files\Orbit\osquery.pid --database_path=C:\Program Files\Orbit\osquery.db --extensions_socket=\\.\pipe\orbit-osquery-extension --enroll_secret_env ENROLL_SECRET --host_identifier=uuid --tls_hostname=fleet.tpsec.co --enroll_tls_endpoint=/api/v1/osquery/enroll --config_plugin=tls --config_tls_endpoint=/api/v1/osquery/config --config_refresh=60 --disable_distributed=false --distributed_plugin=tls --distributed_tls_max_attempts=10 --distributed_tls_read_endpoint=/api/v1/osquery/distributed/read --distributed_tls_write_endpoint=/api/v1/osquery/distributed/write --logger_plugin=tls --logger_tls_endpoint=/api/v1/osquery/log --disable_carver=false --carver_disable_function=false --carver_start_endpoint=/api/v1/osquery/carve/begin --carver_continue_endpoint=/api/v1/osquery/carve/block --carver_block_size=2000000 --tls_server_certs C:\Program Files\Orbit\certs.pem --force --flagfile C:\Program Files\Orbit\osquery.flags” E0330 11:22:40.311066 3380 shutdown.cpp:79] Cannot activate filesystem logger plugin: Could not create file: \Program Files\osquery\log\osqueryd.results.log E0330 11:22:42.511026 5160 shutdown.cpp:79] Worker returned exit status
Also on manually creating folders it works fine. It’s still the issue of not able to create the folders
Lucas Rodriguez
4 months ago
OK, could you run
```
"C:\Program Files\Orbit\bin\orbit\orbit.exe" --version
```
?
Could not create file: \Program Files\osquery\log\osqueryd.results.log On the latest version we changed the path, that looks like the old default path.
o
Ojas
4 months ago
orbit 0.0.6
Lucas Rodriguez
4 months ago
OK, latest is 0.0.7 (and soon 0.0.8). For some reason it's not auto-updating.
Does the host have access to https://tuf.fleetctl.com?
o
Ojas
4 months ago
how do i check that?
Lucas Rodriguez
4 months ago
Any other network error logs related to updating (in
```
C:\Windows\system32\config\systemprofile\AppData\Local\FleetDM\Orbit\Logs\orbit-osquery.log
```
)?
how do i check that? Try visiting the URL from a browser in the host, or using the
```
curl
```
or
```
wget
```
commands (if available).
o
Ojas
4 months ago
i can ping it from the host
2022-03-30T11:19:42Z INF start osqueryd cmd=“C:\Program Files\Orbit\bin\osqueryd\windows\stable\osqueryd.exe --pidfile=C:\Program Files\Orbit\osquery.pid --database_path=C:\Program Files\Orbit\osquery.db --extensions_socket=\\.\pipe\orbit-osquery-extension --enroll_secret_env ENROLL_SECRET --host_identifier=uuid --tls_hostname=fleet.tpsec.co --enroll_tls_endpoint=/api/v1/osquery/enroll --config_plugin=tls --config_tls_endpoint=/api/v1/osquery/config --config_refresh=60 --disable_distributed=false --distributed_plugin=tls --distributed_tls_max_attempts=10 --distributed_tls_read_endpoint=/api/v1/osquery/distributed/read --distributed_tls_write_endpoint=/api/v1/osquery/distributed/write --logger_plugin=tls --logger_tls_endpoint=/api/v1/osquery/log --disable_carver=false --carver_disable_function=false --carver_start_endpoint=/api/v1/osquery/carve/begin --carver_continue_endpoint=/api/v1/osquery/carve/block --carver_block_size=2000000 --tls_server_certs C:\Program Files\Orbit\certs.pem --force --flagfile C:\Program Files\Orbit\osquery.flags” E0330 11:19:44.483722 4888 shutdown.cpp:79] Cannot activate filesystem logger plugin: Could not create file: \Program Files\osquery\log\osqueryd.results.log E0330 11:19:45.709700 5776 shutdown.cpp:79] Worker returned exit status 2022-03-30T11:19:45Z ERR unexpected exit error=“osqueryd exited with error: exit status 78" 2022-03-30T11:19:47Z INF start osqueryd cmd=“C:\Program Files\Orbit\bin\osqueryd\windows\stable\osqueryd.exe --pidfile=C:\Program Files\Orbit\osquery.pid --database_path=C:\Program Files\Orbit\osquery.db --extensions_socket=\\.\pipe\orbit-osquery-extension --enroll_secret_env ENROLL_SECRET --host_identifier=uuid --tls_hostname=fleet.tpsec.co --enroll_tls_endpoint=/api/v1/osquery/enroll --config_plugin=tls --config_tls_endpoint=/api/v1/osquery/config --config_refresh=60 --disable_distributed=false --distributed_plugin=tls --distributed_tls_max_attempts=10 --distributed_tls_read_endpoint=/api/v1/osquery/distributed/read --distributed_tls_write_endpoint=/api/v1/osquery/distributed/write --logger_plugin=tls --logger_tls_endpoint=/api/v1/osquery/log --disable_carver=false --carver_disable_function=false --carver_start_endpoint=/api/v1/osquery/carve/begin --carver_continue_endpoint=/api/v1/osquery/carve/block --carver_block_size=2000000 --tls_server_certs C:\Program Files\Orbit\certs.pem --force --flagfile C:\Program Files\Orbit\osquery.flags” E0330 11:19:48.367669 5808 shutdown.cpp:79] Cannot activate filesystem logger plugin: Could not create file: \Program Files\osquery\log\osqueryd.results.log E0330 11:19:51.128486 3348 shutdown.cpp:79] Worker returned exit status 2022-03-30T11:21:33Z INF start osqueryd cmd=“C:\Program Files\Orbit\bin\osqueryd\windows\stable\osqueryd.exe --pidfile=C:\Program Files\Orbit\osquery.pid --database_path=C:\Program Files\Orbit\osquery.db --extensions_socket=\\.\pipe\orbit-osquery-extension --enroll_secret_env ENROLL_SECRET --host_identifier=uuid --tls_hostname=fleet.tpsec.co --enroll_tls_endpoint=/api/v1/osquery/enroll --config_plugin=tls --config_tls_endpoint=/api/v1/osquery/config --config_refresh=60 --disable_distributed=false --distributed_plugin=tls --distributed_tls_max_attempts=10 --distributed_tls_read_endpoint=/api/v1/osquery/distributed/read --distributed_tls_write_endpoint=/api/v1/osquery/distributed/write --logger_plugin=tls --logger_tls_endpoint=/api/v1/osquery/log --disable_carver=false --carver_disable_function=false --carver_start_endpoint=/api/v1/osquery/carve/begin --carver_continue_endpoint=/api/v1/osquery/carve/block --carver_block_size=2000000 --tls_server_certs C:\Program Files\Orbit\certs.pem --force --flagfile C:\Program Files\Orbit\osquery.flags” E0330 11:21:34.335160 3884 shutdown.cpp:79] Cannot activate filesystem logger plugin: Could not create file: \Program Files\osquery\log\osqueryd.results.log E0330 11:21:37.178841 3876 shutdown.cpp:79] Worker returned exit status 2022-03-30T11:22:38Z INF start osqueryd cmd=“C:\Program Files\Orbit\bin\osqueryd\windows\stable\osqueryd.exe --pidfile=C:\Program Files\Orbit\osquery.pid --database_path=C:\Program Files\Orbit\osquery.db --extensions_socket=\\.\pipe\orbit-osquery-extension --enroll_secret_env ENROLL_SECRET --host_identifier=uuid --tls_hostname=fleet.tpsec.co --enroll_tls_endpoint=/api/v1/osquery/enroll --config_plugin=tls --config_tls_endpoint=/api/v1/osquery/config --config_refresh=60 --disable_distributed=false --distributed_plugin=tls --distributed_tls_max_attempts=10 --distributed_tls_read_endpoint=/api/v1/osquery/distributed/read --distributed_tls_write_endpoint=/api/v1/osquery/distributed/write --logger_plugin=tls --logger_tls_endpoint=/api/v1/osquery/log --disable_carver=false --carver_disable_function=false --carver_start_endpoint=/api/v1/osquery/carve/begin --carver_continue_endpoint=/api/v1/osquery/carve/block --carver_block_size=2000000 --tls_server_certs C:\Program Files\Orbit\certs.pem --force --flagfile C:\Program Files\Orbit\osquery.flags” E0330 11:22:40.311066 3380 shutdown.cpp:79] Cannot activate filesystem logger plugin: Could not create file: \Program Files\osquery\log\osqueryd.results.log E0330 11:22:42.511026 5160 shutdown.cpp:79] Worker returned exit status 2022-03-30T11:25:42Z INF start osqueryd cmd=“C:\Program Files\Orbit\bin\osqueryd\windows\stable\osqueryd.exe --pidfile=C:\Program Files\Orbit\osquery.pid --database_path=C:\Program Files\Orbit\osquery.db --extensions_socket=\\.\pipe\orbit-osquery-extension --enroll_secret_env ENROLL_SECRET --host_identifier=uuid --tls_hostname=fleet.tpsec.co --enroll_tls_endpoint=/api/v1/osquery/enroll --config_plugin=tls --config_tls_endpoint=/api/v1/osquery/config --config_refresh=60 --disable_distributed=false --distributed_plugin=tls --distributed_tls_max_attempts=10 --distributed_tls_read_endpoint=/api/v1/osquery/distributed/read --distributed_tls_write_endpoint=/api/v1/osquery/distributed/write --logger_plugin=tls --logger_tls_endpoint=/api/v1/osquery/log --disable_carver=false --carver_disable_function=false --carver_start_endpoint=/api/v1/osquery/carve/begin --carver_continue_endpoint=/api/v1/osquery/carve/block --carver_block_size=2000000 --tls_server_certs C:\Program Files\Orbit\certs.pem --force --flagfile C:\Program Files\Orbit\osquery.flags” I0330 11:25:43.293851 3316 eventfactory.cpp:156] Event publisher not enabled: ntfs_event_publisher: NTFS event publisher disabled via configuration I0330 11:25:46.206353 1712 interfaces.cpp:102] Failed to retrieve network statistics for interface 4 I0330 11:25:46.266278 1712 interfaces.cpp:102] Failed to retrieve network statistics for interface 1 I0330 11:25:46.269050 1712 interfaces.cpp:130] Failed to retrieve physical state for interface 1 I0330 11:25:46.287117 1712 interfaces.cpp:157] Failed to retrieve DHCP and DNS information for interface 1 I0330 11:25:46.377097 1712 registry.cpp:555] Failed to expand globs: Failed to open registry handle I0330 11:25:46.389892 1712 registry.cpp:555] Failed to expand globs: Failed to open registry handle I0330 11:25:46.390563 1712 registry.cpp:555] Failed to expand globs: Failed to open registry handle I0330 11:25:46.392493 1712 registry.cpp:555] Failed to expand globs: Failed to open registry handle I0330 11:25:46.395962 1712 registry.cpp:555] Failed to expand globs: Failed to open registry handle I0330 11:25:46.396636 1712 registry.cpp:555] Failed to expand globs: Failed to open registry handle I0330 11:25:46.398648 1712 registry.cpp:555] Failed to expand globs: Failed to open registry handle I0330 11:25:46.400411 1712 registry.cpp:555] Failed to expand globs: Failed to open registry handle I0330 11:25:46.401648 1712 registry.cpp:555] Failed to expand globs: Failed to open registry handle W0330 11:25:46.763278 1712 chocolatey_packages.cpp:65] Did not find chocolatey path environment variable E0330 11:25:48.131688 1712 distributed.cpp:144] Error executing distributed query: fleet_policy_query_1: no such table: gatekeeper W0330 11:25:48.153442 1712 bitlocker_info.cpp:52] Error retreiving information from WMI. E0330 11:25:48.159821 1712 distributed.cpp:144] Error executing distributed query: fleet_policy_query_3: no such table: disk_encryption E0330 11:25:48.164297 1712 distributed.cpp:144] Error executing distributed query: fleet_policy_query_4: no such table: disk_encryption E0330 11:25:48.168017 1712 distributed.cpp:144] Error executing distributed query: fleet_policy_query_5: no such table: sip_config E0330 11:25:48.170768 1712 distributed.cpp:144] Error executing distributed query: fleet_policy_query_6: no such table: managed_policies E0330 11:25:48.173740 1712 distributed.cpp:144] Error executing distributed query: fleet_policy_query_7: no such table: managed_policies E0330 11:25:48.176471 1712 distributed.cpp:144] Error executing distributed query: fleet_policy_query_8: no such table: managed_policies E0330 11:25:48.179143 1712 distributed.cpp:144] Error executing distributed query: fleet_policy_query_9: no such table: plist I0330 11:26:12.553747 2764 registry.cpp:555] Failed to expand globs: Failed to open registry handle W0330 11:26:12.555688 2764 virtual_table.cpp:961] The chrome_extensions table returns data based on the current user by default, consider JOINing against the users table W0330 11:26:12.558739 2764 virtual_table.cpp:961] The firefox_addons table returns data based on the current user by default, consider JOINing against the users table W0330 11:26:12.572578 2764 chocolatey_packages.cpp:65] Did not find chocolatey path environment variable W0330 11:26:12.583083 2764 virtual_table.cpp:961] The atom_packages table returns data based on the current user by default, consider JOINing against the users table I0330 11:26:12.585950 2764 query.cpp:102] Storing initial results for new scheduled query: pack_test_Get installed Windows software 2022-03-30T11:29:09Z INF start osqueryd cmd=“C:\Program Files\Orbit\bin\osqueryd\windows\stable\osqueryd.exe --pidfile=C:\Program Files\Orbit\osquery.pid --database_path=C:\Program Files\Orbit\osquery.db --extensions_socket=\\.\pipe\orbit-osquery-extension --enroll_secret_env ENROLL_SECRET --host_identifier=uuid --tls_hostname=fleet.tpsec.co --enroll_tls_endpoint=/api/v1/osquery/enroll --config_plugin=tls --config_tls_endpoint=/api/v1/osquery/config --config_refresh=60 --disable_distributed=false --distributed_plugin=tls --distributed_tls_max_attempts=10 --distributed_tls_read_endpoint=/api/v1/osquery/distributed/read --distributed_tls_write_endpoint=/api/v1/osquery/distributed/write --logger_plugin=tls --logger_tls_endpoint=/api/v1/osquery/log --disable_carver=false --carver_disable_function=false --carver_start_endpoint=/api/v1/osquery/carve/begin --carver_continue_endpoint=/api/v1/osquery/carve/block --carver_block_size=2000000 --tls_server_certs C:\Program Files\Orbit\certs.pem --force --flagfile C:\Program Files\Orbit\osquery.flags” I0330 11:29:09.557931 3980 eventfactory.cpp:156] Event publisher not enabled: ntfs_event_publisher: NTFS event publisher disabled via configuration I0330 11:32:44.828908 3940 registry.cpp:555] Failed to expand globs: Failed to open registry handle W0330 11:32:44.830849 3940 virtual_table.cpp:961] The chrome_extensions table returns data based on the current user by default, consider JOINing against the users table W0330 11:32:44.834173 3940 virtual_table.cpp:961] The firefox_addons table returns data based on the current user by default, consider JOINing against the users table W0330 11:32:44.848613 3940 chocolatey_packages.cpp:65] Did not find chocolatey path environment variable W0330 11:32:44.859524 3940 virtual_table.cpp:961] The atom_packages table returns data based on the current user by default, consider JOINing against the users table
Able to access the site from browser as well.
Lucas Rodriguez
4 months ago
OK, a guess: so due to the issue (which causes orbit to crash), orbit cannot auto-update. Could you disable the
```
filesystem
```
logger configuration from Fleet temporarily? (to give the process a chance to auto-update)
o
Ojas
4 months ago
Do i change it to tls? or just remove it from there all together?
Lucas Rodriguez
4 months ago
Yes, just
```
tls
```
.
o
Ojas
4 months ago
alright. Done config: options: logger_plugin: tls
Lucas Rodriguez
4 months ago
What I fear is that osquery already stored
```
filesystem
```
in its internal rocksdb local storage..., but let's see if this helps.
o
Ojas
4 months ago
should i restart the machine? or just wait few minutes?
Lucas Rodriguez
4 months ago
Let's wait a few minutes first.
✅ 1
Worst case scenario: You will need to regenerate a
```
msi
```
installer and re-install Orbit.
o
Ojas
4 months ago
for that would orbit need to be cleaned from the machine first? or as i run new installer it will auto install new one
Lucas Rodriguez
4 months ago
Yes. Running a new installer should override current installation. Let me know if it doesn't.
o
Ojas
4 months ago
yea with new installer it got updated to orbit 0.0.7
Lucas Rodriguez
4 months ago
OK, let me know if the
```
filesystem
```
change is still not working.
o
Ojas
4 months ago
Tried it on a new machine from scratch. Works nicely. thanks 😃 So everytime there is a patch/update do i need to create a new installer?
Lucas Rodriguez
4 months ago
No, orbit should automatically update. The issue here is that the filesystem configuration was causing issues at startup, so it didn't have the chance to run the auto-updater routine...
🙏 1

Join thread in Slack

View count: 3