https://github.com/osquery/osquery logo
Powered by
#general
Title
# general
s

Stefano Bonicatti

02/15/2020, 12:40 PM
@Prateek Kumar Nischal the master branch of our repo is still targeting osquery 3.3.2, for osquery 4.x you have to check this branch https://github.com/trailofbits/osquery-extensions/tree/extension_porting_4.0.1
p

Prateek Kumar Nischal

02/15/2020, 12:58 PM
Thanks @Stefano Bonicatti, I did move to
Copy code
$ git br
* extension_porting_4.0.1 6c0f9b3 [origin/extension_porting_4.0.1] Other cleanups
  master                  9a6be96 [origin/master] Improve build instructions (#49)
PS: I have removed all extensions for now.. I am just using the 
darwin_unified_log
. Also, modified the submodule config to only include json11.
It's linked and still I get the same issue.. (running from the osquery project root)
Copy code
$ make externals
Built target externals
Exits at this
s

Stefano Bonicatti

02/15/2020, 1:02 PM
How does the CMake configure log looks like?
p

Prateek Kumar Nischal

02/15/2020, 1:04 PM
CMake run at osquery build dir.. ?
s

Stefano Bonicatti

02/15/2020, 1:06 PM
I mean when you run 
cmake -DOSQUERY_TOOLCHAIN_SYSROOT=...
from the build folder, what was the whole output
p

Prateek Kumar Nischal

02/15/2020, 1:07 PM
I was following:
Copy code
# Download source
git clone <https://github.com/osquery/osquery>
cd osquery

# Configure
mkdir build; cd build
cmake -DCMAKE_OSX_DEPLOYMENT_TARGET=10.11 ..

# Build
cmake --build .
and this runs fine..
Copy code
$ cmake -DCMAKE_OSX_DEPLOYMENT_TARGET=10.11 ..
CMake Warning (dev) in CMakeLists.txt:
  No project() command is present.  The top-level CMakeLists.txt file must
  contain a literal, direct call to the project() command.  Add a line of
  code such as

    project(ProjectName)

  near the top of the file, but after cmake_minimum_required().

  CMake is pretending there is a "project(Project)" command on the first
  line.
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) in CMakeLists.txt:
  No cmake_minimum_required command is present.  A line of code such as

    cmake_minimum_required(VERSION 3.16)

  should be added at the top of the file.  The version specified may be lower
  if you wish to support older CMake versions for this project.  For more
  information run "cmake --help-policy CMP0000".
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Configuring done
-- Generating done
-- Build files have been written to: /Users/p0n002h/git/osquery
s

Stefano Bonicatti

02/15/2020, 1:11 PM
So I do see that the 
externals
target indeed doesn't trigger the build, I'll try to understand why, though those messages you see during configuration are not a good sign. How exactly did you remove the other extensions etc?
p

Prateek Kumar Nischal

02/15/2020, 1:13 PM
Just rm.. and then edit the submodule files.. I saw the CMake file.. it didn't have any explicit mention of the specific folders.. so I did not touch that
looks like my osquery build broke.. now 
cmake --build .
is throwing missing openssl cmake file errors..
Copy code
make[2]: *** No rule to make target `installed_formulas/openssl/lib/libssl.a', needed by `osquery/osqueryd'.  Stop.
make[1]: *** [osquery/CMakeFiles/osqueryd.dir/all] Error 2
make: *** [all] Error 2
i'll try to start from scratch
s

Stefano Bonicatti

02/15/2020, 1:31 PM
There's a small update I have to do on our extensions due to the last release, which is not the reason of this failure, but that it will make them fail to compile later. Also, if you want to specify which extensions to build, please use the env var 
TRAILOFBITS_EXTENSIONS_TO_BUILD
; this is mentioned in the README too.
Also, the 
externals
target right now only triggers the build of extensions that do not have their own CMakeLists.txt
In this case you have to build the 
trailofbits_osquery_extensions.ext
target. You can find that by running 
make help | grep trailofbits
p

Prateek Kumar Nischal

02/15/2020, 1:39 PM
I had read about the 
TRAILOFBITS_EXTENSIONS_TO_BUILD
, the reason of deleting the other extensions was to ignore the submodule clone, it was taking wayyy to long to clone the Pcap++.
s

Stefano Bonicatti

02/15/2020, 1:46 PM
I'm not sure I follow, it should be the 
git clone --recurse-submodules <url>
which is cloning all submodules, it's not necessary to delete anything, you can just drop the option from the clone and then 
git submodule update --init <submodule path>
on what you need.
p

Prateek Kumar Nischal

02/15/2020, 1:48 PM
yes.. that's what I realised later.. 😅 That;s just my bad..
s

Stefano Bonicatti

02/15/2020, 2:07 PM
Don't worry! There are still a couple of rough edges.
p

Prateek Kumar Nischal

02/15/2020, 2:08 PM
after running the configure, I was not able to find the trailofbits.. 😕. The configure and build ran successfully..
s

Stefano Bonicatti

02/15/2020, 2:11 PM
That might be caused by a typo in the extensions project name I just fixed ahah
might want to pull again
actually no, it shouldn't be that; you mean you tried to 
make help | grep trailofbits
and it was empty?
did the extensions build at all?
p

Prateek Kumar Nischal

02/15/2020, 2:14 PM
I had started from scratch.. so I just ran the configure and 
cmake --build .
for now and the trailofbits project is linked in the 
externals
directory
do I need to run anything else.. ?
s

Stefano Bonicatti

02/15/2020, 2:17 PM
if you do see 
<build folder>/external/<link name>/trailofbits_osquery_extensions.ext
It mean the extensions are built. I was wondering though why you said that you weren't able to find 
trailofbits
. You mean the target or the extension binary?
p

Prateek Kumar Nischal

02/15/2020, 2:17 PM
Copy code
$ pwd
/Users/p0n002h/git/osquery/build/external
$ ls
CMakeFiles          Makefile            cmake_install.cmake
I was not able to find the target 
trailofbits
It's empty.. so the extension did not build
Hey.. by the way.. appreciate it man.. thanks for helping me.. 😄
The funny thing is that I can't even find the 
example
extension file too.. 😕
Copy code
$ pwd
/Users/p0n002h/git/osquery
$ find . -name "example_extension.ext"
$
@Stefano Bonicatti Any luck.. ?
s

Stefano Bonicatti

02/15/2020, 3:45 PM
could you delete the 
CMakeCache.txt
file in the osquery build folder, then run the cmake configure again and copy here the log?
p

Prateek Kumar Nischal

02/15/2020, 4:33 PM
Looks more interesting this time.
Copy code
-- ================================
-- Trail of Bits osquery extensions
-- Version: 1.2
-- ================================
Importing libraries from '/Users/p0n002h/git/osquery/external/extension_trailofbits/libraries/'
 > extutils
 > firewall
 > hostsfile
Importing extensions from '/Users/p0n002h/git/osquery/external/extension_trailofbits'
 ! Missing '.platforms` file: CMakeFiles. Skipping...
 > darwin_unified_log
 > efigy
-- Found CURL: /usr/lib/libcurl.dylib (found version "7.54.0")
 > fwctl
 > santa
-- ================================
cmake.log
It did pick the files 😄
It failed the build though.. looks like some autogen code.
Copy code
/Users/p0n002h/git/osquery/build/external/extension_trailofbits/osquery_extension_group_main.cpp:42:10: error: no member named 'waitThenShutdown' in 'osquery::Initializer'; did you mean 'waitForShutdown'?
  runner.waitThenShutdown();
         ^~~~~~~~~~~~~~~~
         waitForShutdown
build.log
s

Stefano Bonicatti

02/15/2020, 4:43 PM
yep that is what I was talking about earlier, you do need to change to that function. For now you can do so in the 
tools/codegen/templates/osquery_extension_group_main.cpp.in
file
p

Prateek Kumar Nischal

02/15/2020, 4:51 PM
Yes.. I did change the file and I am able to build
s

Stefano Bonicatti

02/15/2020, 4:53 PM
good!
meanwhile I've opened a PR to fix that
Ah I see you just opened one, keep yours 🙂
p

Prateek Kumar Nischal

02/15/2020, 4:55 PM
😅
thanks..
3 Views
Powered by