Channels
  • Mystery Incorporated

    Mystery Incorporated

    9 months ago
    Hello please see here because the query for python packages is returning different to the python packages installed on the device. https://osquery.slack.com/archives/C01DXJL16D8/p1630781846091900
  • s

    seph

    9 months ago
    Debugging via screenshot can be hard. I have to squint much harder than cut and pasted text in code blocks.
    But, it’s not clear to me what any of that is showing.
    the python table enumerate the python packages at various directories. You can probably tell it what directory to enumerate to.
    Is your pip command using the same directory?
    Who knows. It’s not in your screen shots.
  • Mystery Incorporated

    Mystery Incorporated

    9 months ago
    I was just doing
    pip list

    I was not aware of any directories or anything I just use
    pip install x

    i think
    pip list
    is meant to list all packages installed by pip isn't it?
  • s

    seph

    9 months ago
    Which pip. You could have many.
  • Mystery Incorporated

    Mystery Incorporated

    9 months ago
    @seph well irrespective of which pip,
    SELECT name AS name, version AS version, 'Package (Python)' AS type, 'python_packages' AS source FROM python_packages
    Should list ALL python packages installed on the system right? Even if I had 1 million pips, it's not. I can see for sure that cryptography 3.4.8 is installed, yet that query only lists 2.8 so it is totally broken listing phantom packages that no longer exist and not listing the ones that updated them.
    This is causing the vulnerability checker in fleet to fail, and fleet says it is osquery being the problem due to the erroneous output from that query.
    Here is fleet
    Yet the query above only returns v2.8 (which it should not as it doesn't exist)
  • s

    seph

    9 months ago
    Answering the question about a million pips… I think this is a philosophical question. osquery examines common locations for python packages. These are enumerated by https://github.com/osquery/osquery/blob/482a751f9f55ad31333ecf75b645a2334d943774/osquery/tables/system/python_packages.cpp#L34-L48 It does not search your entire disk looking for likely python packages. It will never find venvs built into random places, or similar. I tend to think osquery’s approach is correct. But it is not always what’s expected.
    I have no idea what fleet is doing, what you’re doing, or any of the larger context needed to debug that. Fleet is, ultimately, just running queries. So if fleet can find it, you should be able to find it otherwise.
    That code snippet is probably incomplete — it looks like other parts also search additional places.
  • Mystery Incorporated

    Mystery Incorporated

    9 months ago
    Ok I see the issue, that cryptography v2.8 is in /usr/lib/python3/dist-packages. But cryptography v3.4.8 is in /usr/local/lib/python3.8/dist-packages/ But osquery is only reporting v2.8 but according to the code you just pointed me too it should be reporting from both paths?
  • s

    seph

    9 months ago
  • Mystery Incorporated

    Mystery Incorporated

    9 months ago
    Yep v3.4.8 is in there, but the query is not returning it
  • s

    seph

    9 months ago
    Are you using osqueryi? Or distributing this via fleet? If the latter, can you repreoduce on the local machine?
  • Mystery Incorporated

    Mystery Incorporated

    9 months ago
    I am just running the query as an on demand query from fleet
  • s

    seph

    9 months ago
    How confident are you that fleet isn’t doing any weird caching here? Can you preproduce this on the local machine?
    (If you have osqueryd, you can invoke it interactively with the
    -S
    option)
  • Mystery Incorporated

    Mystery Incorporated

    9 months ago
    Output from osqueryi on device only lists v2.8
  • s

    seph

    9 months ago
    And that’s running as root, correct?
  • Mystery Incorporated

    Mystery Incorporated

    9 months ago
    I just ran osqueryi with a sudo and same result
    oh hang on it's not in alphabetical order
    there is another list below that list
    so the v3.4.8 is further down the list
  • s

    seph

    9 months ago
    In retrospect, that’s not surprising. osquery searches the directories and appends. If you want to sort it, add an ORDER BY to the sql.
  • Mystery Incorporated

    Mystery Incorporated

    9 months ago
    yea let me see if it's same thing from fleet