Hi Team, I am testing osquery cli on a japanese windows machine and it seems not to recognize japanese characters correctly. The query i run is SELECT path FROM file WHERE path like 'C:\Users\admin.TEST\Desktop\TestFolder\%%'; , and in that folder i have a japanese file with the name ありがとうございます and it is not found, and i have a file name Jap1typeこんにちは.txt and the result that is returned {"path":"C:\\Users\\admin.TEST\\Desktop\\TestFolder\\Jap1type縺薙ｓ縺ｫ縺｡縺ｯ.txt"}, (on an english machine the returned value is Jap1typeήΒΥήΓΥήΒτήΒκήΒψ.txt) , it is the same output in a command prompt and when redirected to a file. Anyone have an idea if osquery supports japanese (unicode) characters correctly and if yes, how can i get the expected result? any advice or direction is appreciated

Hi Chaim. It looks like a path name is not being preserved in Unicode/UTF8, but that's just a guess. Historically, I think osquery has not really been internationalized.

Hi Mike, thanks for the response. If i use Japanese characters in the path itself it is responding correctly, just not in the file name.

I tested by creating the same filenames, and confirmed that they appear incorrectly when shown in osqueryi output, but also in a PowerShell window and in a CMD window, even though they appear "correctly" in Windows Explorer. It appears the problem is not in how osquery retrieves or stores or displays the names, but in literally the font of the console or PowerShell window. It is defaulted to the

Consolas

font, which doesn't have the glyphs for Japanese (or Chinese, or Korean) characters. Switching the shell to

MS Gothic

, the results appear correctly for me.