[Bro] File Extraction Gaps
by Weasel, Gary W Jr CIV DISA RE (US)
Hey Bro List,
So I seem to be running into a problem with file extraction (or perhaps just file analysis in general). I have a basic extraction script running pulling out EXEs that are seen coming across HTTP and for some reason, there are consistently a large number of file gaps in the file it sees. I have a custom log outputting the fuid for any file_gap event (https://www.bro.org/sphinx/scripts/base/bif/event.bif.bro.html#id-file_gap), and I seem to get a wildly varying number of file gap events for a given file.
In my example, I am curling an exe to a server, where that traffic is spanned to my Bro sensor (the exe in question is 1 MB in size). If I curl repeatedly, Bro sees all the files, but the number of file gap events varies wildly (anywhere from 2 or 3 to over 100). The part that gets me, if I tcpdump alongside Bro, and pull the files out of pcap, they're all intact and hash correctly, so I know I'm getting all the packets on wire. Bro and PF_RING report 0 packet loss.
Does anyone have anything that could shed light on what's going on here?