Reviving my old thread - this project was on hiatus for a while, but
is now generating useful data, although not yet in production.
What I ended up doing was transport the syslogs of interest (apache
logs) via stunnel to my bro box, which then runs a custom python
1. Parses and extracts the logs into python variables
2. Construct plausible http session conversation.
3. Packetize the session into TCP packets
4. Push tcp packets onto virtual interface
5. bro listening to virtual interface performs normal IDS monitoring of traffic.
Here's an example:
Jul 3 09:34:54 126.96.36.199 httpd: www.nersc.gov
- - [03/Jul/2013:09:34:54 -0700] "GET /robots.txt HTTP/1.1" 200 82
"Mozilla/5.0 (compatible; Googlebot/2.1;
Bro http logs:
09:34:58 u3qPWFy8m9 188.8.131.52 64555 184.108.40.206 80 GET www.nersc.gov
+http://www.google.com/bot.html) 0 0 200 <empty>(empty) - - - - - -
A few points:
1.Tested creating packets to make the conn logs show the correct
amount of data returned by the server, as reported in the log, but
eventually chose to not do that, as that is of limited value.
2. Support both ipv4 & ipv6 - if a host is dual homed, typically the
syslog entry will be from the ipv4 address, but the requesting ip may
be ipv6, in which case we convert both address to ipv6 using several
The value of this, from our perspective, is that we can now perform
the usual http IDS functions on https connections to our syslogging
webservers, without having to store the certs in our bro system for
decryption. We also have visibility on intrasite traffic to those
Alpha quality code available for the asking.
On Fri, Mar 22, 2013 at 2:18 PM, Jim Mellander <jmellander(a)lbl.gov> wrote:
Well, its unfortunate that we can't feed in data
from other sources
and subject it to the same policies that network traffic is subject
In the meantime, I may just write some code that fakes the data into
pcap files that can be read by bro directly.
On Fri, Mar 22, 2013 at 1:54 PM, Seth Hall <seth(a)icir.org> wrote:
> On Mar 22, 2013, at 4:04 PM, Jim Mellander <jmellander(a)LBL.GOV> wrote:
>> Does anyone have suggestions on how to proceed with this?
> It wouldn't work very well. :)
> Nearly all of the detections rely on the various http_ events. I would go down a
slightly different route with logs than I would with raw traffic. This is something that
I've been talking about for quite a while and I suspect something related to happen
in the next year.
> I think it's really cool that you're importing logs into Bro!
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network