zeek-dev October 2010

zeek-dev@lists.zeek.org

29 discussions

[Bro-Dev] #292: Cluttering of alarm.log with serialization messages

by Bro Tracker

#292: Cluttering of alarm.log with serialization messages ---------------------+------------------------------------------------------ Reporter: vallenti | Owner: robin Type: Problem | Status: new Priority: Normal | Component: BroControl Version: 1.5.1 | Keywords: ---------------------+------------------------------------------------------ In a standard setup with broctl, I find that `alarm.log` gets cluttered with a lot of messages like: {{{ 1288164542.559074 Starting incremental serialization... 1288164542.682096 Finished incremental serialization. 1288165442.559114 Starting incremental serialization... 1288165442.680014 Finished incremental serialization. 1288166342.559481 Starting incremental serialization... 1288166342.679497 Finished incremental serialization. ... }}} I don't remember the exact mechanism to tweak these messages (notice policies?), but since they don't seem critical, I suggest removing them from `alarm.log` in the default setup. -- Ticket URL: <http://tracker.icir.org/bro/ticket/292> Bro Tracker <http://tracker.icir.org/bro> Bro Issue Tracker

10 years, 10 months

[Bro-Dev] #283: Bro should support mixed MPLS and non-MPLS traffic

by Bro Tracker

#283: Bro should support mixed MPLS and non-MPLS traffic ----------------------------+----------------------------------------------- Reporter: robin | Owner: Type: Feature Request | Status: new Priority: Normal | Component: Bro Version: 1.5.1 | Keywords: ----------------------------+----------------------------------------------- Shouldn't be too hard to add based on an older patch for MPLS decapsulation. I have a trace for testing the mixed case, but I can't add that here for privacy reasons. For my own reference: msg-id for trace 4C743FE2.3030900 -- Ticket URL: <http://tracker.icir.org/bro/ticket/283> Bro Tracker <http://tracker.icir.org/bro> Bro Issue Tracker

10 years, 10 months

[Bro-Dev] #289: Crash in broctl due to missing lockfile.

by Bro Tracker

#289: Crash in broctl due to missing lockfile. ------------------------------+--------------------------------------------- Reporter: seth | Owner: robin Type: Problem | Status: new Priority: Normal | Component: BroControl Version: 1.5.2-devel (svn) | Keywords: ------------------------------+--------------------------------------------- I suppose the right way to handle this would just be to ignore it? Here's the stack trace from the python unhandled crash. {{{ waiting for lock ...........warning: removing stale lock Traceback (most recent call last): File "/home/bro/bin/broctl", line 726, in <module> loop.onecmd(line) File "/usr/local/lib/python2.5/cmd.py", line 219, in onecmd return func(arg) File "/home/bro/bin/broctl", line 180, in do_start self.lock() File "/home/bro/bin/broctl", line 96, in lock if not util.lock(): File "/bro/lib/broctl/BroControl/util.py", line 187, in lock while not _aquireLock(): File "/bro/lib/broctl/BroControl/util.py", line 138, in _aquireLock if _breakLock(): File "/bro/lib/broctl/BroControl/util.py", line 104, in _breakLock os.unlink(config.Config.lockfile) OSError: [Errno 2] No such file or directory: '/bro/spool/lock' }}} -- Ticket URL: <http://tracker.icir.org/bro/ticket/289> Bro Tracker <http://tracker.icir.org/bro> Bro Issue Tracker

10 years, 10 months

[Bro-Dev] #291: lookup_addr() only supports IPv4 addresses

by Bro Tracker

#291: lookup_addr() only supports IPv4 addresses -------------------+-------------------------------------------------------- Reporter: leres | Type: Problem Status: new | Priority: Normal Component: Bro | Version: 1.5.1 Keywords: | -------------------+-------------------------------------------------------- We've started deploying IPv6 on one subnet at LBL and bro is missing some IPv6 support; see attached crash report. -- Ticket URL: <http://tracker.icir.org/bro/ticket/291> Bro Tracker <http://tracker.icir.org/bro> Bro Issue Tracker

10 years, 10 months

[Bro-Dev] #293: scripts/local-interfaces cannot find ifconfig

by Bro Tracker

#293: scripts/local-interfaces cannot find ifconfig ---------------------+------------------------------------------------------ Reporter: vallenti | Owner: robin Type: Problem | Status: new Priority: Normal | Component: BroControl Version: 1.5.1 | Keywords: ---------------------+------------------------------------------------------ I find the following error in `debug.log`: {{{ 28 Oct 01:25:01 [local] /usr/local/share/broctl/scripts/local- interfaces 28 Oct 01:25:01 [local] exit code 0 28 Oct 01:25:01 Local IPs: /usr/local/share/broctl/scripts/local- interfaces: line 9: ifconfig: command not found 28 Oct 01:25:01 [local] sh }}} If the log entries reflect serial execution, this is a little weird because `local-interfaces` first terminates cleanly with exit code 0 but then cannot find `ifconfig`. Since `local-interfaces` merely invokes `ifconfig` from bash, I wonder how this can happen. The system I am working with is a vanilla CentOs 5.5 Linux where `ifconfig` is located in `/sbin`. -- Ticket URL: <http://tracker.icir.org/bro/ticket/293> Bro Tracker <http://tracker.icir.org/bro> Bro Issue Tracker

10 years, 10 months

Re: [Bro-Dev] Local MPLS coding support

by Seth Hall

<sending again, I sent it to the wrong mailing list address> Hi Keith! I left the full message you sent me for the benefit of everyone on the bro-dev mailing list. I just pushed a branch to our git repository that has more complete support for MPLS. I'd appreciate it if you could try it. I added some functionality to it that the patch in the tracker doesn't have. * Supports MPLS over Ethernet (which is likely what you're sniffing). * Builds the appropriate BPF filter so that MPLS encapsulated and non-encapsulated traffic is sent through. You just need to make sure and load the mpls.bro script. It *should* just work from that point (no filters to add or anything). If you define your own filter on the command line (with the -f flag), then the auto mpls support will not work. You will have to define the appropriate mpls filter in that case. If this works for you, then you can have your intern work on something else. :P Let us know if you have staff time needing projects to work on, I'm sure we can figure something out. git clone git://git.icir.org/bro git checkout origin/topic/seth/mpls (ignore all of the warnings) <do the normal build and install stuff> .Seth On Oct 28, 2010, at 11:27 AM, Keith Lehigh wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Hi Seth, > Hope all is going well for you with getting your head wrapped around > plans for the future of Bro. > We may have some hours available from an intern here to put to work > on solving our MPLS problem with Bro. This is one of a couple different > projects I have in mind, but before I get too far into considering this, > I wanted to get a little input from you. > I would aim to have this support added in such a way that it is > generic to any flavor of encapsulation and could be contributed back as > patches. Given that this person won't be working for us long term, I'm > not interested in having him make some local hacks which quickly > become unmaintainable. As such, do you think the internals of Bro will > change enough in the near future to make this irrelevant? > This question may be more directed at Robin. Am I being overly > naive in the amount of effort and rewriting that would need to be done > to make this work? I'm pretty sure the intern will need to get some > time in just getting up to speed on Bro, but I think he doesn't have to > be a subject matter expert on Bro. Again, am I being naive? > If we go with this project, I'll direct our intern to put questions > to the Bro list so the community gains from the discussion and have him > make comments/submissions on the tracker ticket for this issue. > Thanks! > > - - Keith > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.5 (Darwin) > > iD8DBQFMyZZmW5AQrvjB4mcRAuEUAJ9Fj7Ti+YuvfvXNzFd9uMAqN7x8OQCeNfUG > a1TeCo9z20D+hjHx5868oYE= > =7VFU > -----END PGP SIGNATURE-----

10 years, 10 months

Re: [Bro-Dev] Local MPLS coding support

by Seth Hall

Hi Keith! I left the full message you sent me for the benefit of everyone on the bro-dev mailing list. I just pushed a branch to our git repository that has more complete support for MPLS. I'd appreciate it if you could try it. I added some functionality to it that the patch in the tracker doesn't have. * Supports MPLS over Ethernet (which is likely what you're sniffing). * Builds the appropriate BPF filter so that MPLS encapsulated and non-encapsulated traffic is sent through. You just need to make sure and load the mpls.bro script. It *should* just work from that point (no filters to add or anything). If you define your own filter on the command line (with the -f flag), then the auto mpls support will not work. You will have to define the appropriate mpls filter in that case. If this works for you, then you can have your intern work on something else. :P Let us know if you have staff time needing projects to work on, I'm sure we can figure something out. git clone git://git.icir.org/bro git checkout origin/topic/seth/mpls (ignore all of the warnings) <do the normal build and install stuff> .Seth On Oct 28, 2010, at 11:27 AM, Keith Lehigh wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Hi Seth, > Hope all is going well for you with getting your head wrapped around > plans for the future of Bro. > We may have some hours available from an intern here to put to work > on solving our MPLS problem with Bro. This is one of a couple different > projects I have in mind, but before I get too far into considering this, > I wanted to get a little input from you. > I would aim to have this support added in such a way that it is > generic to any flavor of encapsulation and could be contributed back as > patches. Given that this person won't be working for us long term, I'm > not interested in having him make some local hacks which quickly > become unmaintainable. As such, do you think the internals of Bro will > change enough in the near future to make this irrelevant? > This question may be more directed at Robin. Am I being overly > naive in the amount of effort and rewriting that would need to be done > to make this work? I'm pretty sure the intern will need to get some > time in just getting up to speed on Bro, but I think he doesn't have to > be a subject matter expert on Bro. Again, am I being naive? > If we go with this project, I'll direct our intern to put questions > to the Bro list so the community gains from the discussion and have him > make comments/submissions on the tracker ticket for this issue. > Thanks! > > - - Keith > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.5 (Darwin) > > iD8DBQFMyZZmW5AQrvjB4mcRAuEUAJ9Fj7Ti+YuvfvXNzFd9uMAqN7x8OQCeNfUG > a1TeCo9z20D+hjHx5868oYE= > =7VFU > -----END PGP SIGNATURE-----

10 years, 10 months

[Bro-Dev] Opinions on trace rewriter?

by Robin Sommer

What are the opinions on completely removing the trace rewriter from the Bro code base? I think I vaguely remember us discussing this, with a conclusion that removing is the way to go, but I wanted to check. Pros of removing: - Rewriter is broken right now anyway - Not used by many people (just the "occasional researcher" :-) - Nobody knows the internals well. - It's quite intrusive to the code, and removing it will get rid of quite a bit of stuff sprinkled across the source. Cons: - It's pretty cool functionality and nobody else has it. I really like the rewriting but I'm thinking it would be better done in an external tool than inside Bro itself. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin(a)icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org

10 years, 11 months

Re: [Bro-Dev] DataSeries for Bro

by Martin Arlitt

hi Robin the technical report is now available at: http://www.hpl.hp.com/techreports/2010/HPL-2010-164.html The work has more focus on Apache than Bro, primarily because I couldn't get Sergey access to Bro on a production network. However, he did integrate DataSeries with Bro and ran some tests. I think his work does show that DataSeries has clear benefits for log collection and analysis with these types of applications. There is at least one thing we would do differently if we started over again, and that is to use an in-memory buffer for log entries before writing an extent to disk. Sergey used a temporary file because he was concerned about messing up Apache's memory management, and then followed the same approach when he added DataSeries logging to Bro. Obviously for those familiar with the Bro source, this shouldn't be an issue. if you or anyone else has questions about this, please let me know. thanks Martin

10 years, 11 months

Re: [Bro-Dev] Cmake for Bro

by Robin Sommer

(Moving this to bro-dev, let's keep dev discussions there). On Mon, Oct 18, 2010 at 11:13 -0500, you wrote: > I can see that happening. I'll plan for that to happen unless we > find some other CMake projects that are doing something more > brilliant. Ok, I think this is my favorite option so far, it will also solve the --help problem. > 1) --enable-shippedpcap, "use the shipped version of libpcap" Please get rid of that, we don't want to ship a pcap anymore at all. > 2) bison vs. byacc I think the main reason for supporting both was that some systems used to come with the former, and others with the latter. I'm not sure how that's today, but I'd be fine supporting just bison. Vern, what do you think? Just looked at the other (Bro-specific) configure options, below are my thoughts. Anybody, please check whether you agree (active mapping and DFA cache expiration I'm most unsure about). Robin --------- cut ------------------------------------------------------- --enable-brov6 enable IPV6 processing -> Keep; we should eventually turn this always on and remove the configure option, but we aren't quire there yet. --enable-int64 enable use of int64 (long long) for integers -> Remove; let's always turns this on and remove the configure option. --enable-activemapping enable active mapping processing -> Do we want to keep active mapping in the code? Is anybody using that? --enable-expire-dfa-states enable DFA state expiration -> Do we want to keep this in code? Nobody is probably using it, and it complicates the DFA code quite a bit. --enable-debug no compiler optimizations -> Keep. --disable-select-loop disable select-based main loop -> Remove; no longer needed. --enable-perftools use Google's perftools -> Keep. --enable-shippedpcap use the shipped version of libpcap -> Remove. --disable-nbdns Disable non-blocking DNS support -> Remove; don't think anybody uses this. --disable-largefile omit support for large files -> Remove. --disable-broccoli Do not build/package Broccoli -> Remove; broccoli will be its own package. --disable-broctl Do not build/package broctl framework -> Remove; BroControl will be its own package. --enable-cluster Configure broctl for cluster usage -> Remove; that's an option for BroControl. --with-openssl=PATH path to OpenSSL (needed for SSL analyzer and secure communication) -> Keep, but I'm wondering whether we should make OpenSSL a mandatory dependency to get rid of all the "#if openssl" conditions. --with-perl=PATH path/name of the Perl interpreter -> Remove; I don't think we're using Perl anywhere anymore. --with-dag=PATH path to the DAG library (for native support for Endace Tech.'s DAG monitoring cards) -> Remove; nobody has been using the native DAG support in ages. --with-binpac=PATH path to a binpac executable for compiling analyzer code -> Keep. -- Robin Sommer * Phone +1 (510) 722-6541 * robin(a)icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org

10 years, 11 months

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

zeek-dev October 2010