Hi all,
Recently I have some problems with Bro and PF_RING in cluster.
On my server, when I have less than 32 worker threads(rings),
everything is okay, but when I use worker threads more than 32, pf_ring
start to receive repeating data packets. For example, rings less than 32, I
send 400000 packets to server and pf_ring info in /proc shows there is
400000 packets in rings, but when rings greater than 32, I can get 800000
packets when 33 rings and 1200000 packets when 34 rings and so on.
I guess if there is some rules that a pf_ring or a bro cluster can only
support less than 32 rings or worker threads on a server or some other
reasons?
Any insight would be helpful.
Hi all,
I'd like to ask guidance on how to contribute to BRO by proposing
extensions to existing protocol analyzers.
For instance, suppose that I realize a patch to the DHCP analyzer that
includes new unsupported options. Such patch would impact on multiple
files like those in src/analyzer/protocol/dhcp,
scripts/base/protocols/dhcp as well as new types to be included in
init-bare.bro.
What would be the best procedure (and format) to submit such a patch?
best,
Valerio
Is there a decoder for Netflow, such that one could use bro to collect and
log Netflow packets seen by a hardware tap, from multiple sources, in a
similar fashion to how Bro handles syslog?
Hi All,
My Bro program shows a wired behavior. We leverage the signature framework
to capture embedded components in HTTP replies (http-reply-body) as well as
the file download (tcp payload). However, we lose many events associated
with the signature (only around 1/3 shown).
The exactly same program actually runs well on another desktop (capturing
all signature matching we issued). I would be appreciate if anyone can have
a clue on the problem.
The machine running bro is fanless computer with Intel Atom and Ubuntu
16.04. It is almost dedicated to the Bro monitoring so it shouldn't be
performance issue.
The signature matching is quite straightforward: we define some simple
signature patterns, load those signatures to BroControl, and pull some
fields from corresponding log files via a broccoli python client.
We do capture some signature matching events, but also lose many that
should be captured. Those events are not shown in signatures.log; it means
that they are either failure of capturing or dropped by Bro Control, rather
than the problem of python client.
BTW, we use File Analysis to capture the file downloads, it works well as
expected.
Thanks very much for any comments~
Cheers,
Hi!
Anyone care to share bro + pfring success story?
What's the speed, what NIC, what's the configuration.
I'm running bro 2.5.1 built with jemalloc and gperftools and against
pf_ring 6.6.0 with ixgbe_zc on CentOS 7.2.
In ZeroCopy mode with zbalance_ipc dividing NIC to 20 application rings (-n
20) I'm getting each CPU core loaded at 100% and around 50% packet drop
(reported by netstats in broctl).
When redirecting from zc to 20 dummy interfaces (zbalance_ipc -r 0:dummy0
and so on) I'm getting around 50% load on each core and a lot less of
packet drop (10% - 15%).
This is with traffic around 700 - 800 Mbit/s
All input will be highly appreciated.
Best regards
Rado
Hello!
In contrast to the normal use case I run Bro mostly from pcaps. When
huge amounts of data (~20 TB) have to be processed, bro in standalone
mode becomes a real bottleneck. So I thought about using the bro cluster
mode.
In the past I thought, the bro workers would communicate with each
other, so when for example one worker sees upstream and the other
downstream, they would combine the information to one log. Seth told me
at BroCon, that Bro needs to be fed complete streams. To do this some
kind of load balancer is needed in front of bro.
When I need to split the flows with a load balancer anyway, is there any
advantage of running bro in cluster mode at all? I do not need any
shared data like tables. Are there any parsers which combine the
information seen by different workers in different flows?
If cluster mode has no added value in my case, I could just load
balance my pcaps to independent bro instances which would make my setup
much easier.
Have a nice weekend!
Franky
The BroCon 2017 list of approved videos have been posted to our YouTube channel. You can view the playlist here:
https://www.youtube.com/watch?v=b6lrNaKRgmA&list=PL2EYTX8UVCMjx7DdqfPEOjCmi…
BroCon slides (and links to videos) are available on our agenda page:
https://www.bro.org/community/brocon2017.html#agenda
If you do not see a link to a slideshow or video it has not been approved for release.
Thanks again to all the speakers, attendees, and sponsors who made this another successful BroCon.
We hope to see you again, either in the audience or on stage, next year.
------
Jeannette Dopheide
Sr. Education, Outreach, and Training Coordinator
National Center for Supercomputing Applications
University of Illinois at Urbana-Champaign
I have been experiencing hash misses so to speak with PE files due to
lack of seen_bytes verse total_bytes. Is this indication of a
performance problem which the sensor is overwhelmed therefore cannot
parse the entire file?
e.g. I have a file that's 300832 in which seen_bytes consistently
matches total_bytes and then a hash is provided. Another file is
774200 total_bytes but the seen_bytes usually does not amount to the
total_bytes (sometimes it does).
I am running Bro on Ubuntu LTS 16.04 and I just upgraded the kernel from
linux-image-4.4.0-62-generic_4.4.0-62 to
linux-image-4.4.0-62-generic_4.4.0-96 along with the headers (I first
stopped bro before upgrading via broctl).
After I rebooted, I went into broctl and tried to re-start bro, but if
failed. It said it couldn’t find /run-bro. So after checking to make sure
all the files were where they were supposed to be, I thought that maybe the
new image install had overwritten something. So I went back into broctl
and ran the install command. Then I tried to start bro again. This time
when it failed to start, the diag said it couldn’t find the interface of
eth0. What was weird since I knew a/ didn’t have an interface with that
name and b/ had already configured bro to use the correctly named
interface. After some poking around, I realized that a new node.cfg had
been created at /etc/bro along with new broccoli.conf, broctl.cfg and
networks.cfg (well, I don’t know if they are new since they are dated back
to 2015). These are different than the originally installed and correct
files located at /usr/local/bro/etc.
How do I get broctl to point back to the correct cfg files?
Thanks.
Craig
Note: please excuse any delays in response as I currently don't have
constant access to Gmail
Howdy,
This is just a heads-up for anyone using the myricom plugin that was
shipped with Bro who is getting ready to upgrade and switch to the
bro-pkg'd version. This is an upcoming change that Seth mentioned at
BroCon.
Long story short, if you're going to do that, wait just a little bit or let
me know and I'll help with some details. It caused our cluster to be down
for several hours while we figured it out and I'd hate for others to have
to debug it again.
We think we've identified a bug in one of the core distributed cmake files
that causes plugins to create a symlink in the installed build environment
back to the .bro-pkg directory. The result of this is when you push Bro to
the cluster w/ 'broctl deploy', the plugin breaks on the workers. We
haven't noticed the issue with anything other than the myricom plugin, but
I believe it could impact others.
(Shout out to Sam Oehlert @ ESnet that figured out the make file issue was
in the Bro core and not part of the plugin itself!)
-Dop
