Dear all
I have server with Ubuntu 8.10 operating system and Bro Ids 1.4, and there are two network interfaces installed in this server eth0 and eth1.
The interface eth0 connected to the internet, and the interface eth1 connected to my special local area network.
How can I make Bro IDS monitor and analyze all the traffic come from eth0? And after that pass the traffic to eth1?
Regards,
Kaled Hussain
M.Sc. IT Student
UUM
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hi guys,
I was doing some testing on bropipe for our migration to 1.4, and
came across a bug that resulted segfaults on malformed input. I've
patched it and am including it. There's some random formatting changes
that you can delete from the diff if you like - the important changes
are around line 463. There's some changes around line 530 that look
like they are legit, but don't seem to be in the trunk, it seems like
they somehow escaped submission earlier.
The patch is an svn diff, let me know if I should submit it
differently.
Steve
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.9 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iEYEARECAAYFAknCj/UACgkQcVd2YI1BWAjGZACggkPbsKKrav9IEiobH6UOXn2t
PDYAoJCU8b6vBR1yiUage8rP5zdYugsp
=BSV/
-----END PGP SIGNATURE-----
Index: bropipe.cc
===================================================================
--- bropipe.cc (revision 6644)
+++ bropipe.cc (working copy)
@@ -185,10 +185,9 @@
int num_tokens = 0;
char delim = '\0';
- for ( unsigned int i = 0; i < str.length(); ++i )
- {
+ for ( unsigned int i = 0; i < str.length(); ++i ) {
while ( isspace(str[i]) )
- ++i;
+ ++i;
string next_arg;
@@ -463,8 +462,11 @@
tokens.clear(); // make sure that the vector is clear
tokenize(inText[j].c_str(), tokens);
-
- if ( (ev = bro_event_new(tokens[0].c_str())) )
+ // if this line didn't tokenize to anything, skip the rest of the block
+ if ( tokens.size() == 0)
+ continue;
+
+ if ( ev = bro_event_new(tokens[0].c_str()) )
{
for ( unsigned int i = 1; i < tokens.size(); ++i )
{
@@ -530,10 +532,10 @@
{
int bbool=0;
- if ( tkn_data == "T" ||
- tkn_data == "TRUE" ||
- tkn_data == "1" )
- bbool = 1;
+ if ( tkn_data.c_str() == "T" ||
+ tkn_data.c_str() == "TRUE" ||
+ tkn_data.c_str() == "1" )
+ bbool = 1;
bro_event_add_val(ev, BRO_TYPE_BOOL, NULL, &bbool);
}
Dear all
Thank for the answers for the my last post.
There are many pre-processors available in Snort like:
Flow
sfPortscan
Stream4
Frag2
Telenet_negotiation
Http_inspect
My question is what are the corresponding for these pre-processors in Bro IDS? And how can I make every one on or off?
Thanks for all
regards
Kaled
Hello Everybody,
I did a (simple) perl script in order to use Bro with "Picviz", this tool is a parallel coordinates
plotter which helps to visualize your data using parallel coordinates plots
(http://en.wikipedia.org/wiki/Parallel_coordinates).
Here are the parameters i plot : time, src_ip, src_port, src_bytes,
dst_ip, dst_port, receive_bytes, duration, state.
The states are the same Bro is reporting about :
"S0,S1,SF,REJ,S2,S3,RSTO,RSTR,RSTOS0,RSTRH,SH,SHR,OTH", I plot them using 3 groups of colors :
"S0,S1,S2,S3, REJ,RSTO,RSTR" as "blue"
"SF" as "green"
"RSTOS0,RSTRH,SH,SHR,OTH" as "red"
}
Using parallel coordinates plots is a very useful way to see what's
happening so happy "Picvizing" :-)
You'll find the files i use at http://www.rootshell.be/~jpli/picviz
and "Picviz" is available at http://www.wallinfire.net/picviz
With regards,
Jean-Philippe.
Robin Sommer wrote:
> On Mon, Mar 02, 2009 at 10:25 -0600, dalbrech(a)illinois.edu wrote:
>
>> real system. So I built a simple tool I call "dns-packet-gen", which
generates
>> pathological DNS packets constructed for their difficulty in parsing. The
tool
>> emits pcap traces containing DNS traffic with many records (50+), all of
which
>> feature deeply-recursive DNS names.
>
> Neat. Is that tool available?
Making it available is on my todo, but it's toward the bottom. My group
currently doesn't have a good web presence, but we're planning to fix that one
of these days as I think it does a lot for a research group's visibility.
>> I'm wondering how you plan to address cache behavior in your multi-
threaded
>> implementation of Bro. Obviously, this is a really hard problem, and I'm not
>> sure how well-studied it is in the literature, especially in the context of IDS.
>
> That's an excellent question and I don't think it has been studied
> in the IDS context yet. Our main strategy is to keep related state
> together, e.g., by running all analysis involving the same
> connection on the same core. It's hard to predict though how cache
> effects will eventually turn out to hurt us, and we expect to do a
> number of measurements in this regard once the multi-threaded Bro is
> in a shape that we can actually measure something. Going from there,
> I'm sure there'll be lots of further optimization potential.
I'm going to start analyzing Bro's memory use and performance; do you have
any recommendations about which policy I use? I'm looking for a reasonably
"representative" workload (if such a thing even exists), something that will test
a lot of code paths through the system.
Thanks again,
DA
--
David R. Albrecht
Graduate Assistant, Hatswitch Group
U. Illinois Urbana-Champaign
(312) 445-0883
Hello.
We're in the process of upgrading our IDS infrastructure here, and I've
been trying for the last week or so to get the "release" version of bro
up and running. It refuses and then it shames me. First, I'll detail
what I've tried, and then I'll tell you where I'm stuck.
I downloaded the "release" version of 1.4, thinking it would be a breeze
to install like 1.3 was.
I attempted to compile and install it, only to find out that the part
that actually allows it to install and run (bro-lite) was not only
deprecated, but was helpfully disabled as shipped in order to prevent me
from blundering into an unsupportable situation.
I read a huge chunk of the mailing list archives and determined that in
order to use the "release" version of bro, I would have to install a
bleeding-edge clustering component, as a test of my mettle.
I followed the instructions at
http://blog.ncsa.uiuc.edu/aashish/2008/10/21/moving-to-bro-14/ and
compiled the clustering component, running it in standalone mode.
When I try running the clustering component, it complains that it
doesn't have the analysis-groups.bro component, which appears to be part
of some changes made to the policy files but only made available to some
inner cabal of bro developers.
Not to be thwarted, I used Google to try to find out about the file, and
found a hidden copy in the web interface of the SVN repository. Naively
thinking this would solve my problem, I installed it
in /usr/local/bro/current/policy/local/ and was finally able to get bro
to start without instantly dying.
That brings me to right now, where I'm stuck. Bro will run for a few
minutes, generating the usual mass of data before suddenly deciding to
segfault and die. As best I can tell, it's dying in
DNS_Mgr::Process() . I'm guessing that's not normal behavior, or
someone else would probably have emailed about it.
Any ideas on how to get a working install of bro 1.4?
Thanks for your help, and for writing it in the first place,
-Lou
(hoping for more to put on his 2008-2009 performance evaluation than
"heroically spent FY08-09 compiling bro over and over again.")
Robin Sommer wrote:
> On Wed, Feb 25, 2009 at 14:26 -0600, dalbrech(a)illinois.edu wrote:
>
>> I've been doing some performance profiling on Bro.
>
> Interesting! Do you have any results you could share?
I saw Vern's talk a while back at UIUC about how Bro works. As I recall, there
were three major components:
1. A packet capture engine, based on libpcap, which makes use of BPF to
selectively discard packets as early as practical (to reduce the overall quantity of
traffic),
2. Protocol parsers, which turn blocks of data from the network into
semantically-meaningful PDU-like data structures, and
3. An event engine, which applies constraints to the lexed protocol data, and
possibly raises alarms if the traffic is suspicious or otherwise "interesting".
My group initially decided to pursue protocol parsing, because our micro-
benchmarks (many of which appeared in our RAID '08 paper) showed that
protocol parser-generators (e.g. binpac and Nikita et al.'s GAPA) leave
significant performance on the table versus a hand-coded parser. In particular,
binpac's use of C++ objects to represent PDUs really suffers on protocols
which feature small, deeply-nested messages, like DNS.
However, the ultimate question is how much the performance gap between a
hand-coded parser vs. a machine-generated one matters in the context of a
real system. So I built a simple tool I call "dns-packet-gen", which generates
pathological DNS packets constructed for their difficulty in parsing. The tool
emits pcap traces containing DNS traffic with many records (50+), all of which
feature deeply-recursive DNS names. For the record, Wireshark routinely took
an hour or more to load traces of 1M of these packets.
Based on my understanding of the Bro code, it looks like parsing "normal" DNS
traffic (real DNS scraped from my own web browsing) only accounts for 4-5%
of all CPU time (I used Shark on OS X with a debug build, and checked the
sampled CPU time of all descendants of UDP_Analyzer::DeliverPacket()). I'm
using a version of the "detect backdoors" script which includes DPD, which
might have skewed the results a little. (If anyone has comments on this, I'd
appreciate hearing them). In either case, my bad DNS traffic only pushed the
binpac-based analyzers up to 13-14% of CPU time, which is still comparatively
tiny in relation to the EventMgr's CPU time.
What I take away from this is that, even if binpac is slow, it's not the piece that
most demands attention. So I started thinking about the systems-level issues
(the Shunt paper took this same direction), and realized the main event loop is
single-threaded. And that got me to where I am today :-)
>
>> I'm wondering what the ICSI folks' position is on threads vs. clustering.
>
> In short: we consider them orthogonal and are pursuing both. :-)
>
> To elaborate a bit: The cluster is the approach that can provide
> significantly increased performance right now; ignoring a few
> implementation glitches we still need to sort out, it's working and
> ready to use[1]. The cluster is also an approach that will work
> long-term: there will always be limits to what a single box can do
> (multi-core or not) and the cluster offers the capability to go
> beyond that.
You're right about scale, modulo communications issues. However, I think it's
important for a large-scale system to be designed in such a way as to gain
more performance as the architecture guys throw faster and faster chips over
the wall. The way I read the tea leaves, the number of CPU cores in commodity
PCs isn't dropping anytime soon, so software that isn't written with multi-
threaded event loops is going to hit a performance wall even as CPUs get
faster.
>
> That said, we are in parallel (no pun intended :) working on a
> multi-threaded Bro implementation. While we have already made quite
> a bit of progress on that, that will however still take a bit to get
> into any kind of usable state; we need to restructure Bro internally
> quite a bit to exploit its concurrency potential. Once finished
> though, we expect Bro's performance to scale pretty well with the
> number of available cores (not considering other aspects of the
> hardware which might turn into bottlenecks at some point). The
> Sarnoff paper has some of the basic ideas for the multi-threaded
> Bro, and there'll be an extended journal version of that coming out
> soon with more details on the parallel execution model (let me know
> if you'd to like to get a draft).
>
> Does that answer your question? If you have any particular thoughts
> on either clustering or the multi-threaded Bro, I'd be happy to hear
> them.
I haven't explored how threadable Bro would be, but it seems you're already on
it. Truthfully, I spent a few weeks picking through the code to convince myself
I knew roughly how to profile it; I'm no expert on Bro internals. I'm going to
have a chat with Nikita soon (my academic adviser) to see what he thinks about
all this, and I'll be in touch. Please send the draft of the Sarnoff paper to my
UIUC email address; I'd love to have a look.
I'm wondering how you plan to address cache behavior in your multi-threaded
implementation of Bro. Obviously, this is a really hard problem, and I'm not
sure how well-studied it is in the literature, especially in the context of IDS.
Fortunately, I have a lot of friends here that do high-performance computing
work (their long-range research agenda includes a 3000-core CPU tapeout), so
I'll talk to them about it.
David