John,
Thanks for the quick response.
> Not sure what Netscalar does, but it all should act the same. The host
> TCP stack would drop any attempted connection for which a session was
> not established regardless of what was upstream from it. Quick and
> dirty, you sould be able to fire up tcpdump and see the session
> initialization.
That's what I'm finding strange. After running a tcpdump capture on
the interface and analyzing it with Wireshark, I do not see any 3-way
handshakes for this particular web application. For any HTTP GET that
I see in Wireshark that pertains to this application, when I "Follow
TCP Stream", the first entry in Wireshark is always the GET message
itself. For all other applications on the network, doing the above
results in the first entry being the SYN.
I've generated a few dumps with the same results. I wonder if the
load balancer is somehow keeping a session active for very long
periods (if this even makes sense).
If you have any suggestions or thoughts, I'd be very interested.
Thanks,
Bill
On Sat, Feb 6, 2010 at 12:51 PM, John Hally <JHally(a)ebscohost.com> wrote:
> Hi Bill,
>
> I've run BRO in the past with load balancers (Arrowpoint/Cisco CSS) and
> was able to see all traffic. In our setup we had 2 segments; a VIP
> access link and a services trunk link where the real/origin servers
> lived. Both of these links had physical network taps and it was as
> simple as plugging in the Ethernet, flipping the interface to
> UP/PROMISC, and starting BRO.
>
> With the CSS, even though the unit would handle the initial connection,
> it would 'snap' that over to the origin server it picked during load
> balancing so you would still see the tcp setup.
>
> Not sure what Netscalar does, but it all should act the same. The host
> TCP stack would drop any attempted connection for which a session was
> not established regardless of what was upstream from it. Quick and
> dirty, you sould be able to fire up tcpdump and see the session
> initialization.
>
> Thoughts?
>
> Tahnks.
>
> John.
>
> -----Original Message-----
> From: bro-bounces(a)ICSI.Berkeley.EDU
> [mailto:bro-bounces@ICSI.Berkeley.EDU] On Behalf Of Bill Jones
> Sent: Saturday, February 06, 2010 10:22 AM
> To: bro(a)ICSI.Berkeley.EDU
> Subject: [Bro] Load Balancers
>
> Hi everyone,
>
> I was curious if anyone has any experience running bro between
> load-balancers (such as Netscaler) and web applications. We are
> currently trying to get HTTP logs generated for a web application. We
> couldn't figure out why bro was not triggering the HTTP analyzer, but
> I now believe that this is because it is never seeing the original SYN
> + SYN/ACK for the conversation. When viewing the conversations in
> Wireshark, I can see that all the TCP streams for this particular
> application begin with the GET and do not include the initial 3-way
> handshake.
>
> Here is an entry in the conn.log for this stream which shows the states:
>
> 1265389087.849048 ? 10.19.120.12 10.19.2.78 http 2232 80 tcp 14785
> 604140 OTH X DdAa
>
> Other web applications on the wire, which do have the 3-way handshake
> visible for all connections, seem to work just fine and I get http
> logs.
>
> My questions are:
>
> Am I correct in assuming that the lack of initial connection
> establishment is why the HTTP analysis is never occurring (and
> therefore I'm not getting entries in http.log)?
>
> Is there a way to force bro to analyze the traffic even though there
> is no proper 3-way handshake visible?
>
>
> Thanks for your time,
> Bill
> _______________________________________________
> Bro mailing list
> bro(a)bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
Hi everyone,
I am puzzled about the outcomes of using ipsumdump or BRO for processing
multiple pcap files.
I am using BRO to analyze anomalities in my 12 hours captured network
traffic which was saved in 4 Gb pcap files. I want that BRO consider the
cases when a connection may have been split in two or more files. I was
using ipsumdump to solve this, but I found that some files have errors and
cause ipsumdump to crush with this message:
ToDump(bigPcap1.pcap): Inappropriate ioctl for device
Using the capinfo tool I detected that some of my files have packet size
larger than normal (65535), so using tshark I cut the part of the file with
problems. For example:
capinfos: An error occurred after reading 3830659 packets from
"trace2.pcap": File contains a record that's not valid.
(pcap: File has 4065648712-byte packet, bigger than maximum of 65535)
So I create a reduced version of trace2.pcap with tshark:
/usr/sbin/tshark -c 3830659 -r trace2.pcap -w trace2-new.pcap
This solution seemed to work fine, all the ***-new.pcap have no errors while
reading with capinfo or wireshark, but even so there are some that still
cause problems for processing. For example:
I processed the following files in 3 different ways:
trace1.pcap, trace2-new.pcap, trace3.pcap (trace2.pcap was replaced because
of the packet size error)
FIRST TRY - using ipsumdump with collate option:
ipsumdump --collate -w - trace* |bro -r - brolite myenvironment -f "tcp or
udp or icmp" dpd_conn_logs=T dpd detect_protocols dyn_disable irc-bot proxy
ftp
Output> 9.7 MB conn.log with 114861 lines (number of connections)
SECOND TRY - using ipsumpdump without collate option
ipsumdump --collate -w - trace* |bro -r - brolite myenvironment -f "tcp or
udp or icmp" dpd_conn_logs=T dpd detect_protocols dyn_disable irc-bot proxy
ftp
Output:
19 Mbytes conn.log with 228922 lines with 950 repeated connections
THIRD TRY - without ipsumdump:
/usr/local/bro/bin/bro -r trace1.pcap -r trace2-new.pcap -r trace3.pcap
brolite todai -f "tcp or udp or icmp" dpd_conn_logs=T dpd detect-protocols
dyn-disable irc-bot proxy ftp 2>bro-error3.log
Output:
15 Mbytes conn.log with 169168 lines, connections are not repeated
COMMENTS:
pcap files has not overlap traffic (it was checked with trace-summary using
first packet seen and last packet seen).
I tried the ipsumdump with both collate and no collate option because when I
used ipsumdump only (without bro), with collate option the resulted larger
pcap file was a 7.9 GB file but without collate option the resulted file was
12.GB (trace1.pcap: 4 MB, trace2-new.pcap: 3.9GB, trace3.pcap: 4GB).
Besides, while using ipsumpdump --collate alone, the progress bar showed
something like this:
66%****************** |8017MB ETAToDump(LargerTrace.pcap):
Success
100%****************************|12113MB
But the progress bar for ipsumdump without the collate option didn't split
and reach the 100% 12113MB.
If anyone can illuminate this matter, it will be a great help.
Veronica
Hi,
I have few network traces to analyze [for traffic classification] which
have vlan headers for certain subnets, and I'm not able to analyze them
using bro. I can use "@load vlan" filter but then I'm only able to read
vlan traffic, and makes Bro skip on normal [non-vlan] traffic.
In my setup, some subnets in my traces are on vlan and further, these
subnets have only inbound traffic on vlan while outbound traffic seems
to be direct [no vlan header in outbound packets]. Due to above
mentioned issue I have to pass each trace twice using different filters
and I'm getting two uni-directional flows for each bidirectional flow.
I searched Bro mailing list and from the previous posts, I feel that Bro
does not support reading vlan and non-vlan traffic concurrently. Is this
assumption correct or there is some way/hack to actually analyze them at
the same time?
Also since I'm using Bro for offline traces, does anyone know a way to
somehow modify the trace file to *fix* vlan traffic and change it to the
normal traffic.
Thanks for the help :)
-Faisal
I am a new user of Bro. I've installed ver 1.5.1 and I can run just fine with a single interface (whichever one is specified in node.cfg) but I can't seem to get other capture interfaces running. I am set up with 4 ethernet interfaces, three of which are taps to different locations within my network and one to the local subnet where the server is located.
What additional information can I provide that might help identify the issue?
Alan Meeks
Information Security Analyst
Angelo State University
www.angelo.edu
325-942-2333 phone
325-942-2109 fax
I am trying to get Time Machine (tm-20090206) running with broccoli to support both interaction with Bro-IDS 1.5.1 and the command line tm-query interface within Time Machine. Time Machine configured, compiles and runs ok (including capturing packets and allowing queries on localhost 42042/tcp), however when I try to connect (anything) to the bro listener, the tm process panics and immediately abends (nothing useful when I run strace against it during the crash).
The only related messages in the tm.log are:
1276102721.251917 broccoli-listen: listening for incoming connections on port 47757...
1276102721.251938 broccoli-init: listen_thread started [a5dc8b90]
1276102731.190267 broccoli-listen: accepted connection
1276102731.190405 broccoli-listen: started Broccoli worker [a4dc3b90]
1276102731.190435 broccoli-worker: running Broccoli worker [a4dc3b90]
Where the broccoli-worker message appears immediately apon connection (via telnet localhost 47757), and the tm process immediately abends.
I'm running on Linux (Ubuntu 9.04 32 bit 2.6.28-11-server) and all the code was compiled with gcc-4.3.3 and g++-4.3.3. The firewall (iptables) is also the Bro-IDS and TM system, and it's monitoring an attacker (backtrack 4.0) running against a web application on an apache 2.2.12 server (all on a MacBook Pro OS X 10.6.3 with VMware Fusion 3.0.1).
Any ideas where I should start looking for the cause of the abends?
Thanks,
AP
Sam,
Here is my node.cfg below:
# $Id: node.cfg.standalone.in 6811 2009-07-06 20:41:10Z robin $
#
# Node configuration for a non-cluster, single-instance setup.
#
[bro]
type=standalone
host=localhost
interface=eth0
-----Original Message-----
From: Sam Oehlert [mailto:soehlert@ncsa.uiuc.edu]
Sent: Thursday, June 10, 2010 3:49 PM
To: Alan J. Meeks
Subject: Re: [Bro] Multiple Capture Interfaces
I see, I didn't think you installed broctl. I wonder if that is why the command hangs. What does your node.cfg look like?
Sam
----- Original Message -----
From: "Alan J. Meeks" <alan.meeks(a)angelo.edu>
To: "Sam Oehlert" <soehlert(a)ncsa.uiuc.edu>
Sent: Thursday, June 10, 2010 3:46:54 PM
Subject: RE: [Bro] Multiple Capture Interfaces
Well, you see that's interesting. I've been starting Bro through
./broctl. When I try to run Bro -i eth0 -i eth1 and so on, the command
hangs and bro does not start.
-----Original Message-----
From: Sam Oehlert [mailto:soehlert@ncsa.uiuc.edu]
Sent: Thursday, June 10, 2010 3:38 PM
To: Alan J. Meeks
Subject: Re: [Bro] Multiple Capture Interfaces
I'm sure there is a better way to do this long term, but if you use the
i flag when calling bro, you can use multiple interfaces. Look at the i
flag here: http://www.bro-ids.org/Bro-reference-manual/Flags.html
Sam
----- Original Message -----
From: "Alan J. Meeks" <alan.meeks(a)angelo.edu>
To: "Sam Oehlert" <soehlert(a)ncsa.uiuc.edu>
Sent: Thursday, June 10, 2010 3:32:30 PM
Subject: RE: [Bro] Multiple Capture Interfaces
The server has 4 ethernet interfaces in it that I have configured 3
connected to taps and one to a regular switchport/local subnet. My Bro
installation can currently only capture from one at a time, whichever
one I have specified in /user/local/bro/etc/node.cfg in the interface
entry.
I am hoping to be able to specify more than one capture interface at a
time.
-----Original Message-----
From: Sam Oehlert [mailto:soehlert@ncsa.uiuc.edu]
Sent: Thursday, June 10, 2010 3:30 PM
To: Alan J. Meeks
Subject: Re: [Bro] Multiple Capture Interfaces
I'm sorry, I am not understanding which four interfaces you are talking
about.
----- Original Message -----
From: "Alan J. Meeks" <alan.meeks(a)angelo.edu>
To: "Sam Oehlert" <soehlert(a)ncsa.uiuc.edu>
Sent: Thursday, June 10, 2010 3:27:32 PM
Subject: RE: [Bro] Multiple Capture Interfaces
Just trying to get the 4 interfaces recognized as capture interfaces.
-----Original Message-----
From: Sam Oehlert [mailto:soehlert@ncsa.uiuc.edu]
Sent: Thursday, June 10, 2010 3:27 PM
To: Alan J. Meeks
Subject: Re: [Bro] Multiple Capture Interfaces
Alan,
That does answer the question I had. I just realized I may have misread
your email, however. Are you attempting to use a cluster setup, or are
you just wanting to use multiple NICs or what? Also, CentOS 5.5 is what
I used for this too, so I have some experience getting it up and running
on there.
Sam
----- Original Message -----
From: "Alan J. Meeks" <alan.meeks(a)angelo.edu>
To: "Sam Oehlert" <soehlert(a)ncsa.uiuc.edu>
Cc: bro(a)ICSI.Berkeley.EDU
Sent: Thursday, June 10, 2010 3:24:28 PM
Subject: RE: [Bro] Multiple Capture Interfaces
Sam,
I believe I installed in a non-clustered or standalone mode. Ran the
./configure, make and make install after ensuring I got all the prereqs
on the server.
I also forgot to mention I was not in the position where I could pick
and choose my hardware and could not get BSD installed. I had to fall
back to CentOS 5.5.
Not sure if that answers your question.
-----Original Message-----
From: Sam Oehlert [mailto:soehlert@ncsa.uiuc.edu]
Sent: Thursday, June 10, 2010 3:20 PM
To: Alan J. Meeks
Cc: bro(a)ICSI.Berkeley.EDU
Subject: Re: [Bro] Multiple Capture Interfaces
What installation steps did you follow? I had problems getting a cluster
up and running because I was installing it incorrectly at first.
Sam
----- Original Message -----
From: "Alan J. Meeks" <alan.meeks(a)angelo.edu>
To: "bro(a)ICSI.Berkeley.EDU" <bro(a)ICSI.Berkeley.EDU>
Sent: Thursday, June 10, 2010 3:17:12 PM
Subject: [Bro] Multiple Capture Interfaces
I am a new user of Bro. I’ve installed ver 1.5.1 and I can run just fine
with a single interface (whichever one is specified in node.cfg) but I
can’t seem to get other capture interfaces running. I am set up with 4
ethernet interfaces, three of which are taps to different locations
within my network and one to the local subnet where the server is
located.
What additional information can I provide that might help identify the
issue?
Alan Meeks
Information Security Analyst
Angelo State University
www.angelo.edu
325-942-2333 phone
325-942-2109 fax
_______________________________________________ Bro mailing list
bro(a)bro-ids.org
http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
I am attempting to figure out how to get a Bro Cluster up and running, but all documentation I see is outdated. I am trying to use the latest SVN (though if you know how to do it with 1.5, I'll figure out the differences between the two), and I am having problems.
First of all, I have the manager and proxy nodes as the main box, with two workers being in virtualized OSes. They can all ping each other, so I know they are connected. I do not know how to start up bro in this method though. Should I be starting bro on the workers, then broctl on the manager? Vice Versa? Only start broctl?
My other problem is a new one, I am now getting an error when I try to start broctl on the manager node. It keeps telling me that the broctl start script can only be run on a manager node, is there some place to tell it this is the manager (it was working before, the errors I got were related to the workers, not the manager).
Sorry for the long email, but I have been working for quite a while and I can't figure this out. I have also spent a long time searching for help that's out there already, sorry if I missed it.
Thank you.
Sam
Hi everyone,
I'm running bro version 1.5.2.2. I am trying to get the
drop_source_and_terminate function working.
I have a test notice set up that calls drop_source_and_terminate. It is
logging the NOTICE_DROP, but the terminate_connection(n$conn) doesn't
seem to be working. I'm not seeing TerminatingConnection or
TerminatingConnectionIgnored notices, and am getting the following error:
1275573771.744509 /usr/local/bro/share/bro/notice-action-filters.bro,
line 74 (terminate_connection): run-time error, value used but not set
Any ideas?
Tyler
--
--
Tyler Schoenke
Network Security Analyst
IT Security Office
University of Colorado - Boulder