John,
Thanks for the quick response.
> Not sure what Netscalar does, but it all should act the same. The host
> TCP stack would drop any attempted connection for which a session was
> not established regardless of what was upstream from it. Quick and
> dirty, you sould be able to fire up tcpdump and see the session
> initialization.
That's what I'm finding strange. After running a tcpdump capture on
the interface and analyzing it with Wireshark, I do not see any 3-way
handshakes for this particular web application. For any HTTP GET that
I see in Wireshark that pertains to this application, when I "Follow
TCP Stream", the first entry in Wireshark is always the GET message
itself. For all other applications on the network, doing the above
results in the first entry being the SYN.
I've generated a few dumps with the same results. I wonder if the
load balancer is somehow keeping a session active for very long
periods (if this even makes sense).
If you have any suggestions or thoughts, I'd be very interested.
Thanks,
Bill
On Sat, Feb 6, 2010 at 12:51 PM, John Hally <JHally(a)ebscohost.com> wrote:
> Hi Bill,
>
> I've run BRO in the past with load balancers (Arrowpoint/Cisco CSS) and
> was able to see all traffic. In our setup we had 2 segments; a VIP
> access link and a services trunk link where the real/origin servers
> lived. Both of these links had physical network taps and it was as
> simple as plugging in the Ethernet, flipping the interface to
> UP/PROMISC, and starting BRO.
>
> With the CSS, even though the unit would handle the initial connection,
> it would 'snap' that over to the origin server it picked during load
> balancing so you would still see the tcp setup.
>
> Not sure what Netscalar does, but it all should act the same. The host
> TCP stack would drop any attempted connection for which a session was
> not established regardless of what was upstream from it. Quick and
> dirty, you sould be able to fire up tcpdump and see the session
> initialization.
>
> Thoughts?
>
> Tahnks.
>
> John.
>
> -----Original Message-----
> From: bro-bounces(a)ICSI.Berkeley.EDU
> [mailto:bro-bounces@ICSI.Berkeley.EDU] On Behalf Of Bill Jones
> Sent: Saturday, February 06, 2010 10:22 AM
> To: bro(a)ICSI.Berkeley.EDU
> Subject: [Bro] Load Balancers
>
> Hi everyone,
>
> I was curious if anyone has any experience running bro between
> load-balancers (such as Netscaler) and web applications. We are
> currently trying to get HTTP logs generated for a web application. We
> couldn't figure out why bro was not triggering the HTTP analyzer, but
> I now believe that this is because it is never seeing the original SYN
> + SYN/ACK for the conversation. When viewing the conversations in
> Wireshark, I can see that all the TCP streams for this particular
> application begin with the GET and do not include the initial 3-way
> handshake.
>
> Here is an entry in the conn.log for this stream which shows the states:
>
> 1265389087.849048 ? 10.19.120.12 10.19.2.78 http 2232 80 tcp 14785
> 604140 OTH X DdAa
>
> Other web applications on the wire, which do have the 3-way handshake
> visible for all connections, seem to work just fine and I get http
> logs.
>
> My questions are:
>
> Am I correct in assuming that the lack of initial connection
> establishment is why the HTTP analysis is never occurring (and
> therefore I'm not getting entries in http.log)?
>
> Is there a way to force bro to analyze the traffic even though there
> is no proper 3-way handshake visible?
>
>
> Thanks for your time,
> Bill
> _______________________________________________
> Bro mailing list
> bro(a)bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
I have tried to read through all the documentation and whatnot, but I don't see anything about this topic. I have bro starting fine now, however, it always quits because it says no work dir found on all of the workers. Do I need to specify a directory for it to use? How do I do that?
Sam
I have a bro cluster set up (or mostly set up) on CentOS 5.5. I am using svn version 7048. I try to run /usr/local/bro/bin/broctl start and it replies with ERROR: Script must be run on manager node. I saw someone else with this issue who added some things to PATH, which I tried, and it still did not work. Does anyone know what could cause this?
Thanks,
Sam
Hi, I'm trying to get BRO installed on FreeBSD amd64. I am having the same issue that kreibich outlined here: http://tracker.icir.org/bro/ticket/256
I've tried compiling broccoli and then compiling bro, but for some reason the main bro build script keeps trying to build broccoli and subsequently failing. Where/how should I include -enable-int64 or -fPIC to get this to build???
Thanks!
Will Urbanski
Hi,
LBNL is looking to recruit a Security Engineer or two....
For details and how to apply see:
http://jobs.lbl.gov/details.asp?jid=24661&p=1
*Summary*
Berkeley Lab, a pioneer in scientific research, has an immediate opening for
a Cyber Security Engineer. The person holding this position will be a member
of the LBNL Computer Protection Program (CPP) which provides cyber security
services and support to Berkeley Lab - an unclassified, university-like
computing environment. The primary responsibilities of this position are to
improve the Lab's cyber protection mechanisms, perform network traffic
analysis, respond to and resolve cyber security incidents, and provide
technical expertise, especially in the area of system protection (unix,
windows). In addition, participation is expected in all aspects of computer
protection that will further the mission of the Lab, such as participation
in: uncovering computer and network vulnerabilities in the LBNL environment,
reducing the risk incurred by vulnerabilities, infusing new security
technologies into the Laboratory environment, and promoting security
awareness and training. Solid interpersonal skills and the ability to work
effectively in a team environment are a must.
*
Specifically, this position will:*
- Perform cyber security monitoring and analysis, incident response, and
the forensic analysis and resolution of cyber security incidents.
- Investigate new technologies and processes to enhance cyber security
capabilities, and implement necessary improvements.
- Participate in the installation, configuration and management of CPP
maintained hardware and software.
--
Anne Hutton
Computer Protection Program
Lawrence Berkeley National Laboratory
(510) 495-2681
Hi,
I've recently begun using Bro and have found it to be a very interesting
tool to work with. I have started poking around some of the scripts and
trying a few of the exercises in the slides from the 2009 Bro workshop. So
far everything is working well.
I have created a few patches to fix small problems I had or add features
that I was looking for. I have attached these patches so that others can use
them if they find them useful. I am working off of svn build 7050. Here are
the list of changes I have made so far:
- Patched main.cc to add the -N command line flag. This effectively enables
BRO_FAKE_DNS for that run. I think this is more useful than using an
environment variable because the flags are listed in the command line help,
and it makes it easier to change from run to run without manipulating
environment variables.
- Removed duplicate login_non_failure_msgs from policy/login.bro. The same
block was listed twice, one with &redef and the other without. Seemed
superfluous.
- Commented out example in policy/scan.bro. It doesn't seem like making the
example value live added any value, and could possibly cause problems.
- Edited policy/ssh.bro to print what port is being used for ssh servers.
Also changed the data structure that tracks the servers to allow for
multiple ssh servers on the same system. I found this useful when using dpd.
- Created policy/dpd.ssh.bro that tells ssh to capture on all tcp ports.
- Changed src/DPM.cc so that the SSH Analyzer is hooked into the analyzer
tree, so that when using dpd.ssh.bro, ssh servers running on any tcp port
are detected.
There are still some more features I plan to work on. Depending on how
difficult it would be, I want to add the ability to hook an analyzer into
the analyzer tree via script, so that the source doesn't need to be edited
for each. Or so that functionality could be toggled on or off depending on
need.
I also started to convert the 6000+ Nmap service probe signatures into dpd
signatures. I have an initial list, but a lot of the Nmap regexs cause
problems with Bro. I am going to try to clean those up so that they can be
useable with Bro.
For the scripts that send packets, such as terminate-connection.bro and the
scripts that load it, I want to rewrite them so they can run in either
Active or Passive mode. In Passive mode they wouldn't send any packets,
which would be the default behavior. I know that currently the default
behavior is similar to this, but being able to designate individual scripts
to be active or passive, or knowing for sure that all scripts are being
passive could be useful and allow for more control.
I have a few more simple ideas that I will try to get working, and I will
probably think of more as I experiment more with Bro. Any feedback would be
greatly appreciated. Thanks for making such a great tool, using it has been
very fun and interesting so far and I hope to learn more from it.
Thanks,
Ben Rosenberg
Hi everyone,
I am puzzled about the outcomes of using ipsumdump or BRO for processing
multiple pcap files.
I am using BRO to analyze anomalities in my 12 hours captured network
traffic which was saved in 4 Gb pcap files. I want that BRO consider the
cases when a connection may have been split in two or more files. I was
using ipsumdump to solve this, but I found that some files have errors and
cause ipsumdump to crush with this message:
ToDump(bigPcap1.pcap): Inappropriate ioctl for device
Using the capinfo tool I detected that some of my files have packet size
larger than normal (65535), so using tshark I cut the part of the file with
problems. For example:
capinfos: An error occurred after reading 3830659 packets from
"trace2.pcap": File contains a record that's not valid.
(pcap: File has 4065648712-byte packet, bigger than maximum of 65535)
So I create a reduced version of trace2.pcap with tshark:
/usr/sbin/tshark -c 3830659 -r trace2.pcap -w trace2-new.pcap
This solution seemed to work fine, all the ***-new.pcap have no errors while
reading with capinfo or wireshark, but even so there are some that still
cause problems for processing. For example:
I processed the following files in 3 different ways:
trace1.pcap, trace2-new.pcap, trace3.pcap (trace2.pcap was replaced because
of the packet size error)
FIRST TRY - using ipsumdump with collate option:
ipsumdump --collate -w - trace* |bro -r - brolite myenvironment -f "tcp or
udp or icmp" dpd_conn_logs=T dpd detect_protocols dyn_disable irc-bot proxy
ftp
Output> 9.7 MB conn.log with 114861 lines (number of connections)
SECOND TRY - using ipsumpdump without collate option
ipsumdump --collate -w - trace* |bro -r - brolite myenvironment -f "tcp or
udp or icmp" dpd_conn_logs=T dpd detect_protocols dyn_disable irc-bot proxy
ftp
Output:
19 Mbytes conn.log with 228922 lines with 950 repeated connections
THIRD TRY - without ipsumdump:
/usr/local/bro/bin/bro -r trace1.pcap -r trace2-new.pcap -r trace3.pcap
brolite todai -f "tcp or udp or icmp" dpd_conn_logs=T dpd detect-protocols
dyn-disable irc-bot proxy ftp 2>bro-error3.log
Output:
15 Mbytes conn.log with 169168 lines, connections are not repeated
COMMENTS:
pcap files has not overlap traffic (it was checked with trace-summary using
first packet seen and last packet seen).
I tried the ipsumdump with both collate and no collate option because when I
used ipsumdump only (without bro), with collate option the resulted larger
pcap file was a 7.9 GB file but without collate option the resulted file was
12.GB (trace1.pcap: 4 MB, trace2-new.pcap: 3.9GB, trace3.pcap: 4GB).
Besides, while using ipsumpdump --collate alone, the progress bar showed
something like this:
66%****************** |8017MB ETAToDump(LargerTrace.pcap):
Success
100%****************************|12113MB
But the progress bar for ipsumdump without the collate option didn't split
and reach the 100% 12113MB.
If anyone can illuminate this matter, it will be a great help.
Veronica
Hi
i'm trying install bro-1.4 :
1. ./configure
...
no error
2.make
...
no error
3.make install
...
no error
4.make install-brolite
...
...
...
make[1]: Leaving directory `/root/Download/bro-1.4/aux'
/bin/chown -R `cat scripts/bro_user_id` /usr/local/bro/
cat: scripts/bro_user_id: No such file or directory
/bin/chown: missing operand after `/usr/local/bro/'
Try `/bin/chown --help' for more information.
make: [install-brolite] Error 1 (ignored)
*********************************************************
Please run "/usr/local/bro/etc/bro.rc --start" to start bro
*********************************************************
does not create the file ‘$BROHOME/etc/bro.cfg’.
please help me for resolve this problem.