John,
Thanks for the quick response.
> Not sure what Netscalar does, but it all should act the same. The host
> TCP stack would drop any attempted connection for which a session was
> not established regardless of what was upstream from it. Quick and
> dirty, you sould be able to fire up tcpdump and see the session
> initialization.
That's what I'm finding strange. After running a tcpdump capture on
the interface and analyzing it with Wireshark, I do not see any 3-way
handshakes for this particular web application. For any HTTP GET that
I see in Wireshark that pertains to this application, when I "Follow
TCP Stream", the first entry in Wireshark is always the GET message
itself. For all other applications on the network, doing the above
results in the first entry being the SYN.
I've generated a few dumps with the same results. I wonder if the
load balancer is somehow keeping a session active for very long
periods (if this even makes sense).
If you have any suggestions or thoughts, I'd be very interested.
Thanks,
Bill
On Sat, Feb 6, 2010 at 12:51 PM, John Hally <JHally(a)ebscohost.com> wrote:
> Hi Bill,
>
> I've run BRO in the past with load balancers (Arrowpoint/Cisco CSS) and
> was able to see all traffic. In our setup we had 2 segments; a VIP
> access link and a services trunk link where the real/origin servers
> lived. Both of these links had physical network taps and it was as
> simple as plugging in the Ethernet, flipping the interface to
> UP/PROMISC, and starting BRO.
>
> With the CSS, even though the unit would handle the initial connection,
> it would 'snap' that over to the origin server it picked during load
> balancing so you would still see the tcp setup.
>
> Not sure what Netscalar does, but it all should act the same. The host
> TCP stack would drop any attempted connection for which a session was
> not established regardless of what was upstream from it. Quick and
> dirty, you sould be able to fire up tcpdump and see the session
> initialization.
>
> Thoughts?
>
> Tahnks.
>
> John.
>
> -----Original Message-----
> From: bro-bounces(a)ICSI.Berkeley.EDU
> [mailto:bro-bounces@ICSI.Berkeley.EDU] On Behalf Of Bill Jones
> Sent: Saturday, February 06, 2010 10:22 AM
> To: bro(a)ICSI.Berkeley.EDU
> Subject: [Bro] Load Balancers
>
> Hi everyone,
>
> I was curious if anyone has any experience running bro between
> load-balancers (such as Netscaler) and web applications. We are
> currently trying to get HTTP logs generated for a web application. We
> couldn't figure out why bro was not triggering the HTTP analyzer, but
> I now believe that this is because it is never seeing the original SYN
> + SYN/ACK for the conversation. When viewing the conversations in
> Wireshark, I can see that all the TCP streams for this particular
> application begin with the GET and do not include the initial 3-way
> handshake.
>
> Here is an entry in the conn.log for this stream which shows the states:
>
> 1265389087.849048 ? 10.19.120.12 10.19.2.78 http 2232 80 tcp 14785
> 604140 OTH X DdAa
>
> Other web applications on the wire, which do have the 3-way handshake
> visible for all connections, seem to work just fine and I get http
> logs.
>
> My questions are:
>
> Am I correct in assuming that the lack of initial connection
> establishment is why the HTTP analysis is never occurring (and
> therefore I'm not getting entries in http.log)?
>
> Is there a way to force bro to analyze the traffic even though there
> is no proper 3-way handshake visible?
>
>
> Thanks for your time,
> Bill
> _______________________________________________
> Bro mailing list
> bro(a)bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
Hello everyone,
First of all, tomorrow is thanksgiving and I would like to thank all of you
for all the feedback I've always received to my posts.
I continue with my research on anomalies, now focus on evasion techniques,
and I need to ask you some help to understand how BRO deals with
fragmentation and TCP overlapping issues. For reference, I am using Bro
1.5.1 in offline analysis.
1. Although I am loading "frag", I am not receiving any event related with
fragmentation.
What could be wrong? libpcap library? my BRO version?
2. What are the possible events triggered by weird analyzer related with tcp
overlapping? (because I am not getting any of them although I think I should
see them on my trace)
3. TCP overlapping problems may generate "partial_ftp_request",
"partial_RPC_request" or other partial events? and also confuse BRO on how
the connection should be flagged? For example a connection with flag "S0",
no reply seen could be related with TCP overlapping problems?
4. How does BRO perform TCP reassembly? I mean, is the traffic on ALL ports
reassembled? Is there any way to apply a default policy for doing TCP
reassembly? Like Policy First or Last or Unix…
5. There is an "active mapping" function to improve TCP reassembly. Can we
define the host profile database without this active function?
6. Can we configure the size of the reassembly buffer? I read in historical
msg (from 2006) there wasn't such config and BRO presented a vulnerability
against an adversary trying to exhaust memory, is this a current
possibility?
7. By doing offline analysis, I understood that BRO will analyze all the
packets without loss even if the CPU is running at 100%. Still, I need
information about dropping packets for other reasons. For example, if BRO
encounters TCP overlapping, Does it drop all the packets? Choose some of
them? Are these actions log somewhere? The same with fragmentations issues.
Where can I check the portion of fragments that where reassembled? how many
frames discarded, etc?
8. I am not seeing any difference in bro logs when I analyze 2 pcap files.
One file contains some malformed packet at the end and wireshark says "the
packet is bigger than 65535", the other pcap file is the same file but
truncated using editcap to avoid this "malformed packet" (if I check the hex
using hd, the part truncated represents 850MB ). All the logs of BRO when
input is one file or the other are identical. Is this the expected result?
Veronica Estrada
Nakao Lab. Network System Research Group
University of Tokyo
Thanks for your mail. Please note that as I'm currently out on
travel, replies may take a bit longer than usual.
Best regards,
Robin
--
Robin Sommer * Phone +1 (510) 722-6541 * robin(a)icir.org
ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org
I was unable to compile Bro-1.5.1 on Solaris 11 Express.Below are the
last lines of warnings make gave me.
./configure did not show any problem.
gcc -g -O2 -I/include -I/usr/include -o hf hf.o setsignal.o
version.o -lreadline -lz -lpcap -lpcap -lssl -lcrypto -lnsl
-lsocket -lpcap -ltermcap
Undefined first referenced
symbol in file
_res hf.o
ld: fatal: symbol referencing errors. No output written to hf
collect2: ld returned 1 exit status
make[4]: *** [hf] Error 1
make[4]: Leaving directory `/home/abar/bro-1.5.1/aux/hf'
make[3]: *** [all-recursive] Error 1
make[3]: Leaving directory `/home/abar/bro-1.5.1/aux'
make[2]: *** [all] Error 2
make[2]: Leaving directory `/home/abar/bro-1.5.1/aux'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/home/abar/bro-1.5.1'
make: *** [all] Error 2
I am trying to setup bro on a CentOS 5.5 server. I am running into problems once I hit the make step.
I get /opt/bro/src/Val.cc:2537: undefined reference to 'debug_logger'
Val.o: In function 'MutableVal::LoggingAccess() const':
/opt/bro/src/Val.h:458: undefined reference to 'debug_logger'
collect2: ld returned 1 exit status
make[3]: ***[bro] Error 1
make[3]: Leaving directory '/opt/bro-1.5.1/src'
make[2]: ***[all] Error 2
make[2]: Leaving directory '/opt/bro-1.5.1/src'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory '/opt/bro-1.5.1/src'
make: *** [all] Error 2
This was a default install of CentOS, nothing but updates done to it so far. The only things I added based on previous installations of bro were:
autoconf bison byacc flex gcc gcc-c++ libtool ncurses-devel openssl openssl-devel python python-devel python-tools ruby
I also installed GeoIP-devel-1.4.5.1 GeoIP-1.4.5.1 libpcap-0.9.7
Any help would be great.
Thanks,
Sam
Hi folks,
I was wondering if anyone had a recipe for changing the log rotate
script to rotate bro logs like regular log rotate does
notice.log
notice.log.1
notice.log.2
notice.log.3
etc
vs
notice.log-10-11-15_13.00.00
notice.log-10-11-15_14.00.00
notice.log-10-11-15_15.00.00
notice.log-10-11-15_16.00.00
etc
And extra points for cleanup of "old" logs.
Thought I'd ask before I did it myself.
Thanks,
-Tim
Hello BRO professionals,
I am using BRO v 1.5.1 to analyze off-line pcap files. When I run BRO
on 4Gb pcap file, one CPU core always reaches 100% but the server
still has more 15 idle cores.
The analysis uses brolite, dpd and detect-protocols.
I am afraid BRO is loosing packets. By the way, how can I measure
packet dropping?
The capture-loss generates this notice:
no=CaptureLossSummary na=NOTICE_ALARM_ALWAYS msg=estimated\ rate\ \=\
0.0082201 tag=@36-6fb3-4a
Are this events or bytes? WHy indicates tag? I cannot find any
reference to this tag in any of the other logs. By reading the
documentation, it seems you don't recommend this metric.
Instead, I will be happy to know the number of packets that BRO
processed. I cannot find where is this number logged.
Best regards
Veronica Estrada
Nakao's Laboratory
Univ. of Tokyo
I'm writing to a global table with two items in a set (global blah: set[addr,string] &write_expire = 4 hrs;)
I'm adding items into the table like this: add blah[a,b];, then adding a notice for an external parser to deal with. Is there any way to only send the notice every four hours?
Thanks guys,
Jason
I recently built and tested Bro Cluster for the Bivio Platform for some
of our customers and wanted to share the information.
The cluster version of Bro is a very native fit for the Bivio
architecture. The internals of the Bivio platform on a single Bivio 7562
can be thought of as a load balancer and 12 separate Linux systems (this
can scale to 48 systems in a single logical unit). The Linux systems
have their own communication plane within the Bivio system that is
separate from the packet acquisition path and can use this to talk to
the workers, proxies, and the manager. The shared file system also
allows for easy setup. Below are the steps I used to setup the system
and Bro Cluster. I also attached my node.cfg for a Bivio 7562, this file
can be edited so that it reflects the number of cpu cores that will be
running systems for Bro.
Installation
--------------------------
1. Unzip Bro
tar -zxvf bro-1.5-release.tar.gz
2. Change into the Bro directory
cd bro-1.5.1/
3. Configure Bro with desired options
./configure --disable-select-loop --enable-cluster
4. Build Bro
make
5. Install Bro with Broctl
make install-broctl
System Configuration
---------------------------
1. Turn off strict key checking to avoid key prompts when logging into
Bro worker cpus
vi /etc/ssh/ssh_config
add
StrictHostKeyChecking no
2. Generate public/private key
ssh-keygen -t rsa -f /root/.ssh/id_rsa
hit return twice for a blank passphrase
3. Add it to the authorized keys
cat /root/.ssh/id_rsa.pub >> /root/.ssh/authorized_keys2
4. Setup your manager,proxy, and worker nodes in your node.cfg
configuration file
# $Id: node.cfg,v 1.1 2010/11/05 19:49:46 jebrahimi Exp $
#
# Node configuration
#
[manager]
type=manager
host=CPU-X
[proxy-1]
type=proxy
host=CPU-X
[worker-1]
type=worker
host=CPU-1c0
interface=default
[worker-2]
type=worker
host=CPU-1c1
interface=default
[worker-3]
type=worker
host=CPU-2c0
interface=default
[worker-4]
type=worker
host=CPU-2c1
interface=default
[worker-5]
type=worker
host=CPU-3c0
interface=default
[worker-6]
type=worker
host=CPU-3c1
interface=default
[worker-7]
type=worker
host=CPU-4c0
interface=default
[worker-8]
type=worker
host=CPU-4c1
interface=default
[worker-9]
type=worker
host=CPU-5c0
interface=default
[worker-10]
type=worker
host=CPU-5c1
interface=default
[worker-11]
type=worker
host=CPU-6c0
interface=default
[worker-12]
type=worker
host=CPU-6c1
interface=default
5. Edit your networks.cfg and broctl.cfg in /usr/local/bro/etc/
6. You will need to add the Bro binaries to you Path
export PATH="$PATH:/usr/local/bro/bin"
7. Install workers and proxies
broctl install
Running Bro
-----------------
1. Since we are running Bro through the cluster shell
and not Bivios nrsp we will need to force on load sharing to the APC
CPUs
nrsp loadshare all on
2. Add the crontab entry for some required Bro tasks, enter cron
crontab -e
then add
0-59/5 * * * * /usr/local/bro/bin/broctl cron
3. Start Bro
broctl start
// Joel
Joel Ebrahimi
Solutions Architect
Bivio Networks Inc.
Hi all,
I'm currently trying to find a method that identifies TCP retransmission
and out-of-order in TCP flows from the monitor's point of view.
Keeping previous sequence numbers (and cleaning them out after the
acknowledgement) in the list and seeing if the current sequence number
is already in the list or not, could be a simple approach for
identifying retransmissions.
However, in this case, we cannot detect segments which are lost before
the monitoring point.
Thus, I think that following scenario should be considered as a
retransmission.
[A] - [B] (lost before the analyzer) - [C] - [B] (Retransmission)
So, the analyzer sees [A] - [C] - [B].
In this case, when the analyzer processes the segment B (the last
segment), the analyzer can realize that the segment is re-sent because
the sequence number of B is smaller than the latest seen segment (C).
Now, the ambiguousness is caused when we consider the out-of-order. See
the following scenario:
[A] - [C] - [B] (Delayed)
The analyzer sees the same sequence numbers in the same order as the
previous scenario shows. However, the segment B here is not a
retransmission.
Is there any good methods for distinguishing retransmissions from
out-of-orders?
Any ideas will be very much appreciated.
Juhoon