John,
Thanks for the quick response.
> Not sure what Netscalar does, but it all should act the same. The host
> TCP stack would drop any attempted connection for which a session was
> not established regardless of what was upstream from it. Quick and
> dirty, you sould be able to fire up tcpdump and see the session
> initialization.
That's what I'm finding strange. After running a tcpdump capture on
the interface and analyzing it with Wireshark, I do not see any 3-way
handshakes for this particular web application. For any HTTP GET that
I see in Wireshark that pertains to this application, when I "Follow
TCP Stream", the first entry in Wireshark is always the GET message
itself. For all other applications on the network, doing the above
results in the first entry being the SYN.
I've generated a few dumps with the same results. I wonder if the
load balancer is somehow keeping a session active for very long
periods (if this even makes sense).
If you have any suggestions or thoughts, I'd be very interested.
Thanks,
Bill
On Sat, Feb 6, 2010 at 12:51 PM, John Hally <JHally(a)ebscohost.com> wrote:
> Hi Bill,
>
> I've run BRO in the past with load balancers (Arrowpoint/Cisco CSS) and
> was able to see all traffic. In our setup we had 2 segments; a VIP
> access link and a services trunk link where the real/origin servers
> lived. Both of these links had physical network taps and it was as
> simple as plugging in the Ethernet, flipping the interface to
> UP/PROMISC, and starting BRO.
>
> With the CSS, even though the unit would handle the initial connection,
> it would 'snap' that over to the origin server it picked during load
> balancing so you would still see the tcp setup.
>
> Not sure what Netscalar does, but it all should act the same. The host
> TCP stack would drop any attempted connection for which a session was
> not established regardless of what was upstream from it. Quick and
> dirty, you sould be able to fire up tcpdump and see the session
> initialization.
>
> Thoughts?
>
> Tahnks.
>
> John.
>
> -----Original Message-----
> From: bro-bounces(a)ICSI.Berkeley.EDU
> [mailto:bro-bounces@ICSI.Berkeley.EDU] On Behalf Of Bill Jones
> Sent: Saturday, February 06, 2010 10:22 AM
> To: bro(a)ICSI.Berkeley.EDU
> Subject: [Bro] Load Balancers
>
> Hi everyone,
>
> I was curious if anyone has any experience running bro between
> load-balancers (such as Netscaler) and web applications. We are
> currently trying to get HTTP logs generated for a web application. We
> couldn't figure out why bro was not triggering the HTTP analyzer, but
> I now believe that this is because it is never seeing the original SYN
> + SYN/ACK for the conversation. When viewing the conversations in
> Wireshark, I can see that all the TCP streams for this particular
> application begin with the GET and do not include the initial 3-way
> handshake.
>
> Here is an entry in the conn.log for this stream which shows the states:
>
> 1265389087.849048 ? 10.19.120.12 10.19.2.78 http 2232 80 tcp 14785
> 604140 OTH X DdAa
>
> Other web applications on the wire, which do have the 3-way handshake
> visible for all connections, seem to work just fine and I get http
> logs.
>
> My questions are:
>
> Am I correct in assuming that the lack of initial connection
> establishment is why the HTTP analysis is never occurring (and
> therefore I'm not getting entries in http.log)?
>
> Is there a way to force bro to analyze the traffic even though there
> is no proper 3-way handshake visible?
>
>
> Thanks for your time,
> Bill
> _______________________________________________
> Bro mailing list
> bro(a)bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
Hi!
I've captured a HTTP packet file using tcpdump. But when I read it with bro,
it shows message as following and I can't get log file:
1271639268.624587 weird: spontaneous_FIN
1271639268.624655 weird: spontaneous_FIN
1271639268.624759 weird: spontaneous_FIN
1271639277.565623 weird: above_hole_data_without_any_acks
1271639281.963865 weird: spontaneous_FIN
1271639282.625769 weird: above_hole_data_without_any_acks
1271639283.776172 weird: spontaneous_FIN
My command is "bro -r XXX.trace http-reply http-header". I can get log file
if I use "bro -r XXX.trace mt", but the output log file is not http
information and is not what I want.
Will anyone help me to fix this?
Thanks!
--
Best regards!
Eric,Ma
Tsinghua University, Beijing
Tel：15210836318
Hi,
I am looking at increasing the performance of Bro by making use of
PF_RING and TNAPI, at the kernel and driver level in Linux. These
existing tools from http://www.ntop.org/news.php allow multi-threaded
applications to drastically increase performance by making use of
functionality that exists in several network cards today.
While PF_RING, TNAPI and other network-card functionality allow for
phenomenal speed-up, they have to be tuned right or else they can lead
to a negative effect on performance. So I'm trying to determine how to
tune them to meet Bro's needs.
As Multi-threaded Bro is not ready yet, I am trying to think of a SIMPLE
application that can best characterize the Bro workload. The design
criteria that I have for this application so far is-
1. Send every packet belonging to a particular connection to the same core.
2. Introduce a small wait on every packet to simulate Bro's processing
of that packet? What will be a good value?
I'd be very interested to have everyone's opinion on this. I really am
looking for a very simple algorithm that will be a right approximation.
Thank you,
Sunjeet Singh
Hi,
I have managed to get TNAPI/PF_RING configured and working with
PF_RING-aware libpcap. http://www.ntop.org/TNAPI.html
Looks like this will be very well suited to the Multiprocessing version
of Bro.
1. At the device driver level, RSS functionality (also, Flow Director in
Intel) allows packets multiplexed to different Receiver Queues (and also
allows packets belonging to a particular connection be sent to the same
RX_Queue) on an I/OAT-supporting network card.
2. By virtue of TNAPI, these multiple RX_Queues get polled concurrently
(by one kernel thread per queue), and sent to PF_RING (along with
information about which queue the packet came from).
3. PF_RING provides a user API which can be used by user-applications
like Bro to directly read from the multiple RX_Queues of a network
interface by using notation like eth0@1, eth0@2, etc. for RX_Queues 1
and 2 belonging to interface eth0. By assigning one thread to one
RX_Queue, we ensure that all packets from one connection are being
processed by the same core.
PF_RING and TNAPI can be used to drastically improve the performance of
any multiprocessing application, but need to be properly tuned and used
by the application. Performance stems from the fact that for Bro, the
packets can bypass the kernel's network stack altogether; one thread
polling per RX_Queue thanks to TNAPI; and PF_RING avoiding the mmap from
Kernel space to User space by directly copying payloads from RX_Queue rings.
Configuration wise, it took a bit of work to change Bro's configure
files to use a PF_RING-aware libpcap instead of the libpcap that Bro
ships with. When running TNAPI and PF_RING, there is a clear performance
improvement in the kernel's ability to receive packets at a higher
packet rate (results on the TNAPI website, I also verified). But using
PF_RING with the existing Bro leads to a performance degradation of Bro
because Bro runs on one user-thread, and when all these packets reach
user-space on different user-threads, they need to be processed by the
core that is running Bro. But from my knowledge on TNAPI/PF_RING and
intuition, multi-threaded Bro can be adapted to PF_RING and will lead to
huge gains in performance.
Here's the summary of results of a brief experiment that I performed on
a 8-core Intel Xeon with32 GB RAM running on Linux and with an Intel
82598EB 10Gbps ethernet card:
Goal: Compare conventional Bro installation against Bro with TNAPI and
PF_RING (I called it Bro-Ring)
Conclusion: Bro-Ring shows a performance drop.
Observations: The values in the table show for varying packet-rates, how
many packets were accepted by the machine running Bro (rest were lost).
Packets/sec Bro-Ring Bro
34000 1368791 1368003
50000 1368546 1367707
65,000 1368614
120000
1224761
130000
1168734
166000 596667
170000 561702
171000 681104
173000 618100
175000 740137
178000 864706
210000
753700
215000
728450
230000 494637
240000
636287
(Note: there was a difference in tcpreplay's input parameter packet-rate
and the actual packet rate achieved, so I could not supply exact values
for packet rate)
Sunjeet Singh
Hi Bro people,
I submit you a small patch.
For ICMP, I wanted to access the payload of the packets.
For SMB, I fixed a small bug and added the processid field.
Hope you find it useful.
Cheers,
Julien Sentier
Une messagerie gratuite, garantie à vie et des services en plus, ça vous tente ?
Je crée ma boîte mail www.laposte.net
Une messagerie gratuite, garantie à vie et des services en plus, ça vous tente ?
Je crée ma boîte mail www.laposte.net
Hi,
Does any one have a Click or other software frontend implementation that
splits traffic to different nodes (and not cores)?
Thank you,
Sunjeet Singh
Thank you very much for the help.
Some feedback:
1.
After including the filter -f "tcp or udp or icmp or (ip[6:2] & 0x3fff
!= 0)" the fragmentations events appeared. Now, I am re-processing the
files but loading only weird and matching unfragmented datagrams only
to accelerate the process.
Command:
bro -r $files -f (ip[6:2] & 0x3fff != 0)" weird
All fragment events are handle by flow weird. When is invoked flow
weird handler?
2.
Related with a retransmission inconsistency. I found the remix
inconsistency events in the notice file.
4.
"Traffic to ports for which there's an analyzer that uses the byte stream.
You can also control this using tcp_reassembler_ports_orig and
tcp_reassembler_ports_resp."
How can I redef these variables? I tried to redef this variables on
my start policy but all I get are errors ((port and 21): error,
arithmetic mixed with non-arithmetic).
const tcp_reassembler_ports_orig: set[port] = {} &redef;
const tcp_reassembler_ports_resp: set[port] = {} &redef;
Could you please illustrate us with an example, for instance include
port 21 & 80 in the tcp_reassebler_ports_orig?
Hi all,
I wanted to send out a quick note about the public availability of the bro-dev mailing list. As some of you know, this list has existed in some form for a long time but it was by invitation only. The list is being opened to everyone to help introduce new developers into the fold and make visible the development process for Bro.
I'd encourage everyone that is interested in developing scripts with Bro to join this list because beginning next month I'm going to be starting bi-weekly script review periods. On Mondays and Wednesdays for a currently undefined time period I'll be sending my thoughts on a several existing Bro scripts to the mailing list and inviting everyone to give their own comments about the scripts and potentially their associated analyzers. The goal is to have a clean and documented policy script directory for the 1.6 release along with adding lots of interesting detections out of the box! Even if you don't have comments about what you'd like to see scripts do, it could be worthwhile lurking to see the direction that Bro is moving.
One final comment I wanted to make is that the list is kind of high volume at the moment due to our ticket tracker sending activity emails to the list and generally a lot of email activity, so subscriber beware (and prepare to filter!). :)
Here's the list interface page:
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
See you on the bro-dev list!
.Seth
Hello everyone,
First of all, tomorrow is thanksgiving and I would like to thank all of you
for all the feedback I've always received to my posts.
I continue with my research on anomalies, now focus on evasion techniques,
and I need to ask you some help to understand how BRO deals with
fragmentation and TCP overlapping issues. For reference, I am using Bro
1.5.1 in offline analysis.
1. Although I am loading "frag", I am not receiving any event related with
fragmentation.
What could be wrong? libpcap library? my BRO version?
2. What are the possible events triggered by weird analyzer related with tcp
overlapping? (because I am not getting any of them although I think I should
see them on my trace)
3. TCP overlapping problems may generate "partial_ftp_request",
"partial_RPC_request" or other partial events? and also confuse BRO on how
the connection should be flagged? For example a connection with flag "S0",
no reply seen could be related with TCP overlapping problems?
4. How does BRO perform TCP reassembly? I mean, is the traffic on ALL ports
reassembled? Is there any way to apply a default policy for doing TCP
reassembly? Like Policy First or Last or Unix…
5. There is an "active mapping" function to improve TCP reassembly. Can we
define the host profile database without this active function?
6. Can we configure the size of the reassembly buffer? I read in historical
msg (from 2006) there wasn't such config and BRO presented a vulnerability
against an adversary trying to exhaust memory, is this a current
possibility?
7. By doing offline analysis, I understood that BRO will analyze all the
packets without loss even if the CPU is running at 100%. Still, I need
information about dropping packets for other reasons. For example, if BRO
encounters TCP overlapping, Does it drop all the packets? Choose some of
them? Are these actions log somewhere? The same with fragmentations issues.
Where can I check the portion of fragments that where reassembled? how many
frames discarded, etc?
8. I am not seeing any difference in bro logs when I analyze 2 pcap files.
One file contains some malformed packet at the end and wireshark says "the
packet is bigger than 65535", the other pcap file is the same file but
truncated using editcap to avoid this "malformed packet" (if I check the hex
using hd, the part truncated represents 850MB ). All the logs of BRO when
input is one file or the other are identical. Is this the expected result?
Veronica Estrada
Nakao Lab. Network System Research Group
University of Tokyo
I've been seeing AddressScan alerts, but when I check conn.log, I can't
find the corresponding entries. I got an alert yesterday about a
5060/udp scan hitting 100 hosts. Below are the conn.log, flowscan, and
notice.log for the entire day matching the IP and port.
conn.log:
Dec 1 11:27:45 0.000000 172.21.210.116 151.32.190.137 other 51272 5060
udp 101 ? S0 L
flowscan:
12/01 11:27:45 172.21.210.116 151.32.190.137 17 51272
5060 1 129
notice.log:
Dec 1 11:27:45 no=AddressScan na=NOTICE_EMAIL es=w5 sa=172.21.210.116
p=5060/udp num=100 msg=172.21.210.116\ has\ scanned\ 100\ hosts\
(5060/udp) tag=@62-7ba5-3df5e6
As you can see, at 11:27, Bro thinks 100 hosts were scanned on
5060/udp. But the conn.log and flowscan data only show one host being
scanned. Any ideas why this alert thinks 100 hosts are being hit when
it is one host with a single SYN?
Tyler
--
--
Tyler Schoenke
Network Security Analyst
IT Security Office
University of Colorado - Boulder
