John,
Thanks for the quick response.
> Not sure what Netscalar does, but it all should act the same. The host
> TCP stack would drop any attempted connection for which a session was
> not established regardless of what was upstream from it. Quick and
> dirty, you sould be able to fire up tcpdump and see the session
> initialization.
That's what I'm finding strange. After running a tcpdump capture on
the interface and analyzing it with Wireshark, I do not see any 3-way
handshakes for this particular web application. For any HTTP GET that
I see in Wireshark that pertains to this application, when I "Follow
TCP Stream", the first entry in Wireshark is always the GET message
itself. For all other applications on the network, doing the above
results in the first entry being the SYN.
I've generated a few dumps with the same results. I wonder if the
load balancer is somehow keeping a session active for very long
periods (if this even makes sense).
If you have any suggestions or thoughts, I'd be very interested.
Thanks,
Bill
On Sat, Feb 6, 2010 at 12:51 PM, John Hally <JHally(a)ebscohost.com> wrote:
> Hi Bill,
>
> I've run BRO in the past with load balancers (Arrowpoint/Cisco CSS) and
> was able to see all traffic. In our setup we had 2 segments; a VIP
> access link and a services trunk link where the real/origin servers
> lived. Both of these links had physical network taps and it was as
> simple as plugging in the Ethernet, flipping the interface to
> UP/PROMISC, and starting BRO.
>
> With the CSS, even though the unit would handle the initial connection,
> it would 'snap' that over to the origin server it picked during load
> balancing so you would still see the tcp setup.
>
> Not sure what Netscalar does, but it all should act the same. The host
> TCP stack would drop any attempted connection for which a session was
> not established regardless of what was upstream from it. Quick and
> dirty, you sould be able to fire up tcpdump and see the session
> initialization.
>
> Thoughts?
>
> Tahnks.
>
> John.
>
> -----Original Message-----
> From: bro-bounces(a)ICSI.Berkeley.EDU
> [mailto:bro-bounces@ICSI.Berkeley.EDU] On Behalf Of Bill Jones
> Sent: Saturday, February 06, 2010 10:22 AM
> To: bro(a)ICSI.Berkeley.EDU
> Subject: [Bro] Load Balancers
>
> Hi everyone,
>
> I was curious if anyone has any experience running bro between
> load-balancers (such as Netscaler) and web applications. We are
> currently trying to get HTTP logs generated for a web application. We
> couldn't figure out why bro was not triggering the HTTP analyzer, but
> I now believe that this is because it is never seeing the original SYN
> + SYN/ACK for the conversation. When viewing the conversations in
> Wireshark, I can see that all the TCP streams for this particular
> application begin with the GET and do not include the initial 3-way
> handshake.
>
> Here is an entry in the conn.log for this stream which shows the states:
>
> 1265389087.849048 ? 10.19.120.12 10.19.2.78 http 2232 80 tcp 14785
> 604140 OTH X DdAa
>
> Other web applications on the wire, which do have the 3-way handshake
> visible for all connections, seem to work just fine and I get http
> logs.
>
> My questions are:
>
> Am I correct in assuming that the lack of initial connection
> establishment is why the HTTP analysis is never occurring (and
> therefore I'm not getting entries in http.log)?
>
> Is there a way to force bro to analyze the traffic even though there
> is no proper 3-way handshake visible?
>
>
> Thanks for your time,
> Bill
> _______________________________________________
> Bro mailing list
> bro(a)bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
The Bro project is looking for an exceptional intern for three months
during the summer of 2011. If you are interesteded in helping us
improve Bro and develop new functionality, please apply!
We are looking for candidates who have excellent programming skills in
C/C++ and Python, are familiar with Unix-style systems, and have a
solid background in network technology. Having implemented network
protocols before is a plus, as is prior involvement with open source
projects.
This position is a paid engineering internship at the International
Computer Science Institute in Berkeley, CA. ICSI will cover travel to
Berkeley as well as provide support with obtaining a temporary US visa
if required.
The deadline for applications is Friday, April 8, 2011. Applicants
will be notified of decisions by March 15, 2011.
If you are interested, please send an application including a cover
letter and a resume to info(a)bro-ids.org (TXT or PDF format only
please). Make sure to mention any relevant programming projects that
you have been working on in the past and describe what your role in
them was. Please also include the name of at least one reference, and
arrange for the reference to submit a supporting letter to
info(a)bro-ids.org via email by the submission deadline.
--
Robin Sommer * Phone +1 (510) 722-6541 * robin(a)icir.org
ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org
On Feb 9, 2011, at 1:14 PM, Neslog wrote:
> How about the file_flush.bro? When I'm testing I lod that one with a
> short time inerval.
Good catch. I had a nagging feeling that I was missing something.
.Seth
--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/
Greetings all,
I'm having an issue with the install of Bro via cluster mode. I receive this error when I run make install-broctl:
( cd aux/broctl && make install-broctl )
make[1]: Entering directory `/home/bro1/bro/bro-1.5.2/aux/broctl'
MAKE_DESTDIR= ./bin/make-wrapper install --make
error: script must be run on manager node
make[1]: *** [install-broctl] Error 1
make[1]: Leaving directory `/home/bro1/bro/bro-1.5.2/aux/broctl'
make: *** [install-broctl] Error 2
This is on the machine I will use as the Managing / Proxy. I also have 2 other machines set as workers (they installed fine - No issues)
Any ideas?
Thanks,
=-=Blake
--
Blake Mattern
Information Security
California Institute of Technology
626-395-3512
mattern(a)caltech.edu
Hi all,
I'm trying Bro Ids for the very first time.
I want to have a log file where I can see which signatures have been triggered.
So I have created a very simple signature and check if it is triggered
with --debug-rules.
Result:
1297262131.735271 SensitiveSignature 192.168.1.60: my signature
So the signature is triggered. However no file is created.
Am I missing something? I have read a lot of information and I didn't
find anything.
BTW, the Bro Reference Manual refers the Bro variable
signatures_files. However it seems that the correct one is
signature_files. Am I wrong?
Many thanks,
David
We're moving from a single production Bro system to a Bro cluster while
also upgrading to 1.5.2. I've got a few questions about customizing
notifications.
1) We have a situation where one of the Bro workers may be monitoring a
shared link. We want all the events sent centrally to the manager still,
but there's a request that an outside entity have access to that worker's
logs as well. One option is to set "redef suppress_local_output = F;" and
export the logs from the worker directly, but there's also the issue of
email alerts. Is there an option to specify a different mail_dest for a
given worker node?
2) Related to that, we want to send some alerts to different email
addresses. Our use case here is sending off new, untuned alerts to a
different address than our normal incident response list. One option is
to just write a function that does that emailing through a script, but I'm
just checking to make sure there isn't a built-in variable for that.
Thanks,
Dop
Unrelated to notification, I have a couple more bro clustering questions.
1) A couple more months from now, asymmetric routing is going to be a real
problem for us. My plan is to correlate possible_split_routing alerts to
identify those situations. Other than writing an external script to
process the logs on the manager node, can the manager do this within Bro?
Essentially it would have to process an event based on an event handed to
it from the worker nodes. I guess this is a more general question, can
the manager programmatically respond to things seen by various workers
that the workers themselves can't see as a whole?
2) It's probably too early to ask as we're just beginning to think about
this, but is it possible to distribute a Time Machine setup across all the
Bro workers?
-Dop
(Last one for today, I promise)
Given these two signatures:
signature s2b-1939-4 {
ip-proto == udp
dst-port == 67
# Not supported: byte_test: 1,>,6,2
event "MISC bootp hardware address length overflow"
payload /\x01/
}
signature s2b-1940-3 {
ip-proto == udp
dst-port == 67
# Not supported: byte_test: 1,>,7,1
event "MISC bootp invalid hardware type"
payload /\x01/
}
We see both of them (which I'm about to ignore), but I don't understand
why one is triggered over the other.
Thanks,
Dop
I wanted to let these communities know about a new open-source project
called StreamDB (http://code.google.com/p/streamdb/) I've just
published that's proven to be extremely helpful for my analysts. It
is a fast and simple tool for quickly viewing traffic related to IDS
alerts (or any IP-based event) which specializes in ultra-fast
retrievals from very large data sets. It can hook into Snorby as it
is OpenFPC compatible. It is also very effective for PCRE searching
traffic from a given source or destination IP address. Streams are
rotated out based on configured retention size in a ring-buffer
fashion. From the project home page:
StreamDB is a high-performance framework for storing network streams.
The current version uses Vortex IDS to read the streams from a file or
network interface and saves them to an indexed DB and data file. Web
code provides an URL-based query interface. There is also a
command-line interface which includes the ability to read piped
queries from STDIN. In addition to almost instant retrieval by IP
address, StreamDB also allows PCRE searches and file type searches on
streams if an IP address is provided as an initial filter. The system
can handle recording gigabit line-speed networks and can retrieve
arbitrary streams from terabytes of data in milliseconds. It is
designed to be a complimentary tool to intrusion detection systems to
aid security analysts.
Here are some query examples:
http://streamdb/?srcip=10.0.0.1http://streamdb/?srcip=10.0.0.1&dstip=1.1.1.1&dstport!80http://streamdb/?srcip=10.0.0.1&dstip=1.1.1.1&dstport=80&start=2 weeks
ago&end=now
http://streamdb/?srcip=10.0.0.1&pcre=example.comhttp://streamdb/?srcip=10.0.0.1&pcre=MZ.*PE\x00\x00http://streamdb/?srcip=10.0.0.1&sort=1&as_hex=1http://streamdb/?srcip=10.0.0.1&raw=1http://streamdb/?srcip=10.0.0.1&offset=1000&limit=200http://streamdb/?srcip=10.0.0.1&filetype=executable
Examples from the CLI:
./sdb --srcip 10.0.0.1 --filetype pdf --headers-only
tail -f /var/log/snort/alert | ./sdb > alert_streams.txt
All of these queries will return in a second or two, assuming that the
IP's referred to aren't busy web servers or NAT points. IP's with
many connections will benefit from more specific search filters for
time and/or destination IP address. Some non-scientific benchmarks on
commodity hardware searching 5 TB of data: PCRE search for a given
srcip with 1000 connections completes in about one second. A similar
search for a srcip with 50,000 connections will take about two
minutes. A lookup for a given srcip/dstip pair will complete in less
than a second, including browser render time.
Example output as text/plain:
Returning 2 of 2 at offset 0 from Sun Jan 30 11:56:11 2011 to Sun Jan
30 11:56:11 2011
2011-01-30 11:56:11 192.168.58.52:4099 -> 131.243.2.191:80 13s 512
bytes FIN ASCII text, with CRLF line terminators
GET /bro-workshop-2009-2/slides/Installation.pdf HTTP/1.1
Host: www.bro-ids.org
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.13)
Gecko/20101206 Ubuntu/10.10 (maverick) Firefox/3.6.13
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115
Referer: http://www.bro-ids.org/bro-workshop-2009-2/slides/
X-Do-Not-Track: 1
Connection: keep-alive
2011-01-30 11:56:11 192.168.58.52:4099 <- 131.243.2.191:80 13s 778247
bytes FIN PDF document, version 1.3
200 OK
Connection: Keep-Alive
Date: Sun, 30 Jan 2011 17:56:11 GMT
Accept-Ranges: bytes
ETag: "8f724e-bde6b-47617252eebc0"
Server: Apache/2.2.17 (FreeBSD) mod_ssl/2.2.17 OpenSSL/0.9.8e DAV/2
PHP/5.3.5 with Suhosin-Patch mod_python/3.3.1 Python/2.6.6
mod_wsgi/2.8
Content-Length: 777835
Content-Type: application/pdf
Last-Modified: Sat, 17 Oct 2009 01:08:07 GMT
Keep-Alive: timeout=5, max=100
X-HTTP-Version: 1.1
%PDF-1.3
%...........
4 0 obj
<< /Length 5 0 R /Filter /FlateDecode >>
stream
x..VM..7 ..W.T..z,i4....I.-.bS..!.a...nl..=. ...%O.Q.....|.Y.(..|.......).
9........l.h.'E....-....&.7]....... .}Fvr....}.x...
.)...^k."U.rC.....w.N<...Z..u<..Z..e.j....4T.Hpj.........u...../g.....n.....o.......R....*.Do.9<.*]>...(...I8....ikJ_.T...:......c|..Ki..Q..>.U.MZ...*...!........jKik~7.qg.iw#.......|..............D/.\Yx..v...2<..d. O./...<...&.DDF....x..K.
Qy...|..f F>.2k.v....^v..{^<..Q..v....t}.f;x.e.S..]...U.7......l..uD...
<clipped for brevity>
I hope that others have found it as useful as we have for rapidly
investigating IDS alerts. Please use the project page or email me to
let me know about any questions, issues, or suggestions.
Thanks,
Martin
