zeek May 2011

zeek@lists.zeek.org

18 participants
17 discussions

[Bro] Current IDS and Data Mining research

by Suman Nandi

Dear Bro Developer and contributer I have been working on IDS and Data Mining .I would like to know the current research in this area that IDS using Data Mining and what are the current reseach areas and objectives where Data Mining can provide solutions to IDS? -- Regards SUMAN KUMAR NANDI HOD-Computer Applications Chitkara University, Punjab India Mobile:+919501105658

9 years, 10 months

[Bro] Open Position at NCSA

by Robin Sommer

Have you always wanted to work on Bro and be paid for it? :-) NCSA has an opening for a Senior Research Programmer: https://jobs.illinois.edu/default.cfm?page=job&jobID=8847&returnPage=sear This is a full-time position that focuses on Bro development in close collaboration with our group here at ICSI. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin(a)icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org

9 years, 10 months

[Bro] handle out of order and retransmitted packets in offline trace

by Song Zhao

Hello, All I am trying to use the policy script http-rewriter.bro in Bro-1.5.1 to anonymize the HTTP message-body of all HTTP packets in a big dumped trace larger than 100GB ( http-rewriter.bro actually deletes all HTTP message-body and add one new header field named X-Actual-Data-Length, right?) . I am not sure if Bro itself and http-rewriter.bro has the ability of reordering all tcp packets and deleting tcp retransmitted packets in every connection of the dumped trace? If they cannot do that, whether I can reorder all packets and delete the retransmitted packets in every connection first by using some tools and then use http-rewriter.bro ? Is this way reasonable? What's your suggestion about the tools I can use? Besides, I want to test if special HTTP packets exist. Special packet here means there are more than one HTTP construct(headers + message body) in one packet. When using http-rewriter.bro on several special pakcets I created, it seems that it can delete the message-body correctly for almost all of cases as long as the packets in the connection are in order and complete. Can http-rewriter.bro handle the special cases correctly as what I found? Expect your answer and thank you very much. Song Zhao

9 years, 10 months

[Bro] Current IDS and Data Mining research

by ssm_as

Hi, Well, it depend on what are you trying to do? For example, do you want to use data mining in IDS alerts analysis (e.g. alerts verification, alerts aggregation, alerts correlation)? In this case you will find a lot of research work submitted in that area. Do you want to use data mining in building IDS to detect intrusions? Then you probably taking about anomaly detection based IDS? In my opinion data mining is not the best approach to do that. Probably, you will need to think about soft computing approaches (neural network, artificial immune system, swarm intelligence, etc). The issue with your question is that you are using very abstract keywords "data mining" and "IDS" . you should be more specific. Thanks, Sherif Saad Ph.D Candidate, University of Victoria --- On Mon, 5/30/11, bro-request(a)bro-ids.org <bro-request(a)bro-ids.org> wrote: From: bro-request(a)bro-ids.org <bro-request(a)bro-ids.org> Subject: Bro Digest, Vol 61, Issue 16 To: bro(a)bro-ids.org Received: Monday, May 30, 2011, 10:00 PM Send Bro mailing list submissions to bro(a)bro-ids.org To subscribe or unsubscribe via the World Wide Web, visit http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro or, via email, send a message with subject or body 'help' to bro-request(a)bro-ids.org You can reach the person managing the list at bro-owner(a)bro-ids.org When replying, please edit your Subject line so it is more specific than "Re: Contents of Bro digest..." Today's Topics: 1. Current IDS and Data Mining research (Suman Nandi) 2. Re: handle out of order and retransmitted packets in offline trace (Song Zhao) ---------------------------------------------------------------------- Message: 1 Date: Mon, 30 May 2011 11:53:57 +0530 From: Suman Nandi <suman.nandi(a)chitkara.edu.in> Subject: [Bro] Current IDS and Data Mining research To: bro(a)bro-ids.org Message-ID: <BANLkTi=pfrxkgD6SzOmN5yrejS3G5MDJxg(a)mail.gmail.com> Content-Type: text/plain; charset="iso-8859-1" Dear Bro Developer and contributer I have been working on IDS and Data Mining .I would like to know the current research in this area that IDS using Data Mining and what are the current reseach areas and objectives where Data Mining can provide solutions to IDS? -- Regards SUMAN KUMAR NANDI HOD-Computer Applications Chitkara University, Punjab India Mobile:+919501105658

9 years, 10 months

Re: [Bro] How to throttle (or limit) the bitrate of a UDP connection using BRO?

by Harkeerat Bedi

Thank you Aaron for your suggestions :) I will look into them. Regards, Harkeerat Bedi On Thu, May 26, 2011 at 4:29 AM, Aaron Glenn <aaron.glenn(a)gmail.com> wrote: > On Wed, May 25, 2011 at 11:25 PM, Harkeerat Bedi <hsbedi(a)memphis.edu> > wrote: > > a way we can accomplish this using BRO? > > BRO is not what you want in this scenario > > > Or, is there any other way? > > Kindly suggest. > > netgraph in freebsd > pf in openbsd > npf in netbsd > I urge you try them all (: > > > Thank you, > > Harkeerat Bedi > > best of luck, > aaron glenn >

9 years, 10 months

[Bro] How to throttle (or limit) the bitrate of a UDP connection using BRO?

by Harkeerat Bedi

Hello All, I am using BRO for a part of my project. Following is what I intend to do: 1. Monitor UDP connections. 2. Compute their bitrates 3. Throttle the bitrates of these UDP connections based on some calculations. I was able to complete tasks 1 and 2. However I don't know how I can accomplish task 3. My current setup includes a Client (Node1) sending UDP data to a Server (Node3). The traffic has to pass through a Gateway (Node2) which is in between the Client and Server and is running BRO. Node1 (Client) <------> Node2 (running BRO) < ------ > Node3 (Server) If I have a UDP connection (between the Client and the Server) with a bit rate of 2Mb/s. How can I reduce its bitrate to a user set value - say: 1Mb/s, using BRO? I am assuming one way may be to drop packets of a connection with a certain frequency such that the overall bitrate of that connection reduces? Is there a way we can accomplish this using BRO? Or, is there any other way? Kindly suggest. Thank you, Harkeerat Bedi

9 years, 10 months

Re: [Bro] Load Balancers

by Bill Jones

John, Thanks for the quick response. > Not sure what Netscalar does, but it all should act the same. The host > TCP stack would drop any attempted connection for which a session was > not established regardless of what was upstream from it. Quick and > dirty, you sould be able to fire up tcpdump and see the session > initialization. That's what I'm finding strange. After running a tcpdump capture on the interface and analyzing it with Wireshark, I do not see any 3-way handshakes for this particular web application. For any HTTP GET that I see in Wireshark that pertains to this application, when I "Follow TCP Stream", the first entry in Wireshark is always the GET message itself. For all other applications on the network, doing the above results in the first entry being the SYN. I've generated a few dumps with the same results. I wonder if the load balancer is somehow keeping a session active for very long periods (if this even makes sense). If you have any suggestions or thoughts, I'd be very interested. Thanks, Bill On Sat, Feb 6, 2010 at 12:51 PM, John Hally <JHally(a)ebscohost.com> wrote: > Hi Bill, > > I've run BRO in the past with load balancers (Arrowpoint/Cisco CSS) and > was able to see all traffic. In our setup we had 2 segments; a VIP > access link and a services trunk link where the real/origin servers > lived. Both of these links had physical network taps and it was as > simple as plugging in the Ethernet, flipping the interface to > UP/PROMISC, and starting BRO. > > With the CSS, even though the unit would handle the initial connection, > it would 'snap' that over to the origin server it picked during load > balancing so you would still see the tcp setup. > > Not sure what Netscalar does, but it all should act the same. The host > TCP stack would drop any attempted connection for which a session was > not established regardless of what was upstream from it. Quick and > dirty, you sould be able to fire up tcpdump and see the session > initialization. > > Thoughts? > > Tahnks. > > John. > > -----Original Message----- > From: bro-bounces(a)ICSI.Berkeley.EDU > [mailto:bro-bounces@ICSI.Berkeley.EDU] On Behalf Of Bill Jones > Sent: Saturday, February 06, 2010 10:22 AM > To: bro(a)ICSI.Berkeley.EDU > Subject: [Bro] Load Balancers > > Hi everyone, > > I was curious if anyone has any experience running bro between > load-balancers (such as Netscaler) and web applications. We are > currently trying to get HTTP logs generated for a web application. We > couldn't figure out why bro was not triggering the HTTP analyzer, but > I now believe that this is because it is never seeing the original SYN > + SYN/ACK for the conversation. When viewing the conversations in > Wireshark, I can see that all the TCP streams for this particular > application begin with the GET and do not include the initial 3-way > handshake. > > Here is an entry in the conn.log for this stream which shows the states: > > 1265389087.849048 ? 10.19.120.12 10.19.2.78 http 2232 80 tcp 14785 > 604140 OTH X DdAa > > Other web applications on the wire, which do have the 3-way handshake > visible for all connections, seem to work just fine and I get http > logs. > > My questions are: > > Am I correct in assuming that the lack of initial connection > establishment is why the HTTP analysis is never occurring (and > therefore I'm not getting entries in http.log)? > > Is there a way to force bro to analyze the traffic even though there > is no proper 3-way handshake visible? > > > Thanks for your time, > Bill > _______________________________________________ > Bro mailing list > bro(a)bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >

9 years, 11 months

[Bro] ICMP Redirect patch

by j.sentier206

Hello, Here is a small and quick patch I made to Bro to get icmp_redirect events with the redirection address. Hope it will be useful to you. Best regards, Julien Une messagerie gratuite, garantie à vie et des services en plus, ça vous tente ? Je crée ma boîte mail www.laposte.net

9 years, 11 months

[Bro] Debian pkg and packet loss metrics

by Louis F Ruppert

Hey, Two questions, too lazy to write two messages: 1. There was some talk about bro 1.6 having the means to easily generate Debian packages with cmake. I see scripts for generating rpm packages and scripts for generating mac packages, but nothing for Debian. Does anyone have pointers on how to do this, or should I plan on generating something in house? 2. How are people non-intrusively measuring packet loss in their clusters? I can get a vague idea of what bro is losing via netstats, but I'd hate to interfere with cluster operations by running it every few minutes. We also split our taps with Click router, so that still wouldn't tell me what, if anything, Click or the kernel itself are losing. (For the record, the kernel and Click both claim to be losing close to nothing, while bro loses ~3-4%, based on a few tests I've done.) Thanks, -Lou

9 years, 11 months

[Bro] 6to4 Tunnelling

by Will

On another note, I know there is alot of progress being made on bro compatibility with IPv6. Are there any groups using bro to detect 6to4 tunnelling or "Teredo"? So, if your network has some devices that are configured to run IPv6 through Toredo (or "need" to for some reason or another?!?), then blocking 3544 isn't acceptable and isn't a great solution regardless. I am wondering if it would be possible to inspect IPv4 UDP traffic for wrapped IPv6 packets. Has anyone looked into this already or doing it? If so, whitelisting known hosts that are allowed to send tunnelled traffic would be trivial. Thanks in advance. -Will Side note: Is "tunnelling" spelled with one "L" or two? Or optional? http://www.merriam-webster.com/dictionary/tunnelling On Thu, May 12, 2011 at 1:19 PM, Will <baxterw3232(a)gmail.com> wrote: > On Wed, May 11, 2011 at 12:54 PM, Aashish SHARMA <aashish043(a)gmail.com> wrote: >> Hello: >> >> HTTP_WatchedMIMEType is declared in bro/share/bro/http-identified-files.bro. >> >> I think you can make the code work by doing the following changes in the http-ext-identified-files.bro >> >> 1) Load http-identified-files >> 2) change "const" to "redef" for the following variables: watched_mime_types, ignored_urls, mime_types_extensions, ignored_signatures >> 3) Comment out declaration of HTTP_IncorrectFileType from http-ext-identified-files.bro >> >> >> + @load http-identified-files >> >> - redef enum Notice += { >> - # This notice is thrown when the file extension doesn't >> - # seem to match the file contents. >> - HTTP_IncorrectFileType, >> - }; >> >> - const watched_mime_types = /application\/x-dosexec/ >> + redef watched_mime_types = /application\/x-dosexec/ >> >> >> - const ignored_urls = /^http:\/\/(au\.|www\.)?download\.windowsupdate\.com\/msdownload\/update/ &redef; >> + redef ignored_urls = /^http:\/\/(au\.|www\.)?download\.windowsupdate\.com\/msdownload\/update/ ; >> >> >> - redef mime_types_extensions: table[string] of pattern = { >> + const mime_types_extensions: table[string] of pattern = { >> >> >> - const ignored_signatures += /^matchfile-/ &redef; >> + redef ignored_signatures += /^matchfile-/; >> >> Aashish >> >> On May 11, 2011, at 6:18 AM, Seth Hall wrote: >> >>> Sorry for not reply earlier. I started a response to your email and never finished it. :) >>> >>> On Apr 1, 2011, at 2:20 PM, Will wrote: >>> >>>> 1. The old way of flagging via 'HTTP_WatchedMIMEType' appears to have gone away >>> >>> Hm, I wonder why I removed that? There will be a solution for this problem in the next release. >>> >>> Did you end up figuring out what was wrong with this? >>> > > Yes, pretty close to what Aashish describes to do above. Though I > don't see what changing the ignored_signatures file does, because it > already looks redef'd. Our "whitelist" is larger and slightly more > custom to our environment, but otherwise just as below. The > mis-matched file type is great for when a file is down loaded with a > random string and doesn't have a "watched" mime type, i.e. a php file > named "WJ4JR874". > > Here is what we are using and seems to be working seemlessly: > > @load global-ext > @load http-ext > @load http-reply > @load http-body > @load signatures > redef signature_files += "http-ext-identified-files.sig"; > > module HTTP; > > export { > redef enum Notice += { > # This notice is thrown when the file extension doesn't > # seem to match the file contents. > HTTP_IncorrectFileType, > > # Generated when we see a MIME type we flagged for watching. > HTTP_WatchedMIMEType, > }; > > # MIME types that you'd like this script to identify and log. > const watched_mime_types = /application\/x-dosexec/ > | /application\/x-executable/ > | /application\/octet-stream/ > | /application\/x-compressed/ > | /application\/x-msdownload/ &redef; > > # URLs included here are not logged and notices are not thrown. > # Take care when defining regexes to not be overly broad. > const ignored_urls = > /^http:\/\/(au\.|www\.)?download\.windowsupdate\.com\/msdownload\/update/ > | /^http:\/\/.*\.adobe\.com\// > | /^http:\/\/.*\.cisco\.com\// > | /^http:\/\/.*\.hp\.com\// > | /^http:\/\/.*\.macromedia\.com\// > | /^http:\/\/.*\.microsoft\.com\// > | /^http:\/\/.*\.sun\.com\// &redef; > > # Create regexes that *should* in be in the urls for specifics > mime types. > # Notices are thrown if the pattern doesn't match the url for > the file type. > const mime_types_extensions: table[string] of pattern = { > ["application/x-dosexec"] = /\.([eE][xX][eE]|[dD][lL][lL])/, > } &redef; > } > > # Don't delete the http sessions at the end of the request! > redef watch_reply=T; > > # Ignore the signatures used to match files > redef ignored_signatures += /^matchfile-/; > > # This script uses the file tagging method to create a separate file. > event bro_init() > { > # Add the tag for log file splitting. > LOG::define_tag("http-ext", "identified-files"); > } > > event signature_match(state: signature_state, msg: string, data: string) > { > # Only signatures matching file types are dealt with here. > if ( /^matchfile/ !in state$id ) return; > > # Not much point in any of this if we don't know about the > # HTTP-ness of the connection. > if ( state$conn$id !in conn_info ) return; > > local si = conn_info[state$conn$id]; > # Set the mime type seen. > si$mime_type = msg; > local defanged_url = gsub(si$url, /\./, "[.]"); > local message = fmt("%s %s", msg, defanged_url); > if ( ignored_urls !in si$url ) > { > if ( watched_mime_types in msg ) > { > NOTICE([$note=HTTP_WatchedMIMEType, > $msg=message, $conn=state$conn, $method=si$method, $URL=si$url]); > # Add a tag for logging purposes. > add si$tags["identified-files"]; > } > > if ( msg in mime_types_extensions && > mime_types_extensions[msg] !in si$url ) > { > NOTICE([$note=HTTP_IncorrectFileType, > $msg=message, $conn=state$conn, $method=si$method, $URL=si$url]); > } > > event file_transferred(state$conn, data, "", msg); > } > } > > Thanks to both! > > -Will > >>> .Seth >>> >>> -- >>> Seth Hall >>> International Computer Science Institute >>> (Bro) because everyone has a network >>> http://www.bro-ids.org/ >>> >>> >>> _______________________________________________ >>> Bro mailing list >>> bro(a)bro-ids.org >>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >> >> >

9 years, 11 months

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

zeek May 2011