Dear Bro Developer and contributer
I have been working on IDS and Data Mining .I would like to know the current
research in this area that IDS using Data Mining and what are the current
reseach areas and objectives where Data Mining can provide solutions to IDS?
--
Regards
SUMAN KUMAR NANDI
HOD-Computer Applications
Chitkara University, Punjab
India
Mobile:+919501105658
Have you always wanted to work on Bro and be paid for it? :-)
NCSA has an opening for a Senior Research Programmer:
https://jobs.illinois.edu/default.cfm?page=job&jobID=8847&returnPage=sear
This is a full-time position that focuses on Bro development in close
collaboration with our group here at ICSI.
Robin
--
Robin Sommer * Phone +1 (510) 722-6541 * robin(a)icir.org
ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org
Hello, All
I am trying to use the policy script http-rewriter.bro in Bro-1.5.1 to
anonymize the HTTP message-body of all HTTP packets in a big dumped trace
larger than 100GB ( http-rewriter.bro actually deletes all HTTP message-body
and add one new header field named X-Actual-Data-Length, right?) .
I am not sure if Bro itself and http-rewriter.bro has the ability of
reordering all tcp packets and deleting tcp retransmitted packets in every
connection of the dumped trace?
If they cannot do that, whether I can reorder all packets and delete the
retransmitted packets in every connection first by using some tools and then
use http-rewriter.bro ? Is this way reasonable? What's your suggestion
about the tools I can use?
Besides, I want to test if special HTTP packets exist. Special packet here
means there are more than one HTTP construct(headers + message body) in one
packet. When using http-rewriter.bro on several special pakcets I created,
it seems that it can delete the message-body correctly for almost all of
cases as long as the packets in the connection are in order and complete.
Can http-rewriter.bro handle the special cases correctly as what I found?
Expect your answer and thank you very much.
Song Zhao
Hi,
Well, it depend on what are you trying to do?
For example, do you want to use data mining in IDS alerts analysis (e.g. alerts verification, alerts aggregation, alerts correlation)? In this case you will find a lot of research work submitted in that area.
Do you want to use data mining in building IDS to detect intrusions? Then you probably taking about anomaly detection based IDS? In my opinion data mining is not the best approach to do that. Probably, you will need to think about soft computing approaches (neural network, artificial immune system, swarm intelligence, etc).
The issue with your question is that you are using very abstract keywords "data mining" and "IDS" . you should be more specific.
Thanks,
Sherif Saad
Ph.D Candidate, University of Victoria
--- On Mon, 5/30/11, bro-request(a)bro-ids.org <bro-request(a)bro-ids.org> wrote:
From: bro-request(a)bro-ids.org <bro-request(a)bro-ids.org>
Subject: Bro Digest, Vol 61, Issue 16
To: bro(a)bro-ids.org
Received: Monday, May 30, 2011, 10:00 PM
Send Bro mailing list submissions to
bro(a)bro-ids.org
To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
or, via email, send a message with subject or body 'help' to
bro-request(a)bro-ids.org
You can reach the person managing the list at
bro-owner(a)bro-ids.org
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Bro digest..."
Today's Topics:
1. Current IDS and Data Mining research (Suman Nandi)
2. Re: handle out of order and retransmitted packets in offline
trace (Song Zhao)
----------------------------------------------------------------------
Message: 1
Date: Mon, 30 May 2011 11:53:57 +0530
From: Suman Nandi <suman.nandi(a)chitkara.edu.in>
Subject: [Bro] Current IDS and Data Mining research
To: bro(a)bro-ids.org
Message-ID: <BANLkTi=pfrxkgD6SzOmN5yrejS3G5MDJxg(a)mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"
Dear Bro Developer and contributer
I have been working on IDS and Data Mining .I would like to know the current
research in this area that IDS using Data Mining and what are the current
reseach areas and objectives where Data Mining can provide solutions to IDS?
--
Regards
SUMAN KUMAR NANDI
HOD-Computer Applications
Chitkara University, Punjab
India
Mobile:+919501105658
Thank you Aaron for your suggestions :) I will look into them.
Regards,
Harkeerat Bedi
On Thu, May 26, 2011 at 4:29 AM, Aaron Glenn <aaron.glenn(a)gmail.com> wrote:
> On Wed, May 25, 2011 at 11:25 PM, Harkeerat Bedi <hsbedi(a)memphis.edu>
> wrote:
> > a way we can accomplish this using BRO?
>
> BRO is not what you want in this scenario
>
> > Or, is there any other way?
> > Kindly suggest.
>
> netgraph in freebsd
> pf in openbsd
> npf in netbsd
> I urge you try them all (:
>
> > Thank you,
> > Harkeerat Bedi
>
> best of luck,
> aaron glenn
>
Hello All,
I am using BRO for a part of my project. Following is what I intend to do:
1. Monitor UDP connections.
2. Compute their bitrates
3. Throttle the bitrates of these UDP connections based on some
calculations.
I was able to complete tasks 1 and 2. However I don't know how I can
accomplish task 3.
My current setup includes a Client (Node1) sending UDP data to a Server
(Node3). The traffic has to pass through a Gateway (Node2) which is in
between the Client and Server and is running BRO.
Node1 (Client) <------> Node2 (running BRO) < ------ > Node3 (Server)
If I have a UDP connection (between the Client and the Server) with a bit
rate of 2Mb/s. How can I reduce its bitrate to a user set value - say:
1Mb/s, using BRO?
I am assuming one way may be to drop packets of a connection with a certain
frequency such that the overall bitrate of that connection reduces? Is there
a way we can accomplish this using BRO?
Or, is there any other way?
Kindly suggest.
Thank you,
Harkeerat Bedi
John,
Thanks for the quick response.
> Not sure what Netscalar does, but it all should act the same. The host
> TCP stack would drop any attempted connection for which a session was
> not established regardless of what was upstream from it. Quick and
> dirty, you sould be able to fire up tcpdump and see the session
> initialization.
That's what I'm finding strange. After running a tcpdump capture on
the interface and analyzing it with Wireshark, I do not see any 3-way
handshakes for this particular web application. For any HTTP GET that
I see in Wireshark that pertains to this application, when I "Follow
TCP Stream", the first entry in Wireshark is always the GET message
itself. For all other applications on the network, doing the above
results in the first entry being the SYN.
I've generated a few dumps with the same results. I wonder if the
load balancer is somehow keeping a session active for very long
periods (if this even makes sense).
If you have any suggestions or thoughts, I'd be very interested.
Thanks,
Bill
On Sat, Feb 6, 2010 at 12:51 PM, John Hally <JHally(a)ebscohost.com> wrote:
> Hi Bill,
>
> I've run BRO in the past with load balancers (Arrowpoint/Cisco CSS) and
> was able to see all traffic. In our setup we had 2 segments; a VIP
> access link and a services trunk link where the real/origin servers
> lived. Both of these links had physical network taps and it was as
> simple as plugging in the Ethernet, flipping the interface to
> UP/PROMISC, and starting BRO.
>
> With the CSS, even though the unit would handle the initial connection,
> it would 'snap' that over to the origin server it picked during load
> balancing so you would still see the tcp setup.
>
> Not sure what Netscalar does, but it all should act the same. The host
> TCP stack would drop any attempted connection for which a session was
> not established regardless of what was upstream from it. Quick and
> dirty, you sould be able to fire up tcpdump and see the session
> initialization.
>
> Thoughts?
>
> Tahnks.
>
> John.
>
> -----Original Message-----
> From: bro-bounces(a)ICSI.Berkeley.EDU
> [mailto:bro-bounces@ICSI.Berkeley.EDU] On Behalf Of Bill Jones
> Sent: Saturday, February 06, 2010 10:22 AM
> To: bro(a)ICSI.Berkeley.EDU
> Subject: [Bro] Load Balancers
>
> Hi everyone,
>
> I was curious if anyone has any experience running bro between
> load-balancers (such as Netscaler) and web applications. We are
> currently trying to get HTTP logs generated for a web application. We
> couldn't figure out why bro was not triggering the HTTP analyzer, but
> I now believe that this is because it is never seeing the original SYN
> + SYN/ACK for the conversation. When viewing the conversations in
> Wireshark, I can see that all the TCP streams for this particular
> application begin with the GET and do not include the initial 3-way
> handshake.
>
> Here is an entry in the conn.log for this stream which shows the states:
>
> 1265389087.849048 ? 10.19.120.12 10.19.2.78 http 2232 80 tcp 14785
> 604140 OTH X DdAa
>
> Other web applications on the wire, which do have the 3-way handshake
> visible for all connections, seem to work just fine and I get http
> logs.
>
> My questions are:
>
> Am I correct in assuming that the lack of initial connection
> establishment is why the HTTP analysis is never occurring (and
> therefore I'm not getting entries in http.log)?
>
> Is there a way to force bro to analyze the traffic even though there
> is no proper 3-way handshake visible?
>
>
> Thanks for your time,
> Bill
> _______________________________________________
> Bro mailing list
> bro(a)bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
Hello,
Here is a small and quick patch I made to Bro to get icmp_redirect events with the redirection address.
Hope it will be useful to you.
Best regards,
Julien
Une messagerie gratuite, garantie à vie et des services en plus, ça vous tente ?
Je crée ma boîte mail www.laposte.net
Hey,
Two questions, too lazy to write two messages:
1. There was some talk about bro 1.6 having the means to easily generate
Debian packages with cmake. I see scripts for generating rpm packages
and scripts for generating mac packages, but nothing for Debian. Does
anyone have pointers on how to do this, or should I plan on generating
something in house?
2. How are people non-intrusively measuring packet loss in their
clusters? I can get a vague idea of what bro is losing via netstats,
but I'd hate to interfere with cluster operations by running it every
few minutes. We also split our taps with Click router, so that still
wouldn't tell me what, if anything, Click or the kernel itself are
losing. (For the record, the kernel and Click both claim to be losing
close to nothing, while bro loses ~3-4%, based on a few tests I've
done.)
Thanks,
-Lou
On another note, I know there is alot of progress being made on bro
compatibility with IPv6. Are there any groups using bro to detect 6to4
tunnelling or "Teredo"?
So, if your network has some devices that are configured to run IPv6
through Toredo (or "need" to for some reason or another?!?), then
blocking 3544 isn't acceptable and isn't a great solution regardless.
I am wondering if it would be possible to inspect IPv4 UDP traffic for
wrapped IPv6 packets. Has anyone looked into this already or doing it?
If so, whitelisting known hosts that are allowed to send tunnelled
traffic would be trivial.
Thanks in advance.
-Will
Side note:
Is "tunnelling" spelled with one "L" or two? Or optional?
http://www.merriam-webster.com/dictionary/tunnelling
On Thu, May 12, 2011 at 1:19 PM, Will <baxterw3232(a)gmail.com> wrote:
> On Wed, May 11, 2011 at 12:54 PM, Aashish SHARMA <aashish043(a)gmail.com> wrote:
>> Hello:
>>
>> HTTP_WatchedMIMEType is declared in bro/share/bro/http-identified-files.bro.
>>
>> I think you can make the code work by doing the following changes in the http-ext-identified-files.bro
>>
>> 1) Load http-identified-files
>> 2) change "const" to "redef" for the following variables: watched_mime_types, ignored_urls, mime_types_extensions, ignored_signatures
>> 3) Comment out declaration of HTTP_IncorrectFileType from http-ext-identified-files.bro
>>
>>
>> + @load http-identified-files
>>
>> - redef enum Notice += {
>> - # This notice is thrown when the file extension doesn't
>> - # seem to match the file contents.
>> - HTTP_IncorrectFileType,
>> - };
>>
>> - const watched_mime_types = /application\/x-dosexec/
>> + redef watched_mime_types = /application\/x-dosexec/
>>
>>
>> - const ignored_urls = /^http:\/\/(au\.|www\.)?download\.windowsupdate\.com\/msdownload\/update/ &redef;
>> + redef ignored_urls = /^http:\/\/(au\.|www\.)?download\.windowsupdate\.com\/msdownload\/update/ ;
>>
>>
>> - redef mime_types_extensions: table[string] of pattern = {
>> + const mime_types_extensions: table[string] of pattern = {
>>
>>
>> - const ignored_signatures += /^matchfile-/ &redef;
>> + redef ignored_signatures += /^matchfile-/;
>>
>> Aashish
>>
>> On May 11, 2011, at 6:18 AM, Seth Hall wrote:
>>
>>> Sorry for not reply earlier. I started a response to your email and never finished it. :)
>>>
>>> On Apr 1, 2011, at 2:20 PM, Will wrote:
>>>
>>>> 1. The old way of flagging via 'HTTP_WatchedMIMEType' appears to have gone away
>>>
>>> Hm, I wonder why I removed that? There will be a solution for this problem in the next release.
>>>
>>> Did you end up figuring out what was wrong with this?
>>>
>
> Yes, pretty close to what Aashish describes to do above. Though I
> don't see what changing the ignored_signatures file does, because it
> already looks redef'd. Our "whitelist" is larger and slightly more
> custom to our environment, but otherwise just as below. The
> mis-matched file type is great for when a file is down loaded with a
> random string and doesn't have a "watched" mime type, i.e. a php file
> named "WJ4JR874".
>
> Here is what we are using and seems to be working seemlessly:
>
> @load global-ext
> @load http-ext
> @load http-reply
> @load http-body
> @load signatures
> redef signature_files += "http-ext-identified-files.sig";
>
> module HTTP;
>
> export {
> redef enum Notice += {
> # This notice is thrown when the file extension doesn't
> # seem to match the file contents.
> HTTP_IncorrectFileType,
>
> # Generated when we see a MIME type we flagged for watching.
> HTTP_WatchedMIMEType,
> };
>
> # MIME types that you'd like this script to identify and log.
> const watched_mime_types = /application\/x-dosexec/
> | /application\/x-executable/
> | /application\/octet-stream/
> | /application\/x-compressed/
> | /application\/x-msdownload/ &redef;
>
> # URLs included here are not logged and notices are not thrown.
> # Take care when defining regexes to not be overly broad.
> const ignored_urls =
> /^http:\/\/(au\.|www\.)?download\.windowsupdate\.com\/msdownload\/update/
> | /^http:\/\/.*\.adobe\.com\//
> | /^http:\/\/.*\.cisco\.com\//
> | /^http:\/\/.*\.hp\.com\//
> | /^http:\/\/.*\.macromedia\.com\//
> | /^http:\/\/.*\.microsoft\.com\//
> | /^http:\/\/.*\.sun\.com\// &redef;
>
> # Create regexes that *should* in be in the urls for specifics
> mime types.
> # Notices are thrown if the pattern doesn't match the url for
> the file type.
> const mime_types_extensions: table[string] of pattern = {
> ["application/x-dosexec"] = /\.([eE][xX][eE]|[dD][lL][lL])/,
> } &redef;
> }
>
> # Don't delete the http sessions at the end of the request!
> redef watch_reply=T;
>
> # Ignore the signatures used to match files
> redef ignored_signatures += /^matchfile-/;
>
> # This script uses the file tagging method to create a separate file.
> event bro_init()
> {
> # Add the tag for log file splitting.
> LOG::define_tag("http-ext", "identified-files");
> }
>
> event signature_match(state: signature_state, msg: string, data: string)
> {
> # Only signatures matching file types are dealt with here.
> if ( /^matchfile/ !in state$id ) return;
>
> # Not much point in any of this if we don't know about the
> # HTTP-ness of the connection.
> if ( state$conn$id !in conn_info ) return;
>
> local si = conn_info[state$conn$id];
> # Set the mime type seen.
> si$mime_type = msg;
> local defanged_url = gsub(si$url, /\./, "[.]");
> local message = fmt("%s %s", msg, defanged_url);
> if ( ignored_urls !in si$url )
> {
> if ( watched_mime_types in msg )
> {
> NOTICE([$note=HTTP_WatchedMIMEType,
> $msg=message, $conn=state$conn, $method=si$method, $URL=si$url]);
> # Add a tag for logging purposes.
> add si$tags["identified-files"];
> }
>
> if ( msg in mime_types_extensions &&
> mime_types_extensions[msg] !in si$url )
> {
> NOTICE([$note=HTTP_IncorrectFileType,
> $msg=message, $conn=state$conn, $method=si$method, $URL=si$url]);
> }
>
> event file_transferred(state$conn, data, "", msg);
> }
> }
>
> Thanks to both!
>
> -Will
>
>>> .Seth
>>>
>>> --
>>> Seth Hall
>>> International Computer Science Institute
>>> (Bro) because everyone has a network
>>> http://www.bro-ids.org/
>>>
>>>
>>> _______________________________________________
>>> Bro mailing list
>>> bro(a)bro-ids.org
>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>
>>
>
