Hi all,
Recently I have some problems with Bro and PF_RING in cluster.
On my server, when I have less than 32 worker threads(rings),
everything is okay, but when I use worker threads more than 32, pf_ring
start to receive repeating data packets. For example, rings less than 32, I
send 400000 packets to server and pf_ring info in /proc shows there is
400000 packets in rings, but when rings greater than 32, I can get 800000
packets when 33 rings and 1200000 packets when 34 rings and so on.
I guess if there is some rules that a pf_ring or a bro cluster can only
support less than 32 rings or worker threads on a server or some other
reasons?
Any insight would be helpful.
Has anyone given any thought as to the possiblity of using a compressed
file analyzer to open and detect embedded flash files in docx files, or
macros in the same? I realize that that means we need a file analyzer
first, but I have been thinking about alternate use cases for the analyzer,
and this one sprung to mind...
I was wondering if anyone can tell me why the sha256 hash functionality
isn't turned on by default for the files log.
I am working on something and needed to turn it on. I normally only use Bro
to process pcap files offline and have never used it on a live network.
Does it cause performance issues?
Thanks,
Shawn
Hi there,
I wrote a new analyzer with BinPAC for a protocol named 'AMS'.
Somehow when I create the analyzer via the binpac python script and name
the analyzer 'AMS' or 'ams', the analyzer won't work. When I name it
'TEST' or 'test', it works fine (same protocol specification, C++ Code,
etc.)
Is there a name convention for new analyzer? Or does anyone know, why
BinPAC/Bro won't accept the name 'ams'?
Thank you!
Where is this line defined? what file would i define this once i create sub
folders for file types? I wish to get cron to compress specific folders to
save disk space.
Files::add_analyzer(f, Files::ANALYZER_EXTRACT, [$extract_filename=fname]);
Cheers,
John
On Tue, Nov 29, 2016 at 7:00 AM, <bro-request(a)bro.org> wrote:
> Send Bro mailing list submissions to
> bro(a)bro.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> or, via email, send a message with subject or body 'help' to
> bro-request(a)bro.org
>
> You can reach the person managing the list at
> bro-owner(a)bro.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Bro digest..."
>
>
> Today's Topics:
>
> 1. Re: Bro 2.5 CPU usage (Drew Dixon)
> 2. File extraction in different directories (maybe day vise)
> (fatema bannatwala)
> 3. Re: File extraction in different directories (maybe day vise)
> (Hosom, Stephen M)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Mon, 28 Nov 2016 13:19:47 -0500
> From: Drew Dixon <dwdixon(a)umich.edu>
> Subject: Re: [Bro] Bro 2.5 CPU usage
> To: Daniel Thayer <dnthayer(a)illinois.edu>
> Cc: bro(a)bro.org
> Message-ID:
> <CA+pNqTOFXBseTFHdQqKM2r=FyZEGUH9Gj4O7i=Sn9PoZLbU-3w@
> mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> Would it be possible for someone quantify what a low bandwidth/low traffic
> setup might be in terms of a bandwidth unit of measurement range where
> Justin's patch would be advised to be used? I.E. Kbps/Mbps etc. What would
> be a cut-off bandwidth/traffic rate value where it would not be advisable
> that this patch be used?
>
> On Fri, Nov 25, 2016 at 1:54 PM, Daniel Thayer <dnthayer(a)illinois.edu>
> wrote:
>
> > Regarding broctl, you can disable the "not seeing any packets"
> > warnings if you set this in your etc/broctl.cfg:
> > StatsLogEnable = 0
> >
> > Doing so will also disable logging to broctl's stats.log (note:
> > this is NOT the stats.log that Bro itself logs), which I'm
> > guessing most people don't need anyway.
> >
> >
> > On 11/25/16 11:43 AM, Michael Shirk wrote:
> > > Is this something worthy of a feature request for low bandwidth setups?
> > >
> > > In addition to something like this, I have to do a patch for very low
> > > network traffic with bro cron reporting network traffic has stopped on
> > > the monitoring interface.
> > >
> > > --
> > > Michael Shirk
> > > Daemon Security, Inc.
> > > http://www.daemon-security.com
> > _______________________________________________
> > Bro mailing list
> > bro(a)bro-ids.org
> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> >
>
My site was an early tester of 2.5_beta, and during most of our testing, we
did not experience much in the way of issues related to log rotation. As
soon as we started using 2.5_beta we were utilizing logger mode, as we run
an extremely busy cluster, and had been experiencing memory utilization
exhaustion issues with the manager being single threaded At one point near
the end of the 2.5 beta, we started seeing some irregular rotation
intervals, which we noted were potentially related to us starting to use
the Intel framework.
I have since moved to 2.5 stable, and I'm still seeing irregular rotation
issues which appear to be related to how busy the manager is at the time of
log rotation. I do notice that the main local-manager process seems to stay
loaded at or near 100% CPU utilization when we are seeing a lot of traffic.
As an example, I rotated logs perfectly from midnight up to 07:00-08:00.
After that, my current logs in spool/logger are running well over into 9
o'clocks data set. I frequently see .rotated files well after a log
rotation interval has run (the following was observed 09:41 today):
-rw-r--r-- 1 root root 18 Nov 30 08:00 .rotated.capture_loss
-rw-r--r-- 1 root root 18 Nov 30 08:00 .rotated.communication
-rw-r--r-- 1 root root 18 Nov 30 08:00 .rotated.conn
-rw-r--r-- 1 root root 18 Nov 30 08:00 .rotated.conn-summary
-rw-r--r-- 1 root root 18 Nov 30 08:00 .rotated.dce_rpc
-rw-r--r-- 1 root root 18 Nov 30 03:00 .rotated.dhcp
-rw-r--r-- 1 root root 18 Nov 30 08:00 .rotated.dns
-rw-r--r-- 1 root root 18 Nov 30 08:00 .rotated.dpd
-rw-r--r-- 1 root root 18 Nov 30 08:00 .rotated.files
-rw-r--r-- 1 root root 18 Nov 30 08:00 .rotated.ftp
-rw-r--r-- 1 root root 18 Nov 30 08:00 .rotated.http
-rw-r--r-- 1 root root 18 Nov 30 08:00 .rotated.intel
-rw-r--r-- 1 root root 18 Nov 30 08:00 .rotated.irc
-rw-r--r-- 1 root root 18 Nov 30 08:00 .rotated.kerberos
-rw-r--r-- 1 root root 18 Nov 30 08:00 .rotated.known_certs
-rw-r--r-- 1 root root 18 Nov 30 08:00 .rotated.known_hosts
-rw-r--r-- 1 root root 18 Nov 30 08:00 .rotated.known_services
-rw-r--r-- 1 root root 18 Nov 29 22:00 .rotated.loaded_scripts
-rw-r--r-- 1 root root 18 Nov 30 08:00 .rotated.mysql
-rw-r--r-- 1 root root 18 Nov 30 08:00 .rotated.notice
-rw-r--r-- 1 root root 18 Nov 29 22:00 .rotated.packet_filter
-rw-r--r-- 1 root root 18 Nov 30 08:00 .rotated.pe
-rw-r--r-- 1 root root 18 Nov 30 08:00 .rotated.radius
-rw-r--r-- 1 root root 18 Nov 30 08:00 .rotated.rdp
-rw-r--r-- 1 root root 18 Nov 29 23:00 .rotated.reporter
-rw-r--r-- 1 root root 18 Nov 30 08:00 .rotated.rfb
-rw-r--r-- 1 root root 18 Nov 30 08:00 .rotated.sip
-rw-r--r-- 1 root root 18 Nov 30 08:00 .rotated.smtp
-rw-r--r-- 1 root root 18 Nov 30 08:00 .rotated.snmp
-rw-r--r-- 1 root root 18 Nov 30 01:00 .rotated.socks
-rw-r--r-- 1 root root 18 Nov 30 08:00 .rotated.software
-rw-r--r-- 1 root root 18 Nov 30 08:00 .rotated.ssh
-rw-r--r-- 1 root root 18 Nov 30 08:00 .rotated.ssl
-rw-r--r-- 1 root root 18 Nov 30 08:00 .rotated.stats
-rw-r--r-- 1 root root 18 Nov 30 08:00 .rotated.syslog
-rw-r--r-- 1 root root 18 Nov 30 08:00 .rotated.tunnel
-rw-r--r-- 1 root root 18 Nov 30 08:00 .rotated.weird
-rw-r--r-- 1 root root 18 Nov 30 08:00 .rotated.x509
Any tips or pointers on how to track down the culprit here and get things
back to normally scheduled log rotation intervals?
FWIW, I'm running on a brand new R730xd with SSD's for the OS, an NVMe
drive for the Bro installation, and RAID 10 & RAID 5 respectively for the
60 day archive and long term storage archive volumes. I'm linting my intel
files to make sure they are properly formatted, and there doesn't seem to
be an issue here...
Respectfully,
-Erin Shelton
Program Manager: Incident Response and Network Security
Office of Information Technology
University of Colorado Boulder
Good afternoon,
To confirm, with the Team Cymru MHR feature enabled, the warnings and
VirusTotal URL outputs would appear in the Bro files.log?
Thanks,
Is there a way, when logging in JSON, to get a readable connection summary
log. When logging in JSON the log is unusable and the tables included in
the log do not get populated. I like the log because it gives a great
overview of the sensors.
Thanks
Tim
I setup a Bro instance on a Raspberry Pi3 with an WLAN monitor interface, for IDS home use.
I got notices and with the
hook Notice::policy(n: Notice::Info)
{
add n$actions[Notice::ACTION_EMAIL];
}
config (example from the mailing list) into my local.bro.I got notices by mail. Works fine.
I also installed critical stack intel feeds, and when I see an Intel file created when I test a banned ip address.
I am new to Bro and have no knowlegde about Bro configuration and scripting language. But I want to make a quickstart
I have two questions:
1: How can the intel also get mailed, when an intel event occurs?
I tried
redef Notice::emailed_types += {
HTTP::IN_HOST_HEADER,
};
Config check is ok but after triggering an intel event I got no mail.
2: I want to incorporate a Bash curl script send alerts to other systems when a notice or an intel event occurs. How to accomplish this?
Thanks in advance.
is it possible that below described statement can be crafted into a bro
script ?
Plz help me if it is possible, let me know what i need to do, to make this
possible.
If my incoming traffic rate exceeds 44Mbps and the average incoming traffic
rate over the last 504seconds exceeds the average incoming traffic rate
over the last 965seconds by more than 70%, send an alert
Thank you Everyone.
MeetGill
