Greetings
I am integrating bro into a larger system so that I can use it to keep track
of connections (which seems easier than trying to write a method from
scratch with pcap). I thought it would be straightforward to grep for print
or email or alarm statements to figure out where to put the hooks for an IPC
message but so far it eludes me. Is there a principal module for outputting
the notifications?
Thanks
Mike
Hi,
I want know , if i can work with paquets of connexion (i.e)
I want recover know if it possibleto recover with bro informations
( duration , protocol, flag, service, ..) before connexion would be
finished.
Thant you for any help.
> 1) In the DNS policy file there is an event for "dns_EDNS_addl" what
> part of the packet is this field in a DNS connection
EDNS is a general mechanism for specifying extensions to DNS.
> and what is the
> "pldsize" value from?
It comes from the framing provided by the EDNS mechanism.
> Is there a way to break out the data from this field?
No, though if there are specific EDNS extensions you're interested in,
we'd certainly encourage you to consider adding analysis for them to
the event engine (in DNS.cc).
> 2) When a DNS record has "DNS_SEC_OK" What is that from the packet connection?
That's also part of EDNS (the 'Z' field), and specifes that the extension
accepts DNSSEC RRs.
Vern
If reduce-memory isn't helping, then most likely the culprit is state you
are building up in script variables. You can generate lightweight periodic
script statistics by @load'ing stats.bro, or heavier-weight and more detailed
using profiling.bro. You can also see the sizes of your script variables
using the function global_sizes() (see for example print-globals.bro, which
simply calls this when Bro exits). Often a way to hone in on which variable
is getting large is to either run on a trace and use print-globals to dump
the sizes when Bro finishes, or set up a timer to print out global_sizes()
every minute or so.
Vern
Hi,
I tried out the suggestion of @load'ing reduce-memory.bro (I appended a
line "@load reduce-memory" at the end of the file mt.bro. I also have a
"@load rotate-logs" line in addition to the mt.bro that comes with the
distribution.), but I still see memory usage increasing monotonically.
Below is a snapshot of memory usage every 10 minutes. Should I be
@load'ing reduce-memory.bro in some other way? If not, is there anything
else I can do to prevent continuous accumulation of state?
Please let me know if any of you have any other suggestions.
Thanks!
Harsha
top - 16:57:19 up 58 days, 2:42, 2 users, load average: 0.55, 0.52, 0.45
9567 root 15 0 13780 9.9m 2608 S 4.0 0.5 0:00.73 bro
top - 17:07:19 up 58 days, 2:52, 2 users, load average: 0.80, 0.73, 0.56
9567 root 15 0 25208 21m 2676 R 15.7 1.0 1:23.21 bro
top - 17:17:20 up 58 days, 3:02, 2 users, load average: 0.78, 0.71, 0.61
9567 root 15 0 30700 26m 2688 S 13.8 1.3 2:51.30 bro
top - 17:27:20 up 58 days, 3:12, 2 users, load average: 0.85, 0.74, 0.65
9567 root 15 0 36796 32m 2688 S 19.8 1.6 4:21.41 bro
top - 17:37:21 up 58 days, 3:23, 2 users, load average: 0.83, 0.71, 0.65
9567 root 15 0 43208 38m 2688 S 11.8 1.9 5:47.97 bro
top - 17:47:22 up 58 days, 3:33, 2 users, load average: 0.56, 0.66, 0.65
9567 root 15 0 47760 43m 2688 S 13.8 2.1 7:03.52 bro
top - 17:57:22 up 58 days, 3:43, 2 users, load average: 0.77, 0.63, 0.61
9567 root 16 0 52308 47m 2696 S 7.9 2.3 8:20.43 bro
top - 18:07:23 up 58 days, 3:53, 2 users, load average: 0.58, 0.62, 0.61
9567 root 15 0 56592 51m 2696 S 13.8 2.5 9:31.94 bro
top - 18:17:23 up 58 days, 4:03, 2 users, load average: 0.56, 0.55, 0.56
9567 root 15 0 62036 56m 2696 S 11.9 2.8 10:47.16 bro
top - 18:27:24 up 58 days, 4:13, 2 users, load average: 0.85, 0.65, 0.57
9567 root 15 0 66576 61m 2696 S 11.9 3.0 12:01.87 bro
top - 18:37:24 up 58 days, 4:23, 2 users, load average: 0.73, 0.65, 0.58
9567 root 15 0 73496 68m 2696 R 11.9 3.4 13:24.69 bro
top - 18:47:25 up 58 days, 4:33, 2 users, load average: 0.50, 0.65, 0.62
9567 root 15 0 77644 72m 2696 S 11.9 3.6 14:43.69 bro
top - 18:57:25 up 58 days, 4:43, 2 users, load average: 0.44, 0.48, 0.54
9567 root 15 0 82576 76m 2696 S 13.8 3.8 16:03.00 bro
......
top - 06:58:03 up 58 days, 16:43, 2 users, load average: 0.51, 0.51, 0.52
9567 root 15 0 349m 345m 2696 S 13.8 17.1 153:08.79 bro
top - 07:08:03 up 58 days, 16:53, 2 users, load average: 0.70, 0.66, 0.57
9567 root 15 0 353m 349m 2696 S 25.7 17.3 155:17.46 bro
top - 07:18:04 up 58 days, 17:03, 2 users, load average: 0.72, 0.67, 0.58
9567 root 15 0 356m 352m 2696 S 27.7 17.4 157:27.63 bro
top - 07:28:04 up 58 days, 17:13, 2 users, load average: 0.43, 0.53, 0.54
9567 root 15 0 365m 361m 2696 R 37.5 17.8 159:45.66 bro
top - 07:38:05 up 58 days, 17:23, 2 users, load average: 0.42, 0.56, 0.54
9567 root 15 0 365m 361m 2696 S 21.7 17.8 162:05.27 bro
top - 07:48:05 up 58 days, 17:33, 2 users, load average: 0.66, 0.52, 0.52
9567 root 15 0 369m 365m 2696 S 11.8 18.1 164:24.92 bro
top - 07:58:06 up 58 days, 17:43, 2 users, load average: 0.70, 0.42, 0.46
9567 root 15 0 372m 368m 2696 S 21.8 18.2 166:41.43 bro
top - 08:08:06 up 58 days, 17:53, 2 users, load average: 0.56, 0.56, 0.49
9567 root 15 0 376m 372m 2696 R 27.7 18.4 169:01.29 bro
top - 08:18:07 up 58 days, 18:03, 2 users, load average: 0.63, 0.52, 0.49
9567 root 15 0 379m 375m 2696 S 19.7 18.5 171:16.30 bro
top - 08:28:07 up 58 days, 18:13, 2 users, load average: 0.75, 0.66, 0.56
9567 root 16 0 383m 379m 2696 S 19.8 18.7 173:36.79 bro
top - 08:38:08 up 58 days, 18:23, 2 users, load average: 1.01, 0.73, 0.60
9567 root 15 0 392m 388m 2696 S 19.7 19.2 176:06.30 bro
top - 08:48:08 up 58 days, 18:33, 2 users, load average: 0.35, 0.54, 0.57
9567 root 15 0 392m 388m 2696 S 27.7 19.2 178:34.94 bro
top - 08:58:09 up 58 days, 18:43, 2 users, load average: 0.85, 0.68, 0.62
9567 root 15 0 394m 391m 2696 R 27.7 19.3 181:01.10 bro
Vern Paxson wrote:
> This will commonly occur simply due to state building up in the variables
> managed by the event engine and the policy scripts. The main problem is
> the need to associate timeouts with the corresponding tables. See our paper:
>
> H. Dreger, A. Feldmann, V. Paxson, and R. Sommer,
> Operational Experiences with High-Volume Network Intrusion Detection,
> Proc. ACM CCS, October 2004
>
> http://www.icir.org/vern/papers/high-volume-ccs04.pdf
>
> for discussion.
>
> You can turn on a bunch (though not an exhaustive set) of these sorts of
> timeouts by @load'ing reduce-memory.bro. Soon we will change Bro so that
> by default it includes this sort of configuration, rather than the user
> needing to enable it specifically.
>
> Vern
Hi,
I have noticed that Bro can provide the user with a fine grained classification
of alarms in the reports (likely unsuccessful, likely successful, ...).
However, in the log, Bro provides me with a less specific classification (alarm
vs no alarm) with no indication of the potential success (or failure) of the
attack. I think that the events in the log correspond to likely successful
attacks only (correct me if I am wrong).
I am wondering if there is any way to get Bro to output all events in the log
WITH their classification (likely successful, likely unsuccessful, ...) or if
this feature is reserved specifically for reports ?
Thank you very much!
---
François Gagnon
Hi Christian,
I had another question that should hopefully be simple.
1) In the DNS policy file there is an event for "dns_EDNS_addl" what part of the packet is this field in a DNS connection and what is the "pldsize" value from? Is there a way to break out the data from this field?
2) When a DNS record has "DNS_SEC_OK" What is that from the packet connection?
Thanks,
Jake
-------------- Original message ----------------------
From: Christian Kreibich <christian(a)whoop.org>
> Hi Jake,
>
> On Tue, 2006-03-21 at 14:57 +0000, jbabbin(a)comcast.net wrote:
> > List,
> > I have a couple of questions that I can't seem to figure out.
> >
> > 1) Brian - Thanks for the SSL patch
> > Once enabled I don't see any way of filtering out hosts from the
> > non-ssl traffic alarm. For example, I have several custom applications
> > that use that port for their traffic...don't ask...so I need to be
> > able to filter them out of the alarms like below.
> >
> > "1141848057.399932 WeirdActivity ** 192.x.x.x/48612 > 206.x.x.x/https:
> > SSL: Skipping connection (not an SSL connection?!)!"
> >
> > The problem seems to be that the detection of non-ssl traffic is done
> > in the source SSLProxy engine and I don't really want to be
> > recompiling every time I need to add another host. Ideas?
>
> have a look at weird_ignore_host set, defined in weird.bro. It allows
> you to filter weird-type events based on the event string and source/
> destination IP addresses.
>
> http://www.bro-ids.org/Bro-reference-manual/weird-variables.html#weird-vari…
>
> Depending on your analysis needs, you could also exclude the custom
> traffic via the pcap filtering expression, though I'd imagine that
> quickly gets tedious.
>
> > 2) Is is possible in a policy file to perform a size comparison on a
> > string?
> > For example, if you wanted to see if a filename was longer than a
> > certain length. How would you sizeof a string value?
>
> Sure. It depends on what version of Bro you're using. In the development
> releases, there's now a magnitude operator |x| that, when given a value,
> returns its length, size, or whatever is most meaningful as magnitude
> (vector length, table size, string length, etc). In older releases (0.9
> and before), the byte_len() function returned a string's length.
>
> Cheers,
> Christian.
> --
> ________________________________________________________________________
> http://www.cl.cam.ac.uk/~cpk25
> http://www.whoop.org
>
> _______________________________________________
> Bro mailing list
> bro(a)bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
This will commonly occur simply due to state building up in the variables
managed by the event engine and the policy scripts. The main problem is
the need to associate timeouts with the corresponding tables. See our paper:
H. Dreger, A. Feldmann, V. Paxson, and R. Sommer,
Operational Experiences with High-Volume Network Intrusion Detection,
Proc. ACM CCS, October 2004
http://www.icir.org/vern/papers/high-volume-ccs04.pdf
for discussion.
You can turn on a bunch (though not an exhaustive set) of these sorts of
timeouts by @load'ing reduce-memory.bro. Soon we will change Bro so that
by default it includes this sort of configuration, rather than the user
needing to enable it specifically.
Vern
Hi,
We have been having some problems with bro. We have been running bro for
a couple of days and we see that the memory usage of bro keeps
increasing monotonically. Bro eventually uses up all the memory on the
machine and crashes. Considering that we are running bro on a machine
with 2GB of memory, I guess this is not expected behavior.
A similar problem was reported way back in 1999, and Vern had then
proposed a fix.
http://mailman.icsi.berkeley.edu/pipermail/bro/1999-April/000813.htmlhttp://mailman.icsi.berkeley.edu/pipermail/bro/1999-May/000818.html
But, we are using version 0.9a11 (the latest stable version available),
which already includes the above fix.
Has anyone else had similar memory problems? Is this a known problem?
Please let me know if you would like a snapshot of the data we are
receiving to diagnose the problem. Below is more information about the
system on which we are running bro and the increasing memory usage that
we notice.
Any help would much appreciated.
Thanks!
Harsha
System config:
Intel Xeon CPU 3.06GHz
2GB RAM
Linux 2.6.11-1.1369_FC4smp
Bro executed as:
sudo ./src/bro -i eth0 mt
Snapshot of bro's memory usage every 10 minutes since it is started:
top - 15:59:54 up 56 days, 1:45, 2 users, load average: 0.02, 0.02, 0.12
31919 root 15 0 35836 31m 2664 S 5.9 1.6 0:06.01 bro
top - 16:09:54 up 56 days, 1:55, 2 users, load average: 0.37, 0.19, 0.12
31919 root 15 0 77988 72m 2684 S 7.9 3.6 0:50.23 bro
top - 16:19:55 up 56 days, 2:05, 2 users, load average: 0.19, 0.20, 0.16
31919 root 15 0 86072 80m 2684 S 9.9 4.0 1:40.03 bro
top - 16:29:55 up 56 days, 2:15, 2 users, load average: 0.11, 0.12, 0.12
31919 root 15 0 90276 84m 2692 S 5.9 4.2 2:31.14 bro
top - 16:39:56 up 56 days, 2:25, 2 users, load average: 0.18, 0.16, 0.12
31919 root 15 0 105m 101m 2692 S 15.8 5.0 3:27.62 bro
top - 16:49:56 up 56 days, 2:35, 2 users, load average: 0.10, 0.12, 0.09
31919 root 15 0 105m 101m 2692 S 5.9 5.0 4:21.60 bro
top - 16:59:57 up 56 days, 2:45, 2 users, load average: 0.02, 0.08, 0.08
31919 root 15 0 105m 101m 2692 S 9.9 5.0 5:13.48 bro
top - 17:09:57 up 56 days, 2:55, 2 users, load average: 0.02, 0.08, 0.08
31919 root 15 0 105m 101m 2692 S 5.9 5.0 6:01.54 bro
top - 17:19:58 up 56 days, 3:05, 2 users, load average: 0.04, 0.08, 0.08
31919 root 15 0 105m 101m 2692 S 11.9 5.0 6:56.49 bro
top - 17:29:58 up 56 days, 3:15, 2 users, load average: 0.18, 0.17, 0.11
31919 root 15 0 111m 107m 2692 R 11.9 5.3 7:54.42 bro
top - 17:39:59 up 56 days, 3:25, 2 users, load average: 0.16, 0.12, 0.09
31919 root 15 0 111m 108m 2692 S 7.9 5.3 8:49.19 bro
top - 17:49:59 up 56 days, 3:35, 2 users, load average: 0.19, 0.21, 0.14
31919 root 15 0 120m 116m 2692 S 7.9 5.7 9:43.56 bro
top - 18:00:00 up 56 days, 3:45, 2 users, load average: 0.08, 0.10, 0.10
31919 root 15 0 120m 117m 2692 R 23.7 5.8 10:41.71 bro
top - 18:10:00 up 56 days, 3:55, 2 users, load average: 0.20, 0.12, 0.09
31919 root 15 0 120m 117m 2692 S 7.9 5.8 11:35.80 bro
top - 18:20:01 up 56 days, 4:05, 2 users, load average: 0.45, 0.26, 0.15
31919 root 16 0 123m 119m 2692 R 7.9 5.9 12:28.89 bro
top - 18:30:01 up 56 days, 4:15, 2 users, load average: 0.13, 0.14, 0.14
31919 root 15 0 131m 128m 2692 R 9.9 6.3 13:25.72 bro
top - 18:40:02 up 56 days, 4:25, 2 users, load average: 0.20, 0.16, 0.12
31919 root 15 0 138m 134m 2692 S 7.9 6.6 14:25.36 bro
top - 18:50:02 up 56 days, 4:35, 2 users, load average: 0.22, 0.15, 0.10
31919 root 15 0 142m 139m 2692 S 11.9 6.9 15:37.64 bro
top - 19:00:03 up 56 days, 4:45, 2 users, load average: 0.11, 0.15, 0.11
31919 root 15 0 142m 139m 2692 S 11.8 6.9 16:51.76 bro
List,
I have a couple of questions that I can't seem to figure out.
1) Brian - Thanks for the SSL patch
Once enabled I don't see any way of filtering out hosts from the non-ssl traffic alarm. For example, I have several custom applications that use that port for their traffic...don't ask...so I need to be able to filter them out of the alarms like below.
"1141848057.399932 WeirdActivity ** 192.x.x.x/48612 > 206.x.x.x/https: SSL: Skipping connection (not an SSL connection?!)!"
The problem seems to be that the detection of non-ssl traffic is done in the source SSLProxy engine and I don't really want to be recompiling every time I need to add another host. Ideas?
2) Is is possible in a policy file to perform a size comparison on a string?
For example, if you wanted to see if a filename was longer than a certain length. How would you sizeof a string value?
Thanks in advance,
Jake Babbin
