Hello,
I would like to bring your attention to one strange and unexpected problem
I faced
while trying to include user agent parser for bro I have developed (plugin:
https://github.com/vitalyrepin/uap-bro ) into the list of uap
implementations https://github.com/ua-parser
The req
--
WBR & WBW, Vitaly
To pick up the idea that you mentioned before - do we also want to make
the new policy/protocols/smb/__load__.bro trigger a reporter warning
that it is deprecated?
Johanna
On 30 Aug 2018, at 14:07, Jonathan Siwek wrote:
> Repository : ssh://git@bro-ids.icir.org/bro
> On branch : master
> Link :
> https://github.com/bro/bro/commit/57a505b0e46d499644a6fb3b063cece0684240b8
>
>> ---------------------------------------------------------------
>
> commit 57a505b0e46d499644a6fb3b063cece0684240b8
> Author: Jon Siwek <jsiwek(a)corelight.com>
> Date: Thu Aug 30 16:05:36 2018 -0500
>
> Allow loading policy/protocols/smb once again
>
> It just redirects to base/protocols/smb
>
>
>> ---------------------------------------------------------------
>
> 57a505b0e46d499644a6fb3b063cece0684240b8
> CHANGES | 4 ++++
> NEWS | 8 ++++++--
> VERSION | 2 +-
> scripts/policy/protocols/smb/__load__.bro | 1 +
> scripts/test-all-policy.bro | 1 +
> 5 files changed, 13 insertions(+), 3 deletions(-)
>
> diff --git a/CHANGES b/CHANGES
> index af31bdea0..15184aa4a 100644
> --- a/CHANGES
> +++ b/CHANGES
> @@ -1,4 +1,8 @@
>
> +2.5-947 | 2018-08-30 16:05:36 -0500
> +
> + * Allow loading policy/protocols/smb once again (Jon Siwek,
> Corelight)
> +
> 2.5-946 | 2018-08-30 09:51:16 -0500
>
> * Update NEWS with more info about runtime options (Daniel Thayer)
> diff --git a/NEWS b/NEWS
> index 0af51ef60..86839427b 100644
> --- a/NEWS
> +++ b/NEWS
> @@ -267,8 +267,12 @@ New Functionality
>
> - Added new NFS events: nfs_proc_symlink, nfs_proc_link,
> nfs_proc_sattr.
>
> -- The SMB scripts in policy/protocols/smb are now moved into
> base/protocols/smb
> - and loaded/enabled by default.
> +- The SMB scripts in policy/protocols/smb are now moved into
> + base/protocols/smb and loaded/enabled by default. If you
> previously
> + loaded these scripts from their policy/ location (in local.bro or
> + other custom scripts) you may now remove/change those although they
> + should still work since policy/protocols/smb is simply a
> placeholder
> + script that redirects to the new base/ location.
>
> - Added new SMB events: smb1_transaction_secondary_request,
> smb1_transaction2_secondary_request, smb1_transaction_response.
> diff --git a/VERSION b/VERSION
> index d522ba4d6..ecd34e707 100644
> --- a/VERSION
> +++ b/VERSION
> @@ -1 +1 @@
> -2.5-946
> +2.5-947
> diff --git a/scripts/policy/protocols/smb/__load__.bro
> b/scripts/policy/protocols/smb/__load__.bro
> new file mode 100644
> index 000000000..8fd733d38
> --- /dev/null
> +++ b/scripts/policy/protocols/smb/__load__.bro
> @@ -0,0 +1 @@
> +@load base/protocols/smb
> diff --git a/scripts/test-all-policy.bro b/scripts/test-all-policy.bro
> index 11824c2c6..d31da6573 100644
> --- a/scripts/test-all-policy.bro
> +++ b/scripts/test-all-policy.bro
> @@ -82,6 +82,7 @@
> @load protocols/modbus/track-memmap.bro
> @load protocols/mysql/software.bro
> @load protocols/rdp/indicate_ssl.bro
> +@load protocols/smb/__load__.bro
> @load protocols/smb/log-cmds.bro
> @load protocols/smtp/blocklists.bro
> @load protocols/smtp/detect-suspicious-orig.bro
>
>
>
> _______________________________________________
> bro-commits mailing list
> bro-commits(a)bro.org
> http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-commits
Hello Everyone,
I am reaching out with the hope that someone will be able to help us with an issue we are having with Bro upgrade from 2.4.1 to 2.5.X.
We have a system with 12 core (3Ghz) ,128GB RAM, and 10G NIC (Intel X520-SR2 10GbE Dual-port), monitoring between 1.5 - 2.5 Gbps traffic.
Bro 2.4.1 is working great and periodically drops 2-5% when traffic peaks at ~ 2.5. However, when we upgrade to Bro 2.5.3/4 on the same exact system the drops go up to 90%.
We are using CentOS-7 and tired installing Bro and Pfring from both rpm and source without any luck. I wonder if anyone has seen this issue and can give some clues to resolve this issue.
Bro Node Conf:
[manager]
type=manager
host=localhost
#
[proxy-1]
type=proxy
host=localhost
#
[worker-1]
type=worker
host=localhost
interface=ens1f1
lb_method=pf_ring
lb_procs=11
pin_cpus=1,2,3,4,5,6,7,8,9,10,11
[root@bro-test ~]# cat /proc/net/pf_ring/info
PF_RING Version : 7.3.0 (unknown)
Total rings : 11
Standard (non ZC) Options
Ring slots : 65534
Slot version : 17
Capture TX : No [RX only]
IP Defragment : No
Socket Mode : Standard
Cluster Fragment Queue : 0
Cluster Fragment Discard : 0
[root@bro-test ~]# tailf /opt/bro/logs/current/capture_loss.log
1535647921.339324 60.000005 worker-1-8 318331 425005 74.900531
1535647921.217853 60.000000 worker-1-5 264716 349078 75.832908
1535647921.241244 60.000021 worker-1-9 265863 364089 73.021432
1535647921.312567 60.000002 worker-1-1 239036 315823 75.686698
1535647922.188607 60.000420 worker-1-4 238192 322818 73.785229
1535647922.760560 60.000029 worker-1-11 250678 338188 74.12386
1535647922.864470 60.000075 worker-1-3 232467 314963 73.807717
1535647923.413121 60.000024 worker-1-10 254241 345382 73.611537
1535647923.205954 60.001556 worker-1-2 259932 354980 73.224407
[root@bro-test ~]# less /opt/bro/logs/current/stats.log | bro-cut ts peer mem pkts_proc bytes_recv pkts_dropped
1535644801.328981 worker-1-8 2854 3523252 2214563854 8841163
1535644801.235592 worker-1-9 2833 3422300 2135680645 9083143
1535644801.299138 worker-1-1 2801 3358673 2089659287 9059868
1535644802.177016 worker-1-4 2727 3262089 2027645336 9155838
1535644801.187590 worker-1-5 2640 3336190 2085853940 9332917
1535644802.750617 worker-1-11 2726 3432674 2153405372 9018943
1535644802.853617 worker-1-3 2816 3448836 2161753414 8929662
1535644803.186853 worker-1-2 2659 3387742 2116043509 9176871
1535644803.395256 worker-1-10 2871 3407486 2132043052 9049047
1535644803.403778 worker-1-7 2821 3278503 2023604941 9966347
1535644850.898433 manager 2340 0 0 -
1535644804.257320 proxy-1 73 0 0 -
[root@bro-test logs]# broctl netstats
worker-1-1: 1535651356.794609 recvd=3501813131 dropped=3589205826 link=3501813131
worker-1-2: 1535651358.808626 recvd=4033892471 dropped=3057179730 link=4033892471
worker-1-3: 1535651358.587316 recvd=3930325145 dropped=3160768660 link=3930325145
worker-1-4: 1535651357.702299 recvd=3561053809 dropped=3530086444 link=3561053809
worker-1-5: 1535651357.650359 recvd=3399338460 dropped=3691836209 link=3399338460
worker-1-6: 1535651334.912244 recvd=3714154738 dropped=3376978237 link=3714154738
worker-1-7: 1535651359.119492 recvd=3684804437 dropped=3406432666 link=3684804437
worker-1-8: 1535651359.668621 recvd=4020016563 dropped=3071265083 link=4020016563
worker-1-9: 1535651359.867601 recvd=3807658264 dropped=3283669188 link=3807658264
worker-1-10: 1535651359.749253 recvd=3703077938 dropped=3388277853 link=3703077938
worker-1-11: 1535651359.907420 recvd=4052516305 dropped=3038874387 link=4052516305
nload output for capture NIC:
[cid:image001.png@01D4407C.0E3A9670]
Jawad Rajput
System Administrator
U.S. Department of Energy
IM-62 /Germantown Building
HQ Network Security Team
Email: Jawad.Rajput(a)hq.doe.gov<mailto:Jawad.Rajput@hq.doe.gov>
Office: 301-903-2176
Office: 301-903-3895
Upgrading between master builds I just ran into this:
fatal error in /bro/share/bro/site/local.bro, line 88: can't open /bro/share/bro/policy/protocols/smb/__load__.bro
I see in NEWS we have
- The SMB scripts in policy/protocols/smb are now moved into base/protocols/smb
and loaded/enabled by default.
But should there be an empty script in there or something that does a reporter warning telling people to update local.bro?
—
Justin Azoff
On Tue, Aug 28, 2018 at 6:35 PM Johanna Amann <johanna(a)icir.org> wrote:
> + If you use these events, you can make your scripts work on old and new versions
> + of Bro by wrapping the event definition in an @if, for example:
> +
> + @if ( Version::at_least("2.6") || ( Version::number == 20500 && Version::info$commit >= [commit number of change] ) )
> + event ssl_client_hello(c: connection, version: count, record_version: count, possible_ts: time, client_random: string, session_id: string, ciphers: index_vec, comp_methods: index_vec)
> + @else
> + event ssl_client_hello(c: connection, version: count, possible_ts: time, client_random: string, session_id: string, ciphers: index_vec)
> + @endif
Since the parser won't be happy with that type of @if usage in old
releases due to [1], should we instead suggest something like:
function my_ssl_client_hello_impl(c: connection, version: count,
possible_ts: time, client_random: string, session_id: string, ciphers:
index_vec, record_version: counter &default=0, comp_methods: index_vec
&default=index_vec())
{
# Copy existing code to here
}
@if ( Version::at_least("2.6") || ( Version::number == 20500 &&
Version::info$commit >= [commit number of change] ) )
event ssl_client_hello(c: connection, version: count, record_version:
count, possible_ts: time, client_random: string, session_id: string,
ciphers: index_vec, comp_methods: index_vec)
{ my_ssl_client_hello_impl(c, version, possible_ts, client_random,
session_id, ciphers, record_version, comp_methods); }
@else
event ssl_client_hello(c: connection, version: count, possible_ts:
time, client_random: string, session_id: string, ciphers: index_vec)
{ my_ssl_client_hello_impl(c, version, possible_ts, client_random,
session_id, ciphers); }
@endif
- Jon
[1] https://bro-tracker.atlassian.net/browse/BIT-1976
Hi,
when I go to tracker.bro.org, the top-right box (Filter result) for me
shows:
"The filter configured for this gadget could not be retrieved. Please
verify it is still valid on the issue navigator.". This seems to be
independent of Browser. I think this used to show the merge-requests.
Can someone perhaps fix that again? :)
Thanks,
Johanna
We are currently writing code for ingesting data directly using Broker’s API. From the docs, it seems that Broker assumes that publishers and subscribers somehow agree on one layout per topic: "senders and receivers will need to agree on a specific data layout for the values exchanged, so that they interpret them in the same way.” [1]
This raises a couple of questions. Primarily: where can Broker users learn the layouts to interpret received data? There’s essentially no hope in deferring a layout, since broker::data doesn’t include any meta information such as field names. Is meta information stored somewhere? Is there a convention how to place and retrieve such meta information in Broker’s data stores? How does Bro itself make such information available? Is there a document that lists all topics used by Bro with their respective broker::data layout?
Dominik
[1] https://bro-broker.readthedocs.io/en/stable/comm.html#exchanging-bro-events <https://bro-broker.readthedocs.io/en/stable/comm.html#exchanging-bro-events>
Hi All,
I am in the process of writing parser for the DNSSEC RR types in DNS
responses, and written RRSIG (type=46) parser by adding code to existing
DNS protocol analyzer in Bro 2.5.4 src code.
I have tested the code by recompiling it on our test server and running it
against a dns pcap, and it correctly parses the RRSIG record and logs it.
And hence have requested a Pull request to merge in the upstream Bro master
repo .
Planning to write the remaining DNSSEC RR types: NSEC, DS and DNSKEY
parsing in Bro DNS analyzer as well, once I get the feedback on the current
merge request of the code for parsing RRSIG.
Thanks,
Fatema.
Johanna mentioned to me that libmaxminddb should be working now in master...
So far I haven't been able to get 'configure' to find it, neither with the
OS packages nor by installing libmaxminddb in /usr/local/ and specifying
--with-geoip.
This is CentOS 7.5.
-Dop
Or: How 13 billion became 1.844674e+19 before becoming 0.
After sending amounts totaling over 13 billion thru Sumstats, a value of 0
came out the other end of the sausage factory, but only for one specific
data item. Debugging this required lots of well placed print statements,
and wild speculation on my part as to what possibly could be broken....
The values being thrown in to be summed are incremental differences from a
previous observation, which *should* be zero or a positive number in the
range of several K, so a 'count' variable was used. However, for some
reason, this value came up negative (or should have) due to (in decreasing
likelihood) logic error in script, one of bro's dark corners, or bro bug.
The reason for the negativity is still TBD. But, in the world of unsigned
64-bit values (aka bro 'count' variables) there is no negativity, only
positivity, and an unsigned underflow creates a number just below 2**64 ~
1.844674e+19 ....
Well, Sumstats tallies in doubles, and naturally this figure (1.844674e+19)
dominated the total. In fact, additional increments to this total pushed
the total value to be greater than 2**64 (with loss of precision, as
doubles only keep 53 bits).
In the processing step at the Sumstats epoch, the value was converted back
to a count using the double_to_count() function which the cheatsheet warns
returns 0, if the double value is <0.0, but it actually returns
2**64-double (with a runtime error), and for values > 2**64 it returns 0
with no runtime error :-(
So, there it is, a value that should have been about 13 billion became
1.844674e+19 and then became 0.
A few suggestions:
1. Conversion routines should saturate at respective minima/maxima of the
type being converted to (possibly with runtime error).
2. Underflow of the 'count' type is almost invariably a bug, and should
trigger a runtime error. Overflow, similarly, although in practice it
seems much less likely to occur as most scripts are dealing with integers
considerably less than 2**64. A similar argument could be made for 'int'.
With some operations, it is difficult to detect overflow/underflow, but for
simple add and subtract, it is relatively easy.
3. Documentation to match behavior.
Jim