zeek-dev August 2018

zeek-dev@lists.zeek.org

13 participants
17 discussions

[Bro-Dev] "bro" name is non-inclusive? (Contribution is rejected based )

by Vitaly Repin

Hello, I would like to bring your attention to one strange and unexpected problem I faced while trying to include user agent parser for bro I have developed (plugin: https://github.com/vitalyrepin/uap-bro ) into the list of uap implementations https://github.com/ua-parser The req -- WBR & WBW, Vitaly

2 years, 4 months

Re: [Bro-Dev] [Bro-Commits] [git/bro] master: Allow loading policy/protocols/smb once again (57a505b0e)

by Johanna Amann

To pick up the idea that you mentioned before - do we also want to make the new policy/protocols/smb/__load__.bro trigger a reporter warning that it is deprecated? Johanna On 30 Aug 2018, at 14:07, Jonathan Siwek wrote: > Repository : ssh://git@bro-ids.icir.org/bro > On branch : master > Link : > https://github.com/bro/bro/commit/57a505b0e46d499644a6fb3b063cece0684240b8 > >> --------------------------------------------------------------- > > commit 57a505b0e46d499644a6fb3b063cece0684240b8 > Author: Jon Siwek <jsiwek(a)corelight.com> > Date: Thu Aug 30 16:05:36 2018 -0500 > > Allow loading policy/protocols/smb once again > > It just redirects to base/protocols/smb > > >> --------------------------------------------------------------- > > 57a505b0e46d499644a6fb3b063cece0684240b8 > CHANGES | 4 ++++ > NEWS | 8 ++++++-- > VERSION | 2 +- > scripts/policy/protocols/smb/__load__.bro | 1 + > scripts/test-all-policy.bro | 1 + > 5 files changed, 13 insertions(+), 3 deletions(-) > > diff --git a/CHANGES b/CHANGES > index af31bdea0..15184aa4a 100644 > --- a/CHANGES > +++ b/CHANGES > @@ -1,4 +1,8 @@ > > +2.5-947 | 2018-08-30 16:05:36 -0500 > + > + * Allow loading policy/protocols/smb once again (Jon Siwek, > Corelight) > + > 2.5-946 | 2018-08-30 09:51:16 -0500 > > * Update NEWS with more info about runtime options (Daniel Thayer) > diff --git a/NEWS b/NEWS > index 0af51ef60..86839427b 100644 > --- a/NEWS > +++ b/NEWS > @@ -267,8 +267,12 @@ New Functionality > > - Added new NFS events: nfs_proc_symlink, nfs_proc_link, > nfs_proc_sattr. > > -- The SMB scripts in policy/protocols/smb are now moved into > base/protocols/smb > - and loaded/enabled by default. > +- The SMB scripts in policy/protocols/smb are now moved into > + base/protocols/smb and loaded/enabled by default. If you > previously > + loaded these scripts from their policy/ location (in local.bro or > + other custom scripts) you may now remove/change those although they > + should still work since policy/protocols/smb is simply a > placeholder > + script that redirects to the new base/ location. > > - Added new SMB events: smb1_transaction_secondary_request, > smb1_transaction2_secondary_request, smb1_transaction_response. > diff --git a/VERSION b/VERSION > index d522ba4d6..ecd34e707 100644 > --- a/VERSION > +++ b/VERSION > @@ -1 +1 @@ > -2.5-946 > +2.5-947 > diff --git a/scripts/policy/protocols/smb/__load__.bro > b/scripts/policy/protocols/smb/__load__.bro > new file mode 100644 > index 000000000..8fd733d38 > --- /dev/null > +++ b/scripts/policy/protocols/smb/__load__.bro > @@ -0,0 +1 @@ > +@load base/protocols/smb > diff --git a/scripts/test-all-policy.bro b/scripts/test-all-policy.bro > index 11824c2c6..d31da6573 100644 > --- a/scripts/test-all-policy.bro > +++ b/scripts/test-all-policy.bro > @@ -82,6 +82,7 @@ > @load protocols/modbus/track-memmap.bro > @load protocols/mysql/software.bro > @load protocols/rdp/indicate_ssl.bro > +@load protocols/smb/__load__.bro > @load protocols/smb/log-cmds.bro > @load protocols/smtp/blocklists.bro > @load protocols/smtp/detect-suspicious-orig.bro > > > > _______________________________________________ > bro-commits mailing list > bro-commits(a)bro.org > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-commits

2 years, 6 months

[Bro-Dev] Bro 2.5 Packet Drop Issue

by Rajput, Jawad (CONTR)

Hello Everyone, I am reaching out with the hope that someone will be able to help us with an issue we are having with Bro upgrade from 2.4.1 to 2.5.X. We have a system with 12 core (3Ghz) ,128GB RAM, and 10G NIC (Intel X520-SR2 10GbE Dual-port), monitoring between 1.5 - 2.5 Gbps traffic. Bro 2.4.1 is working great and periodically drops 2-5% when traffic peaks at ~ 2.5. However, when we upgrade to Bro 2.5.3/4 on the same exact system the drops go up to 90%. We are using CentOS-7 and tired installing Bro and Pfring from both rpm and source without any luck. I wonder if anyone has seen this issue and can give some clues to resolve this issue. Bro Node Conf: [manager] type=manager host=localhost # [proxy-1] type=proxy host=localhost # [worker-1] type=worker host=localhost interface=ens1f1 lb_method=pf_ring lb_procs=11 pin_cpus=1,2,3,4,5,6,7,8,9,10,11 [root@bro-test ~]# cat /proc/net/pf_ring/info PF_RING Version : 7.3.0 (unknown) Total rings : 11 Standard (non ZC) Options Ring slots : 65534 Slot version : 17 Capture TX : No [RX only] IP Defragment : No Socket Mode : Standard Cluster Fragment Queue : 0 Cluster Fragment Discard : 0 [root@bro-test ~]# tailf /opt/bro/logs/current/capture_loss.log 1535647921.339324 60.000005 worker-1-8 318331 425005 74.900531 1535647921.217853 60.000000 worker-1-5 264716 349078 75.832908 1535647921.241244 60.000021 worker-1-9 265863 364089 73.021432 1535647921.312567 60.000002 worker-1-1 239036 315823 75.686698 1535647922.188607 60.000420 worker-1-4 238192 322818 73.785229 1535647922.760560 60.000029 worker-1-11 250678 338188 74.12386 1535647922.864470 60.000075 worker-1-3 232467 314963 73.807717 1535647923.413121 60.000024 worker-1-10 254241 345382 73.611537 1535647923.205954 60.001556 worker-1-2 259932 354980 73.224407 [root@bro-test ~]# less /opt/bro/logs/current/stats.log | bro-cut ts peer mem pkts_proc bytes_recv pkts_dropped 1535644801.328981 worker-1-8 2854 3523252 2214563854 8841163 1535644801.235592 worker-1-9 2833 3422300 2135680645 9083143 1535644801.299138 worker-1-1 2801 3358673 2089659287 9059868 1535644802.177016 worker-1-4 2727 3262089 2027645336 9155838 1535644801.187590 worker-1-5 2640 3336190 2085853940 9332917 1535644802.750617 worker-1-11 2726 3432674 2153405372 9018943 1535644802.853617 worker-1-3 2816 3448836 2161753414 8929662 1535644803.186853 worker-1-2 2659 3387742 2116043509 9176871 1535644803.395256 worker-1-10 2871 3407486 2132043052 9049047 1535644803.403778 worker-1-7 2821 3278503 2023604941 9966347 1535644850.898433 manager 2340 0 0 - 1535644804.257320 proxy-1 73 0 0 - [root@bro-test logs]# broctl netstats worker-1-1: 1535651356.794609 recvd=3501813131 dropped=3589205826 link=3501813131 worker-1-2: 1535651358.808626 recvd=4033892471 dropped=3057179730 link=4033892471 worker-1-3: 1535651358.587316 recvd=3930325145 dropped=3160768660 link=3930325145 worker-1-4: 1535651357.702299 recvd=3561053809 dropped=3530086444 link=3561053809 worker-1-5: 1535651357.650359 recvd=3399338460 dropped=3691836209 link=3399338460 worker-1-6: 1535651334.912244 recvd=3714154738 dropped=3376978237 link=3714154738 worker-1-7: 1535651359.119492 recvd=3684804437 dropped=3406432666 link=3684804437 worker-1-8: 1535651359.668621 recvd=4020016563 dropped=3071265083 link=4020016563 worker-1-9: 1535651359.867601 recvd=3807658264 dropped=3283669188 link=3807658264 worker-1-10: 1535651359.749253 recvd=3703077938 dropped=3388277853 link=3703077938 worker-1-11: 1535651359.907420 recvd=4052516305 dropped=3038874387 link=4052516305 nload output for capture NIC: [cid:image001.png@01D4407C.0E3A9670] Jawad Rajput System Administrator U.S. Department of Energy IM-62 /Germantown Building HQ Network Security Team Email: Jawad.Rajput(a)hq.doe.gov<mailto:Jawad.Rajput@hq.doe.gov> Office: 301-903-2176 Office: 301-903-3895

2 years, 6 months

[Bro-Dev] Compatibilty script for policy/protocols/smb?

by Azoff, Justin S

Upgrading between master builds I just ran into this: fatal error in /bro/share/bro/site/local.bro, line 88: can't open /bro/share/bro/policy/protocols/smb/__load__.bro I see in NEWS we have - The SMB scripts in policy/protocols/smb are now moved into base/protocols/smb and loaded/enabled by default. But should there be an empty script in there or something that does a reporter warning telling people to update local.bro? — Justin Azoff

2 years, 6 months

Re: [Bro-Dev] [Bro-Commits] [git/bro] topic/johanna/tls-more-data: Update NEWS for ssl changes. (3c7c60cf6)

by Jon Siwek

On Tue, Aug 28, 2018 at 6:35 PM Johanna Amann <johanna(a)icir.org> wrote: > + If you use these events, you can make your scripts work on old and new versions > + of Bro by wrapping the event definition in an @if, for example: > + > + @if ( Version::at_least("2.6") || ( Version::number == 20500 && Version::info$commit >= [commit number of change] ) ) > + event ssl_client_hello(c: connection, version: count, record_version: count, possible_ts: time, client_random: string, session_id: string, ciphers: index_vec, comp_methods: index_vec) > + @else > + event ssl_client_hello(c: connection, version: count, possible_ts: time, client_random: string, session_id: string, ciphers: index_vec) > + @endif Since the parser won't be happy with that type of @if usage in old releases due to [1], should we instead suggest something like: function my_ssl_client_hello_impl(c: connection, version: count, possible_ts: time, client_random: string, session_id: string, ciphers: index_vec, record_version: counter &default=0, comp_methods: index_vec &default=index_vec()) { # Copy existing code to here } @if ( Version::at_least("2.6") || ( Version::number == 20500 && Version::info$commit >= [commit number of change] ) ) event ssl_client_hello(c: connection, version: count, record_version: count, possible_ts: time, client_random: string, session_id: string, ciphers: index_vec, comp_methods: index_vec) { my_ssl_client_hello_impl(c, version, possible_ts, client_random, session_id, ciphers, record_version, comp_methods); } @else event ssl_client_hello(c: connection, version: count, possible_ts: time, client_random: string, session_id: string, ciphers: index_vec) { my_ssl_client_hello_impl(c, version, possible_ts, client_random, session_id, ciphers); } @endif - Jon [1] https://bro-tracker.atlassian.net/browse/BIT-1976

2 years, 6 months

[Bro-Dev] Jira filter results

by Johanna Amann

Hi, when I go to tracker.bro.org, the top-right box (Filter result) for me shows: "The filter configured for this gadget could not be retrieved. Please verify it is still valid on the issue navigator.". This seems to be independent of Browser. I think this used to show the merge-requests. Can someone perhaps fix that again? :) Thanks, Johanna

2 years, 6 months

[Bro-Dev] Broker data layouts

by Dominik Charousset

We are currently writing code for ingesting data directly using Broker’s API. From the docs, it seems that Broker assumes that publishers and subscribers somehow agree on one layout per topic: "senders and receivers will need to agree on a specific data layout for the values exchanged, so that they interpret them in the same way.” [1] This raises a couple of questions. Primarily: where can Broker users learn the layouts to interpret received data? There’s essentially no hope in deferring a layout, since broker::data doesn’t include any meta information such as field names. Is meta information stored somewhere? Is there a convention how to place and retrieve such meta information in Broker’s data stores? How does Bro itself make such information available? Is there a document that lists all topics used by Bro with their respective broker::data layout? Dominik [1] https://bro-broker.readthedocs.io/en/stable/comm.html#exchanging-bro-events <https://bro-broker.readthedocs.io/en/stable/comm.html#exchanging-bro-events>

2 years, 6 months

[Bro-Dev] Implementing DNSSEC Parser in Bro.

by fatema bannatwala

Hi All, I am in the process of writing parser for the DNSSEC RR types in DNS responses, and written RRSIG (type=46) parser by adding code to existing DNS protocol analyzer in Bro 2.5.4 src code. I have tested the code by recompiling it on our test server and running it against a dns pcap, and it correctly parses the RRSIG record and logs it. And hence have requested a Pull request to merge in the upstream Bro master repo . Planning to write the remaining DNSSEC RR types: NSEC, DS and DNSKEY parsing in Bro DNS analyzer as well, once I get the feedback on the current merge request of the code for parsing RRSIG. Thanks, Fatema.

2 years, 6 months

[Bro-Dev] libmaxminddb configure issue

by Michael Dopheide

Johanna mentioned to me that libmaxminddb should be working now in master... So far I haven't been able to get 'configure' to find it, neither with the OS packages nor by installing libmaxminddb in /usr/local/ and specifying --with-geoip. This is CentOS 7.5. -Dop

2 years, 6 months

[Bro-Dev] Underflow Considered Harmful

by Jim Mellander

Or: How 13 billion became 1.844674e+19 before becoming 0. After sending amounts totaling over 13 billion thru Sumstats, a value of 0 came out the other end of the sausage factory, but only for one specific data item. Debugging this required lots of well placed print statements, and wild speculation on my part as to what possibly could be broken.... The values being thrown in to be summed are incremental differences from a previous observation, which *should* be zero or a positive number in the range of several K, so a 'count' variable was used. However, for some reason, this value came up negative (or should have) due to (in decreasing likelihood) logic error in script, one of bro's dark corners, or bro bug. The reason for the negativity is still TBD. But, in the world of unsigned 64-bit values (aka bro 'count' variables) there is no negativity, only positivity, and an unsigned underflow creates a number just below 2**64 ~ 1.844674e+19 .... Well, Sumstats tallies in doubles, and naturally this figure (1.844674e+19) dominated the total. In fact, additional increments to this total pushed the total value to be greater than 2**64 (with loss of precision, as doubles only keep 53 bits). In the processing step at the Sumstats epoch, the value was converted back to a count using the double_to_count() function which the cheatsheet warns returns 0, if the double value is <0.0, but it actually returns 2**64-double (with a runtime error), and for values > 2**64 it returns 0 with no runtime error :-( So, there it is, a value that should have been about 13 billion became 1.844674e+19 and then became 0. A few suggestions: 1. Conversion routines should saturate at respective minima/maxima of the type being converted to (possibly with runtime error). 2. Underflow of the 'count' type is almost invariably a bug, and should trigger a runtime error. Overflow, similarly, although in practice it seems much less likely to occur as most scripts are dealing with integers considerably less than 2**64. A similar argument could be made for 'int'. With some operations, it is difficult to detect overflow/underflow, but for simple add and subtract, it is relatively easy. 3. Documentation to match behavior. Jim

2 years, 6 months

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

zeek-dev August 2018