zeek January 2017

zeek@lists.zeek.org

54 participants
75 discussions

[Bro] Bro with PF_RING receive repeating data packets

by Bowen Li

Hi all, Recently I have some problems with Bro and PF_RING in cluster. On my server, when I have less than 32 worker threads(rings), everything is okay, but when I use worker threads more than 32, pf_ring start to receive repeating data packets. For example, rings less than 32, I send 400000 packets to server and pf_ring info in /proc shows there is 400000 packets in rings, but when rings greater than 32, I can get 800000 packets when 33 rings and 1200000 packets when 34 rings and so on. I guess if there is some rules that a pf_ring or a bro cluster can only support less than 32 rings or worker threads on a server or some other reasons? Any insight would be helpful.

5 days, 19 hours

[Bro] Getting SSL events into Python

by Karol Babioch

Hi, I'm currently researching SSL/TLS handshakes and want to process several events Bro provides with the SSL plugin. I've installed Bro along with broccoli and broccoli-python and the "broping" example (from the test directory) is working just fine. For each "ping" event I sent to Bro, a "pong" is received and processed in my Python script. However, in case of the SSL my callbacks are never executed. The most simplified version looks something like this: > #! /usr/bin/env python > > from broccoli import * > > @event > def ssl_established(c): > print('established') > > bc = Connection("127.0.0.1:47760") > > while True: > bc.processInput() To my understanding I don't even have to load the SSL plugin, since it resides within "base", but nevertheless my local.bro contains the following: > @load broping > @load base/protocols/ssl When starting Bro and executing the Python script mentioned above, nothing happens, even if SSL traffic is going through the interface (and/or coming from a recorded pcap). I've also tried to register callbacks for various other SSL related events (ssl_client_hello, ssl_server_hello, etc.), but in no case were my callbacks invoked. The only difference to the "broping.py" from the examples, is that I'm not sending any events, but just want to receive them (hence I'm calling processInput() regularly). What am I missing here? Do I somehow need to enable the SSL functionality within Bro? How can I further debug the problem? Any help is very much appreciated, since I've spent a fair amount of time on this already, with no real progress. Thank you very much! Best regards, Karol Babioch

4 years, 11 months

[Bro] Converting Notice::Info to JSON

by Dave Crawford

I’m creating a script that hooks Notice notice/policy and executes an ActiveHTTP call to submit specific notice events to a REST endpoint. In the submission I’d like to include the Notice::Info object as a JSON data field so tried: to_json(n) But it produces the following error: 1485869266.028563 error in /Users/dave/Projects/bro/share/bro/base/utils/json.bro, line 26: wrong port format, must be /[0-9]{1,5}\/(tcp|udp|icmp)/ (to_port(cat(v))) Do I need to manually re-package all the fields the Notice::Info, and if so, has anyone already done this so I can borrow the code? :-) This is the Notice::Info object I’m testing with: [ts=1485872499.141021, uid=CSRU563utEL1B2yFl5, id=[orig_h=10.0.2.15, orig_p=1381/tcp, resp_h=199.192.156.134, resp_p=443/tcp], conn=<uninitialized>, iconn=<uninitialized>, f=<uninitialized>, fuid=<uninitialized>, file_mime_type=<uninitialized>, file_desc=<uninitialized>, proto=tcp, note=Signatures::Sensitive_Signature, msg=10.0.2.15: ATTACK-RESPONSES Microsoft cmd.exe banner (reverse-shell originator), sub=POST /bbs/info.asp HTTP/1.1\x0d\x0aHost: 199.192.156.134:443\x0d\x0aContent-Length: 165\x0d\x0aConnection: Keep-Alive\x0d\x0aCache-Control: no-cache\x0d\x0a\x0d\x0a3D333531501A..., src=10.0.2.15, dst=199.192.156.134, p=443/tcp, n=<uninitialized>, src_peer=[id=0, host=127.0.0.1, p=0/unknown, is_local=T, descr=bro, class=<uninitialized>], peer_descr=bro, actions={ Phantom::ACTION_PHANTOM, Notice::ACTION_LOG }, email_body_sections=[], email_delay_tokens={ }, identifier=<uninitialized>, suppress_for=1.0 hr, dropped=F, remote_location=<uninitialized>] -Dave

4 years, 11 months

[Bro] Possible system tweak for reducing memory usage

by Azoff, Justin S

TL;DR: It's possible that transparent huge pages and bro do not get along, try doing a # on all nodes echo never > /sys/kernel/mm/transparent_hugepage/enabled # then broctl restart There are ways to make that permanent if it helps. I've been doing some research to try to figure out why some people have more memory issues than others. I think the kernel feature Transparent Huge Pages (THP) and bro may not get along well. It's supposed to help performance for memory allocations, but many services recommend disabling it (mongodb, redis, mysql). For example: > Transparent Huge Pages (THP) is a Linux memory management system that reduces the overhead of Translation Lookaside Buffer (TLB) lookups on machines with large amounts of memory by using larger memory pages. > > However, database workloads often perform poorly with THP, because they tend to have sparse rather than contiguous memory access patterns. You should disable THP on Linux machines to ensure best performance with MongoDB. Bro memory allocations can best described as unpredictable, especially on 'custer in a box' deployments. On our systems, disabling it drops bro worker memory usage by 20% and manager/logger usage by even more, but since we only have one of those it's harder to compare. For workers I disabled THP on half the nodes, and the post bro restart memory usage is consistently lower. -- - Justin Azoff

4 years, 12 months

[Bro] Difference between Bro Clustering method and lb_procs

by Espresso Beanies

Good afternoon, What is the difference between the Bro Clustering method of creating multiple workers and lb_procs? I see the same # of CPUs in-use regardless. Sincerely,

4 years, 12 months

[Bro] files.log need to add id [orig_h,p and resp_h,p]

by ps sunu

Hi, I need to add id [orig_h,orig_p, resp_h, resp_p] in files.log , so i tried to add the content into opt/bro/share/bro/base/frameworks/files/main.bro but its not accepting. I added below code into main.bro id: conn_id &log; and function set_info(f: fa_file) { if ( ! f?$info ) { local tmp: Info = Info($ts=f$last_active,$fuid=f$id, $id=f$conns); f$info = tmp; print "test",f$conns; } any other way to do this , ??

4 years, 12 months

[Bro] Bro4Pros 2017: reminder to cancel vacated registration

by Dopheide, Jeannette M

Attention Bro4Pros attendees, If you are unable to attend Bro4Pros on Thursday February 2nd, please cancel your registration so that we may open the spot to others. For those of you attending Bro4Pros, see you in a couple weeks! Thanks, Jeannette Dopheide ------ Jeannette Dopheide Training and Outreach Coordinator National Center for Supercomputing Applications University of Illinois at Urbana-Champaign

4 years, 12 months

[Bro] Config.log

by James Lay

An odd request I'm sure....almost all other apps that I do the ./configure, make, sudo make install dance leave me with a config.log. Bro does not. Can we get this please? Unless the information is contained somewhere else of course...thank you. James

4 years, 12 months

Re: [Bro] ActiveHTTP

by Jan Grashöfer

> Thanks Jan, what version of Bro are you running and on which platform? I am using Bro 2.5 on Fedora 23 (4.8 kernel). > I have 'bro version 2.5-30’, compiled from Github master, on Debian 8.7 and macOS 10.12.2 and both hang until I ctrl-C, and neither enters the when{} block: $ time bro --pseudo-realtime -r Mswab_Yayih_FD1BE09E499E8E380424B3835FC973A8_2012-03.pcap b.bro [code=302, msg=Found\x0d, body=...] 1485807420.620682 received termination signal real 1m0.583s user 0m26.229s sys 0m34.185s Without "--pseudo-realtime" it seems to hang for me, too. Have you tried using it? Jan

4 years, 12 months

[Bro] ActiveHTTP

by Dave Crawford

I’m testing a new script in 2.5 that uses ActiveHTTP but I'm unable to retrieve the response. With a simple test script of: when ( local resp = ActiveHTTP::request([$url="https://www.google.com/"]) ) { print “Inside the Matrix." } I can see the ActiveHTTP request was successful based on the temporary files created: -rw-r--r-- 1 dave wheel 162 Jan 27 23:43 /tmp/bro-activehttp-HJKhXt6UYXi_body -rw-r--r-- 1 dave wheel 163 Jan 27 23:43 /tmp/bro-activehttp-HJKhXt6UYXi_headers But the print statement within the when block never executes. Any ideas what I’m missing? -Dave

4 years, 12 months

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

zeek January 2017