zeek June 2015

zeek@lists.zeek.org

51 participants
53 discussions

by Troy Jordan

Hello, There are some hilti-based parsers in the Bro docker image. When I run the pcaps for BACnet (/opt/hilti/bro/tests/Traces/bacnet/*.pcap) through Bro (eg bro -r NPDU.pcap) , no event logs are produced in /usr/local/bro/logs). How do I integrate these parsers into Bro? - Troy -- Troy Jordan t r o y j @ m a i n e . e d u GIAC GCIH,GCIA ------------------------------------------------------------ Network Systems Security Analyst Information Technology Security Office University of Maine System ------------------------------------------------------------ 233 Science Building | voice: 207.561.3590 Portland, ME 04103 | fax: 509.351.3650 "As you all know, Security Is Mortals chiefest Enemy" William Shakespeare, Macbeth

5 years, 8 months

[Bro] restrict_filters not preventing logging of selected IP addresses

by Richard Johnson

I think I'm specifying restrict_filters correctly to stop some hosts from being logged, but it's not working as I intend/expect. My local.bro redefinition of restrict_filters (below) is being recognized and propagated by broctl install, as confirmed by print restrict_filters after restarting. As further confirmation that the redef is being noticed, if I specify a pcap syntax impossibility in restrict_filters, I get workers quitting with "fatal error in /raid/bro/share/bro/base/frameworks/packet-filter/./main.bro, line 282: Bad pcap filter ..." on a restart. Yet when the restrict_filter is OK and is seemingly recognized, the IP addresses in the restrict_filters still appear in log entries. This logging continues after a broctl install and update, after a broctl install and restart, as well as after a complete cluster reboot. I'm seeing this under Bro 2.3-7 on CentOS 6.5 with pfring. Whether the capture_filters are redef'ed as shown in the details below, or not, doesn't change the restrict_filters failure I'm seeing. Any ideas for where to take this debugging odyssey? What am I missing that's obvious? Richard ------- Details: [manager-host ~]$ grep capture_filters /raid/bro/share/bro/site/local.bro redef capture_filters = { ["all"] = "ip or not ip" }; [manager-host ~]$ grep restrict_filters /raid/bro/share/bro/site/local.bro redef restrict_filters += { ["not-these-hosts"] = "not host 172.16.1.1 and not host 172.16.22.22 and not host 172.16.39.39 and not host 172.16.88.88" }; [lines condensed for this message by removing extra pretty printing <cr>s] [BroControl] > print capture_filters manager capture_filters = { [all] = ip or not ip } proxy-1 capture_filters = { [all] = ip or not ip } proxy-2 capture_filters = { [all] = ip or not ip } worker-1-1 capture_filters = { [all] = ip or not ip } worker-1-2 capture_filters = { [all] = ip or not ip } worker-1-3 capture_filters = { [all] = ip or not ip } worker-1-4 capture_filters = { [all] = ip or not ip } worker-2-1 capture_filters = { [all] = ip or not ip } worker-2-2 capture_filters = { [all] = ip or not ip } worker-2-3 capture_filters = { [all] = ip or not ip } worker-2-4 capture_filters = { [all] = ip or not ip } [lines condensed for this message by removing extra pretty printing <cr>s] [BroControl] > print restrict_filters manager restrict_filters = { [not-these-hosts] = not host 172.16.1.1 and not host 172.16.22.22 and not host 172.16.39.39 and not host 172.16.88.88 } proxy-1 restrict_filters = { [not-these-hosts] = not host 172.16.1.1 and not host 172.16.22.22 and not host 172.16.39.39 and not host 172.16.88.88 } proxy-2 restrict_filters = { [not-these-hosts] = not host 172.16.1.1 and not host 172.16.22.22 and not host 172.16.39.39 and not host 172.16.88.88 } worker-1-1 restrict_filters = { [not-these-hosts] = not host 172.16.1.1 and not host 172.16.22.22 and not host 172.16.39.39 and not host 172.16.88.88 } worker-1-2 restrict_filters = { [not-these-hosts] = not host 172.16.1.1 and not host 172.16.22.22 and not host 172.16.39.39 and not host 172.16.88.88 } worker-1-3 restrict_filters = { [not-these-hosts] = not host 172.16.1.1 and not host 172.16.22.22 and not host 172.16.39.39 and not host 172.16.88.88 } worker-1-4 restrict_filters = { [not-these-hosts] = not host 172.16.1.1 and not host 172.16.22.22 and not host 172.16.39.39 and not host 172.16.88.88 } worker-2-1 restrict_filters = { [not-these-hosts] = not host 172.16.1.1 and not host 172.16.22.22 and not host 172.16.39.39 and not host 172.16.88.88 } worker-2-2 restrict_filters = { [not-these-hosts] = not host 172.16.1.1 and not host 172.16.22.22 and not host 172.16.39.39 and not host 172.16.88.88 } worker-2-3 restrict_filters = { [not-these-hosts] = not host 172.16.1.1 and not host 172.16.22.22 and not host 172.16.39.39 and not host 172.16.88.88 } worker-2-4 restrict_filters = { [not-these-hosts] = not host 172.16.1.1 and not host 172.16.22.22 and not host 172.16.39.39 and not host 172.16.88.88 } [manager-host current]$ grep 172.16.88.88 conn.log | tail -3 1429461245.805348 CpuepS3Ds2GYzABCtb xx.xx.xx.xx xxxxx 172.16.88.88 443 tcp ssl 4192.655995 14660 16441 S1 F 0ShADda 50 17268 49 19001 (empty) 1429464730.699197 CqVMY53iVvTFSWclAi xx.xx.xx.xx xxxxx 172.16.88.88 443 tcp ssl 1002.988461 5491 4481 SF F 0ShADdaFf 21 6591 17 5377 (empty) 1429464286.982078 CUl3Cl24bUWkgbhAGd xx.xx.xx.xx xxxxx 172.16.88.88 443 tcp ssl 1447.315821 7095 5595 SF F 0ShADdafF 25 8403 21 6699 (empty)

5 years, 10 months

[Bro] BRO 2.3.2 Intel email indicator do not work

by Giedrius Ramas

Hi, I found that BRO 2.3.4 Intel do not work with email's indicators. I have played on my infrastructure to get BRO intel work and found that email indicator won't work. I also tested it on try.bro.org/ the same results . However BRO 2.2 version works well with Intel email's indicators . Please let me know if more details needed to troubleshoot

5 years, 11 months

[Bro] delayed bro operation

by Frank Meier

Hi. A policy forces me to run bro in a separate network. So the captured PCAPs are transfered to the bro network for logging purposes. How would I handle delays in feeding bro with the PCAPS? Would connections spanning multiple PCAPs be a problem? My first idea is to crank up all the timeouts like this: redef tcp_inactivity_timeout = 5 days; redef udp_inactivity_timeout = 5 days; redef icmp_inactivity_timeout = 5 days; redef default_file_timeout_interval = 5 days; What performance penalty will I suffer? I guess the RAM usage will grow, because connections, which were not cleanly terminated, would hang around for a long time. Are there any examples for this kind of setup? How would you search for this? Have a nice weekend! Franky

6 years

[Bro] Bro's limitations with high worker count and memory exhaustion

by Baxter Milliwew

There's some sort of association between memory exhaustion and a high number of workers. The poor man's fix would be to purchase new servers with higher CPU speeds as that would reduce the worker count. Issues with high worker count and/or memory exhaustion appears to be a well know problem based on the mailing list archives. In the current version of bro-2.4 my previous configuration immediately causes the manager to crash: 15 proxies, 155 workers. To resolve this I've lowered the count to 10 proxies and 140 workers. However even with this configuration the manager process will exhaust all memory and crash within about 2 hours. The manager is threaded; I think this is an issue with the threading behavior between manager, proxies, and workers. Debugging threading problems is complex and I'm a complete novice.. my current tutorial is using information from a stack overflow thread: http://stackoverflow.com/questions/981011/c-programming-debugging-with-pthr… Does anyone else have this problem ? What have you tried and what do you suggest ? Thanks 1435347409.458185 worker-2-18 parent - - - info [#10000/10.1.1.1:36994] peer sent class "control" 1435347409.458185 worker-2-18 parent - - - info [#10000/10.1.1.1:36994] phase: handshake 1435347409.661085 worker-2-18 parent - - - info [#10000/10.1.1.1:36994] request for unknown event save_results 1435347409.661085 worker-2-18 parent - - - info [#10000/10.1.1.1:36994] registered for event Control::peer_status_response 1435347409.694858 worker-2-18 parent - - - info [#10000/10.1.1.1:36994] peer does not support 64bit PIDs; using compatibility mode 1435347409.694858 worker-2-18 parent - - - info [#10000/10.1.1.1:36994] peer is a Broccoli 1435347409.694858 worker-2-18 parent - - - info [#10000/10.1.1.1:36994] phase: running

6 years

[Bro] ASCII JSON log stream

by Albert Zaharovits

Hello, I am writing a bro script which creates a ASCII log stream. I would like JSON output only for this stream. I was able to turn on JSON output globally. Any idea? Albert

6 years, 1 month

[Bro] Threat Intelligence Management

by Jan Grashofer

Hi all, I am having a look at Threat Intelligence Management solutions, which can be used with Bro. What do you use and what are your experiences? Regards, Jan

6 years, 1 month

[Bro] Another logging plugin question

by Aaron Gee-Clough

Related to my previous question about a logging plugin, I can get my plugin to compile (and get bro to recognize it as a plugin), but only if I comment out a line that appears in the "elasticsearch" and "dataseries" plugins. The line is: AddComponent(new ::logging::Component("KafkaWriter", ::logging::writer::KafkaWriter::Instantiate)); If I leave this line in, I get compile errors: /home/g-clef/workspace/Bro/Cplusplus/KafkaLogger/src/Plugin.cc: In member function ‘virtual plugin::Configuration plugin::Kafka_KafkaWriter::Plugin::Configure()’: /home/g-clef/workspace/Bro/Cplusplus/KafkaLogger/src/Plugin.cc:13:19: error: expected type-specifier before ‘::’ token AddComponent(new ::logging::Component("KafkaWriter", ::logging::writer::KafkaWriter::Instantiate)); Is this line necessary in logging plugins? Or does it only apply to bro-built-in plugins? Thanks. aaron

6 years, 1 month

[Bro] Myricom + Bro: are these log entries normal?

by Glenn Forbes Fleming Larratt

I'm running 11 worker processes on each of my five clustered listeners, and each of them, like clockwork, spits out 290 apiece (i.e. every five minutes) of these three log entries every day (thank you, logwatch!): Jun 29 14:25:08 bro01.serverfarm.cornell.edu kernel: myri_snf WARN: SNF rx attach: out of free rx rings. app_id=-1 pid=56578 rings_attached=0 rings_requested=0 Jun 29 14:25:08 bro01.serverfarm.cornell.edu kernel: myri_snf WARN: Failed to attach to ring -1 with err=16 Jun 29 14:25:08 bro01.serverfarm.cornell.edu kernel: myri_snf WARN: eth4: endpt 76, early enable failed I'm getting oodles of traffic to both my Bro workers and the Snort instances running as SNF_APP_ID 2 ; are these logs anything I need to worry about, in anyone's experience? Thanks, -- Glenn Forbes Fleming Larratt Cornell University IT Security Office

6 years, 1 month

[Bro] Logging plugin won't configure

by Aaron Gee-Clough

All, I'm attempting to build a logging plugin, and hitting a bit of a brick wall. I have what I think is a reasonable framework, based on the elasticsearch logging plugin, but when I go to configure it, I get : g-clef@yog-sothoth:~/workspace/Bro/C++/KafkaLogger$ ./configure --bro-dist=/home/g-clef/Downloads/bro-2.4/ Build Directory : build Bro Source Directory : /home/g-clef/Downloads/bro-2.4 -- Bro executable : /home/g-clef/Downloads/bro-2.4/build/src/bro -- Bro source : /home/g-clef/Downloads/bro-2.4 -- Bro build : /home/g-clef/Downloads/bro-2.4/build -- Bro install prefix : /usr/local/bro -- Bro plugin directory: /usr/local/bro/lib/bro/plugins -- Bro debug mode : false RegularExpression::compile(): Nested *?+. RegularExpression::compile(): Error in compile. CMake Error at /home/g-clef/Downloads/bro-2.4/cmake/BifCl.cmake:113 (string): string sub-command REGEX, mode REPLACE failed to compile regex "/home/g-clef/workspace/Bro/C++/KafkaLogger/build/src/". Call Stack (most recent call first): /home/g-clef/Downloads/bro-2.4/cmake/BroPluginDynamic.cmake:112 (bif_target) /home/g-clef/Downloads/bro-2.4/cmake/BroPluginCommon.cmake:69 (bro_plugin_bif_dynamic) CMakeLists.txt:10 (bro_plugin_bif) -- Configuring incomplete, errors occurred! See also "/home/g-clef/workspace/Bro/C++/KafkaLogger/build/CMakeFiles/CMakeOutput.log". This looks like a regex error in the bro code, rather than in my code, but I'm not sure what that code is trying to do. Any ideas? Thanks. aaron

6 years, 1 month

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

zeek June 2015