zeek August 2018

zeek@lists.zeek.org

36 participants
33 discussions

[Bro] Bro with PF_RING receive repeating data packets

by Bowen Li

Hi all, Recently I have some problems with Bro and PF_RING in cluster. On my server, when I have less than 32 worker threads(rings), everything is okay, but when I use worker threads more than 32, pf_ring start to receive repeating data packets. For example, rings less than 32, I send 400000 packets to server and pf_ring info in /proc shows there is 400000 packets in rings, but when rings greater than 32, I can get 800000 packets when 33 rings and 1200000 packets when 34 rings and so on. I guess if there is some rules that a pf_ring or a bro cluster can only support less than 32 rings or worker threads on a server or some other reasons? Any insight would be helpful.

5 days, 20 hours

[Bro] Warning of "did not find requested field indicator" from intelligence data file

by wangdj＠ffcs.cn

Hi, According to instruction of intelligence framework, i wrote a intelligence framework text file myintel.txt which content is: #fields indicator indicator_type meta.source meta.desc meta.url 14.215.177.39 Intel::ADDR baidu use baidu search - Very simple. I also wrote a simple bro script file mytest.bro which content is: @load policy/frameworks/intel/seen @load policy/frameworks/intel/do_notice redef Intel::read_files += { "./myintel.txt" }; when i run this script with command "./bro -i eth3 mytest" on a shell terminal and run "ping 14.215.177.39" command on another shell terminal, i got the following warning and : warning: ./myintel.txt/Input::READER_ASCII: Did not find requested field indicator in input data file ./myintel.txt. It seems that there is no error with the myintel.txt file, then what happened leads to this warning. Best Regards DeJin Wang

3 years, 4 months

[Bro] Sysmon integration for Bro

by Neslog

Hello everyone! Has anyone worked on I tegrating Sysmon logs into Bro? Looking to do something like the OSQuery logging.

3 years, 4 months

[Bro] Bro 2.5.5 release (security update)

by Jon Siwek

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 We announce the release of Bro v2.5.5. The new version is now available for download at: https://bro.org/download/index.html or directly at: https://www.bro.org/downloads/bro-2.5.5.tar.gz Binary packages for the new version are currently building and will be available in the next hours at: https://bro.org/download/packages.html This release has the following security fixes: * Fix array bounds checking in BinPAC: for arrays that are fields within a record, the bounds check was based on a pointer to the start of the record rather than the start of the array field, potentially resulting in a buffer over-read. * Fix SMTP command string comparisons: the number of bytes compared was based on the user-supplied string length and can lead to incorrect matches. e.g. giving a command of "X" incorrectly matched "X-ANONYMOUSTLS" (and empty commands match anything). The following changes address potential vectors for Denial of Service reported by Christian Titze & Jan Grashöfer of Karlsruhe Institute of Technology: * "Weird" events are now generally suppressed/sampled by default according to some tunable parameters (see the changelog for more details). These changes help improve performance issues resulting from excessive numbers of weird events. * Improved handling of empty lines in several text protocol analyzers that can cause performance issues when seen in long sequences. * Add 'smtp_excessive_pending_cmds' weird which serves as a notification for when the "pending command" queue has reached an upper limit and been cleared to prevent one from attempting to slowly exhaust memory. Please update your Bro installations as soon as possible. -----BEGIN PGP SIGNATURE----- iQIcBAEBAgAGBQJbhYrrAAoJEMaLSU31asx+ChUQAKsl6Wz5lBhoB51MV8uP/QGX FCL2hbmAqrGKOMkgT0JAGck1jh9tgjV6KCaoeLbOYs4ai+fPktSpuoCFwlNfjkoK jvFkg1dTBRpg+Vd/4yghrVK0N9eTg+i9D/0iOnpwdNfuAaLhJXnUlstTgpaJpPiI powyp15kJmFlKhAeudlJpYXt5FbfLaRlOxHVaVQ5h60T4fEBe+zHA1YFpUOMCdPA hZA6Nz1mkzvSkntG8VwkjUUVr3sEhwSEQO5S+1YHPYyNftYTgAJnHR0KLxG1LWyX MOuomR2LpFagrLE3eFeZ/x9nsttDsBGaV8WXRCrYDknKwj4CBk6NhZESiOAzjCd8 Atv7A6i/ImY3qqkTlVrE4HC6xNsWagTgHeYGEp2nSuet88l9MbJsg/7C5VdiPXbK Xclzczw3aSJ+1Of2kvDnV5OrqfAAZ3+pGIm6Dul/I7CLvXUQispRtyGPtUwtDENE XIDPsG82AwRkZQEOek6DyQHcEPk13eJTgWsbtqmpyHhxWEe5mGfsu+4JWT+mZpLD 51nPNyv8NKNkcfMdNO8mpUekQVOEqYKfHZzoV1s+El2uz1VQ25jdBRZ8qcRBqZlb P7l1iIvRVIS/VFAhtksGLpNQVM++x+CDqYXFS4lq2sF8D4mpoyP5GReG9LaXFWFF cdSmPyEeM92Qsh/o3ySQ =x3s7 -----END PGP SIGNATURE-----

3 years, 5 months

[Bro] trouble of running multiple bro instances

by john Y

Hello all! I am facing with a lot of network traffic saved in pcaps and need to parse them very fast. I tried using broctl but unfortunately it could not use all computer hardware. So, I am running script which invoke a lot of bro instances, one for each pcap. I am using my own bro script which dippend on bro http log and con log files. Because each instance write his logs to the same folder, they run over each other. Creating directory for each instance is too complex and not enough dynamically. Can you offer something better? Maby there is a way to make each instance save his logs to a different dir? My invoke looks something like this : " bro -C -r pcap_path bro_script_path "

3 years, 5 months

[Bro] files.log - no filename over http

by Izik Birka

Hi Why when I download file over HTTP bro doesn't extract the filename ? Here's the http & files log : srv@srv:/nsm/bro/logs/current$ tail -f http_br0.log | grep 192.168.1.1 1534860833.865081 CxLm9G4WxaJ6Z0zqIh 192.168.1.1 31451 77.138.188.44 8080 1 GET 77.138.188.44 http://77.138.188.44/Browsing.exe http://77.138.188.44/ 1.1 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.181 Safari/537.36 0 506576 200 OK - - (empty) - - PROXY-CONNECTION -> keep-alive - - - FI7yey3gl5U0JXLnji - application/x-dosexec srv@srv:/nsm/bro/logs/current$ tail -f files.log | grep 192.168.1.1 1534860834.713869 FI7yey3gl5U0JXLnji 77.138.188.44 192.168.1.1 CxLm9G4WxaJ6Z0zqIh HTTP 0 PE,SHA1,MD5 application/x-dosexec - 0.189665 F F 506576 506576 0 0 F -ea845778462ef5bd2bbf68381df324ca 4af433d0c22067d921c912deae87619b262262f3 - - [Enjoy]<http://www.hot.net.il/> איציק בירקה רכז תחום אבטחת מידע מערכות מידע חטיבת מערכות מידע 077-7077790 | 053-6064571 P חשבו על הסביבה בטרם תדפיסו מייל זה [Enjoy]<http://magazine.hot.net.il/%D7%90%D7%99%D7%98-%D7%92%D7%99%D7%A8%D7%9C%D7%9…> This message (including any attachments) is intended only for the use of the individual or entity to which it is addressed and may contain materials protected by copyright or information that is non-public, proprietary, privileged, confidential, and exempt from disclosure under applicable law or agreement. If you are not the intended recipient, you are hereby notified that any use, dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this communication by error, notify the sender immediately and delete this message immediately. Thank you.

3 years, 5 months

[Bro] CoAP

by erik clark

Does anyone know if there is a Bro plugin for CoAP traffic detection and identification? We have the MQTT plugin for IoT bits, but we would like to support CoAP as well. Thanks!

3 years, 5 months

[Bro] rename field on a per analyzer basis

by erik clark

How can I rename a field based on the analyzer? For example: smtp.log:path -> smtp.log->smtp_path smb_files.log:path -> smb_files.log:smb_path Currently I am using default map, but this does it for all analyzers: redef Log::default_field_name_map = { ["path"] = "smb_path",

3 years, 5 months

[Bro] PE file parser fuzzing

by Maksim Shudrak

Hi everyone, I am trying to search for bugs in bro file parser using libfuzzer. I found the old branch where you tried to fuzz HTTP protocol. So, I have implemented everything on top this branch. I was able to easily make it work for DNP3 protocol but file analyzers are different. The problem is that I can't reach the actual PE parser code from my LLVMFuzzerTestOneInput. The actual code is : analyzer::file::File_Analyzer *filea = new analyzer::file::File_Analyzer("TCP",conn); filea->DeliverStream(DataSize, Data, true); I had problems with file handler which I solved by adding the following line in the Manager.cc --- a/src/file_analysis/Manager.cc +++ b/src/file_analysis/Manager.cc @@ -427,6 +427,7 @@ string Manager::GetFileID(analyzer::Tag tag, Connection* c, bool is_orig) mgr.QueueEvent(get_file_handle, vl); mgr.Drain(); // need file handle immediately so we don't have to buffer data + file_mgr->SetHandle("random_str"); return current_file_id; } In this case, Bro will call "hash" analyzer and ignore PE. It looks like the PE file analyzer is not loaded/initialized, the debug log reports only the following modules being loaded: [FjjsZfY8GArx2E0Ih] Add analyzer MD5 [FjjsZfY8GArx2E0Ih] Add analyzer SHA1 Probably, I am on completely wrong way to make it work. it would be great if you can suggest me some other ways to make it work without significant modification of bro source code. Thank you in advance. ---------------------- Best regards, Maksim Shudrak. tel. +1-415-793-0894 skype: vitality_3

3 years, 5 months

[Bro] BRO Logger crashing due to large DNS log files

by Ron McClellan

All, Having an issue with the bro logger crashing due to large volumes of DNS log traffic, 20-30GB an hour. This is completely a local configuration, on a system with super-fast flash storage, 64 cores, 256GB RAM running BRO 2.5.4. If I disable DNS logging, everything works fine without issue. When I enable it, I get the results below. I thought it might be an issue with gzipping the old logs, so I replaced the standard gzip with pigz and I can manually compress the 30+ gig files in seconds, so don't think that is the issue. I also tried pinning dedicated cores to the logger, currently 6 cores, which should be way more than enough. Any thoughts or suggestions. Thanks, Ron current]# ll -h total 43G -rw-r--r--. 1 root root 3.2K Aug 18 12:00 capture_loss-18-08-18_11.00.00.log -rw-r--r--. 1 root root 3.2K Aug 18 12:18 capture_loss-18-08-18_12.00.00.log -rw-r--r--. 1 root root 2.3M Aug 18 12:00 communication-18-08-18_11.00.00.log -rw-r--r--. 1 root root 1.4M Aug 18 12:18 communication-18-08-18_12.00.00.log -rw-r--r--. 1 root root 4.8K Aug 18 12:18 communication.log -rw-r--r--. 1 root root 19G Aug 18 11:39 dns-18-08-18_10.11.22.log -rw-r--r--. 1 root root 16G Aug 18 12:26 dns-18-08-18_11.00.00.log -rw-r--r--. 1 root root 12M Aug 18 12:00 files-18-08-18_11.00.00.log -rw-r--r--. 1 root root 5.2M Aug 18 12:18 files-18-08-18_12.00.00.log -rw-r--r--. 1 root root 15K Aug 18 12:00 known_certs-18-08-18_11.00.00.log -rw-r--r--. 1 root root 15K Aug 18 12:18 known_certs-18-08-18_12.00.00.log -rw-r--r--. 1 root root 98K Aug 18 12:00 known_hosts-18-08-18_11.00.00.log -rw-r--r--. 1 root root 24K Aug 18 12:18 known_hosts-18-08-18_12.00.00.log -rw-r--r--. 1 root root 71K Aug 18 12:00 known_services-18-08-18_11.00.00.log -rw-r--r--. 1 root root 5.2K Aug 18 12:18 known_services-18-08-18_12.00.00.log -rw-r--r--. 1 root root 1.6K Aug 18 12:00 notice-18-08-18_11.00.00.log -rw-r--r--. 1 root root 954 Aug 18 12:18 notice-18-08-18_12.00.00.log -rw-r--r--. 1 root root 262 Aug 18 12:18 reporter-18-08-18_12.00.00.log -rw-r--r--. 1 root root 23M Aug 18 12:00 smtp-18-08-18_11.00.00.log -rw-r--r--. 1 root root 9.2M Aug 18 12:18 smtp-18-08-18_12.00.00.log -rw-r--r--. 1 root root 1.2M Aug 18 12:00 snmp-18-08-18_11.00.00.log -rw-r--r--. 1 root root 415K Aug 18 12:18 snmp-18-08-18_12.00.00.log -rw-r--r--. 1 root root 81K Aug 18 12:00 software-18-08-18_11.00.00.log -rw-r--r--. 1 root root 8.4K Aug 18 12:18 software-18-08-18_12.00.00.log -rw-r--r--. 1 root root 30K Aug 18 12:00 ssh-18-08-18_11.00.00.log -rw-r--r--. 1 root root 13K Aug 18 12:18 ssh-18-08-18_12.00.00.log -rw-r--r--. 1 root root 217K Aug 18 12:00 ssl-18-08-18_11.00.00.log -rw-r--r--. 1 root root 78K Aug 18 12:18 ssl-18-08-18_12.00.00.log -rw-r--r--. 1 root root 37K Aug 18 12:00 stats-18-08-18_11.00.00.log -rw-r--r--. 1 root root 16K Aug 18 12:18 stats-18-08-18_12.00.00.log -rw-r--r--. 1 root root 28 Aug 18 12:18 stderr.log -rw-r--r--. 1 root root 188 Aug 18 10:11 stdout.log -rw-r--r--. 1 root root 6.8G Aug 18 12:00 weird-18-08-18_11.00.00.log -rw-r--r--. 1 root root 2.5G Aug 18 12:18 weird-18-08-18_12.00.00.log -rw-r--r--. 1 root root 178K Aug 18 12:00 x509-18-08-18_11.00.00.log -rw-r--r--. 1 root root 80K Aug 18 12:18 x509-18-08-18_12.00.00.log # /usr/local/bro/bin/bro --version /usr/local/bro/bin/bro version 2.5.4

3 years, 5 months

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

zeek August 2018