zeek September 2015

zeek@lists.zeek.org

51 participants
52 discussions

by j2om1350＠unibw.de

Hi all, My goal is to integrate a new protocol analyzer in Bro. This protocol (PROFINET dyscovery and Basic Configuration Protocol) is working on layer 2. My question is, are there special considerations to get at the data of the layer 2? My colleague has tried creating an analyzer by following your instructions for coding an analyzer by binpac. Before he went on vacation, he told me, he could access data with binpac of layer 3 but not of layer 2? Is that correct? If so does it work with the new binpac ++? Any pieces of advice or suggestions how to get started would be greatly appreciated. Kind regards Marcel Odenwald

5 years, 4 months

[Bro] exercising binpac++/spicy parsers

by Troy Jordan

Hello, There are some hilti-based parsers in the Bro docker image. When I run the pcaps for BACnet (/opt/hilti/bro/tests/Traces/bacnet/*.pcap) through Bro (eg bro -r NPDU.pcap) , no event logs are produced in /usr/local/bro/logs). How do I integrate these parsers into Bro? - Troy -- Troy Jordan t r o y j @ m a i n e . e d u GIAC GCIH,GCIA ------------------------------------------------------------ Network Systems Security Analyst Information Technology Security Office University of Maine System ------------------------------------------------------------ 233 Science Building | voice: 207.561.3590 Portland, ME 04103 | fax: 509.351.3650 "As you all know, Security Is Mortals chiefest Enemy" William Shakespeare, Macbeth

5 years, 8 months

[Bro] Memory Issue with Bro

by Sampada Kathare

Hello, I have been using the Bro 2.4 to test the performance of SFC driver. I have observed the following issue because of which I am unable to proceed with any analysis - There seems to be a memory leak somewhere as there are times when Bro runs out of memory too soon. These are the instances when drops are also seen too soon even at very low packet rates. When Bro is started, the available free memory keeps going down till a point where the server is extremely sluggish and there are drops being seen - An instance of Bro running out of memory (with 16 workers, no cpu pinning and having sent 155K pps for 7-8 minutes)- [root@dellr620c skathare]# free -m total used free shared buffers cached Mem: 32129 31917 211 3 1 376 --> 211MB : that's very low, considering the system started with some 26GB free memory (and this drop happens just within the first 2 minutes of running the traffic). System becomes very slow at this point and, of course, it has started dropping packets already. -/+ buffers/cache: 31539 589 Swap: 1907 1687 219 [root@dellr620c skathare]# Swap: 1907 1764 142 [root@dellr620c skathare]# cat /proc/meminfo MemTotal: 32900200 kB MemFree: 193384 kB MemAvailable: 480956 kB Buffers: 2464 kB Cached: 471260 kB SwapCached: 74860 kB Active: 23439908 kB Inactive: 3120012 kB Active(anon): 23179296 kB Inactive(anon): 2914628 kB Active(file): 260612 kB Inactive(file): 205384 kB Unevictable: 0 kB Mlocked: 0 kB SwapTotal: 1953076 kB SwapFree: 152264 kB Dirty: 22548 kB Writeback: 8 kB AnonPages: 26017216 kB Mapped: 15200 kB Shmem: 4692 kB Slab: 190556 kB SReclaimable: 93648 kB SUnreclaim: 96908 kB KernelStack: 4288 kB PageTables: 71380 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 15638376 kB Committed_AS: 29119672 kB VmallocTotal: 34359738367 kB VmallocUsed: 374920 kB VmallocChunk: 34342144308 kB HardwareCorrupted: 0 kB AnonHugePages: 243712 kB HugePages_Total: 2700 HugePages_Free: 46 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB DirectMap4k: 345024 kB DirectMap2M: 18483200 kB DirectMap1G: 16777216 kB [root@dellr620c skathare]#top top - 22:48:06 up 70 days, 19:43, 5 users, load average: 18.25, 13.49, 10.04 Tasks: 17 total, 1 running, 16 sleeping, 0 stopped, 0 zombie %Cpu(s): 31.5 us, 2.9 sy, 0.7 ni, 43.7 id, 21.1 wa, 0.0 hi, 0.0 si, 0.0 st KiB Mem: 32900200 total, 32731868 used, 168332 free, 336 buffers KiB Swap: 1953076 total, 1262956 used, 690120 free. 14248 cached Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND P 20504 root 20 0 404644 32724 1464 D 0.0 0.1 6:32.62 bro 31 20552 root 20 0 404692 50108 1496 D 52.1 0.2 6:37.18 bro 29 20564 root 20 0 404824 50960 1508 D 49.8 0.2 6:37.16 bro 27 20574 root 20 0 404652 48748 1476 D 52.0 0.1 6:36.52 bro 24 20567 root 20 0 404684 49948 1456 D 0.0 0.2 6:33.32 bro 23 20561 root 20 0 421440 66672 1412 D 51.9 0.2 6:37.14 bro 18 20569 root 20 0 404708 31904 1508 D 41.1 0.1 6:34.77 bro 16 20495 root 20 0 404620 49936 1408 D 27.1 0.2 6:34.91 bro 13 20515 root 20 0 404684 46324 1500 D 21.9 0.1 6:33.25 bro 13 20548 root 20 0 404704 50188 1504 D 43.6 0.2 6:35.16 bro 13 20474 root 20 0 404736 32704 1508 D 0.0 0.1 6:32.79 bro 12 20502 root 20 0 404636 29300 1464 D 52.0 0.1 6:36.13 bro 12 20539 root 20 0 404748 32784 1484 D 44.7 0.1 6:34.08 bro 11 20537 root 20 0 404668 29284 1464 D 0.0 0.1 6:32.03 bro 4 20559 root 20 0 404684 32644 1444 R 54.6 0.1 6:38.12 bro 3 20542 root 20 0 404728 32704 1504 D 25.1 0.1 6:33.84 bro 1 20289 root 20 0 196768 412 412 S 0.0 0.0 0:00.03 solar_clusterd 0 After stopping the BRO workers (especially after the manager is killed/stopped), memory recovers - top - 22:53:01 up 70 days, 19:48, 5 users, load average: 3.06, 8.83, 9.20 Tasks: 1 total, 0 running, 1 sleeping, 0 stopped, 0 zombie %Cpu(s): 0.1 us, 0.3 sy, 2.2 ni, 96.6 id, 0.8 wa, 0.0 hi, 0.0 si, 0.0 st KiB Mem: 32900200 total, 6702252 used, 26197948 free, 8308 buffers --> This is almost what the system originally started with - 26GB KiB Swap: 1953076 total, 237216 used, 1715860 free. 528032 cached Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND P 20289 root 20 0 196768 1028 484 S 0.0 0.0 0:00.03 solar_clusterd At very high packet rates, the available free memory keeps going down very fast and starts dropping packets. At lower packet rates, the drop in available free memory is comparatively slower, but it is still there and packets are dropped eventually. When the BRO workers are stopped, the available free memory recovers. During the few successful times when I have been able to go till 150Kpps without seeing any packet drops, the available free memory remained a constant at ~23G. It remained at this for the entire duration of the test (more than an hour ) and no drops were seen. The above data is a few days old. When I tried running BRO again today, I saw the memory drop from 18G to 4G in just a matter of few seconds after starting BRO (16 workers, each pinned to a CPU). Is it possible that Bro is accumulating some per-flow state and not freeing it? If so, is there any tuning that should be done to avoid this? Appreciate any help on this! - Sampada The information contained in this message is confidential and is intended for the addressee(s) only. If you have received this message in error, please notify the sender immediately and delete the message. Unless you are an addressee (or authorized to receive for an addressee), you may not use, copy or disclose to anyone this message or any information contained in this message. The unauthorized use, disclosure, copying or alteration of this message is strictly prohibited.

5 years, 9 months

[Bro] General advice on malware hunting?

by nortonperry＠gmail.com

Hi Gents, I have had Bro installed as my gateway for a home network for about nine months now, with a complete (mostly uninterrupted) run of logs. I've also supplemented this with the critical stack plugin since July, with intel feeds up - focused mostly on malware and candc domains. The network is reasonably busy, has probably about 25 discreet hosts of which at any given time between 3 and 10 are up. I have suspected there is malware / a rootkit perhaps on the network for a while as arp -a shows a lot of <undefined> hosts every now and then from the terminal of most systems on the network. Also, Nmap scans often report IP addresses that simply are not there. Also, Bro reports traffic to local NAT IP addresses that don't exist. eg my network is divided into a 192.168.2.x (Internal, all the hosts) and 192.168.1.x(airgap between Bro router and domestic DSL router). The 192.168.1.x network only really ever has two hosts - the bro router and the dsl router, but connections show to other addresses which don't exist. I have tried to put a methodology together for malware hunting based on what I can find online, but nothing has really come to light. I use zcat, bro-cut and regular expressions to query the logs. Would anyone on this list mind assisting me in a bug hunt / provide a methodology for tracking down suspicious traffic? I have looked and looked but can't seem to find any workflow / tolling which can isolate malware effectively. Any advice on this would be very gratefully received! regs Perry

5 years, 10 months

[Bro] File name from fa_file

by Pigott, Nathan

Hello, I'm having problems getting file names from fa_file - the field f$info$filename is showing up uninitialized on every single fa_file in all my tests. Is there a known reason why this would be happening? I'm using Bro 2.3, but I tested on 2.4 as well and got the same results. Are there any alternative ways to get file names? For now I'm parsing the URL returned by Files::describe(f), but this does not work if the URL doesn't contain the file name, or if the file was transferred with a protocol other than HTTP. Thanks, Nathan Pigott

5 years, 10 months

[Bro] Bro Continually Crashes

by Gabriel Dinkins

When starting the Bro service, an error stating that Bro terminated immediately after start-up continually appears. The following is the info I am provided upon running the diag command. --------------------------------------------------------------------------------------------------------------------------------------- Bro 2.4 Linux 2.6.32-573.7.1.el6.x86_64 ==== No reporter.log ==== stderr.log fatal error: problem with interface p3p1 (p3p1: no IPv4 address assigned) ==== stdout.log max memory size (kbytes, -m) unlimited data seg size (kbytes, -d) unlimited virtual memory (kbytes, -v) unlimited core file size (blocks, -c) unlimited ==== .cmdline -i p3p1 -U .status -p broctl -p broctl-live -p standalone -p local -p bro local.bro broctl broctl/standalone broctl/auto ==== .env_vars PATH=/usr/local/bro/bin:/usr/local/bro/share/broctl/scripts:/usr/local/bro/bin:/usr/lib64/qt-3.3/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin BROPATH=/usr/local/bro/spool/installed-scripts-do-not-touch/site::/usr/local/bro/spool/installed-scripts-do-not-touch/auto:/usr/local/bro/share/bro:/usr/local/bro/share/bro/policy:/usr/local/bro/share/bro/site CLUSTER_NODE= ==== .status TERMINATED [atexit] ==== No prof.log ==== No packet_filter.log ==== No loaded_scripts.log ------------------------------------------------------------------------------------------------------------------------------------------ -- Gabriel Dinkins Cyber Security Engineer PUNCH Cyber Analytics Group

5 years, 10 months

[Bro] Reading data into a table

by Damian Gerow

We've run into a few problems with our scripts and the use of &persistent, so we're looking to do some home-grown persistence. These scripts are part of a module called ConnectionValidation; the applicable bits of the scripts are at https://gist.github.com/mutemule/6076cddce3ce8c9e7013. It's worth pointing out that the module as a whole is loaded in to all components, but the persistence layer is only loaded in to the proxy. What I'm seeing is the table being written to disk as expected during bro_done(), but seemingly not being read back in during bro_init(): after startup, the table remains blank in all cluster components. I'd previously tried this with a set instead of a table, but that didn't work. Then I tried using events to populate the set, but that also didn't work. So now I'm on a table, and following the input framework documentation[0] almost exactly, but it's still not doing what I expected. What am I doing wrong? How do I read a table in from disk during initialization/startup? [0] https://www.bro.org/sphinx/frameworks/input.html

5 years, 10 months

[Bro] Raw (eml) Email Extraction Bro 2.4

by V. Garramone

Hi Everyone, I would like to do full email extraction (eml) to file from STMP traffic; should this happen naturally with the new file extraction framework? I found this exchange from a while back, but haven't found anything more recent on the topic: http://mailman.icsi.berkeley.edu/pipermail/bro/2014-July/007224.html I'm currently using Bro 2.4 and a script pretty similar to this one for file extraction: https://github.com/Security-Onion-Solutions/securityonion-bro -scripts/blob/master/file-extraction/extract.bro It looks like I'm getting the message content and attachments, but apparently not the raw email. Any tips would be greatly appreciated! Thanks very much, VG

5 years, 10 months

[Bro] Issue with bro reading a file that capturing live traffic

by Hashem Alaidaros

Hi All, I run tcpdump live to capture the traffic into a file using "-w". Then I run bro to read that file offline using "-r". Both instances are running continuously. First it works fine but then bro stop generating results although it keep running, this means bro didn't continue reading from the file. Is it because bro -r is faster than the live capturing? How to let bro keep reading the file (this file is continuously whitening? My bro version: 2.3 running on ubuntu platform. Thanks

5 years, 10 months

[Bro] Is it possible to export pcap for a given event / connection?

by Dirk Leinenbach

Hi there, does bro provide some mechanism to find the packets that are related to (have caused) a given event or connection? Background: I'd like to be able to export pcap files in some situations for specific events; in that context I'm still able to get to the connection object, but I'd like to be able to see the original data as well for further analysis with Wireshark. One possibility would be to reconstruct filters from the event to filter the original trace retrospectively. But I'm wondering if there is a more direct way to identify / extract the relevant packets. Thanks for your help, Dirk

5 years, 10 months

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

zeek September 2015