zeek September 2019

zeek@lists.zeek.org

32 participants
23 discussions

[Bro] Bro with PF_RING receive repeating data packets

by Bowen Li

Hi all, Recently I have some problems with Bro and PF_RING in cluster. On my server, when I have less than 32 worker threads(rings), everything is okay, but when I use worker threads more than 32, pf_ring start to receive repeating data packets. For example, rings less than 32, I send 400000 packets to server and pf_ring info in /proc shows there is 400000 packets in rings, but when rings greater than 32, I can get 800000 packets when 33 rings and 1200000 packets when 34 rings and so on. I guess if there is some rules that a pf_ring or a bro cluster can only support less than 32 rings or worker threads on a server or some other reasons? Any insight would be helpful.

3 months, 3 weeks

Communication channels

by Woot4moo

Are there any plans to use another communication platform besides IRC? To the best of my knowledge the IRC channel does not record history, so any new member is unaware of prior chats / discoveries. Is there a desire in the community to move to something along the lines of Discord or Slack? Granted Slack would come at a premium.

2 years, 7 months

Why does my logger keep crashing - bro version 2.6.3

by Kayode Enwerem

Hello, Why does my logger keep crashing? Can someone please help me with this. I have provided some system information below: I am running bro version 2.6.3 1. Output of broctl status. The logger is crashed but the manager, proxy and workers are still running. broctl status Name Type Host Status Pid Started logger logger localhost crashed manager manager localhost running 17356 09 Sep 15:42:24 proxy-1 proxy localhost running 17401 09 Sep 15:42:25 worker-1-1 worker localhost running 17573 09 Sep 15:42:27 worker-1-2 worker localhost running 17569 09 Sep 15:42:27 worker-1-3 worker localhost running 17572 09 Sep 15:42:27 worker-1-4 worker localhost running 17587 09 Sep 15:42:27 worker-1-5 worker localhost running 17619 09 Sep 15:42:27 worker-1-6 worker localhost running 17614 09 Sep 15:42:27 worker-1-7 worker localhost running 17625 09 Sep 15:42:27 worker-1-8 worker localhost running 17646 09 Sep 15:42:27 worker-1-9 worker localhost running 17671 09 Sep 15:42:27 worker-1-10 worker localhost running 17663 09 Sep 15:42:27 worker-1-11 worker localhost running 17679 09 Sep 15:42:27 worker-1-12 worker localhost running 17685 09 Sep 15:42:27 worker-1-13 worker localhost running 17698 09 Sep 15:42:27 worker-1-14 worker localhost running 17703 09 Sep 15:42:27 worker-1-15 worker localhost running 17710 09 Sep 15:42:27 worker-1-16 worker localhost running 17717 09 Sep 15:42:27 worker-1-17 worker localhost running 17720 09 Sep 15:42:27 worker-1-18 worker localhost running 17727 09 Sep 15:42:27 worker-1-19 worker localhost running 17728 09 Sep 15:42:27 worker-1-20 worker localhost running 17731 09 Sep 15:42:27 1. Here's my node.cfg settings [logger] type=logger host=localhost [manager] type=manager host=localhost [proxy-1] type=proxy host=localhost [worker-1] type=worker host=localhost interface=af_packet::ens2f0 lb_method=custom #lb_method=pf_ring lb_procs=20 pin_cpus=6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25 af_packet_fanout_id=25 af_packet_fanout_mode=AF_Packet::FANOUT_HASH 1. Heres more information on my CPU. 32 CPUs, model name - AMD, CPU max MHz is 2800.0000 Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 32 On-line CPU(s) list: 0-31 Thread(s) per core: 2 Core(s) per socket: 8 Socket(s): 2 NUMA node(s): 4 Vendor ID: AuthenticAMD CPU family: 21 Model: 2 Model name: AMD Opteron(tm) Processor 6386 SE Stepping: 0 CPU MHz: 1960.000 CPU max MHz: 2800.0000 CPU min MHz: 1400.0000 BogoMIPS: 5585.93 Virtualization: AMD-V L1d cache: 16K L1i cache: 64K L2 cache: 2048K L3 cache: 6144K NUMA node0 CPU(s): 0,2,4,6,8,10,12,14 NUMA node1 CPU(s): 16,18,20,22,24,26,28,30 NUMA node2 CPU(s): 1,3,5,7,9,11,13,15 NUMA node3 CPU(s): 17,19,21,23,25,27,29,31 1. Would also like to know how I can reduce my packet loss. Below is the output of broctl netstats broctl netstats worker-1-1: 1568908277.861813 recvd=12248845422 dropped=5171188999 link=17420313882 worker-1-2: 1568908298.313954 recvd=8636707266 dropped=971489 link=8637678939 worker-1-3: 1568908278.425888 recvd=11684808853 dropped=5617381647 link=17302473791 worker-1-4: 1568908285.731130 recvd=12567242226 dropped=4339688288 link=16907212802 worker-1-5: 1568908298.363911 recvd=8620499351 dropped=24595149 link=8645095758 worker-1-6: 1568908298.372892 recvd=8710565757 dropped=1731022 link=8712297432 worker-1-7: 1568908298.266010 recvd=9065207444 dropped=53523232 link=9118737229 worker-1-8: 1568908286.935607 recvd=11377790124 dropped=3680887247 link=15058934491 worker-1-9: 1568908298.419657 recvd=8931903322 dropped=39696184 link=8971604219 worker-1-10: 1568908298.478576 recvd=8842874030 dropped=2501252 link=8845376352 worker-1-11: 1568908298.506649 recvd=8692769329 dropped=2253413 link=8695025626 worker-1-12: 1568908298.520830 recvd=8749977028 dropped=2314733 link=8752293714 worker-1-13: 1568908298.544573 recvd=9101243757 dropped=1779460 link=9103025399 worker-1-14: 1568908291.370011 recvd=10876925726 dropped=775722632 link=11652810353 worker-1-15: 1568908298.579721 recvd=8503097394 dropped=1420699 link=8504520066 worker-1-16: 1568908298.594942 recvd=8515164266 dropped=1840977 link=8517006779 worker-1-17: 1568908298.646966 recvd=10666567717 dropped=466489754 link=11133059283 worker-1-18: 1568908298.671246 recvd=9023603573 dropped=2037607 link=9025642263 worker-1-19: 1568908298.704675 recvd=8907784186 dropped=1164594 link=8908950238 worker-1-20: 1568908298.718084 recvd=9140525444 dropped=2028593 link=9142555259 Thanks,

2 years, 7 months

Segmentation Fault on Zeek 3.0.0

by TQ

Hello Zeekers, I'm currently in the process of migrating from Bro 2.6.2 to Zeek 3.0.0, and I'm experiencing a small headache with segmentation fault in my plugins. I didn't have this issue with Bro 2.6.2, so I'm not 100% sure what happened here. After making name changes from Bro to Zeek, I was able to successfully compile all of the plugins. When I ran them against pcaps that are specified for the plugin, I noticed that some of the plugins threw a segmentation fault ("Segmentation fault (core dumped)"). I was replaying a pcap file like what I usually do by running: cd ~/Desktop/logs/ && sudo rm -f *.log && zeek -C -t ~/Desktop/logs/output.log -r ~/Desktop/pcap/ testPlugin1_pcap_1.pcapng After some troubleshooting, I noticed that only the ones that had a switch case statement inside a while loop inside main.zeek were affected by this. I do have checks to prevent resource exhaustion, so I'm not sure why the new version is not happy. Anyway, I was able to verify by cd into "/usr/local/zeek/lib/zeek/plugins/Zeek_testPlugin1/scripts" and commenting out the affected section in main.zeek. Even something as simple as this throws segmentation fault: while (index < payload_length) { header = bytestring_to_count(data[index]); len = 0; index += 1; switch (header) { default: ##! test break; } # dummy check as example if (index > 10) { break; } } I've been looking at this for the last 8 hours, so more eyes would be appreciated. Thanks,

2 years, 7 months

How can I reduce my packet loss - bro version 2.6.3

by Kayode Enwerem

Hello, My packet loss on my workers is pretty high. I have done things like CPU pinning but its still high. Can you please assist me in how I can reduce this to under 1%. Below are some of my settings. cat capture_loss.log - Percent lost ranges from 2.7 to about 67.69 #fields ts ts_delta peer gaps acks percent_lost #types time interval string count count double 1569863104.831317 900.000030 worker-1-3 114384 3799935 3.010157 1569863104.851327 900.000002 worker-1-1 162671 3677320 4.423629 1569863104.841705 900.000062 worker-1-9 100444 3393374 2.960004 1569863104.843460 900.000058 worker-1-11 148576 4171807 3.56143 1569863104.855034 900.000116 worker-1-16 165242 3769560 4.383589 1569863104.937666 900.000094 worker-1-23 124377 3891351 3.196242 1569863104.811991 900.000040 worker-1-12 339309 3176448 10.682026 1569863104.853635 900.000036 worker-1-7 304519 3266968 9.32115 1569863105.107706 900.000013 worker-1-8 296921 3475658 8.542872 1569863117.781723 900.739890 worker-1-15 635032 1385280 45.841418 1569863114.375886 900.000010 worker-1-17 631085 2596009 24.309816 1569863118.295945 900.001869 worker-1-6 369130 545290 67.694254 1569863160.141238 900.000229 worker-1-22 774134 1785146 43.365305 1569864004.845052 900.001592 worker-1-11 108257 3871564 2.796208 1569864004.860404 900.000040 worker-1-14 111798 3327087 3.360237 1569864004.937679 900.000013 worker-1-23 148738 3568913 4.167599 1569864004.951235 900.000218 worker-1-19 96672 3509661 2.754454 1569864004.976291 900.000009 worker-1-21 152430 3736550 4.079432 1569864005.211316 900.000025 worker-1-4 176180 3226673 5.460113 1569864005.193565 900.000005 worker-1-10 148535 3986455 3.725992 1569864004.811996 900.000005 worker-1-12 289760 3487997 8.307347 1569864005.107761 900.000055 worker-1-8 270405 3340848 8.093903 1569864014.375894 900.000008 worker-1-17 517282 2462898 21.002981 1569864018.295953 900.000008 worker-1-6 362966 571162 63.548695 1569864060.141335 900.000097 worker-1-22 601802 1520920 39.568288 cat node.cfg (below is the worker config in node.cfg. As you can see I pinned 23 CPUs) [worker-1] type=worker host=localhost interface=af_packet::ens2f0 lb_method=custom #lb_method=pf_ring lb_procs=23 pin_cpus=5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27 af_packet_fanout_id=25 af_packet_fanout_mode=AF_Packet::FANOUT_HASH I have 32 CPUs on this server and CPU model name is - AMD Opteron(tm) Processor 6386 SE CPU MHz: 2800.000 CPU max MHz: 2800.0000 CPU min MHz: 1400.0000 Please assist. Thanks. Thanks,

2 years, 7 months

SSH auth_success state true set, but admin claims no logins

by Collyer, Jeffrey W (jwc3f)

So recently I saw an SSH login to a device from outside the US. I reported it to the end system admin. The Zeek log set the auth_success state to true, but the admin of the box claims no successful login and is pushing back that it is a false positive. Have other Zeek users ever seen this? Is the SSH auth state detection mistaken here? I don’t have pcaps to verify one way to the other, sadly. {"_path":"ssh","_system_name":"corelight","_write_ts":"2019-09-12T22:26:32.106142Z","ts":"2019-09-12T22:26:31.226136Z","uid":"C95i0o2Jl77LXHb2R9","id.orig_h”:”x.x.x.x","id.orig_p":49670,"id.resp_h”:”x.x.x.x","id.resp_p":22,"version":2,"auth_success":true,"auth_attempts":1,"direction":"INBOUND","client":"SSH-2.0-OpenSSH_7.2p2 Ubuntu-4ubuntu2.4","server":"SSH-2.0-OpenSSH_7.4","cipher_alg":"chacha20-poly1305(a)openssh.com","mac_alg":"umac-64-etm(a)openssh.com","compression_alg":"none","kex_alg":"curve25519-sha256(a)libssh.org","host_key_alg":"ecdsa-sha2-nistp256","host_key":"68:1e:68:89:5e:e5:20:72:f7:e6:bf:21:de:07:3a:b1”} Can anyone shed light on this? Thanks Jeff Jeffrey Collyer Information Security Engineer University of Virginia jwc3f(a)virginia.edu

2 years, 8 months

Outdated "starting points" page

by 陈龙

Hi, recently I'm staring looking at Zeek's source code for learning. But I notice that the doc in zeek.org<http://zeek.org> seems to be outdated, for example, the page "starting points” (https://www.zeek.org/development/howtos/starting-points.html). On this page, it mentioned DPM.cc/.h<http://DPM.cc/.h> and AnalyzerTags.h. But I find them in release 2.1rather than release 2.6. It will be more friendly for newbie like me if it be updated or noted with version number. Regards. -- Jason

2 years, 8 months

Zeek 3.0.0 release available

by Jon Siwek

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Zeek release 3.0.0 is now available: https://www.zeek.org/downloads/zeek-3.0.0.tar.gz https://www.zeek.org/downloads/zeek-3.0.0.tar.gz.asc This major release has many additions and changes, the most prominent being a comprehensive switch to use the name Zeek instead of Bro. Please read the release notes for a full description of new features and changes: https://github.com/zeek/zeek/blob/v3.0.0/NEWS The Zeek blog also describes the upcoming release and potential pitfalls to be aware of when upgrading: https://blog.zeek.org/2019/09/zeek-300.html -----BEGIN PGP SIGNATURE----- iQIzBAEBAgAdFiEE6WkLK32KwaGfkhxKxotJTfVqzH4FAl2JGSkACgkQxotJTfVq zH4Nsg/+Kdmvlfe/3OfTQnLm9gjAU6ZZ8Zihwusdv1dttG+Bil7yhvBKCTE5PF2W Ve5tNA6Z/nfCRiBYL26IUP2xPXGlDTWaCwU0uUiOxURDZBD0YXerjNIyBJds4P3l gD+14GjJaEIWh/2Y0iM8nntTKfdqUmfpMF4laXns3leNj/M0KIgHWJGwvxriVAMu UhU87m84/l1+AuoUqscnVf1j5qyX06lQET6v06w8xd5eyrI0C5U8eWXWMolPnzoC oQ5yeuur7o102tNzp5rYS/Pnmn+WQx2HMumB/v6U/iTh8P2cR6n1uzD6d5w35TXN 9zMGss6v5/92SKR7umaUOo5TWM3kS6ieXuEgUwsf77252sE3TpoOPPWQXnzwYdLT NgqJvyYspWkhNlY2cSJ8LFAu6cobIlBdcGFUtwundLH7to8wFaFS3a9WgCWPV7vk K/5b47sJy1p2F8rHJljdqcxUw3LAq57lnob6bhsKtKm5ZZBPgFei8d9S8PUrtade u+mXunraQbLAsyzTdKUhI7hW7gIXF6dRo5NGmYL+Fh1COnAGmhvIrCjOnBftw6LR FkbLwIAWLS1VD/8tyFm1+klVgQItylbvc/UDsy709mWa0anZ/bmyv9s+pW+2Evbu g7CTrS7eb+MoEPgtEnmuR+sem34hkEPsAuwYwqYrcLwwm4leREU= =2Ql2 -----END PGP SIGNATURE-----

2 years, 8 months

Re: [Zeek] Specs needed for 10GBps

by fatema bannatwala

There is this amazing document available for the 100G Intrusion Detection, where the specs of 10Gbps per system are mentioned with a lot of good information to consider while spec-ing out the hardware and designing an efficient monitoring solution. Here's the link to the whitepaper: https://www.cspi.com/wp-content/uploads/2016/09/Berkeley-100GIntrusionDetec… Hope this helps. Fatema

2 years, 8 months

Specs needed for 10GBps

by Jorge Garcia Rodriguez

Hi, We are new in Zeek and we want to use it to monitor traffic with 10 GBps peaks. We need to know the necessary specifications. How much Ram would be necessary? How much CPU? We checked the documentation but found nothing about this, only that we need 1 core for every 250 Mbp/s. Which seems that we need 40 cores or so. Thank you for your help. Regards. Jorge García Rodríguez Technical Consultant Security Infrastructures jgarciar(a)sia.es<mailto:jgarciar@sia.es> Grupo SIA

2 years, 8 months

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

zeek September 2019