Hi all,
Recently I have some problems with Bro and PF_RING in cluster.
On my server, when I have less than 32 worker threads(rings),
everything is okay, but when I use worker threads more than 32, pf_ring
start to receive repeating data packets. For example, rings less than 32, I
send 400000 packets to server and pf_ring info in /proc shows there is
400000 packets in rings, but when rings greater than 32, I can get 800000
packets when 33 rings and 1200000 packets when 34 rings and so on.
I guess if there is some rules that a pf_ring or a bro cluster can only
support less than 32 rings or worker threads on a server or some other
reasons?
Any insight would be helpful.
We recently purchased some Intel XXV710 NICs for our Zeek systems. However,
symmetric hashing does not seem to work on them, at least not completely.
There was some discussion here regarding adding some functionality to the
driver to make it work, however this never landed:
https://sourceforge.net/p/e1000/mailman/message/35199068/
This post discusses how the X710 controller must be configured differently
from the 82599 10G controller (used by the X520 cards):
https://haryachyy.wordpress.com/2019/01/18/learning-dpdk-symmetric-rss/
The odd part is that following the SEPTun-MarkII guide[1] makes it *mostly*
work, but we're consistently finding that ~1-2% of the traffic is not being
symmetrically hashed. We're testing with can-i-use-afpacket-fanout[2] and
Zeek 3.2.
The most damning evidence is an Intel rep telling[3] a customer:
> Unfortunately, we have been informed that the only support to setup
symmetric RSS is via DPDK.
Searching the mailing list archives, I found a couple of posts where people
were encouraged to use X710-based cards, so I'm left wondering: Are there
people using these? Are they also seeing this 1-2% asymmetry? Or am I
missing a configuration tweak?
Thanks,
--Vlad
[1] - <
https://github.com/pevma/SEPTun-Mark-II/blob/master/SEPTun-Mark-II.rst>
[2] - <https://github.com/JustinAzoff/can-i-use-afpacket-fanout>
[3] - <
https://community.intel.com/t5/Ethernet-Products/X-L-710-supports-symmetric…
>
Hi all,
I tried installing Zeek 4.0.1 and 4.0.2
[root@marvin zeek-4.0.2]# zeekctl deploy
Segmentation fault (core dumped)
[root@marvin zeek-4.0.2]# cat /etc/centos-release
CentOS Linux release 8.4.2105
Doesn't give me any other info. I was running 4.0.1 before I updated to
4.0.2, now both version I get a Seg fault.
Thanks
Monah
Hi all,
As the world begins to feel safe and cities are opening up for more
in-person meetings, we are looking at starting up our Zeek Hours (Meetups)
and Zeek Days (Workshops) back up.
We've created a meetup channel on our Zeek Slack Workspace (#meetups), to
help coordinate and plan these events.
We've also updated our meetup group space to the pro version, so that we
can have chapters and manage multiple events, that way no one in the
community has to worry about the cost of the meetup.com space or starting
from scratch etc. * (So please don't create Zeek Meetup groups on
Meetup.com, we'll take care of all that once we start planning these)*
If you or your organization are interested in hosting, sponsoring, leading,
or planning these events please let me know and also please join the slack
channel as well.
Join the Zeek Slack Workspace at: http://bit.ly/ZeekOrgSlackInvite then
join #meetups
I'll schedule the first meeting for July sometime after the community call.
Please let me know if you have any questions.
Thanks,
~Amber
Hi all
I am trying to deploy Zeek 4 on Centos 7
After upgrading many packages (gcc, make and so on), I can only compile the
master version.
For some reason, make fails on Zeek release/4.0 or Zeek v4.0.1 or Zeek
v4.0.2.
Any idea why that could happen ?
Thank you
Hi,
In our setup we use 2 interfaces for capturing bro traffic, one for
inbound and another one for outbound traffic, but we have one issue on
this, session split issue. we are seeing wrong conn_state flags, and tried
different configurations , can any one help on this issue
we are using bro with pf_ring
configuration we tried node.cfg
conf1:
[worker-1]
type=worker
host=localhost
interface='eth10 -i eth11'
lb_method=pf_ring
lb_procs=8
Conf2:
[worker-1]
type=worker
host=localhost
interface=pf_ring::eth10,eth11
lb_method=pf_ring
lb_procs=8
#pin_cpus=24,25,26,27,28,29,30,31
Regards,
Sunu P S
This is great news, I highly recommend this webinar, Aashish is known to
me and is an consummate professional and expert, I am definitely
attending.
Sincerely,
Greg Grasmehr
He/Him/His
Lead Information Security Analyst
California Institute of Technology (Caltech)
GPGMe: 38E2 F9BD A95E 9824 20AB 331A 9E29 D1A1 AAEE 5F42
http://keys.gnupg.net/pks/lookup?search=0x9E29D1A1AAEE5F42
Hi all,
Join Aashish Sharma of Lawrence Berkeley National Laboratory (LBL), Zeek
Leadership Team (LT) and long standing Zeek user and community member as he
shares with you the Top 10 things he thinks you should know about using
Zeek that he wishes someone would have shared with him when he was getting
started with Zeek.
*Top 10 List Includes:*
1. connection logs are equivalent of netflows
2. use UID
3. history field is very useful
4. SF or no SF makes a difference in incident response and
investigation
5. you can manipulate notices to your wish – email, page, action, none,
all
6. you can feed data into zeek real time (input framework
7. you can print values or variables with zeekctl – great for
troubleshooting
8. you can redirect print statements to a file and reporter log
9. you can run other people’s packages and scripts – separate data from
policy model
10. you can create your own detections.
*In addition, Aashish will share his thoughts on: *
- Clustering is quite easy!
- @load and package ordering does make a difference (further goes into
log columns)
Register at: https://event.webinarjam.com/register/28/405xvax5
Please let me know if you have any questions.
Thanks,
~Amber
Hello Everyone,
Testing zeek 4.0.2 code on centos 7. Workers on 4.0.2 code are running with
really high CPU compared to 3.0.13 code even when there is no traffic on
the myricom interface.
Compiled zeek 4.0.2 with following options:
*./configure --spooldir=/var/spool/zeek --logdir=/var/log/zeek
--with-pcap=/opt/snf/ --enable-perftools --disable-zkg
--disable-broker-tests --disable-btest --disable-archiver*
Version of myricom driver we are running:
Product Code Version
10G-PCIE2-8C2-2S 3.0.25.50927
CPU utilization jumps very very high when we load balance worker processes
compared to a single process on 4.0.2 code.
*node.cfg with couples of processes and top output:*
*[logger]type=loggerhost=localhostpin_cpus=16[manager]type=managerhost=localhostpin_cpus=14[proxy-1]type=proxyhost=localhostpin_cpus=12[worker-1]type=workerhost=localhostEnv_Vars
= SNF_APP_ID=3, SNF_NUM_RINGS=2, SNF_RSS_FLAGS=0x31,
SNF_DATARING_SIZE=8192MB, SNF_DESCRING_SIZE=2048MB, SNF_FLAGS=0x1,
SNF_DEBUG_MASK=3lb_method=myricomlb_procs=2pin_cpus=8,10interface=snf0*
Top output:
From 4.0.2 code:
sudo -u zeek /usr/local/zeek/bin/zeekctl top
Name Type Host Pid VSize Rss Cpu Cmd
logger logger localhost 20805 255M 79M 0% zeek
manager manager localhost 20856 216M 77M 0% zeek
proxy-1 proxy localhost 20906 215M 77M 0% zeek
worker-1-1 worker localhost 20966 20G 20G 70% zeek
worker-1-2 worker localhost 20971 20G 20G 70% zeek
From 3.0.13 code:
Name Type Host Pid VSize Rss Cpu Cmd
logger logger localhost 11179 254M 85M 0% zeek
manager manager localhost 11227 207M 86M 12% zeek
proxy-1 proxy localhost 11278 206M 87M 6% zeek
worker-1-1 worker localhost 11339 20G 20G 6% zeek
worker-1-2 worker localhost 11341 20G 20G 12% zeek
node.cfg with single process and top output:
*[logger]type=loggerhost=localhostpin_cpus=16[manager]type=managerhost=localhostpin_cpus=14[proxy-1]type=proxyhost=localhostpin_cpus=12[worker-1]type=workerhost=localhostEnv_Vars
= SNF_APP_ID=3, SNF_NUM_RINGS=1, SNF_RSS_FLAGS=0x31,
SNF_DATARING_SIZE=8192MB, SNF_DESCRING_SIZE=2048MB, SNF_FLAGS=0x1,
SNF_DEBUG_MASK=3pin_cpus=8interface=snf0*
Top output:
From 4.0.2 code:
Name Type Host Pid VSize Rss Cpu Cmd
logger logger localhost 24484 255M 76M 0% zeek
manager manager localhost 24535 216M 80M 0% zeek
proxy-1 proxy localhost 24586 215M 76M 0% zeek
worker-1 worker localhost 24635 20G 20G 18% zeek
From 3.0.13 code:
Name Type Host Pid VSize Rss Cpu Cmd
logger logger localhost 39901 245M 83M 5% zeek
manager manager localhost 39949 206M 86M 0% zeek
proxy-1 proxy localhost 39996 205M 83M 5% zeek
worker-1 worker localhost 40043 20G 20G 11% zeek
Has anyone run into this issue? Am I misconfiguring something on zeek 4.0.2?
--
*Ankith Kumar Hanumanthappa* GCIA, GMON
Security Analyst | University Information Security (UIS)
University of Minnesota | umn.edu
Hi all,
zkg 2.10.0 is out, with the following updates:
https://github.com/zeek/package-manager/blob/bb76430eafab98b93570348103ec0c…
The Zeek master branch now uses this version.
A related heads-up: with the upcoming Zeek 4.1 release we'll modernize
the Zeek package source (https://github.com/zeek/packages) by switching
its index files from bro-pkg.index to zkg.index. This change is
transparent unless you're still using dated versions of zkg (bro-pkg
prior to 2.0.0) -- those versions will no longer "see" packages in the
package source at that point (and already only see a fraction today).
zkg itself continues to support Bro-era metadata files.
Best,
Christian
