Hi all,
Recently I have some problems with Bro and PF_RING in cluster.
On my server, when I have less than 32 worker threads(rings),
everything is okay, but when I use worker threads more than 32, pf_ring
start to receive repeating data packets. For example, rings less than 32, I
send 400000 packets to server and pf_ring info in /proc shows there is
400000 packets in rings, but when rings greater than 32, I can get 800000
packets when 33 rings and 1200000 packets when 34 rings and so on.
I guess if there is some rules that a pf_ring or a bro cluster can only
support less than 32 rings or worker threads on a server or some other
reasons?
Any insight would be helpful.
Hello Everyone,
As you probably know, the next LT elections are coming up in a bit less
than a year – the plan is to hold elections around August 2021. Half
the current LT seats will be up for re-election.
The current election process is documented in our wiki at
https://github.com/zeek/zeek/wiki/Zeek-Project--Leadership-Team----Process-….
It was first used in the 2020 election. In a nutshell, the current
process allows community members to nominate themselves (or someone
else) for a seat on the LT. After contacting all the nominees, the LT
publishes the list of nominations and encourages the community to
provide testimonials for the nominees. The current LT then votes on the
candidates.
For the 2022 election the LT wants to update the above mentioned LT
Process and Description document. As described in the document, we are
hereby seeking community input.
Specifically, the LT Process and Description currently states that we
want to have an open community election starting with the 2022 election.
The LT discussed this in the last few meetings. It is the opinion of the
LT that at the current time there is not yet a sufficient number of
community-led groups to have a meaningful broader voting process. Hence,
we propose to remove the sentences that refer to changing the election
process for the 2022 and following elections, keeping the current voting
process. A future LT can revisit this topic, once there are more active
members in community-led groups.
If you have any feedback about this change, please let us know within
the next four weeks (by 2021-12-15), either by responding to this
message, by mailing the LT at lt(a)lists.zeek.org, or by mailing me
directly.
Thank you,
Johanna Amann (for the Zeek LT)
Problem solved ;)
sob., 27 lis 2021, 01:10 użytkownik Steve Smoot <smoot(a)corelight.com>
napisał:
> Glad it worked! Since I was so weak I didn't cc list so you should do that
> to prevent more troubleshooting
>
> Steve Smoot
> VP, customer success
>
> ------------------------------
> *From:* Jakub Niezabitowski <kuba.michal.n(a)gmail.com>
> *Sent:* Friday, November 26, 2021 1:27:18 PM
> *To:* Steve Smoot <smoot(a)corelight.com>
> *Subject:* Re: [Zeek] Zeek- many duplicate connections
>
> Thank you for quick answer.
>
> You are correct. I had to reinstall pfring and pfring-dkms packages. I
> have been tinkering with suricata so I have compiled pfring from source. In
> my infinite wisdom I have removed those packages...
> Once again it works like a charm.
>
> Thank you for quick reply and sorry silly question!
> Jakub
>
> pt., 26 lis 2021 o 21:06 Steve Smoot <smoot(a)corelight.com> napisał(a):
>
> I've never configured pf ring, but it looks like all the workers are
> reading the same interface which would result in high cpu and duplicate
> flows.... So suggest checking there
> S
>
> Steve Smoot
>
> ------------------------------
> *From:* Jakub Niezabitowski <kuba.michal.n(a)gmail.com>
> *Sent:* Friday, November 26, 2021 4:26 AM
> *To:* zeek(a)lists.zeek.org
> *Subject:* [Zeek] Zeek- many duplicate connections
>
> Hello,
>
> I have noticed (using zeek doctor) that my zeek setup has many duplicate
> connections:
>
> error: 97.80%, 801 out of 819 connections appear to be duplicate
>
> I have been using pfring but I have disabled it to simplify this issue.
>
> This is my node.cfg now:
>
> [logger-1]
> type=logger
> host=localhost
> #
> [manager]
> type=manager
> host=localhost
> #
> [proxy-1]
> type=proxy
> host=localhost
> #
> [worker-1]
> type=worker
> host=localhost
> #lb_procs=7
> #lb_method=pf_ring
> pin_cpus=0
> interface=eth-mirror
> #
> [worker-2]
> type=worker
> host=localhost
> pin_cpus=1
> interface=eth-mirror
>
> [worker-3]
> type=worker
> host=localhost
> pin_cpus=2
> interface=eth-mirror
>
> [worker-4]
> type=worker
> host=localhost
> pin_cpus=3
> interface=eth-mirror
>
> [worker-5]
> type=worker
> host=localhost
> pin_cpus=4
> interface=eth-mirror
>
> What other info should I provide?
>
> Thank you in advance for any help.
> Jakub
>
>
Hello,
I have noticed (using zeek doctor) that my zeek setup has many duplicate
connections:
error: 97.80%, 801 out of 819 connections appear to be duplicate
I have been using pfring but I have disabled it to simplify this issue.
This is my node.cfg now:
[logger-1]
type=logger
host=localhost
#
[manager]
type=manager
host=localhost
#
[proxy-1]
type=proxy
host=localhost
#
[worker-1]
type=worker
host=localhost
#lb_procs=7
#lb_method=pf_ring
pin_cpus=0
interface=eth-mirror
#
[worker-2]
type=worker
host=localhost
pin_cpus=1
interface=eth-mirror
[worker-3]
type=worker
host=localhost
pin_cpus=2
interface=eth-mirror
[worker-4]
type=worker
host=localhost
pin_cpus=3
interface=eth-mirror
[worker-5]
type=worker
host=localhost
pin_cpus=4
interface=eth-mirror
What other info should I provide?
Thank you in advance for any help.
Jakub
Hi Everyone,
As you might have noticed, the mailing lists currently get much fewer
messages than in the past. We think that this is directly related to
most of the community using Slack to communicate.
However, while it is great that our community likes to communicate on
Slack, Slack is not very good as a longer-term resource collection. It
is difficult to find old messages on Slack, pointing users to specific
messages is not easy, and Slack is not optimal for more complex
long-form discussions.
Some members of the Zeek LT expressed that other open source projects
have had good experiences with using Discourse
(https://www.discourse.org/), which provides a forum-like discussion
platform that also can be interacted with via email.
We are considering whether it might make sense to convert from using
mailing lists to using Discourse or another similar project. We’d like
to hear any of our community members who’ve had positive or negative
experiences with using Discourse, or if you happen to know another
project that might work well.
If you have any feedback on this – please either respond to this
message, write the Zeek LT at lt(a)lists.zeek.org, or send an email
directly to me.
Thank you,
Johanna (for the Zeek LT)
We recently purchased some Intel XXV710 NICs for our Zeek systems. However,
symmetric hashing does not seem to work on them, at least not completely.
There was some discussion here regarding adding some functionality to the
driver to make it work, however this never landed:
https://sourceforge.net/p/e1000/mailman/message/35199068/
This post discusses how the X710 controller must be configured differently
from the 82599 10G controller (used by the X520 cards):
https://haryachyy.wordpress.com/2019/01/18/learning-dpdk-symmetric-rss/
The odd part is that following the SEPTun-MarkII guide[1] makes it *mostly*
work, but we're consistently finding that ~1-2% of the traffic is not being
symmetrically hashed. We're testing with can-i-use-afpacket-fanout[2] and
Zeek 3.2.
The most damning evidence is an Intel rep telling[3] a customer:
> Unfortunately, we have been informed that the only support to setup
symmetric RSS is via DPDK.
Searching the mailing list archives, I found a couple of posts where people
were encouraged to use X710-based cards, so I'm left wondering: Are there
people using these? Are they also seeing this 1-2% asymmetry? Or am I
missing a configuration tweak?
Thanks,
--Vlad
[1] - <
https://github.com/pevma/SEPTun-Mark-II/blob/master/SEPTun-Mark-II.rst>
[2] - <https://github.com/JustinAzoff/can-i-use-afpacket-fanout>
[3] - <
https://community.intel.com/t5/Ethernet-Products/X-L-710-supports-symmetric…
>
Hi all,
I have compiled and installed Spicy 1.3.0 in my Zeek 4.0.4 cluster (FreeBSD 13 based), but I try to execute “zeekctl deploy”, in stderr.log in all nodes appears the following error:
/opt/zeek/share/zeekctl/scripts/run-zeek: line 110: 55286 Abort trap nohup "$myzeek" "$@“
Any Idea?
My zkg installed packages are:
zeek/corelight/zeek-community-id (installed: 3.2.1) - "Community ID" flow hash support in conn.log
zeek/corelight/zeek-long-connections (installed: v1.1.0) - Find and log long-lived connections into a "conn_long" log.
zeek/salesforce/hassh (installed: master) - HASSH is used to identify specific Client and Server SSH implementations.
zeek/salesforce/ja3 (installed: master) - JA3 creates 32 character SSL client fingerprints and logs them as a field in ssl.log.
zeek/zeek/spicy-analyzers (installed: v0.2.27)
zeek/zeek/spicy-plugin (installed: v1.3.2)
zeek/zeek/zeek-netmap (installed: v2.0.0) - Packet source plugin that provides native Netmap support.
