Hi all,
Recently I have some problems with Bro and PF_RING in cluster.
On my server, when I have less than 32 worker threads(rings),
everything is okay, but when I use worker threads more than 32, pf_ring
start to receive repeating data packets. For example, rings less than 32, I
send 400000 packets to server and pf_ring info in /proc shows there is
400000 packets in rings, but when rings greater than 32, I can get 800000
packets when 33 rings and 1200000 packets when 34 rings and so on.
I guess if there is some rules that a pf_ring or a bro cluster can only
support less than 32 rings or worker threads on a server or some other
reasons?
Any insight would be helpful.
We recently purchased some Intel XXV710 NICs for our Zeek systems. However,
symmetric hashing does not seem to work on them, at least not completely.
There was some discussion here regarding adding some functionality to the
driver to make it work, however this never landed:
https://sourceforge.net/p/e1000/mailman/message/35199068/
This post discusses how the X710 controller must be configured differently
from the 82599 10G controller (used by the X520 cards):
https://haryachyy.wordpress.com/2019/01/18/learning-dpdk-symmetric-rss/
The odd part is that following the SEPTun-MarkII guide[1] makes it *mostly*
work, but we're consistently finding that ~1-2% of the traffic is not being
symmetrically hashed. We're testing with can-i-use-afpacket-fanout[2] and
Zeek 3.2.
The most damning evidence is an Intel rep telling[3] a customer:
> Unfortunately, we have been informed that the only support to setup
symmetric RSS is via DPDK.
Searching the mailing list archives, I found a couple of posts where people
were encouraged to use X710-based cards, so I'm left wondering: Are there
people using these? Are they also seeing this 1-2% asymmetry? Or am I
missing a configuration tweak?
Thanks,
--Vlad
[1] - <
https://github.com/pevma/SEPTun-Mark-II/blob/master/SEPTun-Mark-II.rst>
[2] - <https://github.com/JustinAzoff/can-i-use-afpacket-fanout>
[3] - <
https://community.intel.com/t5/Ethernet-Products/X-L-710-supports-symmetric…
>
Hi guys,
I wonder if anyone can offer any advice in relation to an issue we have using Zeek (LTS 4.0.3), and a Myricom 10G-PCIE2-8C2-2S. The Myricom card is currently on a SPAN port from a Juniper QFX, albeit we’re planning to move to a Profitap fibre TAP soon.
We’ve compiled Zeek using sources in order to accommodate the snf driver (e.g ./configure --with-pcap=/opt/snf/), and it works well using the following node.cfg configuration -
[worker-1]
type=worker
host=localhost
#pin_cpus=1,3,5,7
interface=snf0
lb_method=myricom
lb_procs=8
Our issue, is that when we try to filter traffic, either using ZeekArgs, or redef PacketFilter::default_capture_filter, workers crash within a few minutes of starting the process.
We’re trying to use a simple capture filter like -
ZeekArgs = -f "not dst host 10.100.48.5 and not dst host 10.100.40.78”
Or
redef PacketFilter::default_capture_filter = "not host 10.100.48.5";
The output of the crash diag is attached, but in short, we experience -
Program terminated with signal SIGSEGV, Segmentation fault.
#0 zeek::packet_analysis::Ethernet::EthernetAnalyzer::AnalyzePacket (this=0x5560e04be680, len=808, data=0x41d853675b0719a8 <error: Cannot access memory at address 0x41d853675b0719a8>, packet=0x5560e171b9c8) at /root/zeek-4.0.3/src/packet_analysis/protocol/ethernet/Ethernet.cc:33
33 if ( data[12] == 0x89 && data[13] == 0x03 )
[Current thread is 1 (Thread 0x7f90ea7172c0 (LWP 4830))]
If we remove the BPF or capture filter, the processes stay online consistently.
Any advise on how to diagnose this would be greatly appreciated.
Best regards
Andy
I haven't redef'd a variable from the command line since Bro 2.5, now
running Zeek 4.2.0-dev.78 and I'm getting the following error:
*error in <params>, line 1: syntax error, at or near "redef" or end of file
./test.zeek*
Example is super simple for trying to understand what I'm doing wrong -
test.zeek:
*const s1 = "world" &redef;*
*print fmt("hello %s", s1);*
On the command line:
> zeek ./test.zeek s1="zeek"
* error in <params>, line 1: syntax error, at or near "redef" or end of
file ./test.zeek*
If I modify the script as follows it works fine:
*const s1 = "world" &redef;redef s1="zeek";*
*print fmt("hello %s", s1);*
> zeek ./test.zeek
hello zeek
What am I missing??
~Troy
Greetings, Zeek community!
It's been a while, but new GA releases of the Brim desktop app (v0.25.0 <https://github.com/brimdata/brim/releases/tag/v0.25.0>) and Zed backend/CLI tooling (v0.30.0 <https://github.com/brimdata/zed/releases/tag/v0.30.0>) have finally arrived!
Since this is our first release since February, there's too many changes to cover in a brief email. There's full details in release notes at the links above. A few highlights:
The storage used by Brim to hold your Zeek/Suricata/other logs is now a "Zed lake". Though the introduction of Zed lakes causes no immediate change to your favorite Brim workflows, they unlock powerful new functionality that will be revealed in Brim going forward, including Git-like branching. See the Zed lake README <https://github.com/brimdata/zed/blob/main/docs/lake/README.md> for details.
Enhancements have been made to the Zed language to unify search and expression syntax, introduce new operators and functions for data exploration and shaping, and more! Review the Zed language docs <https://github.com/brimdata/zed/blob/main/docs/language/README.md> for details.
pcap processing is now handled by a separate, new component called Brimcap. Your favorite pcap workflows in Brim have not changed, but Brimcap also opens up new flexible custom configurations and can be used as a standalone tool. For more info, check out the Brimcap README <https://github.com/brimdata/brimcap/blob/main/README.md> and wiki <https://github.com/brimdata/brimcap/wiki>.
Other links of general interest:
Download page <https://www.brimsecurity.com/download/> for the Brim application
Brim's YouTube channel <https://www.youtube.com/channel/UC0ju7Esmh13oLS8FTS-B3Eg>, which includes app demos and info for developers (admittedly getting a bit dated)
Join our public Slack workspace <https://www.brimsecurity.com/join-slack/> for announcements, Q&A, feedback, and to trade ideas
Have fun!
--
Phil & the Brim team
Hi folks, I am currently struggling how to analyse single big (500MB-1GB+) pcap file with zeek in a short time. I am using standalone mode of zeek and with this command: zeek -Cr sample_500MB.pcap local it took about 1.40min which is a bit too long for my needs...
What do you advise, how can I get zeek -r multithreadded behaviour like? Documentation says, use cluster with several workers (even on one node) but I am not sure if you can read pcap in cluster mode? Another option is tcpreplay and let workers generate logs in cluster mode?
So what do you guys advise, I am doing only offline pcap analysis with single 500+MB pcap and zeek should use ideally 4-6 cores instead of one.
Thank you.
Hi Folks,
I am currently using the Apache Kafka plugin to send logs to the Kafka
server. The current setup is designed to use the plugin to send each zeek
log(5 log streams) to a specific kafka topic as described here
<https://github.com/apache/metron-bro-plugin-kafka#example-4---send-each-zee…>
.
But the requirement now is to:
1 - Continuing to send the existing zeek logs to their specific topics as
described above.
2 - Send all the other *selected* log streams to just one topic name on
kafka cluster.
I have tried to set this up but it does not seem to work. Is this possible
to do?
Appreciate your help!
Thanks,
Jahan
Hi all,
Wednesday 15 Sep, 2021 is going to be a busy, yet fun day. We have a
virtual meetup AND we'll be recording another Zeek Webinar. Details below
on how you can participate in each of these events.
- 15 September 2021 – VIRTUAL ZEEK MEETUP – 10am – 11:30am Pacific/12pm
– 1:30pm Eastern – GUEST SPEAKER – *Richard Bejtlich, *
https://www.taosecurity.com/, will be our speaker at this event. He’ll
be speaking about, “How to monitor your wireless network?” Register to
attend this meetup at:
https://www.meetup.com/greater-boston-area-open-source-zeek-meetup-group/ev…
- 15 September 2021 – ZEEK WEBINAR SERIES: How Zeek and Suricata work
together: Complementary not Competitive! – 11am Pacific/2pm Eastern – On
the Zeek Slack workspace and other areas in the Zeek Community, we are
often asked, “Should I use Zeek or Suricata to monitor my network?” In this
webinar, *Alex Kirk* discusses how Zeek and Suricata can be used
together and how they are complementary not competitive tools. This webinar
is free, but registration is required. Register at:
https://event.webinarjam.com/register/29/vlq6ytvm
Please let me know if you have any questions.
Thanks,
~Amber
