zeek November 2020

zeek@lists.zeek.org

14 participants
13 discussions

by Vlad Grigorescu

We recently purchased some Intel XXV710 NICs for our Zeek systems. However, symmetric hashing does not seem to work on them, at least not completely. There was some discussion here regarding adding some functionality to the driver to make it work, however this never landed: https://sourceforge.net/p/e1000/mailman/message/35199068/ This post discusses how the X710 controller must be configured differently from the 82599 10G controller (used by the X520 cards): https://haryachyy.wordpress.com/2019/01/18/learning-dpdk-symmetric-rss/ The odd part is that following the SEPTun-MarkII guide[1] makes it *mostly* work, but we're consistently finding that ~1-2% of the traffic is not being symmetrically hashed. We're testing with can-i-use-afpacket-fanout[2] and Zeek 3.2. The most damning evidence is an Intel rep telling[3] a customer: > Unfortunately, we have been informed that the only support to setup symmetric RSS is via DPDK. Searching the mailing list archives, I found a couple of posts where people were encouraged to use X710-based cards, so I'm left wondering: Are there people using these? Are they also seeing this 1-2% asymmetry? Or am I missing a configuration tweak? Thanks, --Vlad [1] - < https://github.com/pevma/SEPTun-Mark-II/blob/master/SEPTun-Mark-II.rst> [2] - <https://github.com/JustinAzoff/can-i-use-afpacket-fanout> [3] - < https://community.intel.com/t5/Ethernet-Products/X-L-710-supports-symmetric… >

2 months, 1 week

FTP::Info file_size field

by Darren S.

Greetings, I have a series of FTP file upload tests we're analyzing with Zeek 3.2.2. The environment is as follows: - FTP server: vsftpd 3.0.3 - FTP client: curl 7.64.0 - EPSV mode - Uploads with STOR command There are a number of fields documented for the FTP::Info record [1] that aren't logged (have "-" values for those fields) for these tests in ftp.log. One is `file_size`, documented as "Size of the file if the command indicates a file transfer." Logged records have values logged for args, mime_type, and fuid. But there is no value logged for file_size. The files have been extracted successfully, so the expectation is that given a STOR command was used ("...command indicates a file transfer") and that given a file was extracted and mime type identified, the file size would be logged in ftp.log. Is there potentially an issue in the analyzer? $ egrep '#fields|FsRuCZYQDY8FtmyS2' ftp.log #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p user password command arg mime_type file_size reply_code reply_msg data_channel.passive data_channel.orig_h data_channel.resp_h data_channel.resp_p fuid 1606347082.807480 CNp3Rz21qqtfdKWnG9 10.1.1.5 59888 x.x.219.95 9826 testuser <redacted> STOR ftp://x.x.219.95/home/testuser/archived-unencrypted.zip application/zip - 226 Transfer complete. - - - - FsRuCZYQDY8FtmyS2 1606347082.972373 CNp3Rz21qqtfdKWnG9 10.1.1.5 59888 x.x.219.95 9826 testuser <redacted> EPSV - - - 229 Entering Extended Passive Mode (|||33369|) T 10.1.1.5 x.x.219.95 33369 FsRuCZYQDY8FtmyS2 Also noticing that in files.log there is a record of the extracted file from the data channel, but no associated file name. Is this expected for FTP_DATA since the data channel is just a stream of data with no indication of file name (i.e. not informed by the control channel)? #fields ts fuid tx_hosts rx_hosts conn_uids source depth analyzers mime_type filename duration local_orig is_orig seen_bytes total_bytes missing_bytes overflow_bytes timedout parent_fuid md5 sha1 sha256 extracted extracted_cutoff extracted_size 1606347082.859187 FsRuCZYQDY8FtmyS2 10.1.1.5 x.x.219.95 C66As819fJARn0a3kj FTP_DATA 0 EXTRACT,SHA1,MD5 application/zip - 0.000170 - T 6187 - 0 0 F - 68a7676890bda812d1818269e9b942bc 633cb66a0565b4ed049cf4d65ed689bfe973ee51 - FTP_DATA-FsRuCZYQDY8FtmyS2.zip F - [1] https://docs.zeek.org/en/current/scripts/base/protocols/ftp/info.zeek.html#… - Darren

1 year, 1 month

ciphers and protocols aren't shown in ssl.log

by Rainer Duffner

Hi, I installed zeek 3.2.2 on CentOS 7 (amd64) I want to be able to log the ssl ciphers and protocols used on a host so we can get an overview of how many old clients are connecting However, in the ssl.log, it does not log that information. [root@zeek current]# ll insgesamt 68 -rw-r--r--. 1 root zeek 2316 27. Nov 21:44 conn.log -rw-r--r--. 1 root zeek 581 27. Nov 21:44 dns.log -rw-r--r--. 1 root zeek 26221 27. Nov 21:43 loaded_scripts.log -rw-r--r--. 1 root zeek 600 27. Nov 21:44 ntp.log -rw-r--r--. 1 root zeek 227 27. Nov 21:43 packet_filter.log -rw-r--r--. 1 root zeek 666 27. Nov 21:44 reporter.log -rw-r--r--. 1 root zeek 497 27. Nov 21:44 ssl.log -rw-r--r--. 1 root zeek 686 27. Nov 21:43 stats.log -rw-r--r--. 1 root zeek 20 27. Nov 21:43 stderr.log -rw-r--r--. 1 root zeek 188 27. Nov 21:43 stdout.log -rw-r--r--. 1 root zeek 983 27. Nov 21:44 weird.log [root@zeek current]# cat ssl.log #separator \x09 #set_separator , #empty_field (empty) #unset_field - #path ssl #open 2020-11-27-21-44-28 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p version cipher curve server_name resumed last_alert next_protocol established cert_chain_fuids client_cert_chain_fuids subject issuer client_subject client_issuer validation_status #types time string addr port addr port string string string string bool string string bool vector[string] vector[string] string string string string string 1606509862.986251 CCB2e543osaLm9T38 192.168.1.238 52108 aaa.bb.ccc.68 443 - - - server68.domain.org F - - F - - - - - - - In share/zeek/site/local.zeek I’ve added @load protocols/ssl/weak-keys In share/zeek/policy/protocols/ssl/weak-keys.zeek I’ve set option tls_minimum_version = TLSv12; Is there anything else to do? I run this in a local VMWare Fusion VM. I connected to the host above via curl -v —tlsv1.0 Rainer

1 year, 1 month

Proposal to remove the finger protocol analyzer

by Vlad Grigorescu

(For some additional context, please see: https://github.com/zeek/zeek/pull/1243) I'd like to remove the finger analyzer from core Zeek. The analyzer that's in place today is incomplete, untested, and is showing its age. It's not enabled by default, and the functionality to use it is not in Zeek. As such, I'm proposing that it's simply removed, and not deprecated first. If you rely on this analyzer today, and feel strongly about it staying, please let me know. I'd be interested in hearing your use-case(s) and working on bringing it out of its current state of purgatory, perhaps as a plug-in. Lovers of ancient protocols should fear not, however, as I plan on adding a whois protocol analyzer. The two protocols are closely related, and whois could be a complete analyzer, with tests. In addition, whois seems more operationally useful today, as we see attackers running it on once they gain a foothold, and more tech-savvy users will run it after receiving a suspicious e-mail. Many thanks, --Vlad

1 year, 1 month

2021 People of Zeek Interview Series

by Amber Graner

Hi all, We're interested in interviewing various Zeek users and developers and would like to get this list of potential interviewees lined up prior to January 2021. Is there anyone in the community that you think has an interesting story around their Zeek usage or contributions? Thanks in advance for your suggestions and feedback. With gratitude, ~Amber

1 year, 2 months

how zeek passing parameters to custom events

by qing wang

I have some questions about custom events. I found that someone defined custom event in his zeek script. like that: global myevent: event(ts: time, id: string, context: table[string] of any); but i never seen how he pass parameters to myevent. i want to known how zeek worked with that -- С уважением

1 year, 2 months

Re: Error installing af_packet plugin for Zeek 3.0.11 (SOLVED)

by Carlos Lopez

I have found the problem. It was PATH variable. I don’t know why, but RedHat put the following option in profile’s user: # User specific environment if ! [[ "$PATH" =~ "$HOME/.local/bin:$HOME/bin:" ]] then PATH="$HOME/.local/bin:$HOME/bin:$PATH" fi export PATH As you can see, the error appears due to zkg installs a “zeek” python script wrapper, and it is executed first during zkg tests to install af_packet plugin instead of the real zeek binary located in /opt/zeek/bin, in my case. Regards. From: Carlos Lopez <clopmz(a)outlook.com> Date: Monday, 9 November 2020 at 17:01 To: Vlad Grigorescu <vlad(a)es.net> Cc: "zeek(a)lists.zeek.org" <zeek(a)lists.zeek.org> Subject: Re: [Zeek] Error installing af_packet plugin for Zeek 3.0.11 Uhmm … I have the same config in another RHEL8 host and works without problems: zeek@rhelzeek05:~/.zkg$ ls -al total 8 drwxr-xr-x. 8 zeek idps 133 Nov 9 15:49 . drwx------. 7 zeek idps 167 Nov 9 15:48 .. drwxr-xr-x. 4 zeek idps 35 Nov 9 15:48 clones -rw-r--r--. 1 zeek idps 205 Nov 9 15:59 config drwxr-xr-x. 2 zeek idps 45 Nov 9 15:49 logs -rw-r--r--. 1 zeek idps 164 Nov 9 15:49 manifest.json drwxr-xr-x. 2 zeek idps 6 Nov 9 15:49 plugin_dir drwxr-xr-x. 4 zeek idps 48 Nov 9 15:51 scratch drwxr-xr-x. 2 zeek idps 6 Nov 9 15:49 script_dir drwxr-xr-x. 3 zeek idps 35 Nov 9 15:50 testing zeek@ rhelzeek05:~/.zkg$ more config [sources] zeek = https://github.com/zeek/packages [paths] state_dir = /nsm/zeek/.zkg script_dir = /opt/zeek/share/zeek/site plugin_dir = /opt/zeek/lib/zeek/plugins zeek_dist = /usr/local/src/zeek-3.0.11 From: Vlad Grigorescu <vlad(a)es.net> Date: Monday, 9 November 2020 at 16:55 To: Carlos Lopez <clopmz(a)outlook.com> Cc: "zeek(a)lists.zeek.org" <zeek(a)lists.zeek.org> Subject: Re: [Zeek] Error installing af_packet plugin for Zeek 3.0.11 The diff in the test failure is because Zeek failed to load the plugin. I'm guessing this is due to the non-standard install location. Did you run "zkg autoconfig?" Can you provide your zkg configuration file, ~/.zkg/zkg.conf as whatever user you tried running zkg? --Vlad On Mon, Nov 9, 2020 at 3:04 AM Carlos Lopez <clopmz(a)outlook.com<mailto:clopmz@outlook.com>> wrote: Hi all, I'm trying to install a new server with Zeek 3.0.11 under RHEL8 and when I try to install the af_packet plugin I get the following error: scripts.show-plugin ... failed % 'btest-diff output' failed unexpectedly (exit code 1) % cat .diag == File =============================== == Diff =============================== --- /tmp/test-diff.37542.output.baseline.tmp 2020-11-09 08:58:38.353727203 +0000 +++ /tmp/test-diff.37542.output.tmp 2020-11-09 08:58:38.347727173 +0000 @@ -1,10 +0,0 @@ -Zeek::AF_Packet - Packet acquisition via AF_Packet (dynamic, version) - [Packet Source] AF_PacketReader (interface prefix "af_packet"; supports live input) - [Type] AF_Packet::FanoutMode - [Constant] AF_Packet::buffer_size - [Constant] AF_Packet::enable_hw_timestamping - [Constant] AF_Packet::enable_fanout - [Constant] AF_Packet::enable_defrag - [Constant] AF_Packet::fanout_mode - [Constant] AF_Packet::fanout_id - ======================================= % cat .stderr Traceback (most recent call last): File "/nsm/zeek/.local/bin/zeek", line 11, in <module> sys.exit(main()) File "/nsm/zeek/.local/lib/python3.6/site-packages/zeek.py", line 18, in main cli(auto_envvar_prefix='ZEEK') File "/nsm/zeek/.local/lib/python3.6/site-packages/click/core.py", line 829, in __call__ return self.main(*args, **kwargs) File "/nsm/zeek/.local/lib/python3.6/site-packages/click/core.py", line 760, in main _verify_python3_env() File "/nsm/zeek/.local/lib/python3.6/site-packages/click/_unicodefun.py", line 130, in _verify_python3_env " mitigation steps.{}".format(extra) RuntimeError: Click will abort further execution because Python 3 was configured to use ASCII as encoding for the environment. Consult https://click.palletsprojects.com/python3/ for mitigation steps. This system supports the C.UTF-8 locale which is recommended. You might be able to resolve your issue by exporting the following environment variables: export LC_ALL=C.UTF-8 export LANG=C.UTF-8 1 of 1 test failed ¿¿??? I've never seen a this error installing af_packet plugin ... What does it mean? Regards, C. L. Martinez -- zeek mailing list -- zeek(a)lists.zeek.org<mailto:zeek@lists.zeek.org> To unsubscribe send an email to zeek-leave(a)lists.zeek.org<mailto:zeek-leave@lists.zeek.org>

1 year, 2 months

Error installing af_packet plugin for Zeek 3.0.11

by Carlos Lopez

Hi all, I'm trying to install a new server with Zeek 3.0.11 under RHEL8 and when I try to install the af_packet plugin I get the following error: scripts.show-plugin ... failed % 'btest-diff output' failed unexpectedly (exit code 1) % cat .diag == File =============================== == Diff =============================== --- /tmp/test-diff.37542.output.baseline.tmp 2020-11-09 08:58:38.353727203 +0000 +++ /tmp/test-diff.37542.output.tmp 2020-11-09 08:58:38.347727173 +0000 @@ -1,10 +0,0 @@ -Zeek::AF_Packet - Packet acquisition via AF_Packet (dynamic, version) - [Packet Source] AF_PacketReader (interface prefix "af_packet"; supports live input) - [Type] AF_Packet::FanoutMode - [Constant] AF_Packet::buffer_size - [Constant] AF_Packet::enable_hw_timestamping - [Constant] AF_Packet::enable_fanout - [Constant] AF_Packet::enable_defrag - [Constant] AF_Packet::fanout_mode - [Constant] AF_Packet::fanout_id - ======================================= % cat .stderr Traceback (most recent call last): File "/nsm/zeek/.local/bin/zeek", line 11, in <module> sys.exit(main()) File "/nsm/zeek/.local/lib/python3.6/site-packages/zeek.py", line 18, in main cli(auto_envvar_prefix='ZEEK') File "/nsm/zeek/.local/lib/python3.6/site-packages/click/core.py", line 829, in __call__ return self.main(*args, **kwargs) File "/nsm/zeek/.local/lib/python3.6/site-packages/click/core.py", line 760, in main _verify_python3_env() File "/nsm/zeek/.local/lib/python3.6/site-packages/click/_unicodefun.py", line 130, in _verify_python3_env " mitigation steps.{}".format(extra) RuntimeError: Click will abort further execution because Python 3 was configured to use ASCII as encoding for the environment. Consult https://click.palletsprojects.com/python3/ for mitigation steps. This system supports the C.UTF-8 locale which is recommended. You might be able to resolve your issue by exporting the following environment variables: export LC_ALL=C.UTF-8 export LANG=C.UTF-8 1 of 1 test failed ¿¿??? I've never seen a this error installing af_packet plugin ... What does it mean? Regards, C. L. Martinez

1 year, 2 months

[Reminder] - Monthly Community Call Today

by Amber Graner

Hi all, Just a quick reminder that we have a monthly community call (30 mins) today 6 Nov 2020 from 3-3:30pm Eastern. *Agenda* This is a monthly call with the community to discuss topics related to growing and sustaining the Zeek Community. This call is a public call and will be recorded and shared. Registration is required.(Link below) * ZeekWeek 2020 Summary and high level plans for 2021(10 mins) * Formation of a Docs Team (10 mins) - High Level * Volunteer Opportunities (5 mins) - Areas where you can get involved * Zeek Package Contest (ZPC-3) (5 Mins) - Package Contest extended (update) *Registration* Registration Link https://corelight.zoom.us/meeting/register/tJcldO6qrTMrG9Kwsu6_qHsUeAvdjLmM… <https://www.google.com/url?q=https://corelight.zoom.us/meeting/register/tJc…> Once you register you will be sent the dial in information. Please let me know if you have any questions. Thanks, ~Amber

1 year, 2 months

Virtual ZeekWeek 2020: Summary, Slides and Video - Now Available

by Amber Graner

Hi all, The Virtual ZeekWeek 2020: Summary, Slides and Video are now available and can be found at: https://zeek.org/2020/11/06/virtual-zeekweek-2020-summary-slides-and-video/ Please note the Day 1 training videos are still be edited and will be available soon. Thanks, ~Amber

1 year, 2 months

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

zeek November 2020