We’re currently considering a rewrite of the existing main IO loop in Zeek (see https://github.com/zeek/zeek/issues/264 <https://github.com/zeek/zeek/issues/264> for more detail). As part of this work, we’re planning to remove the ability for a single Zeek process to handle multiple input sources (either interface or pcap) simultaneously, since it would simplify the code somewhat. The case of multiple interfaces is already handled by running multiple workers. Is anyone actively using the multiple pcap functionality that would be broken by this change?
Tim @ Corelight
Hi,
Can I replace a pattern in a string with “” but also return the matched pattern?
Basically I am extracting different blocks of text from a string using match_pattern() but then at the end, I want to search all the remaining text (minus the blocks that were already extracted).
I tried simply calling gsub(data, pattern, “”) just after calling block[n] = match_pattern(data, pattern) but this didn’t seem to benefit performance at all. I’m sure there must be a better way :/
Thanks in Advance,
Jonah
Hi everyone
I am having configure issue.
In node.cfg , ‘eth0 -i eth1’ config makes half of conn.log count than ‘eth0’
there is no traffic in eth1, in live monitoring eth1 is standby.
I test this live traffic and pfsend with pcap file from other server
(pfsend is feeding only to eth0 port)
Because of circumstances I can not use bridge setup.
I must be missing something.
Could any one point me to right direction?
My setup is blow
Zeek(bro) server
cpu: Intel(R) Xeon(R) CPU E5-2650 X 2 (total 32 core)
ram: 64G
zeek(bro) 2.4.2 with pf_ring 7.5.0 (not a zc)
no extra zeek(bro) script
server has two monitoring port
eth0(active), eth1 (standby)
node.cfg 'eth0 -i eth1'
[manager]
type=manager
host=localhost
[proxy-1]
type=proxy
host=localhost
[proxy-2]
type=proxy
host=localhost
[monitor]
type=worker
host=localhost
interface='eth0 -i eth1'
lb_method=pf_ring
lb_procs=10
pin_cpus=1,2,3,4,5,6,7,8,9,10
node.cfg eth0
[manager]
type=manager
host=localhost
[proxy-1]
type=proxy
host=localhost
[proxy-2]
type=proxy
host=localhost
[monitor]
type=worker
host=localhost
interface=eth0
lb_method=pf_ring
lb_procs=10
pin_cpus=1,2,3,4,5,6,7,8,9,10
--
------------------------------------------------------
Hichul Kim 김희철 선임 연구원
Naru Security (주)나루씨큐리티
Hi there
We have a small army of scanners that I want to exclude from zeek, so I
used the BPF filter option. Unfortunately, it's 166 IP addresses and is
triggering this "Too_Long_To_Compile_Filter" warning.
The documentation states "compensation measures may be taken by the
framework to reduce the filter size" - does that mean the filter is being
shortened? Ironically I'm mainly using the filter to remove stuff that zeek
shouldn't bother with - ie I'm removing load - which apparently is in
itself overloading zeek?
Are there other ways of removing noisy and/or masses of uninteresting
traffic - without needing to lean on our network team to start altering
SPAN traffic flows/etc?
Thanks!
--
Cheers
Jason Haar
Information Security Manager, Trimble Navigation Ltd.
Phone: +1 408 481 8171
PGP Fingerprint: 7A2E 0407 C9A6 CAF6 2B9F 8422 C063 5EBB FE1D 66D1
Hi Team
Can you please advice which of the logs contain the full urls to which I have accessed ?
All of the logs that I’ve gone thru contains only the domain and not the full request with the url.
Any help will be appreciated.
Alex.
Hi Everyone,
I'm using the "find_all_urls()" function from urls.zeek to extract all URLs from HTTP bodies. I occasionally errors such as this these:
1485557634.826679 error in /usr/local/zeek/share/zeek/base/utils/urls.zeek, line 122: bad conversion to count (to_count(parts[1]) and answers:PersonalBing:EZBubbleClose) no-repeat center;width:11px;height:11px;background-position-y:-10px}#hp_bottomCell #ezp_notification #ezp_bubble .ezp_bubble_close:hover{background-position-y:0}.ezp_location{font:14px)
1485557634.826679 error in /usr/local/zeek/share/zeek/base/utils/urls.zeek, line 122: bad conversion to count (to_count(parts[1]) and answers:PersonalBing:EZPanelClose) no-repeat center;width:11px;height:11px}.ezp_module{float:left;height:269px;width:255px;margin:25px 0;padding:0 42px}.ezp_module.ezp_module_narrow{width:122px}.ezp_module_leftseparator{border-left:1px solid #222}.ezp_module_title{font-size:20px;line-height:24px;margin-bottom:11px}.ezp_module_desc{font-size:16px;line-height:20px;margin-bottom:20px}.ezp_interests_icon{vertical-align:middle}.ezp_option_control{background:url(rms:)
1485557634.826679 error in /usr/local/zeek/share/zeek/base/utils/urls.zeek, line 122: bad conversion to count (to_count(parts[1]) and answers:PersonalBing:EZPanelClose) no-repeat center;width:11px;height:11px;position:relative;top:-22px;left:-10px}#hp_tbar.ezp_signin_message{background-image:-webkit-gradient(linear,left top,left bottom,from(rgba(0,0,0,.55)),to(rgba(0,0,0,.85)));background-image:-moz-linear-gradient(rgba(0,0,0,.55) 0,rgba(0,0,0,.85) 80%);background-image:-ms-linear-gradient(rgba(0,0,0,.55) 0,rgba(0,0,0,.85) 80%);background-image:-o-linear-gradient(rgba(0,0,0,.55) 0,rgba(0,0,0,.85) 80%);background-image:linear-gradient(rgba(0,0,0,.55) 0,rgba(0,0,0,.85) 80%)}.ezp_opened .ezp_barrier{display:block;background-color:#000;height:111px;margin:0 40px;position:relative;top:-185px;opacity:0}#sc_mdc.loading+.ezp_panelopened{margin-top:-46px}.ezp_icon{position:relative;top:-5px;left:0;cursor:pointer;background-color:rgba(34,34,34,.75);margin-right:1px;margin-bottom:-7px;-webkit-margin-after:-5px}#ezp_bubble_message{position:absolute;left:30px;background-color:rgba(0,0,0,.8);color:#fff;border:1px solid #333;padding:0 12px;font-size:13px;line-height:40px;height:40px;opacity:0}#ezp_bubble_message .ezp_info{vertical-align:middle;margin-right:12px}#ezp_bubble_message .ezp_bubble_down{background:url(rms:)
1378597102.912603 error in /usr/local/zeek/share/zeek/base/utils/urls.zeek, line 122: bad conversion to count (to_count(parts[1]) and )
www.iec.ch\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x16IEC http
I have a couple of questions regarding this:
1) When trying to resolve some of these issues, should I directly modify urls.zeek or will this have unintended consequences regarding other scripts/functionality in Zeek? The reason I ask this is when printing URLs extracted with the find_all_urls() function I get some results which are clearly not valid URLs e.g. "http://www.yootheme.com/license) */" - this should have cut off before the ")" which I believe are bug with urls.zeek rather than simply being intended functionality that I'd like to change.
2) Assuming I don't manage to fix all of these errors and choose to accept some, how can I stop them from printing to console each time I process a PCAP?
3) While trying to fix some of these errors with regex, I ran into the example "www.iec.ch\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x16IEC http". I've tried to strip everthing after the first "\" but this doesn't work due to it being Hex (I guess) rather than an actual "\", any ideas for this specific case?
4) Finally, a regex related question I've been meaning to ask for a while. Because I'm trying to extract URLs from HTML/JS, I need to deal with cases whitespace and multiple types of quote character may be used. When I've written projects in Python, I would create a variable with all of the possible characters in it and then I would use this variable in the regex e.g.
q = r"[\‘\’\'\"\s]*(?:"|')*"
pattern = q+r"userTokens"+q+r"(?::|=)"+q+r"(\w+)"+q
if re.search(pattern, data):
do something..
I can't workout how to do this with regex in Bro/Zeek scripts so I'm having to create incredibly long patterns to ensure all possible cases are met, if anybody can recommend a better way (like how I did it in Python), that would be awesome!
Thanks in Advance,
Jonah (CryptoCat)
Hi all,
I've got a site that i'm running BRO on that is generating TONS of DNS events. About 50% of all log file bytes are DNS related. And most of it is repeated lookup of a single a single domain name.
Is there any way (maybe using restrict_filters, maybe something else) to NOT log these DNS events for this specific hostname? I've done some poking around on google, but nothing's jumping out at me.
Thanks,
jason
<https://www.linkedin.com/company/rheagroup>
Hi Everyone,
This is my first time using this mailing list so I apologise in advance if I’ve followed the wrong format/protocol etc.
I am doing some malware research and making use of the HTTP.log generated by Bro. I’ve noticed some PCAPs fail to generate a HTTP log. I’ve looked at a couple of examples and thought maybe it is because there is no SYN-ACK before the HTTP connection in the PCAP (the researcher who generated the PCAP may have cut this out or not captured it).
Can anybody confirm why the HTTP.log fails to generate (is it the missing SYN-ACK at the start?) and advise if there is some way I can still extract the HTTP traffic from the PCAP using Bro (since it’s clearly all visible in Wireshark).
Note: I’m unable to attach screenshots of any of the problematic PCAPs due to email size..
Thanks in advance,
Jonah (@_CryptoCat)
Hey,
I'm working with Zeek scripts and I am running in an issue getting my
script to execute when zeek is running as a cluster. The script executes
when I start zeek w/ a pcap file. The script executes when I start zeek on
the command line, bind to the interface, and playback that pcap. The
script does not execute when I start zeek as cluster and playback the pcap
file. Other scripts, like 'extract-all-files.bro' run all 3 ways but in
the cluster, will not write my added print outs to the stdout file in
cluster mode. I have also confirmed that my scripts are being loaded by
the logging module when I run "zeekctl diag". I feel like I'm missing
something. Does anyone know what it is?
Thanks
--
