Hi,
Can I replace a pattern in a string with “” but also return the matched pattern?
Basically I am extracting different blocks of text from a string using match_pattern() but then at the end, I want to search all the remaining text (minus the blocks that were already extracted).
I tried simply calling gsub(data, pattern, “”) just after calling block[n] = match_pattern(data, pattern) but this didn’t seem to benefit performance at all. I’m sure there must be a better way :/
Thanks in Advance,
Jonah
Hi everyone
I am having configure issue.
In node.cfg , ‘eth0 -i eth1’ config makes half of conn.log count than ‘eth0’
there is no traffic in eth1, in live monitoring eth1 is standby.
I test this live traffic and pfsend with pcap file from other server
(pfsend is feeding only to eth0 port)
Because of circumstances I can not use bridge setup.
I must be missing something.
Could any one point me to right direction?
My setup is blow
Zeek(bro) server
cpu: Intel(R) Xeon(R) CPU E5-2650 X 2 (total 32 core)
ram: 64G
zeek(bro) 2.4.2 with pf_ring 7.5.0 (not a zc)
no extra zeek(bro) script
server has two monitoring port
eth0(active), eth1 (standby)
node.cfg 'eth0 -i eth1'
[manager]
type=manager
host=localhost
[proxy-1]
type=proxy
host=localhost
[proxy-2]
type=proxy
host=localhost
[monitor]
type=worker
host=localhost
interface='eth0 -i eth1'
lb_method=pf_ring
lb_procs=10
pin_cpus=1,2,3,4,5,6,7,8,9,10
node.cfg eth0
[manager]
type=manager
host=localhost
[proxy-1]
type=proxy
host=localhost
[proxy-2]
type=proxy
host=localhost
[monitor]
type=worker
host=localhost
interface=eth0
lb_method=pf_ring
lb_procs=10
pin_cpus=1,2,3,4,5,6,7,8,9,10
--
------------------------------------------------------
Hichul Kim 김희철 선임 연구원
Naru Security (주)나루씨큐리티
Hi there
We have a small army of scanners that I want to exclude from zeek, so I
used the BPF filter option. Unfortunately, it's 166 IP addresses and is
triggering this "Too_Long_To_Compile_Filter" warning.
The documentation states "compensation measures may be taken by the
framework to reduce the filter size" - does that mean the filter is being
shortened? Ironically I'm mainly using the filter to remove stuff that zeek
shouldn't bother with - ie I'm removing load - which apparently is in
itself overloading zeek?
Are there other ways of removing noisy and/or masses of uninteresting
traffic - without needing to lean on our network team to start altering
SPAN traffic flows/etc?
Thanks!
--
Cheers
Jason Haar
Information Security Manager, Trimble Navigation Ltd.
Phone: +1 408 481 8171
PGP Fingerprint: 7A2E 0407 C9A6 CAF6 2B9F 8422 C063 5EBB FE1D 66D1
Hi Team
Can you please advice which of the logs contain the full urls to which I have accessed ?
All of the logs that I’ve gone thru contains only the domain and not the full request with the url.
Any help will be appreciated.
Alex.
Hi Everyone,
I'm using the "find_all_urls()" function from urls.zeek to extract all URLs from HTTP bodies. I occasionally errors such as this these:
1485557634.826679 error in /usr/local/zeek/share/zeek/base/utils/urls.zeek, line 122: bad conversion to count (to_count(parts[1]) and answers:PersonalBing:EZBubbleClose) no-repeat center;width:11px;height:11px;background-position-y:-10px}#hp_bottomCell #ezp_notification #ezp_bubble .ezp_bubble_close:hover{background-position-y:0}.ezp_location{font:14px)
1485557634.826679 error in /usr/local/zeek/share/zeek/base/utils/urls.zeek, line 122: bad conversion to count (to_count(parts[1]) and answers:PersonalBing:EZPanelClose) no-repeat center;width:11px;height:11px}.ezp_module{float:left;height:269px;width:255px;margin:25px 0;padding:0 42px}.ezp_module.ezp_module_narrow{width:122px}.ezp_module_leftseparator{border-left:1px solid #222}.ezp_module_title{font-size:20px;line-height:24px;margin-bottom:11px}.ezp_module_desc{font-size:16px;line-height:20px;margin-bottom:20px}.ezp_interests_icon{vertical-align:middle}.ezp_option_control{background:url(rms:)
1485557634.826679 error in /usr/local/zeek/share/zeek/base/utils/urls.zeek, line 122: bad conversion to count (to_count(parts[1]) and answers:PersonalBing:EZPanelClose) no-repeat center;width:11px;height:11px;position:relative;top:-22px;left:-10px}#hp_tbar.ezp_signin_message{background-image:-webkit-gradient(linear,left top,left bottom,from(rgba(0,0,0,.55)),to(rgba(0,0,0,.85)));background-image:-moz-linear-gradient(rgba(0,0,0,.55) 0,rgba(0,0,0,.85) 80%);background-image:-ms-linear-gradient(rgba(0,0,0,.55) 0,rgba(0,0,0,.85) 80%);background-image:-o-linear-gradient(rgba(0,0,0,.55) 0,rgba(0,0,0,.85) 80%);background-image:linear-gradient(rgba(0,0,0,.55) 0,rgba(0,0,0,.85) 80%)}.ezp_opened .ezp_barrier{display:block;background-color:#000;height:111px;margin:0 40px;position:relative;top:-185px;opacity:0}#sc_mdc.loading+.ezp_panelopened{margin-top:-46px}.ezp_icon{position:relative;top:-5px;left:0;cursor:pointer;background-color:rgba(34,34,34,.75);margin-right:1px;margin-bottom:-7px;-webkit-margin-after:-5px}#ezp_bubble_message{position:absolute;left:30px;background-color:rgba(0,0,0,.8);color:#fff;border:1px solid #333;padding:0 12px;font-size:13px;line-height:40px;height:40px;opacity:0}#ezp_bubble_message .ezp_info{vertical-align:middle;margin-right:12px}#ezp_bubble_message .ezp_bubble_down{background:url(rms:)
1378597102.912603 error in /usr/local/zeek/share/zeek/base/utils/urls.zeek, line 122: bad conversion to count (to_count(parts[1]) and )
www.iec.ch\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x16IEC http
I have a couple of questions regarding this:
1) When trying to resolve some of these issues, should I directly modify urls.zeek or will this have unintended consequences regarding other scripts/functionality in Zeek? The reason I ask this is when printing URLs extracted with the find_all_urls() function I get some results which are clearly not valid URLs e.g. "http://www.yootheme.com/license) */" - this should have cut off before the ")" which I believe are bug with urls.zeek rather than simply being intended functionality that I'd like to change.
2) Assuming I don't manage to fix all of these errors and choose to accept some, how can I stop them from printing to console each time I process a PCAP?
3) While trying to fix some of these errors with regex, I ran into the example "www.iec.ch\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x16IEC http". I've tried to strip everthing after the first "\" but this doesn't work due to it being Hex (I guess) rather than an actual "\", any ideas for this specific case?
4) Finally, a regex related question I've been meaning to ask for a while. Because I'm trying to extract URLs from HTML/JS, I need to deal with cases whitespace and multiple types of quote character may be used. When I've written projects in Python, I would create a variable with all of the possible characters in it and then I would use this variable in the regex e.g.
q = r"[\‘\’\'\"\s]*(?:"|')*"
pattern = q+r"userTokens"+q+r"(?::|=)"+q+r"(\w+)"+q
if re.search(pattern, data):
do something..
I can't workout how to do this with regex in Bro/Zeek scripts so I'm having to create incredibly long patterns to ensure all possible cases are met, if anybody can recommend a better way (like how I did it in Python), that would be awesome!
Thanks in Advance,
Jonah (CryptoCat)
Hi all,
I've got a site that i'm running BRO on that is generating TONS of DNS events. About 50% of all log file bytes are DNS related. And most of it is repeated lookup of a single a single domain name.
Is there any way (maybe using restrict_filters, maybe something else) to NOT log these DNS events for this specific hostname? I've done some poking around on google, but nothing's jumping out at me.
Thanks,
jason
<https://www.linkedin.com/company/rheagroup>
Hi Everyone,
This is my first time using this mailing list so I apologise in advance if I’ve followed the wrong format/protocol etc.
I am doing some malware research and making use of the HTTP.log generated by Bro. I’ve noticed some PCAPs fail to generate a HTTP log. I’ve looked at a couple of examples and thought maybe it is because there is no SYN-ACK before the HTTP connection in the PCAP (the researcher who generated the PCAP may have cut this out or not captured it).
Can anybody confirm why the HTTP.log fails to generate (is it the missing SYN-ACK at the start?) and advise if there is some way I can still extract the HTTP traffic from the PCAP using Bro (since it’s clearly all visible in Wireshark).
Note: I’m unable to attach screenshots of any of the problematic PCAPs due to email size..
Thanks in advance,
Jonah (@_CryptoCat)
Hey,
I'm working with Zeek scripts and I am running in an issue getting my
script to execute when zeek is running as a cluster. The script executes
when I start zeek w/ a pcap file. The script executes when I start zeek on
the command line, bind to the interface, and playback that pcap. The
script does not execute when I start zeek as cluster and playback the pcap
file. Other scripts, like 'extract-all-files.bro' run all 3 ways but in
the cluster, will not write my added print outs to the stdout file in
cluster mode. I have also confirmed that my scripts are being loaded by
the logging module when I run "zeekctl diag". I feel like I'm missing
something. Does anyone know what it is?
Thanks
--
I'm new to Zeek. I noticed my Zeek is crashing every few days. I have the below bt
Core was generated by `/opt/bro/bin/bro -U .status -p broctl -p broctl-live -p local -p logger local.b'.
Program terminated with signal 11, Segmentation fault.
#0 0x0000000000894795 in logging::Manager::CheckFilterWriterConflict(logging::Manager::WriterInfo const*, logging::Manager::Filter const*) ()
(gdb) bt
#0 0x0000000000894795 in logging::Manager::CheckFilterWriterConflict(logging::Manager::WriterInfo const*, logging::Manager::Filter const*) ()
#1 0x000000000089999f in logging::Manager::Write(EnumVal*, RecordVal*) ()
#2 0x00000000005d5436 in BifFunc::Log::bro___write(Frame*, ValPList*) ()
#3 0x00000000005e502f in BuiltinFunc::Call(ValPList*, Frame*) const ()
#4 0x00000000005c7f46 in CallExpr::Eval(Frame*) const ()
#5 0x0000000000639351 in StmtList::Exec(Frame*, stmt_flow_type&) const ()
#6 0x00000000005f1294 in BroFunc::Call(ValPList*, Frame*) const ()
#7 0x00000000005c7f46 in CallExpr::Eval(Frame*) const ()
#8 0x000000000063aee4 in ExprStmt::Exec(Frame*, stmt_flow_type&) const ()
#9 0x0000000000639351 in StmtList::Exec(Frame*, stmt_flow_type&) const ()
#10 0x00000000005f1294 in BroFunc::Call(ValPList*, Frame*) const ()
#11 0x00000000005ade1a in EventHandler::Call(ValPList*, bool) ()
#12 0x00000000005ad106 in EventMgr::Drain() ()
#13 0x0000000000601b49 in net_run() ()
#14 0x000000000055e4a3 in main ()
I'm not sure what do make out of it. Everything was stable until two things changed. I went from stand alone to clustermode on the one server as I brought in another interface for a different set of networks. So now there are two network interfaces receiving packets in a cluster on one physical server. Any troubleshooting steps appreciated.
thanks
