zeek-dev September 2014

zeek-dev@lists.zeek.org

157 discussions

by Gilbert Clark

Hi list: Does anyone know of work that involves placing hard limits on the amount of time bro is able to spend processing individual packets? Specifically, I'm looking for: * work toward putting hard limits on the number of cycles Bro is allowed to execute per-packet before injecting a hard stop and forcing the engine to move to the next packet, or * work toward emulating buffers / drops based on the number of cycles bro spends processing a particular packet in offline / pseudo-realtime mode Thanks in advance for any references / thoughts! --Gilbert

6 years, 5 months

[Bro-Dev] [JIRA] (BIT-1257) Same file id generated for potentially different files

by Seth Hall (JIRA)

[ https://bro-tracker.atlassian.net/browse/BIT-1257?page=com.atlassian.jira.p… ] Seth Hall reassigned BIT-1257: ------------------------------ Assignee: Seth Hall > Same file id generated for potentially different files > ------------------------------------------------------ > > Key: BIT-1257 > URL: https://bro-tracker.atlassian.net/browse/BIT-1257 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master, 2.3 > Environment: CentOS 6 > Reporter: Jimmy Jones > Assignee: Seth Hall > Attachments: fa.bro, sample-samefileid.pcap > > > Attached sample contains two HTTP downloads of the same URL from the same client, but there are no guarantees that the files is actually the same (no Etags etc - in this case it actually is the same, but lets pretend they were different...). However the file analysis framework seems to give the same file ID in file_name and file_chunk for both downloads. > Think this is something to do with Range requests as doesn't happen if do "normal" HTTP requests. -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003)

6 years, 5 months

[Bro-Dev] [JIRA] (BIT-1257) Same file id generated for potentially different files

by Seth Hall (JIRA)

[ https://bro-tracker.atlassian.net/browse/BIT-1257?page=com.atlassian.jira.p… ] Seth Hall commented on BIT-1257: -------------------------------- Ah! I think it's perfectly reasonable to make our default behavior a bit closer to RFC2616. I'll take a look into it soon. At the very least, if someone does want the more liberal file combining they can add it back with a separate script (which I'll probably include with Bro somewhere). I'll take this ticket to make sure I deal with this soon. > Same file id generated for potentially different files > ------------------------------------------------------ > > Key: BIT-1257 > URL: https://bro-tracker.atlassian.net/browse/BIT-1257 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master, 2.3 > Environment: CentOS 6 > Reporter: Jimmy Jones > Attachments: fa.bro, sample-samefileid.pcap > > > Attached sample contains two HTTP downloads of the same URL from the same client, but there are no guarantees that the files is actually the same (no Etags etc - in this case it actually is the same, but lets pretend they were different...). However the file analysis framework seems to give the same file ID in file_name and file_chunk for both downloads. > Think this is something to do with Range requests as doesn't happen if do "normal" HTTP requests. -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003)

6 years, 5 months

[Bro-Dev] [JIRA] (BIT-1242) Logs: add page listing logs to /script-reference/index.html

by Daniel Thayer (JIRA)

[ https://bro-tracker.atlassian.net/browse/BIT-1242?page=com.atlassian.jira.p… ] Daniel Thayer updated BIT-1242: ------------------------------- Fix Version/s: 2.4 > Logs: add page listing logs to /script-reference/index.html > ----------------------------------------------------------- > > Key: BIT-1242 > URL: https://bro-tracker.atlassian.net/browse/BIT-1242 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Documentation, Website > Reporter: Jeannette Dopheide > Assignee: Jeannette Dopheide > Fix For: 2.4 > > > Add a page to documentation listing the Bro log files. > There's already a page listing common log files: > http://www.bro.org/sphinx/logs/index.html#common-log-files > This page could be moved to http://www.bro.org/sphinx/script-reference/index.html and be expanded to list all known log files. > If this goes well, Robin suggested adding a btest that heuristically checks if something changes with regards to which logs we have. -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003)

6 years, 5 months

[Bro-Dev] [JIRA] (BIT-1242) Logs: add page listing logs to /script-reference/index.html

by Daniel Thayer (JIRA)

[ https://bro-tracker.atlassian.net/browse/BIT-1242?page=com.atlassian.jira.p… ] Daniel Thayer updated BIT-1242: ------------------------------- Description: Add a page to documentation listing the Bro log files. There's already a page listing common log files: http://www.bro.org/sphinx/logs/index.html#common-log-files This page could be moved to http://www.bro.org/sphinx/script-reference/index.html and be expanded to list all known log files. If this goes well, Robin suggested adding a btest that heuristically checks if something changes with regards to which logs we have. was: Add a page to documentation listing the Bro log files. There's already a page listing common log files: http://www.bro.org/sphinx/logs/index.html#common-log-files This page could be moved to http://www.bro.org/sphinx/script-reference/index.html and be expanded to list all known log files. If this goes well, Robin suggested adding a btest that heuristically checks if something changes with regards to which logs we have. I'll open a ticket for that when we're done. > Logs: add page listing logs to /script-reference/index.html > ----------------------------------------------------------- > > Key: BIT-1242 > URL: https://bro-tracker.atlassian.net/browse/BIT-1242 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Documentation, Website > Reporter: Jeannette Dopheide > Assignee: Jeannette Dopheide > > Add a page to documentation listing the Bro log files. > There's already a page listing common log files: > http://www.bro.org/sphinx/logs/index.html#common-log-files > This page could be moved to http://www.bro.org/sphinx/script-reference/index.html and be expanded to list all known log files. > If this goes well, Robin suggested adding a btest that heuristically checks if something changes with regards to which logs we have. -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003)

6 years, 5 months

[Bro-Dev] [JIRA] (BIT-1265) Single sided HTTP POST split

by Jon Siwek (JIRA)

[ https://bro-tracker.atlassian.net/browse/BIT-1265?page=com.atlassian.jira.p… ] Jon Siwek commented on BIT-1265: -------------------------------- [tcp_attempt_delay|https://www.bro.org/sphinx-git/scripts/base/init-bare.bro…] seems to be the relevant option. Related: [connection_attempt|https://www.bro.org/sphinx-git/scripts/base/bif/plugins/…] > Single sided HTTP POST split > ---------------------------- > > Key: BIT-1265 > URL: https://bro-tracker.atlassian.net/browse/BIT-1265 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Environment: CentOS 6 > Reporter: Jimmy Jones > Attachments: sample-upload2-all.pcap, sample-upload2-req.pcap > > > Attached two pcap samples, one is a single sided version of the other, an HTTP POST. > When I process the single sided version (sample-upload2-req) conn.log shows two sessions (the HTTP POST tcp connection that has been split) and http.log shows a partial upload. However processing the original sample (sample-upload2-all) everything is as expected - one connection in conn.log and a complete http.log > Are there any parameters I can tweak to make this work? -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003)

6 years, 5 months

[Bro-Dev] [JIRA] (BIT-1264) HTTP response not detected on nonstandard port

by Jon Siwek (JIRA)

[ https://bro-tracker.atlassian.net/browse/BIT-1264?page=com.atlassian.jira.p… ] Jon Siwek commented on BIT-1264: -------------------------------- The difference here is in [likely_server_ports|https://www.bro.org/sphinx/scripts/base/init-bare.bro.h…]. Because 80/tcp is in the likely_server_ports set, Bro correctly infers the packets belong to the responder, then your signature matches. Because 4321/tcp isn't in the set, Bro thinks the packets are from the originator, then the signature doesn't match because it requires checking against the responder's payload. And if you did force the signature to match by taking away the "is responder" condition, the HTTP analyzer would still ignore the content because it looks like data coming from the originator without having fully set up a TCP connection -- that's generally a situation where the current HTTP analyzer doesn't deal well. > HTTP response not detected on nonstandard port > ---------------------------------------------- > > Key: BIT-1264 > URL: https://bro-tracker.atlassian.net/browse/BIT-1264 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Environment: CentOS 6 > Reporter: Jimmy Jones > Attachments: relaxed.bro, relaxed-http.sig, sample-small2-rsp.pcap, sample-small-rsp.pcap > > > Using the attached bro script I've tweaked the HTTP signature to match on http responses without the corresponding HTTP request TCP session. I know in a proper setup you should never get single sided traffic, but certainly when using bro as a tool you have to deal with it sometimes. > Bro handles this fine when the HTTP is on port 80, but not when on port 4321 (see attached PCAPs). I'm curious as to why? -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003)

6 years, 5 months

[Bro-Dev] [JIRA] (BIT-1265) Single sided HTTP POST split

by Jimmy Jones (JIRA)

Jimmy Jones created BIT-1265: -------------------------------- Summary: Single sided HTTP POST split Key: BIT-1265 URL: https://bro-tracker.atlassian.net/browse/BIT-1265 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Affects Versions: git/master Environment: CentOS 6 Reporter: Jimmy Jones Attachments: sample-upload2-all.pcap, sample-upload2-req.pcap Attached two pcap samples, one is a single sided version of the other, an HTTP POST. When I process the single sided version (sample-upload2-req) conn.log shows two sessions (the HTTP POST tcp connection that has been split) and http.log shows a partial upload. However processing the original sample (sample-upload2-all) everything is as expected - one connection in conn.log and a complete http.log Are there any parameters I can tweak to make this work? -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003)

6 years, 5 months

[Bro-Dev] [JIRA] (BIT-1264) HTTP response not detected on nonstandard port

by Jimmy Jones (JIRA)

Jimmy Jones created BIT-1264: -------------------------------- Summary: HTTP response not detected on nonstandard port Key: BIT-1264 URL: https://bro-tracker.atlassian.net/browse/BIT-1264 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Affects Versions: git/master Environment: CentOS 6 Reporter: Jimmy Jones Attachments: relaxed.bro, relaxed-http.sig, sample-small2-rsp.pcap, sample-small-rsp.pcap Using the attached bro script I've tweaked the HTTP signature to match on http responses without the corresponding HTTP request TCP session. I know in a proper setup you should never get single sided traffic, but certainly when using bro as a tool you have to deal with it sometimes. Bro handles this fine when the HTTP is on port 80, but not when on port 4321 (see attached PCAPs). I'm curious as to why? -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003)

6 years, 5 months

[Bro-Dev] [JIRA] (BIT-1257) Same file id generated for potentially different files

by Jimmy Jones (JIRA)

[ https://bro-tracker.atlassian.net/browse/BIT-1257?page=com.atlassian.jira.p… ] Jimmy Jones commented on BIT-1257: ---------------------------------- Sorry I've not been as clear as I could here. I've changed my own bro instance, but I'm concerned that out of the box, Bro's behaviour while convenient for the majority of cases, isn't correct and will result in irrecoverably corrupted files in some instances (unless you’re lucky enough to keep full captures). I've researched this further and I would argue there is a right answer and the spec is clear, see RFC2616, 10.2.7: bq. A cache MUST NOT combine a 206 response with other previously cached content if the ETag or Last-Modified headers do not match exactly, see 13.5.4. I'd say Bro is a cache in this instance, and for example clients like IE follow this [behavior|http://blogs.msdn.com/b/ieinternals/archive/2011/06/03/send-an-eta…] and Adobe Reader uses the If-Range conditional to ensure the URL is the same document. I agree my change is over-conservative, would you accept something that include ETag and Last-Modified in the hash? Or is the (small) chance of corruption not a concern (which is fine, as long as someone has actively decided not to follow the RFC) > Same file id generated for potentially different files > ------------------------------------------------------ > > Key: BIT-1257 > URL: https://bro-tracker.atlassian.net/browse/BIT-1257 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master, 2.3 > Environment: CentOS 6 > Reporter: Jimmy Jones > Attachments: fa.bro, sample-samefileid.pcap > > > Attached sample contains two HTTP downloads of the same URL from the same client, but there are no guarantees that the files is actually the same (no Etags etc - in this case it actually is the same, but lets pretend they were different...). However the file analysis framework seems to give the same file ID in file_name and file_chunk for both downloads. > Think this is something to do with Range requests as doesn't happen if do "normal" HTTP requests. -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003)

6 years, 5 months

Jump to page:

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

zeek-dev September 2014