(Moving from bro-commits to bro-dev).
On Fri, Jan 31, 2014 at 12:51 -0500, you wrote:
> Instruction counts are probably going to have a strong dependency on
> the compiler version / options used to generate the code. I believe
> these counts could additionally be influenced by e.g. library
> upgrades, even when restricted to a single host and using a specific
> compiler / options.
True, but I'm not sure that's necessarily a bad thing. If the count
changes signficantly, it's worth understanding where it's coming from
I would say. btest won't complain as long as deviations are within a
reasonable range (1% by default, don't know if that's the right
value).
I'm also not sure if instruction count is the right feature; there are
plenty others one could measure, like cycles etc. I was just thinking
this might be the most stable.
> One alternative approach to tracking IDs for timing baselines might be
> to use system tools to gather a list of all libraries bro is linked
> against.
A problem with this is that btest doesn't know about Bro. :-) The way
I'm doing it currently is that instruction count is measured for all
BTEST-EXEC commands that are part of a test, which are then summed up
for a single number. I'd like to keep it the way that btest can
measure arbitrary command lines (which is part of the challenge of
finding a stable way of doing so ...).
> Additionally, formatting the temporary file in a human-readable way
> and keeping it as part of / in addition to the baseline can yield
> potentially useful information when looking into timing differences.
It's, more or less, human-readable:
> cat Baseline/_Timing/2a6b457d90e93b6688f312f87f677c5c
tests.m57-long 705347795206
tests.ipv6 104508274160
tests.m57-short 68458131160
What I'm mostly wondering about is if it's worth commiting data that's
very specific to a single user/machine to the repos?
Robin
--
Robin Sommer * Phone +1 (510) 722-6541 * robin(a)icir.org
ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin
[ https://bro-tracker.atlassian.net/browse/BIT-1119?page=com.atlassian.jira.p… ]
Jon Siwek commented on BIT-1119:
--------------------------------
Added a new commit on the branch to add a script which auto-detects/warns about running on filtered trace.
> topic/jsiwek/tcp-improvements
> -----------------------------
>
> Key: BIT-1119
> URL: https://bro-tracker.atlassian.net/browse/BIT-1119
> Project: Bro Issue Tracker
> Issue Type: Improvement
> Components: Bro
> Affects Versions: git/master
> Reporter: Jon Siwek
> Fix For: 2.3
>
> Attachments: signature.asc
>
>
> This branch is in the bro, bro-testing, and bro-testing-private repos and has a few changes to improve reporting of TCP connection sizes and gaps (commit messages explain in more detail).
> The baseline changes in the external repos all seemed reasonable/explainable (or actually fix a problem). There's too much changed to go through case-by-case and actually check things, but I did do closer examinations of unique differences as I came across them (e.g. try to corroborate Bro results via wireshark). Then for those that seem to follow the same trend as something I already inspected, I wouldn't manually check.
--
This message was sent by Atlassian JIRA
(v6.2-OD-07-028#6211)
[ https://bro-tracker.atlassian.net/browse/BIT-1119?page=com.atlassian.jira.p… ]
Jon Siwek updated BIT-1119:
---------------------------
Status: Merge Request (was: Open)
> topic/jsiwek/tcp-improvements
> -----------------------------
>
> Key: BIT-1119
> URL: https://bro-tracker.atlassian.net/browse/BIT-1119
> Project: Bro Issue Tracker
> Issue Type: Improvement
> Components: Bro
> Affects Versions: git/master
> Reporter: Jon Siwek
> Fix For: 2.3
>
> Attachments: signature.asc
>
>
> This branch is in the bro, bro-testing, and bro-testing-private repos and has a few changes to improve reporting of TCP connection sizes and gaps (commit messages explain in more detail).
> The baseline changes in the external repos all seemed reasonable/explainable (or actually fix a problem). There's too much changed to go through case-by-case and actually check things, but I did do closer examinations of unique differences as I came across them (e.g. try to corroborate Bro results via wireshark). Then for those that seem to follow the same trend as something I already inspected, I wouldn't manually check.
--
This message was sent by Atlassian JIRA
(v6.2-OD-07-028#6211)
Aashish Sharma created BIT-1126:
-----------------------------------
Summary: Logs disappearing after bro termination
Key: BIT-1126
URL: https://bro-tracker.atlassian.net/browse/BIT-1126
Project: Bro Issue Tracker
Issue Type: Problem
Components: BroControl
Affects Versions: 2.2
Environment: freebsd
Reporter: Aashish Sharma
Priority: High
I have noticed several times that in the event of bro termination after expiration of StopTimeout, bro logs disappear.
This is generally seen when log sizes are much bigger (for example after overnight)
This issue was present in bro-2.1 and continue to be present in bro-2.2
I see (kill from control.py - kick in often when stopping or restarting bro) because catch-n-release is still trying to flush its tables (which takes long time). Then there is no logs from overnight!
I can provide more information if desired (or even a test case).
Thanks,
Aashish
--
This message was sent by Atlassian JIRA
(v6.2-OD-07-028#6211)
[ https://bro-tracker.atlassian.net/browse/BIT-1124?page=com.atlassian.jira.p… ]
Robin Sommer updated BIT-1124:
------------------------------
Resolution: Merged (was: Fixed)
Status: Closed (was: Merge Request)
> process command misplaces custom scripts
> ----------------------------------------
>
> Key: BIT-1124
> URL: https://bro-tracker.atlassian.net/browse/BIT-1124
> Project: Bro Issue Tracker
> Issue Type: Problem
> Components: BroControl
> Affects Versions: 2.2
> Reporter: Robin Sommer
> Fix For: 2.3
>
>
> {noformat}
> # cat test.bro
> @load base/utils/site
> print Site::local_nets;
> {noformat}
> {{broctl process trace.pcap test.bro}} gives:
> {noformat}
> error in /usr/local/bro-2.2/share/bro/policy/misc/loaded-scripts.bro, line 4: syntax error, at or near “module"
> {noformat}
> I believe it's due to test.bro being placed in the middle of the command line that {{process}} builds. If I move it to the end, it works fine.
--
This message was sent by Atlassian JIRA
(v6.2-OD-07-028#6211)
[ https://bro-tracker.atlassian.net/browse/BIT-1125?page=com.atlassian.jira.p… ]
Robin Sommer updated BIT-1125:
------------------------------
Resolution: Merged (was: Fixed)
Status: Closed (was: Merge Request)
> topic/jsiwek/http-file-id-caching
> ---------------------------------
>
> Key: BIT-1125
> URL: https://bro-tracker.atlassian.net/browse/BIT-1125
> Project: Bro Issue Tracker
> Issue Type: Improvement
> Components: Bro
> Affects Versions: git/master
> Reporter: Jon Siwek
> Fix For: 2.3
>
> Attachments: signature.asc, signature.asc
>
>
> This branch is in bro and bro-testing repos. It adds a file ID caching / "fast path" mechanism to the file analysis API and adapts HTTP to use it for performance improvement.
--
This message was sent by Atlassian JIRA
(v6.2-OD-07-028#6211)
[ https://bro-tracker.atlassian.net/browse/BIT-1125?page=com.atlassian.jira.p… ]
Robin Sommer commented on BIT-1125:
-----------------------------------
{{I think the cases where there is really only one way to do it are pretty limited.}
Also recursive content inspection of container formats.
> topic/jsiwek/http-file-id-caching
> ---------------------------------
>
> Key: BIT-1125
> URL: https://bro-tracker.atlassian.net/browse/BIT-1125
> Project: Bro Issue Tracker
> Issue Type: Improvement
> Components: Bro
> Affects Versions: git/master
> Reporter: Jon Siwek
> Fix For: 2.3
>
> Attachments: signature.asc, signature.asc
>
>
> This branch is in bro and bro-testing repos. It adds a file ID caching / "fast path" mechanism to the file analysis API and adapts HTTP to use it for performance improvement.
--
This message was sent by Atlassian JIRA
(v6.2-OD-07-028#6211)
[ https://bro-tracker.atlassian.net/browse/BIT-1125?page=com.atlassian.jira.p… ]
Seth Hall updated BIT-1125:
---------------------------
Attachment: signature.asc
True. I think the cases where there is really only one way to do it are pretty limited. Maybe just the old "File" analyzer that is used for FTP and IRC transfers?
Ah, ok. Thanks.
> topic/jsiwek/http-file-id-caching
> ---------------------------------
>
> Key: BIT-1125
> URL: https://bro-tracker.atlassian.net/browse/BIT-1125
> Project: Bro Issue Tracker
> Issue Type: Improvement
> Components: Bro
> Affects Versions: git/master
> Reporter: Jon Siwek
> Fix For: 2.3
>
> Attachments: signature.asc, signature.asc
>
>
> This branch is in bro and bro-testing repos. It adds a file ID caching / "fast path" mechanism to the file analysis API and adapts HTTP to use it for performance improvement.
--
This message was sent by Atlassian JIRA
(v6.2-OD-07-028#6211)
[ https://bro-tracker.atlassian.net/browse/BIT-1125?page=com.atlassian.jira.p… ]
Robin Sommer commented on BIT-1125:
-----------------------------------
Agree with Jon, I think we want the option, it just feels unnecessary to pass through script-land in cases where there's really no question on how to compute the handle. I don't think that's actually different from other low-level decisions analyzers sometimes make on how to process something without asking script-land for its opinion.
Also, analyzers can document whether they offer any customization.
I think I'll rename {{cached_id}} to {{precomputed_id}} then make it cover both cases.
(and I would like to have a document eventually that summarizes the options an analyzer have :-)
> topic/jsiwek/http-file-id-caching
> ---------------------------------
>
> Key: BIT-1125
> URL: https://bro-tracker.atlassian.net/browse/BIT-1125
> Project: Bro Issue Tracker
> Issue Type: Improvement
> Components: Bro
> Affects Versions: git/master
> Reporter: Jon Siwek
> Fix For: 2.3
>
> Attachments: signature.asc, signature.asc
>
>
> This branch is in bro and bro-testing repos. It adds a file ID caching / "fast path" mechanism to the file analysis API and adapts HTTP to use it for performance improvement.
--
This message was sent by Atlassian JIRA
(v6.2-OD-07-028#6211)