zeek-dev September 2016

zeek-dev@lists.zeek.org

9 discussions

[Bro-Dev] input-framework file locations

by Aashish Sharma

I have been thinking and trying different things but for now, it appears that if we are to share policies around, there is no easy way to be able to distribute input-files along with policy files. Basically, right now I use redef Scan::whitelist_ip_file = "/usr/local/bro/feeds/ip-whitelist.scan" ; and then expect everyone to edit path as their setup demands it and place accompanying sample file in the directory or create one for themselves - this all introduces errors as well as slows down deployment. Is there a way I can use relative paths instead of absolute paths for input-framework digestion. At present a new-heuristics dir can have __load__.bro with all policies but input-framework won't read files relative to that directory or where it is placed. redef Scan::whitelist_ip_file = "../feeds/ip-whitelist.scan" ; Something similar to __load__.bro model Also, one question I have is should all input-files go to a 'standard' feeds/input dir in bro or be scattered around along with their accompanied bro policies (ie in individual directories ) Something to think about as with more and more reliance on input-framework i think there is a need for 'standardization' on where to put input-files and how to easily find and read them. Aashish

4 years

[Bro-Dev] Updating NEWS for 2.5

by Robin Sommer

Could folks take a look at NEWS and see what's missing? Couple of things I think we should add at least: - Document the cluster framework's new logger node, with an example how to use it. - Document the recent intel framework updates. - Add BroControl news/changes. Any takers for these? Robin -- Robin Sommer * ICSI/LBNL * robin(a)icir.org * www.icir.org/robin

4 years, 9 months

[Bro-Dev] ICAP Analyzer: BinPAC vs Plugin :: RegEx Issues

by Fernandez, Mark I

In support of submitting the ICAP Analyzer as a Bro Package, I am porting the ICAP Analyzer to build as a dynamic Plugin. Originally, I inserted the ICAP Analyzer straight into the source code tree, under /src/analyzer/protocol/icap, and compiled it as part of the Bro core. But in an effort to make it easier for others to integrate into their existing Bro instantiations, I am making the effort to make it a stand-alone Plugin instead... but the BinPAC parser is not working when I run it as a Plugin. The Plugin builds and installs without error, and I verify that the Plugin is enabled and that my ICAP main.bro script is loaded, but it is not producing any ICAP or HTTP related output: (a) It appears that the parser is not recognizing the ICAP Request messages whatsoever. (b) It starts to parse the ICAP Response messages; but it breaks mid-way thru the packet. I think the problem is within the BinPAC files where I use regular expressions to define a data element within the ICAP packet structures/records. In the ICAP Request message, the very first element is a regex pattern, so that's why it fails to parse these packets at all. In the ICAP Response message, it parses the first element correctly, but then it bombs on the second element, which is a regex pattern. In the BinPAC help/reference document, it contains a section titled, "Running Binpac-Generated Analyzer Standalone" [https://www.bro.org/sphinx/components/binpac/README.html#running-binpac-gen…], which states that to run binpac-generated code independent of Bro, the regex library must be substituted... I presume the stand-alone guidance applies to the Plugin? It must because I did not have this trouble when I built the analyzer straight into the Bro core. The regex library guidance says I need to include three header files: RE.h, bro-dummy.h, and binpac_pcre.h. You provide sample code for each file. Am I to copy-n-paste the sample code directly into my Plugin source code as three new headers files? Or do these three files exist elsewhere in the Bro source? I can find "RE.h" in the source (/src/RE.h). And I can find "binpac_regex.h" in the source (/aux/binpac/lib/binpac_regex.h), which seems similar, but I cannot find "binpac_pcre.h" nor "bro_dummy.h" anywhere. I need a little bit of advice... or a lot of advice :) Can I use RE.h and binpac_regex.h that exist in the Bro 2.4.1 distro? Or do I need to create the three header files and paste the sample code verbatim? Thanks! Mark Mark I. Fernandez MITRE Corporation Email: mfernandez(a)mitre.org<mailto:mfernandez@mitre.org> MITRE is a not-for-profit corporation that operates several Federally Funded Research and Development Centers (FFRDCs) in the interests of the US Government.

4 years, 11 months

[Bro-Dev] Proposal: Operating the Bro package manager offline

by Robin Sommer

At BroCon several people pointed out a need to install Bro packages on machines that do not have a direct external connection. One idea would be some kind of proxy scheme where an intermediary git repository mirrors packages locally; bro-pkg would then pull from there. However, I don't think any of us really liked that idea much. After a few rounds of discussions, Seth and I came up with a different idea that seems easier to manage: extending bro-pkg to bundle packages into deployment files that can be easily pushed to Bro systems simply by copying them over. I’ve tried to flesh this out a bit more, and would be interested to hear what you all think about this approach. And @Jon: Do you think this would be doable that way? Here’s the idea: 1. Generally, one first uses bro-pkg as usual to install packages onto a local Bro system that does have external Internet connectivity (this could be just a dummy Bro installation). One installs new packages there, updates existing ones, etc., until reaching a state that one wants to push out to the actual Bro system. 2. We add a new “bundle” command to bro-pkg that serializes the current state of packages into a single file on disk, a “package bundle”. The bundle contains the complete content of all currently installed packages, using some kind of suitable container format (could be just a ZIP file, or whatever works; the internal representation doesn’t really matter). 3. Users create such a bundle on the local system and then simply copy that bundle file over to all target Bro machines that do not have a external connectivity themselves, using whatever mechanism they have available (e.g., just scp; or maybe through some configuration management system like Ansible etc.). 4. On the target machine, one runs a corresponding “bro-pkg unbundle” command on that bundle file. That command will completely replace the system’s current set of packages with the bundle’s content. As a result, that machine will now have exactly the same packages installed as the original system. This would be the general scheme. A couple of people I talked to at BroCon confirmed that this would offer a viable solution for them, and that they would indeed much prefer copying files around over maintaining local git mirrors. Some additional thoughts on variations/extensions of this basic scheme: - I’m not quite sure if the bundle should contain just the packages themselves or further bro-pkg state as well, such as which packages are currently loaded. Right now I’m learning towards saying “just the packages”; that would basically treat bundles just as a transport mechanism to get packages over to another box. The actual Bro machines would still keep control over which packages to actually load, etc. - As it is described above, Step 1 would require having a local Bro installation into which packages get installed before they can be bundled up. It would be nice to have a mode where bro-pkg can operate without having a Bro around at all, just downloading packages locally somewhere for bundling them up. I could also see offering an even simpler mode where one simply lists packages to bundle on the command-line: “bro-pkg bundle <pkg1> <pkg2> <pkg3>”. That would be particularly useful with configuration management systems I think. - It would be neat if bro-pkg's Python library exposed operations to inspect & retrieve the content of a bundle, such as iterating over the packages inside a bundle and iterating over the files inside a package. That way one could easily build target-side scripts that process and validate bundles before going ahead and installing them (e.g., imposing custom restrictions on what kind of packages one allows to put in place; or ensuring that a bundle always contains a set of packages the site deems mandatory, to avoid configuration mistakes; or even just logging what gets pushed out). What do you guys think about this? Robin -- Robin Sommer * ICSI/LBNL * robin(a)icir.org * www.icir.org/robin

4 years, 11 months

[Bro-Dev] Blog post for bro-pkg

by Slagell, Adam J

Jon, Would you be up for writing a blog post introducing the Bro Package Manager and showing some examples of its use? We probably also want to state that these are unvetted contributions from the community. :Adam ------ Adam J. Slagell Chief Information Security Officer Director, Cybersecurity Division National Center for Supercomputing Applications University of Illinois at Urbana-Champaign www.slagell.info "Under the Illinois Freedom of Information Act (FOIA), any written communication to or from University employees regarding University business is a public record and may be subject to public disclosure."

4 years, 12 months

[Bro-Dev] bro-pkg -> bropkg

by Seth Hall

What does everyone think about making a change to rename bro-pkg to bropkg? I think we're early enough that we could probably get away with it and it fits with more with existing tools like broctl. I find it difficult to type the hyphen too for some reason. Thoughts? .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/

4 years, 12 months

[Bro-Dev] ICAP Analyzer Design Guidance

by Fernandez, Mark I

I am reviewing my source code and scripts for the ICAP Analyzer that I presented last week at BroCon, with the intent of releasing the new analyzer to the Bro community. There is one key aspect which I designed a certain way, but I wonder if it would be acceptable by the community or if it would introduce problems. I appreciate your feedback. In the 'main.bro' script for the ICAP Analyzer, I redefine the 'conn_id' record to include a new element, as follows: redef record conn_id += { orig_u : string &log &optional; } where 'orig_u' is derived from the ICAP header 'X-Authenticated-User' and is associated with the userid on the local domain that originated the HTTP request. At the time I wrote the code, it made perfect sense to extend the 'conn_id' record to include the 'orig_u' element, and it works very well in my operational environment. However, now that I am preparing to release the code to a wider audience, it occurs to me that perhaps it may not be acceptable to the community of users to extend the 'conn_id' record. To be clear, the 'orig_u' element would be present within every log file that records the 'conn_id' record, such as http.log, ftp.log, dns.log, etc. However, the values are meaningful only for http.log. In the other log files, the 'orig_u' column would contain a dash '-' value indicating the value is unset. Design guidance: is it acceptable to redefine/extend the 'conn_id' record as described above? I appreciate your feedback. Thanks! Mark I. Fernandez

4 years, 12 months

[Bro-Dev] Code complexity survey

by Vard Antinyan

Dear developers, We have undertaken a task to assess code complexity triggers and generate recommendations for developing simple and understandable code. Our intension is to share the results with you, developers, so everyone can learn the triggers behind complex software. We need your help for rigorous results. My request to you is - if you get 5-10 min. time, would you please consider to answer the questions of this survey on code complexity? https://goo.gl/forms/h9WXZ8VSEw7BUyHg1 You are welcome to learn preliminary results through this link: https://www.facebook.com/SoftwareCodeQuality/photos/?tab=album&album_id=163… The results will be shared in a public webpage and everyone possible will be invited to learn and discuss them. Your knowledge and experience is vital for achieving substantial and generalizable results, and your effort is much appreciated! Sincerely Vard Antinyan PhD candidate in University of Gothenburg, Sweden Tel: 0046317725707

5 years

[Bro-Dev] Clone Failing: Sanity Check

by anthony kasza

Is it just me or are recursive clones of the GitHub repo failing when trying to clone aux/binpac? -AK

5 years

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

zeek-dev September 2016