zeek-dev March 2016

zeek-dev@lists.zeek.org

198 discussions

[Bro-Dev] [JIRA] (BIT-1547) broctl sets the same state variables over and over

by Justin Azoff (JIRA)

[ https://bro-tracker.atlassian.net/browse/BIT-1547?page=com.atlassian.jira.p… ] Justin Azoff updated BIT-1547: ------------------------------ Resolution: Merged (was: Fixed) Status: Closed (was: Merge Request) > broctl sets the same state variables over and over > -------------------------------------------------- > > Key: BIT-1547 > URL: https://bro-tracker.atlassian.net/browse/BIT-1547 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Affects Versions: git/master > Reporter: Justin Azoff > Assignee: Justin Azoff > Fix For: 2.5 > > > I happened to notice broctl check on one of our test boxes was slow. traced it to sqlite commits() being very slow. Then noticed that broctl seems to call set_state() with the same key, val over and over again... once for each worker.. so a few thousand sets just to run broctl check. > Changing set_state to > {code} > # Set a dynamic state variable. > def set_state(self, key, val): > key = key.lower() > if self.state.get(key) == val: > return > self.state[key] = val > self.state_store.set(key, val) > {code} > Seemed to mostly fix it, aside from this: > {code} > Set manager-port to 47760 > Set manager-port to 47761 > Set manager-port to 47760 > Set manager-port to 47761 > Set manager-port to 47760 > Set manager-port to 47761 > Set manager-port to 47760 > Set manager-port to 47761 > Set manager-port to 47760 > Set manager-port to 47761 > Set manager-port to 47760 > Set manager-port to 47761 > {code} > any idea why that is flipping around like that? > We should possibly add a way for broctl to update state vars without calling commit where it knows it will be setting a large number of state vars in a loop. -- This message was sent by Atlassian JIRA (v7.2.0-OD-05-022#72002)

6 years, 1 month

[Bro-Dev] [JIRA] (BIT-1547) broctl sets the same state variables over and over

by Justin Azoff (JIRA)

[ https://bro-tracker.atlassian.net/browse/BIT-1547?page=com.atlassian.jira.p… ] Justin Azoff commented on BIT-1547: ----------------------------------- Merged.. Can you see about merging master back into topic/mfischer/broctl-broker? This latest change conflicts. I was able to just apply the config.py diff but the install.py change has issues. > broctl sets the same state variables over and over > -------------------------------------------------- > > Key: BIT-1547 > URL: https://bro-tracker.atlassian.net/browse/BIT-1547 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Affects Versions: git/master > Reporter: Justin Azoff > Assignee: Justin Azoff > Fix For: 2.5 > > > I happened to notice broctl check on one of our test boxes was slow. traced it to sqlite commits() being very slow. Then noticed that broctl seems to call set_state() with the same key, val over and over again... once for each worker.. so a few thousand sets just to run broctl check. > Changing set_state to > {code} > # Set a dynamic state variable. > def set_state(self, key, val): > key = key.lower() > if self.state.get(key) == val: > return > self.state[key] = val > self.state_store.set(key, val) > {code} > Seemed to mostly fix it, aside from this: > {code} > Set manager-port to 47760 > Set manager-port to 47761 > Set manager-port to 47760 > Set manager-port to 47761 > Set manager-port to 47760 > Set manager-port to 47761 > Set manager-port to 47760 > Set manager-port to 47761 > Set manager-port to 47760 > Set manager-port to 47761 > Set manager-port to 47760 > Set manager-port to 47761 > {code} > any idea why that is flipping around like that? > We should possibly add a way for broctl to update state vars without calling commit where it knows it will be setting a large number of state vars in a loop. -- This message was sent by Atlassian JIRA (v7.2.0-OD-05-022#72002)

6 years, 1 month

[Bro-Dev] Broker & CAF includes

by Matthias Vallentin

During Broker refactoring, I noticed the following: all headers in broker/* include either standard library headers or Broker headers. This appears to be by design, which makes sense to me. As a library writer, one faces the tricky question of exposing headers from dependencies. For example, Broker currently has it's own broker::util::optional, which ships as a (now outdated) copy of the corresponding CAF source. I am inclined to change this copy to an include that points directly into CAF headers, with the following rationale: Broker already depends on CAF, and a system that has CAF installed always ships with CAF headers. (Strictly speaking, we're not copying the code of <vector> into broker either, but relying on it via an include.) >From a user perspective, nothing changes here. A user will never include a CAF header, but may rely on it during compilation. Here's what I an example of what I want broker/util/optional.hh to look like: #include "caf/optional.hpp" // <--- New include. namespace broker { namespace util { using caf::optional; }} Currently we have: // Note the absence of a CAF include. namespace broker { namespace util { template <class T> class optional { // code copied from CAF }; }} Relying on the former form is more maintainable, and allows us to stay in sync with upstream fixes and improvements on the CAF side. I'm checking in here on the list to see whether anyone has objections. Matthias

6 years, 1 month

[Bro-Dev] [JIRA] (BIT-1558) Bro's ascii formatter writing out scientific notation

by Johanna Amann (JIRA)

[ https://bro-tracker.atlassian.net/browse/BIT-1558?page=com.atlassian.jira.p… ] Johanna Amann updated BIT-1558: ------------------------------- Fix Version/s: 2.5 > Bro's ascii formatter writing out scientific notation > ----------------------------------------------------- > > Key: BIT-1558 > URL: https://bro-tracker.atlassian.net/browse/BIT-1558 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Reporter: Seth Hall > Fix For: 2.5 > > > From the mailing list: > ``` > Hello, > in the x509.log normally the values regarding certificate.not_valid_before & certificate.not_valid_after look like: > 1444082400.000000 1475791199.000000 > I found some value like this: > -3600.000000 2.153226e+09 > Is it possible to modify something in order to have 2153226000 instead 2.153226e+09 ? > ``` > Bro's formatter's shouldn't use scientific notation because it complicates parsing of the data. -- This message was sent by Atlassian JIRA (v7.2.0-OD-05-022#72002)

6 years, 1 month

[Bro-Dev] [Auto] Merge Status

by Merge Tracker

Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- -------------- ------------ ---------- ------------- ---------- ------------------------------------------------------------- BIT-1563 [1] Bro Daniel Thayer - 2016-03-30 2.5 Normal BrokerComm and BrokerStore namespaces should be combined BIT-1561 [2] BroControl Aaron Eppert - 2016-03-25 - Normal Pull Request to fix resource leaking in BroControl Python API BIT-1557 [3] Broccoli Daniel Thayer - 2016-03-21 2.5 Low broccoli code examples don't compile BIT-1547 [4] BroControl Justin Azoff Justin Azoff 2016-03-08 2.5 Normal broctl sets the same state variables over and over BIT-1528 [5] Bro Justin Azoff - 2016-03-24 2.5 Normal SNMP and SIP scans show up in known services. BIT-1507 [6] Bro Jan Grashoefer Seth Hall 2016-01-25 - Low Intel framework does not match mail addresses properly BIT-1498 [7] BroControl scampbell - 2016-03-11 2.5 Trivial add '-q' to ssh execution in ssh_runner.py Open GitHub Pull Requests ========================= Issue Component User Updated Title -------- ----------- --------------- ---------- ------------------------------------------------------------------------- #64 [8] bro aeppert [9] 2016-03-30 Add disable_all_analyzers for connection shunting options [10] #63 [11] bro WilliamTom [12] 2016-03-26 Wrong regex literal in scripting doc [13] #61 [14] bro aeppert [15] 2016-03-22 Add uid to smb_ntlm_authenticate [16] #60 [17] bro aeppert [18] 2016-03-22 Add uid to AuthInfo [19] #52 [20] bro J-Gras [21] 2016-01-18 Fixed matching mail address intel [22] #22 [23] bro-plugins nickwallen [24] 2016-03-23 BIT-1559 Bro-Plugins Send each log stream to different kafka topic [25] #18 [26] bro-plugins jshlbrd [27] 2016-03-03 SSDP analyzer [28] #4 [29] broctl aeppert [30] 2016-03-25 Fix leaking resources from never closing out the Broccoli connection [31] [1] BIT-1563 https://bro-tracker.atlassian.net/browse/BIT-1563 [2] BIT-1561 https://bro-tracker.atlassian.net/browse/BIT-1561 [3] BIT-1557 https://bro-tracker.atlassian.net/browse/BIT-1557 [4] BIT-1547 https://bro-tracker.atlassian.net/browse/BIT-1547 [5] BIT-1528 https://bro-tracker.atlassian.net/browse/BIT-1528 [6] BIT-1507 https://bro-tracker.atlassian.net/browse/BIT-1507 [7] BIT-1498 https://bro-tracker.atlassian.net/browse/BIT-1498 [8] Pull Request #64 https://github.com/bro/bro/pull/64 [9] aeppert https://github.com/aeppert [10] Merge Pull Request #64 with git pull --no-ff --no-commit https://github.com/aeppert/bro.git master [11] Pull Request #63 https://github.com/bro/bro/pull/63 [12] WilliamTom https://github.com/WilliamTom [13] Merge Pull Request #63 with git pull --no-ff --no-commit https://github.com/WilliamTom/bro.git master [14] Pull Request #61 https://github.com/bro/bro/pull/61 [15] aeppert https://github.com/aeppert [16] Merge Pull Request #61 with git pull --no-ff --no-commit https://github.com/aeppert/bro.git patch-6 [17] Pull Request #60 https://github.com/bro/bro/pull/60 [18] aeppert https://github.com/aeppert [19] Merge Pull Request #60 with git pull --no-ff --no-commit https://github.com/aeppert/bro.git patch-5 [20] Pull Request #52 https://github.com/bro/bro/pull/52 [21] J-Gras https://github.com/J-Gras [22] Merge Pull Request #52 with git pull --no-ff --no-commit https://github.com/J-Gras/bro.git topic/jgras/bit-1507 [23] Pull Request #22 https://github.com/bro/bro-plugins/pull/22 [24] nickwallen https://github.com/nickwallen [25] Merge Pull Request #22 with git pull --no-ff --no-commit https://github.com/nickwallen/bro-plugins.git support-many-kafka-topics [26] Pull Request #18 https://github.com/bro/bro-plugins/pull/18 [27] jshlbrd https://github.com/jshlbrd [28] Merge Pull Request #18 with git pull --no-ff --no-commit https://github.com/jshlbrd/bro-plugins-1.git topic/jshlbrd/ssdp [29] Pull Request #4 https://github.com/bro/broctl/pull/4 [30] aeppert https://github.com/aeppert [31] Merge Pull Request #4 with git pull --no-ff --no-commit https://github.com/aeppert/broctl.git patch-1

6 years, 1 month

[Bro-Dev] [JIRA] (BIT-1563) BrokerComm and BrokerStore namespaces should be combined

by Daniel Thayer (JIRA)

[ https://bro-tracker.atlassian.net/browse/BIT-1563?page=com.atlassian.jira.p… ] Daniel Thayer updated BIT-1563: ------------------------------- Status: Merge Request (was: Open) > BrokerComm and BrokerStore namespaces should be combined > -------------------------------------------------------- > > Key: BIT-1563 > URL: https://bro-tracker.atlassian.net/browse/BIT-1563 > Project: Bro Issue Tracker > Issue Type: Task > Components: Bro > Reporter: Daniel Thayer > Fix For: 2.5 > > > The BrokerComm and BrokerStore namespaces should be combined to > just "Broker". -- This message was sent by Atlassian JIRA (v7.2.0-OD-04-029#72002)

6 years, 1 month

[Bro-Dev] [JIRA] (BIT-1563) BrokerComm and BrokerStore namespaces should be combined

by Daniel Thayer (JIRA)

[ https://bro-tracker.atlassian.net/browse/BIT-1563?page=com.atlassian.jira.p… ] Daniel Thayer commented on BIT-1563: ------------------------------------ Branch "topic/dnthayer/broker-namespace" in the bro git repo contains this change. I also split the broker main.bro into two scripts, because they will become much bigger when BIF script wrappers are added to them. > BrokerComm and BrokerStore namespaces should be combined > -------------------------------------------------------- > > Key: BIT-1563 > URL: https://bro-tracker.atlassian.net/browse/BIT-1563 > Project: Bro Issue Tracker > Issue Type: Task > Components: Bro > Reporter: Daniel Thayer > Fix For: 2.5 > > > The BrokerComm and BrokerStore namespaces should be combined to > just "Broker". -- This message was sent by Atlassian JIRA (v7.2.0-OD-04-029#72002)

6 years, 1 month

[Bro-Dev] [JIRA] (BIT-1563) BrokerComm and BrokerStore namespaces should be combined

by Daniel Thayer (JIRA)

[ https://bro-tracker.atlassian.net/browse/BIT-1563?page=com.atlassian.jira.p… ] Daniel Thayer updated BIT-1563: ------------------------------- Fix Version/s: 2.5 > BrokerComm and BrokerStore namespaces should be combined > -------------------------------------------------------- > > Key: BIT-1563 > URL: https://bro-tracker.atlassian.net/browse/BIT-1563 > Project: Bro Issue Tracker > Issue Type: Task > Components: Bro > Reporter: Daniel Thayer > Fix For: 2.5 > > > The BrokerComm and BrokerStore namespaces should be combined to > just "Broker". -- This message was sent by Atlassian JIRA (v7.2.0-OD-04-029#72002)

6 years, 1 month

[Bro-Dev] [JIRA] (BIT-1563) BrokerComm and BrokerStore namespaces should be combined

by Daniel Thayer (JIRA)

Daniel Thayer created BIT-1563: ---------------------------------- Summary: BrokerComm and BrokerStore namespaces should be combined Key: BIT-1563 URL: https://bro-tracker.atlassian.net/browse/BIT-1563 Project: Bro Issue Tracker Issue Type: Task Components: Bro Reporter: Daniel Thayer The BrokerComm and BrokerStore namespaces should be combined to just "Broker". -- This message was sent by Atlassian JIRA (v7.2.0-OD-04-029#72002)

6 years, 1 month

[Bro-Dev] [JIRA] (BIT-1472) Bif for a new function to calculates haversine distance between two geoip locations

by Aashish Sharma (JIRA)

[ https://bro-tracker.atlassian.net/browse/BIT-1472?page=com.atlassian.jira.p… ] Aashish Sharma commented on BIT-1472: ------------------------------------- Until you are set to update libGeoIP2 API, could you add this bif to bro.bif you can later eliminate this from bro.bif or reintegrate as you see fit. > Bif for a new function to calculates haversine distance between two geoip locations > ----------------------------------------------------------------------------------- > > Key: BIT-1472 > URL: https://bro-tracker.atlassian.net/browse/BIT-1472 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: 2.4 > Reporter: Aashish Sharma > Assignee: Justin Azoff > Priority: Low > Labels: bif, function > Fix For: 2.5 > > > Merge request for: > topic/aashish/haversine > ## ## Calculates haversine distance between two geoip locations > ## > ## > ## lat1, long1, lat2, long2 > ## > ## Returns: distance in miles > ## function haversine_distance%(lat1:double, long1:double, lat2:double, long2:double %): double > accompanying bro policy in base/utils/haversine_distance_ip.bro > module GLOBAL; > ## Returns the haversine distance between two IP addresses based on GeoIP > ## database locations > ## > ## > ## orig: the address of orig connection > ## resp: the address of resp server > ## Returns: the GeoIP distance between orig and resp in miles > function haversine_distance_ip(orig: addr, resp: addr): double -- This message was sent by Atlassian JIRA (v7.2.0-OD-04-029#72002)

6 years, 1 month

Jump to page:

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

zeek-dev March 2016