#464: Bug with complex data structures
---------------------+--------------------
Reporter: seth | Owner:
Type: Problem | Status: new
Priority: High | Milestone: Bro1.6
Component: Bro | Version:
Keywords: |
---------------------+--------------------
This script:
{{{
type MetaData: record {
a: count;
tags: set[string];
};
global ip_data: table[addr] of set[MetaData] = table();
local m: MetaData = [$a=4, $tags=set("blah")];
if ( 1.2.3.4 !in ip_data )
ip_data[1.2.3.4] = set(m);
else
add ip_data[1.2.3.4][m];
}}}
Results in this error:
{{{
1306867820.149493 <no location> and ./test29.bro, line 10
([a=4, tags={blah}] and list of record { a:count; tags:set[string]; }):
error, index type doesn't match table
}}}
--
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/464>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker
#461: internal_error called for async DNS failures
---------------------+--------------------
Reporter: seth | Owner:
Type: Problem | Status: new
Priority: Normal | Milestone: Bro1.6
Component: Bro | Version:
Keywords: |
---------------------+--------------------
Bro shouldn't be shutting down when DNS lookups fail which is what
internal_error leads to. It seems that the main area people are having
problems is the handling of the return value of nb_dns_activity, but I
think that all internal_error calls in DNS_Mgr.cc should be audited.
Maybe for the next release we can change the places where internal_error
is called based on the nb_dns_activity return value and audit the rest of
the internal_error calls later.
--
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/461>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker
#460: Problem with extended records and empty container types
---------------------+------------------------
Reporter: seth | Owner:
Type: Problem | Status: new
Priority: Normal | Milestone: Bro1.6
Component: Bro | Version: git/master
Keywords: |
---------------------+------------------------
If you extend a record, the empty container types don't work anymore. The
following code::
{{{
type Blah: record {
whatever: string;
};
redef record Blah += {
test2: set[count] &default=set();
};
global foobar: Blah;
}}}
Results in this crash::
{{{
[build (topic/policy-scripts-new)]$ gdb --args ./src/bro test23.bro
GNU gdb 6.3.50-20050815 (Apple version gdb-1469) (Wed May 5 04:36:56 UTC
2010)
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you
are
welcome to change it and/or distribute copies of it under certain
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for
details.
This GDB was configured as "x86_64-apple-darwin"...Reading symbols for
shared libraries ........ done
(gdb) r
Starting program: ./src/bro test23.bro
Reading symbols for shared libraries .+++++++ done
./test23.bro, line 6 (set() and set[count]): error, &default value has
inconsistent type
Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_INVALID_ADDRESS at address: 0x0000000000000028
0x0000000100085138 in BroType::Tag (this=0x0) at Type.h:75
75 TypeTag Tag() const { return tag; }
(gdb) bt
#0 0x0000000100085138 in BroType::Tag (this=0x0) at Type.h:75
#1 0x0000000100224e60 in same_type (t1=0x100cb2be0, t2=0x0, is_init=0) at
./src/Type.cc:1645
#2 0x00000001000a5fe3 in Attributes::CheckAttr (this=0x100cb2d70,
a=0x100cb2c90) at ./src/Attr.cc:268
#3 0x00000001000a6e2a in Attributes::AddAttr (this=0x100cb2d70,
attr=0x100cb2c90) at ./src/Attr.cc:134
#4 0x00000001000a73b0 in Attributes::Attributes (this=0x100cb2d70,
a=0x100cb2710, t=0x100cb27c0, arg_in_record=false) at ./src/Attr.cc:105
#5 0x0000000100220de3 in TypeDecl::TypeDecl (this=0x100cb2d50,
t=0x100cb27c0, i=0x100b6e0c0 "test2", arg_attrs=0x100cb2710,
in_record=false) at ./src/Type.cc:789
#6 0x0000000100080f22 in yyparse () at parse.y:954
#7 0x0000000100094a1d in main (argc=2, argv=0x7fff5fbfeba0) at
./src/main.cc:736
(gdb)
}}}
--
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/460>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker
#458: Debian Packaging via CPack
--------------------+--------------------
Reporter: jsiwek | Owner: jsiwek
Type: Task | Status: new
Priority: Normal | Milestone: Bro1.6
Component: Bro | Version:
Keywords: |
--------------------+--------------------
This should be (re)investigated because, as of CMake 2.8.3, automatic
dependency detection looks possible. This was a major reason why we
didn't originally create scripts to generate DEBs -- they wouldn't
automatically include the dependency meta-info.
See http://public.kitware.com/Bug/view.php?id=10292
Main points of this task:
* Add a `make-deb-packages` script analogous to the current mac/rpm ones
* LD_LIBRARY_PATH might need to point to `./opt/bro/lib` during `make
package` to workaround RPATH bug (see CMake ticket above)
* In `cmake/ConfigurePackaging.cmake`'s `SetPackageGenerators` macro for
"Linux" platform:
* Add DEB to CPACK_GENERATOR
* Set CPACK_DEBIAN_PACKAGE_SHLIBDEPS
* Test that the pre/post install package scripts work
--
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/458>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker
#462: topic/seth/ssl-binpac - core SSL/TLS analyzer work
---------------------------+--------------------
Reporter: seth | Owner:
Type: Merge Request | Status: new
Priority: Normal | Milestone: Bro1.6
Component: Bro | Version:
Keywords: |
---------------------------+--------------------
This is the core analyzer work minus the scripts which will be developed
in the policy-scripts-new branch.
* The core events have been reworked. Less state is kept in the analyzer
itself so that more can be done in policy scripts.
* All certificates (including chained) are passed through to script-land
as DER certificates and they are just strings in Bro.
* A new BiF for doing certificate validation.
* NotValidBefore and NotValidAfter dates extracted from certificates.
* Lots of simplification to the core analyzer.
--
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/462>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker
(This is mostly for Robin and Jon, but I'm CC-ing bro-dev as well to solicit others thoughts about the topic)
I'd appreciate if you guys took a look at the metrics framework and let me know what you think about it. There's a lot missing still, but it's a decent start. Running it on a tracefile with those metrics/*-example.bro scripts and taking a look at the output is probably one of the best ways to learn what it's doing. I'll document some features I'm still planning on adding to it here too.
Currently, in the metrics framework a metric is just a key or keys that is connected to a number which is collected over some interval before being written to disk and reset. One metric that you could collect is the number of established TCP connections. Going further, you can imagine only wanting to collect the metric for local addresses and collecting the metric separately for every /24 or every known internally allocated subnet. Changing the break time for the metric could be useful too because you may only care about some value per hour while others you want to collect the metric every 15 seconds. A subnet is one aggregation technique and results in one of the possible indexes. A user could configure that they want all metrics aggregated into /24 netmask length instead of calculating the metric per individual IP address. The other index is a freeform string field which can represent anything. It's possible to use either, both, or no indexes to aggregate the data.
Here are some thoughts about missing features:
- Missing support for cluster deployment. The way this will work (I think) is that the manager will handle the break interval (the time between metrics collection/logging and reset) and it will send a metrics collection event to the workers which will send their metrics data to the manager where the data from each worker will be added together and logged. This would essentially act as a lazy synchronization technique.
- Missing statistical support. I want to be able to define when notices should happen based on rate of change of a metric (per the break interval) increasing much faster than you think it should (SSH failed logins). There's probably a lot of other stuff in this area I haven't thought of.
- Need a way to refine metrics to add things like netmask aggregation or aggregation table without defining it in the Metrics::create function so that we can ship a metrics collection script that the user can subsequently reconfigure. For example, maybe they want to track successful outbound TCP connections by department. They could supply a table like:
const subnets_to_depts = table( [1.2.3.0/24] = "Accounting", [4.3.0.0/16] = "Sales", [4.3.2.0/24] = "Accounting" );
and using that as the address aggregation table would aggregate the metrics per department instead of just per /24 or some other arbitrary netmask length.
- I need to write a command line tool to convert the log into something that Graphviz can understand because I'd like to be able to generate time-series graphs from these metrics really easily.
Here's the list of metrics I'm working towards currently...
1. Youtube video views.
2. HTTPS requests per host header (using a new SSL analyzer that provides the information from the SSL establishment), this is an example of a non-IP address based metric too.
3. TCP connections originated and responded to.
4. HTTP SQL injection requests (raise a notice when too many attacks)
5. HTTP requests subindexed by returned status code from server. How many 404,200,500 status codes seen per client IP address?
6. SSH failed connections (too many looks like scanning obviously).
7. DNS requests (watch for spiking could possibly find DNS tunnels or DNS amplification attacks)
Sorry for the rambling email, but my thoughts on this script are still mixing around a bit. :)
Thanks,
.Seth
--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/
#442: Hardcode Python path
----------------------------+------------------------
Reporter: robin | Owner: jsiwek
Type: Merge Request | Status: new
Priority: Normal | Milestone: Bro1.6
Component: BroControl | Version: git/master
Resolution: | Keywords:
----------------------------+------------------------
Changes (by jsiwek):
* version: => git/master
* type: Problem => Merge Request
Comment:
I took this a step further than just doing broctl and tried to standardize
all use of "#!/usr/bin/env <shell>" in installed shell scripts that may be
executed by users. This is implemented in the `topic/jsiwek/abs-interp-
path` branch in the broctl, trace-summary, and bro-aux repositories.
A caveat is that the hashbangs are left alone when configuring in binary
packaging mode (the `--binary-package` option to configure) for creating
DEBs, RPMs, Mac packages. This is because we currently intend to build
binary packages on NMI platforms which won't (can't) always have Python
installed in a standard location for a given distro.
--
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/442#comment:3>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker
#442: Hardcode Python path
-------------------------+--------------------
Reporter: robin | Owner: jsiwek
Type: Problem | Status: new
Priority: Normal | Milestone: Bro1.6
Component: BroControl | Version:
Resolution: | Keywords:
-------------------------+--------------------
Comment (by jsiwek):
Here's Craig's response that convinced me of taking the approach he
proposed:
{{{
>> BroControl currently uses `#! /usr/bin/env` to find the Python
>> binary at runtime. Craig suggested this instead:
>
> Craig, can you explain more about what the problem was in this ticket?
> I didn't understand how configuring the python shebang to a full path at
build time was better than using the env shebang.
If you use env, you'll pick up whatever version of python is first on
the path instead of the version that was first on the path when broctl
was built. This will be different for different users and can result in
unexpected results and possible brokenness.
The specific python binary path should be considered part of the broctl
config so that the package builder is able to control it by setting the
path when building broctl. Also, the end user only as to have broctl on
his path to be able to run it.
For example on FreeBSD, /usr/local/bin is not on the default path; the
default path is used at bootup so if broctl doesn't have the path built
in, it can't find python when it's run from a rc.d script. You could add
/usr/local/bin to the path but that won't be right 100% of the time for
all installations.
> If an absolute path to a python interpreter is set in the shebang
> at configure/build time, then to change the python interpreter that
> is used, the user has to either (1) edit the script(s) or (2)
> re-configure/build/install broctl. These seem like the more
> "difficult" options to me.
(How often does this actually happen?)
Editing the scripts seems wrong; if you later rebuild (say to install a
newer version) your changes will get over written.
The binary executable is part of the broctl configuration so having to
re-configure/build/install broctl to use a different python sounds
completely reasonable to me.
From my perspective, the inconvenience of rebuilding broctl seems minor
compared to having different users picking up different versions of
python.
Finally, some folks consider it a security issue to use #!/usr/bin/env
}}}
--
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/442#comment:2>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker
#463: Create a test for large packet support
-------------------------------+-----------------
Reporter: seth | Owner:
Type: Test Case Missing | Status: new
Priority: Low | Milestone:
Component: Bro | Version:
Keywords: |
-------------------------------+-----------------
This requires that we find a tracefile with large packets to verify that
we can successfully process them.
--
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/463>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker
