Hi Robin,
I've been looking at the serialization code in Bro for a while now and
I've hit a dead end. It'd be really cool if you could help me out
because I think I simply don't get it :)
I think I've basically understood the concepts of SerialObj, Serializer
and SerializationFormat -- I think I would have structured things the
same way. I'm getting lost in the details though:
- When exactly and why does a class have to implement Unserialize() and
Serialize()? What's their relationship to DoSerialize() and
DoUnserialize()? The comments in SerialObj.h are a bit vague in that
regard.
- It seems the idea of sending ahead an identifier to the receiving end
that tells it what to do with the following data exists three times:
* in the SER_xxx constants and the factory approach in
IMPLEMENT_SERIAL
* in the character constants 'i', 'e', 's' etc in Serializer.cc
* in the MSG_xxx constants in RemoteSerializer.cc.
I think the latter are partially internal to the remote<->local
communication and can hence mostly be ignored for understanding the
serialization code, right? If you could quickly explain the difference
between the first two that'd be great.
The next thing I noticed are the hardcoded (un)seralization methods in
Serializer:
bool Serialize(const ID& id, const SerialInfo& info);
bool Serialize(const char* func, val_list* args);
bool Serialize(const StateAccess& s);
bool Serialize(const Connection& c);
bool Serialize(const Timer& t);
virtual void GotID(ID* id, Val* val) = 0;
virtual void GotEvent(const char* name, double time,
EventHandlerPtr event, val_list* a) = 0;
virtual void GotFunctionCall(const char* name, double time,
Func* func, val_list* args) = 0;
virtual void GotStateAccess(StateAccess* s) = 0;
virtual void GotTimer(Timer* t) = 0;
virtual void GotConnection(Connection* c) = 0;
bool UnserializeID();
bool UnserializeCall();
bool UnserializeStateAccess();
bool UnserializeTimer();
bool UnserializeConnection();
Are these special in a way to have them implemented this way? Couldn't
there be a "received" callback per SER_xxx constant that resides as
a static method in the serializable classes themselves? So we can avoid
hardcoding anything?
- Following the comments in SerialObj.h, I see what I need to do to make
a class's objects serializable. I presume that the correct way to ship
an object to a serializer is by calling SerialObj::Serialize() with the
appropriate serializer. What are my options for picking them up at the
receiving end?
Oh and RemoteSerializer::ProcessSerialization() calls Unserialize()
passing a SerialInfo, but Serializer::Unserialize() expects a bool -- is
that intended?
The reason why I'm looking at this is that I'm trying to find the right
knobs to tweak to allow arbitrary *local* client applications to feed
information into Bro (like a tuned sshd that can feed its events and
traffic to a local Bro) without reinventing the wheel ...
Thanks so much!
Best,
Christian.
--
________________________________________________________________________
http://www.cl.cam.ac.uk/~cpk25http://www.whoop.org
On Fri, Apr 23, 2004 at 02:44:49PM +0800, ?????? wrote:
> We are intresting in Bro.But when we test it under a simulative network,
> it was killed a few minutes by the linux systerm,giving the message of"
> Out of memory",while Bro 's message is "internal error:double signal".
The amount of memory bro uses heavily depends on the policy scripts
that you are running. If you additionally load the script
statistics.bro you'll get a statistics.log file which should provide
you information where the all the memory has gone. One common trick is
to tune the various timeouts like the script reduce-memory.bro does.
(Note: reduce memory sets timeouts that are perhaps not suitable to
your needs)
Holger
--
Holger Dreger * http://www.net.in.tum.de * Phone: +49 (0)89 289-18006
Computer Science Department * Technische Universitaet Muenchen
Sorry to immediately bug folks with another release, but this one, available
from the usual location:
ftp://ftp.ee.lbl.gov/bro-pub-0.8-current.tar.gz
fixes a rarely-tickled-but-fatal bug in timer processing. CHANGES
and the one-character fix appended.
Vern
-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
0.8a82 Tue Apr 27 11:53:24 PDT 2004
- Fixed inactivity timer loop when a packet arrives exactly when
the timer is set to expire.
-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
--- Conn.cc 2004/03/15 20:14:19 1.49
+++ Conn.cc 2004/04/27 19:00:13
@@ -1,4 +1,4 @@
-// $Id: Conn.cc,v 1.49 2004/03/15 20:14:19 vern Exp $
+// $Id: Conn.cc,v 1.50 2004/04/27 19:00:03 vern Exp $
//
// Copyright (c) 1996, 1997, 1998, 1999, 2000, 2001, 2002
// The Regents of the University of California. All rights reserved.
@@ -294,7 +294,7 @@
// timeout once, but it's disabled now. We do nothing then.
if ( inactivity_timeout )
{
- if ( last_time + inactivity_timeout < t )
+ if ( last_time + inactivity_timeout <= t )
{
Event(connection_timeout);
sessions->Remove(this, 1);
> I'd like to extract 41 features and their corresponding attack classes based on
> the DARPA 1999 dataset and 2000 dataset
Bro doesn't have these features directly coded into its analyzers.
You could probably add them with not that much work, but I'd advise you
to first consider whether you really want to do so: those datasets, while
invaluable for the evaluations for which they were originally developed,
are notorious for how they are misapplied for subsequent intrusion detection
research. The main problem is that they have artifacts due to their synthetic
nature. In particular, the feature set from the KDD Cup is known to be
seriously flawed. See McHugh's critique of the original datasets and
Mahoney/Chen's RAID 2003 paper on the problems with the KDD Cup feature set.
Vern
> $ /usr/local/bin/bro08a81 -r
> /home/f/200401121453-iptcpudpfrag-telefonicaESP.tcpdump mt
> /L/policy/scan.bro, line 91: internal error: NB-DNS error in
> DNS_Mgr::WaitForReplies (recvfrom(): Connection refused)
You can do this by editing Makefile and removing "nb_dns.o" from
the "OBJ =" target (it's at the very end), and editing config.h to
change
#define HAVE_NB_DNS
to
#undef HAVE_NB_DNS
Finally, set the environment variable $BRO_DNS_FAKE.
If someone would contribute the necessary autoconf magic to do this
automatically, that would be great - it's beyond my meager autoconf
skills.
Vern
Hi,
Possible disable dns request
simply on last bro (a81) ?
because I don't have resolver on my fbsd host.
$ /usr/local/bin/bro08a81 -r
/home/f/200401121453-iptcpudpfrag-telefonicaESP.tcpdump mt
/L/policy/scan.bro, line 91: internal error: NB-DNS error in
DNS_Mgr::WaitForReplies (recvfrom(): Connection refused)
Thanks
Rmkml(a)Wanadoo.fr
An updated "CURRENT" version of Bro is now available from the usual location:
ftp://ftp.ee.lbl.gov/bro-pub-0.8-current.tar.gz
This version has several minor-but-handy changes to the last "CURRENT"
version (0.8a79).
Vern
-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
0.8a81 Mon Apr 26 22:46:37 PDT 2004
- A bunch of memory leaks fixed (Chema Gonzalez).
- A new HTTP analyzer variable, content_truncation_limit, controls
how much of an HTTP request/reply contents element (i.e., what's
passed to http_entity_data - this is *not* the entire content, but
the next chunk's worth) is logged in the log file. It defaults
to 40 bytes. Setting it to 0 means "log all of it".
- Fix to avoid crashing for malformed RPC requests.
- Improved OpenSSL auto-configuration (Robin Sommer).
- Fix for compiling without OpenSSL.
- A new built-in, double_to_count(), converts a value of type "double"
to the corresponding "count" (Chema Gonzalez). We should probably
add floor(), ceil(), etc.
- Parameterization of trw.bro tweaked (Jaeyeon Jung).
> One possibility is to load the contents.bro script. It will write the
> contents of every connection to two files (contents-*), one for each
> direction. Note that it does writing for every connection, not just HTTP
> ones. If you want the latter, you might want to adapt the script
> accordingly.
Note, depending on your broader use, you can avoid adapting the script
by using capture_filters to only capture tcp port 80.
Vern
> I'd like to log http payloads for each connection seen on my network.
>
> In fact, I'd like to get something like :
> Src_IP;Dst_IP;Request_Payload;Reply_Payload
>
> but with entire payloads (not only URIs, but also banners...)
You can get all of this via
bro -r http.trace http-reply http-header http-body
The only problem is that http-body has hardwired into it that contents
(not headers) larger than 40 bytes are truncated to 40 bytes. I will
fix that. In the interim, if you need entire contents you can make a
copy of the script with the limitation removed and use that instead.
Vern
Hi everybody !!!
I'd like to log http payloads for each connection seen on my network.
In fact, I'd like to get something like :
Src_IP;Dst_IP;Request_Payload;Reply_Payload
but with entire payloads (not only URIs, but also banners...)
At the moment, the only way I found to manage that is to load the
signature module and write a signature file using payload /.*/, in order
to get the payloads on signature_match events with the data string.
It works, but unfortunately, this solution is a bit heavy in term of CPU
usage. There's probably a way to get these payloads using a built-in
function (and avoiding signatures module) ??? In fact, having a look at
the http-related modules, I can't find how I can handle this problem...
Anybody to help me ??? (or just to tell me it's not possible with
built-in functionnalities... ;-( )
Yohann.
