Greetings
I am integrating bro into a larger system so that I can use it to keep track
of connections (which seems easier than trying to write a method from
scratch with pcap). I thought it would be straightforward to grep for print
or email or alarm statements to figure out where to put the hooks for an IPC
message but so far it eludes me. Is there a principal module for outputting
the notifications?
Thanks
Mike
Hi All,
I was trying to implement an 'OR' condition in the signature payload to
match either of the two patterns
given in payload.
For example:
signature abc-21 {
ip-proto == tcp
. . . .
. . . .
payload /.*(abc) | (xyz).*/
}
When I run Bro with this signature, I was able to see a log for the packet
that matches the pattern first.i.e., the packet with
abc or xyz (whichever comes first) gets logged and the rest doesn't generate
a log.
Only one pattern matches always and the others go unnoticed.
Is there anything wrong in writing the 'OR' condition?
Thanks in advance,
Dhanesh.
Hello All
I have a question about asymmetric routing and Bro IDS
Consider a situation where a traffic to and from an organization takes
different routes and the IDS is deployed where only one directional of
the conversation can be monitored (either client to server OR server
to client).
In such a situation does the TCP analysis of Bro work ? or does it
need to see both sides of the conversation ?
Thanks for the reply.
Thomas
I'm testing Bro on a 10 G link. In some cases, it seems that the
tracefile created by Bro truncates the received packet.
Here's an instance where the Bro running the 10G card cut off the packet
early:
>From Bro running on 10G:
1161587285.381234 128.3.7.51.80 > 66.94.232.246.46965: FP
2067756330:2067756438(108) ack 53063837 win 24616 <nop,nop,timestamp
1275337162 783243679> (DF)
4500 00a0 e4dc 4000 3e06 a4f0 8003 0733
425e e8f6 0050 b775 7b3f 752a 0329 b09d
8019 6028 6a20 0000 0101 080a 4c04 19ca
2eaf 599f
Bro running on 1G:
1161587285.381098 128.3.7.51.80 > 66.94.232.246.46965: FP
2067756330:2067756438(108) ack 53063837 win 24616 <nop,nop,timestamp
1275337162 783243679> (DF)
4500 00a0 e4dc 4000 3e06 a4f0 8003 0733
425e e8f6 0050 b775 7b3f 752a 0329 b09d
8019 6028 6a20 0000 0101 080a 4c04 19ca
2eaf 599f 636b 7175 6f74 653e 3c2f 7464
3e0d 0a20 203c 2f74 723e 0d0a 3c2f 7461
626c 653e 0d0a 0d0a 3c70 3e3c 6120 6872
6566 3d22 2e2f 6c65 6164 5f65 7870 6f73
7572 652e 7368 746d 6c22 3e4e 6578 743e
3c2f 613e 0d0a 0909 0909 0909 0d0a 0d0a
3c2f 626f 6479 3e0d 0a3c 2f68 746d 6c3e
Looking at other examples, it looks like Bro cuts off FIN's with data
when recording packets, sometimes. I've seen this same effect on the 1
Gig traffic too, so I don't think it is a problem with the 10 G card.
In the Bro code, TCP_Rewriter.cc, we see:
void PacketDumper::DumpPacket(const struct pcap_pkthdr* hdr,
const u_char* pkt, int len)
{
if ( pkt_dump )
{
struct pcap_pkthdr h = *hdr;
h.caplen = len;
if ( h.caplen > hdr->caplen )
internal_error("bad modified caplen");
pcap_dump((u_char*) pkt_dump, &h, pkt);
}
}
So it looks like if passed a smaller len than the original capture
length, the output packet will be smaller than the original received
packet. Does anyone know why this is being done? It would be nice if
the entire packet was recorded....
--
Jim Mellander
Incident Response Manager
Computer Protection Program
Lawrence Berkeley National Laboratory
(510) 486-7204
The reason you are having computer problems is:
Just type 'mv * /dev/null'.
> All,
>
> I am having an issue getting ACLD working. I have expect and Acld is running on my system. When a "bad guy" is identified by Bro. It tries and shim in the ACL. When that occurs I see the following error:
>
> acl.sh: Oct 26 16:10:18 drop 2.2.2.2
> couldn't execute "socket": no such file or directoryerror setting blocking mode: resource temporarily unavailable
> while executing
> "spawn socket -c $aclipaddr $aclport"
> (file "/usr/local/bro/bin/acl.sh" line 249)
>
> Am I missing a binary? Or could this be an old version of acl.sh?
>
> Thank you,
>
> Marc
>
> Marc Weisbrod
> Senior Security Engineer
> University of California, San Francisco
> 1855 Folsom Street, Room 602
> San Francisco, CA 94103
>
>
>
Ennobled bro users and developers,
I'm looking for some clarification on the use of bro and multiple
interfaces.
FreeBSD 6.1 machine with two em* (Intel 1000 fibre) interfaces. Each
interface's RX port is connected to one of the two TX ports on a
regenerative tap.
Bro.cfg was originally configured as:
BRO_CAPTURE_INTERFACE="em0 em1"
Additionally, we tried enabling and disabling:
BRO_BPFBOND_ENABLE="YES"
and
BRO_BPFBOND_FLAGS="em0 em1"
In all cases above, we got indications that this configuration was not
correct, and that bro might not be getting all of the traffic across
both interfaces properly (see example #1 below, with content gaps in the
smtp log).
We then set up a bond interface:
ifconfig bridge0 create
ifconfig bridge0 addm em0 addm em1 up
..and changed our bro.cfg to:
BRO_CAPTURE_INTERFACE="bond0"
BRO_BPFBOND_ENABLE="NO"
This seems to work properly now; at least we no longer get content gaps
logged to the smtp log (see example #2 below).
My questions are: Is this (bridge device method) the "right" way to
handle multiple interfaces for my hardware/software? The documentation
mentions kernel patches to enable bpf bonding on FreeBSD 4.1. Is this
not necessary on later FreeBSD releases?
Thanks,
Matt Cuttler
===
example #1, using em0 and em1:
1.2.3.4/1880 > 5.6.7.8/smtp start internal
1.2.3.4/1880 > 5.6.7.8/smtp: unexpected: content gap: \
seq = 30, len = 33
1.2.3.4/1880 < 5.6.7.8/smtp: unusual command/reply: \
(UNKNOWN)() --> 250(OK)
1.2.3.4/1880 > 5.6.7.8/smtp: unexpected: unexpected \
command: RCPT reply = 0 state = 12
1.2.3.4/1880 < 5.6.7.8/smtp: unexpected: content gap: \
seq = 139, len = 14
1.2.3.4/1880 < 5.6.7.8/smtp: unexpected: content gap: \
seq = 153, len = 14
1.2.3.4/1880 < 5.6.7.8/smtp: unusual command/reply: \
(UNKNOWN)() --> 250(Accepted)
1.2.3.4/1880 > 5.6.7.8/smtp: unexpected: unexpected \
command: DATA reply = 0 state = 12
1.2.3.4/1880 > 5.6.7.8/smtp: unexpected: content gap: \
seq = 149, len = 1460
1.2.3.4/1880 > 5.6.7.8/smtp: unexpected: content gap: \
seq = 1609, len = 1697
1.2.3.4/1880 < 5.6.7.8/smtp: unexpected: content gap: \
seq = 237, len = 28
1.2.3.4/1880 < 5.6.7.8/smtp: unusual command/reply: \
(UNKNOWN)() --> 221(mail.host.net closing connection)
finish
===
===
Example #2, using bond0:
1.2.3.4/19100 > 5.6.7.8/smtp start external
recipient: <user(a)email.address>
finish
===
A few people have asked about whether or not you need to register for the
SC'06 conference to attend the workshop. The workshop is not affiliated
with the conference, so you do not need to register. However, depending
on where the workshop room is located, there is a chance that people may
need to purchase a guest pass to get to the room. I'd like to try and
prevent this from happening, but we'll try to let people know if that
is going to be the case as soon as we get more info.
--
James J. Barlow <jbarlow(a)ncsa.uiuc.edu>
Head of Security Operations and Incident Response
National Center for Supercomputing Applications Voice : (217)244-6403
1205 West Clark Street, Urbana, IL 61801 Cell : (217)840-0601
http://www.ncsa.uiuc.edu/~jbarlow Fax : (217)244-1987
Hi!
Nodes: A -- B -- C
The function send_ping (bro.bif) is used for sending a ping only to the
given event_peer, not to all peers im connected with, right? That means,
B can ping C, without sending the ping to A.(?)
Is there a function to do this with events?
Example:
A has requested all events from B
B has requested all events from A and C
C has requested all events from B
Now B wants to send a special question event to A, but not to C. I think
there is no function yet? -> A and C are receiving this event.
My solution is that the event 'question' has a destination-address and
every peer receiving the event compares this address with its own.
event question(dest: addr, [...])
{
if( is_remote_event() && my_addr == dest)
{do something, awnser}
}
@robin
>> more or less, i'm writing a short (german) documentation how
>> inter-bro communication can be used for mobile WLAN clients
> Cool! Can you send me a copy when you're done?
Sure!!
> (Or even before if you'd like to get feedback.)
Yes, Thanks!! :-))
Thanks, Sandro
Bro release 1.2 is now available from:
ftp://bro-ids.org/bro-1.x-current.tar.gz
This becomes the new CURRENT release. The 1.1 branch (formerly CURRENT)
is now the STABLE release:
ftp://bro-ids.org/bro-1.1-stable.tar.gz
The most significant new features with 1.2 are dynamic protocol detection
and a large set of enhancements to the BinPAC system for generating protocol
analyzers. The appended changelog lists numerous other features/changes/fixes.
The old STABLE release, based on the 0.9 release, remains available at
ftp://bro-ids.org/bro-pub-0.9-stable.tar.gz
We do not anticipate making any further changes to it.
Vern
-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1.2 Tue Oct 17 12:09:49 PDT 2006
- Bro now supports DPD, dynamic protocol detection (Robin Sommer, Holger
Dreger, and Michael Mai). With DPD, Bro can analyze protocols regardless
of what port numbers they use: it infers the protocol based on which
application analyzers can parse it without error. Adding this functionality
involved extensive changes to Bro's internals, but also now enables
multiple Bro analyzers to work on the same connection, either concurrently
or one nested inside the other (we have not taken much advantage of this
latter capability yet, but see the FTP events discussed below).
There are a number of new policy scripts, events, and variables associated
with DPD processing, as follows.
Scripts:
You activate DPD by @load'ing dpd.bro. It in turn instructs Bro
to load the signature file policy/sigs/dpd.sig. Note that Bro
uses signatures to expedite deciding which analyzers to try on
a given connection; it does *not* simply use the signatures to
make the determination of which protocol is in use, as this is
insufficiently robust. (At this point, Bro provides signatures
for FTP, IRC, HTTP, SMTP, and SSH. In the future we plan to add
other protocols.)
Along with dpd.bro, you need to @load detect-protocols.bro or
detect-protocols-http.bro. The former enables general detection
of application-layer protocols, while the latter does further
inspection of HTTP sessions to characterize applications running
on top of HTTP such as Gnutella or SOAP. (Loading dpd.bro
is separate from loading one of these scripts because in principle
Bro could use a different means than signatures to activate
the analyzers, although currently it does not.)
If you @load dyn-disable.bro, then once an analyzer determines
that it does not match a given connection, it is deactivated
(and a Notice is generated). Otherwise, it still proceeds to try
its best to analyze the connection (to possibly be more robust
against evasion).
The scripts dce.bro and smb.bro enable DPD for the Windows DCE and
SMB protocols, respectively. (Note that analysis of these protocols
is undergoing a major expansion, not yet complete.)
Events:
event protocol_confirmation(c: connection, atype: count, aid: count)
Generated when the given connection has been confirmed as
conforming with the application type (protocol) specified
by atype. aid is a globally unique analyzer ID that identifies
a particular analyzer instance.
The values for atype are symbolic names associated with
each of Bro's analyzers, such as ANALYZER_IRC. See the
initialization at the beginning of Analyzer.cc for the
full set of names.
The function analyzer_name(atype: count): string translates
these symbolic names into text. For example,
analyzer_name(ANALYZER_IRC)
yields "IRC".
event protocol_violation(c: connection, atype: count, aid: count,
reason: string)
Generated when the given connection has been found to
violate the protocol of the given application type, with
"reason" giving details.
Variables:
dpd_buffer_size: count (default 1024)
Specifies how much pending data Bro keeps for connections
that have not been classified yet. Once this fills, the
data is deleted, though classification can still continue
(see below).
dpd_match_only_beginning: bool (default T)
If set, specifies that Bro should stop signature matching
if it has processed dpd_buffer_size bytes.
dpd_ignore_ports: bool (default F)
If set, then Bro does not take into consideration the port
numbers associated with connections when attempting to
classify them (which can otherwise help the process in
some cases).
dpd_reassemble_first_packets: bool (default T)
If set, then Bro does TCP stream reassembly before applying
signature-matching to detect protocols.
likely_server_ports: set[port]
Specifies a list of ports that Bro will consider as likely
used by servers. For example, if Bro sees a connection
that has already been established (so it does not know
which side sent the initial SYN), and one side uses a port
in this set, then it will assume that that side is the
server (connection responder). The set is empty unless
you populate it or @load server-ports.bro, which specifies
a large number of values.
dpd_config: table[AnalyzerTag] of dpd_protocol_config
Specifies the DPD configuration associated with each tag.
The type dpd_protocol_config is simply:
type dpd_protocol_config: record {
ports: set[port] &optional;
};
i.e., an optional $ports field specifying a set of ports
associatd with the tag. For example, ftp.bro now includes
the equivalent of:
redef dpd_config += {
[ANALYZER_FTP] = [$ports = 21/tcp]
};
Functions:
The function
expect_connection(orig: addr, resp: addr, resp_p: port,
analyzer: count, tout: interval)
is called to alert Bro that a new connection is expected, initiated
by orig to a server running on resp's port resp_p (note: orig's port
is not specified) which will correspond to the specified analyzer
(e.g., "FILE", which is used to analyze files transferred by FTP -
see next item). "tout" is a timeout to associate with the waiting.
The function
function disable_analyzer(cid: conn_id, aid: count)
instructs Bro to disable the analyzer that generated the current
event, assuming the analyzer is associated with the given connection
ID. This is used by the dyn-disable.bro script discussed above.
- A much more complete BinPAC compiler, along with new HTTP, DNS, and
RPC/Portmap analyzers in binpac (Ruoming Pang). The flag "--use-binpac"
activates the BinPAC-based analyzers (currently for HTTP and DNS).
See www.cs.princeton.edu/~rpang/binpac-paper.pdf for a description of
BinPAC, and let Ruoming know if you are interested in using BinPAC to build
new analyzers.
- A new type of analyzer, FILE, analyzes the contents of a connection as
though it were a data file (Robin Sommer). Currently, it can generate
two events:
event file_transferred(c: connection, prefix: string, descr: string,
mime_type: string)
Indicates that the connection transferred a file. "prefix"
is the beginning of the file's data; "descr" and "mime_type"
are indicators of the file's type, as reported by the
"libmagic" library.
descr/mime_type are only set if Bro is configured on a
system that includes the "libmagic" library.
event file_virus(c: connection, virname: string)
Indicates the connection transferred an executable
corresponding to a known virus of the given name.
This functionality is only available if Bro is configured
on a system that includes the "libclamav" library.
Note, this analyzer is enabled via a call to expect_connection by
the FTP analyzer.
- New events relating to IRC analysis (Robin Sommer):
event irc_client(c: connection, prefix: string, data: string)
Generated upon seing a client message sent over the given
IRC connection. "prefix" is the command's prefix as defined
by the IRC protocol. It is used by servers to indicate the
true origin of the message; it may be empty. "data" contains
the message.
event irc_server(c: connection, prefix: string, data: string)
Same for server messages.
event irc_user_message(c: connection, user: string, host: string,
server: string, real_name: string)
Generated upon seeing an IRC "USER" command.
event irc_password_message(c: connection, password: string)
Generated upon seeing an IRC "PASS" command.
event irc_channel_topic(c: connection, channel: string, topic: string)
Generated upon seeing an IRC server reply that includes
the channel topic.
event irc_global_users(c: connection, prefix: string, msg: string)
Generated upon seeing an IRC server reply that includes
a count of the number of IRC users.
- The new experimental script irc-bot.bro tracks IRC-based bots (Robin Sommer).
The accompanying script irc-bot-syslog.bro syslog's the state of the
bot analysis every IrcBot::summary_interval seconds (default 1 minute).
- The new script proxy.bro looks for open Web proxies by matching incoming
requests to a server with outgoing requests it makes (Robin Sommer). It
generates HTTPProxyFound Notices when it finds one.
- Changes to notices.bro (Robin Sommer):
- notice_policy_item's now have a default $result of
NOTICE_FILE and a default $priority of 1.
- The new notice_action_filter, notice_alarm_per_orig, alarms
on the first NoticeType from a specific source. Subsequent
instances are tallied.
- notice_action_filters now reside in the new script
notice-action-filter.bro (automatically loaded by notice.bro).
- The notice actions NOTICE_ALARM_PER_CONN, NOTICE_ALARM_PER_ORIG,
and NOTICE_ALARM_ONCE have been removed, as they were never
actually implemented.
- If the notice_policy returns IGNORE or FILE, the action_filters
filters are no longer consulted.
- A new attribute for tables and sets, &mergeable, changes the semantics
of assignments, as follows (Robin Sommer). Given two &mergeable tables/sets
A and B, an assignment "A = B" becomes actually a join "A = A \cup B"
(i.e., union). The envisoned use is to help avoid race conditions
when doing remote state synchronization.
- The semantics of &synchronized expire_funcs has changed (Robin Sommer).
Now, when a table entry is expired and the operation is propagated to a
a peer, the peer will call its expire_function.
- TRW analysis now skips UDP traffic because it currently treats
all UDP connections as failures (Robin Sommer).
- trw.bro has been split into trw-impl.bro (the algorithm) and
trw.bro (which simply activates the analysis), to facilitate writing
scripts that have hooks into TRW analysis but don't presume it's
active (Robin Sommer).
- The option report_remote_notices in remote.bro has been replaced
by a new script you include, remote-report-notices.bro (Robin Sommer).
- The new function connect_peer() explicitly connects to a remote host
(Robin Sommer).
- The new script remote-send-id.bro sends the current value of an ID
to a remote Bro and then terminates processing (Robin Sommer). It's
intended for use from the command-line, as in
bro -e "redef dst="<dst>" id="<name-of-id>" <scripts> remote-send-id
The other scripts must set up the connection. <dst> is an index into
Remote::destinations corresponding to the destination.
- New built-ins {suspend,resume}_state_updates() can be called to
temporarily avoid propagating updates to &sync'ed values (Robin Sommer).
This can avoid duplicated activity.
- The new function terminate_communication() instructs Bro to end its
communication with remote peers (Robin Sommer).
- The new event remote_state_access_performed is raised when remote state
access has been performed (Robin Sommer). This is primarily for debugging.
- The log() built-in has been renamed to ln() to avoid conflict (Vern Paxson).
- bifcl now generates event generation wrapper functions from event.bif
(Ruoming Pang). For example, to generate event http_reply, currently
one writes:
val_list* vl = new val_list;
vl->append(BuildConnVal());
vl->append(new StringVal(fmt("%.1f", reply_version)));
vl->append(new Val(reply_code, TYPE_COUNT));
if ( reply_reason_phrase )
vl->append(reply_reason_phrase);
else
vl->append(new StringVal("<empty>"));
ConnectionEvent(http_reply, vl);
In the future, one will be able to just call bro_event_http_reply(), and
the code generated by bifcl looks like:
void bro_event_http_reply(Connection* c, StringVal* version,
bro_uint_t code, StringVal* reason)
{
val_list* vl = new val_list;
vl->append(c->BuildConnVal());
vl->append(version);
vl->append(new Val(code, TYPE_COUNT));
vl->append(reason);
mgr.QueueEvent(http_reply, vl, SOURCE_LOCAL, c);
}
Accompanying this change is a semantic shift to types "string" and "port"
in .bif files. They used to be translated to C++ types BroString* and
uint32, respectively. Now they are translated to StringVal* and PortVal*.
The functions in bro.bif are changed accordingly, and please be aware
of this change when you write built-in functions in future.
Also for this change, the parameter 'new' for rsh_request has been renamed
'new_session', as 'new' is a reserved word for C++.
- Some ICMP "connections" now have services identified ("icmp-echo",
"icmp-unreach") rather than just listing the service as "other"
(Ruoming Pang).
- The new option remote_trace_sync_interval specifies an interval after
which each Bro will stop processing its trace and wait for all others
to signal that they have reached the same time (Robin Sommer). The
intent is support for operating Bro in a distributed cluster fashion
(and in particular for debugging such clusters when running off-line
on traces).
This option only works in pseudo-realtime mode, and requires the new
global remote_trace_sync_peers to give the total number of remote peers
(not including self). Signaling is done via a new communication message
type.
- Extensions for DNS transformation/anonymization, including introduction
of trace transformation for protocols other than TCP (Jason Lee).
Not yet fully developed/debugged.
- Extensions for HTTP transformation/anonymization (Martin Casado).
Not yet fully developed/debugged.
- The $conn field is now included in HTTPProxyFound notices (Robin Sommer).
- Changed service inference algorithm to favor lower-numbered
likely-servers over higher-numbered ones (Vern Paxson).
- In pseudo-realtime mode, Bro now uses real-time for deciding which
peer should send state (Robin Sommer).
- Time synchronization for Bro's running on traces in pseudo-realtime mode
added (Robin Sommer).
- Avoidance of false content gaps improved when sorting packets with
out-of-order timestamps (Ruoming Pang).
- Packets from the packet sorter are now more robustly drained upon
termination of input (Ruoming Pang).
- Documentation for deep-copy updated (Christian Kreibich).
- Nasty fragment reassembly bug fixed (Vern Paxson).
- Serious bugs in EDNS0 processing fixed (Vern Paxson).
- Fixed significant misfeature of interconn.bro that stopped all processing
of a connection once it makes a detection (Vern Paxson).
- Fixes for &read_expire operation across synchronizes tables (Robin Sommer).
- Fixes for multiple peers exchanging initial &sync state simultaneously
(Robin Sommer).
- Improvements to graceful termination of Bro when communicating with
remote peers (Robin Sommer).
- Fix for ICMP analyzer not always generating icmp_sent events
(Robin Sommer). This appears to still need some work, as now
it generates redundant events.
- Fix for initial exchange of &sync state which could lead to
referencing unknown IDs (Robin Sommer).
- Fix to scan detection for differing semantics of connection compressor
vs. non-compressor (Robin Sommer).
- Bug fix for distinguishing regular expression matches of length 0 from
those of length 1 (Ruoming Pang).
- Fix for SSH version parsing in the presence of content gaps (Robin Sommer).
- Bug fix for IRC that could lead to crashes (Robin Sommer).
- Bug fix to refrain from adding new timers when a connection has
already been removed from the connection table (Robin Sommer).
- Bug fix for packet_contents not including the transport-layer header
(Robin Sommer).
- Some memory leaks fixed (Robin Sommer).
- A bunch of portability and distribution problems fixed (Christian
Kreibich, Robin Sommer, Vern Paxson).