zeek October 2006

zeek@lists.zeek.org

21 participants
20 discussions

by Mike Muratet

Greetings I am integrating bro into a larger system so that I can use it to keep track of connections (which seems easier than trying to write a method from scratch with pcap). I thought it would be straightforward to grep for print or email or alarm statements to figure out where to put the hooks for an IPC message but so far it eludes me. Is there a principal module for outputting the notifications? Thanks Mike

14 years, 1 month

[Bro] Using a 'OR' condition in Signature payloads

by Jaya Dhanesh

Hi All, I was trying to implement an 'OR' condition in the signature payload to match either of the two patterns given in payload. For example: signature abc-21 { ip-proto == tcp . . . . . . . . payload /.*(abc) | (xyz).*/ } When I run Bro with this signature, I was able to see a log for the packet that matches the pattern first.i.e., the packet with abc or xyz (whichever comes first) gets logged and the rest doesn't generate a log. Only one pattern matches always and the others go unnoticed. Is there anything wrong in writing the 'OR' condition? Thanks in advance, Dhanesh.

14 years, 10 months

[Bro] Bro

by Sergejs Kravchenko

14 years, 10 months

[Bro] Bro and asymmetric routing

by Adayadil Thomas

Hello All I have a question about asymmetric routing and Bro IDS Consider a situation where a traffic to and from an organization takes different routes and the IDS is deployed where only one directional of the conversation can be monitored (either client to server OR server to client). In such a situation does the TCP analysis of Bro work ? or does it need to see both sides of the conversation ? Thanks for the reply. Thomas

14 years, 10 months

[Bro] Bro trace files - packets truncated in some circumstances?

by Jim Mellander

I'm testing Bro on a 10 G link. In some cases, it seems that the tracefile created by Bro truncates the received packet. Here's an instance where the Bro running the 10G card cut off the packet early: >From Bro running on 10G: 1161587285.381234 128.3.7.51.80 > 66.94.232.246.46965: FP 2067756330:2067756438(108) ack 53063837 win 24616 <nop,nop,timestamp 1275337162 783243679> (DF) 4500 00a0 e4dc 4000 3e06 a4f0 8003 0733 425e e8f6 0050 b775 7b3f 752a 0329 b09d 8019 6028 6a20 0000 0101 080a 4c04 19ca 2eaf 599f Bro running on 1G: 1161587285.381098 128.3.7.51.80 > 66.94.232.246.46965: FP 2067756330:2067756438(108) ack 53063837 win 24616 <nop,nop,timestamp 1275337162 783243679> (DF) 4500 00a0 e4dc 4000 3e06 a4f0 8003 0733 425e e8f6 0050 b775 7b3f 752a 0329 b09d 8019 6028 6a20 0000 0101 080a 4c04 19ca 2eaf 599f 636b 7175 6f74 653e 3c2f 7464 3e0d 0a20 203c 2f74 723e 0d0a 3c2f 7461 626c 653e 0d0a 0d0a 3c70 3e3c 6120 6872 6566 3d22 2e2f 6c65 6164 5f65 7870 6f73 7572 652e 7368 746d 6c22 3e4e 6578 743e 3c2f 613e 0d0a 0909 0909 0909 0d0a 0d0a 3c2f 626f 6479 3e0d 0a3c 2f68 746d 6c3e Looking at other examples, it looks like Bro cuts off FIN's with data when recording packets, sometimes. I've seen this same effect on the 1 Gig traffic too, so I don't think it is a problem with the 10 G card. In the Bro code, TCP_Rewriter.cc, we see: void PacketDumper::DumpPacket(const struct pcap_pkthdr* hdr, const u_char* pkt, int len) { if ( pkt_dump ) { struct pcap_pkthdr h = *hdr; h.caplen = len; if ( h.caplen > hdr->caplen ) internal_error("bad modified caplen"); pcap_dump((u_char*) pkt_dump, &h, pkt); } } So it looks like if passed a smaller len than the original capture length, the output packet will be smaller than the original received packet. Does anyone know why this is being done? It would be nice if the entire packet was recorded.... -- Jim Mellander Incident Response Manager Computer Protection Program Lawrence Berkeley National Laboratory (510) 486-7204 The reason you are having computer problems is: Just type 'mv * /dev/null'.

14 years, 10 months

[Bro] FW: ACLD implementation

by Weisbrod, Marc

> All, > > I am having an issue getting ACLD working. I have expect and Acld is running on my system. When a "bad guy" is identified by Bro. It tries and shim in the ACL. When that occurs I see the following error: > > acl.sh: Oct 26 16:10:18 drop 2.2.2.2 > couldn't execute "socket": no such file or directoryerror setting blocking mode: resource temporarily unavailable > while executing > "spawn socket -c $aclipaddr $aclport" > (file "/usr/local/bro/bin/acl.sh" line 249) > > Am I missing a binary? Or could this be an old version of acl.sh? > > Thank you, > > Marc > > Marc Weisbrod > Senior Security Engineer > University of California, San Francisco > 1855 Folsom Street, Room 602 > San Francisco, CA 94103 > > >

14 years, 11 months

[Bro] bridge interface vs. bpf bonding (patch?) on FreeBSD 6.1

by Matt Cuttler

Ennobled bro users and developers, I'm looking for some clarification on the use of bro and multiple interfaces. FreeBSD 6.1 machine with two em* (Intel 1000 fibre) interfaces. Each interface's RX port is connected to one of the two TX ports on a regenerative tap. Bro.cfg was originally configured as: BRO_CAPTURE_INTERFACE="em0 em1" Additionally, we tried enabling and disabling: BRO_BPFBOND_ENABLE="YES" and BRO_BPFBOND_FLAGS="em0 em1" In all cases above, we got indications that this configuration was not correct, and that bro might not be getting all of the traffic across both interfaces properly (see example #1 below, with content gaps in the smtp log). We then set up a bond interface: ifconfig bridge0 create ifconfig bridge0 addm em0 addm em1 up ..and changed our bro.cfg to: BRO_CAPTURE_INTERFACE="bond0" BRO_BPFBOND_ENABLE="NO" This seems to work properly now; at least we no longer get content gaps logged to the smtp log (see example #2 below). My questions are: Is this (bridge device method) the "right" way to handle multiple interfaces for my hardware/software? The documentation mentions kernel patches to enable bpf bonding on FreeBSD 4.1. Is this not necessary on later FreeBSD releases? Thanks, Matt Cuttler === example #1, using em0 and em1: 1.2.3.4/1880 > 5.6.7.8/smtp start internal 1.2.3.4/1880 > 5.6.7.8/smtp: unexpected: content gap: \ seq = 30, len = 33 1.2.3.4/1880 < 5.6.7.8/smtp: unusual command/reply: \ (UNKNOWN)() --> 250(OK) 1.2.3.4/1880 > 5.6.7.8/smtp: unexpected: unexpected \ command: RCPT reply = 0 state = 12 1.2.3.4/1880 < 5.6.7.8/smtp: unexpected: content gap: \ seq = 139, len = 14 1.2.3.4/1880 < 5.6.7.8/smtp: unexpected: content gap: \ seq = 153, len = 14 1.2.3.4/1880 < 5.6.7.8/smtp: unusual command/reply: \ (UNKNOWN)() --> 250(Accepted) 1.2.3.4/1880 > 5.6.7.8/smtp: unexpected: unexpected \ command: DATA reply = 0 state = 12 1.2.3.4/1880 > 5.6.7.8/smtp: unexpected: content gap: \ seq = 149, len = 1460 1.2.3.4/1880 > 5.6.7.8/smtp: unexpected: content gap: \ seq = 1609, len = 1697 1.2.3.4/1880 < 5.6.7.8/smtp: unexpected: content gap: \ seq = 237, len = 28 1.2.3.4/1880 < 5.6.7.8/smtp: unusual command/reply: \ (UNKNOWN)() --> 221(mail.host.net closing connection) finish === === Example #2, using bond0: 1.2.3.4/19100 > 5.6.7.8/smtp start external recipient: <user(a)email.address> finish ===

14 years, 11 months

[Bro] More info on Bro workshop at SC'06

by James J. Barlow

A few people have asked about whether or not you need to register for the SC'06 conference to attend the workshop. The workshop is not affiliated with the conference, so you do not need to register. However, depending on where the workshop room is located, there is a chance that people may need to purchase a guest pass to get to the room. I'd like to try and prevent this from happening, but we'll try to let people know if that is going to be the case as soon as we get more info. -- James J. Barlow <jbarlow(a)ncsa.uiuc.edu> Head of Security Operations and Incident Response National Center for Supercomputing Applications Voice : (217)244-6403 1205 West Clark Street, Urbana, IL 61801 Cell : (217)840-0601 http://www.ncsa.uiuc.edu/~jbarlow Fax : (217)244-1987

14 years, 11 months

[Bro] sending a question to a special peer

by Sandro Reichert

Hi! Nodes: A -- B -- C The function send_ping (bro.bif) is used for sending a ping only to the given event_peer, not to all peers im connected with, right? That means, B can ping C, without sending the ping to A.(?) Is there a function to do this with events? Example: A has requested all events from B B has requested all events from A and C C has requested all events from B Now B wants to send a special question event to A, but not to C. I think there is no function yet? -> A and C are receiving this event. My solution is that the event 'question' has a destination-address and every peer receiving the event compares this address with its own. event question(dest: addr, [...]) { if( is_remote_event() && my_addr == dest) {do something, awnser} } @robin >> more or less, i'm writing a short (german) documentation how >> inter-bro communication can be used for mobile WLAN clients > Cool! Can you send me a copy when you're done? Sure!! > (Or even before if you'd like to get feedback.) Yes, Thanks!! :-)) Thanks, Sandro

14 years, 11 months

[Bro] new Bro CURRENT and STABLE releases (1.2 and 1.1)

by Vern Paxson

Bro release 1.2 is now available from: ftp://bro-ids.org/bro-1.x-current.tar.gz This becomes the new CURRENT release. The 1.1 branch (formerly CURRENT) is now the STABLE release: ftp://bro-ids.org/bro-1.1-stable.tar.gz The most significant new features with 1.2 are dynamic protocol detection and a large set of enhancements to the BinPAC system for generating protocol analyzers. The appended changelog lists numerous other features/changes/fixes. The old STABLE release, based on the 0.9 release, remains available at ftp://bro-ids.org/bro-pub-0.9-stable.tar.gz We do not anticipate making any further changes to it. Vern -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1.2 Tue Oct 17 12:09:49 PDT 2006 - Bro now supports DPD, dynamic protocol detection (Robin Sommer, Holger Dreger, and Michael Mai). With DPD, Bro can analyze protocols regardless of what port numbers they use: it infers the protocol based on which application analyzers can parse it without error. Adding this functionality involved extensive changes to Bro's internals, but also now enables multiple Bro analyzers to work on the same connection, either concurrently or one nested inside the other (we have not taken much advantage of this latter capability yet, but see the FTP events discussed below). There are a number of new policy scripts, events, and variables associated with DPD processing, as follows. Scripts: You activate DPD by @load'ing dpd.bro. It in turn instructs Bro to load the signature file policy/sigs/dpd.sig. Note that Bro uses signatures to expedite deciding which analyzers to try on a given connection; it does *not* simply use the signatures to make the determination of which protocol is in use, as this is insufficiently robust. (At this point, Bro provides signatures for FTP, IRC, HTTP, SMTP, and SSH. In the future we plan to add other protocols.) Along with dpd.bro, you need to @load detect-protocols.bro or detect-protocols-http.bro. The former enables general detection of application-layer protocols, while the latter does further inspection of HTTP sessions to characterize applications running on top of HTTP such as Gnutella or SOAP. (Loading dpd.bro is separate from loading one of these scripts because in principle Bro could use a different means than signatures to activate the analyzers, although currently it does not.) If you @load dyn-disable.bro, then once an analyzer determines that it does not match a given connection, it is deactivated (and a Notice is generated). Otherwise, it still proceeds to try its best to analyze the connection (to possibly be more robust against evasion). The scripts dce.bro and smb.bro enable DPD for the Windows DCE and SMB protocols, respectively. (Note that analysis of these protocols is undergoing a major expansion, not yet complete.) Events: event protocol_confirmation(c: connection, atype: count, aid: count) Generated when the given connection has been confirmed as conforming with the application type (protocol) specified by atype. aid is a globally unique analyzer ID that identifies a particular analyzer instance. The values for atype are symbolic names associated with each of Bro's analyzers, such as ANALYZER_IRC. See the initialization at the beginning of Analyzer.cc for the full set of names. The function analyzer_name(atype: count): string translates these symbolic names into text. For example, analyzer_name(ANALYZER_IRC) yields "IRC". event protocol_violation(c: connection, atype: count, aid: count, reason: string) Generated when the given connection has been found to violate the protocol of the given application type, with "reason" giving details. Variables: dpd_buffer_size: count (default 1024) Specifies how much pending data Bro keeps for connections that have not been classified yet. Once this fills, the data is deleted, though classification can still continue (see below). dpd_match_only_beginning: bool (default T) If set, specifies that Bro should stop signature matching if it has processed dpd_buffer_size bytes. dpd_ignore_ports: bool (default F) If set, then Bro does not take into consideration the port numbers associated with connections when attempting to classify them (which can otherwise help the process in some cases). dpd_reassemble_first_packets: bool (default T) If set, then Bro does TCP stream reassembly before applying signature-matching to detect protocols. likely_server_ports: set[port] Specifies a list of ports that Bro will consider as likely used by servers. For example, if Bro sees a connection that has already been established (so it does not know which side sent the initial SYN), and one side uses a port in this set, then it will assume that that side is the server (connection responder). The set is empty unless you populate it or @load server-ports.bro, which specifies a large number of values. dpd_config: table[AnalyzerTag] of dpd_protocol_config Specifies the DPD configuration associated with each tag. The type dpd_protocol_config is simply: type dpd_protocol_config: record { ports: set[port] &optional; }; i.e., an optional $ports field specifying a set of ports associatd with the tag. For example, ftp.bro now includes the equivalent of: redef dpd_config += { [ANALYZER_FTP] = [$ports = 21/tcp] }; Functions: The function expect_connection(orig: addr, resp: addr, resp_p: port, analyzer: count, tout: interval) is called to alert Bro that a new connection is expected, initiated by orig to a server running on resp's port resp_p (note: orig's port is not specified) which will correspond to the specified analyzer (e.g., "FILE", which is used to analyze files transferred by FTP - see next item). "tout" is a timeout to associate with the waiting. The function function disable_analyzer(cid: conn_id, aid: count) instructs Bro to disable the analyzer that generated the current event, assuming the analyzer is associated with the given connection ID. This is used by the dyn-disable.bro script discussed above. - A much more complete BinPAC compiler, along with new HTTP, DNS, and RPC/Portmap analyzers in binpac (Ruoming Pang). The flag "--use-binpac" activates the BinPAC-based analyzers (currently for HTTP and DNS). See www.cs.princeton.edu/~rpang/binpac-paper.pdf for a description of BinPAC, and let Ruoming know if you are interested in using BinPAC to build new analyzers. - A new type of analyzer, FILE, analyzes the contents of a connection as though it were a data file (Robin Sommer). Currently, it can generate two events: event file_transferred(c: connection, prefix: string, descr: string, mime_type: string) Indicates that the connection transferred a file. "prefix" is the beginning of the file's data; "descr" and "mime_type" are indicators of the file's type, as reported by the "libmagic" library. descr/mime_type are only set if Bro is configured on a system that includes the "libmagic" library. event file_virus(c: connection, virname: string) Indicates the connection transferred an executable corresponding to a known virus of the given name. This functionality is only available if Bro is configured on a system that includes the "libclamav" library. Note, this analyzer is enabled via a call to expect_connection by the FTP analyzer. - New events relating to IRC analysis (Robin Sommer): event irc_client(c: connection, prefix: string, data: string) Generated upon seing a client message sent over the given IRC connection. "prefix" is the command's prefix as defined by the IRC protocol. It is used by servers to indicate the true origin of the message; it may be empty. "data" contains the message. event irc_server(c: connection, prefix: string, data: string) Same for server messages. event irc_user_message(c: connection, user: string, host: string, server: string, real_name: string) Generated upon seeing an IRC "USER" command. event irc_password_message(c: connection, password: string) Generated upon seeing an IRC "PASS" command. event irc_channel_topic(c: connection, channel: string, topic: string) Generated upon seeing an IRC server reply that includes the channel topic. event irc_global_users(c: connection, prefix: string, msg: string) Generated upon seeing an IRC server reply that includes a count of the number of IRC users. - The new experimental script irc-bot.bro tracks IRC-based bots (Robin Sommer). The accompanying script irc-bot-syslog.bro syslog's the state of the bot analysis every IrcBot::summary_interval seconds (default 1 minute). - The new script proxy.bro looks for open Web proxies by matching incoming requests to a server with outgoing requests it makes (Robin Sommer). It generates HTTPProxyFound Notices when it finds one. - Changes to notices.bro (Robin Sommer): - notice_policy_item's now have a default $result of NOTICE_FILE and a default $priority of 1. - The new notice_action_filter, notice_alarm_per_orig, alarms on the first NoticeType from a specific source. Subsequent instances are tallied. - notice_action_filters now reside in the new script notice-action-filter.bro (automatically loaded by notice.bro). - The notice actions NOTICE_ALARM_PER_CONN, NOTICE_ALARM_PER_ORIG, and NOTICE_ALARM_ONCE have been removed, as they were never actually implemented. - If the notice_policy returns IGNORE or FILE, the action_filters filters are no longer consulted. - A new attribute for tables and sets, &mergeable, changes the semantics of assignments, as follows (Robin Sommer). Given two &mergeable tables/sets A and B, an assignment "A = B" becomes actually a join "A = A \cup B" (i.e., union). The envisoned use is to help avoid race conditions when doing remote state synchronization. - The semantics of &synchronized expire_funcs has changed (Robin Sommer). Now, when a table entry is expired and the operation is propagated to a a peer, the peer will call its expire_function. - TRW analysis now skips UDP traffic because it currently treats all UDP connections as failures (Robin Sommer). - trw.bro has been split into trw-impl.bro (the algorithm) and trw.bro (which simply activates the analysis), to facilitate writing scripts that have hooks into TRW analysis but don't presume it's active (Robin Sommer). - The option report_remote_notices in remote.bro has been replaced by a new script you include, remote-report-notices.bro (Robin Sommer). - The new function connect_peer() explicitly connects to a remote host (Robin Sommer). - The new script remote-send-id.bro sends the current value of an ID to a remote Bro and then terminates processing (Robin Sommer). It's intended for use from the command-line, as in bro -e "redef dst="<dst>" id="<name-of-id>" <scripts> remote-send-id The other scripts must set up the connection. <dst> is an index into Remote::destinations corresponding to the destination. - New built-ins {suspend,resume}_state_updates() can be called to temporarily avoid propagating updates to &sync'ed values (Robin Sommer). This can avoid duplicated activity. - The new function terminate_communication() instructs Bro to end its communication with remote peers (Robin Sommer). - The new event remote_state_access_performed is raised when remote state access has been performed (Robin Sommer). This is primarily for debugging. - The log() built-in has been renamed to ln() to avoid conflict (Vern Paxson). - bifcl now generates event generation wrapper functions from event.bif (Ruoming Pang). For example, to generate event http_reply, currently one writes: val_list* vl = new val_list; vl->append(BuildConnVal()); vl->append(new StringVal(fmt("%.1f", reply_version))); vl->append(new Val(reply_code, TYPE_COUNT)); if ( reply_reason_phrase ) vl->append(reply_reason_phrase); else vl->append(new StringVal("<empty>")); ConnectionEvent(http_reply, vl); In the future, one will be able to just call bro_event_http_reply(), and the code generated by bifcl looks like: void bro_event_http_reply(Connection* c, StringVal* version, bro_uint_t code, StringVal* reason) { val_list* vl = new val_list; vl->append(c->BuildConnVal()); vl->append(version); vl->append(new Val(code, TYPE_COUNT)); vl->append(reason); mgr.QueueEvent(http_reply, vl, SOURCE_LOCAL, c); } Accompanying this change is a semantic shift to types "string" and "port" in .bif files. They used to be translated to C++ types BroString* and uint32, respectively. Now they are translated to StringVal* and PortVal*. The functions in bro.bif are changed accordingly, and please be aware of this change when you write built-in functions in future. Also for this change, the parameter 'new' for rsh_request has been renamed 'new_session', as 'new' is a reserved word for C++. - Some ICMP "connections" now have services identified ("icmp-echo", "icmp-unreach") rather than just listing the service as "other" (Ruoming Pang). - The new option remote_trace_sync_interval specifies an interval after which each Bro will stop processing its trace and wait for all others to signal that they have reached the same time (Robin Sommer). The intent is support for operating Bro in a distributed cluster fashion (and in particular for debugging such clusters when running off-line on traces). This option only works in pseudo-realtime mode, and requires the new global remote_trace_sync_peers to give the total number of remote peers (not including self). Signaling is done via a new communication message type. - Extensions for DNS transformation/anonymization, including introduction of trace transformation for protocols other than TCP (Jason Lee). Not yet fully developed/debugged. - Extensions for HTTP transformation/anonymization (Martin Casado). Not yet fully developed/debugged. - The $conn field is now included in HTTPProxyFound notices (Robin Sommer). - Changed service inference algorithm to favor lower-numbered likely-servers over higher-numbered ones (Vern Paxson). - In pseudo-realtime mode, Bro now uses real-time for deciding which peer should send state (Robin Sommer). - Time synchronization for Bro's running on traces in pseudo-realtime mode added (Robin Sommer). - Avoidance of false content gaps improved when sorting packets with out-of-order timestamps (Ruoming Pang). - Packets from the packet sorter are now more robustly drained upon termination of input (Ruoming Pang). - Documentation for deep-copy updated (Christian Kreibich). - Nasty fragment reassembly bug fixed (Vern Paxson). - Serious bugs in EDNS0 processing fixed (Vern Paxson). - Fixed significant misfeature of interconn.bro that stopped all processing of a connection once it makes a detection (Vern Paxson). - Fixes for &read_expire operation across synchronizes tables (Robin Sommer). - Fixes for multiple peers exchanging initial &sync state simultaneously (Robin Sommer). - Improvements to graceful termination of Bro when communicating with remote peers (Robin Sommer). - Fix for ICMP analyzer not always generating icmp_sent events (Robin Sommer). This appears to still need some work, as now it generates redundant events. - Fix for initial exchange of &sync state which could lead to referencing unknown IDs (Robin Sommer). - Fix to scan detection for differing semantics of connection compressor vs. non-compressor (Robin Sommer). - Bug fix for distinguishing regular expression matches of length 0 from those of length 1 (Ruoming Pang). - Fix for SSH version parsing in the presence of content gaps (Robin Sommer). - Bug fix for IRC that could lead to crashes (Robin Sommer). - Bug fix to refrain from adding new timers when a connection has already been removed from the connection table (Robin Sommer). - Bug fix for packet_contents not including the transport-layer header (Robin Sommer). - Some memory leaks fixed (Robin Sommer). - A bunch of portability and distribution problems fixed (Christian Kreibich, Robin Sommer, Vern Paxson).

14 years, 11 months

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

zeek October 2006