Hi all,
My goal is to integrate a new protocol analyzer in Bro. This protocol
(PROFINET dyscovery and Basic Configuration Protocol) is working on layer
2. My question is, are there special considerations to get at the data of
the layer 2? My colleague has tried creating an analyzer by following your
instructions for coding an analyzer by binpac. Before he went on vacation,
he told me, he could access data with binpac of layer 3 but not of layer
2? Is that correct? If so does it work with the new binpac ++? Any pieces
of advice or suggestions how to get started would be greatly appreciated.
Kind regards
Marcel Odenwald
Hello,
There are some hilti-based parsers in the Bro docker image. When I run
the pcaps for BACnet (/opt/hilti/bro/tests/Traces/bacnet/*.pcap) through
Bro (eg bro -r NPDU.pcap) , no event logs are produced in
/usr/local/bro/logs).
How do I integrate these parsers into Bro?
- Troy
--
Troy Jordan
t r o y j @ m a i n e . e d u
GIAC GCIH,GCIA
------------------------------------------------------------
Network Systems Security Analyst
Information Technology Security Office
University of Maine System
------------------------------------------------------------
233 Science Building | voice: 207.561.3590
Portland, ME 04103 | fax: 509.351.3650
"As you all know, Security Is Mortals chiefest Enemy"
William Shakespeare, Macbeth
Hello,
I have been using the Bro 2.4 to test the performance of SFC driver. I have observed the following issue because of which I am unable to proceed with any analysis -
There seems to be a memory leak somewhere as there are times when Bro runs out of memory too soon. These are the instances when drops are also seen too soon even at very low packet rates.
When Bro is started, the available free memory keeps going down till a point where the server is extremely sluggish and there are drops being seen -
An instance of Bro running out of memory (with 16 workers, no cpu pinning and having sent 155K pps for 7-8 minutes)-
[root@dellr620c skathare]# free -m
total used free shared buffers cached
Mem: 32129 31917 211 3 1 376 --> 211MB : that's very low, considering the system started with some 26GB free memory (and this drop happens just within the first 2 minutes of running the traffic). System becomes very slow at this point and, of course, it has started dropping packets already.
-/+ buffers/cache: 31539 589
Swap: 1907 1687 219
[root@dellr620c skathare]#
Swap: 1907 1764 142
[root@dellr620c skathare]# cat /proc/meminfo
MemTotal: 32900200 kB
MemFree: 193384 kB
MemAvailable: 480956 kB
Buffers: 2464 kB
Cached: 471260 kB
SwapCached: 74860 kB
Active: 23439908 kB
Inactive: 3120012 kB
Active(anon): 23179296 kB
Inactive(anon): 2914628 kB
Active(file): 260612 kB
Inactive(file): 205384 kB
Unevictable: 0 kB
Mlocked: 0 kB
SwapTotal: 1953076 kB
SwapFree: 152264 kB
Dirty: 22548 kB
Writeback: 8 kB
AnonPages: 26017216 kB
Mapped: 15200 kB
Shmem: 4692 kB
Slab: 190556 kB
SReclaimable: 93648 kB
SUnreclaim: 96908 kB
KernelStack: 4288 kB
PageTables: 71380 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 15638376 kB
Committed_AS: 29119672 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 374920 kB
VmallocChunk: 34342144308 kB
HardwareCorrupted: 0 kB
AnonHugePages: 243712 kB
HugePages_Total: 2700
HugePages_Free: 46
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
DirectMap4k: 345024 kB
DirectMap2M: 18483200 kB
DirectMap1G: 16777216 kB
[root@dellr620c skathare]#top
top - 22:48:06 up 70 days, 19:43, 5 users, load average: 18.25, 13.49, 10.04
Tasks: 17 total, 1 running, 16 sleeping, 0 stopped, 0 zombie
%Cpu(s): 31.5 us, 2.9 sy, 0.7 ni, 43.7 id, 21.1 wa, 0.0 hi, 0.0 si, 0.0 st
KiB Mem: 32900200 total, 32731868 used, 168332 free, 336 buffers
KiB Swap: 1953076 total, 1262956 used, 690120 free. 14248 cached Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND P
20504 root 20 0 404644 32724 1464 D 0.0 0.1 6:32.62 bro 31
20552 root 20 0 404692 50108 1496 D 52.1 0.2 6:37.18 bro 29
20564 root 20 0 404824 50960 1508 D 49.8 0.2 6:37.16 bro 27
20574 root 20 0 404652 48748 1476 D 52.0 0.1 6:36.52 bro 24
20567 root 20 0 404684 49948 1456 D 0.0 0.2 6:33.32 bro 23
20561 root 20 0 421440 66672 1412 D 51.9 0.2 6:37.14 bro 18
20569 root 20 0 404708 31904 1508 D 41.1 0.1 6:34.77 bro 16
20495 root 20 0 404620 49936 1408 D 27.1 0.2 6:34.91 bro 13
20515 root 20 0 404684 46324 1500 D 21.9 0.1 6:33.25 bro 13
20548 root 20 0 404704 50188 1504 D 43.6 0.2 6:35.16 bro 13
20474 root 20 0 404736 32704 1508 D 0.0 0.1 6:32.79 bro 12
20502 root 20 0 404636 29300 1464 D 52.0 0.1 6:36.13 bro 12
20539 root 20 0 404748 32784 1484 D 44.7 0.1 6:34.08 bro 11
20537 root 20 0 404668 29284 1464 D 0.0 0.1 6:32.03 bro 4
20559 root 20 0 404684 32644 1444 R 54.6 0.1 6:38.12 bro 3
20542 root 20 0 404728 32704 1504 D 25.1 0.1 6:33.84 bro 1
20289 root 20 0 196768 412 412 S 0.0 0.0 0:00.03 solar_clusterd 0
After stopping the BRO workers (especially after the manager is killed/stopped), memory recovers -
top - 22:53:01 up 70 days, 19:48, 5 users, load average: 3.06, 8.83, 9.20
Tasks: 1 total, 0 running, 1 sleeping, 0 stopped, 0 zombie
%Cpu(s): 0.1 us, 0.3 sy, 2.2 ni, 96.6 id, 0.8 wa, 0.0 hi, 0.0 si, 0.0 st
KiB Mem: 32900200 total, 6702252 used, 26197948 free, 8308 buffers --> This is almost what the system originally started with - 26GB
KiB Swap: 1953076 total, 237216 used, 1715860 free. 528032 cached Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND P
20289 root 20 0 196768 1028 484 S 0.0 0.0 0:00.03 solar_clusterd
At very high packet rates, the available free memory keeps going down very fast and starts dropping packets. At lower packet rates, the drop in available free memory is comparatively slower, but it is still there and packets are dropped eventually. When the BRO workers are stopped, the available free memory recovers. During the few successful times when I have been able to go till 150Kpps without seeing any packet drops, the available free memory remained a constant at ~23G. It remained at this for the entire duration of the test (more than an hour ) and no drops were seen.
The above data is a few days old. When I tried running BRO again today, I saw the memory drop from 18G to 4G in just a matter of few seconds after starting BRO (16 workers, each pinned to a CPU). Is it possible that Bro is accumulating some per-flow state and not freeing it? If so, is there any tuning that should be done to avoid this?
Appreciate any help on this!
-
Sampada
The information contained in this message is confidential and is intended for the addressee(s) only. If you have received this message in error, please notify the sender immediately and delete the message. Unless you are an addressee (or authorized to receive for an addressee), you may not use, copy or disclose to anyone this message or any information contained in this message. The unauthorized use, disclosure, copying or alteration of this message is strictly prohibited.
Hi Gents,
I have had Bro installed as my gateway for a home network for about nine
months now, with a complete (mostly uninterrupted) run of logs. I've also
supplemented this with the critical stack plugin since July, with intel
feeds up - focused mostly on malware and candc domains.
The network is reasonably busy, has probably about 25 discreet hosts of
which at any given time between 3 and 10 are up. I have suspected there is
malware / a rootkit perhaps on the network for a while as arp -a shows a
lot of <undefined> hosts every now and then from the terminal of most
systems on the network. Also, Nmap scans often report IP addresses that
simply are not there.
Also, Bro reports traffic to local NAT IP addresses that don't exist. eg my
network is divided into a 192.168.2.x (Internal, all the hosts) and
192.168.1.x(airgap between Bro router and domestic DSL router). The
192.168.1.x network only really ever has two hosts - the bro router and the
dsl router, but connections show to other addresses which don't exist.
I have tried to put a methodology together for malware hunting based on
what I can find online, but nothing has really come to light. I use zcat,
bro-cut and regular expressions to query the logs.
Would anyone on this list mind assisting me in a bug hunt / provide a
methodology for tracking down suspicious traffic?
I have looked and looked but can't seem to find any workflow / tolling
which can isolate malware effectively. Any advice on this would be very
gratefully received!
regs
Perry
Hello,
I'm having problems getting file names from fa_file - the field f$info$filename is showing up uninitialized on every single fa_file in all my tests. Is there a known reason why this would be happening? I'm using Bro 2.3, but I tested on 2.4 as well and got the same results.
Are there any alternative ways to get file names? For now I'm parsing the URL returned by Files::describe(f), but this does not work if the URL doesn't contain the file name, or if the file was transferred with a protocol other than HTTP.
Thanks,
Nathan Pigott
When starting the Bro service, an error stating that Bro terminated
immediately after start-up continually appears. The following is the info I
am provided upon running the diag command.
---------------------------------------------------------------------------------------------------------------------------------------
Bro 2.4
Linux 2.6.32-573.7.1.el6.x86_64
==== No reporter.log
==== stderr.log
fatal error: problem with interface p3p1 (p3p1: no IPv4 address assigned)
==== stdout.log
max memory size (kbytes, -m) unlimited
data seg size (kbytes, -d) unlimited
virtual memory (kbytes, -v) unlimited
core file size (blocks, -c) unlimited
==== .cmdline
-i p3p1 -U .status -p broctl -p broctl-live -p standalone -p local -p bro
local.bro broctl broctl/standalone broctl/auto
==== .env_vars
PATH=/usr/local/bro/bin:/usr/local/bro/share/broctl/scripts:/usr/local/bro/bin:/usr/lib64/qt-3.3/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin
BROPATH=/usr/local/bro/spool/installed-scripts-do-not-touch/site::/usr/local/bro/spool/installed-scripts-do-not-touch/auto:/usr/local/bro/share/bro:/usr/local/bro/share/bro/policy:/usr/local/bro/share/bro/site
CLUSTER_NODE=
==== .status
TERMINATED [atexit]
==== No prof.log
==== No packet_filter.log
==== No loaded_scripts.log
------------------------------------------------------------------------------------------------------------------------------------------
--
Gabriel Dinkins
Cyber Security Engineer
PUNCH Cyber Analytics Group
We've run into a few problems with our scripts and the use of &persistent,
so we're looking to do some home-grown persistence. These scripts are part
of a module called ConnectionValidation; the applicable bits of the scripts
are at https://gist.github.com/mutemule/6076cddce3ce8c9e7013. It's worth
pointing out that the module as a whole is loaded in to all components, but
the persistence layer is only loaded in to the proxy.
What I'm seeing is the table being written to disk as expected during
bro_done(), but seemingly not being read back in during bro_init(): after
startup, the table remains blank in all cluster components.
I'd previously tried this with a set instead of a table, but that didn't
work. Then I tried using events to populate the set, but that also didn't
work. So now I'm on a table, and following the input framework
documentation[0] almost exactly, but it's still not doing what I expected.
What am I doing wrong? How do I read a table in from disk during
initialization/startup?
[0] https://www.bro.org/sphinx/frameworks/input.html
Hi Everyone,
I would like to do full email extraction (eml) to file from STMP traffic;
should this happen naturally with the new file extraction framework?
I found this exchange from a while back, but haven't found anything more
recent on the topic:
http://mailman.icsi.berkeley.edu/pipermail/bro/2014-July/007224.html
I'm currently using Bro 2.4 and a script pretty similar to this one for
file extraction:
https://github.com/Security-Onion-Solutions/securityonion-bro
-scripts/blob/master/file-extraction/extract.bro
It looks like I'm getting the message content and attachments, but
apparently not the raw email.
Any tips would be greatly appreciated!
Thanks very much,
VG
Hi All,
I run tcpdump live to capture the traffic into a file using "-w".
Then I run bro to read that file offline using "-r".
Both instances are running continuously. First it works fine but then bro
stop generating results although it keep running, this means bro didn't
continue reading from the file. Is it because bro -r is faster than the
live capturing?
How to let bro keep reading the file (this file is continuously whitening?
My bro version: 2.3 running on ubuntu platform.
Thanks
Hi there,
does bro provide some mechanism to find the packets that are related to (have caused) a given event or connection?
Background: I'd like to be able to export pcap files in some situations for specific events; in that context I'm still able to get to the connection object, but I'd like to be able to see the original data as well for further analysis with Wireshark.
One possibility would be to reconstruct filters from the event to filter the original trace retrospectively. But I'm wondering if there is a more direct way to identify / extract the relevant packets.
Thanks for your help,
Dirk
