10 Feb
2016
10 Feb
'16
1:55 p.m.
I was wondering if anyone can tell me why the sha256 hash functionality
isn't turned on by default for the files log.
I am working on something and needed to turn it on. I normally only use Bro
to process pcap files offline and have never used it on a live network.
Does it cause performance issues?
Thanks,
Shawn
11 Feb
11 Feb
7:40 a.m.
On Feb 10, 2016, at 4:55 PM, Shawn Homan
<shawn.homan(a)gmail.com> wrote:
I was wondering if anyone can tell me why the sha256 hash functionality isn't turned
on by default for the files log.
I am working on something and needed to turn it on. I normally only use Bro to process
pcap files offline and have never used it on a live network.
Does it cause performance issues?
When I was setting the default behavior a few years ago, I did some very weak testing and
noticed that if I had md5 and sha1 turned on, the performance impact was ~1%, but it
jumped up somewhere between 3-4% when I enabled SHA256. That measurement should be
revisited sometime soon though and perhaps even better measurements done to see if that
performance impact is still there.
Generally though, there is nothing in place which is stopping you from enabling SHA256
file hashes.
.Seth
--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/
3:39 p.m.
Thanks for the information. I have it turned on in my offline system, but
not sure how to measure performance.
On Thu, Feb 11, 2016 at 10:40 AM, Seth Hall <seth(a)icir.org> wrote:
On Feb 10, 2016, at 4:55 PM, Shawn Homan
<shawn.homan(a)gmail.com> wrote:
I was wondering if anyone can tell me why the sha256 hash functionality
isn't
turned on by default for the files log.
I am working on something and needed to turn it on. I normally only use
Bro to
process pcap files offline and have never used it on a live network.
Does it cause performance issues?
When I was setting the default behavior a few years ago, I did some very
weak testing and noticed that if I had md5 and sha1 turned on, the
performance impact was ~1%, but it jumped up somewhere between 3-4% when I
enabled SHA256. That measurement should be revisited sometime soon though
and perhaps even better measurements done to see if that performance impact
is still there.
Generally though, there is nothing in place which is stopping you from
enabling SHA256 file hashes.
.Seth
--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/
30 Dec
30 Dec
8 a.m.
I'm curious if anyone has this turned on at scale, on production systems? If so, can
you speak to the performance impacts Seth mentioned below?
Seth,
any thoughts if this would be the same with 2.5 as it was when you originally posted? I
didn't see anything specific about it in release notes, so would we be correct to
assume the SHA256 analyzer would probably perform the same as what you saw back in Feb
16?
Thanks,
ryan
Sent from Outlook<http://aka.ms/weboutlook>
________________________________
From: bro-bounces(a)bro.org <bro-bounces(a)bro.org> on behalf of Shawn Homan
<shawn.homan(a)gmail.com>
Sent: Thursday, February 11, 2016 5:39 PM
To: Seth Hall
Cc: bro(a)bro.org
Subject: Re: [Bro] SHA256 Hash File Analyzer
Thanks for the information. I have it turned on in my offline system, but not sure how to
measure performance.
On Thu, Feb 11, 2016 at 10:40 AM, Seth Hall
<seth@icir.org<mailto:seth@icir.org>> wrote:
On Feb 10, 2016, at 4:55 PM, Shawn Homan
<shawn.homan@gmail.com<mailto:shawn.homan@gmail.com>> wrote:
I was wondering if anyone can tell me why the sha256 hash functionality isn't turned
on by default for the files log.
I am working on something and needed to turn it on. I normally only use Bro to process
pcap files offline and have never used it on a live network.
Does it cause performance issues?
When I was setting the default behavior a few years ago, I did some very weak testing and
noticed that if I had md5 and sha1 turned on, the performance impact was ~1%, but it
jumped up somewhere between 3-4% when I enabled SHA256. That measurement should be
revisited sometime soon though and perhaps even better measurements done to see if that
performance impact is still there.
Generally though, there is nothing in place which is stopping you from enabling SHA256
file hashes.
.Seth
--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/
8:27 a.m.
On Dec 30, 2016, at 11:00 AM, Ryan Stillions <ryanstillions(a)hotmail.com> wrote:
I'm curious if anyone has this turned on at scale, on production systems? If so,
can you speak to the performance impacts Seth mentioned below?
Seth,
any thoughts if this would be the same with 2.5 as it was when you originally posted? I
didn't see anything specific about it in release notes, so would we be correct to
assume the SHA256 analyzer would probably perform the same as what you saw back in Feb 16?
The analyzer really just delegates to openssl to do all the hashing, so you should be able
to use openssl to gauge the performance impact:
$ openssl speed md5 sha1 sha256
Doing md5 for 3s on 16 size blocks: 6879766 md5's in 3.00s
Doing md5 for 3s on 64 size blocks: 5066897 md5's in 3.00s
Doing md5 for 3s on 256 size blocks: 2814019 md5's in 3.00s
Doing md5 for 3s on 1024 size blocks: 1016906 md5's in 3.00s
Doing md5 for 3s on 8192 size blocks: 147949 md5's in 3.00s
Doing sha1 for 3s on 16 size blocks: 7763902 sha1's in 3.00s
Doing sha1 for 3s on 64 size blocks: 5420584 sha1's in 3.00s
Doing sha1 for 3s on 256 size blocks: 2965390 sha1's in 3.00s
Doing sha1 for 3s on 1024 size blocks: 1054003 sha1's in 3.00s
Doing sha1 for 3s on 8192 size blocks: 147866 sha1's in 3.00s
Doing sha256 for 3s on 16 size blocks: 4896135 sha256's in 3.00s
Doing sha256 for 3s on 64 size blocks: 2682706 sha256's in 3.00s
Doing sha256 for 3s on 256 size blocks: 1131865 sha256's in 3.00s
Doing sha256 for 3s on 1024 size blocks: 342980 sha256's in 3.00s
Doing sha256 for 3s on 8192 size blocks: 45549 sha256's in 3.00s
OpenSSL 1.0.1e-fips 11 Feb 2013
built on: Tue Sep 27 13:37:25 UTC 2016
options:bn(64,64) md2(int) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int)
blowfish(idx)
compiler: gcc -fPIC -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN
-DHAVE_DLFCN_H -DKRB5_MIT -m64 -DL_ENDIAN -DTERMIO -Wall -O2 -g -pipe -Wall
-Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4
-grecord-gcc-switches -m64 -mtune=generic -Wa,--noexecstack -DPURIFY -DOPENSSL_IA32_SSE2
-DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM
-DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes
md5 36692.09k 108093.80k 240129.62k 347103.91k 403999.40k
sha1 41407.48k 115639.13k 253046.61k 359766.36k 403772.76k
sha256 26112.72k 57231.06k 96585.81k 117070.51k 124379.14k
On a different machine with a different distribution and newer CPUs I get
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes
md5 50302.28k 175259.63k 373751.13k 536014.85k 632668.98k
sha1 62768.79k 170994.71k 358746.20k 509927.77k 569868.29k
sha256 50775.24k 110530.33k 188262.14k 241865.05k 270240.43k
The 1024 byte block size and below would be the most relevant for bro. Unless you're
using jumbo frames bro shouldn't be doing much with blocks larger than 1500.
--
- Justin Azoff
3 Jan
3 Jan
2:05 p.m.
On Dec 30, 2016, at 11:00 AM, Ryan Stillions
<ryanstillions(a)hotmail.com> wrote:
any thoughts if this would be the same with 2.5 as it was when you originally posted? I
didn't see anything specific about it in release notes, so would we be correct to
assume the SHA256 analyzer would probably perform the same as what you saw back in Feb 16?
I would expect to still see a similar performance hit from enabling SHA256, but I
don't really know. Someone needs to test it. Justin's point about the
performance of OpenSSL is most of the picture, but there is still some additional overhead
due to having another analyzer attached to a file so I wouldn't go totally off of the
openssl benchmark testing.
.Seth
--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/
1966
days inactive
2294
days old
5 comments
4 participants
参加者 (4)
-
Azoff, Justin S -
Ryan Stillions -
Seth Hall -
Shawn Homan