Hi all,
Recently I have some problems with Bro and PF_RING in cluster.
On my server, when I have less than 32 worker threads(rings),
everything is okay, but when I use worker threads more than 32, pf_ring
start to receive repeating data packets. For example, rings less than 32, I
send 400000 packets to server and pf_ring info in /proc shows there is
400000 packets in rings, but when rings greater than 32, I can get 800000
packets when 33 rings and 1200000 packets when 34 rings and so on.
I guess if there is some rules that a pf_ring or a bro cluster can only
support less than 32 rings or worker threads on a server or some other
reasons?
Any insight would be helpful.
Hi to all. :-)
When running:
bro -r my.pcap -b -C base/protocols/rdp
(ubuntu server 16.04 - bro version 2.4.1)
I'm getting an error:
Error: 1521789435.202907 internal error: file analyzer instantiation failed
I've found nothing on google or the docs.
How can I fix this?
Thanks,
Assaf.
Yes, I will do that.
On Fri, 27 Apr 2018 at 11:54 PM Vlad Grigorescu <vlad(a)es.net> wrote:
> Would you mind also sending your reply to the bro mailing list? That way
> other people can also help you, and it will provide information to anyone
> else that might run into this same issue in the future. Thanks.
>
> On Fri, Apr 27, 2018 at 2:49 PM, Seong Hyeok Seo <pulgrims(a)gmail.com>
> wrote:
>
>> we’re working on 2 machines. we set one worker on a single server and a
>> manager and a proxy on the other one.
>> and actually we emailed to a pfring developer and they replied this...
>> “it seems that Bro is not setting up a pf_ring cluster to distribute the
>> traffic across the instances (it should call pfring_set_cluster),
>> please write to the Bro mailing list as we are not maintaining that code
>> sorry.”
>>
>>
>> On Fri, 27 Apr 2018 at 11:33 PM Vlad Grigorescu <vlad(a)es.net> wrote:
>>
>>> Could you provide a bit more detail about your setup? Are the workers
>>> all running on a single server, or are they distributed across multiple
>>> servers?
>>>
>>> What I'm trying to determine is at what point the duplication is
>>> happening.
>>>
>>> On Fri, Apr 27, 2018 at 9:47 AM, Seong Hyeok Seo <pulgrims(a)gmail.com>
>>> wrote:
>>>
>>>> Hi, we're doing a job that collecting traffic by using Bro and PF_RING
>>>> , but we found that each Bro worker got the same full traffic stream.
>>>> We think the packet is duplicated as much as the process number that we
>>>> set in a config file(bro/etc/node.cfg)
>>>>
>>>> These are OS, Bro, PF_RING Ver. that we're using.
>>>>
>>>>
>>>> OS: CentOS 7.4.1708 (Core)
>>>> Bro: 2.5.3
>>>> PF RING: 7.1.0-1859
>>>>
>>>> we installed those things, referring this page,
>>>> https://www.bro.org/documentation/load-balancing.html
>>>> and node.cfg is like this
>>>> ------------------------------------------
>>>>
>>>> [manager]
>>>> type=manager
>>>> host=X.X.X.X
>>>>
>>>> [proxy-1]
>>>> type=proxy
>>>> host=X.X.X.X
>>>>
>>>> [worker-1]
>>>> type=worker
>>>> host=X.X.X.X
>>>> interface=eth0
>>>> lb_method=pf_ring
>>>> lb_procs=8
>>>> --------------------------------------------------
>>>>
>>>> please, help us to fix this and thank you in advance.
>>>>
>>>> Sincerely,
>>>> Seonghyoek
>>>>
>>>>
>>>> _______________________________________________
>>>> Bro mailing list
>>>> bro(a)bro-ids.org
>>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>>>
>>>
>>>
>
Hi,
Sorry if my questions have already been answered but it would be really
helpful if anyone can provide information on the following.
1. Does bro capture_loss indicate that packets that are mirrored using a
switch's SPAN/TAP port to a server running bro, drop packets in the
mirroring process somewhere upstream?
In our particular setting, we are seeing zero packet drops reported by
"broctl netstats" but more than 40% packet losses in capture_loss. Does
that imply that the server running bro is not dropping any packets but that
packets are being dropped upstream? Bidirectional traffic is sent to the
server running bro using SPAN ports.
2. Is there a document that explains in detail how capture loss is
computed?
It says "Reported loss is computed in terms of the number of “gap events”
(ACKs for a sequence number that’s above a gap)."
What exactly is a gap event and how is the function call "get_gap_stats()"
defined? The code in "capture-loss.bro" does not explain how acks and gaps
can be used to estimate capture loss. Any detailed documentation would be
useful.
Thanks and regards,
Sourav Maji
We are looking to set up a proper ES cluster and dumping bro logs into it
via logstash. The thought is to have 6 ES nodes (2 dedicated master, 4 data
nodes). If we are dumping 15 TB of data into the cluster a year (possibly
as high as 20 or 25TB) from Bro, is 4 data nodes sufficient? The boxen will
only have 64 gigs of ram (30 for java heap, 34 for system use) and probably
16 discrete cores. I have a feeling that this is horribly insufficient for
a data cluster of that size.
1 year retention, 1x replication.
On Fri, Apr 27, 2018 at 10:39 AM, Kinkead, Tanner <
Tanner.Kinkead(a)franklintempleton.com> wrote:
> How long do you expect to retain logs? Are you using replica shards?
>
>
>
> *From:* bro-bounces(a)bro.org [mailto:bro-bounces@bro.org] *On Behalf Of *erik
> clark
> *Sent:* Friday, April 27, 2018 5:48 AM
> *To:* Bro-IDS <bro(a)bro.org>
> *Subject:* Re: [Bro] ES cluster and logstash
>
>
>
> We are looking to set up a proper ES cluster and dumping bro logs into it
> via logstash. The thought is to have 6 ES nodes (2 dedicated master, 4 data
> nodes). If we are dumping 15 TB of data into the cluster a year (possibly
> as high as 20 or 25TB) from Bro, is 4 data nodes sufficient? The boxen will
> only have 64 gigs of ram (30 for java heap, 34 for system use) and probably
> 16 discrete cores. I have a feeling that this is horribly insufficient for
> a data cluster of that size.
>
>
>
> Notice: All email and instant messages (including attachments) sent to
> or from Franklin Templeton Investments (FTI) personnel may be retained,
> monitored and/or reviewed by FTI and its agents, or authorized
> law enforcement personnel, without further notice or consent.
> .
>
>
Hi, we're doing a job that collecting traffic by using Bro and PF_RING
, but we found that each Bro worker got the same full traffic stream.
We think the packet is duplicated as much as the process number that we set
in a config file(bro/etc/node.cfg)
These are OS, Bro, PF_RING Ver. that we're using.
OS: CentOS 7.4.1708 (Core)
Bro: 2.5.3
PF RING: 7.1.0-1859
we installed those things, referring this page, https://www.bro.org/
documentation/load-balancing.html
and node.cfg is like this
------------------------------------------
[manager]
type=manager
host=X.X.X.X
[proxy-1]
type=proxy
host=X.X.X.X
[worker-1]
type=worker
host=X.X.X.X
interface=eth0
lb_method=pf_ring
lb_procs=8
--------------------------------------------------
please, help us to fix this and thank you in advance.
Sincerely,
Seonghyoek
