Jon Siwek created BIT-1100:
------------------------------
Summary: topic/jsiwek/broccoli-vectors
Key: BIT-1100
URL: https://bro-tracker.atlassian.net/browse/BIT-1100
Project: Bro Issue Tracker
Issue Type: New Feature
Components: Bro, Broccoli
Affects Versions: git/master
Reporter: Jon Siwek
Fix For: 2.3
This branch is in the bro and broccoli repos and adds support for broccoli clients to receive events that have arguments w/ vector values.
Sending events that have arguments w/ vector values is still unsupported. (Broccoli generally seems to be limited in the complexity of types it can create compared to Bro).
--
This message was sent by Atlassian JIRA
(v6.2-OD-03#6206)
[ https://bro-tracker.atlassian.net/browse/BIT-1100?page=com.atlassian.jira.p… ]
Jon Siwek updated BIT-1100:
---------------------------
Status: Merge Request (was: Open)
> topic/jsiwek/broccoli-vectors
> -----------------------------
>
> Key: BIT-1100
> URL: https://bro-tracker.atlassian.net/browse/BIT-1100
> Project: Bro Issue Tracker
> Issue Type: New Feature
> Components: Bro, Broccoli
> Affects Versions: git/master
> Reporter: Jon Siwek
> Fix For: 2.3
>
>
> This branch is in the bro and broccoli repos and adds support for broccoli clients to receive events that have arguments w/ vector values.
> Sending events that have arguments w/ vector values is still unsupported. (Broccoli generally seems to be limited in the complexity of types it can create compared to Bro).
--
This message was sent by Atlassian JIRA
(v6.2-OD-03#6206)
[ https://bro-tracker.atlassian.net/browse/BIT-1099?page=com.atlassian.jira.p… ]
grigorescu edited comment on BIT-1099 at 12/2/13 6:59 PM:
----------------------------------------------------------
Hi Tyler,
First off - is this actually causing a problem for you?
It's set this way to work with both SNF1 and SNF2. You're looking at SNF2, which doesn't define 0x100. SNF1 does:
{code}
# If incoming data is to be partitioned across rings via RSS, an
# alternative receive mode is to set the SNF_F_RX_PRIVATE=0x100 flag
# which reduces the amount of shared SNF references at the cost of
# an additional copy. This approach may provide better capture behavior
# when the RSS distribution is unbalanced or more generally, when the
# consumption rate of each process varies enough to cause large amounts
# of packet drops. Here we set the SNF_F_RX_PRIVATE=0x100 flag with
# SNF_F_PSHARED=0x1 flag to allow process-sharing.
# export SNF_FLAGS=0x101
{code}
Setting this to 0x101 shouldn't affect SNF2 drivers; it will be like 0x1 to them.
was (Author: grigorescu):
Hi Tyler,
First off - is this actually causing a problem for you?
It's set this way to work with both SNF1 and SNF2. You're looking at SNF2, which doesn't define 0x100. SNF1 does:
{quote}
# If incoming data is to be partitioned across rings via RSS, an
# alternative receive mode is to set the SNF_F_RX_PRIVATE=0x100 flag
# which reduces the amount of shared SNF references at the cost of
# an additional copy. This approach may provide better capture behavior
# when the RSS distribution is unbalanced or more generally, when the
# consumption rate of each process varies enough to cause large amounts
# of packet drops. Here we set the SNF_F_RX_PRIVATE=0x100 flag with
# SNF_F_PSHARED=0x1 flag to allow process-sharing.
# export SNF_FLAGS=0x101
{quote}
Setting this to 0x101 shouldn't affect SNF2 drivers; it will be like 0x1 to them.
> ./build/bro-git-20131129/lib/broctl/plugins/lb_myricom.py
> ---------------------------------------------------------
>
> Key: BIT-1099
> URL: https://bro-tracker.atlassian.net/browse/BIT-1099
> Project: Bro Issue Tracker
> Issue Type: Problem
> Components: BroControl
> Affects Versions: git/master
> Environment: RHEL6
> Myricom 10G
> Reporter: tyler.schoenke
> Labels: Myricom, RHEL6
>
> Hi All,
> First, unrelated, thanks for fixing whatever was wrong with PktSrc.cc:222. I was core dumping on that with the Myricom 10G until I upgraded to the latest git version of Bro.
> I had been running with my Myricom environment variables configured in .bash_profile, but decided to move them to etc/node.cfg because that feature is now available in Bro 2.2.
> When I tried to start the workers, they failed and I got the message to run diag. When I ran diag, I noticed it showed the following.
> 28366 snf.0.-1 P (environ) SNF_DATARING_SIZE = 2147483648 (0x80000000) (2048.0 MiB)
> 28366 snf.0.-1 P (default) SNF_DESCRING_SIZE = 536870912 (0x20000000) (512.0 MiB)
> 28366 snf.0.-1 P (userset) SNF_FLAGS = 257 (0x101)
> 28366 snf.0.-1 P (environ) SNF_DEBUG_MASK = 3 (0x3)
> 28366 snf.0.-1 P (default) SNF_DEBUG_FILENAME = stderr
> Everything looks great except for SNF_FLAGS. Myricom defaults it to 0x1, and that was working when I ran bro from the command line.
> I found it was getting set in ./build/bro-git-20131129/lib/broctl/plugins/lb_myricom.py
> According to /opt/snf/include/snf.h, the only valid values for SNF_FLAGS are 0x1, 0x2, and 0x300. No possible combination of those will add up to 0x101. I'm guessing that is a typo. If not, please explain why it is defaulted to that, otherwise you may want to default to 0x1.
> Thanks,
> Tyler
--
This message was sent by Atlassian JIRA
(v6.2-OD-03#6206)
[ https://bro-tracker.atlassian.net/browse/BIT-1099?page=com.atlassian.jira.p… ]
grigorescu edited comment on BIT-1099 at 12/2/13 6:59 PM:
----------------------------------------------------------
Hi Tyler,
First off - is this actually causing a problem for you?
It's set this way to work with both SNF1 and SNF2. You're looking at SNF2, which doesn't define 0x100. SNF1 does:
{quote}
# If incoming data is to be partitioned across rings via RSS, an
# alternative receive mode is to set the SNF_F_RX_PRIVATE=0x100 flag
# which reduces the amount of shared SNF references at the cost of
# an additional copy. This approach may provide better capture behavior
# when the RSS distribution is unbalanced or more generally, when the
# consumption rate of each process varies enough to cause large amounts
# of packet drops. Here we set the SNF_F_RX_PRIVATE=0x100 flag with
# SNF_F_PSHARED=0x1 flag to allow process-sharing.
# export SNF_FLAGS=0x101
{quote}
Setting this to 0x101 shouldn't affect SNF2 drivers; it will be like 0x1 to them.
was (Author: grigorescu):
Hi Tyler,
First off - is this actually causing a problem for you?
It's set this way to work with both SNF1 and SNF2. You're looking at SNF2, which doesn't define 0x100. SNF1 does:
# If incoming data is to be partitioned across rings via RSS, an
# alternative receive mode is to set the SNF_F_RX_PRIVATE=0x100 flag
# which reduces the amount of shared SNF references at the cost of
# an additional copy. This approach may provide better capture behavior
# when the RSS distribution is unbalanced or more generally, when the
# consumption rate of each process varies enough to cause large amounts
# of packet drops. Here we set the SNF_F_RX_PRIVATE=0x100 flag with
# SNF_F_PSHARED=0x1 flag to allow process-sharing.
# export SNF_FLAGS=0x101
Setting this to 0x101 shouldn't affect SNF2 drivers; it will be like 0x1 to them.
> ./build/bro-git-20131129/lib/broctl/plugins/lb_myricom.py
> ---------------------------------------------------------
>
> Key: BIT-1099
> URL: https://bro-tracker.atlassian.net/browse/BIT-1099
> Project: Bro Issue Tracker
> Issue Type: Problem
> Components: BroControl
> Affects Versions: git/master
> Environment: RHEL6
> Myricom 10G
> Reporter: tyler.schoenke
> Labels: Myricom, RHEL6
>
> Hi All,
> First, unrelated, thanks for fixing whatever was wrong with PktSrc.cc:222. I was core dumping on that with the Myricom 10G until I upgraded to the latest git version of Bro.
> I had been running with my Myricom environment variables configured in .bash_profile, but decided to move them to etc/node.cfg because that feature is now available in Bro 2.2.
> When I tried to start the workers, they failed and I got the message to run diag. When I ran diag, I noticed it showed the following.
> 28366 snf.0.-1 P (environ) SNF_DATARING_SIZE = 2147483648 (0x80000000) (2048.0 MiB)
> 28366 snf.0.-1 P (default) SNF_DESCRING_SIZE = 536870912 (0x20000000) (512.0 MiB)
> 28366 snf.0.-1 P (userset) SNF_FLAGS = 257 (0x101)
> 28366 snf.0.-1 P (environ) SNF_DEBUG_MASK = 3 (0x3)
> 28366 snf.0.-1 P (default) SNF_DEBUG_FILENAME = stderr
> Everything looks great except for SNF_FLAGS. Myricom defaults it to 0x1, and that was working when I ran bro from the command line.
> I found it was getting set in ./build/bro-git-20131129/lib/broctl/plugins/lb_myricom.py
> According to /opt/snf/include/snf.h, the only valid values for SNF_FLAGS are 0x1, 0x2, and 0x300. No possible combination of those will add up to 0x101. I'm guessing that is a typo. If not, please explain why it is defaulted to that, otherwise you may want to default to 0x1.
> Thanks,
> Tyler
--
This message was sent by Atlassian JIRA
(v6.2-OD-03#6206)
[ https://bro-tracker.atlassian.net/browse/BIT-1099?page=com.atlassian.jira.p… ]
grigorescu commented on BIT-1099:
---------------------------------
Hi Tyler,
First off - is this actually causing a problem for you?
It's set this way to work with both SNF1 and SNF2. You're looking at SNF2, which doesn't define 0x100. SNF1 does:
# If incoming data is to be partitioned across rings via RSS, an
# alternative receive mode is to set the SNF_F_RX_PRIVATE=0x100 flag
# which reduces the amount of shared SNF references at the cost of
# an additional copy. This approach may provide better capture behavior
# when the RSS distribution is unbalanced or more generally, when the
# consumption rate of each process varies enough to cause large amounts
# of packet drops. Here we set the SNF_F_RX_PRIVATE=0x100 flag with
# SNF_F_PSHARED=0x1 flag to allow process-sharing.
# export SNF_FLAGS=0x101
Setting this to 0x101 shouldn't affect SNF2 drivers; it will be like 0x1 to them.
> ./build/bro-git-20131129/lib/broctl/plugins/lb_myricom.py
> ---------------------------------------------------------
>
> Key: BIT-1099
> URL: https://bro-tracker.atlassian.net/browse/BIT-1099
> Project: Bro Issue Tracker
> Issue Type: Problem
> Components: BroControl
> Affects Versions: git/master
> Environment: RHEL6
> Myricom 10G
> Reporter: tyler.schoenke
> Labels: Myricom, RHEL6
>
> Hi All,
> First, unrelated, thanks for fixing whatever was wrong with PktSrc.cc:222. I was core dumping on that with the Myricom 10G until I upgraded to the latest git version of Bro.
> I had been running with my Myricom environment variables configured in .bash_profile, but decided to move them to etc/node.cfg because that feature is now available in Bro 2.2.
> When I tried to start the workers, they failed and I got the message to run diag. When I ran diag, I noticed it showed the following.
> 28366 snf.0.-1 P (environ) SNF_DATARING_SIZE = 2147483648 (0x80000000) (2048.0 MiB)
> 28366 snf.0.-1 P (default) SNF_DESCRING_SIZE = 536870912 (0x20000000) (512.0 MiB)
> 28366 snf.0.-1 P (userset) SNF_FLAGS = 257 (0x101)
> 28366 snf.0.-1 P (environ) SNF_DEBUG_MASK = 3 (0x3)
> 28366 snf.0.-1 P (default) SNF_DEBUG_FILENAME = stderr
> Everything looks great except for SNF_FLAGS. Myricom defaults it to 0x1, and that was working when I ran bro from the command line.
> I found it was getting set in ./build/bro-git-20131129/lib/broctl/plugins/lb_myricom.py
> According to /opt/snf/include/snf.h, the only valid values for SNF_FLAGS are 0x1, 0x2, and 0x300. No possible combination of those will add up to 0x101. I'm guessing that is a typo. If not, please explain why it is defaulted to that, otherwise you may want to default to 0x1.
> Thanks,
> Tyler
--
This message was sent by Atlassian JIRA
(v6.2-OD-03#6206)
tyler.schoenke created TM-16:
--------------------------------
Summary: Index not working when traffic encapsulated in 802.1q trunk
Key: TM-16
URL: https://bro-tracker.atlassian.net/browse/TM-16
Project: Time Machine
Issue Type: Problem
Affects Versions: git/master
Environment: Ubuntu 10.04 , pf_ring
Reporter: tyler.schoenke
Hi All,
When I query the time machine index, I am not receiving any results.
I just restarted time machine, and checked one of the recent class files to see there is traffic for a particular IP address.
tcpdump -e -v -n -r class_all_1385406639.023206 "vlan and host 128.138.44.198"
It shows some traffic, example:
128.138.44.198.54014 > 74.125.225.209.443: Flags [.], cksum 0x8d2c (correct), seq 1283940799:1283940800, ack 615539104, win 16311, length 1
19:11:00.571731632 10:8c:cf:57:46:00 > 00:1d:09:6a:d9:a9, ethertype 802.1Q (0x8100), length 70: vlan 987, p 0, ethertype IPv4, (tos 0x0, ttl 56, id 17482, offset 0, flags [none], proto TCP (6), length 52)
When I telnet localhost 42042 and run the following command, I don't receive any results.
query to_file "128.138.44.198.pcap" index ip "128.138.44.198"
In the above tcpdump, you can see my traffic is 802.1Q trunked. I have to use the "vlan" BPF to extract it with tcpdump, and am wondering if the trunking is causing problems with indexing?
I tested the same version of time machine on non-trunked traffic, and the index works fine.
Let me know if you need any other configuration info.
Tyler
--
This message was sent by Atlassian JIRA
(v6.2-OD-03#6206)
tyler.schoenke created BIT-1099:
-----------------------------------
Summary: ./build/bro-git-20131129/lib/broctl/plugins/lb_myricom.py
Key: BIT-1099
URL: https://bro-tracker.atlassian.net/browse/BIT-1099
Project: Bro Issue Tracker
Issue Type: Problem
Components: BroControl
Affects Versions: git/master
Environment: RHEL6
Myricom 10G
Reporter: tyler.schoenke
Hi All,
First, unrelated, thanks for fixing whatever was wrong with PktSrc.cc:222. I was core dumping on that with the Myricom 10G until I upgraded to the latest git version of Bro.
I had been running with my Myricom environment variables configured in .bash_profile, but decided to move them to etc/node.cfg because that feature is now available in Bro 2.2.
When I tried to start the workers, they failed and I got the message to run diag. When I ran diag, I noticed it showed the following.
28366 snf.0.-1 P (environ) SNF_DATARING_SIZE = 2147483648 (0x80000000) (2048.0 MiB)
28366 snf.0.-1 P (default) SNF_DESCRING_SIZE = 536870912 (0x20000000) (512.0 MiB)
28366 snf.0.-1 P (userset) SNF_FLAGS = 257 (0x101)
28366 snf.0.-1 P (environ) SNF_DEBUG_MASK = 3 (0x3)
28366 snf.0.-1 P (default) SNF_DEBUG_FILENAME = stderr
Everything looks great except for SNF_FLAGS. Myricom defaults it to 0x1, and that was working when I ran bro from the command line.
I found it was getting set in ./build/bro-git-20131129/lib/broctl/plugins/lb_myricom.py
According to /opt/snf/include/snf.h, the only valid values for SNF_FLAGS are 0x1, 0x2, and 0x300. No possible combination of those will add up to 0x101. I'm guessing that is a typo. If not, please explain why it is defaulted to that, otherwise you may want to default to 0x1.
Thanks,
Tyler
--
This message was sent by Atlassian JIRA
(v6.2-OD-03#6206)