Hello,
I would like to bring your attention to one strange and unexpected problem
I faced
while trying to include user agent parser for bro I have developed (plugin:
https://github.com/vitalyrepin/uap-bro ) into the list of uap
implementations https://github.com/ua-parser
The req
--
WBR & WBW, Vitaly
I'm working on some scripts that use sets and vectors, sometimes together,
and am finding it clunky that Bro doesn't offer much in the way of operators
for this. To that end, I'm thinking of implementing some along the following
lines, where values starting with 's' are sets, 'v' are vectors, and 'e'
are type-compatible elements:
s1 + s2 Set union (for sets of the same type, of course)
s1 || s2 Set union
s1 && s2 Set intersection
s1 - s2 Set difference
s += e Add the element 'e' to set 's'
(same as the current "add s[e]")
s -= e Remove the 'e' element from 's', if present
(same as the current "delete s[e]")
s1 += s2 Same as "s1 = s1 + s2"
s1 -= s2 Same as "s1 = s1 - s2"
v += e Append the element 'e' to the vector 'v'
v += s Append the elements of 's' to the vector 'v',
with the order not being defined
s += v Add the elements of 'v' to 's'
s -= v Remove the elements of 'v' from 's', if present
These strike me as pretty straightforward, but please chime in if you
have comments!
Vern
The latest version of the new Broker-ized cluster/communication system
for Bro in 'topic/actor-system' branch is wrapping up and, in my
opinion, ready to be merged into Bro's 'master' branch.
However, since it's such a big change, I'd like a last round of feedback
before merging. If you want to test, the build process should now be as
simple as:
$ git clone --recursive --branch=topic/actor-system git://git.bro.org/bro
$ cd bro && ./configure && make
Configuring BroControl is not any different from before.
If you had custom scripts, they may require porting. There's a guide
and examples for that at [1] and [2] (hyperlinks in those docs will
render more nicely when it's up on bro.org).
Though, for this round of testing, I'd be most interested just in any
general stability issues or major feature breakages from a vanilla Bro
installation. Mild performance issues, minor bugs, or other issues w/
porting custom scripts are things I think we can iron out even after
merging into 'master'.
- Jon
[1]
https://github.com/bro/bro/blob/topic/actor-system/doc/frameworks/broker.rst
[2] https://github.com/bro/bro/tree/topic/actor-system/doc/frameworks/broker
I have a couple of branches that, for whatever reason, aren't headed for a
merge request into master. The branches were left around for reference, in
case someone wanted to pick up and continue the work. However, this too now
looks very unlikely, as some of these are several years old, and of
questionable value at this point.
I'm torn between deleting the branches, in an effort to not clog up git
with unneeded branches, and leaving them around or perhaps archiving them
somewhere, in order to not completely lose the work in case it's of value
to someone down the road.
I'm curious if anyone has thoughts on the best way to proceed.
--Vlad
In implementing bitwise operations for counts (now pretty much done!),
I found that Bro's internals actually support "||" and "&&" operators for
patterns:
p1 || p2 returns a pattern that matches either p1 or p2
p1 && p2 returns a pattern that matches p1 followed by p2
(Note that this second is not obvious. One might instead expect it to
return a pattern that matches only if both p1 and p2 match, where the two
could overlap.)
This doesn't appear to be documented anywhere. I discussed it with Robin
and we both agree that this seems weird and likely not used anywhere.
Therefore we're thinking we should deprecate it and eventually remove it.
Anyone think otherwise?
Vern
Take a look
http://try.bro.org/#/trybro/saved/228251
In this instance, the client sent the Key Exchange Init packet first and
the server sent it's Key Exchange Init packet second. The client packet
actually contained the list of encryption algorithms seen here, but it's
being printed out when I specify is_server == T, it should be printed when
is_server == F, right?
It also looks like ssh_capabilities is only capturing details within the
first Key Exchange Init packet, whether that be the one from the server or
the client, and ignoring the second one.
So sometimes the server will send the KEI first, Bro captures that, then
the client sends it's KEI and it looks like Bro ignores it. Same thing
happens when it's the other way around. I want to be able to look at the
details in both KEI's.
Does this make sense? Does anyone know how this can be fixed or maybe I'm
doing something wrong here?
Thanks!
John
While testing other stuff, I realized that if input-framework cannot find a file
its now generating reporter_warning event instead of reporter_error ?
Did "error" changed to "warning" for some reason ? Wasn't previously this a
error condition ?
0.000000 Reporter::WARNING /DATA/feeds/BRO-feeds/WIRED.blocknet.2/Input::READER_ASCII: Init: cannot open /DATA/feeds/BRO-feeds/WIRED.blocknet.2 (empty)
1) Also in what situations input-framework would generate WARNINGS vs ERRORS ?
2) Does warnings means READER_ASCII will try to read file again after some time
or does it just gives up and waits for me to tap into reporter_warning event to
handle the situation ?
Thanks,
Aashish
Hi,
I will be updating the Bro git/webserver. There might be intermittent
outages in the next few hours (mostly of git - the webserver should
automatically fail over).
Johanna
I have a aggregation policy where I am trying to keep counts of number of
connections an IP made in a cluster setup.
For now, I am using table on workers and manager and using expire_func to
trigger worker2manager and manager2worker events.
All works great until tables grow to > 1 million after which expire_functions
start clogging on manager and slowing down.
Example of Timer from prof.log on manager:
1523636760.591416 Timers: current=57509 max=68053 mem=4942K lag=0.44s
1523636943.983521 Timers: current=54653 max=68053 mem=4696K lag=168.39s
1523638289.808519 Timers: current=49623 max=68053 mem=4264K lag=1330.82s
1523638364.873338 Timers: current=48441 max=68053 mem=4162K lag=60.06s
1523638380.344700 Timers: current=50841 max=68053 mem=4369K lag=0.47s
So Instead of using &expire_func, I can probably try schedule {} ; but I am not
sure how scheduling events are any different internally then scheduling
expire_funcs ?
I'd like to think/guess that scheduling events is probably less taxing. but
wanted to check with the greater group on thoughts - esp insights into their
internal processing queues.
Thanks,
Aashish
So I am seeing some weird stuff in my sample pcap of scanners. May be too
obvious and I am just not seeing why/how of it.
Here is the issue : ( I have time in human format for easier read):
SO I just pick one session from conn.log and this is the connection in
question: (there are many more like this):
$ fgrep CspAa42NoEGEaXK4ci conn.log | cf
Apr 12 05:37:42 CspAa42NoEGEaXK4ci 191.254.157.138 45107 128.3.97.204 23 tcp - - - - S0 F T 0 S 1 40 0 0 -
Now as part of debugging I have dumped network_time for various events which
process this connection:
Apr 12 05:37:42 new_connection CspAa42NoEGEaXK4ci
Apr 12 06:13:48 connection_attempt CspAa42NoEGEaXK4ci
Apr 12 06:13:48 connection_state_remove CspAa42NoEGEaXK4ci
Now my understanding is there are various timers involved upon whose expirations
bro infers events such as connection_attempt, connection_reset etc etc. Timers
such as tcp_attempt_delay, tcp_SYN_timeout, tcp_close_delay amongst others. But
all these timers are generally 5 seconds.
Q. Why would connection_attempt event kick in after 36 minutes and 6 seconds ? (
06:13:48 - 05:37:42 ) ?
I have a pcap to share if anyone is interested and replicate on their end.
Aashish