On 14/03/2019 07:50, Mauro Palumbo wrote:
However, I noticed that this works fine if new fields
are added to the
intel data file, but NOT if some fields are removed (for example if an
ip address previously believed to be malicious is removed from the intel
file because it was later realized to be safe). At the script level in
the intel framework, intel data are stored into
global data_store: DataStore &redef;
and there are some functions for removing items from the record (
remove(item: Item, purge_indicator: bool), remove_meta_data(item: Item):
bool ). But I am not sure they are really called anywhere.
that's correct. One has to keep in mind, that the files you provide to
Zeek are just "input" for the data store. However, there are different
approaches to remove Intel data from the internal data store.
1. You can use item expiration to expire items. If you update the intel
file periodically, rereading items will reset the expiration timeout so
that items removed from the file will eventually expire while the others
are kept in the data store.
2. You could define which items to remove explicitly. Either by
processing a file of items to delete, introducing a new meta data field
indicating that an item should be removed or interfacing Zeek in any
other way. This approach would require writing some custom script but
shouldn't be too hard.
There is a blog post that provides some more details:
I hope that helps!