It looks like ipsumdump might be changing the snaplen to 2000 bytes when it writes out the pcap file. I don't see an runtime option to change the snaplen.
Another tool you can try to merge those files is tcpslice from ftp://ftp.ee.lbl.gov/tcpslice.tar.gz. I have been able to preserve the snaplen using tcpslice.
tcpslice trace*.pcap -w - | bro -r - ...
SriOn Fri, Jun 18, 2010 at 2:09 AM, Veronica Estrada <email@example.com> wrote:
_______________________________________________Hi everyone,I am puzzled about the outcomes of using ipsumdump or BRO for processing multiple pcap files.I am using BRO to analyze anomalities in my 12 hours captured network traffic which was saved in 4 Gb pcap files. I want that BRO consider the cases when a connection may have been split in two or more files. I was using ipsumdump to solve this, but I found that some files have errors and cause ipsumdump to crush with this message:ToDump(bigPcap1.pcap): Inappropriate ioctl for deviceUsing the capinfo tool I detected that some of my files have packet size larger than normal (65535), so using tshark I cut the part of the file with problems. For example:capinfos: An error occurred after reading 3830659 packets from "trace2.pcap": File contains a record that's not valid.(pcap: File has 4065648712-byte packet, bigger than maximum of 65535)So I create a reduced version of trace2.pcap with tshark:/usr/sbin/tshark -c 3830659 -r trace2.pcap -w trace2-new.pcapThis solution seemed to work fine, all the ***-new.pcap have no errors while reading with capinfo or wireshark, but even so there are some that still cause problems for processing. For example:I processed the following files in 3 different ways:trace1.pcap, trace2-new.pcap, trace3.pcap (trace2.pcap was replaced because of the packet size error)FIRST TRY - using ipsumdump with collate option:ipsumdump --collate -w - trace* |bro -r - brolite myenvironment -f "tcp or udp or icmp" dpd_conn_logs=T dpd detect_protocols dyn_disable irc-bot proxy ftpOutput> 9.7 MB conn.log with 114861 lines (number of connections)SECOND TRY - using ipsumpdump without collate optionipsumdump --collate -w - trace* |bro -r - brolite myenvironment -f "tcp or udp or icmp" dpd_conn_logs=T dpd detect_protocols dyn_disable irc-bot proxy ftpOutput:19 Mbytes conn.log with 228922 lines with 950 repeated connectionsTHIRD TRY - without ipsumdump:/usr/local/bro/bin/bro -r trace1.pcap -r trace2-new.pcap -r trace3.pcap brolite todai -f "tcp or udp or icmp" dpd_conn_logs=T dpd detect-protocols dyn-disable irc-bot proxy ftp 2>bro-error3.logOutput:15 Mbytes conn.log with 169168 lines, connections are not repeatedCOMMENTS:pcap files has not overlap traffic (it was checked with trace-summary using first packet seen and last packet seen).I tried the ipsumdump with both collate and no collate option because when I used ipsumdump only (without bro), with collate option the resulted larger pcap file was a 7.9 GB file but without collate option the resulted file was 12.GB (trace1.pcap: 4 MB, trace2-new.pcap: 3.9GB, trace3.pcap: 4GB). Besides, while using ipsumpdump --collate alone, the progress bar showed something like this:66%****************** |8017MB ETAToDump(LargerTrace.pcap): Success100%****************************|12113MBBut the progress bar for ipsumdump without the collate option didn't split and reach the 100% 12113MB.If anyone can illuminate this matter, it will be a great help.Veronica
Bro mailing list