PCAP Tuning

PCAP is a specialized topology that is a Spout-only topology. Both Kafka topic consumption and HDFS writing is done within a spout to avoid the additional network hop required if using an additional bolt.

General Storm topology properties

```
topology.workers=16
topology.ackers.executors: 0
```

                                +__Spout and Bolt properties__
                                +```
                                +kafkaSpout
                                +    parallelism: 128
                                +    poll.timeout.ms=100
                                +    offset.commit.period.ms=30000
                                +    session.timeout.ms=39000
                                +    max.uncommitted.offsets=200000000
                                +    max.poll.interval.ms=10
                                +    max.poll.records=200000
                                +    receive.buffer.bytes=431072
                                +    max.partition.fetch.bytes=10000000
                                +    enable.auto.commit=false
                                +    setMaxUncommittedOffsets=20000000
                                +    setOffsetCommitPeriodMs=30000
                                +
                                +writerConfig
                                +    withNumPackets=1265625
                                +    withMaxTimeMS=0
                                +    withReplicationFactor=1
                                +    withSyncEvery=80000
                                +    withHDFSConfig
                                +        io.file.buffer.size=1000000
                                +        dfs.blocksize=1073741824
                                +```
                                +