I’m happy to announce the release of Bottlenose Nerve Center 2.0 today. Analyzing 3 billion messages an hour (72 billion messages a day), and doing real-time predictive analytics on nearly 300 million data points an hour, it’s a big step in real-time big data analytics.
Think about it for a moment: 3 billion messages is several times more data volume than the entire daily Twitter firehose, and we’re analyzing this much data every single hour, continuously. For one thing this level of real-time big data analytics cannot be done today with Hadoop — Hadoop is actually not capable of doing huge data aggregations this fast — so we’re using new technologies, like ElasticSearch (our team actually contributes to the ElasticSearch codebase) and Cassandra under the hood. We’re now analyzing in under a second what would take about an hour with Hadoop.
It’s an impressive technical accomplishment and I’m very proud of the incredibly talented engineering team that built this. I just want to give a shout out to my amazing co-founder Dominiek ter Heide and our product and engineering team for their work. This level of real-time analytics is truly game-changing.