I follow the 37 Signals blog regularly. A great bunch of very smart people doing good work.
Noah at 37 Signals ate crow. He used to think Big Data was a bunch of bull for companies that don’t have the massive sets of data like Facebook, Google et al.
He changed his perspective on Big Data for small companies by testing platforms and databases for accessing large sets of data for these smaller companies. Here are a few quotes to summarize the post:
“The typical analytical workload with this data is a few gigabytes or tens of gigabytes – sometimes big enough to fit in RAM”
“None of this is an insurmountable problem, and it’s all pretty typical of “medium” data”
“The challenges of this medium amount of data are, however, enough that I occasionally wish for a better solution”
He then finds through his not-totally-scientific-testing that Impala from Cloudera provided good results.
“I’m thrilled to have been proven wrong about the utility of these technologies for businesses at sub-Facebook scale, and I’m more than happy to eat crow in this case.”
It’s a good quick read, check it out.