Dean Wampler (author of
Programming Scala) gave a really interesting talk at the Lambda Jam conference on why
Copious Data is the Killer App for Functional Programming. He looks at the factors that make FP a great tool for dealing with "Big Data", and shows examples of how existing FP-based tools such as Scalding or Cascalog already make it much easier to work with Hadoop's map-reduce framework, which is after all based on the FP "flatmap" and "reduce" operations. More provocatively - and entertainingly - he describes Hadoop as the "EJB of our times" and sees a need for better alternatives to allow scalable "big data" processing without all the ceremony and inflexibility of Hadoop/Java and the limitations of the current map-reduce model, and he presents FP as a better paradigm on which to base the next generation of "big data" processing.
Anyway, it's interesting stuff, and speaking as an ex-database application developer keen to move into FP and "big data", I find it very encouraging that smarter people than me are also spotting the ways in which FP and big data seem like a natural match for each other.
