I have been following Big Data, Hadoop in particular, for the past four years and a lot has changed since then. At the time I was looking for the next big wave of technologies, especially on the back end of development as that was my forte at the time. After our initial introduction, the more I learned the more I felt Hadoop was the right tool to get the job done. Focusing upon the development aspects of job vs actual maintenance of the servers/cluster has always been a no-brainer for me. Now that Hadoop has been adopted by a wide variety of industries and has grown quite popular, it has become much more apparent that there are indeed areas where Hadoop is lacking. Security and real-time analytics are two of these concerns which both affect real-time processing and reporting, but fortunately there is a lot of current work happening to address these limitations. Consequently, there is a group of big data vendors who are investing a large number of resources as a remedy, bringing us newer tools such as Yarn, Tez, Storm, etc. Also, acquisitions of XA Secure by Hortonworks and Gazzang by Cloudera are contributing to the solving these issues around security.
Last week the LA R meetup featured 5 speakers each highlighting a few things they had found interesting at the useR! 2014 conference (which I had the pleasure of helping to organize this year in Los Angeles). This meetup proved to be interesting even to those of us who attended the conference (which was about half of our audience), simply because there were so many cool things to hear and enough parallel sessions at the conference that no one could possibly capture all of the action.
Last week, I got the opportunity to share the highlights from the eyeo festival at the DataVis LA meetup. The eyeo festival is a yearly event held in Minneapolis that draws together artists, technologists, designers, and many more people who work with art and computers. As I said in the meetup description, a lot of the things that caught my eye this year could be described better by the term “art” than straight “data visualization,” although there’s usually both at the festival. I also really liked that this year’s festival emphasized the history of computer art and data visualization. By bringing in Frieder Nake, Lillian Schwartz, and Roman Verostko, the organizers grounded us in the history that started in the 1960s.
Today I am happy to announce that we are expanding our partnership on several fronts with my friend Subash D’Souza, a leader of the Big Data community in LA.
During the past few years I have been organizing several data science meetups in Los Angeles (LA R, DataVis LA, Data Science/Machine Learning) and Subash has provided the same kind of leadership on the big data front (LA Hadoop, HBase, LA Big Data). While I think of big data as primarily technology and of data science as a blend of statistics and technology (and not only for large data sets), there is clearly a significant overlap between these two fields. Considering this overlap, we have already been collaborating for some time regarding these various meetups, e.g. organizing joint events or speaking at one another’s meetups.
We have a few pieces of swag that we are raffling off to the community at our upcoming meetup events. Our stash consists of 25 data-oriented books, primarily from O’Reilly plus 25 T-shirts from useR! 2014 conference and Revolution Analytics (see picture above). Did we get your attention?
I am excited to announce that we are starting a new meetup group - Python Data Science LA - to serve the needs of the LA data science community using Python!
At an earlier Data Science/Machine Learning meetup we conducted an informal survey on tools for Data Science which showed that a large majority of data scientists use R or Python, with R taking a slight lead. (I know, those attending the meetups bias the results towards open source tools.)
Data Science @ IRIS.TV
by Dr. Thomas Sullivan, Chief Data Scientist and Joel Spitalnik, VP Engineering
While video recommendations best serve the viewer experience when presented in real time, machine learning is a computationally intensive process that is often done on the back end. IRIS.TV’s utilizes a hybrid process to synchronize computational tasks for learning so that engineering and data science teams can shrink the latency between analyzing history and presentation of recommended videos to improve the experience.
