One of the best things I like in working at CARD.com is that I am not only crunching R code in 24/7, but I also have the chance to interact with and improve the related data infrastructure with some interesting technologies.
After joining the company in January, I soon realized that while Impala is a very powerful database for handling data that do not comfortably fit in MySQL, it’s still not as fast as one might expect when querying large amount of data from R. Sometimes I had to wait several minutes for a query to run! So I used this spare time to think about how to improve the workflow.
The LA Data Warehouse, Business Analytics and Analytics meetup had a fantastic first event with a great audience (full house), an awesome host (thanks Edmunds: Punnoose for taking the lead and the Team onsite and helping) and a great mix of 3 talks overviewing the field.
There comes a time in a software toolchain’s lifecycle where the focus shifts from developer productivity and preference to the ability to deploy solutions in production which coexist gracefully in the enterprise. Configuration management and dependency management are two of the first points of friction in this transition, as the free-wheeling methods preferred by developers are at odds with the structured methods and isolation desired by administrators, who serve as gatekeepers to enterprise deployment. R is currently at this very point in the transition, in the midst of transformation from a language and environment used primarily in academic research to its growing place in the enterprise. At useR! 2014 there were a series of presentations on how dependency management can be addressed within the R ecosystem, ranging from “provider” side options (e.g. freezing CRAN and private repositories) to the client side options, where a project maintains its own dependencies as part of its codebase.
(Please click here if the video above doesn’t load.)
The video above is from an event at useR! 2014 consisting of Yasmin Lucero’s presentation plus an incredible panel discussion, which is included here. The title of the panel was “R’s Place in the Production Environment”, and was moderated by Tareef Kawaf, President at RStudio. Five high-profile members of the R community were on this panel, spanning the gamut of the enterprise:
(Please click here if the video above doesn’t load.)
During useR! 2014, the LA R Meetup (run by Szilard Pafka) organized an event focused on R’s status in industry. This event took the form of a presentation by Yasmin Lucero, Senior Statistician at AOL-Gravity, followed by a panel discussion. The video above captures the presentation, where Yasmin provides an excellent overview of the current status of R in the enterprise and breaks down the taxonomy of solutions for which R is commonly chosen as a development platform. From batch reporting and dashboards for internal deployment to full blown public facing analytics data products, R has certainly found a home in business. Yasmin’s talk concludes with specific, operational aspects of managing R in the enterprise, such as library and dependency management. Enjoy the video and don’t forget to tune in for Part 2 of this session where the panel discussed “R’s Place in the Production Environment.”
