R

At this meetup we’re fortunate to have RStudio’s lead developer Joe Cheng flying in from Seattle to talk about Shiny. Shiny makes it possible to develop interactive web applications by writing only a handful of R code. It lowers the barriers to constructing interactive dashboards that you can use to better explain your results to your boss and/or co-workers, or it can help you productionalize your analysis so that others in your company can make use of it regularly without the need to continually bother you.

Last week the LA R meetup featured another round of 5 speakers each highlighting a few things they had found interesting at the useR! 2014 conference. This was the second such event after the first one in September - probably because useR! was awesome enough that it inspired 10 volunteers to stand up on stage and talk more about it.

Part 2 of the dplyr tutorial is now available! Hadley will be covering Grouped Mutate/Filter, Joins, Do, and Databases. (You can find Part 1 of the tutorial here.) For our viewers who have just discovered the wonders of R, keep in mind that this tutorial was filmed in early July of 2014. As R packages naturally update over time (well, depending on the programmers!), please make sure to check Hadley’s Githubfor the latest changes to the dplyr package.

At this meetup, five more LA R meetup members will present short talks of what they have learned at useR! 2014. This is the 2nd round of talks about useR! since we had twice as many volunteers to speak as we could accommodate in one meetup (indeed, useR! was that awesome). The first round of talks at our September meetup is summarized here (with slides).

JJ Allaire has courageously bet, time and time again, on the cutting edge of technologies. In doing so, he has not only benefited from the fertile rains but has often times himself been the rainmaker. He has, to a great degree, democratized development on the web by bringing the power to create interactive web applications to a wider audience than those select few who had access to both a shell account and Unix knowledge. ColdFusion was a platform by the web and for the web, and its adoption revolutionized programming ‘for the web’ as its own worthy pursuit. This story also resolutely validates the idea that by creating tools for developers at the beginning of a time of revolutionary change in technology, one could not only validate the tools but also help stage the unfolding narrative as one with many participants.

Hadley Wickham (perhaps you’ve heard of his work) presented a 2 hour workshop on dplyr at this year’s useR! conference at UCLA. This tutorial was definitely a highlight of the week-long conference for me, and working on this tutorial video has also made me very appreciative of how versatile the dplyr package can be. It clearly is the chef’s knife of data science tools.

Hadley Wickham is hard at work, releasing packages which leverage the expressive power of R to keep easy things intuitive, and to make hard things possible. Having already developed a number of packages which address other steps in the data processing and analysis pipeline, he chose to address the process by which R developers ingest, manipulate, and transform data. This package, dplyr, is a narrowly focused library, designed to provide 90% of the functionality required in most data munging tasks. It may not solve every possible problem in data munging, but dplyr’s built in primitives will reduce your need to drop to “base R” for most common tasks. It does so by leveraging the magritr package’s “pipe” operator, and is inspired by dataflow programming constructs. It is fast, it is powerful, and it has been the single largest productivity boost that many of us here at DataScience.LA have adopted in 2014. In this video, Hadley provides a very quick introduction to the dplyr package, describing his philosophy behind the package’s design while providing a number of illustrative examples as to the flexibility, expressiveness, and power of this tool.

Joe Cheng is a software engineer. Unfortunately the term gets thrown around pretty lightly, so I’d like to make sure that I convey the appropriate amount of gravitas when I refer to him as such. Joe Cheng is a *great* Software Engineer. He has worked for some of the largest technology companies on the planet, building cutting edge tools which empowered the web browser as a full blown platform instead of a historical curiosity. Joe is one of those software engineers who saw a future with the web as a platform for modern software, then worked hard to make it a reality. When the time came to build the next generation of data science IDEs, an IDE targeting the web as a platform, RStudio called Joe.

There comes a time in a software toolchain’s lifecycle where the focus shifts from developer productivity and preference to the ability to deploy solutions in production which coexist gracefully in the enterprise. Configuration management and dependency management are two of the first points of friction in this transition, as the free-wheeling methods preferred by developers are at odds with the structured methods and isolation desired by administrators, who serve as gatekeepers to enterprise deployment. R is currently at this very point in the transition, in the midst of transformation from a language and environment used primarily in academic research to its growing place in the enterprise. At useR! 2014 there were a series of presentations on how dependency management can be addressed within the R ecosystem, ranging from “provider” side options (e.g. freezing CRAN and private repositories) to the client side options, where a project maintains its own dependencies as part of its codebase.

This is a re-publication of a blog post from a blog I created not long before we got the idea to start DataScience.LA, and in many aspects it is the genesis of DSLA (the blog’s name, the people getting together - thanks Eduardo, Amelia and Leigh). It was also the spark that lead to the creation of the Python Data Science LA meetup.

First things first, Dirk Eddelbuettel was recently named ordinary. This seems contradictory, since Dirk is a known HPC expert, an organizer of the R in Finance conference, the creator of Rcpp, and a Debian contributor. These are only a few of the many accolades bestowed upon Dirk without even a hint of puffery. And yet, Dirk Eddelbuettel is considered ordinary. What makes Dirk ordinary? It should be mentioned that to the Vienna-based nonprofit that provides R’s leadership, the R Foundation for Statistical Computing, ‘ordinary’ means something quite different to the lay person. Ordinary members provide guidance and direction for the R Project for Statistical Computing. It’s hard to imagine someone more qualified than Dirk for this task.

At useR! 2014 Ramnath Vaidyanathan gave a tutorial and a presentation on one of his current project, rCharts. This R package allows you to create, customize and share interactive visualizations straight from R. rCharts leverages several existing javascript visualization libraries such as polychart, MorrisJS, NVD3, xCharts, HighCharts, leaflet, and Rickshaw, providing R users a fantastic array of new features and visualization styles.

At the useR! 2014 conference, without a doubt one of the overriding themes was R’s history, legacy, and future as an interface into the “best of the best” algorithms which were available. Romain Francois’ package, Rcpp11, is very much the spirit and the letter of this ethos and telos of R as an interface. This package is a reimplementation of Dirk Eddelbuettel’s Rcpp package with an emphasis on C++ 11’s features. Whether it’s an R user hitting performance limitations, or someone trying to integrate a C/C++ library into their analysis, Romain’s tools are there to help.

Martin Maechler is a member of R-Core. This distinction puts him in the very apex of understanding of the R programming language and environment. He is a professor of statistics with wide and varied interests and a leading expert in statistical computing environments. He is also a prolific programmer, evidenced by his maintenance of the officially recommended cluster package for unsupervised clustering and analytics, the lead developer of Emacs Speaks Statistics, and co-maintainer of the Matrix package… plus too many other contributions to list them all here.

Last week the LA R meetup featured 5 speakers each highlighting a few things they had found interesting at the useR! 2014 conference (which I had the pleasure of helping to organize this year in Los Angeles). This meetup proved to be interesting even to those of us who attended the conference (which was about half of our audience), simply because there were so many cool things to hear and enough parallel sessions at the conference that no one could possibly capture all of the action.

Heather Turner is a biostatistician and Associate Fellow at the University of Warwick, as well as a consultant. She is a former editor of the R Journal, was a local organizer for the useR! 2011 conference, and holds a wealth of information about the field of data science, the state of the technology, and the wider community.

At useR! 2014, one of the most anticipated presentations was Joe Cheng’s Shiny: R made interactive. It was but one of the of a fantastic series of talks by RStudio representatives; check out Winston Chang’s ggvis: Interactive graphics in R for another great talk. Shiny begins with your R code and ends with the customer’s view: an interactive, browser-accessible application. This enables you to place a great deal of power into your users’ hands to customize their view and provides a variety of new and innovative ways to interact with the data, such as by tuning parameters and focusing on sub-populations. Essentially, Shiny functions by allowing you to “wire up” up a reactive application, greatly speeding up the development of a data and analysis based web app.

As someone who has spent a significant amount of time ‘data wrangling’ outside the field of data science, I found this recent New York Times article both highly interesting and nothing new - I’ve seen the same issue discussed in a variety of fields with similar automation solutions presented each time. Granted, there are fields where this janitorial drudgery is absolutely necessary, such as molecular biology. As a scientist working on researching mRNA expression you certainly have to keep RNase-free conditions, meaning you have to ‘clean up’ your bench in a more detailed way than you’d ever imagine before you can run your first PCR cycle. This includes specific protocols for washing your dishes several times with special water before you even start because if you don’t, you’re not likely to get any bands in your electrophoresis gel and you’ll lose several days (or weeks!) of work.

We have a few pieces of swag that we are raffling off to the community at our upcoming meetup events. Our stash consists of 25 data-oriented books, primarily from O’Reilly plus 25 T-shirts from useR! 2014 conference and Revolution Analytics (see picture above). Did we get your attention?

In science consensus is irrelevant. What is relevant is reproducible results. The greatest scientists in history are great precisely because they broke with the consensus. – Michael Crichton Yihui Xie intended to make it easier to do his homework, but instead found himself tackling one of the greatest problems in modern science: the reproducibility of results.