R

Come and have fun with local useRs at SatRday LA on April 6, 2019. This is the first SatRday in LA, and the second in the states. If you have not heard of SatRday, it is a one-day affordable, inclusive, non-profit R conference organized by local R users.

Update: Video of the 2nd half of the talk is here (1st half was not recorded, sorry): This video is no longer available on YouTube (original link). In our next meetup, we’ll have Hadley (I think no introduction needed). Packages made easy by Hadley Wickham

Update: Video recording of the meetup here: ▶ Machine Learning in Production with Szilard Pafka Slides here:

Update: Slides click here. Video recording here: This video is no longer available on YouTube (original link). Stream processing with R in AWS by Gergely Daróczi Abstract: R is rarely mentioned among the big data tools, although it’s fairly well scalable for most data science problems and ETL tasks. This talk presents an open-source R package to interact with Amazon Kinesis via the MultiLangDaemon bundled with the Amazon KCL to start multiple R sessions on a machine or cluster of nodes to process data from theoretically any number of Kinesis shards. Besides the technical background and a quick introduction on how Kinesis works, this talk will feature some stream processing use-cases at CARD.com, and will also provide an overview and hands-on demos on the related data infrastructure built on the top of Docker, Amazon ECS, ECR, KMS, Redshift and a bunch of third-party APIs.

It has been over 2 months since the first satRday conference ended and I shared some quick impressions on that, where I also promised a follow-up post on my experiences as ending up being the main organizer of the event – this is what you are reading now. Although the idea was born at the EARL Boston conference last year, and I decided to take part in the project while living in Los Angeles – the conference took place in Budapest after all as logistics seemed to be easier there thanks to my closer relationship to the Hungarian R User Group.

XGBoost is a fantastic open source implementation of Gradient Boosting Machines, a general purpose supervised learning method that achieves the highest accuracy on a wide range of datasets in practical applications. Deep learning is all the hype now, but apart from specific domains such as images, speech or text (i.e. problems with higher-level abstractions to be learnt and/or perception problems, where deep learning achieved some remarkable results indeed), it is usually outperformed by Gradient Boosting in a majority of general business domains and supervised learning applications. Proof of this and also because XGBoost has an easy-to-use interface from both R and Python, XGBoost has become a favorite tool in Kaggle competitions. Besides feature engineering, cross-validation and ensembling, XGBoost is a key ingredient for achieving the highest accuracy in many data science competitions and more importantly in practical applications.

What has Kaggle learned from 2 million machine learning models? Anthony Goldbloom, founder and CEO of Kaggle UPDATE:Video recording of the talk available here Kaggle is a community of almost 450K data scientists who have built almost 2MM machine learning models to participate in our competitions. Data scientists come to Kaggle to learn, collaborate and develop the state of the art in machine learning. This talk will cover some of the lessons on winning techniques we have learned from the Kaggle community.

With so much hype about “big data” and the industry pushing for “big data” analytical tools for everyone, the question arises how many people have big data (for analytics) and how many of them really need these tools (which are more complex and often more immature compared to the traditional tools for analytics).

One of the best things I like in working at CARD.com is that I am not only crunching R code in 24/7, but I also have the chance to interact with and improve the related data infrastructure with some interesting technologies. After joining the company in January, I soon realized that while Impala is a very powerful database for handling data that do not comfortably fit in MySQL, it’s still not as fast as one might expect when querying large amount of data from R. Sometimes I had to wait several minutes for a query to run! So I used this spare time to think about how to improve the workflow.

I currently have the need for machine learning tools that can deal with observations of the order of 10 millions in the context of binary classification. That kind of data is a few GBs in size and it fits comfortably nowadays in the RAM of a decent single machine. It is a trivial task for linear models, as there are plenty of open source tools that can train a logistic regression with this amount of data on a single machine in a few seconds, even while using only 1 processor core (many of these tools are single-threaded). Linear models are also the gold standard of large-scale machine learning that can run on clusters, processing very large distributed datasets.

What is DataFest? # DataFest is a team competition, where a team of up to 5 undergraduate students has less than 48 hours to provide the best insight gleaned from a large, complex data set — and communicate that insight to an audience of their peers and the DataFest judges. DataFest 2015 will be April 24th through the 26th, beginning at 6pm on Friday the 24th.

This meetup will be hosted by Google and we’ll have Peter Lipman and Pete Meyer talk about the data science/statistics projects they have been working on at Google. Human Evaluation Comparisons: Common Framework with Applications We designed a study to determine whether being on device influenced a quality evaluator’s perception of ads quality. Here, we present a common framework, developed into an internal R package, that was used to analyze the study.

The Shiny framework, by RStudio, is growing in popularity as a tool that makes it easy for R developers to leverage the web as a deployment platform for sharing their analysis. However, since the very earliest release of Shiny, the development group has been bombarded with the same series questions over and over again, questions like “Does it work with d3.js?” and “Can I use JavaScript in Shiny?”

The Shiny framework is a web application framework for the R programming language, designed by Joe Cheng, to help R programmers turn their analysis into interactive web applications. Fortunately, it does without requiring that the R programmer know HTML, CSS or JavaScript - greatly widening the possible audience of users. At the January DataScience.LA R Meetup, we were fortunate to have Joe Cheng, Software Engineer at RStudio and original architect of Shiny present an introduction to Shiny.

This was originally posted on the author’s LinkedIn You will find lots of advice on how to get a job as a data scientist. I want to add some signal to the noise. I happen to be hiring for a Data Scientist or two as of this writing, and was thinking of doing a blog post on offering advice for those looking. But then I thought of something better and more insightful: It is a cut-and-paste of an email that I sent to our Director of Talent Acquisition last week when he asked me, “Can you share with me what you’re looking for in regards to your Data Scientist candidates?”

Shiny takes inputs from UI elements and sends them to the server, where the application can access them as R variables. While Shiny has security measures in place, as in any typical web application, it remains the developer’s responsibility to sanitize the inputs before using them. For example, Shiny has no way to protect you if you are using an input in a SQL query such as select ... from ... where field = 'input'. Someone manipulating the websocket communication can craft a specially-formatted input that can force the database to execute a query that it is not supposed to do, termed an SQL injection. This might give an attacker access to private data or the ability to do other nefarious things, and it is a common security issue.

It pains me to admit it, but even though I had visited their site, created an account, and played around with their tools, I didn’t really get the value proposition behind Plot.ly. I already use ggplot, bokeh, and d3.js, I already use knitr and ipython notebooks, so why do I need a new way of posting my plots on a web page?

Plotly is a web-based platform for making graphs and analyzing data. Plotly’s APIs and web app let teams work together online to make 2D, 3D, and streaming graphs using R, Python, MATLAB, Julia, SAS, and and Excel, all together, all in one platform. The goal is to support technical and non-technical collaboration at all levels. Plotly translates ggplot2, matplotlib, and MATLAB plots into interactive, web-based, shareable plots.

Yesterday, I published a post here on DataScience.LA with a simple/basic benchmark for dplyr. It was the main part of a short talk at our last Los Angeles R meetup. Below, I continue my thoughts with some reflections on dplyr’s API and philosophy, AKA the other part of my talk.

Earlier this month I gave a short talk about dplyr at the Los Angeles R meetup. It was one of the 5 talks about the useR! 2014 conference, and you can see the summaries and slides from the other 4 talks here.