NYU Data Science newsletter – October 6, 2015

GROUP CURATION: N/A

Today as organizations attempt to leverage Big Data and compete on analytics, there are “kinks” in the data supply chain—slowing the process that collects, stores, analyzes, and transforms data into insights. The kinks are information silos, and they stand between analysis and actionable insight.

Unfortunately, the task of unclogging the data supply chain often falls to various subject matter experts (SMEs), each of whom may only know what a small subset of an organization’s data sources contain.

bqplot is a plotting system for the Jupyter notebook.

provide a unified framework for 2d visualizations with a pythonic API.

provide a sensible API for adding user interactions (panning, zooming, selection, etc)

I recently gave a talk about recommender systems at the Data Science Sydney meetup (the slides are available here). This post roughly follows the outline of the talk, expanding on some of the key points in non-slide form (i.e., complete sentences and paragraphs!). The first few sections give a broad overview of the field and the common recommendation paradigms, while the final part is dedicated to debunking five common myths about recommender systems.

Please help us make this a great book! This is still only a draft and can be improved in many ways. We’ve included our TODO marks to help you see which sections are still under rapid development. If you see a section that we appear to have finished writing and you find it unclear or inaccurate, please let us know. Please specify (by date) which version you are commenting on.

For those who don’t know, this post is in direct reply to arguably the worst, most misleading, idiotic chart ever created (which was widely shared by other idiotic people). … 1388 comments, as of 10/9

It can take weeks to reprogram an industrial robot to perform a complicated new task, which makes retooling a modern manufacturing line painfully expensive and slow.

The process could be sped up significantly if robots were able to learn how to do a new job by watching others do it first. That’s the idea behind a project underway at the University of Maryland, where researchers are teaching robots to be attentive students.

The National Institutes of Health has awarded UCSF $9.75 million to develop a platform, over the next five years, that researchers can use to conduct digital health studies. This platform, called Health ePeople, will provide researchers with access to a large group of volunteers that have agreed to participate in research and the infrastructure to collect participant health data through mobile and wireless technologies.

“The primary goal of Health ePeople is to provide a resource enabling convenient and efficient mobile and wireless health research,” a co-principal investigator, Jeffrey Olgin, who is also a professor of medicine and chief of cardiology at UCSF.

… This week, the indoor health revolution took a huge step forward with the opening of the Well Living Lab at the Mayo Clinic in Rochester, Minn. Delos and the Mayo Clinic partnered to create a 7,500-square-foot research lab that will be the testing ground for a host of different studies on indoor health. The goal is to quantify the way we interact with the indoor environment so we can gain a better understanding of how it impacts our mental and physical well being. Ultimately, says Scialla, the hope is that we’ll begin to design buildings around this kind of research.

The lab is designed to be totally modular—the floor, walls, ceiling, furniture, plumbing, and lighting fixtures can all be swapped out and replaced to mimic the interior design of business and residential spaces, depending on what the particular study calls for.

… A study by researchers at Cornell University and Stanford shows that a more advanced system could be trained to recognize the body language and behavior that precedes a particular maneuver. This could help trigger an early warning system, such as a blind spot alert, much earlier—perhaps thereby helping to prevent serious accidents, according to the academics involved.

“Imagine you are driving on a highway,” says Saxena Ashutosh, the director of a project called Robo Brain at Cornell University and Stanford who oversaw the driving project.

… Insight offers a Fellowship three times a year where academics learn the applied data science skills they need to work in industry. Klein had already used machine learning in his research – it comes in handy when interpreting fMRI data – but at Insight he picked up industry standard tools and data science workflows to handle messy datasets. As a long-time wearable enthusiast, Klein was fascinated by the Health eHeart study at UCSF – where over 30,000 participants contributed Fitbit step data and anonymized clinical information. After many hours of data wrangling and feature engineering, Mike was able to detect early warning signs of heart conditions based on Fitbit walking patterns.

“I was excited to be able to leverage my existing knowledge of machine learning techniques to attack a problem in a new domain area with very different sorts of data. Both the industry focused tools and the pace (a sprint!) were very different at Insight and, in the process, I learned a great deal about working with sparse/unbalanced datasets.” says Mike Klein

Klein wasn’t the only one in his cohort to come up with a data-driven solution during the intensive 7-week fellowship.

Riley Newman is the Head of Data Science at Airbnb.com, was the company’s first Data Scientist and one of the first 10 employees. Since Airbnb was founded in 2008, they have grown roughly 43,000%, currently have over 1.5 million listings in 34,000 cities across 190 countries and Riley and his team manage all of that data. … We caught up with Riley to discuss the early days of Airbnb, what are his daily priorities and his picks as the top 3 funkiest places listed on Airbnb.

The other day I was catching up with neighborhood news, and saw this article about “people counters” in San Francisco’s tourist district. These are cameras watching the sidewalks and totaling up how many pedestrians are walking past. The results weren’t earth-shattering, but I was fascinated because I’d never heard of the technology before. Digging in deeper, I discovered there’s a whole industry of competing vendors offering similar devices.

Why am I so interested in these? Traditionally we’ve always thought about cameras as devices to capture pictures for humans to watch. People counters only use images as an intermediate stage in their data pipeline, their real output is just the coordinates of nearby pedestrians. Right now this is a very niche application, because the systems cost $2,100 each. What happens when something similar costs $2, or even 20 cents? And how about combining that price point with rapidly-improving computer vision, allowing far more information to be derived from images?

Those trends are why I think we’re going to see a lot of “Semantic Sensors” emerging.