NYU Data Science newsletter – September 14, 2015

GROUP CURATION: N/A

Yesterday, major health insurance providers Lifetime Healthcare Companies and its subsidiary BlueCross BlueShield announced that they had been hacked, affecting a total of 10.5 million patients. These aren’t the first healthcare companies to be hacked this year, and they certainly won’t be the last; though data breaches have become an unfortunate reality for many companies, health information is especially at risk.

Healthcare data is the cash cow of the hacker world. A hacker will get $10 on the black market for each individual healthcare profile, 10 or 20 times the amount they would receive for credit card information, according to a report from Reuters published last year.

Successful applications of reinforcement learning in real-world problems often require dealing with partially observable states. It is in general very challenging to construct and infer hidden states as they often depend on the agent’s entire interaction history and may require substantial domain knowledge. In this work, we investigate a deep-learning approach to learning the representation of states in partially observable tasks, with minimal prior knowledge of the domain. In particular, we study reinforcement learning with deep neural networks, including RNN and LSTM, which are equipped with the desired property of being able to capture long-term dependency on history, and thus providing an effective way of learning the representation of hidden states. We further develop a hybrid approach that combines the strength of both supervised learning (for representing hidden states) and reinforcement learning (for optimizing control) with joint training. Extensive experiments based on a KDD Cup 1998 direct mailing campaign problem demonstrate the effectiveness and advantages of the proposed approach, which performs the best across the board.

So many data scientists select an analytic technique in hopes of achieving a magical solution, but in the end, the solution simply may not even be possible due to other limiting factors. It is important for organizations working with analytic capabilities to understand the various constraints of implementation most real-world applications will encounter. When developing a solution one has to consider: data complexity, speed, analytic complexity, accuracy & precision, and data size. Data Scientists, nor the organizations they work for, will be able to be the best in each category simultaneously; however, it will prove necessary to understand the trade-offs of each.

In this post, I will show you how to build this app. I will be using the dataset for yellow taxis in the month of January 2015 provided by the NYC Taxi & Limousine Commission. You will need RStudio for this. Since the dataset is very big, I created a smaller dataset that doesn’t contain as many rows. The smaller dataset can be found here.

Services like Google Translate and Bing work great with English and Spanish, because they have plentiful and deep data sets to draw upon: lots of stuff exists in both of those languages. But the trouble with big data is that it needs big data. This leaves languages like Galician, Welsh, and Faroese in the cold, translationally wise, because there’s just not much of them online to work with.

So linguists from the University of Copenhagen found a different solution for the translation of these minority languages. And that solution is the Bible.

The U.S. Environmental Protection Agency has awarded an $800,000 Science to Achieve Results (STAR) grant to UC Santa Barbara’s Roger Nisbet. He will use the funding to develop a model to better understand biological and ecological consequences of exposure to metals, nanoparticles and certain flame retardants in industrial and consumer products. Such materials could pose a threat to human and environmental health.

A new model for predicting tornado activity could allow experts to prepare forecasts months or even seasons in advance, researchers at the University of Toronto say.

“The aim is to predict ahead to the following year or subsequent years about whether we’ll get above or below average tornado activity in a given area,” said Vincent Cheng, a postdoctoral fellow in the Ecological Modelling Lab at the University of Toronto Scarborough (UTSC).

If dark matter is the glue holding galaxies together, dark energy is its doppelganger, pushing the universe apart at increasing speeds. Dark energy is thought to make up three-quarters of the universe yet its basic nature remains poorly understood.

In a new approach to cracking the dark energy puzzle, Columbia astronomers are working with computer scientists to wring more information from high-resolution images of about a billion galaxies in our universe. Their project, drawing on statistics, and computer self-learning and face-recognition algorithms, is funded by a two-year, $200,000 grant awarded by the Office of the Provost and administered by the Data Science Institute.

As part of the BBC’s Intelligent Machines season, Google’s Eric Schmidt has penned an exclusive article on how he sees artificial intelligence developing, why it is experiencing such a renaissance and where it will go next.

I spent most of the day at Columbia’s Data Science Institute, participating in a workshop on data science in the natural sciences. I learned a huge amount! There were way too many cool things to mention them all here, but here are some personal highlights:

Andrew Gelman (Columbia) talked about the trade-off between spatial resolution and what he called “statistical resolution”; he compared this trade-off to that in political science between conceptual resolution (the number of questions we are trying to ask) and statistical resolution (the confidence with which you can answer those questions).

… Arizona State University is feeding that boom. The program in the W. P. Carey School of Business has seen enrollment in its master’s of business analytics degree program triple in the three years it has been offered — from 54 in 2013 to 154 this year.

ASU added an undergraduate program in analytics in 2014 and an online master’s degree program this year.

Big Data For Humans, the U.K. startup that offers a data science-as-a-service for the retail and travel sector, has picked up seed funding.

When the company wanted a team of roboticists, it raided a university lab to get them. Can high-tech academia survive today’s Silicon Valley talent binge?

Can a computer tell if you’re in a good mood or ready to rip someone’s head off? Sort of.

Artificial intelligence (AI) startups like Affectiva and Emotient are making headway in this area and their technology has already been applied in market research and advertising applications where the difference between a commercial that bores you and one that captivates (or even enrages) you is the difference between success and failure.

… [researchers] grappled with the challenge of processing the massive volume of human dynamics data. A challenge initially imposed by a lack of data had become a problem of having too much data—a 180-degree swing from one extreme to the other!

But a solution to the problem of crunching the big data soon arrived, when Li’s project received a Microsoft Azure for Research Award. As Julie Zhu, a doctoral student on the project team noted, “The program arrived at exactly the right time. We were just looking into building multi-node clusters.”

It took nearly a decade of high-speed travel for NASA’s New Horizons mission to cross the 6 billion miles to the dwarf planet Pluto — and before that, many years of painstaking research and planning to make the mission possible.

Having spent much of his career working toward that epochal moment, MIT professor of planetary sciences Richard Binzel had some pithy words of advice for students in the field at a talk Wednesday afternoon: “Start young!”