NYU Data Science newsletter – December 3, 2015

GROUP CURATION: N/A

Large amount of Electronic Health Record (EHR) data have been collected over millions of patients over multiple years. The rich longitudinal EHR data documented the collective experiences of physicians including diagnosis, medication prescription and procedures. We argue it is possible now to leverage the EHR data to model how physicians behave, and we call our model Doctor AI. Towards this direction of modeling clinical bahavior of physicians, we develop a successful application of Recurrent Neural Networks (RNN) to jointly forecast the future disease diagnosis and medication prescription along with their timing. Unlike a traditional classification model where a single target is of interest, our model can assess entire history of patients and make continuous and multilabel prediction based on patients’ historical data. We evaluate the performance of the proposed method on a large real-world EHR data over 250K patients over 8 years. We observe Doctor AI achieves up to 79% recall@30, significantly higher than several baselines.

Can intelligent systems help data scientists? I recognize this is a massive, far-reaching question but it’s one that we need to consider and a question that inspired last week’s gathering of executives from around the world. O’Reilly’s Next: Economy conference, an event dedicated to addressing the transformation of work and business in the digital age, was an opportunity to closely examine the specific business models and industries being rapidly upended.

I was invited to speak on how intelligent systems like natural-language generation will augment our work and which roles will evolve accordingly. My presentation garnered a range of reactions as I zeroed in on the data scientist and pointed out that there are many aspects of this job that can be automated.

Announcing Lilt beta. Interactive/adaptive translation based on @stanfordnlp and @uwdata research.

Lilt is a translation tool that learns
the way you translate.

Machine Learning and AI are fast becoming ubiquitous in data driven businesses, that is to say, an awful lot of businesses. Here I choose a few areas where it’s possible that big corporations haven’t already eaten everybody’s lunch. It’s not uncharted territory?—?if I could think of the next killer application, I’d be trying to do it!

Accurate and timely estimates of population characteristics are a critical input to social and economic research and policy. In industrialized economies, novel sources of data are enabling new approaches to demographic profiling, but in developing countries, fewer sources of big data exist. We show that an individual’s past history of mobile phone use can be used to infer his or her socioeconomic status. Furthermore, we demonstrate that the predicted attributes of millions of individuals can, in turn, accurately reconstruct the distribution of wealth of an entire nation or to infer the asset distribution of microregions composed of just a few households. In resource-constrained environments where censuses and household surveys are rare, this approach creates an option for gathering localized and timely information at a fraction of the cost of traditional methods.

Greetings from the West Big Data Innovation Hub (WBDIH)!

We are incredibly excited about our launch this month, and the potential of what we can create together. Our mission is to build multi-sector and multi-state partnerships to address societal challenges with Big Data innovation. To help us grow our community, please forward this note widely and tell your colleagues to join our mailing list!

With the support of the Gordon and Betty Moore Foundation, we now have the opportunity to hire an Associate Director. The Associate Director is one of the two key roles providing leadership to Data Carpentry’s core efforts and is expected to shape the organization’s operational functioning, influence training, and contribute to strategic planning. The main focus of the Associate Director’s role will be in community engagement and education as well as overseeing communications.

Have you ever wondered how Google Photos helps you find all your favorite dog photos? With today’s release of Google Cloud Vision API, developers can now build powerful applications that can see, and more importantly understand, the content of images. The uses of Cloud Vision API are game changing to developers of all types of applications and we are very excited to see what happens next!

A common question we’ve been getting is why we picked Clojure as our primary server language at Metabase. Now, choosing a programming language for a project is usually decided on a mix of what a team is most comfortable with, beer-fueled debates on dynamic vs static typing, benchmarks of marginal relevance and that elusive quest for Dev-Hipster points. In our case, we ended up on our primary backend language by a rather circuitous path.

Do you do data driven discovery, data-intensive research, or data science? Please answer some questions about resources!

Survey results on data driven discovery are immediately available under CC0.

Social media have provided instrumental means of communication in many recent political protests. The efficiency of online networks in disseminating timely information has been praised by many commentators; at the same time, users are often derided as “slacktivists” because of the shallow commitment involved in clicking a forwarding button. Here we consider the role of these peripheral online participants, the immense majority of users who surround the small epicenter of protests, representing layers of diminishing online activity around the committed minority. We analyze three datasets tracking protest communication in different languages and political contexts through the social media platform Twitter and employ a network decomposition technique to examine their hierarchical structure. We provide consistent evidence that peripheral participants are critical in increasing the reach of protest messages and generating online content at levels that are comparable to core participants.