NYU Data Science newsletter – April 1, 2016

GROUP CURATION: N/A

“An AI utopia is a place where people have income guaranteed because their machines are working for them,” [Oren Etzioni] explains on a new episode of GeekWire’s radio show. “Instead, they focus on activities that they want to do, that are personally meaningful like art or where human creativity still shines, in science. They’re engaged in those activities because of the interaction. Another one would be, of course, interaction between people and not because they need to make a buck.” [audio, 31:52]

Last year, The University of San Diego’s California Institute for Telecommunications and Information Technology’s (Calit2) data sharing initiative, called Health Data Exploration (HDE), awarded a total of $200,000 to five projects that aim to use aggregated personal health data to advance research. Now, the five HDE projects have released results from their first year. … New York University Assistant Professor Rumi Chunara received $50,000 last year to develop a platform that aggregates Runkeeper data and uses it to study the relationship between the environment and how types and amounts of exercise vary over time. Researchers created a program with Open Humans that allows Runkeeper users to send their data to the study securely.

Martin Lindstrom has spent time with 2,000 families in more than 77 countries to get clues to how they live — resulting in the acquisition of what he likes to call Small Data. In his new book, Small Data: The Tiny Clues That Uncover Huge Trends, he argues that the Small Data explains the why behind what Big Data reveals. Knowledge@Wharton recently spoke with Lindstrom on the Knowledge@Wharton show. [audio, 20:11]

With the Zika virus spreading largely unchecked in Latin America and the Caribbean by way of a now-notorious insect, some of the nation’s leading mosquito researchers are striving to assemble a state-of-the-art DNA map that they say will help them fight the disease with the mosquito’s own genetic code.

The NYU Tandon School of Engineering today announced it has been awarded a highly competitive $260,000 grant from the National Endowment for the Humanities to digitize the City Record. The project, which will be led by Jonathan Soffer, Professor of History and Chair, Department of Technology, Culture & Society at Tandon, will digitize the 1,723 volumes – or one million pages – of the City Record, from 1873-1998 and make them openly and freely available to the public on nyc.gov.

These volumes contain copious data on every aspect of the city’s politics, society, economy, real estate and infrastructure development, employment, and expenditures and will aid scholars studying the city because of the depth and breadth of the data it contains, offering digitized resources unmatched by any other city.

We present a framework for efficient inference in structured image models that explicitly reason about objects. We achieve this by performing probabilistic inference using a recurrent neural network that attends to scene elements and processes them one at a time. Crucially, the model itself learns to choose the appropriate number of inference steps. We use this scheme to learn to perform inference in partially specified 2D models (variable-sized variational auto-encoders) and fully specified 3D models (probabilistic renderers). We show that such models learn to identify multiple objects – counting, locating and classifying the elements of a scene – without any supervision, e.g., decomposing 3D images with various numbers of objects in a single forward pass of a neural network. We further show that the networks produce accurate inferences when compared to supervised counterparts, and that their structure leads to improved generalization.

The belief that our life offline is separate from our life online has been denounced as digital dualism. But there’s less of a debate when it comes to differentiating between analog objects and digital data. Yes, the print and electronic copies of the same book contain the same words, but it’s obvious to most people (and, increasingly, to researchers) that the two reading experiences are quite different.

We need to understand such differences because the world is going to see a lot more digital data in the near future.

Before I started playing Guess the Correlation, I didn’t expect to spend an hour of my Easter weekend obsessing over an 8-bit video game, much less one based on something that many scientists do every day. I also didn’t expect to be hypnotized by graph after graph of black dots, trying to accurately gauge the patterns they concealed, in exchange for points and a place on a leaderboard. And I definitely didn’t expect to have fun doing it.

Guess the Correlation is the brainchild of Omar Wagih, a graduate student at the European Bioinformatics Institute, and nefarious devourer of the thing I once called “my free time.” On paper, it sounds incredibly boring. In practice, it is inexplicably addictive. Try it.

… As I spoke to each botmaker, it became increasingly clear that the community at large was tied together by crisscrossing lines of influence. There is a well-known body of talks, essays, and blog posts that form a common ethical code. The botmakers have even created open source blacklists of slurs that have become Step 0 in keeping their bots in line.

“A lot of people in the botmaking community were perturbed to see someone coming in out of nowhere and assuming they knew better—without doing a little bit of research into prior art,” Rob Dubbin, a long-time botmaker, told me.

… Researchers at Indiana University’s Network Science Institute who analysed links shared on Twitter and searches on AOL, a web portal, showed that the sites reached from social media are much less diverse than those reached from a search engine. Pablo Barberá, formerly of SMaPP and soon to join the University of Southern California, who examined the political Twitterspheres in America, Germany and Spain, found they were indeed polarised, particularly in America.

This video looks at how to access the pixels of an HTML5 canvas in p5.js.

Feather: fast, interoperable binary data frame storage for Python, R, and more powered by Apache Arrow

This past January, we (Hadley and Wes) met and discussed some of the systems challenges facing the Python and R open source communities. In particular, we wanted to explore opportunities to collaborate on tools for improving interoperability between Python, R, and external compute and storage systems.
… In discussing Arrow in the context of Python and R, we wanted to see if we could design a very fast file format for storing data frames that could be used by both languages. Thus, the Feather format was born.

See GitHub – wesm/feather.

Caravel is a data exploration platform [created and used by Airbnb] designed to be visual, intuitive and interactive.