NYU Data Science newsletter – February 26, 2016

GROUP CURATION: N/A

In June of last year, we published a story about a visualization techniques that helped to understand how neural networks carried out difficult visual classification tasks. In addition to helping us gain a deeper understanding of how NNs worked, these techniques also produced strange, wonderful and oddly compelling images.

Following that blog post, and especially after we released the source code, dubbed DeepDream, we witnessed a tremendous interest not only from the machine learning community but also from the creative coding community.

One of the first Knights Landing, Omni-Path supercomputers will be hitting the floor in Colorado in the coming months, and while one of the lead decision-makers for the system says they are expecting to see it in May (well ahead of when Knights Landing and Omni-Path were expected to appear, even for early ship programs), that buffer time provides a chance to make the necessary tweaks and optimizations to ensure that a scientific computing software stack is primed and ready for the changes Omni-Path in particular will bring about.

According to Peter Ruprecht, Senior HPC analyst and lead for the KNL, Omni-Path machine at the University of Colorado, Boulder, where the new cluster will live, much the system architecture was developed with the goal of suiting as many varied user applications as possible.

In the past, the bots I’ve made have primarily been a tool for exploring systems and culture. I feed the bot data or text and then design constraints around how it can recombine or augment that material to create insight, humor, or strange robot poetry. … Lately, however, I’ve become much more interested how bots become social creatures. What does it mean to have a conversation with a computer or a bot? What does that conversation feel like and what kind of underlying relationship does it create or reflect?

PLOS and the PLOS ONE editorial team would like to express our gratitude to all those individuals who participated in the peer review process of submissions to PLOS ONE over this past year. During 2015 PLOS ONE published over 28,000 research articles. This would not have been possible without the contribution of more than 76,000 reviewers from around the world and a wide range of disciplines.

Tobias Weyand, a computer vision specialist at Google, and a couple of pals have trained a deep-learning machine to work out the location of almost any photo using only the pixels it contains.

Their new machine significantly outperforms humans and can even use a clever trick to determine the location of indoor images and pictures of specific things such as pets, food, and so on that have no location cues.

When Daniel Nadler woke on Nov. 6, he had just enough time to pour himself a glass of orange juice and open his laptop before the Bureau of Labor Statistics released its monthly employment report at 8:30 a.m. He sat at the kitchen table in his one-bedroom apartment in Chelsea, nervously refreshing his web browser — Command-R, Command-R, Command-R — as the software of his company, Kensho, scraped the data from the bureau’s website. Within two minutes, an automated Kensho analysis popped up on his screen: a brief overview, followed by 13 exhibits predicting the performance of investments based on their past response to similar employment reports.

Nadler couldn’t have double-checked all this analysis if he wanted to. It was based on thousands of numbers drawn from dozens of databases. He just wanted to make sure that Kensho had pulled the right number — the overall growth in American payrolls — from the employment report. It was the least he could do, given that within minutes, at 8:35 a.m., Kensho’s analysis would be made available to employees at Goldman Sachs.

New research from the McKinsey Global Institute looks at how all types of global flows influence economic growth. We find that over the course of a decade, cross-border flows of goods, services, finance, people, and data increased world GDP by roughly 10 percent over what would have occurred in a world without any flows. This value was equivalent to $7.8 trillion in 2014 alone.

What really jumps out from our findings is that data flows account for $2.8 trillion of this effect.

We founded DeepMind to solve intelligence and use it to make the world a better place by developing technologies that help address some of society’s toughest challenges. It was clear to us that we should focus on healthcare because it’s an area where we believe we can make a real difference to people’s lives across the world.

We’re starting in the UK, where the National Health Service is hugely important to our team.

How often do people conducting surveys simply fabricate some or all of the data? Several high-profile cases of fraud over the past few years have shone a spotlight on that question, but the full scope of the problem has remained unknown. Yesterday, at a meeting in Washington, D.C., a pair of well-known researchers, Michael Robbins and Noble Kuriakose, presented a statistical test for detecting fabricated data in survey answers. When they applied it to more than 1000 public data sets from international surveys, a worrying picture emerged: About one in five of the surveys failed, indicating a high likelihood of fabricated data.

A botifesto

We live in a world of bots. Generally speaking, these sets of algorithms are responsible for so much on the backend of the internet, from making Google searches possible to filling up your spam folder. But an emergent kind of bot, capable of interacting with humans and acting on their behalf, is playing a more active role in our everyday lives.

Also, more Bots essays from the Data & Society:Points blog:

Bots: A definition and some historical threads (Allison Parrish, February 24)

Our friends, the bots? (Alexis Lloyd, February 25)

What is the Value of a Bot? (Danah Boyd, February 26)

Johan Bollen at Indiana University in Bloomington and a few pals [including NYU CDS’ Bruno Goncalves] have found the first evidence of a happiness paradox on Twitter. They say that it is good evidence that social network use can affect the well-being of a significant proportion of the planet’s population.

The friendship paradox is straightforward to explain. It comes about because of the skewed way people collect friends on online social networks such as Twitter and Facebook. Most people have a small number of friends—a few dozen or so. But a tiny fraction of people have huge numbers of friends millions or tens of millions of followers in some cases.

Yahoo! Inc., is getting into the artificial intelligence (AI) game with the release of new internally-built software under an open-source license. Called CaffeOnSpark, the software is able to perform ‘deep learning’ on the vast ocean of data kept in Yahoo’s Hadoop file system. Now, the company has made it available on GitHub for everyone to use.

This repository contains data and code offering a demo of the NBA Expected Possession Value model presented in the paper “A Multiresolution Stochastic Process Model for Predicting NBA Possession Outcomes.”

The main document that introduces and illustrates the code/data is EPV_demo.pdf. The source .tex for the tutorial file EPV_demo.pdf can be built from EPV_demo.Rnw using RStudio.

During the seven-week Insight Data Engineering Fellows Program recent grads and experienced software engineers learn the latest open source technologies by building a data platform to handle large, real-time datasets. Ryan Walker (now a Data Engineer at Casetext) discusses his project of building a streaming search platform.

Collaborative knowledge bases that make their data freely available in a machine-readable form are central for the data strategy of many projects and organizations. The two major collaborative knowledge bases are Wikimedia’s Wikidata and Google’s Freebase. Due to the success of Wikidata, Google decided in 2014 to offer the content of Freebase to the Wikidata community. In this paper, we report on the ongoing transfer efforts and data mapping challenges, and provide an analysis of the effort so far. We describe the Primary Sources Tool, which aims to facilitate this and future data migrations. Throughout the migration, we have gained deep insights into both Wikidata and Freebase, and share and discuss detailed statistics on both knowledge bases.