NYU Data Science newsletter – July 15, 2016

GROUP CURATION: N/A

As SAILORS returns for the second summer, the new campers are giddy with excitement. After grabbing breakfast and getting to know one another, the girls situate themselves in a lecture room in the Gates Computer Science building at Stanford University. Professor Fei-Fei Li, director of the SAILORS program and the AI Lab as a whole, warmly welcomes the campers to the summer program, imparting the grounds on which the idea of an all-girls, two-week research-intensive program came about just two years ago.

The AP is now using bots to handle some game write-ups, but some say all is not lost for sports journalism

The IoT has implications for two distinct aspects of journalism – newsgathering and consumption. Smart devices connected to each other can be used to provide better context to a story, such as data on traffic, weather, population density or power consumption.

Inside a 350,000-square-foot building in the hills of Prineville, OR, slotted inside a nondescript server rack, is one of Facebook’s most valuable artificial intelligence tools. It’s called Big Sur, and it’s a hardware system for training software to improve itself over time. It uses enormous amounts of data, funneled in from all over the world, and taps into the building’s extraordinary computing power to accelerate a process that once took months down to a matter of hours. With Big Sur, Facebook is able to train AI processes that power board-game-playing programs and help software “read” photos and explain their contents back to people.

Microsoft has unveiled a new degree program as it looks to address the “significant skills gap” that exists in the field of data science.

Announced at its Worldwide Partner Conference yesterday, the Microsoft Professional Degree (MPD) program is touted as “the first program of its kind to offer employer-endorsed, university-caliber curriculum for professionals at any stage of their career.” The course will be offered through Edx.org.

Are we giving AI engineers and creators the right tools to be ethical and create ethical products, algorithms, and software?

More from recent conferences:

useR 2016, A Love Story (July 12, Medium, Moore Data, Chris Mentzel)

Dispatch: The White House’s and NYU’s Artificial Intelligence Workshop #AINow (July 12, LinkedIn, Khurram Nasir Gore)

Fast Forward Labs: What We Liked at AINow (July 08, Fast Forward Labs Blog)

Machined Learnings: ICML 2016 Thoughts (July 04, Paul Mineiro, Machined Learnings blog)

Microsoft Research at IJCAI 2016: Developing technologies that allow people and machines to collaborate (July 08, Microsoft Research, Eric Horvitz)

As you may have suspected, your car is spying on you. Fire up a new model and it updates more than 100,000 data points, including rather personal details like the front-seat passenger’s weight. The navigation system tracks every mile and remembers your route to work. The vehicular brain is smart enough to help avoid traffic jams or score parking spaces, and soon will be able to log not only your itineraries but your internet shopping patterns.

A clash between European Union bureaucracy and artificial intelligence is a plot worthy of a cyberpunk thriller. It will take place in real life in 2018, once some European data protection laws, passed earlier this year, go into effect. And, though we might instinctively be tempted to endorse progress over regulation, the EU is on the side of the angels in this battle.

The EU’s General Data Protection Regulation and a separate directive contain provisions to protect people against decisions made automatically by algorithms.

I loved the useR 2016 conference, and here’s why: though I am not a statistician, and I have never programmed in R, I found my first time at this yearly geekfest of R-ficionados to be a friendly, accessible, and instructive event.

Our program in Data-Driven Discovery focuses on supporting the people and practice of data science for natural science research. Many of the people we support use R, and many others create new approaches to leveraging data for new discoveries through developing new R packages and programs. So it was really about time that one of us here dive into the community of practitioners. It was illuminating.

The devil’s in the data. A new report from NYU Tandon’s Governance Lab attempts to define guidelines for its deployment.

We have a long way to go if we want virtual assistants to understand us.

Levels of global biodiversity loss may negatively impact on ecosystem function and the sustainability of human societies, according to UCL-led research.
biodiversity hotspots

“This is the first time we’ve quantified the effect of habitat loss on biodiversity globally in such detail and we’ve found that across most of the world biodiversity loss is no longer within the safe limit suggested by ecologists” explained lead researcher, Dr Tim Newbold.

This workshop will bring together researchers across disciplines to discuss the future of research on the mathematical foundations of human computation, with particular emphasis on the ways in which theorists can learn from the existing empirical literature on human computation and the ways in which applied and empirical work on human computation can benefit from mathematical foundations.

Austin, TX Held at HCOMP 2016 on Thursday, November 3.

Deadline for submissions is Monday, August 1.

In this blog, I explore three sets of APIs—RDDs, DataFrames, and Datasets—available in a pre-release preview of Apache Spark 2.0; why and when you should use each set; outline their performance and optimization benefits; and enumerate scenarios when to use DataFrames and Datasets instead of RDDs. Mostly, I will focus on DataFrames and Datasets, because in Apache Spark 2.0, these two APIs are unified.

Our primary motivation behind this unification is our quest to simplify Spark by limiting the number of concepts that you have to learn and by offering ways to process structured data.

The primary aim of this page is to share information about useful new Python 3 features that may be useful to scientists for everyday work, as well as information about things you can do right now to prepare for the Python 3 transition, and how to try Python 3 (without necessarily switching over completely).

Distributed DQN framework for training OpenAI Gym (https://gym.openai.com/) environments over multiple GPUs. Can also be configured to run in a cluster of hosts.