Data Science newsletter – November 2, 2018

GROUP CURATION: N/A

An S-76B commercial helicopter flew over a small crowd gathered at Fort Eustis, Virginia, landed in an adjacent field after adjusting to miss a vehicle, and rose up to hover perfectly motionless for several minutes. The mid-October demonstration was remarkable because the pilot carried out the maneuvers using supervised autonomy in an aircraft equipped with DARPA’s Aircrew Labor In-Cockpit Automation System (ALIAS). He operated the system via novel control interceptors and a tablet he had used for the first time just three days beforehand.

“Hovering in adverse winds is a task that consumes a human pilot’s attention, but automated flight control achieves ‘rock steady’ precision,” said Graham Drozeski, the DARPA program manager for ALIAS, explaining how offloading pilots’ cognitive burden frees them to focus on mission execution.

When playing a video game, what motivates you to carry on?

This question is perhaps too broad to yield a single answer, but if you had to sum up why you accept that next quest, jump into a new level, or cave and play just one more turn, the simplest explanation might be “curiosity” — just to see what happens next. And as it turns out, curiosity is a very effective motivator when teaching AI to play video games, too.

Research published this week by artificial intelligence lab OpenAI explains how an AI agent with a sense of curiosity outperformed its predecessors playing the classic 1984 Atari game Montezuma’s Revenge. Becoming skilled at Montezuma’s Revenge is not a milestone equivalent to beating Go or Dota 2, but it’s still a notable advance. When the Google-owned DeepMind published its seminal 2015 paper explaining how it beat a number of Atari games using deep learning, Montezuma’s Revenge was the only game it scored 0 percent on.

The classic rubber ducky has many attributes. It’s cute, tough, and super buoyant. For one team of coders, those kid-friendly, bath-time qualities inspired something more: a disaster response startup that just won $200,000 and the chance for worldwide implementation through IBM’s inaugural Call for Code competition.

Project Owl–an acronym that stands for “organization, whereabouts, and logistics”–will air-drop (likely via drone) their plucky “Clusterduck” armada into a disaster area (they can float in water or just sit wherever they land). The devices are small, hexagonal rubber balls that are waterproof, durable, and house mini-Wi-Fi relays, which can work together to create an ad hoc mobile network.

Over the past two years, since the former Department of Statistics expanded into what is now called DS2, its leaders have assembled faculty with a wide-ranging body of research and academic expertise. Their collaborations delve into fields as varied as astrophysics, genetics, forestry, engineering, economics, computer science, radiology, mathematics, and law.

It’s no accident. If data can be found in every part of Yale, DS2 leaders say, then DS2 should be there as well.

“Like many things at Yale, it has started with teaching and education,” said John Lafferty, the John C. Malone Professor of Statistics and Data Science, who joined the department in 2017. “The new Statistics and Data Science major has been popular right from the start when it was introduced last year. Going forward, Yale is embracing the interdisciplinary nature of data science. It will be important to join forces across traditional departmental boundaries to advance data science and its application in different fields.”

One major goal for elected officials and policymakers is to improve the happiness of their constituents in cost-effective ways. But how do individuals’ circumstances influence which policies are likely to make them feel more satisfied with their lives? In new research which draws on a large Connecticut-based survey, Christopher Barrington-Leigh and Jan Wollenberg find that improvements in areas of people’s lives such as food security and social engagement can lead to much greater improvements in their happiness compared to increasing their incomes. With that in mind, they recommend that by targeting specific groups and needs, state and local agencies may be much more effective at improving happiness compared to more blanket measures which affect everyone.

In a paper being presented this week at the Conference on Empirical Methods in Natural Language Processing, researchers from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) describe a model that runs faster and more efficiently than these monolingual models.

The model leverages a metric in statistics, called Gromov-Wasserstein distance, that essentially measures distances between points in one computational space and matches them to similarly distanced points in another space. They apply that technique to “word embeddings” of two languages, which are words represented as vectors — basically, arrays of numbers — with words of similar meanings clustered closer together. In doing so, the model quickly aligns the words, or vectors, in both embeddings that are most closely correlated by relative distances, meaning they’re likely to be direct translations.

In a paper being presented at this week’s Empirical Methods in Natural Language Processing conference, MIT researchers describe a parser that learns through observation to more closely mimic a child’s language-acquisition process, which could greatly extend the parser’s capabilities. To learn the structure of language, the parser observes captioned videos, with no other information, and associates the words with recorded objects and actions. Given a new sentence, the parser can then use what it’s learned about the structure of the language to accurately predict a sentence’s meaning, without the video.

This “weakly supervised” approach — meaning it requires limited training data — mimics how children can observe the world around them and learn language, without anyone providing direct context. The approach could expand the types of data and reduce the effort needed for training parsers, according to the researchers. A few directly annotated sentences, for instance, could be combined with many captioned videos, which are easier to come by, to improve performance.

Applying artificial intelligence to self-driving cars to smooth traffic, reduce fuel consumption, and improve air quality predictions may sound like the stuff of science fiction, but researchers at the Department of Energy’s Lawrence Berkeley National Laboratory (Berkeley Lab) have launched two research projects to do just that.

In collaboration with UC Berkeley, Berkeley Lab scientists are using deep reinforcement learning, a computational tool for training controllers, to make transportation more sustainable. One project uses deep reinforcement learning to train autonomous vehicles to drive in ways to simultaneously improve traffic flow and reduce energy consumption. A second uses deep learning algorithms to analyze satellite images combined with traffic information from cell phones and data already being collected by environmental sensors to improve air quality predictions.

One of the mysteries of Facebook is that whenever public sentiment about the company feels most mixed, it delivers smashing results for Wall Street that keep any social consequences from depressing the company’s share price. This was true even during the depths of the Cambridge Analytica scandal, which began with a major price drop and ended with Facebook at a new all-time high, $209.94 a share, ahead of its earnings announcement in mid-July.

Since that report, which revealed slowing user and profit growth, Facebook’s share price has been tumbling steadily, falling to about $150. Even Tuesday’s earnings, which crushed expectations, did not right the Facebook ship. User growth is still slowing, and CEO Mark Zuckerberg cautioned that the product that built the Facebook-advertising empire, News Feed, has become outmoded. It won’t disappear overnight, but it will capture less and less of the total attention inside Facebook’s ecosystem. Replacing it, bit by bit, will be a mix of things: Stories, messaging, video, and more targeted “community” features like Marketplace and Dating.

Today, we’re happy to announce our newest white paper, “The Inevitability of AI Law & Policy: Preparing Government for the Era of Autonomous Machines,” by Public Knowledge General Counsel Ryan Clough. The paper argues that the rapid and pervasive rise of artificial intelligence risks exploiting the most marginalized and vulnerable in our society. To mitigate these harms, Clough advocates for a new federal authority to help the U.S. government implement fair and equitable AI. Such an authority should provide the rest of the government with the expertise and experience needed to achieve five goals crucial to building ethical AI systems:

Morgan Stanley just hired a physician who created a futuristic doctor’s office in an effort to improve the health and wellness benefits it offers to its workers.

At Morgan Stanley, Dr. David Stark will serve as the bank’s first chief medical officer as well as the head of HR data and analytics, with the aim of tackling rising healthcare costs and improving employee wellness. Stark was the creator of Lab100, a futuristic clinic built in partnership with Mount Sinai and was an assistant professor at the health system’s Icahn School of Medicine.

Data-driven and evidence-based social policy innovation can help governments serve communities better, smarter, and faster. Integrated Data Systems (IDS) use data that government agencies routinely collect in the course of delivering public services to shape local policy and practice. They can inform the design and implementation of programs, help measure and evaluate outcomes across the lifecourse, and enable policy-makers to better address complex social problems.

Respecting privacy is paramount to IDS’ success. The use of IDS to link sensitive personal data is typically governed by stringent local, state, and federal privacy laws and regulations, as well as rigorous technical safeguards and ethical norms. Nevertheless, individuals and communities routinely have questions and concerns about how their information is used and protected.

For lasting success, IDS need to develop “social license” to integrate data.

Video games are the great equalizer. No matter where you live and your level of experience, everyone plays the same product: A student in Barcelona and a salaryman in Japan see the exact same content and scenarios when playing Minecraft, and the same Fortnite gliders are available to a player in Seattle and another in Taipei. But the future of gaming lies in real-time personalization, driven by the very data we feed into the game.

The video game industry is playing catch-up to Big Tech when it comes to customization. Most companies already deliver a unique experience based on each user’s preferences: Google provides highly customized search results by collecting data on your searches, translations, and emails; Spotify creates individual playlists based on what you’ve been listening to; and Netflix and Amazon present you with shows and movies you’re likely to find interesting based on what you’ve viewed in the past.

Links to conference videos now available.

MotionRugs is a novel dense pixel display technique for the visualization of collective movement. The technique makes use of spatial linearization strategies to create a condensed, static and two-dimensional representation of spatio-temporal datasets.

According to Russinovich, the best way to understand microservice infrastructure is to look at the previous generation architecture of the client-server era. The era of monoliths. Typically, there were three tiers: application (front end); the middle tier (business logic) and then the third tier of backend for things like a database server. [audio, 10:42]