NYU Data Science newsletter – March 11, 2016

GROUP CURATION: N/A

… This is the home of Terra Bella—the satellite company, formerly known as Skybox, that Google purchased for $500 million in June 2014. In the next 18 months, it plans to put more than a dozen new satellites into orbit. This will increase its imagery “refresh rate”—that is, how often any one spot on Earth is photographed—from one new image every three days to four to five new images per day.

Terra Bella is part of a larger group of satellite companies that promise to transform the way we see Earth. Planet Labs is another: An independent startup based in San Francisco, it estimates that in the next 12 months, it will have more than 100 satellites beaming imagery down to Earth. That will give it an almost-daily imagery refresh rate.

Google created the PhD Fellowship program in 2009 to recognize and support outstanding graduate students doing exceptional research in Computer Science and related disciplines. Now in its eighth year, our fellowship program has supported hundreds of future faculty, industry researchers, innovators and entrepreneurs.

Reflecting our continuing commitment to supporting and building relationships with the academic community, we are excited to announce the 39 recipients from North America, Europe and the Middle East. We offer our sincere congratulations to Google’s 2016 Class of PhD Fellows.

After more than four hours of tight play and a rapid-fire endgame, Google’s artificially intelligent Go-playing computer system has won a second contest against grandmaster Lee Sedol, taking a two-games-to-none lead in their historic best-of-five match in downtown Seoul.

The surprisingly skillful Google machine, known as AlphaGo, now needs only one more win to claim victory in the match. The Korean-born Lee Sedol will go down in defeat unless he takes each of the match’s last three games.

Watch DeepMind’s program AlphaGo take on the legendary Lee Sedol (9-dan pro), the top Go player of the past decade, in a $1M 5-game challenge match in Seoul. [hours and hours of video]

Is AlphaGo unstoppable? It felt like it today, as Google’s artificial intelligence crushed the best that humanity has to offer in Go. It wasn’t even close.

After yesterday’s historic round one loss, it may have been a restless night for legendary player Lee Sedol. Lee arrived in the arena noticeably worn, his eyes blinking.

The Google DeepMind challenge match will pit the world’s top player of the ancient Chinese board game against the world’s most sophisticated Artificial Intelligence programme. Here is everything you need to know about this clash between advanced technology and old-fashioned human wit.

Also:

All Matches – Google DeepMind Challenge Match: Lee Sedol vs AlphaGo (YouTube, Deep Mind)

Google’s AI Wins Pivotal Second Game in Match With Go Grandmaster (WIRED Business, March 10)

As the era of ‘big data’ unfolds, researchers are increasingly engaging with large, complex data sets compiled from heterogeneous sources and distributed across networked technologies. The nature of these data sets makes it difficult to grasp and manipulate their materiality. We argue that moments of breakdown – points at which progress is stopped due to a material limitation – provide opportunities for researchers to develop new imaginations and configurations of their data sets’ materiality, and serve as underappreciated resources for knowledge production. In our ethnographic study of data-intensive research in an academic setting, we emphasize the layers of repair work required to address breakdown, and highlight incremental innovations that stem from this work. We suggest that a focus on the breakdown–repair process can facilitate nuanced understandings of the relationships and labour involved in constituting data assemblages and constructing knowledge from them.

D4GX is Bloomberg’s Data for Good Exchange event.

Our “call for papers” is open and we are encouraging data scientists, academics and industry experts to share their success stories, challenges and visions for future applications of data science that can help solve problems for the social good.

New York, NY on Sunday, September 25, at Bloomberg Global Headquarters (731 Lexington Ave)

Deadline for paper abstract submissions is Friday, July 1.

Communications between embassies, government entities, and diplomats take the form of classified diplomatic cables. In 2010, over 150,000 of these cables were released by Wikileaks, a nonprofit organization that publishes classified government documents. The effect of the leak was twofold; not only did previously secret information become readily available, but now, the general population could glimpse into the inter-workings of diplomacy.

Last year, Arthur Spirling, an Associate Professor of Politics and Data Science at New York University, co-authored a paper titled, “Dimensions of Diplomacy,” regarding his research on these Wikileaks cables. We got the chance to ask him a few questions about his research, his findings, and the nature of governmental secrecy.

statsmodels is a Python module that provides classes and functions for the estimation of many different statistical models, as well as for conducting statistical tests, and statistical data exploration. An extensive list of result statistics are avalable for each estimator. The results are tested against existing statistical packages to ensure that they are correct. The package is released under the open source Modified BSD (3-clause) license. The online documentation is hosted at sourceforge.

In data science, models can involve abstract features in high dimensional spaces, or they can be more concrete, in lower dimensions, and more readily understood by humans; that is, they are interpretable. What’s the role of interpretable models in data science, especially when working with less technical partners from the business? When, and why, should we favor model interpretability?

The key here is figuring out the audience. Who is going to use the model and to what purpose? Let’s take a simple but specific example. Last week, I was working on a typical cold-start predictive modeling problem for e-commerce: how do you predict initial sales for new products if you’ve never sold them before?

As someone who creates open-source software, I spend a lot of time thinking about how to make software better.

This is unavoidable: there’s an unending stream of pleas for help on Stack Overflow, in GitHub issues and Slack mentions, in emails and direct messages. Fortunately, you also see people succeed and make fantastic things beyond your imagination, and knowing you helped is a powerful motivation to keep at it.

So you wonder: what qualities of software lead people to succeed or fail? How can I improve my software and empower more people to be successful? Can I articulate any guiding principles, or do I just have intuition that I apply on a case-by-case basis?

It can be challenging to understand the impact of your programs and whether or not they are functioning as intended. The Goldilocks Initiative exists to help solve these challenges by finding right-fit monitoring and evaluation (M&E) systems tailored to your organization. … [Innovations for Poverty Action] launched the Goldilocks Initiative to complement our traditional randomized evaluation work and help find the right-fit between collecting too much data that doesn’t get used and not collecting enough, to understand how to allocate limited funding for the greatest impact. The initiative provides resources and consulting services for organizations, donors, and governments in designing and supporting the implementation of cost-effective, appropriately-sized M&E systems.