NYU Data Science newsletter – May 11, 2016

GROUP CURATION: N/A

Students taking an online course at Georgia Tech’s School of Interactive Computing were duped into thinking one of their teaching assistants, named Jill Watson, was an actual human. And how can you blame them—the virtual TA managed to answer many of their questions with 97 percent certainty.

Biden took to the stage at Health Datapalooza not just to share his own experiences, but to put out a call to action: While the government has taken great strides to increase access to technology-enabled healthcare, it’s still not enough, he said.

More hospitals, researchers, scientists and providers need to “open access to their data to prevent cancer,” said Biden.

Technology that helps coaches and athletes fine-tune performance? An app that can streamline the modular-construction process? A low-cost Internet-enabled diagnostic tool with the potential to revolutionize healthcare in developing nations? Each one is inarguably a very good idea.

You might be surprised to know that these innovations didn’t stem from the minds of seasoned engineers with decades of experience. Much to the contrary, each was developed by a team of students competing in the 2016 InnoVention Competition at NYU Tandon.

When the DNA sequence of the human genome was revealed in 2000, many people expected it to start a revolution. Researchers would be able to discover the genes that caused or influenced diseases. And drug companies would be able to use that knowledge to create better medicines. Until recently, though, it has been a case of “revolution postponed”. The flood of promised discoveries has been more like a trickle.

Much of the reason for the unfulfilled promises was naivity about how straightforward the link between different versions of genes and particular diseases would be. But that naivity has gone, and the fact that complex illnesses often have contributions from large numbers of genes is now recognised. This recognition, plus better computing and sequencing power, mean researchers are indeed beginning to pick the relationships between genes and disease apart.

Teaching statistics in a spectacular and humorous way. It seems impossible, however, Swedish professor Hans Rosling (67) proofs it isn’t. His mission: to correct our outdated worldview.

Last month, the JPMorgan Chase shareholders’ letter went up online, and the bank’s chief executive, Jamie Dimon, had some harsh words for start-ups selling services that try to improve upon his bank’s offerings.

To the Mints and the Acorns and the Pennys and the scores of other services and their partners that use Chase checking and credit card data in their work — and, more important, to their customers — he declared the following:

These start-ups take more of your data than they need to.

Many of them sell the data to outsiders in a way that benefits them but not you.

They often take your data every day, for years, even if your account is inactive.

If your money disappears because of fraud, it’s on you, not the bank.

Recorded along the lovely Boston Harbor (enjoy the outdoor sounds), in this week’s episode of The PolicyViz Podcast, I chat with Lane Harrison, Assistant Professor in the Computer Science Department at Worcester Polytechnic Institute. We not only talk about Lane’s interesting research in data visualization–including how emotions affect our perception of graphs and content–but also the current state of the field, Github, and Amazon’s Mechanical Turk. [audio, 22:42]

This post is a discussion of Nick Bostrom’s book Superintelligence. The book has had an effect on the thinking of many of the world’s thought leaders. Not just in artificial intelligence, but in a range of different domains (politicians, physicists, business leaders). In that light, and given this series of blog posts is about the “Future of AI”, it seemed important to read the book and discuss his ideas.

A special report on gentrification (PDF) by the NYU Furman Center explores gentrification within the context of New York City’s neighborhoods. Of the city’s 55 neighborhoods, the report classifies 15 as “gentrifying” and analyzes how their housing and population have changed over the past two decades.

The analysis was released as part of the NYU Furman Center’s 15th annual State of New York City’s Housing and Neighborhoods in 2015 report, which provides a compendium of data and analysis about the city’s housing and neighborhoods.

Open access to data is commonly required by funding agencies, journals, and public policy, despite the lack of
agreement on the concept of “open data.” We present findings from two longitudinal case studies of major
scientific collaborations, the Sloan Digital Sky Survey in astronomy and the Center for Dark Energy Biosphere
Investigations in deep subseafloor biosphere studies. These sites offer comparisons in rationales and policy
interpretations of open data, which are shaped by their differing scientific objectives. While policy rationales and
implementations shape infrastructures for scientific data, these rationales also are shaped by pre-existing
infrastructure. Meanings of the term “open data” are contingent on project objectives and on the infrastructures
to which they have access.

In the early days of the Internet, scientists erected their own online network, a digital utopia that still stands today. Here, astronomers, physicists, mathematicians, computational biologists, and computer scientists come together to discuss heady, cosmic topics. They exchange knowledge—without exchanging money. It’s called arXiv, and it’s where researchers go to post their ideas for discussion, sharing PDFs of their scientific articles before they’re locked behind a journal’s paywall.

ArXiv is about to celebrate its 25th birthday. It can now officially rent a car without paying extra, and that means it has to grow up and start thinking about its future. The repository still excels at its primary goal—to quickly and freely disseminate papers about black holes, baryons, and Bayesian statistics—but it runs on old legacy code. “Under the hood, the service is facing significant pressures,” says Oya Rieger, arXiv’s program director.

For the past few weeks, whenever I’ve found myself looking for a place to eat or drink, I’ve turned to Ozlo, a bot on my phone, for help. Meeting up with a friend one Sunday morning, I wrote “coffee nearby,” and he quickly returned with a list of local coffee shops. When I asked Ozlo to show me the shops on a map, he dropped one into the conversation. (By the way, Ozlo is male in gender according to his creators.)

Ozlo is another in a rapidly lengthening conga line of AI-powered chatbots. More than two years in the making, he currently specializes in food and drink. Flush with with $14 million in venture capital funding from the likes of Greylock Partners and Yahoo co-founder Jerry Yang’s AME Cloud Ventures, Ozlo is designed to field most any question about dining out — not just where to eat, but what.

Data science brings new understanding to healthcare and other fields. But does healthcare want to be told something different from such an understanding?

The MGM Grand Garden Arena in Las Vegas is a 16,800-seat venue that frequently hosts Ultimate Fighting bouts, rock stars and pop idols. On a Tuesday morning last October the arena was thrumming yet again, except this time the star was a young man in a T-shirt and jeans with a giant number-filled spreadsheet projected behind him. The developer from Tableau Software merely had to show off a new feature that automatically combines two separate data sources into one row and thousands of spectators erupted in applause.

This kind of over-the-top geek-out is now fairly normal stuff for Tableau cofounder and chief scientist Pat Hanrahan, who reflected on his employee’s performance a few hours later: “There are so many people here now using data–and enjoying it. If I were to say that on day one, you would have laughed. No one believed us that data was cool.”

Submissions are solicited for the Thirtieth Annual Conference on Neural Information Processing Systems, an interdisciplinary conference that brings together researchers in all aspects of neural and statistical information processing and computation, and their applications.

Barcelona, SpainMonday-Saturday, December 5-10, NIPS 2016 will be at Centre Convencions Internacional.

Deadline for Paper Submissions is Friday, May 20.

Previously Rong’s post and Ben’s post show that (noisy) gradient descent can converge to local minimum of a non-convex function, and in (large) polynomial time (Ge et al.’15). This post describes a simple framework that can sometimes be used to design/analyse algorithms that can quickly reach an approximate global optimum of the nonconvex function. The framework —which was used to analyse alternating minimization algorithms for sparse coding in our COLT’15 paper with Ge and Moitra—generalizes many other sufficient conditions for convergence (usually gradient-based) that were formulated in recent papers.

There are many technologies for streaming data: simple event processors, stream processors, and complex event processors. Even within the open-source community there is a bewildering amount of options with sometimes few major differences that are not well documented or easy to find. That’s why I’ve decided to create an overview of Apache streaming technologies, including Flume, NiFi, Gearpump, Apex, Kafka Streams, Spark Streaming, Storm (and Trident), Flink, Samza, Ignite, and Beam.

Nobody doubts the value of data today. The USPTO has been in the business of “open data” since the very beginning of our agency’s history. We created this portal to improve the discoverability, accessibility, and usability of public patent and trademark data to harness the power of data. The Developer Hub establishes a shareable, and “social” platform, for anyone in this community to showcase unique ways they’re using our data, combining it with other data sets, such as economic and geographic data. Through this forum users can leverage the power of the crowd to unlock our data to answer questions about trends in technology and innovation but also to provide input on other types of data sets we should release.

We will soon be releasing APIs so that anyone with the most basic programming experience will be able to explore our data according to their own interests, curiosity, and business needs. This makes it easier for innovators to further mine this data, helping to inform our customers where to spend their limited research and development resources, and providing a much more detailed view of the competitive landscape than previously available.

Quickly dive-in to the most current topics in the field of deep learning, with this curated collection from Strata + Hadoop World 2016. This video features presentations from speakers in both academia and industry, covering the essentials of deep learning, and how to use the latest tools and techniques.