NYU Data Science newsletter – July 11, 2016

GROUP CURATION: N/A

New software from researchers at MIT’s Computer Science and Artificial Intelligence Laboratory could make databases much easier for laypeople to work with. The program’s home screen looks like a spreadsheet, but it lets users build their own database queries and reports by combining functions familiar to any spreadsheet user.

Simple drop-down menus let the user pull data into the tool from multiple sources. The user can then sort and filter the data, recombine it using algebraic functions, and hide unneeded columns and rows, and the tool will automatically generate the corresponding database queries.

a startup in London has finally figured a genuinely useful application: brewing quality beers.

IntelligentX offers four basic beers, including a classic British golden ale, a British bitter kissed with grapefruit, a hoppy American pale ale and a smokey Marmite brew. Once you’ve tasted them, you can chat with the company’s Messenger bot to share your feedback, which its AI (built using IntelligentX’s own machine learning algorithm) uses to improve on its recipes.

Cortana Intelligence with Bing Predicts preview is an end-to-end consulting program that brings the power of Microsoft’s unique corpus of social, search and web data to let customers enrich and augment their Cortana Intelligence Suite solutions resulting in more accurate outcomes across a wide variety of business problems. The program springs from the highly successful and well-regarded Bing Predicts consumer experience, where Bing correctly predicted every knockout game of the 2014 soccer World Cup and 95% of the 2014 US mid-term elections. This program was born when we saw the opportunity to take the unique data assets that we have from our consumer businesses and help our commercial customers.

If there’s any promise I can make about the field of AI, it’s that the hype will always overtake the research. In this article, I won’t quibble over definitions, simply taking the broadest term as used in the media: artificial intelligence is whenever a system appears more intelligent than we expect it to be. This is in recognition of both the AI effect (where well defined applications of AI/ML are no longer considered intelligence) and that even if a system is simply an application of basic statistics, it is likely to be reported as AI if it appears intelligent.

As supercomputing centers look to future exascale systems, among the other pressing concerns (power consumption in particular) is adopting the right programming approach to scale applications across millions of cores.

And while this might sound like a big enough challenge on its own, it gets more complicated because it might just be that a new programming model (or system) might not be the scalability and performance answer either. It could just be that tweaking existing tools and methods can move programming evolution to programming revolution, that is, of course, if the supercomputing programmer community can agree.

The gig: Technology companies collect data about the pages people look at, the links they click on, and the things they search for. At Airbnb Inc., it’s Elena Grewal’s job to figure out what all that data means. Grewal, 31, is the interim lead for data science at the $25.5-billion San Francisco start-up.

You can hardly talk to a technology executive or developer today without talking about artificial intelligence, machine learning or bots.

While everyone agrees on the importance of machine learning to their company and industry, few companies have adequate expertise to do what they wanted the technology to do. Here are some insights into what we can expect in the coming years around ML and AI.

This year’s fellows are seeking to create, “a positive impact in the world,” (Thomas Dinsley), and, “work on meaningful projects that combine data and urbanism within a social-minded context,” (Rachael Dottle).

Within the confines of one of four projects, the fellows seek to develop strategies for improving urban issues such as unsafe food product consumption and transit inefficiency. “I am very excited to work on this project,” writes fellow Kaicheng Tan, who is working on a project to improve pedestrian wayfinding through crowdsourcing.

A team of scientists has developed a system that can forecast the outbreak of dengue fever by simply analyzing the calling behavior of citizens to a public-health hotline. This telephone-based disease surveillance system can forecast two to three weeks ahead of time, and with intra-city granularity, the outbreak of dengue fever, a mosquito-borne virus that infects up to 400,000 people each year.

“Thousands of lives are lost every year in developing countries for failing to detect epidemics early because of the lack of real-time data on reported cases,” observes Lakshminarayanan Subramanian, a professor at New York University’s Courant Institute of Mathematical Sciences and part of the research team. “We think our technique can be of use to public-health officials in their fight against the spread of crippling diseases.”

Facebook isn’t in the wireless business. But it continues to build all sorts of new-fangled wireless hardware.

Today, Mark Zuckerberg and company unveiled a creation they call OpenCellular. This is a Sunday-dinner-platter-sized hardware device that attaches to a tree or a street lamp or a telephone pole, and from there it can drive a wireless network, including traditional 2G cell-phone networks, higher speed LTE cellular networks, and smaller Wi-Fi networks like those inside your home, office, or local coffee shop.

Machine learning can provide an easier way to program computers to do things that are incredibly difficult to code by hand. For example, a deep learning neural network can be trained to recognize dogs in photographs or video footage with remarkable accuracy provided it sees enough examples. The flip side is that it can be more complicated to understand how these systems work.

In an article for Science, two Stanford researchers in collaboration with a Stanford alumnus now at Purdue have revealed a new and more useful approach for describing and, ultimately, managing complex networks. Their method simplifies what would otherwise be a daunting undertaking by envisioning the whole as a series of “motifs” – modules comprised of smaller network chunks.

“In the past few years, there has been progress in describing networks through ‘nodes’ and ‘edges,’” said Jure Leskovec, associate professor of computer science and senior author of the Science article.

Barcelona, Spain The 11th WiML Workshop is co-located with NIPS on Monday, December 05, 2016.

Deadline for abstract submission is Friday, August 26.

ICON is a comprehensive index of research-quality network data sets from all domains of network science, including social, web, information, biological, ecological, connectome, transportation, and technological networks.

Project Malmo is a platform for Artificial Intelligence experimentation and research built on top of Minecraft. We aim to inspire a new generation of research into challenging new problems presented by this unique environment.

At Metamarkets we run one of the largest production Druid clusters out there, so when it comes to scalability, we are almost always the first ones to encounter issues of running Druid at scale. Sometimes, however, performance problems are much simpler, and the downside of a large cluster is that it tends to average out problems that are hiding in plain sight, making them harder to pinpoint.

Recently, we started noticing that, despite being able to scale our cluster almost horizontally, performance would not always increase accordingly. While we don’t expect a linear increase in speed, some of the numbers were still puzzling. Initially, we attributed those discrepancies to other known bottlenecks in our system but we were not fully satisfied with that explanation. It turns out, however, we had a much simpler problem that goes back to the way we distribute the data throughout our cluster.

A simple website with just an overview of all possible calls in Elasticsearch.