NYU Data Science newsletter – June 15, 2015

GROUP CURATION: N/A

Sparkit-learn aims to provide scikit-learn functionality and API on PySpark. The main goal of the library is to create an API that stays close to sklearn’s.

Sometimes predictions may be out of whack despite best tuning efforts given available resources. When this happens, what are the steps one must take in order to improve the results?

We analyze the associations between the top Big Data, Data Mining, and Data Science tools based on the results of 2015 KDnuggets Software Poll. Download anonymized data and analyze it yourself.

… Earlier this year Complete Genomics (now acquired by BGI) announced they would launch two sequencer systems, one for small jobs and one for “nation-scale” genome sequencing. At the European Human Genetics Conference this weekend, the big unit was announced. Unfortunately, BGI hasn’t seen it appropriate to reach out to the blogging community, nor are there a wealth of details. But some general characteristics of the machine can be grasped.

First, it is an end-to-end beast. Included in the $12M pricetag are robots for sample preparation and library prep, which are fully automated. The instrument has a rated throughput of 10K genomes per year, with planned expansion to 30K/year.

… As DNA data flood in, researchers say, the mass-genome approach will paint an increasingly accurate picture of the past and show how ancient events shaped modern humanity — from what we eat to the diseases that ail us. “Christ, what does this mean?” says Greger Larson, an evolutionary geneticist at the University of Oxford, UK. “In another five years, we’ll be talking about tens of thousands of ancient genomes.”

Genomics have been heralded as ushering in the era of personalized medicine and enabling the development of more targeted and precise therapies. As the industry continues to work toward a stronger approach to personalized medicine—in other words, really being able to stratify patients and match treatments to the right patients at the right time—the use of tissue data in conjunction with genetic data is becoming ever more important. This is particularly true in oncology, where tissue remains the primary factor in cancer diagnosis, tumor heterogeneity, and drug behavior.

The realization of true personalized medicine will ultimately come from the combination and correlation of genomic information, tissue imaging data, and clinical outcomes. In particular, a big data approach to biomarker discovery and clinical diagnostic assay development in oncology will help enable the discovery and development of precise diagnostics for more effective therapies.

When some of the world’s most advanced rescue robots are foiled by nothing more complex than a doorknob, you get a good sense of the challenge of making our homes and workplaces more automated.

At the DARPA Robotics Challenge, a contest held over the weekend in California, two dozen extremely sophisticated robots did their best to perform a series of tasks on an outdoor course, including turning a valve, climbing some steps, and opening a door (see “A Transformer Wins DARPA’s $2 Million Robotics Challenge”). Although a couple of robots managed to complete the course, others grasped thin air, walked into walls, or simply toppled over as if overcome with the sheer impossibility of it all. At the same time, efforts by human controllers to help the robots through their tasks may offer clues as to how human-machine collaboration could be deployed in various other settings.

The official map of the human genome can’t tell you everything about your genes. Does graph theory offer a better way?

1st day of Kensho hackathon, Goldman team wore suits. On 3rd day, hoodies & Beats headphones http://nyti.ms/1Gc8CZb

On Monday, Apple revealed a more inquisitive Siri for iOS 9—a direct competitor to Android’s Google Now. Each smartphone feature seeks to anticipate your next move by offering up suggestions, directions, and the like. Hilary Mason, who is founder and CEO of Fast Forward Labs, says Siri and Google Now are examples of “companies using all of the data they’ve gathered about us [to] actually make our lives better.” She joins Ira to discuss how much insight our smartphones have into our behavior, and whether or not there are limits to what they can predict about us. [audio, 7:53]

Hundreds of thousands of Serengeti “selfies” are giving researchers a candid, and often amusing, picture of what life is like on the African plain.

Zebras munch on dried grass, lion cubs play with their mother, elephants go for a stroll, and birds hitch a ride on the back of a wart hog.

The images were captured by 225 “camera traps” set up throughout Tanzania’s Serengeti National Park. The cameras took a picture when they detected motion nearby. Often, the photos were snapped with the animals staring directly into the lens.

In our latest From the Field, Frontline SMS CEO Sean McDonald talks about their new environmental sensor, designed to bridge the environmental data divide with open hardware.

It’s official: Kyunghyun Cho is joining NYU as an Assistant Professor in the Computer Science Department and the Center for Data Science.

Kyunghyun will be reinforcing our already-considerable strength in deep learning, and more generally our efforts in AI, machine learning and data science.

He will be joining us following his post-doctoral fellowship at MILA, the Montréal Institute for Learning Algorithms, founded and directed by Yoshua Bengio at Université de Montréal.