NYU Data Science newsletter – April 29, 2015

GROUP CURATION: N/A

I recently came across a good and brief explanation on Quora by Mateusz Opala – “How to become a Data Scientist” and we share it here, with added links.

Elasticsearch is booming. Together with Logstash, a tool for collecting and processing logs, and Kibana, a tool for searching and visualizing data in Elasticsearch (aka, the “ELK” stack), adoption of Elasticsearch continues to grow by leaps and bounds. When it comes to actually using Elasticsearch, there are tons of metrics generated. Instead of taking on the formidable task of tackling all-things-metrics in one blog post, I’ll take a look at 10 Elasticsearch metrics to watch. This should be helpful to anyone new to Elasticsearch, and also to experienced users who want a quick start into performance monitoring of Elasticsearch.

One of the great things about R is that there’s so much available to use with it: there are several interfaces to choose from, thousands of add-on packages to extend its capabilites, hundreds of books and on-line tutorials — an abundance of riches to improve your R experience. But with that abundance comes a problem: how to find the best add-ons to R.

Qin Wenfeng has taken the trouble to curate the best add-ons to R in their list, awesome-R: A curated list of awesome R frameworks, packages and software. The list provides several (but not too many!) recommendations for R users in the areas of IDEs, data manipulation packages, database integration frameworks, machine learning suites, R-related websites, and much more.

It was always kind of shady that Facebook let you volunteer your friends’ status updates, check-ins, location, interests and more to third-party apps. While this let developers build powerful, personalized products, the privacy concerns led Facebook to announce at F8 2014 that it would shut down the Friends data API in a year. Now that time has come, with the forced migration to Graph API v2.0 leading to the friends’ data API shutting down, and a few other changes happening on April 30.

Twitter wants to make it simpler for digital advertisers to get personal with potential customers. What’s more, it doesn’t care much whether prospects are using a Web browser or a mobile device to research a purchase. Or both.

The social network plans to pay an undisclosed sum for TellApart, a six-year-old marketing software company that drives “hundreds of millions in annual revenue” for retailers like Neiman Marcus, Brookstone, Pottery Barn, and eBags. If all goes as plans, the deal should close by June 1.

The outlook for data scientists: less hype, more hiring.

The exuberance surrounding big data has passed its peak and is trending down, the technology research firm Gartner declared last August in its annual “hype cycle” report on perceptions of technology.

Perhaps, but it remains a rising market for data scientists. Salaries rose 8 percent on average in the last year, with bonuses adding $56,000, according to a salary and employment survey released on Tuesday by Burtch Works, a recruiter of professionals with quantitative skills.

Clarkson University will offer a new master of science degree in the highly sought-after field of data analytics, beginning this fall. The new graduate degree was approved by the New York State Education Department earlier this month for Clarkson, Potsdam, N.Y.

In Rwanda, people have to deal with all kinds of threats to their health: malaria, HIV/AIDS, severe diarrhea. But in late 2012, Agnes Binagwaho, Rwanda’s Minister of Health, realized her country’s key health enemy was something far more innocuous. The thing causing the most harm to her people, the leading risk factor for premature death and disability, was inside their own homes: Dirty indoor air, from cooking food over burning dung and vegetation in poorly ventilated huts. Within weeks, Binagwaho announced a program to distribute one million clean cookstoves to the poorest households in the young, mostly rural country.

Binagwaho was able to improve millions of lives thanks to a new kind of medical record-keeping, only possible in this era of big data.

Insurance giant Humana Inc., whose earnings are under pressure from rising medical costs, has a prescription it hopes will bring it relief: helping its 13.8 million members improve their own health. … Across the street from its headquarters here, the insurer has set up a mobile-apps lab that designs tools like HumanaVitality, which lets people set personal health targets and challenge themselves or one another to fitness goals.

For my money, the #1 neglected topic in statistics is measurement.