NYU Data Science newsletter – August 30, 2016

GROUP CURATION: N/A

… while IoT companies tout their successes, the truth is that many IoT projects fail. Reasons include high start-up costs; lengthy, complex projects; and enormous amounts of data to capture and analyze.

Also, in trying to realize value from their data, businesses might not know where to start. An accelerator built around specific use cases—for example condition-based monitoring—establishes the value proposition right away, rather than having the business spend a lot of capital on software and infrastructure just to get a seat on the “Big Data” bandwagon, with no idea where they’re going.

It appears that more consolidation is on the way for the analytics ecosystem. An anonymous insider has told VentureBeat that SAP SA is in the final steps of closing a $125-million-plus agreement to acquire Altiscale Inc., a Palo Alto-based startup that provides managed data processing services.

The outfit started out in 2012 with a hosted implementation of Hadoop and later added a premium edition called Altiscale Insight Cloud that layers business intelligence functionality on top of the system.

In this episode of the O’Reilly Data Show, I spoke with Adam Marcus, co-founder and CTO of B12, a startup focused on building human-in-the-loop intelligent applications. We talked about the open source platform Orchestra,for coordinating human-in-the-loop projects; the current wave of human-assisted AI applications; best practices for reviewing and scoring experts; and flash teams. [audio, 44:01]

AS BOSSES go, Linus Torvalds and Andy Jassy couldn’t be more different. Mr Torvalds works, often in his bathrobe, out of his home in Portland, Oregon. He leads an army of volunteer developers whose software can be had for nothing. The office of Mr Jassy, who usually sports business casual, is in a tower in Seattle. His employees operate dozens of huge data centres around the world and work to create new online services that his firm can charge for.

Yet their organisations share an anniversary and an intertwined history. On August 25th 1991 Mr Torvalds asked other developers to comment on a computer operating system he had written, which became known as Linux. It has since become the world’s most-used piece of software of this type. On the same day in 2006 Mr Jassy’s team made available a beta version of “Elastic Compute Cloud” (EC2), the central offering of Amazon Web Services (AWS), the cloud-computing arm of Amazon, an e-commerce giant. Over the past 12 months the division racked up sales of $11 billion.

These two organisations have been central to the rise of cloud computing, the provision of all kinds of number-crunching services over the internet.

“Several years of research backed by two extensive studies involving thousands of participants convinced an international team of music psychologists that personality plays a much bigger part in musical preferences than anyone had ever imagined. Bigger even than age, gender, culture, or education.”

More data + psychology:

CrowdEmotion Case Study: Spotify Music Mashup Video (August 26, Medium, Sense Makers, Steven Mulvey)

Ode to Recall: To Remember Events in Order, We Rely on the Brain as a Symphony (August 26, NYU News, describes research by Lila Davachi)

Instagram Probably Can’t Predict Depression. GPS, Though… (August 24, WIRED, Science, Nick Stockton)

New mental health technology tells your doctor what you won’t (August 23, ReadWrite, David Curry)

Insurance companies have their IT work cut out for them. This article provides a framework for their efforts that is based on three questions:

What are the main technological building blocks of a digital insurer?

What emerging architectural strategies can help insurers accommodate future developments in technology?

How do incumbent insurers jump-start digital implementation and stay ahead of the competition?

Big data is hardly new at this point — nor has it wrought anywhere near its potential effects on many companies and institutions insulated by inertia and red tape. A summer program at the University of Washington called Data Science for Good shows that fresh eyes and good code can make more in 10 weeks than some have done in as many years.

It’s all part of UW’s eScience Institute, and the program is based on similar ones at the University of Chicago and Georgia Tech. Teams of students from around the country apply to take part, then travel to Seattle and work with experts who connect them with data and tools.

Yandex may not have quite the same name recognition it does in Russia, but the Russian search engine positively dominates its domestic search market — in 2015, it handled 57.3 percent of the country’s web queries.

But while search is the company’s bread and butter, Yandex has branched out in recent years — it offers antivirus software, a real-time bidding platform, a topographical mapping solution, language translation, and more. And this week, it dipped a toe into yet another territory: user feedback interpreted by machine intelligence.

Yandex said the new technology, which it began rolling out on Tuesday, is an attempt to build a “better relationship” with its users. It’s debuting in the form of a complaint button within its web browser for Android.

Securing an IoT device’s communication with cloud services from end to end can be a daunting challenge. One of the biggest problems is the assignment, protection, and management of public decryption keys and certificates for authentication of deployed devices. To simplify those efforts for developers, Microchip in conjunction with Amazon Web Services (AWS) has developed a secure provisioning platform and associated support chip that streamline IoT security.

We’ve designed a distributed system for sharing enormous datasets – for researchers, by researchers. The result is a scalable, secure, and fault-tolerant repository for data, with blazing fast download speeds.

A lot of machine learning tutorials are geared toward Mac. One major advantage of using Linux is it’s free and it supports using TensorFlow with your GPU. The accelerated parallel computing power of GPUs are one of the reasons for such major advancements in machine learning.

“In this post I’m going to go over some critical packages that I use almost every time I work in R, and their counterpart(s) in Python.”

“Currently, Amazon Web Services is the undisputed cloud leader, with more than 30 percent of the infrastructure as a service (IaaS) market according to Synergy Research Group. Microsoft Azure experienced 100 percent revenue growth during the second quarter, and now has a firm hold on second place with more than 10 percent of the market.”

In machine learning, computers apply statistical learning techniques to automatically identify patterns in data. These techniques can be used to make highly accurate predictions.

The U.S. Department of Transportation is releasing an open data set that contains detailed, anonymized information about each of these tragic incidents. As the new data being released show, and as DOT reported earlier this summer, 2015 showed a marked increase in traffic fatalities nationwide.

To be precise, 7.2% more people died in traffic-related accidents in 2015 than in 2014. This unfortunate data point breaks a recent historical trend of fewer deaths occurring per year.