Data Science newsletter – November 2, 2016

GROUP CURATION: N/A

Even in analytically sophisticated organizations, machine learning often operates in “silos of expertise.” For example, the financial crimes unit in a bank may use advanced techniques to catch anti-money laundering; the credit risk team uses completely different and incompatible tools to predict loan defaults and set risk-based pricing; while treasury uses still other tools to predict cash flow. Meanwhile, customer service and branch operations do not use machine learning at all because they lack the critical mass of specialists and software.

Here’s a keynote speech that Commissioner Kara Stein of the Securities and Exchange Commission gave to the “Big Data in Finance Conference” last week, titled “A Vision for Data at the SEC.” It’s about how the SEC does and should use data to catch crimes and prevent crashes and so forth. But it seems to me that Stein raises and then passes over the most interesting questions.

The U.S. Army is coming to grips with a problem faced by the intelligence community: finding actionable insight in a rising tide of data. Artificial intelligence promises to help, though how and when AI will be factored into military requirements is an open question.

The number of sensors and communications modes now deployed are drowning soldiers in data and making it difficult to make decisions. Lieutenant General Robert Ashley Jr., the Army’s intelligence chief, wants to use AI to help soldiers process data and share it with partners, according to ExecutiveGov.

For the past few election cycles, it’s become generally understood that single surveys can be noisy and that averaging across polls gives a better sense of trends in public opinion. But how do you do your averaging? Our current election forecasts and poll aggregators fall in one of two camps: either transparent and simple (for example, the moving averages of RealClearPolitics) or complicated with proprietary “special sauce” (for example, whatever Nate Silver is currently doing).

But the model I’m talking about here is the best of both worlds. It includes state and national poll data, models of correlations between states, a pre-election forecast, state-level and electoral college forecasts, and pretty graphics—but all in an open-source framework, so anyone can check any part of the data or model.

A new center at the University of Chicago will make connections in areas as diverse as health care, real estate and social networks by focusing on space.

The Center for Spatial Data Science is concerned with geography, proximity and other location-based measures that hold great promise for unlocking discoveries from data across many fields. While methods based on time have long dominated statistical analysis, the opening of the center—and arrival of its director Luc Anselin, the Stein-Freiler Distinguished Service Professor in Sociology—will expand the use of spatial data science in research across campus.

If silicon has its limits, ingenuity may not. Better algorithms and new kinds of hardware circuits could help scientists continue to make computers that can do more and at a lower cost.

“It’s been a fun ride,” said Thomas M. Conte, an electrical engineer at the Georgia Institute of Technology. “Today you’re entering this patchwork world where you are going to find a better solution for a particular problem, and that’s how we’re going to advance in the future.”

As electronic medical records (EMRs) began to enter the scene, clinicians were asked to document their care of patients in more detail than ever before. The goals for this effort were to improve communication among caregivers and to inform payment reimbursement. But as with the asthma trial above, these data are having another life.

Today, EMR data are being used to predict domestic abuse, characterize genetic risks for rheumatoid arthritis and non-responsiveness to antidepressants, as well as to identify distinct subgroups of children with autism. Unknown signals can often lurk in a mountain of electronic data.

Enthusiasm for precision medicine, from the White House down to everyday physicians, is at an all-time high. But serious problems with the databases used to interpret patients’ genetic profiles can lead to “inappropriate treatment” with “devastating consequences,” researchers at the Mayo Clinic warned on Monday.

Their report describes the cases of some two dozen people who were told they had a potentially fatal illness and one who had a heart defibrillator surgically implanted but, it turns out, never needed it. The individuals were family members who underwent genetic testing after a young relative died of a heart syndrome. Test results indicated that they carried a mutation in a heart-related gene — and the database that the testing company used indicated it caused a serious disorder.

The pace of change these days means that L&D staff simply can’t create all the content that learners require. The good news is there is a huge amount of content readily available on the web that can be accessed and made available to support learners. Let’s look at what it takes to be a great content curator for learning, and some resources that can help you.

Whether you decide to build or buy, you can benefit from the lessons we learned while building our own real-time pipeline. At least that way you’ll be armed with what you need to meet the challenge.

Lesson #1: With analytics data, scale matters

This site contains links to interactive demos, websites, evaluation data, and other complementary material about research presented at the IEEE VIS conference, Oct 2016, in Baltimore. We hope this website helps you finding inspiration in browsing all the amazing output the scientific visualization community is producing — besides publishing scientific papers.

For a number of years, I led the data science team for Google Search logs. We were often asked to make sense of confusing results, measure new phenomena from logged behavior, validate analyses done by others, and interpret metrics of user behavior. Some people seemed to be naturally good at doing this kind of high quality data analysis. These engineers and analysts were often described as “careful” and “methodical”. But what do those adjectives actually mean? What actions earn you these labels?

To answer those questions, I put together a document shared Google-wide which I optimistically and simply titled “Good Data Analysis.” To my surprise, this document has been read more than anything else I’ve done at Google over the last eleven years. Even four years after the last major update, I find that there are multiple Googlers with the document open any time I check.