NYU Data Science newsletter – August 8, 2016

GROUP CURATION: N/A

I’m a data science student at NYU’s Business School, so I see and talk to fellow students all the time about the careers they’re interested in, and unsurprisingly, data science is never at the top. Along those lines, and considering what drove me to choose data science as my major, I think that there are two key issues holding students back from being interested. If you’re a business looking to poach data talent straight out of college, you should read this.

In an ideal medical research landscape, clinical data would quickly be made available to all public and private researchers in the quest to speed up medical advances. The research community is getting closer to that ideal thanks, in part, to the work of the Yale Open Data Access (YODA) Project, but more work needs to be done, according to an article in the Aug. 4 New England Journal of Medicine.

The article on the current state of clinical trial data sharing is co-authored by Yale YODA Project leader Harlan Krumholz, M.D., and Johnson & Johnson chief medical officer Joanne Waldstreicher, M.D.

Forget telephoto lenses and fake mustaches: The most important tools for America’s 35,000 private investigators are database subscription services. For more than a decade, professional snoops have been able to search troves of public and nonpublic records—known addresses, DMV records, photographs of a person’s car—and condense them into comprehensive reports costing as little as $10. Now they can combine that information with the kinds of things marketers know about you, such as which politicians you donate to, what you spend on groceries, and whether it’s weird that you ate in last night, to create a portrait of your life and predict your behavior.

IDI, a year-old company in the so-called data-fusion business, is the first to centralize and weaponize all that information for its customers.

CuratedAI is a literary magazine with a twist–?all stories and poems are generated by machines using the tricks of the Artificial Intelligence trade. Editing, for now, is still the domain of us humans, but we aim to keep our touch as light as possible.

Using algorithmic rankers as an example, we argue that syntactic transparency alone will not lead to true algorithmic accountability (Angwin). This is true even if the complete input data is publicly available. We advocate instead for interpretability, which rests on making explicit the interactions between the program and the data on which it acts. An interpretable algorithm allows stakeholders to understand the outcomes, not merely the process by which outcomes were produced.

For the last 4 years, I’ve been through quite a startup journey at PasswordBox. We went through multiple phases: launching our consumer product, raising a Series A, improving our product and optimizing, growing to millions of users and eventually getting acquired.

Throughout the journey, I’ve been part of the data team where I’ve been doing all things data: analysis, presentations, engineering, training, etc. We started with a very rudimentary stack to track and understand our product usage. Over time, we improved our infrastructure and our tooling. We built dashboards, data pipelines, reports etc. We invested heavily on the data culture. We trained our colleagues to leverage our tools to make better decisions with data.

While we are now working harder than ever to improve our data stack, I think we are at a stage where we can take a step back and reflect on our past.

THINK of all the people with whom you interact during the course of a day, week, month and year. The many souls with whom you might exchange a greeting or give a warm embrace; engage in chitchat or have a deeper conversation. All those who, by some accident of fate, inhabit your world. And then ask yourself who among them are your friends — your true friends. Recent research indicates that only about half of perceived friendships are mutual. That is, someone you think is your friend might not be so keen on you. Or, vice versa, as when someone you feel you hardly know claims you as a bestie.

It’s a startling finding that has prompted much discussion among psychologists, neuroscientists, organizational behavior experts, sociologists and philosophers.

People are using Fitbits, smartphones and other wearable technology to track leisure goals like limiting computer use and working out more

When Uber decided to develop its own self-driving car, it went big. The company came to Carnegie Mellon University, the epicenter for autonomous driving research for three decades, and hired away four professors and 36 technical staff members.

A lot of news reports described that as “poaching.” I call it embracing the free market for brilliant people.

More academic appointments & awards:

CMU School of Computer Science – New Faculty Profiles [pdf] (August 08, Carnegie Mellon University, Andrew W. Moore)

Johns Hopkins Discovery Awards given to 24 cross-divisional research teams (August 05, Johns Hopkins University, Hub)

UMD Names Dr. Keith Marzullo as Dean of College of Information Studies (August 01, University of Maryland, UMD Right Now)

There is a movement happening within the health field for Americans to have access to their full medical record, including the notes doctors, nurses and other clinicians write after a visit. The engine behind this movement: OpenNotes.

OpenNotes hit an important milestone this week with 10 million Americans now able to read their medical notes securely online. Among the 10 million patients who can now access their notes online, more than three million are Veterans.

Data science can be defined as either the intersection or union of software engineering and statistics. In recent years, the field seems to be gravitating towards the broader unifying definition, where everyone who touches data in some way can call themselves a data scientist. Hence, while many people whose job title is Data Scientist do very useful work, the title itself has become fairly useless as an indication of what the title holder actually does. This post briefly discusses how we got to this point, where I think the field is likely to go, and what data scientists can do to remain relevant.

Quito, Ecuador In October 2016, 40,000+ people will meet in Quito, Ecuador for the largest cities summit in United Nations history – Habitat III. We are launching an open call for visualizations to find new ways of exploring cities and sustainable futures.

Deadline for the chance to be included in the Habitat III exhibition is Wednesday, August 31.

The Demo Expo at NYCML16 is an opportunity to meet representatives from industry, potential investors and collaborators for new media projects and ideas. It is the biggest pan-university NYC media showcase of the year. More than $25,000 in prizes will be awarded at NYCML16.

Deadline for submissions is Thursday, September 8.

Fed up with a ton of tutorials but no easy way to find exercises I decided to create a repo just with exercises to practice pandas. Don’t get me wrong, tutorials are great resources, but to learn is to do. So unless you practice you won’t learn.