NYU Data Science newsletter – September 15, 2015

GROUP CURATION: N/A

The U.S. Department of Education just released an amazing dataset about the costs of going to college, earnings potential, etc. They’re doing so many things right, it’s really great. But what is still lacking is the metadata, making analysis harder than it needs to be.

… At Trifacta, we think about data wrangling as a process that includes six core activities:

Discovering

Structuring

Cleaning

Enriching

Validating

Publishing

When dealing with data, it helps to have a well defined workflow. Specifically, whether we want to perform an analysis with the sole intent of “telling the story” (Data Visualisation/Journalism) or build a system that relies on data to model a certain task (Data Mining), process matters. By defining a methodology in advance, teams are in sync and it is easier to avoid losing time trying to figure out what the next step should be. This enables a faster production of results and publication of materials.

With that in mind, and following the previous blogpost about the Ashley Madison leak data analysis, we saw an opportunity to show the workflow that we are currently using. This workflow is used not only to analyse data leaks (such as the case of AshMad), but also to analyse our own internal data.

… Beyond the project home pages and documentation, there are several excellent sources available to teach the core concepts behind machine learning. While there are hundreds (even thousands) of books and tutorials on ML, I’ve tried to focus on those targeted towards programmers and less on those that are more rigorous or focused too much on the math behind the scenes. While that stuff is important in the long run, it is often impedes engineers in the getting started phase from trying out real systems with real data.

Students who aren’t already planning to major in science often regard it as a waste of their time … This approach is a major mistake, and having offered some advice to future science majors, let me offer some encouragement for non-scientists facing the prospect of having to take science in college. There are lots of reasons why you should take science, or at least shouldn’t avoid it; here are a few.

… Web and mobile apps are getting smarter and smarter about each of us and recommending things to us that until recently we had to figure out all by ourselves. It almost seems like recommenders are table stakes these days. You can’t even play in the game unless you can do this sort of thing. And that requires a data science team to sift through all the data on your service and make smart recommendations to your users.

The Global Burden of Disease, Injuries, and Risk Factor study 2013 (GBD 2013) is the first of a series of annual updates of the GBD. Risk factor quantification, particularly of modifiable risk factors, can help to identify emerging threats to population health and opportunities for prevention. The GBD 2013 provides a timely opportunity to update the comparative risk assessment with new data for exposure, relative risks, and evidence on the appropriate counterfactual risk distribution.

There’s a playlist on Spotify I love called Discover Weekly. It’s updated every Monday with a mix of songs, some I know and some I’ve never heard, crossing into almost every genre with no discernible pattern. Like magic, it just knows what I want to hear.

It’s one of the reasons why I’m listening to Spotify more than ever. And I’m not alone.

The convergence of budget EEG gear and big-data analysis tools is leading to a revolution in DIY brain research for mind-reading technology.

Intelligent Machines is a BBC News series looking at AI and robotics

The quest for artificial intelligence (AI) began over 70 years ago, with the idea that computers would one day be able to think like us. Ambitious predictions attracted generous funding, but after a few decades there was little to show for it.

But, in the last 25 years, new approaches to AI, coupled with advances in technology, mean that we may now be on the brink of realising those pioneers’ dreams.

Sir Nigel Shadbolt, Professor of Artificial Intelligence at Southampton University, believes in the power of open data. With Sir Tim Berners-Lee he persuaded two UK Prime Ministers of the importance of letting us all get our hands on information that’s been collected about us by the government and other organisations. But, this has brought him into conflict with people who think there’s money to be made from this data. And open data raises issues of privacy.

Nigel Shadbolt talks to Jim al-Khalili about how a degree in psychology and philosophy lead to a career researching artificial intelligence and a passion for open data. [audio, 27:46]

Last Friday, NYU’s Center for Data Science welcomed two companies, TEK systems, and Medidata Solutions to the first weekly set of career information sessions. These sessions give students networking opportunities to find how their Data Science coursework and internships are applicable in various careers.

Having sponsored the Facebook smartUP last Spring, as well as sponsoring two more smartUps taking place this Fall and Winter, Medidata has continued to partner with the NYU on multiple fronts. Even one of the Masters of Data Science students from the first graduating class, Patricia Allen, began working as a statistical analyst in June after interning for Medidata Solutions during her studies at the Center for Data Science.

As part of the Moore-Sloan Data Science Initiative’s ongoing commitment to promoting diversity, we are highlighting the work of 5 exceptional women in the field of data science. The first profile in our series is on Katy Huff, a Berkeley Institute for Data Science, Moore/Sloan fellow. … Kathryn Huff is using computer modeling and simulation to eliminate human error in nuclear reactions, and find safer ways to continue the practice of nuclear energy.