NYU Data Science newsletter – February 23, 2016

GROUP CURATION: N/A

More companies are creating data science capabilities to enable competitive advantages. Because data science talent is rare and the demand for such talent is high, organizations often work with outsourced partners to fill important skill gaps. Here are a few reasons to consider outsourcing. What can go right and wrong along the way?

This is a crucial but often overlooked point in the debate around algorithms and the future of work: Most human jobs will not be replaced but rather reconfigured in the near future. We absolutely need to worry about the long-term implications on the demand for human labor and how this will affect the economy. But if we only focus on the question of whether and when humans will be replaced, we miss the impact algorithms are already having on work and the opportunities to make choices, as designers and consumers, about how algorithms can disrupt or enforce existing power dynamics in the future.

A recently released report from McKinsey’s Global Institute makes a similar point. Even as it suggests tantalizing statistics about the potential efficiency of automation, it points out that fewer than 5 percent of jobs, as in occupations, can be entirely automated in the near future. This stands in contrast to the widely reported 2014 Oxford study that warned that 47 percent of U.S. employment was at risk from automation. The McKinsey report concludes that the future effects of automation in the workplace should be examined from the perspective of job tasks, rather than jobs in and of themselves.

In a nondescript office building here, one of the world’s most powerful weather supercomputers quietly hums on a 24/7 mission to analyze billions of pieces of data that ultimately will tell you whether you need a sweater or sunscreen when you leave the house.

Forecasts, critical not only for your wardrobe choices but for ship captains, airline pilots and shipping companies, depend on sophisticated data crunching and computer models, but three years ago European models delivered a blow to the U.S. weather apparatus. The European weather models accurately predicted the path and strength of the devastating Hurricane Sandy that hit the New Jersey coastline and caused $65 billion in damage.

Now, the U.S. is on the rebound with this monumental supercomputer that collects, processes and analyzes billions of observations from weather satellites, weather balloons, airplanes, buoys and surface stations from around the world to help meteorologists make better weather forecasts.

The “Nation, Genre and Gender” research project at [University College Dublin] is currently creating a large digital corpus of Irish and English novels from the period 1800–1922. Our objective is to compare gender, genre and the nationality of the author or setting in shaping social structures in fiction, building on ideas from researchers such as Franco Moretti who have advocated the “distant reading” approach to studying literature from a macro level viewpoint. As part of this, we are looking at how techniques from Social Network Analysis (SNA), often applied to online networks such as Twitter and Facebook, can be applied to provide a new perspective on literary texts. … we are interested in exploring the social structures in 19th-20th century literature. Here I will outline how we go from the original text of a novel such as Oliver Twist by Charles Dickens to a visualisation of the final social network.

To take on Google and Amazon, CEO Nadella is sprinkling machine learning like fairy dust on everything Microsoft touches.

BIDS senior fellow and faculty director Saul Perlmutter will lead a team of 29 scientists from 15 institutions for the Department of Energy’s space telescope project. The team will “explore mysteries of dark energy, hunt for distant planets, [and] retrace universe’s history.”

With many educators pushing for students to use resources on the Internet with class work, the federal government is now grappling with a stark disparity in access to technology, between students who have high-speed Internet at home and an estimated five million families who are without it and who are struggling to keep up.

New York University’s Todd Gureckis, an associate professor in the Department of Psychology, has been awarded a Presidential Early Career Award for Scientists and Engineers (PECASE). The awards, announced by the White House, are the highest honor bestowed by the U.S. government on science and engineering professionals in the early stages of their independent research careers.

If you’re hankering for a snazzier way to read the posts on Hacker News (especially on mobile devices), here’s a roundup of 21 better ways to read Hacker News without the eyestrain.

Learning from data in order to gain useful predictions and insights. This course introduces methods for five key facets of an investigation: data wrangling, cleaning, and sampling to get a suitable data set; data management to be able to access big data quickly and reliably; exploratory data analysis to generate hypotheses and intuition; prediction based on statistical methods such as regression and classification; and communication of results through visualization, stories, and interpretable summaries.

We will be using Python for all programming assignments and projects. All lectures will be posted here and should be available 24 hours after meeting time.