NYU Data Science newsletter – May 11, 2015

GROUP CURATION: N/A

This extension adds a browser button for easy access to Google Scholar from any web page.

Back in 2012, I carried out a survey to find out which Python, NumPy, and SciPy versions scientists are currently using for their daily work, in order to better understand which versions should be supported. The main finding was that a large fraction of people have reasonably up-to-date Python installations, although virtually no-one was using Python 3 for daily work.

This year, I decided to repeat the experiment: last January I advertised a survey which asked users to provide information about their Python installation(s) for research/production work, as well as more general information about their Python experience, which packages they used regularly, why they are not using Python 3 if they were still using Python 2, and so on.

Yesterday, the journal Science released a study by Facebook employees examining what content you do (and don’t) see on Facebook’s news feed. Its conclusion, at first glance, was that the Facebook news feed algorithm does not keep users from seeing opinions they disagree with (a reference to the so-called filter bubble of social media, in which you assume most people agree with you because you are not exposed to other viewpoints). But after prominent media outlets covered the study’s findings, data scientists began to speak up. Actually, they argued, the study has major flaws, and its conclusion suggests that the news feed algorithm does hide news stories it thinks you will disagree with.

IT’S humans versus machine at the Rivers casino in Pittsburgh, Pennsylvania. Four professional poker players are squaring up to an artificial intelligence over two weeks, duking it out by playing a total of 80,000 hands of poker for a $100,000 cash prize.

Data Scientists will be around for at least the next 10 – 20 years, but will salaries fall once expectations could be downwardly adjusted for these positions?

new study has found that the financial health of Social Security, the program that millions of Americans have relied on for decades as a crucial part of their income, has been dramatically overstated.

The polling fiasco in the 2015 UK general election is just the latest in a string of high-profile failures over the last few months. This contrasts with the good performance of prediction markets, and Hypermind in particular.

In modern societies, cultural change seems ceaseless. The flux of fashion is especially obvious for popular music. While much has been written about the origin and evolution of pop, most claims about its history are anecdotal rather than scientific in nature. To rectify this, we investigate the US Billboard Hot 100 between 1960 and 2010. Using music information retrieval and text-mining tools, we analyse the musical properties of approximately 17?000 recordings that appeared in the charts and demonstrate quantitative trends in their harmonic and timbral properties. We then use these properties to produce an audio-based classification of musical styles and study the evolution of musical diversity and disparity, testing, and rejecting, several classical theories of cultural change. Finally, we investigate whether pop musical evolution has been gradual or punctuated. We show that, although pop music has evolved continuously, it did so with particular rapidity during three stylistic ‘revolutions’ around 1964, 1983 and 1991. We conclude by discussing how our study points the way to a quantitative science of cultural change.

What happens when a nuclear missile hits Washington, D.C.?

Chris Barrett, Executive Director of Virginia Bioinformatics Institute and Professor Virginia Tech came to speak at the Computational Social Science department of the Krasnow Institute about how big data plays a role in massively interactive systems on April 17 in a talk titled ‘Massively Interactive Systems: Thinking and Deciding in the Age of Big Data’.

To get respect in a field, scientists need to consider not just their work, but also their interactions with others.

The scholarly literature is meant to be a permanent record of science. So it is an embarrassing state of affairs that many of the web references in research papers are broken: click on them, and there’s a fair chance they will point nowhere or to a site that may have altered since the paper referred to it.

Herbert Van de Sompel, an information scientist at the Los Alamos National Laboratory Research Library in New Mexico, quantified the alarming extent of this ‘link rot’ and ‘content drift’ (together, ‘reference rot’) in a paper published last December (M. Klein et al. PLoS ONE 9, e115253; 2014). With a group of researchers under the auspices of the Hiberlink project (http://hiberlink.org), he analysed more than 1 million ‘web-at-large’ links (defined as those beginning with ‘http://’ that point to sites other than research articles) in some 3.5 million articles published between 1997 and 2012. The Hiberlink team found that in articles from 2012, 13% of hyperlinks in arXiv papers and 22% of hyperlinks in papers from Elsevier journals were rotten (the proportion rises in older articles), and overall some 75% of links were not cached on any Internet archiving site within two weeks of the article’s publication date, meaning their content might no longer reflect the citing author’s original intent — although the reader may not know this.