NYU Data Science newsletter – November 5, 2015

GROUP CURATION: N/A

Beginning in 2011, the NEH Office of Digital Humanities (ODH) began requiring a Data Management Plan (DMP) for the majority of its grant programs. In the past year, NEH has received a number of Freedom of Information Act (FOIA) requests to view some or all of the DMPs submitted as a component of successful grant applications since 2011. Due to the high level of interest from scholars and the general public in the DMPs submitted, NEH has bundled the plans in a zip file and is making them available for download via the NEH FOIA Library [the link entitled “Data Management Plans From Successful Grant Applications (2011 – 2014)” leads to a 15.1mb zip file]: http://www.neh.gov/about/foia/library.

This article introduces the Special Issue on simple versus complex methods in forecasting. Simplicity in forecasting requires that (1) method, (2) representation of cumulative knowledge, (3) relationships in models, and (4) relationships among models, forecasts, and decisions are all sufficiently uncomplicated as to be easily understood by decision-makers. Our review of studies comparing simple and complex methods — including those in this special issue — found 97 comparisons in 32 papers. None of the papers provide a balance of evidence that complexity improves forecast accuracy. Complexity increases forecast error by 27 percent on average in the 25 papers with quantitative comparisons. The finding is consistent with prior research to identify valid forecasting methods: all 22 previously identified evidence-based forecasting procedures are simple. Nevertheless, complexity remains popular among researchers, forecasters, and clients. Some evidence suggests that the popularity of complexity may be due to incentives: (1) researchers are rewarded for publishing in highly ranked journals, which favor complexity; (2) forecasters can use complex methods to provide forecasts that support decision-makers’ plans; and (3) forecasters’ clients may be reassured by incomprehensibility. Clients who prefer accuracy should accept forecasts only from simple evidence-based procedures. They can rate the simplicity of forecasters’ procedures using the questionnaire at the simple-forecasting website.

There’s an old saying that when the only tool you have is a hammer, everything looks like a nail. I know, it’s cliched and painfully overused, but stay with me here. In the world of working with scientific data, and processing data using a computer and a programming environment, this–cliched or not–is the situation many young scientists find themselves in. In coming up through academia on a scientific path (rather than engineering or computer science route), we find ourselves frequently working with data and programming languages without having had the fundamental training in software best practices. This results in a lot of clunky code, and also in scientists wasting their time doing things the hard way–using a hammer for every task they need to perform on their data.

Many young scientists find themselves in an even worse situation: with a very complete theoretical understanding of hammers, and of how they are manufactured, but nothing to actually bang nails in with except for a nearby rock (let’s call that rock “excel”). Which makes for data analyses that are fragile and not easily reproduced.

In my efforts to work with remove sensing and geospatial data, and my interest in finding practical methods of working with the tsunami of data soon to come from unmanned aircraft, I was immediately drawn to the idea of a Software Carpentry Workshop; a place where I could learn about all of the fundamental skills and tools that I had missed along the way. With a generous travel grant from the University of Alaska Fairbanks’s Graduate School, I was able to attend a Software Carpentry Workshop at the University of Washington. Here I am excited to report on the experience.

The social network fundamentally changed in early 2014. And that’s causing big problems for the company.

With the coming release of our pending Health Data Analytics Value Chain report, we wanted to start a discussion around one of the core elements of the value chain that we present: Data Governance. Data governance is the foundational piece of the analytics value chain, establishing frameworks for managing data and information across an entire HCO.

Data governance sets the rules of the road for integrating data, ensuring that quality and use remains intact across multiple use cases. It also establishes the codes for data stewardship; the approach that organizations use to manage data and links to identities.

Data stewardship concerns are nearly countless ranging from data collection, aggregation, role-based sharing, consent management and security across different stakeholders.

NYU is a great place to become the founder of a new startup. In 2014, the university was deemed one of the top schools for venture capital backed entrepreneurs and last month it was named the sixth best college for female founders. … Given all these resources, we thought it was time to gather some of the most promising NYU startups that are taking advantage of what the university has to offer. Here are four that are killing the game.

The National Science Foundation announced new Big Data Hubs and the release of a solicitation for the new Big Data Spokes Initiative. More:

UC Berkeley BIDS announcement

UW eScience Institute announcement

Columbia Data Science Institute announcement

NSF announcement

… “Big Data is the headache; deep learning is the solution,” said well-respected venture capitalist Steve Jurvetson, partner at Draper Fisher Jurvetson and an early investor in multiple billion-dollar companies including SolarCity, Tesla Motors and Twitter.

During an onstage interview at VERGE, Jurvetson said it is no longer enough simply to find patterns in data — something that many software applications already do pretty well. The next imperative is teaching the software to make connections that are too complex for humans to perceive, a field that often goes by the name “machine learning.”

Many people think of Facebook as just the big blue app, or even as the website, but in recent years we’ve been building a family of apps and services that provide a wide range of ways for people to connect and share. From text to photos, through video and soon VR, the amount of information being generated in the world is only increasing. In fact, the amount of data we need to consider when we serve your News Feed has been growing by about 50 percent year over year — and from what I can tell, our waking hours aren’t keeping up with that growth rate. The best way I can think of to keep pace with this growth is to build intelligent systems that will help us sort through the deluge of content.

To tackle this, Facebook AI Research (FAIR) has been conducting ambitious research in areas like image recognition and natural language understanding. They’ve already published a series of groundbreaking papers in these areas, and today we’re announcing a few more milestones.

In defending his company against assertions that Uber drivers should be classified as employees, Uber CEO Travis Kalanick often wields the algorithm. Uber isn’t a boss, he argues. It’s a software platform that balances supply and demand to connect entrepreneurs with customers.

A new academic paper pokes holes in that argument.

Researchers at the Data and Society research institute at New York University point out that Uber uses software to exert similar control over workers that a human manager would. The company’s algorithm uses performance metrics, scheduling prompts, behavioral suggestions, dynamic prices, and information asymmetry “as a substitute for direct managerial power and control,” they wrote.

This Big Data journal special issue on Visualization in Data Science, expected to be published in March 2016, will showcase research in which data visualization is used to solve a data science problem.

Deadline for Manuscript Submissions: Friday, January 1, 2016

… As podcasting is fundamentally a broadcast medium (oldschool, we know 😉 this is also a great way for us to get in touch with our mysterious listenership.