NYU Data Science newsletter – July 27, 2015

NYU Data Science Newsletter features journalism, research papers, events, tools/software, and jobs for July 27, 2015

GROUP CURATION: N/A

 
Data Science News



Big data, interactive access: How Apache Drill makes it easy – O’Reilly Radar

O'Reilly Radar, Ellen Friedman


from July 24, 2015

Big data techniques are becoming mainstream in an increasing number of businesses, but how do people get self-service, interactive access to their big data? And how do they do this without having to train their SQL-literate employees to be advanced developers?

One solution is to take advantage of the rapidly maturing open source, open community software tool known as Apache Drill. Drill is not the first SQL-on-Hadoop tool. It is, however, a new and very sophisticated highly scalable SQL query engine that has been built from the ground up to be appropriate for use even in production settings.

 

Testing for data scientists by Trey Causey

slides.com, Trey Causey


from July 18, 2015

from presentation at PyData 2015

 

loosening group boundaries

Harold Jarche


from June 28, 2015

The Triple A Organization (Awareness, Alternatives, Action) by Valdis Krebs takes this into consideration, promoting organizational dynamics that connect unique group boundaries but do not destroy them. … Successful AAA organizations will enable the flow of opposing ideas, as well as the space and time to test new ideas. Understanding our communication and knowledge flows, through social and organizational network analysis, such as that provided by Valdis, gives us the visualization necessary to ensure that we have the foundation for working in the network era. Balance in the network era will be achieved in a constant dance of engaging people and ideas. This continuous cognitive movement will ensure that people and organizations can adapt to the new challenges brought through our increasing connectivity.

 

In Connecticut, the Twilight of a Trading Hub – The New York Times

The New York Times, Dealbook blog


from July 23, 2015

… For much of the last 30 years, the story of Wall Street was one of ceaseless growth and enormous wealth, as the clubby investment banks and trading firms that used to be based in Lower Manhattan grew into giants of the global economy, with outposts in cities like Stamford.

Now these firms are in the middle of a contraction that many in the industry think could last for some time. It’s particularly obvious in a place like Stamford, where tax breaks lured banking operations and a large block of the city was remade for them. But the cuts are happening everywhere.

The number of front-office employees, who are generally highly paid, at the 10 largest Wall Street banks has shrunk more than 20 percent since 2011 — to 51,000 from 64,000 — according to Coalition, an industry data provider. A study of the top Wall Street banks, released in May by the Boston Consulting Group, found that revenue had been steadily falling across the entire industry since 2012, while costs had not.

 

US tailored-medicine project aims for ethnic balance

Nature News & Comment


from July 21, 2015

The clock is ticking for experts charged with designing a US government programme to collect genetic, physiological and other health data from one million volunteers over the next two decades. The plan for the effort, part of the US$215-million Precision Medicine Initiative (PMI) announced in January, is due in the next few weeks — a daunting deadline, in part because the effort’s priorities include filling racial and socio-economic gaps left by other long-term studies.

 

Environmental science: Agree on biodiversity metrics to track from space

Nature News & Comment


from July 22, 2015

Global biodiversity loss is intensifying. But it is hard to assess progress towards the Aichi Biodiversity Targets for 2011–20 set by the Convention on Biological Diversity (CBD). Target 5, for instance, aims to halve global deforestation rates by 2020; but reliable indicators for deforestation that can be monitored remotely have not been developed or agreed on. National biodiversity monitoring programmes differ widely, most data sets are inconsistent, and few data are shared openly.

To focus priorities, ecologists have proposed classes of ‘essential biodiversity variables’ — including species traits and populations, and ecosystem function and structure1. But measuring these on the ground is laborious and limited.

 

This “World’s First” Artificially Intelligent Ad Is A Test of Automated Creativity

Fast Company, Co.Create


from July 23, 2015

M&C Saatchi has developed an outdoor campaign which “evolves” unique ads based on its audience’s reactions, which makes it the world’s first artificial intelligence poster campaign, the agency claims.

Built around a single site in central London, the campaign, for invented coffee brand Bahio, is powered by a genetic algorithm that tests different executions by analyzing the strengths of various features such as copy, layout, font and image.

 

Mapping Systemic Risk: Critical Degree and Failures Distribution in Financial Networks

PLOS One


from July 24, 2015

The financial crisis illustrated the need for a functional understanding of systemic risk in strongly interconnected financial structures. Dynamic processes on complex networks being intrinsically difficult to model analytically, most recent studies of this problem have relied on numerical simulations. Here we report analytical results in a network model of interbank lending based on directly relevant financial parameters, such as interest rates and leverage ratios. We obtain a closed-form formula for the “critical degree” (the number of creditors per bank below which an individual shock can propagate throughout the network), and relate failures distributions to network topologies, in particular scalefree ones. Our criterion for the onset of contagion turns out to be isomorphic to the condition for cooperation to evolve on graphs and social networks, as recently formulated in evolutionary game theory. This remarkable connection supports recent calls for a methodological rapprochement between finance and ecology.

 

Big Data & the Risky Eclipse of Statisticians

Hacker Rank blog


from July 20, 2015

If statisticians have historically been leaders of data, why was there a need for a brand new breed of data scientists? While the world is exploding with bounties of valuable data, statisticians are strangely working quietly in the shadows. Statistics is the science of learning from data, so why aren’t statisticians reigning as kings of today’s Big Data revolution?

In 2009, when Google was still fine tuning its PageRank algorithm based on the statistical innovation Markov Chain, Google’s Chief Economist Hal Varian declared statistician as the sexiest job of the decade. We’re about halfway through, and it seems that Varian missed the target.

 

Data Science for Social Good 2015, Project blogs

UW eScience Institute, Data Science for Social Good 2015


from July 14, 2015

As part of the DSSG program, participants have been blogging about their experience. In team blogs devoted to the different projects, they have been sharing their thoughts coming into the program, and will continue to share their thoughts about the challenges and victories they encounter along the way. As we enter the second half of the program, a few of the interesting results will also start appearing on these blogs, documenting the progress of the projects.

 

A Look Inside Google and Carnegie-Mellon’s IoT Campus

Fast Company


from July 24, 2015

For the “Internet of Things” to thrive, all it needs is for all devices to get along—which is currently wishful thinking. Last week, however, Google announced a partnership with Carnegie-Mellon University, which is leading a collaboration of faculty from several other academic institutions on a project to jumpstart the Internet of Things revolution. Their plan: Build a universal platform that lets any device talk to any other device. And fittingly, that master-key solution will be open source.

The problem is that IoT software and devices are mostly proprietary, built by each company and working well within their own sandboxes, but they don’t communicate well together. The joint project between CMU, Cornell, Stanford, Illinois at Urbana-Champaign, and Google wants to wipe away the private-industry middlemen that keep sensors in separate sandboxes by creating a new, open platform: GIoTTO.

 
CDS News



Rooting Out Corruption or Rooting for Corruption? The Heterogeneous Electoral Consequences of Scandals Political Science Research and Methods – Rooting Out Corruptio

Political Science Research and Methods


from July 22, 2015

Corruption scandals have been found to have significant but mild electoral effects in the comparative literature (Golden 2006). However, most studies have assumed that voters punish all kinds of illegal practices. This article challenges this assumption by distinguishing between two types of corruption, according to the type of welfare consequences they have for the constituency. This hypothesis is tested using data from the 2011 Spanish local elections. We exploit the abundance of corruption allegations associated with the Spanish housing boom, which generated income gains for a wide segment of the electorate in the short term. We find that voters ignore corruption when there are side benefits to it, and that punishment is only administered in those cases in which they do not receive compensation

 

Leave a Comment

Your email address will not be published.