NYU Data Science newsletter – June 24, 2015

GROUP CURATION: N/A

The Freedmen’s Bureau Project is helping African Americans reconnect with their Civil War­-era ancestors. Join us in restoring thousands of records, and begin building your own family tree.

A few weeks ago I helped organize and instruct a Software Carpentry workshop geared towards social scientists, with the great help from folks at UW-Madison’s Advanced Computing Institute. Aside from tweaking a few examples (e.g. replacing an example using fake cochlear implant data with one of fake survey data), the curriculum was largely the same. The Software Carpentry curriculum is made to help researchers, mostly in STEM fields, to write code for reproducibility and collaboration. There’s instruction in the Unix shell, a scripting language of your choice (we did Python), and collaboration with Git.

We had a good mix of folks at the workshop, many who had some familiarity with coding to those who had zero experience. There were a number of questions at the workshop about how folks could use these tools in their research, a lot of them coming from qualitative researchers.

I was curious about what other ways researchers who use qualitative methods could incorporate programming into their research routine. So I took to Facebook and Twitter.

At MapD our goal is to build the world’s fastest big data analytics and visualization platform that enables lag-free interactive exploration of multi-billion row datasets. MapD supports standard SQL queries as well as a visualization API that maps OpenGL primitives onto SQL result sets.

… A modern data scientist often has to work on multiple platforms with multiple languages. Some projects may be in R, others in Python. Or perhaps you have to work on a cluster with no gui. Or maybe you need to write papers with latex. You can do all that with Emacs and customize it to do whatever you like. I won’t lie though. The learning curve can be steep, but I think the investment is worth it.

Below are some key features that I think make Emacs an excellent editor for any data scientist.

I see a low use of Mathematica in Kaggle competitions.

Why would one use the Wolfram Language versus R, Python, or Julia for machine learning?

Besides prettier plots and the Manipulate function, do we have something that is useful for ml that other languages are lacking?

Data security and information governance are the hot practice areas for law firms these days, and nearly every month a new law firm unveils a rebranding effort or a new practice group angling to grow its business in this area. The latest is Akerman, which launched an 18-person Data Law group this month that will focus on data security, information governance and eDiscovery.

Andrew Connolly is a professor in the University of Washington Department of Astronomy. He is one of several UW professors working on the Large Synoptic Survey Telescope, or LSST, which will begin scanning the sky in 2022 from its location atop Cerro Pachón, a mountain in northern Chile.

He has called it “one of the most exciting experiments in astrophysics today,” adding, “it could completely transform our knowledge of the universe, from understanding how dark energy drives the expansion of the universe, to identifying asteroids that may one day impact the Earth.”

What makes a good [soccer] manager? The answer depends in part on his job description. Some managers are in charge of recruitment, while others must leave it to a director of football or a transfer committee. Most managers take charge of tactics on the field, but a few give their coaches minute-to-minute control. All managers, however, try to make the whole greater than the sum of the parts, bringing players together in a lineup that works. And that’s what I’ve tried to measure with these new ratings.

… political scientists at Harvard and Stanford universities, drawing from longstanding social psychology research, have concluded that a person’s political attitudes are actually a consequence of the actions he or she has taken — and not their cause.

In a new working paper, the researchers say changing political attitudes can be understood in the context of “cognitive dissonance,” a theory of behavioral psychology that asserts that people experience uneasiness after acting in a way that appears to conflict with their beliefs and preferences about themselves or others. To minimize that mental discomfort, the theory posits, a person will adapt his or her attitude to better fit with or justify previous actions.

Last year, astronomers were surprised to detect 47 galaxies in the Coma Cluster that were made almost entirely of dark matter. So how much more surprised are they to see 800 more dark galaxies in the same cluster?

Even that many concentrations of mysterious dark matter may be merely the “tip of the iceberg,” said Jin Koda, an astrophysicist at Stony Brook University in New York.

… We want a solution that can not only learn how and what data to extract from data sources like spreadsheets, word processor files, and computer generated PDF files but also learn how to map the data found in these documents to specified target fields in a database. … This is a qualifying problem which requires you to process MLS (Multiple Listing Service) data from 300 MLS training records into a database. You will be provided with a document summarizing the correct format of each field to be extracted.

Submission Deadline is Monday, January 11, 2016. Submissions began on June 8.