NYU Data Science newsletter – January 15, 2016

GROUP CURATION: N/A

We Google things we’re too lazy to remember or ask another human. We’ve become accustomed to asking Siri and Cortana about the weather. The current generation of artificial intelligence can pull facts from the web, keep track of your appointments and even crack jokes. What if there were a virtual assistant to help you make real-life decisions, like whether you should start a brewery or go to business school? Project Sapphire, a collaboration between IBM and the University of Michigan [led by David Nahamoo from IBM and Satinder Baveja from Michigan], is aimed at building an artificially intelligent academic advisor that guides undergraduate students through their course options, helps pick extracurricular activities and eventually dishes out advice on their careers.

38th U.S. Secretary of Commerce Penny Pritzker speaks at the 2016 AMS [American Meteorological Association] Annual Meeting in New Orleans. She was joined by NOAA Administrator Kathryn Sullivan and former AMS President Marshall Shepherd. Much of the discussion featured weather data initiatives and NOAA Big Data projects.

… For as long as we have been a nation, we (the public) have demanded from our government (and our contractors) a thorough accounting for the money spent and the accomplishments achieved. In today’s government, that means data and that’s all good.

The issue? A little taste for good data leads to cravings for more. Without realizing it, we’ve become zombies with insatiable appetites for fleshy spreadsheets. That’s not so good.

Why? Data is expensive.

A veritable who’s who in artificial intelligence spent today discussing the future of their field and how to ensure it will be a good one. This exciting conference was organized by Yann LeCun, head of Facebook’s AI Research, together with a team of his colleagues at New York University. We plan to post a more detailed report once the conference is over, but in the mean time, here are some highlights from today.

One recurrent theme has been optimism, both about the pace at which AI is progressing and about it’s ultimate potential for making the world a better place.

Recommendation Systems have become an integral part of many businesses today. They have variety of applications. For example, on an e-commerce website like Amazon or Walmart, it would recommend you more relevant products to buy or in completely different setting they which would suggest you to add new people to your social network like that on Facebook or Twitter. Recommendation Systems are fueled and motivated by desire of Business growth and expansion. And just like other things backed by a desire to personal growth and earning profits, there is always a chance that it can cross ethical boundaries and act irresponsibly.

Meetup.com organized a talk given by Evan Estola, one of the company’s senior Machine Learning Engineer, to discuss the scenarios when “Recommendation Systems go Bad.” He talked about various aspects of recommendation systems, including the technical details as well as what would be an irresponsible behavior for a Recommendation system.

In this episode of the O’Reilly Data Show, I sat down with Vasant Dhar, a professor at the Stern School of Business and Center for Data Science at NYU, founder of SCT Capital Management, and editor-in-chief of the Big Data Journal. We talked about the early days of AI and data mining, and recent applications of data science to financial investing and other domains. [audio, 41:42]

People have been asking for videos of our lessons more and more frequently over the past few months, so we have begun archiving links on the lessons page. If you have more, please send them our way (either as a pull request against the lessons page in our website’s GitHub repo or by email.

Julia Computing has recently received funding from the Gordon and Betty Moore Foundation’s Data-Driven Discovery Initiative. One of the main components of this project is to improve the statistics and data science functionality available in the Julia ecosystem. I thought it would be useful to set out our current plans in this regard. These may change as the work develops, but they represent what we think are the most potentially useful contributions to the community.

Scientists, engineers, and other technical professionals require skills in computing and data analysis to do their jobs. We refer to these as data science skills.

Examples of data science skills abound. Biologists search thousands of genomes for DNA sequences with special characteristics, such as genes that transcribe non-coding RNA that is “anti-sense” to messenger RNAs. Astronomers search, integrate, and visualize data from many instruments that produce terabytes of complex data. Social scientists do text analytics on massive repositories of social media data to distill patterns in topics and trends in sentiment.

This course teaches graduate students the software engineering skills to do research in data science fields and to be successful technical professionals in the 21st Century.