Data Science newsletter – August 30, 2017

GROUP CURATION: N/A

Deborah Thomas, associate professor of math and computer science, explains that preparations for this major have been in process for a while. “We were already offering the courses, getting students exposed to the field that is very hot right now,” she says. Since data science can be applied to any field, this major will allow graduates to enter the job market with ease. “The amount of data being collected is constantly increasing and will never go away,” says Thomas. “Most often, we have acquired a ton of data with no easy way of extracting hidden patterns from it. Data science allows us to do that easily. The number of opportunities for our students will only increase.”

Carnegie Mellon University’s School of Computer Science received $1 million to pay for machine learning projects.

There’s a catch, however.

The students and faculty must use the money for good.

Uptake, a Chicago-based predictive analytics software company, gave the money to CMU to establish the Machine Learning for Social Good fund.

When it comes to televised sporting events, the ability of the cameras to follow the action — both smoothly and accurately — can make or break the viewing experience. That’s why, last year, alongside California Institute of Technology Assistant Professor Yisong Yue, I and a team of collaborators from Disney Research and the University of British Columbia published a paper on automating broadcast cameras using machine learning.

The project was designed to improve the television broadcast experience by teaching automatic cameras to be more human like. This involved creating machine learning algorithms that trained automated cameras to strike a delicate balance between capturing all movements accurately and maintaining a smooth viewing experience. The Walt Disney Company plans to use this research to improve its soccer and basketball coverage.

Before I elaborate on the strategies we used to achieve this objective, which involved combining a deep neural network with a model-based approach, I want to take a step back and discuss, at a higher level, machine learning, deep learning, and imitation learning.

University of Washington researchers are developing an app that could allow people to easily screen for pancreatic cancer and other diseases — by snapping a smartphone selfie.

BiliScreen uses a smartphone camera, computer vision algorithms and machine learning tools to detect increased bilirubin levels in a person’s sclera, or the white part of the eye. The app is described in a paper to be presented Sept. 13 at Ubicomp 2017, the Association for Computing Machinery’s International Joint Conference on Pervasive and Ubiquitous Computing.

Countless NoSQL databases competed to be the database of choice. MongoDB’s marketing strategy helped it become the winner.

Scientific advancement increasingly relies on large and complicated databases and models to thrive. Earth and space scientists, in particular, often grapple with complex systems to help society manage the threats of natural hazards and climate change. The scientific community faces many challenges to preserve often nonreproducible data and to catalog and store them in ways that are searchable and understandable.

To address these challenges, the Laura and John Arnold Foundation has awarded a grant to a coalition of scientific groups, convened by the American Geophysical Union (AGU), representing the international Earth and space science community. The grant will allow the coalition to develop standards to connect researchers, publishers, and data repositories across the sciences. These standards are meant to render data findable, accessible, interoperable, and reusable (FAIR), with the aim of enhancing the integrity and reproducibility of data and accelerating scientific discovery.

To perpetuate the pace of innovation and progress in microelectronics technology over the past half-century, it will take an enormous village rife with innovators. This week, about 100 of those innovators throughout the broader technology ecosystem, including participants from the military, commercial, and academic sectors, gathered at DARPA headquarters at the kickoff meeting for the Agency’s new CHIPS program, known in long form as the Common Heterogeneous Integration and Intellectual Property (IP) Reuse Strategies program.

“Now we are moving beyond pretty pictures and mere words, and we are rolling up our sleeves to do the hard work it will take to change the way we think about, design, and build our microelectronic systems,” said Dan Green, the CHIPS program manager. The crux of the program is to develop a new technological framework in which different functionalities and blocks of intellectual property—among them data storage, computation, signal processing, and managing the form and flow of data—can be segregated into small chiplets, which then can be mixed, matched, and combined onto an interposer, somewhat like joining the pieces of a jigsaw puzzle. Conceivably an entire conventional circuit board with a variety of different but full-sized chips could be shrunk down onto a much smaller interposer hosting a huddle of yet far smaller chiplets.

Early in August, NYU professor Siddharth Garg checked for traffic, and then put a yellow Post-it onto a stop sign outside the Brooklyn building in which he works. When he and two colleagues showed a photo of the scene to their road-sign detector software, it was 95 percent sure the stop sign in fact displayed a speed limit.

The stunt demonstrated a potential security headache for engineers working with machine learning software. The researchers showed that it’s possible to embed silent, nasty surprises into artificial neural networks, the type of learning software used for tasks such as recognizing speech or understanding photos.

Malicious actors can design that behavior to emerge only in response to a very specific, secret signal, as in the case of Garg’s Post-it. Such “backdoors” could be a problem for companies that want to outsource work on neural networks to third parties, or build products on top of freely available neural networks available online.

Known as the “Fish Guy,” Adam Summers earned his moniker for an odd hobby turned academic obsession: giving dead fish a computerized tomography, or CT, scan. The biomechanist at Friday Harbor Laboratories on San Juan Island, Washington, has been haphazardly scanning fluid-preserved collections of fish for 20 years—”I literally traded Snickers bars for CT scans when I started out,” he says—to create detailed 3D representations of the animals and study the intricacies of their internal architecture. Whenever he posted the beautifully rendered skeletons online, fellow fish admirers would eagerly ask what was up next. Summers’s half-joking reply: “Don’t worry, they are all next. I am scanning all fishes.”

Then last year David Blackburn, a herpetologist at the Florida Museum of Natural History in Gainesville, saw Summers’s #scanAllFish hashtag on Twitter and light-heartedly countered that he would “scan all frogs.”

For years, U.S. forecasters have envied their colleagues at the European Centre for Medium-Range Weather Forecasts (ECMWF) in Reading, U.K., whose hurricane prediction models remain the gold standard. Infamously, the National Weather Service (NWS) in 2012 failed to predict Hurricane Sandy’s turn into New Jersey, whereas ECMWF was spot on. But two innovations tested during Hurricane Harvey, one from NASA and another from the National Oceanic and Atmospheric Administration (NOAA), could help level the playing field.

NOAA’s offering is a brand-new forecasting model. Two years ago, NOAA’s Geophysical Fluid Dynamics Laboratory (GFDL) in Princeton, New Jersey, won a competition to provide the computer code for the next-generation weather model of NWS. Current NWS models must wait for results from a time-consuming global simulation before they can zoom in on a smaller area and run a high-resolution model for hurricanes.

Cities which are effectively using data to solve policy problems often choose to partner with universities or contract out to businesses for technical assistance in collecting and analyzing data. This can be a complex and, often, overwhelming process for cities without their own capacity to evaluate technology and analyze data.

Recently, Columbia University’s School of International and Public Affairs and Data Science Institute had the opportunity to partner with New York City’s Department of Environmental Protection on a project called “Stopping Trash Where It Starts.” The City’s goal was to limit floatable trash in the waterways as part of an overall plan to reduce water pollution. Since the City’s data collection efforts indicated that street litter is the primary source of floatable trash, our project focused on helping the City develop a better understanding of the causes of street litter. Our data analysis was intended to inform policy recommendations and new initiatives for reducing litter on the streets.

Washington, DC December 12, organized by Computing Research Association. “The goal of the summit is to put the issue of technology and jobs on the national agenda in an informed and deliberate manner.” [registration opens on September 18]

New York, NY October 13-15. 200 of the brightest minds in the property and technology industries will hack cutting edge solutions to real-world challenges faced by the world’s leading real estate companies. [$$]

CrateDB operates in a shared-nothing architecture as a cluster of identically configured servers (nodes) that coordinate seamlessly with each other. Execution of write and query operations are automatically distributed across the nodes in the cluster.

Increasing or decreasing database capacity is a simple matter of adding or removing nodes. We worked hard on the “simple” part by automating the sharding, replication (for fault tolerance), and rebalancing of data as the cluster changes size. CrateDB was born in the container era and allows you to scale and administer it easily via container orchestration platforms like Docker or Kubernetes in a microservices environment.

“In modern applications, statisticians are faced with integrating heterogeneous data modalities relevant for an inference, prediction, or decision problem.” … “In this article, we investigate why these modular approaches might be preferable to the full model in misspecified settings. We propose principled criteria to choose between modular and full-model approaches. The question arises in many applied settings, including large stochastic dynamical systems, meta-analysis, epidemiological models, air pollution models, pharmacokinetics-pharmacodynamics, and causal inference with propensity scores.”

Este repositorio contiene una serie de material sobre un breve tutorial sobre scikit-learn en Python. Está basado en el tutorial de scikit-learn realizado en la conferencia Scipy2017 (ver referencias).

There is a lot for companies to like about data lakes. Because data are loaded in “raw” formats rather than preconfigured as they enter company systems, they can be used in ways that go beyond just basic capture. For instance, data scientists who may not know exactly what they are looking for can find and access data quickly, regardless of format. Indeed, a well-maintained and governed “raw data zone” can be a gold mine for data scientists seeking to establish a robust advanced-analytics program. And as companies extend their use of data lakes beyond just small pilot projects, they may be able to establish “self-service” options for business users in which they could generate their own data analyses and reports.

However, it can be time consuming and complicated to integrate data lakes with other elements of the technology architecture, establish appropriate rules for company-wide use of data lakes, and identify the supporting products, talent, and capabilities needed to deploy data lakes and realize significant business benefits from them. For instance, companies typically lack expertise in certain data-management approaches and need to find staffers who are fluent in emerging data-flow technologies such as Flume and Spark.