Times Insider|As Augmented Reality Evolves, the Reporting Is All Around You
Times Insider explains who we are and what we do, and delivers behind-the-scenes insights into how our journalism comes together.
As wildfires ravaged the West Coast and toxic smoke billowed and tainted the air this summer, those safe in their homes in other parts of the country were able to get a glimpse of the air quality just by holding up their phones. Through the use of a specialized Instagram filter, viewers of The New York Times on Instagram could project floating smoke particles into their living rooms or backyards. They could see what Westerners were living through.
Through the use of augmented reality, or AR, which lays a computer-generated image or animation over a user’s view of the real world, The New York Times was able to create an immersive experience intended to produce a deeper level of understanding.
In a recently announced collaboration with Facebook, which owns Instagram, The Times has started an Instagram-driven augmented reality initiative meant to create more personal and interactive experiences for users.
The project will use Spark AR, a developer platform owned by Facebook, which gives creators access to a suite of tools and software needed to create augmented reality filters and camera effects and then distribute them on Instagram and Facebook. The social media conglomerate is not involved in any storytelling or editorial decisions.
Though The Times has published journalism using augmented reality, or AR, on its website and through its own app since 2018, this series is the first time the technology has been published solely to be consumed through social platforms.
“We’re reaching a newer audience who is more familiar with this storytelling medium,” said Karthik Patanjali, graphics editor for special projects. Mr. Patanjali began experimenting with AR and Times journalism four years ago. “We knew this medium had a lot of potential, but nobody had used it for journalistic storytelling,” he said. “It was all dancing hot dogs.”
The Times’s new AR Lab, composed of around a dozen reporters, editors, developers, project managers, 3-D artists and designers, will work to produce content accessible not only to The Times’s 10.7 million Instagram followers but to anyone who uses the social media platform, making pieces of the organization’s journalism reachable for a new audience. According to Dan Sanchez, an editor for emerging platforms at The Times, the team hopes to use AR to create an immersive hook into the journalism The Times already produces online and make use of Instagram’s “swipe up” feature, linking long-form pieces, to reach people who might not already have a direct connection to the newspaper.
To help the storytelling process, the AR team works closely with desks across the newsroom, from Climate to Cooking to Games, to find opportunities to incorporate the technology in methods that feel complementary and purposeful. So far, along with the project that highlighted the toxicity caused by the wildfires, the initiative has also brought to life interactive artifacts to commemorate the centennial of the 19th Amendment and a filter that demonstrates decreases in air pollution around the world during the Covid-19 lockdowns.
“The whole point of this project is to really dig into how we can connect the physical world to layers of visual information,” Mr. Sanchez said. “If you’re an Instagram user, you can actually see those layers of information over top of the physical world, and you can actually manipulate them and remix them, and experience them on your own.”
The team’s latest project, which was published on Saturday, is an immersive experience informed by infectious-disease experts that illustrates how different masks trap and hold assorted sizes of microscopic particles. By creating these participatory filters, those working on the project hope to help users create a new world and space they can explore on their own.
“If we can take a piece of evidence and put it right in front of you so that you can see it, sense it and know its scale, I think that’s pretty huge,” said Noah Pisner, a 3-D immersive editor. “There’s a lot of ways we can use it to just improve the work that journalists are already doing. We want it to be something that’s additive.”
According to those on the team, augmented reality’s current role in journalism is meant to be supplemental the same way a video, a photo or a graphic might be. While it’s meant to enhance the narrative experience at the moment, some editors believe this shift toward AR indicates not only a shift in journalism, but a shift in how we obtain and view information as a whole, Mr. Patanjali said.
“You won’t be consuming information like this forever,” he said. “It’s not going to be on a sliver of glass forever. It’s going to be around you. These are all steps toward that future we’re preparing ourselves for. The world is 3-D. Why shouldn’t the information we present also be?”