The era of the ‘always-on’ smartphone camera is here

Back in the early 1980s, American singer Rockwell released his sole chart-topping single, “Somebody’s Watching Me“. Featuring backing vocals from Michael and Jermaine Jackson, the song made a rather catchy hit of its protagonist’s deeply paranoid delusions. But had it been released in 2022 instead, those same feelings of paranoia might have seemed rather more rational thanks to some recent news from wireless tech giant Qualcomm.

During a deep dive into its upcoming Snapdragon 8 Gen 1 system-on-chip (SoC) at the recent Snapdragon Tech Summit 2021, Qualcomm detailed the latest iteration of the chip and its Spectra image signal processors (ISP). The 8G1 SoC is likely to power many of the next generations of Android flagships, and to be fair to Qualcomm, it has plenty of worthwhile new features. But among its updates, you’ll also find one highly controversial change: Support for “always-on” cameras.

Qualcomm recognizes that this is a divisive feature

The company clearly recognizes the potential for privacy concerns, as in detailing the Image Signal Processors located within the 8G1 SoC, it went to pains to distinguish the always-on ISP from the rest. It took nearly 45 minutes after first describing the three on-demand ISPs in detail before Qualcomm even mentioned the existence of that fourth, always-on ISP.

A lengthy discussion of security tellingly separated the two ISP deep-dives. And even once that mention finally came, the new, all-seeing ISP was described in fuzzy, reassuring terms as being “tucked away in the sensing hub for maximum security”. An accompanying graphic likewise gave the always-on ISP a tiny, insignificant-looking block in the system topology, located far away from the other ISPs to which the company would probably rather you paid attention.

There is a potential security benefit, but at what cost?

Qualcomm was quick to point out the potential for convenience from an always-on camera, noting in a glossily-produced video that it could be used to unlock your phone in an entirely hands-free manner. And in fairness, something which makes it less painful to unlock your phone means you might be more likely to use a screen lock in the first place, potentially improving your security.

But that gain has to be offset against the potential downsides, and they’re significant. An always-on camera can provide access to a vast amount of information about you and your environment. While we’re sure Qualcomm and its manufacturer partners have good intentions and intend to limit the function to things like locking and unlocking your phone, security breaches do happen. If or when a hacker manages to get their hands on that data stream, a little extra convenience might feel rather more trivial.

And there’s precedent to consider, too. Even if this generation of technology proves secure and is used entirely responsibly, it lays the groundwork by normalizing the idea of an always-on camera in the first place. Future versions enabling new uses for always-on cameras might also reset the window for what’s acceptable when it comes to letting third parties make use of that always-on video feed or the information that could be gleaned from within its frame.

It’s impressive in other respects

Other than its always-on ISP, the Snapdragon 8 Gen 1 certainly impresses with its other still and video-capture improvements. Qualcomm product management VP Judd Jeape describes it as “the biggest by far leap forward we’ve taken in camera, period.”

The 8G1 SoC is said to be the first smartphone chipset to make a jump from 14-bit to 18-bit ISPs, allowing not just greater dynamic range and smoother color gradients but also 18-bit RAW output. Between all active cameras, the 8G1 SoC can process up to 3.2 gigapixels of data per second, a rate that’s up 19% from the prior generation and claimed to be the fastest in mobile. And at 12-megapixels resolution, the SoC can process 240 photos per second.

Its newly-branded “Mega Multi-Frame Engine” promises better night photos with improved computational photography algorithms able to combine 30 18-bit frames of source data, a more than fivefold increase in data from the six 14-bit frames of the Snapdragon 888. And it does so while tracking motion at every pixel location, where the prior generation operated at just 1/4 the linear resolution.

Huge improvements for face detection and video capture

300-point AI-based algorithms—up from 115 points in Snapdragon 888—now allow for more accurate real-time face detection, which in turn can allow better autofocus and autoexposure performance. Yet despite crunching more data, face detection performance is said to be 300% better and the system can function in light levels as low as 0.5 lux (moonlight is roughly equivalent to 1 lux).

Snapdragon 8 Gen 1 can now record 8K HDR videos and can clock up to 64 megapixels off the sensor at 30 fps, allowing for both electronic image stabilization (with a crop) and high-res stills during video capture. Updated computational HDR algorithms allow for three exposures per frame instead of two, and the ISP can now capture HDR streams from two cameras at once, allowing for smoother handoffs between cameras.

Better efficiency, but bokeh errors remain an issue

When recording in HDR10+ format the 8G1 SoC is twice as power-efficient as the previous generation. It can now also correct for chromatic aberration in either stills or video in real-time, and supports an artificial bokeh effect for video capture as well. This looks quite nice at first glance, but on closer inspection suffers from artifacts at the periphery of your subject, and obvious errors also remain noticeable in stills.

Still, some weakness in the ArcSoft-derived bokeh algorithms aside this new SoC clearly has a lot to offer. And it’s important to note that its one truly divisive feature—the always-on camera—could potentially be disabled in software by design or better yet, disabled by default on the hardware side by Qualcomm. (And Qualcomm product management VP Judd Heape has indicated to the Verge that the company is open to this.)

Looking forward

As for the question of whether or not these new features outweigh the potential—or indeed, the likelihood—that some manufacturers will now start shipping their phones with always-on cameras… well, within the next few months it’s poised to become academic. As of this writing per Android Authority, no less than 10 manufacturers have announced their plans to start shipping new flagships based on the Snapdragon 8 Gen 1 chipset.

The first deliveries—and along with them, the departure of the genie from its bottle—are expected to arrive in Q1 2022.