If you want to take your smartphone photography to the next level, Adobe is developing a camera app to help you accomplish just that.
Marc Levoy, who joined Adobe two years ago as a vice president to help push the initiative, said that the company aims to introduce an app within the next year or two that combines the computing power of modern phones with the artistic options that serious photographers often demand.
When it comes to credentials, Levoy is top-notch: He was a pioneer in the field of computational photography while working at Stanford University and then went on to head Google’s illustrious Pixel camera app team.
In an exclusive interview, Levoy stated, “What I did at Google was to democratise decent photography.” What I want to accomplish at Adobe is to make creative photography more accessible, so that anyone can have an open dialogue with their camera.
If it takes off, the app has the potential to further the smartphone photography revolution beyond the capabilities currently prioritised by major manufacturers like Apple, Google, and Samsung.
The image quality of small, physically restricted smartphone cameras has been vastly improved thanks to computational photography. And you can now use previously hidden capabilities like panoramic stitching, portrait mode to remove distracting backdrops, and night modes for higher quality shots after dark.
Conversational Camera Apps for The Shutterbug
In other words, Adobe isn’t launching an app for the masses, but rather for photography lovers and professionals who are likely already using Photoshop and Lightroom. Such photographers typically have to practise with manual controls, including autofocus, shutter speed, colour, lens length, and aperture.
Some camera applications, such as Open Camera for Android and Halide for iPhones, have manual controls that are analogous to those found on conventional cameras. Some of those features can be found in Adobe’s own camera app, which is a part of the Lightroom mobile app.
However, Adobe is taking a different approach with its new camera app, emphasising rather a “conversation” between the photographer and the programme as they work together to capture the perfect image.
“photographers who want to think a little bit more carefully about the shot that they are taking and are willing to interact a bit more with the camera while they are taking it,” Levoy said, describing Adobe’s target audience. “Many doors have just been opened.
That’s always been on my list of things to achieve, and now I can finally realise that dream thanks to Adobe.”
Google and its smartphone rivals, on the other hand, are keen to avoid alienating their more mainstream customers. “Let’s focus on the consumer and the single button press,” they’d say every time Levoy proposed a feature that required more than one button press.
Ideation and Implementation Details for The Adobe Camera App
While Levoy won’t reveal too more about what his programme can do just yet, he did add that Adobe is developing a way to eliminate reflections in window shots. He noted that Adobe’s strategy brings fresh AI techniques to the table.
Levoy remarked, “I wish I could get rid of window reflections.” I’d like to send it back because it messes up so many of my pictures.
However, Levoy does anticipate development in a number of key areas.
An image is “relighted” when issues like harsh shadows on faces are fixed. Decisions on where in a scene to place lights can be informed by the iPhone’s lidar sensor or other methods of creating a 3D “depth map” of the area.
Attempting to provide higher-resolution images or greater granularity when digitally zooming in by computationally generating new pixels is the new “superresolution” method.
Levoy posits that the multi-frame and AI techniques for image enhancement could be combined, citing the examples of Google’s Super Res Zoom and Adobe’s AI-based Image Enhancement tool. And I’m collaborating with the folks at Adobe who made that,” he stated.
Make sure everyone is smiling and no one is blinking in a group photo by digitally combining numerous shots into one.
Inconsistent performance is common with this type of technology: “It originally debuted in Google Photos and has been there for a while. After users started uploading a wide variety of disturbing content, we quickly pulled the plug “To quote Levoy:
Modern image sensors. As an example, Sony produces a polarised light sensor that could be beneficial in smartphones, and this feature has long been lauded by photographers for its ability to reduce glare and reflections.
Instead of filtering the entire image, it would provide scene information for more nuanced processing, such as eliminating reflections on a person’s sweaty face.
Computational video “barely touched the surface,” according to Levoy, which involves using the same tactics in the video as are now prevalent with photographs.
As an illustration, he wishes there was a tool similar to Google Pixel’s Magic Eraser that could be used to delete unwanted footage from recordings. He predicted that the video’s prominence will continue to grow, citing the success of TikTok as evidence.
Imagery that changes shape depending on the devices used to view it. Photos on mobile devices tend to appear best with increased contrast and deeper colours, but the same photo on a larger screen like a laptop or television can look garish.
Levoy speculated that Adobe’s DNG file format would enable viewers to modify such settings to taste. A combination of real and artificial visuals, such as those created by OpenAI’s DALL-E artificial intelligence system, which Levoy describes as “wonderful.”
He also mentioned that your own photos might be used to drive AI-generated images, demonstrating Adobe’s commitment to encouraging originality.
Objects’ true colours, such as whether they are blue or white but appear blue due to shadowing, could be better discerned with the use of data gathered by multispectral imaging sensors, which collect ultraviolet and infrared light beyond the range of human eyesight.