An excellent mobile camera is built with sturdy imaging hardware, traditional imaging algorithms, and an increasingly important AI engine, which can convert these numbers 1 and 0 into beautiful snapshots. As the power of machine learning and computational photography continues to increase, the latter is advancing by leaps and bounds. Artificial intelligence technology is helping smartphones produce professional-quality photos, including beautiful bokeh, high dynamic range images and film, and even shooting videos in near-dark conditions. But continuously improving software technology also requires more powerful machine learning and image processing hardware, such as the hardware in the Qualcomm Snapdragon 888 5G mobile platform.
Qualcomm Snapdragon 888 runs as a heterogeneous computing platform and distributes machine learning workloads to the most appropriate components. In photography and video, the Spectra 580 image signal processor (ISP) and Hexagon 780 processor of the platform play a central role.
Intelligent image processing, understand your pictures
Snapdragon 888’s Spectra 580 ISP handles all typical photography and video processing stages. It is the core component that acquires the image from the camera sensor and converts it into the numbers 1 and 0 stored in the smartphone gallery. But dissatisfied with the basics of image processing, Qualcomm added additional features of its ISP to help users capture perfect pictures. Let’s delve into some examples.
Spectra 580 is actually a “triple ISP” capable of processing 2.7 huge pixels per second.This is sufficient data throughput Record 8K video Or capture a huge 200 megapixel image, which is equivalent to more detail than ever before. But the triple ISP also allows the processor to capture pictures and videos from up to three cameras at a time, enabling picture-in-picture capture, such as the director’s view in the Samsung Galaxy S21 series. ISP also provides a complete 10-bit image pipeline, allowing the full capture and storage of more than 1 billion tones using the HEIC image format, as seen in Oppo Find X3 Pro. This is much more than traditional 8-bit JPEG image data.
Qualcomm’s zero shutter lag function has “pre-capture” images, so you can shoot immediately after you press the shutter button, so you don’t miss the perfect fast-motion shooting. ISP also processes multiple image frames to reduce noise and high dynamic range, ensuring that details look clear and smooth even in harsh lighting environments. When their images are incorrect, no one wants to retake them, and the intelligent image processing technology used by Qualcomm ensures that there is no need to do so.
But this is not all. The Spectra 580 in Snapdragon 888 also has a “computer vision” function directly built into the chip, injecting artificial intelligence imaging intelligence into it.
Autofocus, auto exposure, and auto white balance are the cornerstones of imaging, ensuring that your images are of professional quality no matter what scene you shoot. For example, you need to shoot different exposures for landscapes and portraits. By directly using AI intelligence in the ISP chain, Qualcomm’s camera settings can make these important adjustments in real time without taking up valuable processing resources and battery life. But most importantly, whether you are taking macro photos, portrait photos or beautiful landscapes, every time you press the shutter button, it can ensure that your photos are always in focus, correctly exposed, and have the correct colors.
If you want to understand the power of these technologies, check out these pictures taken on Snapdragon devices.
AI intelligence brings professional touch
Although the ISP of Snapdragon 888 handles core image and video processing tasks, the Hexagon 780 processor in the 6th-generation Qualcomm artificial intelligence engine is ready to step in when additional machine learning horsepower is needed. Use cases involving the Hexagon 780 processor range from sharpening the appearance of pictures to completely changing the imaging capabilities of smartphones.
Software bokeh blur is a very popular photography and video function, and Qualcomm uses its AI engine to produce high-end effects. Hardware-accelerated machine learning calculates the depth and edges of the foreground and background, while graphics delivery applies software blur to the image. The result is a natural and soft appearance of the image, which is usually associated with a high-quality digital SLR camera. Qualcomm’s AI engine is even fast enough to run all these mathematical operations in real time, applying bokeh blur to 4K 30fps video clips.
To give another example, Rainbowsoft has developed an artificial intelligence auto-scaling function, which is accelerated on Qualcomm’s artificial intelligence engine to automatically track and zoom objects and characters in the scene.This feature is included in Smartphone for Snapdragon Insider, The real-time application allows you to switch the subject, automatically zoom in and out, and even switch to the most suitable camera lens in the multi-camera smartphone settings. When you shoot the video again, you will never forget fast-moving objects and subjects.
Qualcomm is also cooperating with BlinkAI, which uses the AI engine of Snapdragon 888 to greatly improve the night video shooting of Xiaomi Mi 11 smartphones. BlinkAI’s computing solution can increase the effective lighting by up to 5 times, thereby greatly improving image quality in difficult shooting environments. When shooting in low light, a long period of dark, gloomy and grainy video footage.
Powerful hardware allows computational video capture to go a step further, so that our usual multi-frame imaging technology associated with photography can now be applied to video. For example, the use of digital overlay (DOL) and interlaced HDR image sensors combined with processing intelligence can combine multiple frames into powerful HDR video even at high resolutions. This technology is used in OnePlus 9 Pro, Oppo Find X3 Pro and Xiaomi Mi 11 Ultra.
Computational photography is the future
Robert Triggs/Android Authority
In addition to improved camera lens and sensor hardware, cutting-edge AI engine functions are essential for driving smartphone photography and videography innovation forward. But your phone needs the necessary hardware to achieve this goal.
The combination of Spectra 580 triple ISP and Hexagon 780 artificial intelligence processor inside Qualcomm Snapdragon 888 provides the functions required for next-generation mobile photography. ISP provides the necessary data throughput for the 4K video and multi-camera technology that now penetrates the smartphone industry. Hexagon 780 provides additional processing power for the most advanced computational photography technology, including object segmentation and real-time bokeh blur. In addition to improving the consistency and quality of photos, Qualcomm’s AI engine is also promoting the capture of perfect snapshots and videos in new ways.
The important thing is that the AI camera function is creating the main difference between products, providing users with more powerful tools that can not only capture the perfect shot every time, but also make their photography and video more creative. Whether it is simulating the soft depth of field of a DSLR camera, accurately tracking your subject, or replacing the entire background in real time, the powerful AI engine pushes mobile photography to new heights.