The software giant Adobe has introduced a new AI feature, known as VideoGigaGAN, that promises to upscale videos by eight times the original resolution while minimizing common visual artifacts. This means you can turn a 720p HD resolution video into 4K with very little loss in quality.
The Rise of AI in Video Editing
Adobe has been integrating artificial intelligence into its products at an impressive pace. Some particularly impressive features include AI-driven object addition and removal in Premiere Pro and text-based image generation in Photoshop.
VideoGigaGAN represents the latest in Adobe’s suite of AI tools. The new tool enhances video sharpness and detail without the usual downsides like flickering or distortion.
The technology leverages advancements in General Adversarial Networks (GANs), a deep learning architecture comprised of two different neural networks — a generator and a discriminator. The generator’s role is to create data so convincing that it could be mistaken for real data from the target distribution, while the discriminator’s job is to distinguish between the generator’s fake data and the actual real data (or the training data). The two neural networks are pitted against each other. Through their competition, the generator learns to produce increasingly realistic data in order to one-up the discriminator, and the discriminator becomes better at detecting subtleties that differentiate real from fake.
GANs have pushed the boundaries of image, video, and voice AI generation. VideoGigaGAN is the next iteration in this exciting field, using its advanced deep-learning model to ‘fill in the blanks’ by adding many new pixels to upscale videos. It excels in both adding sharpness and preserving detail, something that previous models could not achieve simultaneously.
Understanding VideoGigaGAN
VideoGigaGAN’s secret sauce is a mix of neural networks, including “temporal attention,” a system that reduces artifacts that can accumulate over time, and feature propagation, which is responsible for adding details where none previously existed. Sprinkled on top there’s some anti-aliasing and a feature called “HF shuttle” to enhance high-frequency features.
The result is impressive. Low-resolution videos look much more crisp after they are processed through VideoGigaGAN. It does well even with things that have been known to be challenging, such as hair and skin texture.
It’s pretty cool — but this is also just generative AI, at the end of the day. What this means is that the added details don’t exist in reality. These are newly added pixels that have been predicted with a high degree of confidence by the AI. Perhaps the AI adds a small pimple of coloring on your face that doesn’t exist. That’s certainly a possibility. Bearing this in mind, this kind of technology will never stand in court as forensic evidence — just in case you were getting CSI vibes (I certainly did).
Despite its limitations, the technology presents an exciting step forward. Video producers might soon have the ability to transform older, lower-quality footage into high-resolution creations with enhanced detail. Adobe is not alone either. There’s a GAN-like competition among software developers too, with NVIDIA and Microsoft working hard on their own AI upscaling technologies.
One thing is clear: the future of video editing looks sharper than ever, thanks to AI.
VideoGigaGAN was described in a recent study published in the pre-print server arXiv.
