Improving AI-generated images, pixel by pixel

edited by Swati Mestri, reviewed by Robert Egan

This article has been reviewed according to Science X's editorial process and policies. Editors have highlighted the following attributes while ensuring the content's credibility:

Add to Preferred Sources Qualitative results on ImageNet-256×256. Selected PixelDiT2-H/16 class-conditional samples. Credit: arXiv (2026). DOI: 10.48550/arxiv.2609.24919 If you have ever used an artificial intelligence text-to-image generator like ChatGPT or Google Gemini, you may have encountered a common but maddening problem: The image looks nearly perfect, but when you ask the AI to fix one small detail, it messes up something unrelated.

The issue is caused by the way the AI models are built. Most state-of-the-art AI image generators use a diffusion model in latent space, meaning they compress concepts such as a cat or a dog into their essential details and store them so they can be retrieved quickly and efficiently when it's time to create an image. This is part of what gives the models their seemingly magical ability to generate the images you describe in seconds. But the method has its drawbacks.

"Using the latent space helps cut down the computation, but the problem is that you lose controllability because you are not adjusting individual pixels," says Jiebo Luo, the Albert Arendt Hopeman professor of engineering at the University of Rochester. "As a result, you can't easily control specific parts of the images you generate."

Luo's lab and collaborators at the AI infrastructure company NVIDIA formed one of the first teams to develop a model that trades latent space for pixel space, dubbed PixelDiT.

They presented their first iteration at the 2026 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). The model allows users to zero in on certain regions of an image, like fixing just the pixels used to illustrate a pair of glasses on a person's face or adjusting the color of a car.

The team has since created an even more powerful model, PixelDiT2, that implements pixel space diffusion efficiently to achieve almost the same performance as the state-of-the-art latent space models while providing control over individual pixels.

The team will present a paper showcasing PixelDiT2 at the Neural Information Processing Systems conference (NeurIPS) this December. The findings are available on the arXiv preprint server.

Luo and his collaborators, who include computer science Ph.D. student and NVIDIA intern Yongsheng Yu as well as alumnus Wei Xiong '22 Ph.D. (computer science), a senior researcher at NVIDIA, have now turned their attention to an even more difficult problem: video generation.

"Video is a big challenge because you have to worry about smooth transitions from frame to frame; otherwise, it looks funny to the human eye," says Luo. "You also have a lot more data to work with."

But Luo says the progress the team has made with PixelDiT2 leaves them in a strong position to pursue AI video generation in pixel space, which would provide users with powerful new levels of control over the videos they generate.

Yongsheng Yu et al., PixelDiT2: Representation-Grounded Pixel Diffusion Transformers, arXiv (2026). DOI: 10.48550/arxiv.2609.24919 Journal information: arXiv

Swati Mestri holds a bachelor's degree in Electronics Engineering and has worked as a content editor since 2019. She has experience editing research documents across technology, health care, and materials science, and has a particular interest in technology and space. Full profile →

Bachelor's in mathematical biology, Master's in creative writing. Well-traveled with unique perspectives on science and language. Full profile →