PixelUMM replaces separate semantic and reconstruction-oriented visual encoders with a shared pixel-space interface. Images become spatial patches, videos become spatiotemporal tubelets, and lightweight linear projections connect both to one multimodal backbone.
The model jointly learns autoregressive text prediction for understanding and pixel-space flow matching for generation. It achieves competitive results across image and video tasks while avoiding duplicated visual streams that increase context length, memory use, and integration complexity.
Controlled experiments indicate that lighter spatial and temporal compression reduces generation loss, while convolutional output heads help suppress visible patch artifacts.