MatAnyone-E

High-resolution video matting requires preserving fine details, such as hair strands and soft transparency, while maintaining consistency across frames. Memory-based methods achieve strong matting quality by matching current-frame features with information stored from earlier frames, but this process becomes increasingly costly as resolution grows. We present MatAnyone-E, an efficient extension of MatAnyone 2 that focuses computation where it is most needed. Our approach first performs low-resolution propagation to identify uncertain regions that require closer inspection and the memory candidates most relevant to them. It then performs high-resolution matching only for these regions using the selected memory, while reusing low-resolution matching results elsewhere. This strategy reduces both the number of locations requiring detailed matching and the amount of memory searched, while preserving the information needed to produce a complete matte. We further reduce the cost of generating high-resolution mattes with a lightweight decoder designed around measured runtime bottlenecks and trained to reproduce the original decoder's representations. Experiments show that MatAnyone-E maintains matting quality comparable to MatAnyone 2 while achieving an estimated 7.4× inference speedup and reducing peak GPU memory usage to 7.6% of MatAnyone 2 at 4K. With compilation-enabled inference on a single NVIDIA H200 GPU, it reaches 73.07 FPS at 1080p and 28.62 FPS at 4K.

MatAnyone-E keeps the architecture of MatAnyone 2 and replaces the two parts whose cost grows fastest with resolution. Sparse Memory Propagation first runs a cheap low-resolution pass. Its uncertainty selects the query locations that need detailed matching (query pruning), and its affinities select the memory most relevant to them (memory pruning). High-resolution matching then runs only on these query–memory pairs, and the upsampled low-resolution readout is reused everywhere else. A lightweight decoder, designed from stage-wise latency profiling and trained with feature distillation from the original decoder, further cuts the cost of decoding high-resolution mattes.