Hugging Face Transformers v5.21.0 Released with New Models Including MiniCPM-OCR

Hugging Face released Transformers v5.21.0. New model additions include MiniCPM-OCR, a document-understanding model from OpenBMB's MiniCPM-V family. It maintains the MiniCPM-V 4.6 architecture, pairing a SigLIP vision encoder and a window-attention merger with a Qwen3.5 language model. It is specialized for reading text-dense images such as scanned pages, screenshots, receipts, and tables.

Support for Flint, a mixture-of-experts language model from Recursive, was also added. Compared to a standard SwiGLU MoE transformer such as Qwen3-MoE, Flint uses hybrid attention: sliding-window attention with rotary embeddings over the current token and the previous 512 tokens in most layers, and full attention without positional encoding in the remaining layers. It also features learned attention sinks, as in GPT-OSS, with per-head RMS-normalized queries and keys. Each sparse layer routes a token to 6 of 256 experts by sigmoid score and adds an ungated shared expert, while the first two layers are dense.

Additionally, support for UltraVBert was included. UltraVBert extends ModernVBert by pairing the UltraBERT text encoder with a SigLIP vision encoder for visual document understanding and retrieval. Its image processor splits each page into 512-pixel tiles plus a global view.