Unsloth Releases Inkling-GGUF: A Multimodal MoE Model for Developers

What Changed

Unsloth has officially released Inkling-GGUF, a new general-purpose multimodal model available on Hugging Face. This release provides a GGUF quantized version of the Inkling model, making it accessible for a wider range of local deployments and applications. Inkling is designed to accept text, image, and audio inputs, and subsequently generate text outputs. The model is intended for use in English and other languages, supporting various coding languages, and is geared towards developers creating AI-powered applications.

Technical Details

Inkling is an autoregressive transformer model with a 66-layer decoder-only architecture. A key feature of its design is a sparse Mixture-of-Experts (MoE) feed-forward backbone. In this configuration, each token is routed to 6 out of 256 experts, with an additional 2 shared experts active on every token. The attention mechanism employs a hybrid of local and global layers.

The model is natively multimodal, processing images and video through a hierarchical patch encoder, and audio via discrete token encoding. All input modalities are projected into a shared hidden space and processed jointly by the decoder. Inkling boasts 975 billion total parameters, with 41 billion active parameters. It supports BF16 and NVFP4 numerics.

Input modalities include UTF-8 encoded text, pixel-based image formats (ideally between 40px and 4096px per dimension), and WAV audio sampled at 16kHz (ideally under 20 minutes in length). The model generates output as UTF-8 encoded text.

Training data for Inkling was sourced from a broad variety of content types, including publicly available text, images, audio, and video, as well as third-party acquisitions and synthetically generated or augmented data. The curation process involved cleaning, processing, and modifying datasets, including deduplication and filtering for quality and safety.

For local deployment, Inkling supports integration with several open-source libraries, including SGLang, vLLM, TokenSpeed, Unsloth, and Huggingface. API access is also available through third-party inference providers.

Benchmark Analysis

Evaluations for Inkling were conducted at an effort of 0.99, with comparison scores from the r/acc leaderboard as of July 14, 2026. The benchmarks compare Inkling against both open-weights models (Nemotron 3 Ultra, Kimi K2.5, Kimi K2.6, GLM 5.2, DeepSeek V4 Pro) and closed-weights models (Gemini 3.1 Pro, Claude Fable 5, GPT 5.6 Sol).

In reasoning tasks, Inkling scored 30.0% on HLE (text only) and 46.0% on HLE (with tools). It achieved 97.1% on AIME 2026 and 87.9% on GPQA Diamond. For agentic coding, Inkling demonstrated 77.6% on SWEBench Verified, 54.3% on SWEBench Pro (Public), and 63.8 on Terminal Bench 2.1 (Best Harness). In general agentic tasks, it scored 74.1% on MCP Atlas and 22.3% on Tau 3 Banking.

Factuality benchmarks show Inkling at 77.1% on BrowseComp (w/ Ctx), 43.9% on SimpleQA Verified, and 1.0% on AA Omniscience. In chat evaluations, Inkling achieved 79.8% on IFBench and 88.7% on Global-MMLU-Lite. Vision capabilities were assessed at 73.3% on MMMU Pro (Standard 10), 78.1% on Charxiv RQ, and 82.0% on Charxiv RQ (with python). Audio benchmarks include 56.6% on Audio MC, 77.2% on MMAU, and 91.4% on VoiceBench.

Safety evaluations reported Inkling at 78.0% on FORTRESS (Adversarial), 95.9% on FORTRESS (Benign), and 98.6% on StrongREJECT.

Developer Implications

Inkling's multimodal capabilities, supporting text, image, and audio inputs, offer developers a versatile foundation for building diverse AI applications. Its open-weight release facilitates research, fine-tuning, and integration into third-party products. The model's design for agentic and tool-use systems, coding assistants, chatbots, and retrieval-augmented generation systems positions it as a tool for various development needs.

Developers can leverage Inkling for general-purpose conversational use and instruction-following tasks. The availability of local deployment options through SGLang, vLLM, TokenSpeed, Unsloth, and Huggingface provides flexibility in deployment environments. The GGUF format further enhances its accessibility for local inference on consumer hardware.

However, developers should be aware of common foundation model limitations, including potential for hallucination, occasional instruction-following failures, and degraded performance in extended multi-turn conversations. Biases from training data may also be reflected in outputs. Unsloth recommends human oversight for high-stakes or safety-critical contexts and encourages developers to conduct their own evaluations for specific use cases. Implementing additional safeguards, such as content filtering and rate limiting, at the application layer is advised, especially for open deployment contexts.

Bottom Line

Unsloth's Inkling-GGUF provides a multimodal, sparse Mixture-of-Experts model designed for a range of AI development tasks. Its ability to process text, image, and audio inputs and generate text outputs, combined with multilingual support and local deployment options, makes it a flexible tool for developers. While offering strong performance across various benchmarks, developers should implement robust safeguards and human oversight, particularly in sensitive applications, to mitigate inherent risks associated with large foundation models.