Researchers have published Lang-SVG, a method that reconstructs raster images as hierarchical, semantically labeled SVG trees rather than flat layers of vector primitives. Published on arXiv on November 1, 2025, the work addresses a persistent bottleneck in image vectorization: while recent differentiable methods excel at matching pixel-level fidelity, they typically organize SVG primitives as unstructured layers, scattering the parts of a single object across multiple layers and making targeted edits difficult.
Lang-SVG replaces the flat layer model with an explicit parent-child tree structure. The authors initialize this hierarchy using open-vocabulary segmentation models like Semantic-SAM and UnSAM, which provide coarse-to-fine masks. They then refine the structure using depth-adaptive simplification, tree-guided pruning, and a multimodal large language model (MLLM) to assign semantic tags to unlabeled nodes.
Building a Semantic SVG Tree
The pipeline begins by extracting masks from open-vocabulary segmentation models. These masks are organized into a tree by evaluating spatial containment: a node becomes a child of the deepest valid parent based on mask intersection, breaking ties with intersection-over-union (IoU) scores. If no valid parent exists, the node attaches to a full-image root. This geometric assignment rule supports masks at different granularities, though the authors note that imperfect segmentation and partially overlapping regions can still produce ambiguous relations.
Converting these masks into vector primitives introduces a complexity trade-off. Coarse masks at the top of the tree often have irregular boundaries, but they are occluded by deeper layers and contribute less to the final rendered image. Fitting them with high-complexity polygons wastes the available control-point budget. Lang-SVG applies depth-adaptive simplification, using a higher approximation tolerance for root nodes to yield simpler polygons, while leaf nodes use lower tolerance to preserve fine-grained details. This allocates geometric complexity in proportion to visual contribution.
To prevent the optimization from allocating primitives to high-frequency textures rather than meaningful structures, the authors introduce L1-SAM-Prompting. Traditional vectorization methods initialize primitives by clustering pixels based on L1 loss maps, which tends to fit noise and high-frequency discrepancies. Instead, Lang-SVG extracts bounding boxes and mean points from the L1 loss map to prompt SAM. This generates supplementary masks that emphasize coherent structures over pixel-level noise, bridging the gap between loss-driven clustering and foundation model segmentation.
The resulting tree often contains redundant or overlapping regions. Lang-SVG uses tree-guided pruning and merging to remove them. Rather than relying solely on geometric overlap, the method compares the color of an SVG layer to its parent. If an unlabeled child node shares high color similarity with its parent, it is removed. Parent-child or sibling nodes can be merged if they exhibit similar color and sufficient spatial overlap, though nodes with existing semantic tags are preserved to retain semantic information.
Because initial segmentation masks rarely cover every region or assign perfect labels, the authors employ a tree-guided semantic tagging step. They pass the full image, a crop of the target SVG region, and its RGB color to Qwen3.5-27B. The model is prompted with the node's parent, ancestor, sibling, and child tags to infer a semantic label. This structured decision task improves tagging accuracy by leveraging hierarchical context that flat representations lack. The authors evaluate this in both closed-set and open-vocabulary settings, assigning a default texture or unknown tag to ambiguous nodes to avoid unreliable semantic over-assignment.
Recent lines of work have begun reformulating vectorization as an SVG code generation task with MLLMs. Models like StarVector, OmniSVG, and InternSVG predict richer primitives directly in the SVG code space, indicating a shift toward editability and semantic decomposition. However, these methods still represent SVGs as an ordered set of primitives, or first separate semantic regions in the raster domain and then vectorize them independently. Lang-SVG explicitly encodes parent-child relations and semantics among SVG nodes, enabling applications such as grouping and semantic analysis of vector primitives that flat representations cannot support.
Illustrative image: Broesis / Pixabay
Structural and Fidelity Results
The authors evaluate Lang-SVG on ADE20K-Part234, a dataset annotated with object, part, and subpart masks, as well as 40 complex SVG-style images from Iconfont. They introduce a new evaluation protocol that measures structural quality and semantic part alignment alongside traditional reconstruction metrics. Standard photometric and perceptual metrics cannot determine whether SVG primitives align with human-perceived object-part structures, so the authors combine StructXLIP, a structure-aware vision-language model, with part/subpart recall at different IoU thresholds.
On ADE20K, Lang-SVG improves the StructXLIP score from 0.462 to 0.506 and increases average part recall from 28.8% to 48.5% at a budget of 576 control points. The tree structure also proves useful for semantic reasoning: incorporating SVG tree context into the MLLM prompt improves semantic tagging accuracy from 58.5% to 81.3% in masked-label evaluations.
The method maintains high rendering fidelity while achieving these structural gains. Under the same 576 control point budget, Lang-SVG outperforms baselines like DiffVG, LIVE, SGLIVE, and LIVSS on both natural and SVG-style images. The following table reports performance on the SVG-style images:
| Method | SSIM | PSNR | LPIPS | S-XLIP | CLIP |
|---|---|---|---|---|---|
| DiffVG | 0.9125 | 26.46 | 0.1274 | 0.8899 | 0.9397 |
| LIVE | 0.9349 | 28.19 | 0.0882 | 0.9616 | 0.9704 |
| SGLIVE | 0.9308 | 28.62 | 0.0815 | 0.9546 | 0.9728 |
| LIVSS | 0.9319 | 28.24 | 0.0945 | 0.9491 | 0.9696 |
| Lang-SVG | 0.9384 | 29.28 | 0.0782 | 0.9549 | 0.9766 |
The results are measured on a fixed control point budget, which is the standard constraint in vectorization benchmarks. The authors note that Lang-SVG achieves the best reconstruction performance among the compared methods while simultaneously improving structural alignment. SGLIVE previously incorporated SAM to improve structural alignment, and LIVSS used diffusion-based simplification to generate multi-level semantic masks, but Lang-SVG avoids the geometric distortions that diffusion-based approaches can introduce.
Practical Constraints
Lang-SVG shifts the vectorization workflow from a purely differentiable optimization loop to a pipeline that relies heavily on external foundation models. The method depends on Semantic-SAM or UnSAM for mask generation and Qwen3.5-27B for semantic tagging. This introduces significant inference latency and computational overhead compared to end-to-end differentiable vectorizers like DiffVG or Bézier Splatting. Practitioners looking to deploy this in production will need to account for the cost of running large vision-language models and segmentation networks alongside the vectorization pipeline.
Furthermore, the quality of the resulting SVG tree is bounded by the accuracy of the initial segmentation masks. The authors acknowledge that imperfect segmentation and partially overlapping regions can still produce ambiguous parent-child relations, requiring the pruning and merging steps to clean up the hierarchy. The pipeline also assumes that a limited control-point budget can be distributed effectively across the tree, which may not hold for images with extremely dense, non-hierarchical textures.
The code and project page are available at https://xiliu8006.github.io/Lang-SVG/. The work is published as a preprint on arXiv.
Sources
PNEUMETRON EDITORIAL
Pneumetron's pipeline pairs AI-assisted drafting with human editorial review before publishing — our goal is to make staying informed easier, not to replace real reporting.




