Pixal3D: Open Source 3D Asset Generation from a Single Image (MIT)

Tencent ARC Lab, in collaboration with Tsinghua University and Victoria University of Wellington, released Pixal3D under the MIT license in May 2026. The model generates high fidelity 3D assets from a single image, producing detailed geometry and PBR (Physically Based Rendering) textures in one pipeline. The paper was accepted to SIGGRAPH 2026, the principal peer reviewed venue for computer graphics research.

The Problem It Solves

Existing image-to-3D models generate assets in a canonical pose, introducing an ambiguity: the model must infer spatial structure from image pixels without a direct link between them. Pixal3D's authors describe this as "an implicit 2D-3D correspondence issue" that reduces fidelity in existing approaches.

Pixal3D generates 3D directly in the coordinate space of the input image rather than a canonical one. That alignment removes the ambiguity by preserving the spatial relationship between input pixels and output geometry throughout the generation process.

How It Works

The result is a 3D mesh in GLB format, a standard production format compatible with Blender, Unreal Engine, and most professional 3D pipelines. PBR textures are output alongside the geometry, removing the need for a separate texturing step.

Example Outputs

The model also supports input from multiple views when additional reference images are available. The Image Back Projection Conditioner aggregates feature volumes from each view, improving accuracy for complex assets. A scene synthesis pipeline extends the single asset workflow to modular, object separated 3D scenes.

Two Resolution Modes

Pixal3D runs in two configurations based on available VRAM. The standard mode operates at resolution 1536 for the highest output fidelity. A low VRAM mode runs at resolution 1024, making it accessible on consumer grade hardware. Both modes produce GLB output compatible with the same production pipeline.

Access and Hardware Requirements

The standard mode at resolution 1536 requires a GPU with 24GB of VRAM, which covers current generation professional cards including the NVIDIA RTX 4090 and A6000. The low VRAM mode at resolution 1024 runs on GPUs from 12GB upward, making it accessible on hardware available to individual artists and small studios.

Model weights are hosted on HuggingFace under the TencentARC organization. The repository includes inference scripts, a Gradio demo for local testing, and documentation for integrating Pixal3D output into downstream pipelines. Total model size is approximately 8.5GB. Runtime on a 24GB GPU for a single asset at full resolution is under two minutes.

The Scene Synthesis Extension

Beyond single asset generation, the team released a scene synthesis pipeline that decomposes input images into individual objects, generates a 3D asset for each one, and assembles them into a spatially coherent scene. The pipeline operates object by object rather than generating a monolithic scene mesh, which preserves individual asset detail and allows selective replacement of specific elements.

That approach has direct applications for set extension and prop replacement in visual effects. A production photographing a practical set could generate 3D replacements for specific objects, composite them back into the original footage, and achieve consistent photorealistic rendering without modeling the objects from scratch. The pipeline processes each object independently, so replacing one element does not require regenerating the entire scene.

SIGGRAPH Acceptance and What It Signals

SIGGRAPH 2026 acceptance means the Pixal3D paper went through peer review by computer graphics researchers. The venue publishes roughly 25% of submissions across all categories. For an image-to-3D method to clear that bar, the reviewers need to be satisfied that the results are reproducible, the comparisons against baselines are fair, and the method contributes something the field did not already have.

The paper's specific contribution, pixel aligned geometry generation in the input image's coordinate space rather than a canonical pose, addresses a problem explicitly identified in prior work on single image 3D reconstruction. SIGGRAPH acceptance does not mean Pixal3D is the best method for every 3D generation task. It means the method is a verifiable advance on at least one dimension that the research community recognizes as meaningful.

Comparison With Prior Open Source Options

Before Pixal3D, the practical open source alternatives for image-to-3D generation included Zero123++ and the TRELLIS family. Zero123++ generates novel views rather than explicit 3D geometry, requiring a separate reconstruction step. TRELLIS generates structured latents that can be decoded into meshes, but its canonical pose assumption introduces the correspondence ambiguity Pixal3D is designed to avoid.

For production teams, the practical difference between these approaches shows up in the time between input image and usable GLB file. Methods requiring multiple reconstruction steps add latency and introduce error accumulation between stages. Pixal3D's single pipeline from input to GLB with PBR textures compresses that workflow, though the 24GB VRAM requirement for the full resolution mode limits where it can run.

What It Means for VFX and Filmmakers

PBR texture output is the practical differentiator from most open source 3D generation models. Geometry only output requires a separate texturing stage before assets are usable in production. Pixal3D delivers both in a single inference pass, compressing the pipeline from concept image to production ready 3D asset.

Acceptance to SIGGRAPH 2026 gives the method peer reviewed credibility, which matters for studios evaluating open source 3D tools for professional workflows. The MIT license removes licensing risk for commercial productions.

For character design, prop modeling, and set dressing, a single image input means a reference photo or concept art can become a 3D asset without a full modeling session. The approach complements 3DreamBooth for 3D subject driven video generation, which focuses on consistency across frames rather than single asset output fidelity. For teams building 3D to video pipelines, VideoFrom3D provides the downstream step that converts generated 3D assets into animated video sequences. Input reference images can be generated through AI FILMS Studio's image workspace.

Why PBR Textures Matter for Production Use

Physically Based Rendering is the standard shading model used in modern real-time and offline renderers, including Unreal Engine, Unity, Blender's Cycles and EEVEE renderers, and most VFX compositing software. PBR textures specify surface properties, roughness, metalness, and base color, in a format that behaves predictably under different lighting conditions rather than baking a specific lighting setup into the texture.

A 3D asset with PBR textures can be placed into a new environment and relit by the environment's lighting without the surface appearing inconsistent with its surroundings. A 3D asset with only a diffuse texture, or no texture at all, requires manual material assignment before it integrates naturally into a production scene. For productions that need to reuse assets across multiple scenes with different lighting, PBR output is the baseline for production viability.

The Research Team and Institutional Context

Pixal3D was produced by a four institution collaboration. Tencent ARC Lab, the research division of one of China's largest technology companies, led the project. Tsinghua University contributed academic researchers. Victoria University of Wellington, a New Zealand research university with an active computational media program, provided additional collaborators. The collaboration structure reflects how much AI research now crosses institutional and national boundaries.

Tencent ARC Lab has produced a consistent stream of open source computer vision and generative media research over the past three years. Previous releases include AnyDoor, an image customization model, and RAVE, a video editing approach. Releasing under MIT rather than a more restrictive research license is a deliberate strategy. Open commercial licensing drives adoption, which produces citations and follow up work, which strengthens the lab's research standing.

For studios evaluating the institutional provenance of open source tools, Tencent ARC Lab's publication record provides a different risk profile than an anonymous GitHub repository. The SIGGRAPH acceptance and the authorship list spanning multiple institutions are credibility signals that matter in corporate procurement and legal review processes.

How Pixal3D Handles Textureless Regions

One practical limitation of single image 3D generation is that the input image can only provide texture information for visible surfaces. Regions of the object that face away from the camera in the input image have no texture information to draw from. Pixal3D addresses this through a learned prior over surface texture distributions: the model infers likely texture for occluded regions based on training data rather than leaving them blank.

The quality of inferred texture on occluded regions varies with the training distribution. Common object categories, such as furniture, vehicles, and consumer products, have well-represented texture distributions in the training data. Novel or unusual objects may have inferred textures that require manual correction. The low VRAM mode, running at resolution 1024, produces less detailed inferred texture on occluded surfaces than the full resolution mode, which is the main practical difference between the two configurations beyond raw output fidelity.

Implications for Production Asset Pipelines

The conventional 3D asset pipeline for a visual effects production runs from concept art through modeling, UV unwrapping, and texture creation, typically requiring a dedicated modeling artist and multiple review cycles. For hero assets, that pipeline is appropriate: the detail and control it provides justify the time. For secondary assets and environmental props, the time-to-asset ratio becomes harder to justify as production scales.

Pixal3D compresses the secondary asset pipeline significantly. A props department can generate a reference photograph of an object, run it through Pixal3D, and receive a textured GLB within two minutes. That output may require cleanup for close-up use but is production ready for mid-ground and background appearances. The workflow shifts time from modeling to quality review, which is a more scalable division of labor on a production with hundreds of props.

The MIT license means studios can integrate Pixal3D into proprietary pipelines without licensing negotiations. For studios with internal tool development teams, that removes a category of legal and procurement friction that has historically slowed open source adoption in production environments.

Deployment Considerations for Independent Productions

Independent productions evaluating Pixal3D face a practical split based on hardware. Studios with an RTX 4090 or equivalent can run the full resolution mode locally with inference times under two minutes. Productions without that hardware have two options: the low VRAM mode at 1024 resolution, or cloud inference through the HuggingFace Space, which allows testing without any local GPU requirement.

Cloud inference via the HuggingFace Space is the practical entry point for most independent productions before a hardware investment decision. Output at 1024 resolution is suitable for evaluating workflow fit and checking whether the model handles the specific asset categories a production needs. The step to 1536 resolution and local deployment makes sense once a production has confirmed that Pixal3D produces usable assets for its specific requirements.

What the Research Community Will Test Next

SIGGRAPH 2026 acceptance invites follow up work from the research community. The natural next comparisons are accuracy on different asset categories, performance on low-texture objects where the color and material information in the input image is limited, and scaling behavior when the multi-view pipeline receives more than four or five input images.

The scene synthesis extension is likely to attract independent evaluation, since assembling multiple AI generated assets into a coherent scene introduces additional error modes beyond single asset generation. Object boundary alignment, shadow consistency, and perspective coherence across multiple AI generated objects in a single scene are problems the current paper addresses architecturally but does not benchmark in detail. Those benchmarks will appear in follow up work over the next year.

---

Sources

arXiv: Pixal3D: Pixel-Aligned 3D Generation from Images GitHub: TencentARC/Pixal3D HuggingFace: TencentARC/Pixal3D Project Page: ldyang694.github.io/projects/pixal3d/