Observe
Encode a flexible number of RGB views with calibrated camera poses, resolutions, and aspect ratios.
NeurIPS 2026
Any 3D Scene is Worth 1K Tokens.
3DRAE turns calibrated multi-view observations into a compact, view-decoupled 3D representation. 3DDiT generates directly in that latent space.
256~1Kfixed latent tokens
≈2s3D generation
Flexibleview count & trajectory
3D-grounded scene representation and generationLooping teaser
One representation, many views
Represent once. Query anywhere.
3DRAE removes repeated information across views while retaining semantic appearance and scene geometry.
Ground
Fuse view-coupled 2D foundation features into a fixed-length, view-decoupled 3D latent.
Generate & decode
Denoise directly in 3D latent space, then query RGB images and optional point maps at arbitrary cameras.
Open research release
Models, data, and runnable code.
The static Space showcases the release. Inference runs locally from the published checkpoints and examples.
Released variants
Choose semantic backbone, representation size, and output capabilities for your use case.
DINOv2 · 1024 tokens3DRAE · 3DDiT (RGB & point maps)
DINOv2 · 256 tokens3DRAE (RGB)
SigLIP2 · 1024 tokens3DRAE · 3DDiT (RGB)
Quick start
git clone https://github.com/WU-CVGL/3DRAE.git
cd 3DRAE
uv sync --python 3.10
source .venv/bin/activate
hf download tracywei/3drae \
--exclude "README.md" ".gitattributes" \
--local-dir checkpoints
python scripts/generate.py \
--scene examples/scene_000 \
--appearance-condition-frame-ids 0 \
--trajectory-mode synthetic-orbit \
--output-dir outputs/generationExplore 3DRAE
Everything in one release.
Citation
Build on 3D-grounded tokens.
BibTeX
@inproceedings{wei20263drae,
title = {Any 3D Scene is Worth 1K Tokens: 3D-Grounded Representation for Scene Generation at Scale},
author = {Wei, Dongxu and Xu, Qi and Li, Zhiqi and Zhou, Hangning and Qiu, Cong and Qin, Hailong and Yang, Mu and Cui, Zhaopeng and Liu, Peidong},
booktitle = {Advances in Neural Information Processing Systems},
year = {2026}
}