Spatial Retrieval Augmented Autonomous Driving

SpatialRetrievalAD Dataset & Devkit

Xiaosong Jia^1*, Chenhe Zhang^1*, Yule Jiang^2*, Songbur Wong^2*
Zhiyuan Zhang², Chen Chen³, Shaofeng Zhang⁴, Xuanhe Zhou², Xue Yang^2†
Junchi Yan^2†, Yu-Gang Jiang¹

¹Institute of Trustworthy Embodied AI, Fudan University
²Shanghai Jiao Tong University
³Key Laboratory of Target Cognition and Application Technology, Aerospace Information Research Institute, Chinese Academy of Sciences
⁴University of Science and Technology of China

^*Equal contribution   ^†Corresponding authors

📧 Primary Contact: Xiaosong Jia (jiaxiaosong@fudan.edu.cn)

SpatialRetrievalAD.mp4

🌍 Introduction

This repository provides the official devkit for the nuScenes-Geography dataset introduced in our paper, "Spatial Retrieval Augmented Autonomous Driving".

We introduce a novel Spatial Retrieval Paradigm that retrieves offline geographic images (Satellite/Streetview) based on GPS coordinates to enhance autonomous driving tasks. For multi-task learning, we design a plug-and-play Spatial Retrieval Adapter and a Reliability Estimation Gate to robustly fuse this external knowledge into model representations, followed retrieval injection mode of Bench2Drive-R.

The following figure shows the spatial distribution and coverage status of our released nuScenes-Geography dataset across the nuScenes scenes. Please refer to our paper for a detailed description and analysis.

📖 Table of Contents

• Introduction
• News
• Multi-Task Implementations
• Dataset & Devkit Installation
  • Installing the Devkit
  • Downloading the Dataset
• Dataset Reconstruction
• Usage in Your Own Project
• Acknowledgments

🔥 News

• [2025-12-09] The nuScenes-Geography dataset and curation tools are released.

🚀 Multi-Task Implementations

All implementation repositories are hosted under the SpatialRetrievalAD organization.

Tasks	Repositories
Generative World Model	Generative-World-Model
End-to-End Planning	End2End-Planning
Online Mapping	Online Mapping
Occupancy Prediction	Occupancy-Prediction
3D Detection	3D-Detection

📦 Dataset & Devkit Installation

🛠️ Step 1: Installing the Devkit

Clone the official devkit repository from GitHub and install it in editable mode:

git clone https://github.com/SpatialRetrievalAD/SpatialRetrievalAD-Dataset-Devkit.git
cd SpatialRetrievalAD-Dataset-Devkit
pip install -e .

⬇️ Step 2: Downloading the Dataset

Download the dataset from Hugging Face:

👉 SpatialRetrievalAD/nuScenes-Geography-Data

hf download SpatialRetrievalAD/nuScenes-Geography-Data --repo-type=dataset

The dataset directory is organized as follows:

nuScenes-Geography-Data
├── frame_metadata.json
├── pano_metadata.json
├── unavailable_metadata.json
├── sat
│   ├── boston-seaport.png
│   ├── singapore-hollandvillage.png
│   ├── singapore-onenorth.png
│   └── singapore-queenstown.png
└── streetview
    ├── quality_labels.json
    └── panos
        ├── <pano_id_0>.jpg
        └── <pano_id_1>.jpg

The following figure show the correspondence between Geography images and nuScenes images:

🔍 Usage in Your Own Project

Get started with the dataset by following the Usage in Your Own Project guide.

🔧 Dataset Reconstruction

For more details, please refer to Dataset Reconstruction

🖊️ Citation

@misc{spad,
      title={Spatial Retrieval Augmented Autonomous Driving}, 
      author={Xiaosong Jia and Chenhe Zhang and Yule Jiang and Songbur Wong and Zhiyuan Zhang and Chen Chen and Shaofeng Zhang and Xuanhe Zhou and Xue Yang and Junchi Yan and Yu-Gang Jiang},
      year={2025},
      eprint={2512.06865},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2512.06865}, 
}

🙏 Acknowledgments

We thank the following projects for their contributions to the development of this project: BEVDet, BEVFormer, FB-OCC, FlashOCC, MagicDriveDiT, MapTR, MapTRv2, nuScenes, PETR, UniMLVG, VAD