Spatial Retrieval Augmented Autonomous Driving

🗺️ Task: 3D Occupancy Prediction

📖 Introduction

This repository contains the official implementation of the Occupancy Prediction task from our paper: "Spatial Retrieval Augmented Autonomous Driving".

We introduce a novel Spatial Retrieval Paradigm that retrieves offline geographic images (Satellite/Streetview) based on GPS coordinates to enhance autonomous driving tasks. For Online Mapping, we design a plug-and-play Spatial Retrieval Adapter and a Reliability Estimation Gate to robustly fuse this external knowledge into BEV representations.

We provides the implementation based on FB-OCC.

Note: If you are looking for the implementation based on FlashOcc, please go to flashocc branch.

🚀 News

• [2025-12-09] Code and checkpoints for Occupancy Prediction (FB-OCC & FlashOcc) are released!

📊 Main Results

FB-OCC - main branch

Method	Modality	mIOU	Config	Download
FB-OCC	C	39.11	5.82	-
FB-OCC + Geo	C + Geo	39.74	config	model

C: Camera, Geo: Geographic Images.

FlashOcc - flashocc branch

Method	Modality	mIOU	Config	Download
FlashOcc	C	31.92	-	-
FlashOcc + Geo	C + Geo	32.35	-	-

C: Camera, Geo: Geographic Images.

📦 Installation

Please follow the official installation instructions to configure the environment: Install

📂 Data Preparation

Step 1: NuScenes and Occ3D-nuScenes Data

Download nuScenes V1.0 full dataset data.

For Occupancy Prediction Task, you need to download extra annotation from https://github.com/Tsinghua-MARS-Lab/Occ3D

Genetate custom annotation files following FB-BEV

python tools/create_data_bevdet.py

Step 2: Geographic Data

Install nuGeo:

Following the instructions in the SpatialRetrievalAD-Dataset-Devkit project, prepare both the nuScenes-Geography dataset and its devkit.

After preparation, place the dataset directory nuScenes-Geography under data/

Step 3: Integrating Retrieved Geographic Information

Once both the base nuScenes/Occ3D data and the geographic dataset are ready, merge the geographic information into the dataset by executing:

python tools/merge_data.py

Final Folder Structure

Occupancy-FBOCC/
...
├── ckpts/
└── data/
    ├── nuscenes/
    │   ├── gts/
    │   ├── maps/
    │   ├── samples/
    │   ├── sweeps/
    │   ├── v1.0-test/
    │   ├── v1.0-trainval/
    │   ├── bevdetv2-nuscenes_infos_train.pkl
    │   └── bevdetv2-nuscenes_infos_val.pkl
    └── nuScenes-Geography-Data
        ├── satellite_data_v1.0-trainval.pkl
        ├── streetview_data_v1.0-trainval.pkl
        ├── frame_metadata.json
        ├── pano_metadata.json
        ├── unavailable_metadata.json
        ├── sat/
        ├── sat_slice/
        └── streetview
            ├── cams/
            ├── panos/
            └── quality_labels.json

🚄 Training & Evaluation

Train Occupancy-FBOCC with 8 GPUs

bash ./tools/dist_train.sh occupancy_configs/geo_fb_occ/fbocc-r50-cbgs_depth_16f_16x4_20e_geo.py 8

Eval Occupancy-FBOCC with 8 GPUs

bash ./tools/dist_test.sh occupancy_configs/geo_fb_occ/fbocc-r50-cbgs_depth_16f_16x4_20e_geo.py ./path/to/ckpts.pth 8

🖊️ Citation

@misc{spad,
      title={Spatial Retrieval Augmented Autonomous Driving}, 
      author={Xiaosong Jia and Chenhe Zhang and Yule Jiang and Songbur Wong and Zhiyuan Zhang and Chen Chen and Shaofeng Zhang and Xuanhe Zhou and Xue Yang and Junchi Yan and Yu-Gang Jiang},
      year={2025},
      eprint={2512.06865},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2512.06865}, 
}

🙏 Acknowledgements

This work is based on FB-OCC. It is also greatly inspired by the following outstanding contributions to the open-source community: BEVFormer, PETR.