GTR-Seg

Gated Token Recurrence for Efficient Dense Prediction

Zhe Feng^1,2,3, Longfei Liu², Wei Liu¹, Kai Chen⁵, Jiangang Kong¹, Wei Zhou¹, Yifeng Qian⁵, Dexiong Chen⁴, Xuanlong Yu², Xi Shen²

¹ Didi International Business Group,
² Intellindust AI Lab,
³ Institute of Automation, Chinese Academy of Sciences,
⁴ The Hong Kong University of Science and Technology (Guangzhou),
⁵ Didi Research

() corresponding author, shenxiluc@gmail.com

We introduce Gated Token Recurrence (GTR), a softmax-free recurrent vision backbone that combines gated linear attention, alternating spatial scan directions, and spatially enhanced SwiGLU blocks. GTR models deliver strong dense predictions across segmentation, pose, depth, and detection, while scaling efficiently to high-resolution inputs.

One GTR backbone, five dense prediction tasks on the same driving clip: object detection, instance segmentation, pose estimation, semantic segmentation and metric depth.

Metric depth predicted by GTR, unprojected into a 3D point cloud: an outdoor driving scene (left) and an indoor scene (right).

🔥 Changelog

• [2026-10-15] 🚀 Optimized training infrastructure with batched matching-index transfers and caching, reducing CPU–GPU synchronization.
• [2026-09-27] 🥳 Thanks to Xuban Ceccon (@EHxuban11) for integrating GTR into LibreYOLO!

📚 Table of Contents

• Overview
• Chunkwise CUDA Operator
• Getting Started
  • Prepare Environment
  • Install Packages
  • Prepare Data
• Model Zoo
• Evaluation
• Training
• Inference and Benchmarks
• TensorRT Deployment
• Acknowledgements
• Citation

Overview

Each backbone block combines GLA with Spatial SwiGLU, whose 3×3 depthwise convolution mixes neighboring patch tokens in the value branch. The 12-layer backbone alternates two-dimensional scan directions and exposes intermediate features to a lightweight three-scale projector. A query-based head decodes the resulting feature pyramid.

Chunkwise CUDA operator

Isolated GLA operator latency on RTX 4090 (log scale), comparing FLA v0.5.0 with our inference-only operator under FP16 inputs and CUDA Graph execution. The source is in engine/gtr/backbone/csrc.

Getting Started

Prepare Environment

Reference environment: Python 3.11, CUDA 12.8, PyTorch 2.11.0.

git clone https://github.com/Intellindust-AI-Lab/GTR.git && cd GTR

Install Packages

pip install -r requirements.txt
bash engine/gtr/backbone/csrc/build.sh   # optional: fused CUDA GLA kernel used for the latency numbers

Without the CUDA extension (or with GTR_DISABLE_CUDA_GLA=1) the backbone falls back to the flash-linear-attention Triton kernel.

Prepare Data

The dataset configs in configs/dataset/ expect the following layout under ./dataset:

dataset/
├── coco/{train2017,val2017,annotations}
├── cityscapes/{leftImg8bit,gtFine,train.txt,val.txt}
├── nyu/{nyu_depth_v2_labeled.mat,splits.mat}
└── DOTA/...

Model zoo

All checkpoints live in one Hugging Face repository: GTR. Fetch everything with

hf download Phoenix8125/GTR --local-dir weights

Every task checkpoint stores both the raw weights (model) and the EMA weights (ema). All reported numbers are evaluated with the EMA weights, which is what --test-only uses. Latency is the median single-image FP16 forward time on one RTX 4090 (see the paper for the full protocol).

Evaluation

# detection / instance segmentation / pose / semantic segmentation
python train.py -c configs/det/coco_finetune/gtr_s.yml             --test-only -r weights/det/gtr_s_coco.pth
python train.py -c configs/seg/coco_seg_finetune/gtrseg_s.yml      --test-only -r weights/seg/gtrseg_s_coco.pth
python train.py -c configs/pose/coco_pose_finetune/gtrpose_s.yml   --test-only -r weights/pose/gtrpose_s_coco.pth
python train.py -c configs/semseg/cityscapes_finetune/gtrsemseg_s.yml --test-only -r weights/semseg/gtrsemseg_s_cityscapes.pth

# monocular depth on NYU Depth V2 (paper protocol)
python tools/eval_nyu_depth.py -c configs/depth/pretrain/gtrdepth_s.yml -r weights/depth/gtrdepth_s.pth

# DOTA-v1.0 test: writes the Task1 files to upload to the evaluation server
python tools/dota_submit.py -c configs/obb/dota_finetune/gtrobb_s.yml -r weights/obb/gtrobb_s_dota.pth \
    --weights ema --img-dir <test patches> --out-dir outputs/dota_submit/gtrobb_s

Training

CONFIG=configs/det/coco_finetune/gtr_s.yml ./train.sh -t weights/init/gtr_s_obj365_full_raw.pth

python tools/export_init_weights.py --ckpt weights/obj365/gtr_s_obj365.pth \
    --out weights/init/gtr_s_obj365_full_raw.pth
python tools/export_init_weights.py --ckpt weights/obj365/gtr_s_obj365.pth \
    --out weights/init/gtr_s_obj365_backbone_raw.pth --backbone-only

CONFIG=configs/det/coco_finetune/gtr_s.yml              ./train.sh -t weights/init/gtr_s_obj365_full_raw.pth
CONFIG=configs/seg/coco_seg_finetune/gtrseg_s.yml       ./train.sh -t weights/init/gtr_s_coco_full_raw.pth
CONFIG=configs/pose/coco_pose_finetune/gtrpose_s.yml SEED=1 ./train.sh -t weights/init/gtr_s_coco_backbone_raw.pth
CONFIG=configs/semseg/cityscapes_finetune/gtrsemseg_s.yml ./train.sh -t weights/init/gtr_s_obj365_backbone_raw.pth
CONFIG=configs/obb/dota_finetune/gtrobb_s.yml           ./train.sh -t weights/init/gtr_s_obj365_backbone_raw.pth
CONFIG=configs/depth/pretrain/gtrdepth_s.yml            ./train.sh -t weights/obj365/gtr_s_obj365.pth

Inference and benchmarks

python tools/inference/torch_inf.py -c configs/det/coco_finetune/gtr_s.yml -r weights/det/gtr_s_coco.pth -i image.jpg
python tools/benchmark/flops.py --config configs/det/coco_finetune/gtr_s.yml
python tools/benchmark/torch_speed.py --configs configs/det/coco_finetune/gtr_s.yml --dtype fp16

TensorRT deployment🚀

TensorRT support lives in tensorrt_plugin/. It includes ONNX export, fp32/fp16 engine builds with trtexec, and PyTorch-vs-TensorRT checks. GLA is handled by a TensorRT plugin backed by the CUDA kernels in engine/gtr/backbone/csrc.

More details are in tensorrt_plugin/README.md.

Acknowledgements💗

This code base builds on EdgeCrafter, RF-DETR, DPT, RiO-DETR and flash-linear-attention.

We thank Zhangchi Hu for helpful suggestions on the oriented object detection task.

Citation

If you find GTR useful in your research or applications, please consider giving us a star ⭐ and citing it by the following BibTeX entry.

@article{gtr2026feng,
  title   = {GTR: Gated Token Recurrence for Efficient Dense Prediction},
  author  = {Feng, Zhe and Liu, Longfei and Liu, Wei and Chen, Kai and Kong, Jiangang and Zhou, Wei and Qian, Yifeng and Chen, Dexiong and Yu, Xuanlong and Shen, Xi},
  journal = {arXiv preprint arXiv:2609.26590},
  year    = {2026}
}