Skip to content
ModelsThatRun

Grounding DINO

IDEA Research's open-set detector: boxes for whatever a text prompt names. Tiny and base, Apache-2.0, through transformers.

Size

Commercial use OK for size tiny: code Apache-2.0, weights Apache-2.0. Measured on our RTX 4090: 118 ms median, 2.9 GiB VRAM peak. We scored it 48.3 box AP on COCO val2017. safetensors weights, 657 MB.

License

Commercial use OK

Code
Apache-2.0checked 26 Oct 2026
Weights
Apache-2.0all sizes, checked 26 Oct 2026

On our RTX 4090, measured

118ms median

p90 120 ms, fp32, batch 1, 1199x800

VRAM peak
2.9 GiB
Status
Runs as is
Environment, 27 Oct 2026

NVIDIA GeForce RTX 4090, driver 580.173.02, CUDA 13.0, torch 2.14.1+cu130, Python 3.12.9, transformers==5.20.2. Baseline 848 MiB. 10 warm-up + 100 timed. Scope: predict(): pre-processing, forward pass and post-processing; input file decoded beforehand.

Accuracy

48.3box APmeasured

COCO val2017, box AP, 5,000 images, fp32, pycocotools==2.0.11 · 26 Oct 2026

Paper 48.4 COCO AP, zero-shotREADME, model zoo table (GroundingDINO-T, O365, GoldG, Cap4M)

Weights

safetensors

Size
657 MB
sha256
1a2412ef99bd…c042f3
Download tiny weights

Same photos and one score threshold for every model of this task. Failures are marked.

Sizes

SizeLatencyVRAMAccuracyWeightsRunsLicense
tiny118 ms1199x8002.9 GiB48.3 box AP, measured48.4 COCO AP, zero-shot paper657 MBAs isCommercial OK
base141 ms1199x8003.3 GiB55.5 box AP, measured56.7 COCO AP paper890 MBAs isCommercial OK

From the authors

In their words, with their numbers. Ours are in the cards at the top of this page.

Paper

Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection

Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang and 6 others

arXiv 2303.05499, 9 Mar 2023

Abstract

In this paper, we present an open-set object detector, called Grounding DINO, by marrying Transformer-based detector DINO with grounded pre-training, which can detect arbitrary objects with human inputs such as category names or referring expressions. The key solution of open-set object detection is introducing language to a closed-set detector for open-set concept generalization. To effectively fuse language and vision modalities, we conceptually divide a closed-set detector into three phases and propose a tight fusion solution, which includes a feature enhancer, a language-guided query selection, and a cross-modality decoder for cross-modality fusion. While previous works mainly evaluate open-set object detection on novel categories, we propose to also perform evaluations on referring expression comprehension for objects specified with attributes. Grounding DINO performs remarkably well on all three settings, including benchmarks on COCO, LVIS, ODinW, and RefCOCO/+/g. Grounding DINO achieves a $52.5$ AP on the COCO detection zero-shot transfer benchmark, i.e., without any training data from COCO. It sets a new record on the ODinW zero-shot benchmark with a mean $26.1$ AP. Code will be available at \url{https://github.com/IDEA-Research/GroundingDINO}.

README

Copied from the authors' repository at commit 856dde2 on 3 Nov 2026. Numbers and license claims in it are theirs, not measured or checked by us. We removed badges, replaced images with links and adjusted headings.

IDEA-CVR, IDEA-Research

Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, Lei Zhang.

[Paper] [Demo] [BibTex]

PyTorch implementation and pretrained models for Grounding DINO. For details, see the paper Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection.

Helpful Tutorial

Highlight Projects

Highlight

  • Open-Set Detection. Detect everything with language!
  • High Performance. COCO zero-shot 52.5 AP (training without COCO data!). COCO fine-tune 63.0 AP.
  • Flexible. Collaboration with Stable Diffusion for Image Editting.

News

  • 2023/07/18: We release Semantic-SAM, a universal image segmentation model to enable segment and recognize anything at any desired granularity. Code and checkpoint are available!
  • 2023/06/17: We provide an example to evaluate Grounding DINO on COCO zero-shot performance.
  • 2023/04/15: Refer to CV in the Wild Readings for those who are interested in open-set recognition!
  • 2023/04/08: We release demos to combine Grounding DINO with GLIGEN for more controllable image editings.
  • 2023/04/08: We release demos to combine Grounding DINO with Stable Diffusion for image editings.
  • 2023/04/06: We build a new demo by marrying GroundingDINO with Segment-Anything named Grounded-Segment-Anything aims to support segmentation in GroundingDINO.
  • 2023/03/28: A YouTube video about Grounding DINO and basic object detection prompt engineering. [SkalskiP]
  • 2023/03/28: Add a demo on Hugging Face Space!
  • 2023/03/27: Support CPU-only mode. Now the model can run on machines without GPUs.
  • 2023/03/25: A demo for Grounding DINO is available at Colab. [SkalskiP]
  • 2023/03/22: Code is available Now!
Description Paper introduction. Figure ODinW Marrying Grounding DINO and GLIGEN Figure gd gligen

Explanations/Tips for Grounding DINO Inputs and Outputs

  • Grounding DINO accepts an (image, text) pair as inputs.
  • It outputs 900 (by default) object boxes. Each box has similarity scores across all input words. (as shown in Figures below.)
  • We defaultly choose the boxes whose highest similarities are higher than a box_threshold.
  • We extract the words whose similarities are higher than the text_threshold as predicted labels.
  • If you want to obtain objects of specific phrases, like the dogs in the sentence two dogs with a stick., you can select the boxes with highest text similarities with dogs as final outputs.
  • Note that each word can be split to more than one tokens with different tokenlizers. The number of words in a sentence may not equal to the number of text tokens.
  • We suggest separating different category names with . for Grounding DINO. Figure model explain1 Figure model explain2

TODO

  • ☑ Release inference code and demo.
  • ☑ Release checkpoints.
  • ☑ Grounding DINO with Stable Diffusion and GLIGEN demos.
  • ☐ Release training codes.

Install

Note:

  1. If you have a CUDA environment, please make sure the environment variable CUDA_HOME is set. It will be compiled under CPU-only mode if no CUDA available.

Please make sure following the installation steps strictly, otherwise the program may produce:

NameError: name '_C' is not defined

If this happened, please reinstalled the groundingDINO by reclone the git and do all the installation steps again.

how to check cuda:
echo $CUDA_HOME

If it print nothing, then it means you haven't set up the path/

Run this so the environment variable will be set under current shell.

export CUDA_HOME=/path/to/cuda-11.3

Notice the version of cuda should be aligned with your CUDA runtime, for there might exists multiple cuda at the same time.

If you want to set the CUDA_HOME permanently, store it using:

echo 'export CUDA_HOME=/path/to/cuda' >> ~/.bashrc

after that, source the bashrc file and check CUDA_HOME:

source ~/.bashrc
echo $CUDA_HOME

In this example, /path/to/cuda-11.3 should be replaced with the path where your CUDA toolkit is installed. You can find this by typing which nvcc in your terminal:

For instance, if the output is /usr/local/cuda/bin/nvcc, then:

export CUDA_HOME=/usr/local/cuda

Installation:

1.Clone the GroundingDINO repository from GitHub.

git clone https://github.com/IDEA-Research/GroundingDINO.git
  1. Change the current directory to the GroundingDINO folder.
cd GroundingDINO/
  1. Install the required dependencies in the current directory.
pip install -e .
  1. Download pre-trained model weights.
mkdir weights
cd weights
wget -q https://github.com/IDEA-Research/GroundingDINO/releases/download/v0.1.0-alpha/groundingdino_swint_ogc.pth
cd ..

Demo

Check your GPU ID (only if you're using a GPU)

nvidia-smi

Replace {GPU ID}, image_you_want_to_detect.jpg, and "dir you want to save the output" with appropriate values in the following command

CUDA_VISIBLE_DEVICES={GPU ID} python demo/inference_on_a_image.py \
-c groundingdino/config/GroundingDINO_SwinT_OGC.py \
-p weights/groundingdino_swint_ogc.pth \
-i image_you_want_to_detect.jpg \
-o "dir you want to save the output" \
-t "chair"
 [--cpu-only] # open it for cpu mode

If you would like to specify the phrases to detect, here is a demo:

CUDA_VISIBLE_DEVICES={GPU ID} python demo/inference_on_a_image.py \
-c groundingdino/config/GroundingDINO_SwinT_OGC.py \
-p ./groundingdino_swint_ogc.pth \
-i .asset/cat_dog.jpeg \
-o logs/1111 \
-t "There is a cat and a dog in the image ." \
--token_spans "[[[9, 10], [11, 14]], [[19, 20], [21, 24]]]"
 [--cpu-only] # open it for cpu mode

The token_spans specify the start and end positions of a phrases. For example, the first phrase is [[9, 10], [11, 14]]. "There is a cat and a dog in the image ."[9:10] = 'a', "There is a cat and a dog in the image ."[11:14] = 'cat'. Hence it refers to the phrase a cat . Similarly, the [[19, 20], [21, 24]] refers to the phrase a dog.

See the demo/inference_on_a_image.py for more details.

Running with Python:

from groundingdino.util.inference import load_model, load_image, predict, annotate
import cv2

model = load_model("groundingdino/config/GroundingDINO_SwinT_OGC.py", "weights/groundingdino_swint_ogc.pth")
IMAGE_PATH = "weights/dog-3.jpeg"
TEXT_PROMPT = "chair . person . dog ."
BOX_TRESHOLD = 0.35
TEXT_TRESHOLD = 0.25

image_source, image = load_image(IMAGE_PATH)

boxes, logits, phrases = predict(
    model=model,
    image=image,
    caption=TEXT_PROMPT,
    box_threshold=BOX_TRESHOLD,
    text_threshold=TEXT_TRESHOLD
)

annotated_frame = annotate(image_source=image_source, boxes=boxes, logits=logits, phrases=phrases)
cv2.imwrite("annotated_image.jpg", annotated_frame)

Web UI

We also provide a demo code to integrate Grounding DINO with Gradio Web UI. See the file demo/gradio_app.py for more details.

Notebooks

COCO Zero-shot Evaluations

We provide an example to evaluate Grounding DINO zero-shot performance on COCO. The results should be 48.5.

CUDA_VISIBLE_DEVICES=0 \
python demo/test_ap_on_coco.py \
 -c groundingdino/config/GroundingDINO_SwinT_OGC.py \
 -p weights/groundingdino_swint_ogc.pth \
 --anno_path /path/to/annoataions/ie/instances_val2017.json \
 --image_dir /path/to/imagedir/ie/val2017

Checkpoints

name backbone Data box AP on COCO Checkpoint Config
1 GroundingDINO-T Swin-T O365,GoldG,Cap4M 48.4 (zero-shot) / 57.2 (fine-tune) GitHub link | HF link link
2 GroundingDINO-B Swin-B COCO,O365,GoldG,Cap4M,OpenImage,ODinW-35,RefCOCO 56.7 GitHub link | HF link link

Results

COCO Object Detection Results Figure COCO
ODinW Object Detection Results Figure ODinW
Marrying Grounding DINO with Stable Diffusion for Image Editing See our example notebook for more details. Figure GD SD
Marrying Grounding DINO with GLIGEN for more Detailed Image Editing. See our example notebook for more details. Figure GD GLIGEN

Model: Grounding DINO

Includes: a text backbone, an image backbone, a feature enhancer, a language-guided query selection, and a cross-modality decoder.

Figure arch

Acknowledgement

Our model is related to DINO and GLIP. Thanks for their great work!

We also thank great previous work including DETR, Deformable DETR, SMCA, Conditional DETR, Anchor DETR, Dynamic DETR, DAB-DETR, DN-DETR, etc. More related work are available at Awesome Detection Transformer. A new toolbox detrex is available as well.

Thanks Stable Diffusion and GLIGEN for their awesome models.

Citation

If you find our work helpful for your research, please consider citing the following BibTeX entry.

@article{liu2023grounding,
  title={Grounding dino: Marrying dino with grounded pre-training for open-set object detection},
  author={Liu, Shilong and Zeng, Zhaoyang and Ren, Tianhe and Li, Feng and Zhang, Hao and Yang, Jie and Li, Chunyuan and Yang, Jianwei and Su, Hang and Zhu, Jun and others},
  journal={arXiv preprint arXiv:2303.05499},
  year={2023}
}

Alternatives

Every model on this photo
Model output
Input test photo
⇆
InputOutput