feat: Feature/dino batching (HumanSignal#388)

* Add GroundingDINO backend * Clean comments * Feature/shondle sam with dino (HumanSignal#352) * Add baseline SAM code * Add TO DOs * Add SAM integration fixes * Add regular SAM * Add SAM capability and instructions * Update README.md * Remove whitespace * Remove whitespace * Adding env variables and code cleaning * Fix environment variable issues * Change truncation of ID * Remove exec * Adding SAM batch support * Add batch dino for multiple tasks * Add DINO Batching * Adding debug statements * Add support for more than three images at a time * Use Batching DINO Instead * Fix labelling scheme * Add batching instructions * Update README.md * Uncomment SAM download * Add batching example video * Remove extra print statements * Remove debug argument --------- Co-authored-by: shondle <�106922533+shondle@users.noreply.github.com> Co-authored-by: Chris Hoge <chris@hogepodge.com>

Author: 3 people

label_studio_ml/examples/grounding_dino/.dockerignore

@@ -0,0 +1,2 @@
+node_modules/*
+docker-data/*

label_studio_ml/examples/grounding_dino/Dockerfile

@@ -16,19 +16,20 @@ ENV PYTHONUNBUFFERED=True \
 COPY requirements.txt .
 RUN pip install --no-cache-dir -r requirements.txt
 
-RUN git clone https://github.com/IDEA-Research/GroundingDINO.git
-WORKDIR /app/GroundingDINO
+RUN git clone https://github.com/yuwenmichael/Grounding-DINO-Batch-Inference.git
+WORKDIR /app/Grounding-DINO-Batch-Inference/GroundingDINO
 RUN pip -q install -e .
 RUN mkdir weights
-WORKDIR /app/GroundingDINO/weights
+WORKDIR /app/Grounding-DINO-Batch-Inference/GroundingDINO/weights
+
 RUN wget -q https://github.com/IDEA-Research/GroundingDINO/releases/download/v0.1.0-alpha/groundingdino_swint_ogc.pth
 
 WORKDIR /app
 RUN wget -q https://github.com/ChaoningZhang/MobileSAM/raw/master/weights/mobile_sam.pt
 RUN wget -q https://dl.fbaipublicfiles.com/segment_anything/sam_vit_h_4b8939.pth
 
-
+RUN pip install olefile
 COPY . ./
 
 
-CMD gunicorn --preload --bind :$PORT --workers $WORKERS --threads $THREADS --timeout 0 _wsgi:app
+CMD gunicorn --preload --bind :$PORT --workers $WORKERS --threads $THREADS --timeout 0 _wsgi:app

label_studio_ml/examples/grounding_dino/README.md

@@ -9,6 +9,8 @@ Use text prompts for zero-shot detection of objects in images! Specify the detec
 See [here](https://github.com/IDEA-Research/GroundingDINO) for more details about the pretrained GroundingDINO model. 
 
 
+## Quickstart
+=======
 Quickstart
 
 1. Make sure docker is installed
@@ -21,7 +23,7 @@ Example-
 3. Run `docker compose up`
 4. Check the IP of your backend using `docker ps` and add it to the Machine Learning backend in the Label Studio software. Usually this is on `http://localhost:9090`.
 
-5. Edit the labelling config to the below.
+5. Create a project and edit the labelling config (an example is provided below). When editing the labeling config, make sure to add all rectangle labels under the RectangleLabels tag, and all corresponding brush labels under the BrushLabels tag.
 
 ```
 <View>
@@ -34,13 +36,16 @@ Example-
  </View>
  <RectangleLabels name="label" toName="image">
  <Label value="cats" background="yellow"/>
+ <Label value="house" background="blue"/>
  </RectangleLabels>
+ <BrushLabels name="label2" toName="image">
+ <Label value="cats" background="yellow"/>
+ <Label value="house" background="blue"/>
+ </BrushLabels>
 </View>
 ```
 
-This may be adjusted to your needs, but please keep the promp section and some rectangle labels.
-
-6. Go to an image task in one of your projects. Turn on the Auto-annotation switch. Then, type in the prompt box and press add. After this, you should receive your predictions. See the video above for a demo. 
+6. Go to an image task in your project. Turn on the Auto-annotation switch. Then, type in the prompt box and press add. After this, you should receive your predictions. See the video above for a demo. 
 
 
 ## Using GroundingSAM
@@ -52,6 +57,24 @@ To do this, set `USE_SAM=True` before running.
 If you want to use a more efficient version of SAM, set `USE_MOBILE_SAM=True` as well.
 
 
+## Batching Inputs
+
+https://github.com/HumanSignal/label-studio-ml-backend/assets/106922533/79b788e3-9147-47c0-90db-0404066ee43f
+
+> Note: this is an experimental feature.
+
+1. Clone the label studio branch with the added batching features.
+
+`git clone -b feature/dino-support https://github.com/HumanSignal/label-studio.git`
+
+2. Run this branch with `docker compose up`
+3. Do steps 2-5 from the [quickstart section](#quickstart), now using access code and host IP info of the newly clones Label Studio branch. GroundingSAM is supported.
+4. Go to the task menu inside your project and select the tasks you would like to annotate.
+5. Click the dropdown in the upper left hand side and select `Add Text Prompt for GroundingDINO`
+6. Enter in the prompt you would like to retrieve predictions for and press submit.
+- If your prompt is different from the label values you have assigned, you can use the underscore to give the correct label values to your prompt outputs. For example, if I wanted to select all brown cats but still give them the label value "cats" from my labeling config, my prompt would be "brown cat_cats".
+
+
 ## Other Environment Variables
 
 Adjust `BOX_THRESHOLD` and `TEXT_THRESHOLD` values in the Dockerfile to a number between 0 to 1 if experimenting. Defaults are set in `dino.py`. See explanation of these values in this [section](https://github.com/IDEA-Research/GroundingDINO#star-explanationstips-for-grounding-dino-inputs-and-outputs).