Skip to content

Issue Encountered When Using DiffMOT Detection Text Files #23

@Flowercaty

Description

@Flowercaty

I am attempting to perform tracking tasks using the MOT17 dataset. I have updated the detection subpath in the dataset configuration to point to DiffMOT detections. The configuration in CAMELTrack/.venv/lib/python3.12/site-packages/tracklab/configs/dataset/mot17.yaml is as follows:

defaults:
  - default

_target_: tracklab.wrappers.MOT17

dataset_path: ${data_dir}/MOT17
eval_set: "train"
public_dets_subpath: "det/det_.txt"  # Updated to DiffMOT detections

track_eval:
  dataset_class: MotChallenge2DBox
  BENCHMARK: "MOT17"
  DO_PREPROC: True

After merging the detection results, I placed them in the corresponding det folder. Below is the partial file structure of the MOT17 dataset:
.
├── states
│ ├── mot17-test.pklz
│ ├── mot17-train.pklz
│ └── mot17-val.pklz
├── test
│ ├── MOT17-01
│ ├── ... (other sequences)
└── train
├── MOT17-02
│ ├── det
│ │ ├── det.txt # Merged detections
│ │ └── det_.txt # DiffMOT detections (referenced in config)
│ ├── gt
│ ├── img1
│ └── seqinfo.ini
├── MOT17-04
├── MOT17-05
├── MOT17-09
├── MOT17-10
├── MOT17-11
└── MOT17-13
When executing:
CUDA_VISIBLE_DEVICES=0 uv run tracklab -cn cameltrack dataset=mot17 dataset.eval_set=train
I encounter the following error:

CUDA_VISIBLE_DEVICES=0 uv run tracklab -cn cameltrack_0725 dataset=mot17 dataset.eval_set=train
[07/27/25 18:59:54] INFO Run directory: /home/disk/sdb/one/hcy/CAMELTrack/outputs/CAMELTrack/2025-07-27/18-59-53 main.py:104
INFO Using device: 'cuda'. main.py:105
INFO dataset: main.py:108
dataset_path: ${data_dir}/MOT17
nvid: -1
nframes: -1
eval_set: train
vids_dict:
train: []
val: []
test: []
track_eval:
dataset_class: MotChallenge2DBox
BENCHMARK: MOT17
GT_FOLDER: eval/gt
GT_LOC_FORMAT: '{gt_folder}/{seq}.txt'
TRACKERS_FOLDER: eval/pred
TRACKER_SUB_FOLDER: ''
OUTPUT_FOLDER: eval/results
OUTPUT_SUB_FOLDER: ''
SPLIT_TO_EVAL: ${dataset.eval_set}
PRINT_CONFIG: false
DO_PREPROC: true
target: tracklab.wrappers.MOT17
public_dets_subpath: det/det_.txt
eval:
target: tracklab.wrappers.TrackEvalEvaluator
show_progressbar: true
eval_set: ${dataset.eval_set}
dataset_path: ${dataset.dataset_path}
cfg:
save_gt: true
bbox_column_for_eval: bbox_ltwh
metrics:
- CLEAR
- HOTA
- Identity
eval:
USE_PARALLEL: true
NUM_PARALLEL_CORES: ${num_cores}
BREAK_ON_ERROR: false
PRINT_RESULTS: true
PRINT_ONLY_COMBINED: true
PRINT_CONFIG: false
TIME_PROGRESS: false
DISPLAY_LESS_PROGRESS: false
OUTPUT_SUMMARY: true
OUTPUT_EMPTY_CLASSES: false
OUTPUT_DETAILED: true
PLOT_CURVES: true
dataset: ${dataset.track_eval}
engine:
target: tracklab.engine.OfflineTrackingEngine
num_workers: ${num_cores}
callbacks:
progress:
target: tracklab.callbacks.Progressbar
use_rich: ${use_rich}
vis: ${visualization}
visualization:
colors:
bbox:
no_id: null
prediction: track_id
ground_truth:
- 0
- 255
- 0
text:
no_id: null
prediction: track_id
ground_truth:
- 0
- 255
- 0
keypoint:
no_id: null
prediction: track_id
ground_truth:
- 0
- 255
- 0
default:
no_id: null
prediction: track_id
ground_truth: null
'N': 10
target: tracklab.visualization.VisualizationEngine
save_videos: true
visualizers:
detections:
target: tracklab.visualization.DefaultDetectionVisualizer
modules:
pose_estimator:
target: tracklab.wrappers.pose_estimator.yolo_ultralytics_pose_api.YOLOUltralyticsPose
batch_size: 4
cfg:
path_to_checkpoint: ${model_dir}/yolo_ultralytics/yolo11m-pose.pt
min_confidence: 0.4
reid:
dataset:
name: PoseTrack21
nickname: pt21
fig_size:
- 384
- 128
mask_size:
- 64
- 32
max_crop_size:
- 256
- 128
masks_mode: pose_on_img_crops
enable_human_parsing_labels: true
eval_metric: mot_intra_video
multi_video_queries_only: false
columns: []
sam_checkpoint: ''
enable_sam: false
train:
set_name: train
min_vis: 0.3
min_h: 10
min_w: 10
min_samples_per_id: 4
max_samples_per_id: 20
max_total_ids: 1000
test:
set_name: val
min_vis: 0.0
min_h: 0
min_w: 0
min_samples_per_id: 4
max_samples_per_id: 10
max_total_ids: -1
ratio_query_per_id: 0.2
target: tracklab.wrappers.reid.kpreid_api.KPReId
training_enabled: false
batch_size: 2
save_path: reid
use_keypoints_visibility_scores_for_reid: false
cfg:
model:
load_weights: ${model_dir}/reid/kpr_dancetrack_sportsmot_posetrack21_occludedduke_market_split0.pth.tar
pretrained: false
discard_test_params: false
save_model_flag: false
kpr:
dim_reduce_output: 512
keypoints:
enabled: false
prompt_masks: keypoints_gaussian
prompt_preprocess: cck6
masks:
dir: ''
preprocess: five_v
transreid:
sie_camera: false
promptable_trans:
masks_prompting: false
project:
name: ''
experiment_name: ''
notes: ''
tags: []
job_id: ''
logger:
use_wandb: false
data:
save_dir: ''
sources:
- market1501
targets:
- market1501
track:
target: cameltrack.CAMELTrack
training_enabled: false
override_camel_cfg:
sim_threshold: 0.5
min_det_conf: 0.4
min_init_det_conf: 0.6
min_num_hits: 0
max_wo_hits: 150
max_track_gallery_size: 50
CAMEL:
target: cameltrack.CAMEL
sim_threshold: 0.1
use_computed_sim_threshold: false
gaffe:
target: cameltrack.architecture.GAFFE
emb_dim: 1024
n_heads: 8
n_layers: 4
dim_feedforward: 2048
checkpoint_path: null
temporal_encoders:
app_encoder:
target: cameltrack.architecture.TemporalEncoder
hidden_dim: 512
n_heads: 8
n_layers: 4
dim_feedforward: 1024
checkpoint_path: null
det_tokenizer:
target: cameltrack.architecture.PartsEmbeddingsLinProj
use_parts: true
kp_encoder:
target: cameltrack.architecture.TemporalEncoder
hidden_dim: 128
n_heads: 8
n_layers: 4
dim_feedforward: 256
checkpoint_path: null
det_tokenizer:
target: cameltrack.architecture.KeypointsLinProj
use_conf: false
bbox_encoder:
target: cameltrack.architecture.TemporalEncoder
hidden_dim: 64
n_heads: 8
n_layers: 4
dim_feedforward: 128
checkpoint_path: null
det_tokenizer:
target: cameltrack.architecture.BBoxLinProj
use_conf: true
optimizer:
init_lr: 0.0001
weight_decay: 0.0001
train_cfg:
use_wandb: ${use_wandb}
use_rich: ${use_rich}
evaluate_only: false
model_selection_criteria: last
pl_trainer:
max_epochs: 2
precision: 32
gradient_clip_val: 0
accumulate_grad_batches: 1
enable_progress_bar: true
enable_model_summary: false
profiler: null
num_sanity_val_steps: 0
log_every_n_steps: 50
check_val_every_n_epochs: 1
val_check_interval: null
fast_dev_run: false
datamodule_cfg:
target: cameltrack.train.CAMELDataModule
batch_transforms:
train:
target: cameltrack.train.transforms.Compose
transforms:
- target: cameltrack.train.transforms.AppEmbNoise
alpha: 0.4
- target: cameltrack.train.transforms.KeypointsShake
alpha: 0.05
- target: cameltrack.train.transforms.BBoxShake
alpha: 0.05
tracklet_transforms:
train:
target: cameltrack.train.transforms.Compose
transforms:
- target: cameltrack.train.transforms.MaxNumObs
max_num_obs: 150
- target: cameltrack.train.transforms.DropoutFeatures
p_drop_app: 0.4
p_drop_bbox: 0.1
p_drop_kps: 0.1
- target: cameltrack.train.transforms.ProbabilisticTransform
probs:
- 0.4
- 0.4
- 0.1
transforms:
- target: cameltrack.train.transforms.SwapSporadic
p_swap: 0.2
sigma: 50
method: gaussian
- target: cameltrack.train.transforms.DropoutSporadic
p_drop: 0.2
sigma: 50
method: gaussian
- target: cameltrack.train.transforms.SwapOccluded
p_swap: 0.5
min_iou: 0.5
sampler: simple
sampler_args:
fill_samples: true
multi_dataset_training: false
num_videos: null
num_workers: ${num_cores}
max_length: 50
batch_size: 8
num_samples: 32
samples_per_video: 100
train_add_val: false
dataset_splits:
- train
- val
name: camel
path: ${dataset.dataset_path}/states/train_camel
tracker_states:
train: ${dataset.dataset_path}/states/train.pklz
val: ${dataset.dataset_path}/states/val.pklz
dataset_transforms:
- add_detections
- add_last_obs_counter
- add_occlusions
- add_dim_image
recursive: false
pipeline:
- pose_estimator
- reid
- track
num_cores: 4
experiment_name: CAMELTrack
print_config: true
use_wandb: false
data_dir: ${project_dir}/data
model_dir: ${project_dir}/pretrained_models
state:
load_file: null
save_file: ${project_dir}/states/${experiment_name}-${dataset.eval_set}.pklz
load_from_public_dets: true
load_from_groundtruth: false
test_tracking: true
eval_tracking: true
use_rich: true
project_dir: ${hydra:runtime.cwd}

[07/27/25 18:59:58] INFO Loading MOT17 dataset from /home/disk/sdb/one/hcy/CAMELTrack/data/MOT17. mot17.py:29
Loading MOT17 'train' set videos ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0% -:--:-- WARNING The val split does not exist. common.py:56
WARNING Warning: The test split does not contain ground truth. common.py:122
WARNING Warning: The test split does not contain public detections. common.py:137
Loading MOT17 'test' set videos ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00
Loading MOT17 'train' set videos ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:03
[07/27/25 19:00:05] INFO dict_keys(['train', 'val', 'test']) common.py:47
INFO MOT17= train set: 7; test set: 21 main.py:33
Overwriting current config with config loaded from /home/disk/sdb/one/hcy/CAMELTrack/pretrained_models/reid/kpr_dancetrack_sportsmot_posetrack21_occludedduke_market_split0.pth.tar
WARNING: the train dataset of the loaded model is different from the target dataset in the current config.
Successfully loaded pretrained weights from "/home/disk/sdb/one/hcy/CAMELTrack/pretrained_models/reid/kpr_dancetrack_sportsmot_posetrack21_occludedduke_market_split0.pth.tar"
[07/27/25 19:00:08] INFO Pipeline: YOLOUltralyticsPose -> KPReId -> CAMELTrack module.py:67
INFO Starting tracking operation on train set. main.py:51
INFO Saving TrackerState to /home/disk/sdb/one/hcy/CAMELTrack/states/CAMELTrack-train.pklz tracker_state.py:52
INFO Loading {'detection': ['visibility', 'video_id', 'image_id', 'class', 'person_id', 'id'], 'image': ['frame', 'id', tracker_state.py:102
'is_labeled', 'file_path', 'video_id']} from None
INFO Pipeline has been validated
Traceback (most recent call last):
File "/home/disk/sdb/one/hcy/CAMELTrack/.venv/bin/tracklab", line 10, in
sys.exit(main())
^^^^^^
File "/home/disk/sdb/one/hcy/CAMELTrack/.venv/lib/python3.12/site-packages/hydra/main.py", line 94, in decorated_main
_run_hydra(
File "/home/disk/sdb/one/hcy/CAMELTrack/.venv/lib/python3.12/site-packages/hydra/_internal/utils.py", line 394, in _run_hydra
_run_app(
File "/home/disk/sdb/one/hcy/CAMELTrack/.venv/lib/python3.12/site-packages/hydra/_internal/utils.py", line 457, in _run_app
run_and_report(
File "/home/disk/sdb/one/hcy/CAMELTrack/.venv/lib/python3.12/site-packages/hydra/_internal/utils.py", line 223, in run_and_report
raise ex
File "/home/disk/sdb/one/hcy/CAMELTrack/.venv/lib/python3.12/site-packages/hydra/_internal/utils.py", line 220, in run_and_report
return func()
^^^^^^
File "/home/disk/sdb/one/hcy/CAMELTrack/.venv/lib/python3.12/site-packages/hydra/_internal/utils.py", line 458, in
lambda: hydra.run(
^^^^^^^^^^
File "/home/disk/sdb/one/hcy/CAMELTrack/.venv/lib/python3.12/site-packages/hydra/_internal/hydra.py", line 132, in run
_ = ret.return_value
^^^^^^^^^^^^^^^^
File "/home/disk/sdb/one/hcy/CAMELTrack/.venv/lib/python3.12/site-packages/hydra/core/utils.py", line 260, in return_value
raise self._return_value
File "/home/disk/sdb/one/hcy/CAMELTrack/.venv/lib/python3.12/site-packages/hydra/core/utils.py", line 186, in run_job
ret.return_value = task_function(task_cfg)
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/disk/sdb/one/hcy/CAMELTrack/.venv/lib/python3.12/site-packages/tracklab/main.py", line 63, in main
tracking_engine.track_dataset()
File "/home/disk/sdb/one/hcy/CAMELTrack/.venv/lib/python3.12/site-packages/tracklab/engine/engine.py", line 118, in track_dataset
detections, image_pred = self.video_loop(tracker_state, video_metadata, video_idx)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/disk/sdb/one/hcy/CAMELTrack/.venv/lib/python3.12/site-packages/tracklab/engine/offline.py", line 30, in video_loop
for batch in self.dataloaders[model_name]:
~~~~~~~~~~~~~~~~^^^^^^^^^^^^
File "/home/disk/sdb/one/hcy/CAMELTrack/.venv/lib/python3.12/site-packages/torch/utils/data/dataloader.py", line 708, in next
data = self._next_data()
^^^^^^^^^^^^^^^^^
File "/home/disk/sdb/one/hcy/CAMELTrack/.venv/lib/python3.12/site-packages/torch/utils/data/dataloader.py", line 1480, in _next_data
return self._process_data(data)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/disk/sdb/one/hcy/CAMELTrack/.venv/lib/python3.12/site-packages/torch/utils/data/dataloader.py", line 1505, in _process_data
data.reraise()
File "/home/disk/sdb/one/hcy/CAMELTrack/.venv/lib/python3.12/site-packages/torch/_utils.py", line 733, in reraise
raise exception
cv2.error: Caught error in DataLoader worker process 1.
Original Traceback (most recent call last):
File "/home/disk/sdb/one/hcy/CAMELTrack/.venv/lib/python3.12/site-packages/torch/utils/data/_utils/worker.py", line 349, in _worker_loop
data = fetcher.fetch(index) # type: ignore[possibly-undefined]
^^^^^^^^^^^^^^^^^^^^
File "/home/disk/sdb/one/hcy/CAMELTrack/.venv/lib/python3.12/site-packages/torch/utils/data/_utils/fetch.py", line 52, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
~~~~~~~~~~~~^^^^^
File "/home/disk/sdb/one/hcy/CAMELTrack/.venv/lib/python3.12/site-packages/tracklab/datastruct/datapipe.py", line 36, in getitem
self.model.preprocess(image=image, detection=detection, metadata=metadata),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/disk/sdb/one/hcy/CAMELTrack/.venv/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/disk/sdb/one/hcy/CAMELTrack/.venv/lib/python3.12/site-packages/tracklab/wrappers/reid/kpreid_api.py", line 133, in preprocess
batch = ImageDataset.getitem(
^^^^^^^^^^^^^^^^^^^^^
File "/home/disk/sdb/one/hcy/CAMELTrack/.venv/lib/python3.12/site-packages/torchreid/data/datasets/dataset.py", line 454, in getitem
result = transforms(**transf_args)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/disk/sdb/one/hcy/CAMELTrack/.venv/lib/python3.12/site-packages/albumentations/core/composition.py", line 210, in call
data = t(**data)
^^^^^^^^^
File "/home/disk/sdb/one/hcy/CAMELTrack/.venv/lib/python3.12/site-packages/albumentations/core/transforms_interface.py", line 118, in call
return self.apply_with_params(params, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/disk/sdb/one/hcy/CAMELTrack/.venv/lib/python3.12/site-packages/albumentations/core/transforms_interface.py", line 131, in apply_with_params
res[key] = target_function(arg, **dict(params, **target_dependencies))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/disk/sdb/one/hcy/CAMELTrack/.venv/lib/python3.12/site-packages/torchreid/data/masks_transforms/resize.py", line 38, in apply
return functional.resize(img, height=self.height, width=self.width, interpolation=interpolation)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/disk/sdb/one/hcy/CAMELTrack/.venv/lib/python3.12/site-packages/albumentations/augmentations/utils.py", line 123, in wrapped_function
result = func(img, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/disk/sdb/one/hcy/CAMELTrack/.venv/lib/python3.12/site-packages/albumentations/augmentations/geometric/functional.py", line 392, in resize
return resize_fn(img)
^^^^^^^^^^^^^^
File "/home/disk/sdb/one/hcy/CAMELTrack/.venv/lib/python3.12/site-packages/albumentations/augmentations/utils.py", line 209, in __process_fn
img = process_fn(img, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
cv2.error: OpenCV(4.11.0) /io/opencv/modules/imgproc/src/resize.cpp:4208: error: (-215:Assertion failed) !ssize.empty() in function 'resize'

Tracking videos (MOT17-02) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/7 0:03:44 • -:--:--
Kpreid ━━━━╸━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1953/17033 0:03:22 • 0:26:08

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions