ValueError: shapes not aligned (when running deep sort with yolov8 object detector)

[Vishnu15018]

I tried yolov8 object detection , and deep sort object tracking to track vehicles , using the Nicolai Nielsen tutorials.I got this below error.

Trace if the error.

`ValueError Traceback (most recent call last)
in <cell line: 26>()
69 # Update tracker with bounding boxes
70 print(list1)
---> 71 tracks = object_tracker.update_tracks(list1,frame)
72
73 # Process each tracked object

8 frames
/usr/local/lib/python3.10/dist-packages/deep_sort_realtime/deepsort_tracker.py in update_tracks(self, raw_detections, embeds, frame, today, others, instance_masks)
226 # Update tracker.
227 self.tracker.predict()
--> 228 self.tracker.update(detections, today=today)
229
230 return self.tracker.tracks

/usr/local/lib/python3.10/dist-packages/deep_sort_realtime/deep_sort/tracker.py in update(self, detections, today)
95
96 # Run matching cascade.
---> 97 matches, unmatched_tracks, unmatched_detections = self._match(detections)
98
99 # Update track set.

/usr/local/lib/python3.10/dist-packages/deep_sort_realtime/deep_sort/tracker.py in _match(self, detections)
149 unmatched_tracks_a,
150 unmatched_detections,
--> 151 ) = linear_assignment.matching_cascade(
152 gated_metric,
153 self.metric.matching_threshold,

/usr/local/lib/python3.10/dist-packages/deep_sort_realtime/deep_sort/linear_assignment.py in matching_cascade(distance_metric, max_distance, cascade_depth, tracks, detections, track_indices, detection_indices)
145 continue
146
--> 147 matches_l, _, unmatched_detections = min_cost_matching(
148 distance_metric,
149 max_distance,

/usr/local/lib/python3.10/dist-packages/deep_sort_realtime/deep_sort/linear_assignment.py in min_cost_matching(distance_metric, max_distance, tracks, detections, track_indices, detection_indices)
60 return [], track_indices, detection_indices # Nothing to match.
61
---> 62 cost_matrix = distance_metric(tracks, detections, track_indices, detection_indices)
63 cost_matrix[cost_matrix > max_distance] = max_distance + 1e-5
64 # indices = linear_assignment(cost_matrix)

/usr/local/lib/python3.10/dist-packages/deep_sort_realtime/deep_sort/tracker.py in gated_metric(tracks, dets, track_indices, detection_indices)
131 features = np.array([dets[i].feature for i in detection_indices])
132 targets = np.array([tracks[i].track_id for i in track_indices])
--> 133 cost_matrix = self.metric.distance(features, targets)
134 cost_matrix = linear_assignment.gate_cost_matrix(
135 self.kf, cost_matrix, tracks, dets, track_indices, detection_indices, only_position=self.gating_only_position

/usr/local/lib/python3.10/dist-packages/deep_sort_realtime/deep_sort/nn_matching.py in distance(self, features, targets)
172 cost_matrix = np.zeros((len(targets), len(features)))
173 for i, target in enumerate(targets):
--> 174 cost_matrix[i, :] = self._metric(self.samples[target], features)
175 return cost_matrix

/usr/local/lib/python3.10/dist-packages/deep_sort_realtime/deep_sort/nn_matching.py in _nn_cosine_distance(x, y)
93
94 """
---> 95 distances = _cosine_distance(x, y)
96 return distances.min(axis=0)
97

/usr/local/lib/python3.10/dist-packages/deep_sort_realtime/deep_sort/nn_matching.py in _cosine_distance(a, b, data_is_normalized)
52 a = np.asarray(a) / np.linalg.norm(a, axis=1, keepdims=True)
53 b = np.asarray(b) / np.linalg.norm(b, axis=1, keepdims=True)
---> 54 return 1.0 - np.dot(a, b.T)
55
56

ValueError: shapes (2,1280,3) and (3,1280,2) not aligned: 3 (dim 2) != 1280 (dim 1)`

code snippet:

while True :
ret, frame = cap.read()
print("frame:", frame.shape)
if not ret:
break
count += 1
# Perform object detection on the frame (assuming you have the model defined somewhere)
results = model.predict(frame, save=True, conf=0.25)
annotated=results[0].plot()
cv2_imshow(annotated)
z=results[0].boxes
list1 = []

cords=z.xywhn.cpu().numpy()
confs=list(z.conf)
cls_ids=(z.cls)
for i in range(len(cords)):
    x1=cords[i][0]
    y1=cords[i][1]
    w=cords[i][2]
    h=cords[i][3]
    c=(confs[i].cpu()).item()
    d=str(cls_ids[i].item())
    list1.append(([x1,y1,w,h],c,d))             #([left,top,w,h],confidence,class_name)
# Update tracker with bounding boxes
print(list1)
#here I am getting the error
tracks = object_tracker.update_tracks(list1,frame)  
# Process each tracked object
print(tracks)
for track in tracks:

    if not track.is_confirmed():
        continue
    tk_id = int(track.track_id)
    ltrb = track.to_ltrb()
    print(ltrb)
    x3,y3,x4,y4=ltrb
    cx=int(x3+x4)//2
    cy=int(y3+y4)//2

Deep sort Initialisation:

from deep_sort_realtime.deepsort_tracker import DeepSort

object_tracker = DeepSort(max_age=5,
n_init=2,
nms_max_overlap=0.9,
max_cosine_distance=0.3,
nn_budget=None,
override_track_class = None,
embedder="mobilenet",
half=True,
bgr=True,
embedder_gpu=True,
embedder_model_name=None,
embedder_wts=None,
polygon=False,
today=None)

I am new to object tracking, provide me with the solution or any article related to this error. Input image size is (720, 1280, 3) Thanks in advance.

[irjkf129]

Hello! You have error in str tracks = object_tracker.update_tracks(list1,frame)
Need use arghument "frame"
tracks = object_tracker.update_tracks(list1,frame=frame)

[mrsahabu]

any solution?

[Vishnu15018]

any solution?
No, solution. I copy pasted the code directly from Nicolai Nielsen tutorials GitHub, repo on deepsort.

[Woodthorne]

I found that by adding a row a, b = np.squeeze(a), np.squeeze(b) after row 53 inside of the if not data_is_normalized portion of function _cosine_distance in nn_matching.py allows the program to proceed. I now face issues elsewhere in my program, but I haven't been able to diagnose fully if they are related to this error and fix.

[Woodthorne]

After running the fix for a few days now I can say that the issues that followed were unrelated and squeezing the variables seems to be working.