compute_coco_map ignores classes with Ground Truth but zero predictions

[iocalangiu]

HI, while writing a test for compute_coco_map() I identified a bug.

# Setup: Class 1 and 2 have predictions.
# Result: mAP is ~0.39 (averaged over 2 classes)

# Act: Add Class 3 (1 GT box, 0 Predictions)
metrics_factory.update(
    np.array([[100, 100, 110, 110]]), [3], # GT for Class 3
    np.empty((0, 4)), [], []               # 0 Preds
)

mAP_after = metrics_factory.compute_coco_map()

# Assert
# Expected: mAP should drop (averaged over 3 classes: [AP1 + AP2 + 0] / 3)
# Actual: mAP remains ~0.39 (still averaged over 2 classes)

The issue is that ground truth counts are updated too late

        # Update ground truth counts
        for g_label in gt_labels:
            self.gt_counts[int(g_label)] += 1

Specifically, after

        # Handle case where there is ground truth but no predictions
        if len(pred_boxes) == 0:
            for g_label in gt_labels:
                self.results[g_label].append((None, -1))  # All are false negatives
            return

I moved the code snippet that counts ground truths prior to all the early returns and the mAP is now lower (3rd class was included).

[dpascualhe]

Good catch! I think you might be right, I guess that as long as gt_labels is not empty, self.gt_counts should aways be updated. @SakhinetiPraveena can you validate that this is indeed a bug?

[JaisalJain]

Hi, I've reproduced the bug locally and traced the root cause.

Root Cause

In the update() method, gt_counts is updated at the bottom of the function,
after the early return that handles the no-predictions case:

Early return - gt_counts never gets updated for this case

if len(pred_boxes) == 0:
    for g_label in gt_labels:
        self.results[g_label].append((None, -1))
    return  # exits before gt_counts update

# This never runs when pred_boxes is empty

for g_label in gt_labels:
    self.gt_counts[int(g_label)] += 1

As a result, classes with ground truth but zero predictions are absent from gt_counts, and are therefore skipped in compute_coco_map() via the if self.gt_counts.get(int(label), 0) == 0: continue.

Reproduced with:

• mAP over 2 classes (both with predictions): 1.000
• mAP after adding a 3rd class (GT exists, zero predictions): 1.000 (should drop)
• gt_counts confirms: {1: 1, 2: 1} - Class 3 missing entirely

Proposed Fix

Move the gt_counts update to before all early returns in update(), so ground truth is always counted regardless of whether predictions exist.

---

I'd like to submit a PR with the fix. Please let me know if this approach looks good.

[iocalangiu]

Hi, thanks for also looking into this. I proposed the same solution, so given this and the fact that I found the bug, I think I should open the PR. I am just waiting for the main contributors to reply.

[JaisalJain]

Hi, thanks for clarifying, that makes sense. I see you had already proposed the same fix in the issue description. Happy to help test or review if needed.