About Finding Duplicates By Multiple Fields

One of the ways to find duplicates of instances with multiple fields is to concatenate the fields into duplicate_id and search for the multiple instances with the same duplicate_id:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
from django.db.models.functions import Concat, Cast
from django.db import models

queryset = MyModel.objects.annotate(
    duplicate_id=Concat(
        Cast(models.F("content_type_id"), models.CharField()),
        models.Value("|"),
        Cast(models.F("object_id"), models.CharField()),
        output_field=models.CharField(),
    )
)

for duplicate_id in (
    queryset.values_list("duplicate_id", flat=True)
    .annotate(duplicate_count=models.Count("duplicate_id"))
    .filter(duplicate_count__gt=1)
):
    the_first = None
    for instance in queryset.filter(
        duplicate_id=duplicate_id,
    ).order_by("created"):
        if not the_first:
            the_first = instance
        else:
            other = instance

Tips and Tricks Programming Databases Django 4.2 Django 3.2 Django 2.2