model's id is not uniqe

Using Django
1

Hello everyone,
I run into a weird bug in my program.
I have a model called CatalogImage but at one point of the project (not sure when or what I did to make it happen) I noticed I have duplicates in the database with the same id and values.
for example:

CatalogImage.objects.filter(id=19)
<QuerySet [<CatalogImage: מגף ריינו שחור>, <CatalogImage: מגף ריינו שחור>]>
CatalogImage.objects.get(id=19)
<CatalogImage: מגף ריינו שחור>
CatalogImage.objects.filter(id=19).values()
<QuerySet [{‘id’: 19, ‘title’: ‘מגף ריינו שחור’, ‘description’: ‘* ריינו מגף עבודה\r\n* מגף מקצועי ונוח עשוי מעור איכותי\r\n* אפשרות למגף עם מיגון מלא או ללא מיגון\r\n 02/S3 \r\n* צד אלסטי המאפשר נוחות מירבית לנעילה/ חליצה של הנעל.\r\n* מדרך פנימי ארגונומי ונוח לשעות רבות של עבודה.\r\n* סוליית פוליאוריטן דו שכבתית PU/PU\r\n* אימום אנטומי ורחב במיוחד ,בולם זעזועים מובנה בסוליה.\r\n* SRC-עמידות גבוהה בהחלקה’, ‘image’: ‘שקופית13_optimized_9It5WB6_0nmQKq6_acSUbCz_kuGhiXk_ZXxiRTV_AiwRKrv_IMFkFzk_RSLpsAX_b2JnZ_H6jyZDE.jpg’, ‘image_thumbnail’: ‘image_thumbnail_שקופית13_optimized_9It5WB6_0nmQKq6_acSUbCz_kuGhiXk_ZXxiRTV_AiwRKrv_IMFkF_hUNtq8q.jpg’, ‘big_discount’: True, ‘discount’: ‘/static/assets/catalog/imgs/discount_20.png’}, {‘id’: 19, ‘title’: ‘מגף ריינו שחור’, ‘description’: ‘* ריינו מגף עבודה\r\n* מגף
מקצועי ונוח עשוי מעור איכותי\r\n* אפשרות למגף עם מיגון מלא או ללא מיגון\r\n 02/S3 \r\n* צד אלסטי המאפשר נוחות מירבית לנעילה/ חליצה של הנעל.\r\n* מדרך פנימי ארגונומי ונוח לשעות רבות של עבודה.\r\n* סוליית פוליאוריטן דו שכבתית PU/PU\r\n* אימום אנטומי ורחב במיוחד ,בולם זעזועים מובנה בסוליה.\r\n* SRC-עמידות גבוהה בהחלקה’, ‘image’: ‘שקופית13_optimized_9It5WB6_0nmQKq6_acSUbCz_kuGhiXk_ZXxiRTV_AiwRKrv_IMFkFzk_RSLpsAX_b2JnZ_H6jyZDE.jpg’, ‘image_thumbnail’: ‘image_thumbnail_שקופית13_optimized_9It5WB6_0nmQKq6_acSUbCz_kuGhiXk_ZXxiRTV_AiwRKrv_IMFkF_hUNtq8q.jpg’, ‘big_discount’: True, ‘discount’: ‘/static/assets/catalog/imgs/discount_20.png’}]>

as you can see I have 2 objects with the same id and values, I tried to change the values of one of them from the admin panel and the 2 changed, same about deleting them. how can I get rid of the duplicates and when they created?
Thank you!

2

We may need to see your model for CatalogImage

Did you define another field as the primary key?

Have you only used Django migrations to create and alter this model?

Are you using a custom manager for this model? (If so, we’ll need to see this, too.)

3

Thank you for the response.
this is my CatalogImage:

class CatalogImage(models.Model):
    title = models.CharField(max_length=120, verbose_name=_("title"), unique=False)
    description = models.TextField(verbose_name=_("description"))
    image = models.ImageField(verbose_name=_("image"))
    image_thumbnail = models.ImageField(verbose_name=_("image thumbnail"), null=True, blank=True)
    colors = models.ManyToManyField(to=Color)
    sizes = models.ManyToManyField(to=ProductSize)
    big_discount = models.BooleanField(default=False)
    class Meta():
        verbose_name = _('Catalog image')
        verbose_name_plural = _('Catalog images')
        ordering = ['throughimage__image_order']

and in my CatalogAlbum/models.py:

from mptt.models import MPTTModel, TreeForeignKey
import datetime

class CatalogAlbum(MPTTModel):
    title = models.CharField(max_length=120, verbose_name=_("title"))
    slug = models.SlugField(max_length=120, verbose_name=_("slug"))
    description= models.TextField(verbose_name=_('description'), default='', blank=True)
    fotter = models.TextField(verbose_name=_('fotter'), default='', blank=True)
    keywords = models.TextField(verbose_name=_('keyworks'), default='', blank=True)
    images = models.ManyToManyField(to=CatalogImage, related_name='album', blank=True, through='ThroughImage')
    parent = TreeForeignKey('self', on_delete=models.CASCADE, null=True, blank=True, related_name='children')
    is_public = models.BooleanField(verbose_name=_('is public'), default=True)
    renew_for = models.DurationField(null=True, blank=True, default=datetime.timedelta(days=3))
    renew_after = models.DurationField(null=True, blank=True, default=datetime.timedelta(days=1))
    timer = models.DateTimeField(null=True, blank=True)
    class MPTTMeta:
        order_insertion_by = ['title']
    class Meta:
        unique_together = ('slug', 'parent',)

in the catalogAlbum/models.py I have this too:

from adminsortable.fields import SortableForeignKey
from adminsortable.models import Sortable

class ThroughImage(Sortable):
    catalogImage = SortableForeignKey(CatalogImage, on_delete=models.CASCADE)
    catalogAlbum = models.ForeignKey(CatalogAlbum, on_delete=models.CASCADE)
    image_order = models.PositiveIntegerField(default=0, editable=False, db_index=True)
    class Meta(Sortable.Meta):
        ordering = ['image_order']

I only use python manage.py makemigrations, python manage.py migrate.
my database is currently sqlite

Did you define another field as the primary key? not me, but im not completely sure how django-admin-sortable==2.2.3 or django-mptt-admin==2.0.0 effect the catalogImage modal

Are you using a custom manager for this model? No, still didn’t learn it yet :smiley:

Edit:
I looked in the database running this command SELECT * FROM 'catalogImages_catalogimage'
and there were no duplicates, so the problem is in polling the info from the database if I’m not mistaken

4

Just making guesses here since I really don’t have a solid idea but I do have a hunch. I’d quickly try two things to narrow this down.

  1. Remove the ordering from the CatalogImage model Meta class. (Note: I’m not suggesting this be removed permanently - just as a test to see if it’s the base cause of the behavior you’re seeing.)

  2. Check your ‘ThroughImage’ table to see if you’ve got two separate rows with the catalogImage field = 19. (I’m guessing you do.)

My guess is that since you’re sorting through a backward foreign key relationship, you’re getting the same row returned twice because of the filter. This doesn’t mean that you have multiple copies of the same row, just that the way the query is constructed is causing the same row to be returned twice.

You do have the option of adding the distinct() method on your query to prevent it from returning duplicate rows.

5

Thank you!,
looks like that was the problem
so ThroughImage dose have 2 records of the same CatalogImage:

ThroughImage.objects.filter(catalogImage_id=19).values()
<QuerySet [{'id': 112, 'order': 24, 'catalogImage_id': 19, 'catalogAlbum_id': 1, 'image_order': 0}, {'id': 155, 'order': 1, 'catalogImage_id': 19, 'catalogAlbum_id': 4, 'image_order': 1}]>

removing the ordering from the catalogImage solve the problem but I still want the images field in the CatalogAlbum to be sorted
This was before:
I have this in my CatalogAlbum.views.py:

def catalogView2(request, *args, **wkargs):
    albums = CatalogAlbum.objects.prefetch_related('images')
    context = {'albums':albums}
    return render(request, 'catalog.html', context=context)

and in the template (catalog.html)
{%for img in album.images.all%}

I changed to this in the template:
{%for img in album.sorted_image_set%}

and added this @property in the CatalogAlbum/models.py:

@property
def sorted_image_set(self):
      return self.images.order_by('throughimage__image_order')

not sure if this is the best way to solve it but looks like it works.
Thank you!

6

I think you did a lot more work than necessary…

I would have left your original code as-it, and then approached the situation from the side of any queries directly retrieving CatalogImage. You didn’t express any issues with the original code, only with the Admin and a manually-entered .filter query.

So my suggestion would have been to select one of:

  • Ignore it. (Now that you know the cause and that you don’t actually have duplicate rows, is it really a problem?)
  • Add a custom ModelAdmin class to retrieve distinct values
  • Add a custom Manager to retrieve distinct values.