Migrating from postgresql to elasticsearch

Using Django Using the ORM
1

I join two models with select_related and insert their content into elasticsearch with iterator(). It seems that I cannot obtain the content of the left joined Abstract table. How can I obtain the column paper_abstract in the example bellow?

class Paper(models.Model):
    paper_title = models.CharField(max_length=2)

class Abstract(models.Model):
    paper = models.OneToOneField(Paper,
                                 on_delete=models.CASCADE,
                                 related_name='abstracts',
                                 primary_key=True)
    paper_abstract = models.TextField()

And the bulk_update.py that I use with python manage.py bulk_update.

from django.core.management.base import BaseCommand

from elasticsearch_dsl import connections
from elasticsearch.helpers import bulk

from django.db import connection

from appln.models import Paper, Abstract

class Command(BaseCommand):
    help = 'Bulk indexing.'

    def _document_generator(self):

        for paper_abst in Paper.objects.select_related('abstracts').iterator():

            yield {
                '_index': 'paper_dev',
                'paper_id': paper_abst.id,
                'paper_title': paper_abst.title,
                'paper_abstract': paper_abst.paper_abstract
            }

    def handle(self, *args, **kwargs):
        index = 'paper_dev'
        self.stdout.write(f'Bulk idexing "{index}" ...')
        connection = connections.get_connection()
        succeeded, _ = bulk(connection, actions=self._document_generator(), stats_only=True)
        self.stdout.write(f'Updated {succeeded} successfully')
2

paper_abst is a reference to the Paper object in your query. You need to follow the reference abstracts to access the Abstract object before you can retrieve the paper_abstract field from it.
e.g. paper_abst.abstracts.paper_abstract.

3

Thanks for your answer. I have changed the _document_generator as follows;

def _document_generator(self):

        for paper_abst in Paper.objects.select_related('abstracts').iterator():

            pprint(vars(paper_abst))

            yield {
                '_index': 'paper_dev',
                'paper_id': paper_abst.id,
                'paper_title': paper_abst.title,
                'paper_abstract': paper_abst.abstracts.paper_abstract
            }

The pprint doesn’t give abstracts and when I run the bulk_update.py I got the following error:

appln.models.Paper.abstracts.RelatedObjectDoesNotExist: Paper has no abstracts.
4

Have you verified that the Abstract exists for every Paper being referenced?

Have you tried verifying the query in the Django shell?

5

My mistake. Thanks a lot.