accessors on recursive foreign keys

I have a model, Code, with a recursive foreign key to itself (‘self’). This field is called ‘childof’, and is meant to model a hierarchy in Code, i.e., section1 is a childof chapter2, so section1.childof returns chapter2.

Therefore, chapter2.code_set should return section1 and its siblings. However, I have overridden code_set with the related_name ‘parent’ on the childof field. Nevertheless, both parent and code_set appear in dir() - as does childof. There is no ‘childof_set’ or ‘self_set’ in dir().

Here is my problem: chapter2.parent returns a Manager object. But chapter2.parent.all() returns the ‘Manager cannot be accessed by instances’ error. My best guess is that this is happening because chapter2 is itself a childof subtitle3, but that’s just a guess.

The same error occurs with chapter2.code_set. Here is the relevant portion of my model code and some actual samples from the repl.


    childof = models.ForeignKey(
        'self', on_delete=models.CASCADE, related_name='parent', blank=True, null=True,
        default=None)

cg = Code.objects.filter(short_arrow__startswith="TITLE 14-COAST GUARD")

cg1 = Code.objects.get(pk='287ea0df')
cg1
<Code: TITLE 14-COAST GUARD USCE SUBCHAPTER III-PROCUREMENT>

cg2 = Code.objects.get(short_arrow__contains='CHAPTER 11-ACQUISITIONS')
cg2
<Code: TITLE 14-COAST GUARD USCE CHAPTER 11-ACQUISITIONS>
cg2.childof
<Code: TITLE 14-COAST GUARD USCE SUBTITLE I-ESTABLISHMENT, POWERS, DUTIES, AND ADMINISTRATION>
cg2.parent
<django.db.models.fields.related_descriptors.create_reverse_many_to_one_manager.<locals>.RelatedManager at 0x7f17f4b49df0>

cg1.childof
<Code: TITLE 14-COAST GUARD USCE CHAPTER 11-ACQUISITIONS>    
cg1.childof == cg2
True

cg2.parent.all()
Out[18]: Traceback (most recent call last):
...
AttributeError: Manager isn't accessible via Code instances

cg2.code_set()
Traceback (most recent call last):
...
AttributeError: 'RelatedManager' object has no attribute 'code_set'

cg2.parent.count
<bound method BaseManager._get_queryset_methods.<locals>.create_method.<locals>.manager_method of <django.db.models.fields.related_descriptors.create_reverse_many_to_one_manager.<locals>.RelatedManager object at 0x7f17f4b49430>>

cg2.parent.count()
Traceback (most recent call last):
...
AttributeError: Manager isn't accessible via Code instances

cg4 = cg.select_related().filter(short_arrow__contains='CHAPTER 11-ACQUISITIONS')

cg4
<QuerySet [<Code: TITLE 14-COAST GUARD USCE CHAPTER 11-ACQUISITIONS>]>

I have two other recursive fk to self, one without a related name:

 approval = models.ForeignKey(
        'self', on_delete=models.PROTECT, blank=True, null=True, default=None)

And one with:

 authorizing_statute = models.ForeignKey(
        'self', on_delete=models.PROTECT, related_name='source_statute', blank=True, null=True,  default=None)

My code passes check and I am not getting any accessor errors.

Please advise. Thanks.

This would be cg2.code_set.all(). Code_set is a manager on which you call queryset functions such as all, filter or get.

Thank you for pointing that out. I can’t believe I forgot / skipped that option. However, the
problem remains:

from codeAT.models import Code
cg = Code.objects.filter(short_arrow_startswith='TITLE 14-COAST')
cg1 = cg.get(short_arrow='TITLE 14-COAST GUARD USCE SUBCHAPTER III-PROCUREMENT')
cg1.code_set.all()
Out[6]: Traceback (most recent call last):
  File "/home/malikarumi/.virtualenvs/hattie/lib/python3.8/site-packages/IPython/core/formatters.py", line 222, in catch_format_error
    r = method(self, *args, **kwargs)
  File "/home/malikarumi/.virtualenvs/hattie/lib/python3.8/site-packages/IPython/core/formatters.py", line 707, in __call__
    printer.pretty(obj)
  File "/home/malikarumi/.virtualenvs/hattie/lib/python3.8/site-packages/IPython/lib/pretty.py", line 410, in pretty
    return _repr_pprint(obj, self, cycle)
  File "/home/malikarumi/.virtualenvs/hattie/lib/python3.8/site-packages/IPython/lib/pretty.py", line 778, in _repr_pprint
    output = repr(obj)
  File "/home/malikarumi/.virtualenvs/hattie/lib/python3.8/site-packages/django/db/models/query.py", line 296, in __repr__
    data = list(self[: REPR_OUTPUT_SIZE + 1])
  File "/home/malikarumi/.virtualenvs/hattie/lib/python3.8/site-packages/django/db/models/query.py", line 347, in __getitem__
    qs = self._chain()
  File "/home/malikarumi/.virtualenvs/hattie/lib/python3.8/site-packages/django/db/models/query.py", line 1480, in _chain
    obj = self._clone()
  File "/home/malikarumi/.virtualenvs/hattie/lib/python3.8/site-packages/django/db/models/query.py", line 1493, in _clone
    query=self.query.chain(),
  File "/home/malikarumi/.virtualenvs/hattie/lib/python3.8/site-packages/django/db/models/query.py", line 236, in query
    self._filter_or_exclude_inplace(negate, args, kwargs)
  File "/home/malikarumi/.virtualenvs/hattie/lib/python3.8/site-packages/django/db/models/query.py", line 1096, in _filter_or_exclude_inplace
    self._query.add_q(Q(*args, **kwargs))
  File "/home/malikarumi/.virtualenvs/hattie/lib/python3.8/site-packages/django/db/models/sql/query.py", line 1502, in add_q
    clause, _ = self._add_q(q_object, self.used_aliases)
  File "/home/malikarumi/.virtualenvs/hattie/lib/python3.8/site-packages/django/db/models/sql/query.py", line 1532, in _add_q
    child_clause, needed_inner = self.build_filter(
  File "/home/malikarumi/.virtualenvs/hattie/lib/python3.8/site-packages/django/db/models/sql/query.py", line 1414, in build_filter
    value = list(value)
  File "/home/malikarumi/Projects/hattie/codeAT/models.py", line 122, in __next__
    if self.index >= len(self.objects):
  File "/home/malikarumi/.virtualenvs/hattie/lib/python3.8/site-packages/django/db/models/manager.py", line 186, in __get__
    raise AttributeError(
AttributeError: Manager isn't accessible via Code instances
cg2 = cg.get(short_title__contains='Sec. 1152')
cg2.childof
Out[8]: <Code: TITLE 14-COAST GUARD USCE SUBCHAPTER III-PROCUREMENT>

Because cg2 is a child of cg1, I would expect cg2 to show up in cg1’s code_set(), or parent list,
or something, and maybe it would, if I could figure out either a) which object manager works or b)
how to fix the ones I have.

Questions:
When we use related_name, is that the same as directly changing the name of the default
Manager?
If not, are parent and code_set aliases of each other?
I don’t get why they both still exist.
Do I need to rewrite my model to get this to work? Any references to knowledgeable sources to help
me work out how?
Do you have any other suggestions, workaround, fixes?
After my first post on this thread, I went looking at Django-Treebeard. Do you have any thoughts
about that?

What version of Django and Python are you using?

Which database engine?

Couple different random thoughts here -

  • I can’t recreate the problem you’re describing here. Based on the information you’ve provided, everything that I have now tried has worked as expected.

  • With the related_name parameter set, I don’t have access to an attribute named code_set

  • This all leads me to believe one of:

    • There’s a third-party library causing problems
    • Something you’ve defined in your models is causing the problem
    • Something in your code creating the instances of your model is causing problems
  • We use Treebeard extensively. We love it.

1 Like

Ok, thanks. I’ve been slow on the uptake, but will be taking a hard look at this today. Hopefully with a solution…

@KenWhitesell et al

  1. I started by commenting out the other two recursive fields and migrating. That eliminated
    code_set, but did not change the reverse accessor issue. Is there a way to hack the RelatedManager to fix this? I don’t know what else to do here.

  2. I saw a suggestion to use an explicit junction table on Stack Overflow, but making a round trip -
    or two - to an external table seems like an awful lot of overhead for this situation.

  3. If instead of a junction table, if I made an explicit parentto field on the model, would that
    work?

Presumably the related name on the childof field would still fail like it does now,
but I would instead have the explicit parent field to work with. I still would not have the
automatic reverse relation, and I would have to come up with a script to fill in the parentto
field, but that might solve my problem:

    # pseudocode
    family = c.itertools.groupby(instance.childof)  
    family = c.pandas.groupby(instance.childof)  
    for f in family:
        paren = c.objects.get(instance.childof)
        OR
        paren = instance.childof
        c.objects.update(paren.parentto=f)  # where parentto is a Postgresql ArrayField
        OR  # https://docs.djangoproject.com/en/4.0/ref/models/relations/#django.db.models
        .fields.related.RelatedManager.add:
            >>> b = Blog.objects.get(id=1)
            >>> e = Entry.objects.get(id=234)
            >>> b.entry_set.add(e) # Associates Entry e with Blog b
            SEE ALSO: https://docs.djangoproject.com/en/4.0/topics/db/examples/many_to_one/

If I did this two field hack, then I don’t really need either one to be ForeignKeys any more,
do I? They could be a CharField and an ArrayField, couldn’t they? It breaks the extended lookup chain - but I don’t have that now, anyway - or at least, I only have it in one direction.

Please advise. Thanks.

UPDATE:

Hey @KenWhitesell Ken, et al…

I was trying to see if add on the RelatedManager would work: Related objects reference | Django documentation | Django

Here’s the prior attempt:

st1 = ch1.childof
st1.parent
Out[14]: <django.db.models.fields.related_descriptors.create_reverse_many_to_one_manager.<locals>.RelatedManager at 0x7fb78bebcd30>
st1.parent.count()
Traceback (most recent call last):
<..snip..>

Then I did this:

st1.parent.add(ch1)  # <-- no error!
st1.parent.count()
Traceback (most recent call last):

But check out the bottom of the traceback:

File “/home/malikarumi/.virtualenvs/hattie/lib/python3.8/site-packages/IPython/core/interactiveshell.py”, line 3398, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File “”, line 1, in <cell line: 1>
st1.parent.count()
File “/home/malikarumi/.virtualenvs/hattie/lib/python3.8/site-packages/django/db/models/manager.py”, line 85, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File “/home/malikarumi/.virtualenvs/hattie/lib/python3.8/site-packages/django/db/models/query.py”, line 470, in count
return self.query.get_count(using=self.db)
File “/home/malikarumi/.virtualenvs/hattie/lib/python3.8/site-packages/django/db/models/query.py”, line 236, in query
self._filter_or_exclude_inplace(negate, args, kwargs)
File “/home/malikarumi/.virtualenvs/hattie/lib/python3.8/site-packages/django/db/models/query.py”, line 1096, in _filter_or_exclude_inplace
self._query.add_q(Q(*args, **kwargs))
File “/home/malikarumi/.virtualenvs/hattie/lib/python3.8/site-packages/django/db/models/sql/query.py”, line 1502, in add_q
clause, _ = self._add_q(q_object, self.used_aliases)
File “/home/malikarumi/.virtualenvs/hattie/lib/python3.8/site-packages/django/db/models/sql/query.py”, line 1532, in _add_q
child_clause, needed_inner = self.build_filter(
File “/home/malikarumi/.virtualenvs/hattie/lib/python3.8/site-packages/django/db/models/sql/query.py”, line 1414, in build_filter
value = list(value)

File "/home/malikarumi/Projects/hattie/codeAT/models.py", line 124, in __next__
    if self.index >= len(self.objects):

File “/home/malikarumi/.virtualenvs/hattie/lib/python3.8/site-packages/django/db/models/manager.py”, line 186, in get
raise AttributeError(
AttributeError: Manager isn’t accessible via Code instances

What the heck is a call to next doing here? Isn’t that a mistake?

Here’s manager.py:

class ManagerDescriptor:
    def __init__(self, manager):
        self.manager = manager

    def __get__(self, instance, cls=None):
        if instance is not None:
            raise AttributeError(
                "Manager isn't accessible via %s instances" % cls.__name__
            )

And this is line 1414 in query.py:

            # Prevent iterator from being consumed by check_related_objects()
            if isinstance(value, Iterator):
                value = list(value)
            self.check_related_objects(join_info.final_field, value, join_info.opts)

I don’t see what the len of model instances has to do with the _set of an instance’s reverse foreign keys, except if the _set is longer than the set of available instances on the other model - not very likely, I would think.

If Django is calling next, why didn’t:
1. I get StopIteration instead of the Manager error?
2.
ManagerDescriptor.get() is clear that if it is an instance, call the error, but, as you know, in this case that defeats the whole purpose of a recursive foreign key to ‘self’. So shouldn’t it be calling something else, something I haven’t found yet, like class RelatedManager or something?

I’n going to keep working on this while I wait to hear back from you

Please see my previous response. I cannot recreate the errors you are describing based upon the information you’ve presented here so far. There’s something else involved affecting this.