Trying to serialize python dictionary including Models and Forms

Using Django Forms & APIs
1

Hello,

I’m trying to serialize and object in my view. This is an object i’m building for the context of my page and it is a python dictionnary, including django models and forms.

  • As i’m trying to cache this object with the low-level API of django i first tried the classic way :
cache.set("my_dict", my_dict, 60*10)

but i had this error :

Can’t pickle <function …”

The default serializer of django can’t serialize this dictionnary.

  • So i tried to serialize it with json with these lines :
cache.set("my_dict", serializers.serialize('json',my_dict), 10*60)
#or
cache.set("my_dict", json.dumps(my_dict), 10*60)

but i also had an error

‘str’ object has no attribute ‘_meta’

  • Then, i wanted to test to convert it in string with
cache.set("my_dict",str(my_dict), 10*60)

but here the error was when i tried to load the cache with

my_dic = eval(cache.get("my_dict"))

i had

{‘key_1’: <MyModel : MY_OBJECT (96)>, …}
_______^
SyntaxError: invalid syntax

And now I’m a bit out of ideas…

Would you have an idea?
Feel free to ask me questions if I wasn’t specific enough.

Thanks in advance

2

We would need to see how my_dict is created and how all the data within it is obtained to be able to answer this.

3

Thank you for your answer.

my_dict is created with several django requests, also SQL requests in an other database but at the end the data in my_dict are in string or int types, and some functions to generate python lists or dictionnaries.

So if you need all of the types included in my_dict here is an exemple :

my_dict = { "key_1" : <MyModel: MY_OBJECT (96)>,   #Model object
            "key_2": ['1','2','3','4'],            #Python list
            "key_3" : {'id_1':[<MyModel: MY_OBJECT (96)> , <MyForm bound=False, valid=Unknown, fields=(field_1;field_2)> ],...},  #python lists of django objects and forms
            "key_4": datetime.datetime(2020, 5, 17)  #Datetime object

It is a fictitional dictionnary but you have a tour of all types of objects included in it.

And to give you the full error i had when using pickle serializer :

Can’t pickle <function paginator_number …>: it’s not the same object as django.contrib.admin.templatetags.admin_list.paginator_number

What’s strange is that i don"t use pagination in this view, nor in the model’s methods or forms.

I hope my answer is sufficient.
Thank you

4

Unfortunately, it’s not.

You’re trying to address a very specific error. Posting a “representative example” of what you think my_dict contains when this error occurs is not sufficient for answering your question about how a paginator is involved here.

For us to be able to have any realistic chance to diagnose this, we’re going to need to see the complete view where my_dict is being created, and that’s just as a starting point. This may extend into the forms and model definitions as well.

5

I understand yes.
The situation has changed a little, I now know which parts of the dictionary generate this error and it is the django forms.

I also decided to use the session method of django instead of cache directly, even if it still use cache system, but the error is the same cause it also needs to serialize my object.

I generate pre-filled forms that I integrate into my_dict, and they create the serialization error.

Here is my form:

class PatientForm(forms.Form):
    patient_sex = forms.ChoiceField(
        choices=Patient._meta.get_field("patient_sex").choices, 
        required=False,
    )
    patient_weight= forms.DecimalField(
        required=False,
        widget=forms.NumberInput(attrs={'class': 'form-control'}),
    )
    patient_abo=forms.ChoiceField(
        choices= Patient._meta.get_field("patient_abo").choices,
        required=False,
    )
    patient_cmv=forms.CharField(
        max_length=15,
        required=False,
    )

and here the view i tested : (i know my_dict i not a dictionnary in this exemple but it is to keep consistency with my previous posts)

def Main(request, abo, cmv):
    #session cache update
    my_dict= request.session.get("my_dict", None)
    if not my_dict:
        my_dict = PatientForm(initial={'patient_abo':abo,'patient_cmv':cmv}) #prefilled form
        request.session['my_dict'] = my_dict #ERROR HERE when trying to add in session cache

    #if form submit in template
    if request.method == 'POST' and 'submitForm' in request.POST:
        my_form = PatientForm(request.POST)
        is my_form.is_valid():
            patient_object = my_form.save(commit=False)
            patient_object.save()

            #update session
            request.session['my_dict']= PatientForm(initial={'patient_abo':patient_object.patient_abo,'patient_cmv':patient_object.patient_cmv})

    return render(request, 'Register/main.html', {'my_dict':my_dict})

and the error is still the same :

_pickle.PicklingError: Can’t pickle <function paginator_number at 0x0000015D4200C160>: it’s not the same object as django.contrib.admin.templatetags.admin_list.paginator_number

Thank you

6

Good, you’re getting closer.

Am I to conclude that PatientForm is really RecipientGeneralInfoForm as shown above?

What are the abo and cmv objects being passed into Main?

7

Yes sorry about the form name i edited my post.

abo is a string from a list of choices. And cmv a free string.
I am sure of the good compatibility of these variables with the form. I tested it several times.

8

It’s not a question of it being compatible with the forms. Forms are designed to work with as many datatypes as makes sense.

It’s a question of whether or not those objects are directly serializable.

Edit: Playing around a little bit in the shell is starting to give me the impression that Form objects are not serializable - at least not by the built-in serializer. It simply may be that you need to extract the data from the fields and serialize that independently of the Form object itself - or provide a custom serializer.

From Serializing Django objects | Django documentation | Django

Django’s serialization framework provides a mechanism for “translating” Django models into other formats.

[Emphasis added]

9

i understand. My big question is about the error on a “function”, because I did not define a function in my form. And even less about pagination so i assume it may come from something else.

I didn’t find anything in forums about this specific error. Maybe have you seen other types of pickle’ error about function ? If so, what was the source ?

10

You may not directly have done that, but I guarantee that the base class (forms.Form) defines a number of functions.

11

I see. Well i’ll try to define a custom serializer because i already tried ‘json’ and ‘xml’ and it didn’t work too.

Thank you for your research

12

I would suggest that you’d be better off just serializing the initial attribute of the form and using that to reconstruct the form when you need it.

Forms are constructed with a fair amount of metaclass code, similar to how Models are built. They aren’t “trivial” objects. (see the source code in django.forms.forms)

13

i understand thank you, i’ll try this instead.