Correct approach to manipulating JSON

Using Django
1

I have an app that I am developing to retrieve some data as JSON from a remote server and populate some fields on a form. The basic process is up and running, in that I can send the javascript request from the browser, to my test server, and the test server relays that on to the remote server with the browser getting the JSON it requires and populating the fields.

Essentially I have two questions:

(i) my approach (explained below) seems a bit hacky to me, is there a better way?

(ii) I am having problems with encoding, that I don’t seem to be able to resolve – in that I am either getting HTML entities in the returned JSON, or I am at risk of getting invalid JSON because there are unescaped characters (principally ").

Here’s the approach:

After a lot of messsing around and and not getting very far I made a simple view, whereby an “id” is passed to the view and it in turn gets a JSON response from a remote server.

Because the JSON returned is both more extensive and more complex than actually required by the browser, I used a template to return something more limited (I don’t really want to have the browser churning through the JSON):

views.py

def release(request):
	rawdata = {}
	if 'id' in request.GET :
		id = request.GET['id']
		
		url = 'https://api.discogs.com/releases/' + id
        #886973923825
        #0602557970821
        #886973922323
		response = requests.get(url)
		rawdata = response.json()
    
	return render(request, 'core/release.html', {
		'rawdata': rawdata,
		#'trackstring': trackstring,
		#'artists': rawdata['artists'],
	})

release.html

{% autoescape off %}
    {% block content %}
        {% if rawdata.id %}
            { "tracks" : "
            {% for track in rawdata.tracklist %}
                {% if forloop.last %}
                    {{forloop.counter}} 
                    ({{ track.position }}) 
                    {{ track.title }} 
                    {{ track.duration }}
                {% else %}
                    {{forloop.counter}}
                    ({{ track.position }}) 
                    {{ track.title }} 
                    {{ track.duration }}
                    \n
                {% endif %}
            {% endfor %}"}
        {% else %}
            { "tracks" : "No results", "value" : ""}
        {% endif %}
    {% endblock %}
{% endautoescape %}

As you can see, the template concatenates the separate track elements into a single tracks element with some readability/humanising, which is insterted into a textarea. There will be a couple of other simplified elements too.

As will also be obvious, I have autoescape off, because otherwise I end up with HTML entities in the tracks element, which then get inserted raw into the textarea. However, this brings its own problems, in that double quotes can appear in the strings, that are unescaped, and consequently cause the browser parsing to fail.

I tried {{ track.title|escapejs }} but this has the odd effect of escaping/encoding most extended characters as unicode (\u+nnn) but double quotes as HTML entites. The former seems to suit the javascript in the browser, but not the latter.

So, is there a way to encode the data appropriately, and/or am I barking up the wrong tree with this method of adapting/reformatting the JSON for the browser?

Thanks

2

I would think that you’d be a lot better off reading the JSON and using the json.loads method to convert it to python objects (lists, dicts).

You can then manipulate / alter those objects in any desired manner (adding or deleting keys or entries), and then turn around and use the Django JsonResponse to return the proper response.

I wouldn’t be involving templates at all in this process.

3

I tried going down that route, but did not seem able to progress it.

I added this to the view:

…
        response = requests.get(url)
		rawdata = response.json()
		print('testing type, etc')
		print(type(rawdata))
		print(dict(rawdata))
		print(rawdata.id)

Which output this in the terminal:

testing type, etc
<class 'dict'>
{'id': 12111831, 'status': 'Accepted',…
Internal Server Error: /release/
…
    print(rawdata.id)
AttributeError: 'dict' object has no attribute 'id'

Isn’t it showing that the first element of the dict is ‘id’!?

If I try something like:

rawdata = json.loads(response.json())

I get the error:

TypeError: the JSON object must be str, bytes or bytearray, not dict

So response.json is already a dict!?

4

You are correct, rawdata is going to be the decoded json data from the api. But it’s a dict, not an object.

would more appropriately be:
print(rawdata['id'])

5

Thanks, that helped resolve it :smile: