Typeerror: String Indices Must Be Integers While Parsing Json Using Python?

Question

I am confuse now why I am not able to parse this JSON string. Similar code works fine on other JSON string but not on this one - I am trying to parse JSON String and extract script from the JSON.

Below is my code.

for step in steps:
    step_path = '/example/v1' +'/'+step

    data, stat = zk.get(step_path)
    jsonStr = data.decode("utf-8")
    print(jsonStr)
    j = json.loads(json.dumps(jsonStr))
    print(j)
    shell_script = j['script']
    print(shell_script)

So the first print(jsonStr) will print out something like this -

{"script":"#!/bin/bash\necho Hello world1\n"}

And the second print(j) will print out something like this -

{"script":"#!/bin/bash\necho Hello world1\n"}

And then the third print doesn't gets printed out and it gives this error -

Traceback (most recent call last):
  File "test5.py", line 33, in <module>
    shell_script = j['script']
TypeError: string indices must be integers

So I am wondering what wrong I am doing here?

I have used same above code to parse the JSON and it works fine..

Accepted Answer

The problem is that jsonStr is a string that encodes some object in JSON, not the actual object.

You obviously knew it was a string, because you called it jsonStr. And it's proven by the fact that this line works:

jsonStr = data.decode("utf-8")

So, jsonStr is a string. Calling json.dumps on a string is perfectly legal. It doesn't matter whether that string was the JSON encoding of some object, or your last name; you can encode that string in JSON. And then you can decode that string, getting back the original string.

So, this:

j = json.loads(json.dumps(jsonStr))

… is going to give you back the exact same string as jsonStr in j. Which you still haven't decoded to the original object.

To do that, just don't do the extra encode:

j = json.loads(jsonStr)

If that isn't clear, try playing with it an interactive terminal:

>>> obj = ['abc', {'a': 1, 'b': 2}]
>>> type(obj)
list
>>> obj[1]['b']
2
>>> j = json.dumps(obj)
>>> type(j)
str
>>> j[1]['b']
TypeError: string indices must be integers
>>> jj = json.dumps(j)
>>> type(jj)
str
>>> j
'["abc", {"a": 1, "b": 2}]'
>>> jj
'"[\\"abc\\", {\\"a\\": 1, \\"b\\": 2}]"'
>>> json.loads(j)
['abc', {'a': 1, 'b': 2}]
>>> json.loads(j) == obj
True
>>> json.loads(jj)
'["abc", {"a": 1, "b": 2}]'
>>> json.loads(jj) == j
True
>>> json.loads(jj) == obj
False

Accepted Answer

Try replacing j = json.loads(json.dumps(jsonStr)) with j = json.loads(jsonStr).

Answer 3

Ok... So for people who are still lost because they are used to JS this is what I understood after having tested multiple use cases :

json.dumps does not make your string ready to be loaded with json.loads. It will only encode it to JSON specs (by adding escapes pretty much everywhere) !
json.loads will transform a correctly formatted JSON string to a python dictionary. It will only work if the JSON follows the JSON specs (no single quotes, uppercase for boolean's first letter, etc).

Must Read

Dumping JSON - An encoding story

Lets take an example !

$ obj = {"foobar": True}

This is NOT json ! This is a python dictionary that uses python types (like booleans).

True is not compatible with the JSON specs so in order to send this to an API you would have to serialize it to REAL JSON. That's where json.dumps comes in !

$ json.dumps({"foobar": True})
'{"foobar": true}'

See ? True became true which is real JSON. You now have a string that you can send to the real world. Good job.

Typeerror: String Indices Must Be Integers While Parsing Json Using Python?

3 Answers

Must Read

Dumping JSON - An encoding story

Sophia Al-Mansoor