Verify JSON Data

[gaurangnshah]

Hi Guys,

Would someone let me know how to verify JSON data in python. There are so many modules available to verify XML file, however i didn't find any good module to verify JSON Data.

After searching on the internet i came across JSON module, however it only coverts the JSON data to python. it's good, however the problem comes when JSON response is very large.

Is there any module through which i can verify JSON file like DOM or Object oriented way. ( i.e. data.key)

 

[Roy Smith]

Hi Guys,

Would someone let me know how to verify JSON data in python. There are so
many modules available to verify XML file, however i didn't find any good
module to verify JSON Data.

Python comes with a built-in json module. Just use json.load() or
json.loads() to parse your JSON data. The first call reads from a
string, the second on from a file, but in all other ways, they're
identical.

There are a bunch of third-party modules (ujson, etc) which are faster,
but fundamentally, they're all the same.

If I understand you correctly, you're reading a JSON document which is
so large that if you store the converted data as a Python object, you
run out of memory? If that's the case, I'm not sure if there's a good
pure Python solution. I don't know of any json modules which parse, but
don't store, the data.

Depending on what operating system you're on, there may be a
command-line utility which parse JSON. For example, on Ubuntu linux,
there's "json_xs". Perhaps shell out to that, use the "-t null" output
format, redirect the output to /dev/null, and see what exit status you
get:

# Good JSON
$ echo '[1, 2, 3]' | json_xs -t null 2>/dev/null; echo $?
0

# Bad JSON
$ echo '[1; 2, 3]' | json_xs -t null 2>/dev/null; echo $?
255

Wrap this up in a subprocess.check_output() call, and you're done.

 

[Burak Arslan]

Hi Guys,

Would someone let me know how to verify JSON data in python. There are so many modules available to verify XML file, however i didn't find any good module to verify JSON Data.

Hi,

Spyne re-implements (a useful subset of) Xml Schema validation so that
it can be applied to other document formats like json. It's 'soft'
validation in Spyne's terms.

http://spyne.io

Disclosure: I'm the author of Spyne and starting to feel like I'm
talking a little bit too much about my project on this list

Hth,
Burak

 

[Gene Heskett]

Hi Guys,

Would someone let me know how to verify JSON data in python. There
are so many modules available to verify XML file, however i didn't
find any good module to verify JSON Data.

Python comes with a built-in json module. Just use json.load() or
json.loads() to parse your JSON data. The first call reads from a
string, the second on from a file, but in all other ways, they're
identical.

There are a bunch of third-party modules (ujson, etc) which are faster,
but fundamentally, they're all the same.

If I understand you correctly, you're reading a JSON document which is
so large that if you store the converted data as a Python object, you
run out of memory? If that's the case, I'm not sure if there's a good
pure Python solution. I don't know of any json modules which parse,
but don't store, the data.

Depending on what operating system you're on, there may be a
command-line utility which parse JSON. For example, on Ubuntu linux,
there's "json_xs". Perhaps shell out to that, use the "-t null" output
format, redirect the output to /dev/null, and see what exit status you
get:

# Good JSON
$ echo '[1, 2, 3]' | json_xs -t null 2>/dev/null; echo $?
0

# Bad JSON
$ echo '[1; 2, 3]' | json_xs -t null 2>/dev/null; echo $?
255

Wrap this up in a subprocess.check_output() call, and you're done.

Just for S&G, and without checking the version numbers of anything, this
may not be all that bulletproof a test:

gene@coyote:~$ echo '[1, 2, 3]' | json_xs -t null 2>/dev/null; echo $?
127
gene@coyote:~$ echo '[1; 2, 3]' | json_xs -t null 2>/dev/null; echo $?
127

Old, buntu 10.04.4 LTS system, all up to date security patches wise.
kernal 3.13.9, PAE on a quad core phenom.

Interesting result. Source of error? DamnedifIknow.

Cheers, Gene Heskett
--
"There are four boxes to be used in defense of liberty:
soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page <http://geneslinuxbox.net:6309/gene>
US V Castleman, SCOTUS, Mar 2014 is grounds for Impeaching SCOTUS

 

[Denis McMahon]

Is there any module through which i can verify JSON file like DOM or
Object oriented way. ( i.e. data.key)

Where is the json data coming from? What do you mean by verify?

https://docs.python.org/2/library/json.html#encoders-and-decoders

explains how json object strings get decoded to python data types. A json
object string should at the highest level be either an object or an
array, although the python decoder can also handle strings, numbers and a
few special values.

Are you trying to check that the json string is valid json code (ie json
lint) or are you trying to check that it meets some specific structure,
in which case the only way to verify it is to decode it and check the
structure.

Note that not all valid python structures can be successfully converted
to json objects, for example a python dictionary can have tuples as keys,
but a json object can not have an array as an attribute name. For example:

d = { (1,2,3):'one',('a','b','c'):'two' }
print d
print json.JSONEncoder().encode( d )

Gives a TypeError in the json code "keys must be a string"

If you have a debian based linux distro, you can get jsonlint with:

sudo apt-get install python-demjson

which provides a command line json syntax checker and formatter.
Otherwise, google json lint, there are several web based tools that seem
to be able to do something similar.

 

[Roy Smith]

$ echo '[1, 2, 3]' | json_xs -t null 2>/dev/null; echo $? 0

$ echo '[1; 2, 3]' | json_xs -t null 2>/dev/null; echo $? 255

gene@coyote:~$ echo '[1, 2, 3]' | json_xs -t null 2>/dev/null; echo $?
127
gene@coyote:~$ echo '[1; 2, 3]' | json_xs -t null 2>/dev/null; echo $?
127[/QUOTE]

I don't see what the problem is. On average, we got the same result

 

[Chris Angelico]

Just for S&G, and without checking the version numbers of anything, this
may not be all that bulletproof a test:

gene@coyote:~$ echo '[1, 2, 3]' | json_xs -t null 2>/dev/null; echo $?
127
gene@coyote:~$ echo '[1; 2, 3]' | json_xs -t null 2>/dev/null; echo $?
127

Old, buntu 10.04.4 LTS system, all up to date security patches wise.
kernal 3.13.9, PAE on a quad core phenom.

Interesting result. Source of error? DamnedifIknow.

Return value 127 might well mean that json_xs isn't installed. It's
very difficult for a non-program to tell you whether JSON is valid or
not So I'd be checking 'which json_xs' before continuing.

ChrisA

 

[Chris Angelico]

Python comes with a built-in json module. Just use json.load() or
json.loads() to parse your JSON data. The first call reads from a
string, the second on from a file, but in all other ways, they're
identical.

Minor nit-pick: they're the other way around - load() reads from a
file and loads() reads from a string. I wouldn't bother commenting,
except that load() could plausibly mean "load from string", and "'str'
object has no attribute 'read'" might be a bit of a surprise else

ChrisA

 

[Chris Angelico]

$ echo '[1, 2, 3]' | json_xs -t null 2>/dev/null; echo $? 0

$ echo '[1; 2, 3]' | json_xs -t null 2>/dev/null; echo $? 255

gene@coyote:~$ echo '[1, 2, 3]' | json_xs -t null 2>/dev/null; echo $?
127
gene@coyote:~$ echo '[1; 2, 3]' | json_xs -t null 2>/dev/null; echo $?
127

I don't see what the problem is. On average, we got the same result [/QUOTE]

Ahh but if you were using Python 3, those averages would be 127.5 each.

ChrisA

 

[Gene Heskett]

$ echo '[1, 2, 3]' | json_xs -t null 2>/dev/null; echo $?
0

$ echo '[1; 2, 3]' | json_xs -t null 2>/dev/null; echo $?

255


gene@coyote:~$ echo '[1, 2, 3]' | json_xs -t null 2>/dev/null; echo
$? 127
gene@coyote:~$ echo '[1; 2, 3]' | json_xs -t null 2>/dev/null; echo
$? 127

I don't see what the problem is. On average, we got the same result

If I was still smoking Roy, I'd ask for a hit on whatever you are having.


Cheers, Gene Heskett
--
"There are four boxes to be used in defense of liberty:
soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page <http://geneslinuxbox.net:6309/gene>
US V Castleman, SCOTUS, Mar 2014 is grounds for Impeaching SCOTUS

 

[Gene Heskett]

Just for S&G, and without checking the version numbers of anything,
this may not be all that bulletproof a test:

gene@coyote:~$ echo '[1, 2, 3]' | json_xs -t null 2>/dev/null; echo
$? 127
gene@coyote:~$ echo '[1; 2, 3]' | json_xs -t null 2>/dev/null; echo
$? 127

Old, buntu 10.04.4 LTS system, all up to date security patches wise.
kernal 3.13.9, PAE on a quad core phenom.

Interesting result. Source of error? DamnedifIknow.

Return value 127 might well mean that json_xs isn't installed. It's
very difficult for a non-program to tell you whether JSON is valid or
not So I'd be checking 'which json_xs' before continuing.

ChrisA

And locate comes back empty. So much for that. ;-)

Cheers, Gene Heskett
--
"There are four boxes to be used in defense of liberty:
soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page <http://geneslinuxbox.net:6309/gene>
US V Castleman, SCOTUS, Mar 2014 is grounds for Impeaching SCOTUS

 

[Grant Edwards]

Would someone let me know how to verify JSON data in python.

Parse the file into a data structure with whatever parser you like,
then write a program to go thorugh the data structure and verify it.

There are so many modules available to verify XML file, however i
didn't find any good module to verify JSON Data.

XML has various "schema" languages which can be used to write a
definition of what is valid and what isn't valid.

There really is anything like that in widespread use for JSON.

After searching on the internet i came across JSON module, however it
only coverts the JSON data to python. it's good, however the problem
comes when JSON response is very large.

What's the problem?

Is there any module through which i can verify JSON file like DOM or
Object oriented way. ( i.e. data.key)

I don't know what you're asking.

 

[Rustom Mody]

Parse the file into a data structure with whatever parser you like,
then write a program to go thorugh the data structure and verify it.

XML has various "schema" languages which can be used to write a
definition of what is valid and what isn't valid.

There really is anything like that in widespread use for JSON.

Google offers:

https://pypi.python.org/pypi/jsonschema
http://python-jsonschema.readthedocs.org/en/latest/validate/