Verify JSON Data

Discussion in 'Python' started by gaurangnshah, May 26, 2014.

  1. gaurangnshah

    gaurangnshah Guest

    Hi Guys,

    Would someone let me know how to verify JSON data in python. There are so many modules available to verify XML file, however i didn't find any good module to verify JSON Data.

    After searching on the internet i came across JSON module, however it only coverts the JSON data to python. it's good, however the problem comes when JSON response is very large.

    Is there any module through which i can verify JSON file like DOM or Object oriented way. ( i.e. data.key)
     
    gaurangnshah, May 26, 2014
    #1
    1. Advertisements

  2. gaurangnshah

    Roy Smith Guest

    Python comes with a built-in json module. Just use json.load() or
    json.loads() to parse your JSON data. The first call reads from a
    string, the second on from a file, but in all other ways, they're
    identical.

    There are a bunch of third-party modules (ujson, etc) which are faster,
    but fundamentally, they're all the same.

    If I understand you correctly, you're reading a JSON document which is
    so large that if you store the converted data as a Python object, you
    run out of memory? If that's the case, I'm not sure if there's a good
    pure Python solution. I don't know of any json modules which parse, but
    don't store, the data.

    Depending on what operating system you're on, there may be a
    command-line utility which parse JSON. For example, on Ubuntu linux,
    there's "json_xs". Perhaps shell out to that, use the "-t null" output
    format, redirect the output to /dev/null, and see what exit status you
    get:

    # Good JSON
    $ echo '[1, 2, 3]' | json_xs -t null 2>/dev/null; echo $?
    0

    # Bad JSON
    $ echo '[1; 2, 3]' | json_xs -t null 2>/dev/null; echo $?
    255

    Wrap this up in a subprocess.check_output() call, and you're done.
     
    Roy Smith, May 26, 2014
    #2
    1. Advertisements

  3. gaurangnshah

    Burak Arslan Guest

    Hi,

    Spyne re-implements (a useful subset of) Xml Schema validation so that
    it can be applied to other document formats like json. It's 'soft'
    validation in Spyne's terms.

    http://spyne.io

    Disclosure: I'm the author of Spyne and starting to feel like I'm
    talking a little bit too much about my project on this list :)

    Hth,
    Burak
     
    Burak Arslan, May 26, 2014
    #3
  4. gaurangnshah

    Gene Heskett Guest

    Just for S&G, and without checking the version numbers of anything, this
    may not be all that bulletproof a test:

    gene@coyote:~$ echo '[1, 2, 3]' | json_xs -t null 2>/dev/null; echo $?
    127
    gene@coyote:~$ echo '[1; 2, 3]' | json_xs -t null 2>/dev/null; echo $?
    127

    Old, buntu 10.04.4 LTS system, all up to date security patches wise.
    kernal 3.13.9, PAE on a quad core phenom.

    Interesting result. Source of error? DamnedifIknow.

    Cheers, Gene Heskett
    --
    "There are four boxes to be used in defense of liberty:
    soap, ballot, jury, and ammo. Please use in that order."
    -Ed Howdershelt (Author)
    Genes Web page <http://geneslinuxbox.net:6309/gene>
    US V Castleman, SCOTUS, Mar 2014 is grounds for Impeaching SCOTUS
     
    Gene Heskett, May 26, 2014
    #4
  5. Where is the json data coming from? What do you mean by verify?

    https://docs.python.org/2/library/json.html#encoders-and-decoders

    explains how json object strings get decoded to python data types. A json
    object string should at the highest level be either an object or an
    array, although the python decoder can also handle strings, numbers and a
    few special values.

    Are you trying to check that the json string is valid json code (ie json
    lint) or are you trying to check that it meets some specific structure,
    in which case the only way to verify it is to decode it and check the
    structure.

    Note that not all valid python structures can be successfully converted
    to json objects, for example a python dictionary can have tuples as keys,
    but a json object can not have an array as an attribute name. For example:

    d = { (1,2,3):'one',('a','b','c'):'two' }
    print d
    print json.JSONEncoder().encode( d )

    Gives a TypeError in the json code "keys must be a string"

    If you have a debian based linux distro, you can get jsonlint with:

    sudo apt-get install python-demjson

    which provides a command line json syntax checker and formatter.
    Otherwise, google json lint, there are several web based tools that seem
    to be able to do something similar.
     
    Denis McMahon, May 26, 2014
    #5
  6. gaurangnshah

    Roy Smith Guest


    gene@coyote:~$ echo '[1, 2, 3]' | json_xs -t null 2>/dev/null; echo $?
    127
    gene@coyote:~$ echo '[1; 2, 3]' | json_xs -t null 2>/dev/null; echo $?
    127[/QUOTE]

    I don't see what the problem is. On average, we got the same result :)
     
    Roy Smith, May 26, 2014
    #6
  7. Return value 127 might well mean that json_xs isn't installed. It's
    very difficult for a non-program to tell you whether JSON is valid or
    not :) So I'd be checking 'which json_xs' before continuing.

    ChrisA
     
    Chris Angelico, May 26, 2014
    #7
  8. Minor nit-pick: they're the other way around - load() reads from a
    file and loads() reads from a string. I wouldn't bother commenting,
    except that load() could plausibly mean "load from string", and "'str'
    object has no attribute 'read'" might be a bit of a surprise else :)

    ChrisA
     
    Chris Angelico, May 26, 2014
    #8
  9. I don't see what the problem is. On average, we got the same result :)[/QUOTE]

    Ahh but if you were using Python 3, those averages would be 127.5 each.

    ChrisA
     
    Chris Angelico, May 26, 2014
    #9
  10. gaurangnshah

    Gene Heskett Guest

    If I was still smoking Roy, I'd ask for a hit on whatever you are having.
    :)

    Cheers, Gene Heskett
    --
    "There are four boxes to be used in defense of liberty:
    soap, ballot, jury, and ammo. Please use in that order."
    -Ed Howdershelt (Author)
    Genes Web page <http://geneslinuxbox.net:6309/gene>
    US V Castleman, SCOTUS, Mar 2014 is grounds for Impeaching SCOTUS
     
    Gene Heskett, May 26, 2014
    #10
  11. gaurangnshah

    Gene Heskett Guest

     
    Gene Heskett, May 26, 2014
    #11
  12. Parse the file into a data structure with whatever parser you like,
    then write a program to go thorugh the data structure and verify it.
    XML has various "schema" languages which can be used to write a
    definition of what is valid and what isn't valid.

    There really is anything like that in widespread use for JSON.
    What's the problem?
    I don't know what you're asking.
     
    Grant Edwards, May 26, 2014
    #12
  13. gaurangnshah

    Rustom Mody Guest

    Rustom Mody, May 27, 2014
    #13
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.