XML, JSON, or what?

Discussion in 'Python' started by Frank Millman, Jun 8, 2006.

  1. Hi all

    I am writing a multi-user accounting/business application, which uses
    sockets to communicate between server and client. The server contains
    all the business logic. It has no direct knowledge of the client. I
    have devised a simple message format to exchange information between
    the two.

    At first, I used XML as a message format. Then I read the article that
    recommended not using XML for Python-to-Python, so I changed it to a
    pickled collection of Python objects. It works just as well.

    At present the client uses wxPython. One of my medium-term goals is to
    write a web-based client. I don't think it will be easy to reproduce
    all the functionality that I have at present, but I hope to get close.
    I have not done any serious research yet, but I am pretty sure I will
    use javascript on the client, to make it as universal as possible.

    Ideally, the server should be able to handle a wxPython client or a web
    client, without even knowing which one it is talking to. Obviously I
    cannot use Pickle for this. So my question is, what is the ideal format
    to use? I could go back to XML, or I could switch to JSON - I have read
    a bit about it and it does not seem complicated. My messages are not
    very large (maximum about 2k so far for a complex screen layout) so I
    don't think performance will be an issue.

    I would rather make a decision now, otherwise I will have a lot of
    changes to make later on. Does anyone have any recommendations?

    Thanks

    Frank Millman
    Frank Millman, Jun 8, 2006
    #1
    1. Advertising

  2. Frank Millman

    A.T.Hofkamp Guest

    On 2006-06-08, Frank Millman <> wrote:
    > I would rather make a decision now, otherwise I will have a lot of
    > changes to make later on. Does anyone have any recommendations?


    Did you consider XMPP?
    With XMPP you create XML streams between your server and the client.
    XMPP is an open standard (by the IETF).

    Jabber (jabber.org) is based on the XMPP standard. They develop protocols on
    XMPP for all kinds of data exchange. Currently, it is mainly used for chatting,
    but we use it in-house as publish/subscribe to control some machines too.
    They do a lot of thinking about scalabilty, privacy, authentication, and all
    that other stuff you need to do when you put some service on the internet.

    They also have protocols for using a GUI interactively (much like the wizards
    of Win* do, you get one page of things to decide, you press 'next', you get the
    next page, etc).

    Albert
    A.T.Hofkamp, Jun 8, 2006
    #2
    1. Advertising

  3. Frank Millman

    Ant Guest

    > to use? I could go back to XML, or I could switch to JSON - I have read

    I'd favour JSON if the data structures are simple personally. XML is
    comparatively speaking a pain to deal with, where with JSON you can
    simply eval() the data and you have a Python dictionary at your
    disposal.

    I recently used JSON as a way of passing data from a Java backend to a
    web page for Javascript to deal with, with the added side effect that
    my Python testing scripts could also easily read the same data.
    Ant, Jun 8, 2006
    #3
  4. Frank Millman

    Alan Kennedy Guest

    [Frank Millman]
    > I am writing a multi-user accounting/business application, which uses
    > sockets to communicate between server and client. The server contains
    > all the business logic. It has no direct knowledge of the client. I
    > have devised a simple message format to exchange information between
    > the two.
    >
    > At first, I used XML as a message format. Then I read the article that
    > recommended not using XML for Python-to-Python, so I changed it to a
    > pickled collection of Python objects. It works just as well.


    If you were just communicating python to python, I'd recommend Pyro,
    since it has all the socket management, etc, already taken care of.

    http://pyro.sourceforge.net

    > At present the client uses wxPython. One of my medium-term goals is to
    > write a web-based client. I don't think it will be easy to reproduce
    > all the functionality that I have at present, but I hope to get close.
    > I have not done any serious research yet, but I am pretty sure I will
    > use javascript on the client, to make it as universal as possible.


    If you're going to mix javascript client and python server, you
    definitely need something cross platform, like XML or JSON.

    > Ideally, the server should be able to handle a wxPython client or a web
    > client, without even knowing which one it is talking to. Obviously I
    > cannot use Pickle for this. So my question is, what is the ideal format
    > to use? I could go back to XML, or I could switch to JSON - I have read
    > a bit about it and it does not seem complicated.


    JSON is indeed (mostly) as simple as it looks: it fits your need very
    well. And you should be up and running with it very quickly.

    > My messages are not
    > very large (maximum about 2k so far for a complex screen layout) so I
    > don't think performance will be an issue.


    And parsing JSON will almost certainly be faster than parsing XML. You
    should easily be able to parse hundreds or maybe thousands of 2K JSON
    messages a second. And JSON generation and parsing is at least as well
    supported and robust as XML, in most languages.

    > I would rather make a decision now, otherwise I will have a lot of
    > changes to make later on. Does anyone have any recommendations?


    I'd go with JSON, for simplicity and portability. If you have any
    specific questions about it, ask.

    regards,

    --
    alan kennedy
    ------------------------------------------------------
    email alan: http://xhaus.com/contact/alan
    Alan Kennedy, Jun 8, 2006
    #4
  5. Frank Millman

    Steve Holden Guest

    Ant wrote:
    >>to use? I could go back to XML, or I could switch to JSON - I have read

    >
    >
    > I'd favour JSON if the data structures are simple personally. XML is
    > comparatively speaking a pain to deal with, where with JSON you can
    > simply eval() the data and you have a Python dictionary at your
    > disposal.
    >

    Modulo any security problems that alert and malicious users are able to
    inject into your application. Simply using eval() uncritically on
    whatever comes down the pipe is a train wreck waiting to happen.

    > I recently used JSON as a way of passing data from a Java backend to a
    > web page for Javascript to deal with, with the added side effect that
    > my Python testing scripts could also easily read the same data.
    >


    Oh, well as ling as we're talking about the *web* that's all right, then :)

    regards
    Steve
    --
    Steve Holden +44 150 684 7255 +1 800 494 3119
    Holden Web LLC/Ltd http://www.holdenweb.com
    Love me, love my blog http://holdenweb.blogspot.com
    Recent Ramblings http://del.icio.us/steve.holden
    Steve Holden, Jun 8, 2006
    #5
  6. Frank Millman

    Alan Kennedy Guest

    [Ant]
    >> I'd favour JSON if the data structures are simple personally. XML is
    >> comparatively speaking a pain to deal with, where with JSON you can
    >> simply eval() the data and you have a Python dictionary at your
    >> disposal.


    [Steve]
    > Modulo any security problems that alert and malicious users are able to
    > inject into your application. Simply using eval() uncritically on
    > whatever comes down the pipe is a train wreck waiting to happen.


    Yes, evaling JSON, or any other text coming from the web, is definitely
    a bad idea.

    But there's no need for eval: there are safe JSON codecs for python,

    http://cheeseshop.python.org/pypi?:action=search&description=json

    And one for javascript,

    http://www.json.org/js.html
    http://www.json.org/json.js

    And most other languages you're likely to come across.

    http://www.json.org/

    regards,

    --
    alan kennedy
    ------------------------------------------------------
    email alan: http://xhaus.com/contact/alan
    Alan Kennedy, Jun 8, 2006
    #6
  7. Frank Millman

    Ant Guest


    > Yes, evaling JSON, or any other text coming from the web, is definitely
    > a bad idea.
    >
    > But there's no need for eval: there are safe JSON codecs for python,


    Fair enough. And I should imagine that the codecs are still much faster
    and easier to use than XML for the same purpose.

    For my purposes, the JSON is pushed out to the web from our Java
    web-app, and eval'd in the test scripts which screen scrape the JSON
    structure from the web page - no danger in this case for me. But yes -
    I wouldn't be eval'ing random 'JSON' code from the web :)
    Ant, Jun 8, 2006
    #7
  8. Frank Millman wrote:
    > Hi all
    >
    > I am writing a multi-user accounting/business application, which uses
    > sockets to communicate between server and client. The server contains
    > all the business logic. It has no direct knowledge of the client. I
    > have devised a simple message format to exchange information between
    > the two.
    >

    [...]
    >
    > I would rather make a decision now, otherwise I will have a lot of
    > changes to make later on. Does anyone have any recommendations?
    >
    > Thanks
    >
    > Frank Millman


    Thanks for the replies, guys.

    Albert, thank you for the pointer to XMPP. I had not heard of it, but
    it looks very interesting. I will have to investigate further before
    deciding whether it is right for my application. If not, JSON it is.

    Frank
    Frank Millman, Jun 9, 2006
    #8
  9. Alan Kennedy wrote:
    > [Frank Millman]
    > > I am writing a multi-user accounting/business application, which uses
    > > sockets to communicate between server and client. The server contains
    > > all the business logic. It has no direct knowledge of the client. I
    > > have devised a simple message format to exchange information between
    > > the two.
    > >

    >
    > If you're going to mix javascript client and python server, you
    > definitely need something cross platform, like XML or JSON.
    >

    [...]
    >
    > I'd go with JSON, for simplicity and portability. If you have any
    > specific questions about it, ask.
    >


    Thanks for the offer. I have just tried it out, and instantly bumped my
    head. I think I know the problem, and I don't think there is a simple
    answer, but I will ask here first before starting to figure out a way
    around it.

    My client-server is Python-to-Python. At present, I am using cPickle to
    transfer objects between the two. Among other things, I sometimes
    transfer a tuple. Using JSON it appears on the other side as a list. As
    I sometimes use the tuple as a dictionary key, this fails, as you
    obviously cannot use a list as a key.

    I am using 'simplejson'. I see there is also something called json-py,
    but I have not tried it yet, as I am assuming (maybe wrongly) that it
    will have the same problem. My hunch is that javascript/JSON does not
    have the concept of a tuple, and therefore the problem is inherent.

    Can someone confirm this, or is there an easy workaround?

    Thanks

    Frank
    Frank Millman, Jun 16, 2006
    #9
  10. > My client-server is Python-to-Python. At present, I am using cPickle to
    > transfer objects between the two. Among other things, I sometimes
    > transfer a tuple. Using JSON it appears on the other side as a list. As
    > I sometimes use the tuple as a dictionary key, this fails, as you
    > obviously cannot use a list as a key.

    [...]
    > Can someone confirm this, or is there an easy workaround?


    You can always convert a list to a tuple using the tuple () builtin
    right before you use it as a key. But you have to be sure that it is a
    list. tuple ("abc") => ('a', 'b', 'c') is probably not what you intend
    to do.

    Daniel
    Daniel Dittmar, Jun 16, 2006
    #10
  11. Daniel Dittmar wrote:
    > > My client-server is Python-to-Python. At present, I am using cPickle to
    > > transfer objects between the two. Among other things, I sometimes
    > > transfer a tuple. Using JSON it appears on the other side as a list. As
    > > I sometimes use the tuple as a dictionary key, this fails, as you
    > > obviously cannot use a list as a key.

    > [...]
    > > Can someone confirm this, or is there an easy workaround?

    >
    > You can always convert a list to a tuple using the tuple () builtin
    > right before you use it as a key. But you have to be sure that it is a
    > list. tuple ("abc") => ('a', 'b', 'c') is probably not what you intend
    > to do.
    >
    > Daniel


    Thanks,Daniel. After sleeping on it, I had also come to the conclusion
    that this is the easiest solution -
    if isinstance(key,list):
    key = tuple(key)

    A bit of a kludge, but I can live with it. I will keep checking to see
    if any other anomalies crop up.

    Frank
    Frank Millman, Jun 17, 2006
    #11
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Florian Frank
    Replies:
    0
    Views:
    227
    Florian Frank
    Jun 30, 2009
  2. sajuptpm
    Replies:
    2
    Views:
    316
    sajuptpm
    Dec 28, 2012
  3. Acácio Centeno
    Replies:
    1
    Views:
    241
    dieter
    Feb 15, 2013
  4. Bryan Britten
    Replies:
    9
    Views:
    256
    Bryan Britten
    May 28, 2013
  5. David Karr
    Replies:
    1
    Views:
    153
    David Karr
    Jun 17, 2013
Loading...

Share This Page