Strategy to Verify Python Program is POST'ing to a web server.

Discussion in 'Python' started by mzagursk@gmail.com, Jun 18, 2011.

  1. Guest

    Hello Folks,

    I am wondering what your strategies are for ensuring that data
    transmitted to a website via a python program is indeed from that
    program, and not from someone submitting POST data using some other
    means. I find it likely that there is no solution, in which case what
    is the best solution for sending data to a remote server from a python
    program and ensuring that it is from that program?

    For example, if I create a website that tracks some sort of
    statistical information and don't ensure that my program is the one
    that is uploading it, the statistics can be thrown off by people
    entering false POST data onto the data upload page. Any remedy?

    Thanks
     
    , Jun 18, 2011
    #1
    1. Advertising

  2. Eden Kirin Guest

    On 18.06.2011 13:34, wrote:
    > Hello Folks,
    >
    > I am wondering what your strategies are for ensuring that data
    > transmitted to a website via a python program is indeed from that
    > program, and not from someone submitting POST data using some other
    > means. I find it likely that there is no solution, in which case what
    > is the best solution for sending data to a remote server from a python
    > program and ensuring that it is from that program?
    >
    > For example, if I create a website that tracks some sort of
    > statistical information and don't ensure that my program is the one
    > that is uploading it, the statistics can be thrown off by people
    > entering false POST data onto the data upload page. Any remedy?


    Include some hash check in hidden field.

    For example, from your python program you will include hidden fields
    random_number and hash:

    import random, hashlib
    my_secret_key = "MySecretKey"
    random_number = "%f" % random.random()
    hash = hashlib.sha1("%s %s" % (my_secret_key, random_number)).hexdigest()

    On the server side check hash with random_number and secret key to
    ensure the data is POSTed from your application.

    --
    www.vikendi.com -/- www.svimi.net
     
    Eden Kirin, Jun 18, 2011
    #2
    1. Advertising

  3. Authentication by client SSL certificate is best.

    You should also look into restricting access on the server side by IP address.

    Michael

    On Sat, Jun 18, 2011 at 7:34 AM, <> wrote:
    > Hello Folks,
    >
    > I am wondering what your strategies are for ensuring that data
    > transmitted to a website via a python program is indeed from that
    > program, and not from someone submitting POST data using some other
    > means.  I find it likely that there is no solution, in which case what
    > is the best solution for sending data to a remote server from a python
    > program and ensuring that it is from that program?
    >
    > For example, if I create a website that tracks some sort of
    > statistical information and don't ensure that my program is the one
    > that is uploading it, the statistics can be thrown off by people
    > entering false POST data onto the data upload page.  Any remedy?
    >
    > Thanks
    > --
    > http://mail.python.org/mailman/listinfo/python-list
    >
     
    Michael Hrivnak, Jun 18, 2011
    #3
  4. On Sat, Jun 18, 2011 at 9:34 PM, <> wrote:
    > I am wondering what your strategies are for ensuring that data
    > transmitted to a website via a python program is indeed from that
    > program, and not from someone submitting POST data using some other
    > means.  I find it likely that there is no solution, in which case what
    > is the best solution for sending data to a remote server from a python
    > program and ensuring that it is from that program?


    You're correct there: there is no solution. Everything on the other
    side of your network cable should be treated as hostile and spoofed.
    But the real question is, how much effort are people likely to go to
    to avoid using your program?

    SSL certificates are good, but they can be stolen (very easily if the
    client is open source). Anything algorithmic suffers from the same
    issue.

    In the example you gave, there's no solution. Someone could easily
    spoof it and stuff the ballot. But if you make that more difficult
    than the survey is worth, then you can largely trust your data.

    The other common reason for wanting to be sure that the far end really
    is your script is when you're trusting the client to do data
    validation. There's a solution to that one: repeat the validation on
    the server, and then it doesn't matter if they use your program or
    not. (And before you cry "Isn't that obvious?", a lot of people have
    completely missed that point.) In neither case can you prove what
    program was on the far end. You're working with network packets, so
    anything can be spoofed. You could go a long way toward it, though, by
    using something ridiculously complex, such as:

    * Client connects via SSL to host, using a known certificate.
    * Server verifies certificate, and sends client some Python code to execute..
    * Client verifies the server's certificate (vital!).
    * Client executes the code it's given, and based on the result, plus
    some other data, sends the server a hash value.
    * Server executes the same code it gave the client, knows the data it
    was working with, and calculates the equivalent hash.
    * If the two hashes match, the client is deemed to be valid.

    This is a variant of the usual nonce-based hashing systems, where the
    nonce in question is actually executable code. By randomizing the
    code, you can make it difficult for any non-Python program to
    duplicate the hash algorithm. But it still won't provide certainty, by
    any means.

    I've spent quite a bit of time this past fortnight explaining some of
    these concepts to my boss and one of my coworkers; they were building
    a rather elaborate system but didn't realise that, apart from
    requiring about three times as much data from /dev/random, it wasn't
    materially different from a simple SSL cert check...

    Chris Angelico
     
    Chris Angelico, Jun 18, 2011
    #4
  5. On Sat, Jun 18, 2011 at 1:26 PM, Chris Angelico <> wrote:
    > SSL certificates are good, but they can be stolen (very easily if the
    > client is open source). Anything algorithmic suffers from the same
    > issue.


    This is only true if you distribute your app with one built-in
    certificate, which does indeed seem like a bad idea. When you know
    your user base though, especially if this is a situation with a small
    number of deployments, than you can distribute a unique certificate to
    each client, signed by your CA. Not knowing what kind of statistics
    the OP is trying to collect, we really don't know if this client will
    be running in one place or thousands.

    Even if there will be thousands of deployments, you could generate an
    RSA key-pair on the client similar to how an ssh client does, and use
    that to sign the data. Then you can at least track which client each
    submission came from (storing the public key and IP address), and then
    remove submissions as necessary if you detect abuse.

    > In the example you gave, there's no solution. Someone could easily
    > spoof it and stuff the ballot. But if you make that more difficult
    > than the survey is worth, then you can largely trust your data.
    >
    > You could go a long way toward it, though, by
    > using something ridiculously complex, such as:
    >
    > * Client connects via SSL to host, using a known certificate.
    > * Server verifies certificate, and sends client some Python code to execute.
    > * Client verifies the server's certificate (vital!).
    > * Client executes the code it's given, and based on the result, plus
    > some other data, sends the server a hash value.
    > * Server executes the same code it gave the client, knows the data it
    > was working with, and calculates the equivalent hash.
    > * If the two hashes match, the client is deemed to be valid.


    An authentication process that involves the client executing code
    supplied by the server opens up one single point of failure (server is
    compromised or man-in-the-middle attack is happening) by which
    arbitrary code could get executed on the client. Yikes! It's ok to
    execute server-supplied code in a sandbox (i.e. javascript), but I
    would never want to use software that sends me code over the network
    to be executed directly on my system (unless that's the express
    purpose of the software, like celery). Besides, it seems that all
    you've accomplished is verifying that the client can execute python
    code and you've made it a bit less convenient to attack.

    The TLS handshake really does verify that the client has a certificate
    which has been previously signed by the CA. If you can get signed
    certs to each deployment, that is spectacular security that will serve
    you well. The above sounds a bit like you're trying to create a new
    cipher based on exchanged code that gets executed. I encourage you to
    not reinvent the wheel, and stick with the ciphers that are already
    standard in the SSL/TLS handshake.

    If you cannot uniquely authenticate each client (either through a
    signed cert or by having the user supply credentials interactively),
    then you'll have to accept that you cannot trust the submitted data
    100%, and just take measures to mitigate abuse.

    Michael
     
    Michael Hrivnak, Jun 18, 2011
    #5
  6. Paul Rubin Guest

    "" <> writes:
    > For example, if I create a website that tracks some sort of
    > statistical information and don't ensure that my program is the one
    > that is uploading it, the statistics can be thrown off by people
    > entering false POST data onto the data upload page. Any remedy?


    If you're concerned about unauthorized users posting random crap, the
    obvious solution is configure your web server to put password protection
    on the page.

    If you're saying AUTHORIZED users (those allowed to use the program to
    post stuff) aren't trusted to not bypass the program, you've basically
    got a DRM problem, especially if you think the users might
    reverse-engineer the program to figure out the protocol. The most
    effective approaches generally involve delivering the program in the
    form of a hardware product that's difficult to tamper with. That's what
    cable TV boxes amount to, for example.

    What is the application, if you can say? That might help get better
    answers.
     
    Paul Rubin, Jun 18, 2011
    #6
  7. Terry Reedy Guest

    On 6/18/2011 7:34 AM, wrote:
    > Hello Folks,
    >
    > I am wondering what your strategies are for ensuring that data
    > transmitted to a website via a python program is indeed from that
    > program, and not from someone submitting POST data using some other
    > means. I find it likely that there is no solution, in which case what
    > is the best solution for sending data to a remote server from a python
    > program and ensuring that it is from that program?
    >
    > For example, if I create a website that tracks some sort of
    > statistical information and don't ensure that my program is the one
    > that is uploading it, the statistics can be thrown off by people
    > entering false POST data onto the data upload page. Any remedy?


    You have not specified all the parameters of the problem. Are there a
    limited number of copies of your program or are they distrubuted freely?
    What about multiple votes from one program?

    Corporate proxy votes (which are a legally important type of statistical
    information) work as follows. Each shareholder is mailed or emailed a
    'control number'. Attend stockholder meeting in person, mail proxy vote,
    or login with any browser with control number. Repeat votes by the same
    control id supercede previous vote. There should be a 'thank you for
    voting' response for each vote. I suspect IP addr. is recorded with vote
    too. I have not heard of specific problems with electronic proxy voting.

    --
    Terry Jan Reedy
     
    Terry Reedy, Jun 18, 2011
    #7
  8. On Sun, Jun 19, 2011 at 6:40 AM, Michael Hrivnak <> wrote:
    > On Sat, Jun 18, 2011 at 1:26 PM, Chris Angelico <> wrote:
    >> SSL certificates are good, but they can be stolen (very easily if the
    >> client is open source). Anything algorithmic suffers from the same
    >> issue.

    >
    > This is only true if you distribute your app with one built-in
    > certificate, which does indeed seem like a bad idea.  When you know
    > your user base though, especially if this is a situation with a small
    > number of deployments, than you can distribute a unique certificate to
    > each client, signed by your CA.


    That changes it from verifying the program to verifying the user. It's
    a somewhat different beast, but it still leaves the possibility of
    snagging the cert and using it in another program. Same with IP
    address checks. You can't prove that the other end is a particular
    program.

    >> You could go a long way toward it, though, by
    >> using something ridiculously complex, such as:
    >> ...

    >
    > An authentication process that involves the client executing code
    > supplied by the server opens up one single point of failure (server is
    > compromised or man-in-the-middle attack is happening) by which
    > arbitrary code could get executed on the client.  Yikes!


    Yeah, hence the part of verifying the server's cert too. That one is a
    bit safer though; nobody but you will have that certificate, so it's
    not as easy to take and put into another program. But this whole
    scheme was meant from the start to be ridiculous.

    > If ...
    > then you'll have to accept that you cannot trust the submitted data
    > 100%, and just take measures to mitigate abuse.


    I still stand by my original point, namely that the "if" on here is
    superfluous, and the "then" is unconditional. But the measures you
    describe _do_ reduce the likelihood significantly.

    ChrisA
     
    Chris Angelico, Jun 19, 2011
    #8
  9. Michael Hrivnak wrote:
    > Besides, it seems that all
    > you've accomplished is verifying that the client can execute python
    > code and you've made it a bit less convenient to attack.


    And that only if the attacker isn't a Python programmer.
    If he is, he's probably writing his attack program in
    Python anyway. :)

    Although if you were devious, and you detected that such
    an attack was in progress, you could lull him into a sense
    of security and then send him some Python code to pwn his
    machine...

    --
    Greg
     
    Gregory Ewing, Jun 19, 2011
    #9
  10. On Sun, Jun 19, 2011 at 10:38 AM, Gregory Ewing
    <> wrote:
    > And that only if the attacker isn't a Python programmer.
    > If he is, he's probably writing his attack program in
    > Python anyway. :)
    >


    I was thinking you'd have it call on various functions defined
    elsewhere in the program, forcing him to pretty much have the whole
    original code in there. :)

    ChrisA
     
    Chris Angelico, Jun 19, 2011
    #10
  11. Nobody Guest

    On Sat, 18 Jun 2011 04:34:55 -0700, wrote:

    > I am wondering what your strategies are for ensuring that data
    > transmitted to a website via a python program is indeed from that
    > program, and not from someone submitting POST data using some other
    > means.


    > Any remedy?


    Supply the client with tamper-proof hardware containing a private key.

    Either that, or just accept that it cannot be done. Compare the amount of
    effort game developers put into trying to implement tamper-proofing in
    software with how little success they've had.
     
    Nobody, Jun 19, 2011
    #11
  12. On Sun, 19 Jun 2011 05:47:30 +0100, Nobody wrote:

    > On Sat, 18 Jun 2011 04:34:55 -0700, wrote:
    >
    >> I am wondering what your strategies are for ensuring that data
    >> transmitted to a website via a python program is indeed from that
    >> program, and not from someone submitting POST data using some other
    >> means.

    >
    >> Any remedy?

    >
    > Supply the client with tamper-proof hardware containing a private key.


    Is that resistant to man-in-the-middle attacks by somebody with a packet
    sniffer watching the traffic between the device and the website?

    > Either that, or just accept that it cannot be done. Compare the amount
    > of effort game developers put into trying to implement tamper-proofing
    > in software with how little success they've had.


    Exactly.



    --
    Steven
     
    Steven D'Aprano, Jun 19, 2011
    #12
  13. Paul Rubin Guest

    Steven D'Aprano <> writes:
    >> Supply the client with tamper-proof hardware containing a private key.

    >
    > Is that resistant to man-in-the-middle attacks by somebody with a packet
    > sniffer watching the traffic between the device and the website?


    Sure, why not? As long as the crypto is done properly, that is.

    But, there is also the matter of securing the path from the data to the
    hardware. I don't have the impression that the OP has really thought
    this through.
     
    Paul Rubin, Jun 19, 2011
    #13
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Jonathan

    POST-ing data to Http server

    Jonathan, Jan 16, 2004, in forum: Java
    Replies:
    6
    Views:
    18,866
    Andrew Thompson
    Jan 31, 2004
  2. Corey Ellsworth

    "POST"ing Image Data to ASPX File?

    Corey Ellsworth, Jun 12, 2006, in forum: ASP .Net
    Replies:
    2
    Views:
    718
    Corey Ellsworth
    Jun 12, 2006
  3. Dan Greenblatt

    'post'ing Tkinter menus on windows.....

    Dan Greenblatt, Aug 13, 2004, in forum: Python
    Replies:
    0
    Views:
    269
    Dan Greenblatt
    Aug 13, 2004
  4. Chris

    question about webservices and "post"ing data

    Chris, Dec 5, 2005, in forum: ASP .Net Web Services
    Replies:
    1
    Views:
    109
  5. Bob

    POST'ing XML from VB ASP to Perl

    Bob, Dec 2, 2004, in forum: Perl Misc
    Replies:
    1
    Views:
    136
    Sherm Pendley
    Dec 2, 2004
Loading...

Share This Page