why encodeURIComponent() for data in body of POST XHR request?

Discussion in 'Javascript' started by Peter Michaux, Aug 2, 2007.

  1. Hi,

    All Ajax libraries I've read use encodeURIComponent() on the name-
    value pairs extracted from forms before POST ing the result to the
    server with and xmlhttprequest. I can understand why this encoding is
    required in the case of a GET request and the form data is attached as
    a URI query string; however, why is the encoding necessary for POST
    requests?

    Thanks,
    Peter
    Peter Michaux, Aug 2, 2007
    #1
    1. Advertising

  2. Peter Michaux

    David Mark Guest

    On Aug 2, 12:43 am, Peter Michaux <> wrote:
    > Hi,
    >
    > All Ajax libraries I've read use encodeURIComponent() on the name-
    > value pairs extracted from forms before POST ing the result to the
    > server with and xmlhttprequest. I can understand why this encoding is
    > required in the case of a GET request and the form data is attached as
    > a URI query string; however, why is the encoding necessary for POST
    > requests?
    >
    > Thanks,
    > Peter


    URL encoding is simply the standard for name-value pairs in an HTTP
    request. GET or POST doesn't enter into it (the pairs look exactly
    the same in a POST request as they do when tacked onto the query
    string for a GET.)

    For example, imagine what would happen if you didn't encode the pairs
    and one of the values had an ampersand in it.
    David Mark, Aug 2, 2007
    #2
    1. Advertising

  3. Peter Michaux

    Evertjan. Guest

    David Mark wrote on 02 aug 2007 in comp.lang.javascript:

    > On Aug 2, 12:43 am, Peter Michaux <> wrote:
    >>
    >> All Ajax libraries I've read use encodeURIComponent() on the name-
    >> value pairs extracted from forms before POST ing the result to the
    >> server with and xmlhttprequest. I can understand why this encoding is
    >> required in the case of a GET request and the form data is attached as
    >> a URI query string; however, why is the encoding necessary for POST
    >> requests?

    >
    > URL encoding is simply the standard for name-value pairs in an HTTP
    > request. GET or POST doesn't enter into it (the pairs look exactly
    > the same in a POST request as they do when tacked onto the query
    > string for a GET.)
    > For example, imagine what would happen if you didn't encode the pairs
    > and one of the values had an ampersand in it.


    When using POST, nothing would, I would imagine.


    --
    Evertjan.
    The Netherlands.
    (Please change the x'es to dots in my emailaddress)
    Evertjan., Aug 2, 2007
    #3
  4. Peter Michaux

    David Mark Guest

    On Aug 2, 3:18 am, "Evertjan." <> wrote:
    > David Mark wrote on 02 aug 2007 in comp.lang.javascript:
    >
    > > On Aug 2, 12:43 am, Peter Michaux <> wrote:

    >
    > >> All Ajax libraries I've read use encodeURIComponent() on the name-
    > >> value pairs extracted from forms before POST ing the result to the
    > >> server with and xmlhttprequest. I can understand why this encoding is
    > >> required in the case of a GET request and the form data is attached as
    > >> a URI query string; however, why is the encoding necessary for POST
    > >> requests?

    >
    > > URL encoding is simply the standard for name-value pairs in an HTTP
    > > request. GET or POST doesn't enter into it (the pairs look exactly
    > > the same in a POST request as they do when tacked onto the query
    > > string for a GET.)
    > > For example, imagine what would happen if you didn't encode the pairs
    > > and one of the values had an ampersand in it.

    >
    > When using POST, nothing would, I would imagine.
    >


    What does that mean? You can certainly POST form data with ampersands
    and they are normally encoded the same way as GET requests.
    David Mark, Aug 2, 2007
    #4
  5. Peter Michaux

    Evertjan. Guest

    David Mark wrote on 02 aug 2007 in comp.lang.javascript:

    > On Aug 2, 3:18 am, "Evertjan." <> wrote:
    >> David Mark wrote on 02 aug 2007 in comp.lang.javascript:
    >>
    >> > On Aug 2, 12:43 am, Peter Michaux <> wrote:

    >>
    >> >> All Ajax libraries I've read use encodeURIComponent() on the name-
    >> >> value pairs extracted from forms before POST ing the result to the
    >> >> server with and xmlhttprequest. I can understand why this encoding
    >> >> is required in the case of a GET request and the form data is
    >> >> attached as a URI query string; however, why is the encoding
    >> >> necessary for POST requests?

    >>
    >> > URL encoding is simply the standard for name-value pairs in an HTTP
    >> > request. GET or POST doesn't enter into it (the pairs look exactly
    >> > the same in a POST request as they do when tacked onto the query
    >> > string for a GET.)
    >> > For example, imagine what would happen if you didn't encode the
    >> > pairs and one of the values had an ampersand in it.

    >>
    >> When using POST, nothing would, I would imagine.


    > What does that mean?


    Eh?
    Nothing extaordinary would happen.

    > You can certainly POST form data with ampersands
    > and they are normally encoded the same way as GET requests.


    With a simple querystring encoding is not automatic.

    --
    Evertjan.
    The Netherlands.
    (Please change the x'es to dots in my emailaddress)
    Evertjan., Aug 2, 2007
    #5
  6. Peter Michaux wrote:

    > All Ajax libraries I've read use encodeURIComponent() on the name-
    > value pairs extracted from forms before POST ing the result to the
    > server with and xmlhttprequest. I can understand why this encoding is
    > required in the case of a GET request and the form data is attached as
    > a URI query string; however, why is the encoding necessary for POST
    > requests?


    Because POST uses the same characters to split ('&' and '=') by
    default. The method doesn't matter; the data is just offered in
    another part of the request.

    GET looks like this (header):

    GET /file.html?name=Bart&nr=4 HTTP/1.0

    while default POST looks like this (body):

    POST /file.html HTTP/1.0
    Content-length: 15

    name=Bart&nr=4

    It's like in email conventions, where stuff can be stored in header or
    in body.

    You can see this mechanism very well in Perl's CGI processing, where
    GET reads the input from the URL (a line in the header of the
    request), while POST takes it from <STDIN> (body of request). The
    further processing is identical; and the URL decoding is an important
    step here.

    I've seen the term "un-webify" for this kind of processing rather than
    "URL-decoding"; the latter indeed sounds like it refers to GET only.

    But the encoding of POST-ed data is only the default behaviour of the
    browser (which is done in Ajax "by hand" in such libraries).

    <form method="post">

    actually means:

    <form method="post"
    enctype="application/x-www-form-urlencoded">

    But it's possible to disable this URL-encoding for POST-ed data,
    mostly to transfer (binary) files to the gateway software. In the
    following example, you tell the form not to encode anything:

    <form method="post" enctype="multipart/form-data">

    This is the only case where 'é' will be passed as 'é'; under default
    GET/POST rules it will always be passed as '%E9'.

    Again, this is identical to email; all mail attachments are sent using
    multipart (but, unlike HTTP, they are additionally base64-encoded).

    So as a general conclusion: Ajax libraries must invoke
    encodeURIComponent() when sending POST-requests in the "application/x-
    www-form-urlencoded" encoding type. They must always invoke
    encodeURIComponent() for GET. They must not invoke
    encodeURIComponent() in the POST "multipart/form-data" type.

    Obviously, it also depends on how the gateway software is configured
    how to handle incoming data (this will be URL decoded by default).

    Hope this helps,

    --
    Bart
    Bart Van der Donck, Aug 2, 2007
    #6
  7. "Evertjan." wrote:

    David Mark wrote:

    >> You can certainly POST form data with ampersands
    >> and they are normally encoded the same way as GET requests.

    >
    > With a simple querystring encoding is not automatic.


    URL-encoding is 'automatic' in browsers as long as you don't override
    this setting. POST or GET doesn't matter; browser will URL-encode all
    form values by default before sending them to server.

    If you don't want this to happen, you should use

    <form method="post" enctype="multipart/form-data">

    to receive non-encoded data at server.

    --
    Bart
    Bart Van der Donck, Aug 2, 2007
    #7
  8. On Aug 1, 10:19 pm, David Mark <> wrote:
    > On Aug 2, 12:43 am, Peter Michaux <> wrote:
    >
    > > Hi,

    >
    > > All Ajax libraries I've read use encodeURIComponent() on the name-
    > > value pairs extracted from forms before POST ing the result to the
    > > server with and xmlhttprequest. I can understand why this encoding is
    > > required in the case of a GET request and the form data is attached as
    > > a URI query string; however, why is the encoding necessary for POST
    > > requests?

    >
    > > Thanks,
    > > Peter

    >
    > URL encoding is simply the standard for name-value pairs in an HTTP
    > request. GET or POST doesn't enter into it (the pairs look exactly
    > the same in a POST request as they do when tacked onto the query
    > string for a GET.)
    >
    > For example, imagine what would happen if you didn't encode the pairs
    > and one of the values had an ampersand in it.


    Thanks, David. Now it is obvious using '&' as a separator implies the
    keys and values can't have '&'. I've let myself be tripped up by
    thinking these requests are more magical than just text transfer
    before.

    Peter
    Peter Michaux, Aug 2, 2007
    #8
  9. On Aug 2, 2:39 am, Bart Van der Donck <> wrote:
    > Peter Michaux wrote:
    > > All Ajax libraries I've read use encodeURIComponent() on the name-
    > > value pairs extracted from forms before POST ing the result to the
    > > server with and xmlhttprequest. I can understand why this encoding is
    > > required in the case of a GET request and the form data is attached as
    > > a URI query string; however, why is the encoding necessary for POST
    > > requests?

    >
    > Because POST uses the same characters to split ('&' and '=') by
    > default. The method doesn't matter; the data is just offered in
    > another part of the request.
    >
    > GET looks like this (header):
    >
    > GET /file.html?name=Bart&nr=4 HTTP/1.0
    >
    > while default POST looks like this (body):
    >
    > POST /file.html HTTP/1.0
    > Content-length: 15
    >
    > name=Bart&nr=4
    >
    > It's like in email conventions, where stuff can be stored in header or
    > in body.
    >
    > You can see this mechanism very well in Perl's CGI processing, where
    > GET reads the input from the URL (a line in the header of the
    > request), while POST takes it from <STDIN> (body of request). The
    > further processing is identical; and the URL decoding is an important
    > step here.
    >
    > I've seen the term "un-webify" for this kind of processing rather than
    > "URL-decoding"; the latter indeed sounds like it refers to GET only.
    >
    > But the encoding of POST-ed data is only the default behaviour of the
    > browser (which is done in Ajax "by hand" in such libraries).
    >
    > <form method="post">
    >
    > actually means:
    >
    > <form method="post"
    > enctype="application/x-www-form-urlencoded">
    >
    > But it's possible to disable this URL-encoding for POST-ed data,
    > mostly to transfer (binary) files to the gateway software. In the
    > following example, you tell the form not to encode anything:
    >
    > <form method="post" enctype="multipart/form-data">
    >
    > This is the only case where 'é' will be passed as 'é'; under default
    > GET/POST rules it will always be passed as '%E9'.
    >
    > Again, this is identical to email; all mail attachments are sent using
    > multipart (but, unlike HTTP, they are additionally base64-encoded).
    >
    > So as a general conclusion: Ajax libraries must invoke
    > encodeURIComponent() when sending POST-requests in the "application/x-
    > www-form-urlencoded" encoding type. They must always invoke
    > encodeURIComponent() for GET. They must not invoke
    > encodeURIComponent() in the POST "multipart/form-data" type.
    >
    > Obviously, it also depends on how the gateway software is configured
    > how to handle incoming data (this will be URL decoded by default).
    >
    > Hope this helps,


    It helps a lot. Thanks.

    Peter
    Peter Michaux, Aug 2, 2007
    #9
  10. Peter Michaux

    Evertjan. Guest

    Bart Van der Donck wrote on 02 aug 2007 in comp.lang.javascript:

    > "Evertjan." wrote:
    >
    > David Mark wrote:
    >
    >>> You can certainly POST form data with ampersands
    >>> and they are normally encoded the same way as GET requests.

    >>
    >> With a simple querystring encoding is not automatic.

    >
    > URL-encoding is 'automatic' in browsers as long as you don't override
    > this setting. POST or GET doesn't matter; browser will URL-encode all
    > form values by default before sending them to server.
    >
    > If you don't want this to happen, you should use
    >
    > <form method="post" enctype="multipart/form-data">
    >
    > to receive non-encoded data at server.


    I wrote:
    >> With a simple querystring encoding is not automatic.


    var q = 'text=me&you'
    location.href = 'http://cnn.com/?' + q

    will be encoded?


    --
    Evertjan.
    The Netherlands.
    (Please change the x'es to dots in my emailaddress)
    Evertjan., Aug 2, 2007
    #10
  11. "Evertjan." wrote:

    > Bart Van der Donck wrote on 02 aug 2007 in comp.lang.javascript:


    >> URL-encoding is 'automatic' in browsers as long as you don't
    >> override this setting. POST or GET doesn't matter; browser will
    >> URL-encode all form values by default before sending them to
    >> server.
    >> If you don't want this to happen, you should use
    >> <form method="post" enctype="multipart/form-data">
    >> to receive non-encoded data at server.


    > var q = 'text=me&you'
    > location.href = 'http://cnn.com/?'+ q
    >
    > will be encoded?


    No, that's why Ajax libraries should use encodeURIComponent() here :)

    --
    Bart
    Bart Van der Donck, Aug 2, 2007
    #11
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. int80
    Replies:
    4
    Views:
    13,860
    randomguy
    Aug 25, 2008
  2. Mr. SweatyFinger
    Replies:
    2
    Views:
    1,850
    Smokey Grindel
    Dec 2, 2006
  3. Replies:
    0
    Views:
    7,787
  4. fochie
    Replies:
    5
    Views:
    142
    Thomas 'PointedEars' Lahn
    Jun 17, 2005
  5. Une Bévue

    XHR and responseXML == null why ?

    Une Bévue, Apr 26, 2008, in forum: Javascript
    Replies:
    8
    Views:
    153
    Thomas 'PointedEars' Lahn
    Apr 28, 2008
Loading...

Share This Page