Posting an XML document to a protected API

[Maruthy Mentireddi]

I am working on the FrontEnd of a website and need to make a RESTful
HTTP POST to a Backend system which checks for the presence of an API
KEY and a SESSSION KEY in the request URL as part of the security model.

Eg: To make a HTTP POST request, the URL should be of the format:
http://#{API_KEY}@#{BACKEND_HOST}:#{BACKEND_PORT}/#{PATH_TO_RESOURCE}?session_key=#{SESSION_KEY}

Now the problem I am facing is how do I specify the API_KEY in the URL
using the following code:

********************************************************************
server_addr = "http://#{API_KEY}@#{BACKEND_HOST_AND_PORT}"
api_path =
"#{BACKEND_HOST_PATH}/users/#{session[:user_id]}/contact_groups.xml?session_key=#{session[:session_key]}"

url = URI.parse(server_addr + api_path)
request =
Net::HTTP:ost.new(url.path+"?session_key=#{session[:session_key]}")

request.body = "<?xml version='1.0'
encoding='UTF-8'><contact_groups>SOME NESTED DATA GOES
HERE</contact_groups>"
response = Net::HTTP.start(url.host, url.port) {|http|
http.request(request)}
********************************************************************

When using the exact same code as above, the Backend responds back with
a HTTP 401 - !ruby/object:Net::HTTPUnauthorized error code.

And if I change the last line to include the API KEY as shown below:

response = Net::HTTP.start(url.user + "@" + url.host, url.port) {|http|
http.request(request)}

Ruby gives a"getaddrinfo: nodename nor servname provided, or not known"
error.

What is the right way to make this post to the Backend. Any response in
this regard will be greatly appreciated.

Thanks,

 

[7stud --]

The first thing I would do is try to get a response from the server
without all the variables in your request string. To begin with, you
need a valid API key and a valid session key. Then you can try
something like this:

require 'net/http'

Net::HTTP.start("(e-mail address removed):1200) do |http|
req =
Net::HTTP:ost.new("/users/John/contact_groups.xml?session_key=123456789")

resp = http.request(req).body
puts resp
end

 

[Engine Yard]

I already know both the API KEY and the SESSION KEY. Infact I have
everything needed to make the post, all that I need is to be able to
specify the "1234@" parameter as part of the URL:

Eg: The URL should be:
http://[email protected]:8080/books/palentology/common_groups.xml?session_key=a123j32nf4nfs23
where "asdfghlkji" is my API KEY and "a123j32nf4nfs23" is my SESSION
KEY.

Currently I can only create a URL of the form:
http://myserver.com:8080/books/palentology/common_groups.xml?session_key=a123j32nf4nfs23

Because Net::HTTP.start(url) will not allow me to input something like
Net::HTTP.start([email protected]:8080) or Net::HTTP.start([email protected],
8080), it gives a "getaddrinfo: nodename nor servname provided, or not
known" error when I attempt either of the above.

If I omit the "1234@" part and do something like
Net::HTTP.start(host.com, 8080), my backend server gives me a 401
Unauthorizes Access error.


When I implement your solution I still get the "getaddrinfo: nodename
nor servname provided, or not known" error because it looks like the
Net::HTTP.start method does not take the "1234@" parameter as it is
supposed to.

Is there any way to add the "1234@" parameter to the URL.

 

[7stud --]

The first thing I would do is try to get a response from the server
without all the variables in your request string. To begin with, you
need a valid API key and a valid session key. Then you can try
something like this:

I lied. That's not the first thing I would try. The first thing I
would try is posting your url into your browser's address bar to see if
you get a response.

 

[7stud --]

I lied. That's not the first thing I would try. The first thing I
would try is posting your url into your browser's address bar to see if
you get a response.

If that didn't work, then I would start looking for the specs on valid
host names.

----
Restrictions on valid host names

Hostnames are composed of series of labels concatenated with dots, as
are all domain names[1]. For example, "en.wikipedia.org" is a hostname.
Each label must be between 1 and 63 characters long, and the entire
hostname has a maximum of 255 characters.

RFCs mandate that a hostname's labels may contain only the ASCII letters
'a' through 'z' (case-insensitive), the digits '0' through '9', and the
hyphen. Hostname labels cannot begin or end with a hyphen. No other
symbols, punctuation characters, or blank spaces are permitted.

 

[7stud --]

I lied. That's not the first thing I would try. The first thing I
would try is posting your url into your browser's address bar to see if
you get a response.

If that didn't work, then I would start looking for the specs on valid
host names.

----
Restrictions on valid host names

Hostnames are composed of series of labels concatenated with dots, as
are all domain names[1]. For example, "en.wikipedia.org" is a hostname.
Each label must be between 1 and 63 characters long, and the entire
hostname has a maximum of 255 characters.

RFCs mandate that a hostname's labels may contain only the ASCII letters
'a' through 'z' (case-insensitive), the digits '0' through '9', and the
hyphen. Hostname labels cannot begin or end with a hyphen. No other
symbols, punctuation characters, or blank spaces are permitted.
------

Then I would determine that this format is a mistake:

the URL should be of the format:

http://#{API_KEY}@#{BACKEND_HOST}:#{BACKEND_PORT}/#{PATH_TO_RESOURCE}?>session_key=#{SESSION_KEY}

and I would try the following instead:

http://#{BACKEND_HOST}:#{BACKEND_PORT}/#{API_KEY}@#{BACKEND_HOST}:#{BACKEND_PORT}#{PATH_TO_RESOURCE}?session_key=#{SESSION_KEY}

or some variation thereof.

 

[Engine Yard]

I actually have many HTTP GET API calls of the same format, they are all
based on ActiveResource models

Eg:
response = User.findone, :from =>
"#{BACKEND_HOST_PATH}/users/#{session[:user_id]}/get_preferences.xml",
arams => {:session_key => session[:session_key]})

and the User model defines self.site as:

self.site = "http://#{BACKEND_API_KEY}@#{BACKEND_HOST_AND_PORT}"

so I am pretty sure that the
http://APIKEY@DOMAIN_NAME:PORT_NUMBER/PATH_TO_RESOURCE.RESPONSE_FORMAT/?PARAM_NAME=PARAM_VALUE

works fine. The main reason I am not using this pattern to make the POST
call is: The xml content that is contained the POST message's body is a
little complex.

Is there anyother way I can make a POST to the API.

7stud -- wrote:
The first thing I would do is try to get a response from the server
without all the variables in your request string. To begin with, you
need a valid API key and a valid session key. Then you can try
something like this:


I lied. That's not the first thing I would try. The first thing I
would try is posting your url into your browser's address bar to see if
you get a response.

If that didn't work, then I would start looking for the specs on valid
host names.

----
Restrictions on valid host names

Hostnames are composed of series of labels concatenated with dots, as
are all domain names[1]. For example, "en.wikipedia.org" is a hostname.
Each label must be between 1 and 63 characters long, and the entire
hostname has a maximum of 255 characters.

RFCs mandate that a hostname's labels may contain only the ASCII letters
'a' through 'z' (case-insensitive), the digits '0' through '9', and the
hyphen. Hostname labels cannot begin or end with a hyphen. No other
symbols, punctuation characters, or blank spaces are permitted.
------

Then I would determine that this format is a mistake:

the URL should be of the format:

http://#{API_KEY}@#{BACKEND_HOST}:#{BACKEND_PORT}/#{PATH_TO_RESOURCE}?>session_key=#{SESSION_KEY}

and I would try the following instead:

http://#{BACKEND_HOST}:#{BACKEND_PORT}/#{API_KEY}@#{BACKEND_HOST}:#{BACKEND_PORT}#{PATH_TO_RESOURCE}?session_key=#{SESSION_KEY}

or some variation thereof.

 

[7stud --]

7stud -- wrote:
The first thing I would do is try to get a response from the server
without all the variables in your request string. To begin with, you
need a valid API key and a valid session key. Then you can try
something like this:


I lied. That's not the first thing I would try. The first thing I
would try is posting your url into your browser's address bar to see if
you get a response.

If that didn't work, then I would start looking for the specs on valid
host names.

----
Restrictions on valid host names

Hostnames are composed of series of labels concatenated with dots, as
are all domain names[1]. For example, "en.wikipedia.org" is a hostname.
Each label must be between 1 and 63 characters long, and the entire
hostname has a maximum of 255 characters.

RFCs mandate that a hostname's labels may contain only the ASCII letters
'a' through 'z' (case-insensitive), the digits '0' through '9', and the
hyphen. Hostname labels cannot begin or end with a hyphen. No other
symbols, punctuation characters, or blank spaces are permitted.
------

Then I would determine that this format is a mistake:

the URL should be of the format:

http://#{API_KEY}@#{BACKEND_HOST}:#{BACKEND_PORT}/#{PATH_TO_RESOURCE}?>session_key=#{SESSION_KEY}

and I would try the following instead:

http://#{BACKEND_HOST}:#{BACKEND_PORT}/#{API_KEY}@#{BACKEND_HOST}:#{BACKEND_PORT}#{PATH_TO_RESOURCE}?session_key=#{SESSION_KEY}

or some variation thereof.

Also, after looking around some more, I would try a variation based on
the following format:

-----------------
2.1. The main parts of URLs

A full BNF description of the URL syntax is given in Section 5.

In general, URLs are written as follows:

<scheme>:<scheme-specific-part>

3. Specific Schemes

The mapping for some existing standard and experimental protocols is
outlined in the BNF syntax definition. Notes on particular protocols
follow. The schemes covered are:

ftp File Transfer protocol
http Hypertext Transfer Protocol
gopher The Gopher protocol
mailto Electronic mail address
news USENET news
nntp USENET news using NNTP access
telnet Reference to interactive sessions
wais Wide Area Information Servers
file Host-specific file names
prospero Prospero Directory Service

Other schemes may be specified by future specifications. Section 4 of
this document describes how new schemes may be registered, and lists
some scheme names that are under development.


3.1. Common Internet Scheme Syntax

While the syntax for the rest of the URL may vary depending on the
particular scheme selected, URL schemes that involve the direct use
of an IP-based protocol to a specified host on the Internet use a
common syntax for the scheme-specific data:

//<user>:<password>@<host>:<port>/<url-path>

Some or all of the parts "<user>:<password>@", ":<password>",
":<port>", and "/<url-path>" may be excluded. The scheme specific
data start with a double slash "//" to indicate that it complies with
the common Internet scheme syntax. The different components obey the
following rules:

user
An optional user name. Some schemes (e.g., ftp) allow the
specification of a user name.

password
An optional password. If present, it follows the user
name separated from it by a colon.

The user name (and password), if present, are followed by a
commercial at-sign "@". Within the user and password field, any ":",
"@", or "/" must be encoded.




Berners-Lee, Masinter & McCahill [Page 5]

RFC 1738 Uniform Resource Locators (URL) December 1994


Note that an empty user name or password is different than no user
name or password; there is no way to specify a password without
specifying a user name. E.g., <URL:ftp://@host.com/> has an empty
user name and no password, <URL:ftp://host.com/> has no user name,
while <URL:ftp://foohost.com/> has a user name of "foo" and an
empty password.

host
The fully qualified domain name of a network host, or its IP
address as a set of four decimal digit groups separated by
".". Fully qualified domain names take the form as described
in Section 3.5 of RFC 1034 [13] and Section 2.1 of RFC 1123
[5]: a sequence of domain labels separated by ".", each domain
label starting and ending with an alphanumerical character and
possibly also containing "-" characters. The rightmost domain
label will never start with a digit, though, which
syntactically distinguishes all domain names from the IP
addresses.

port
The port number to connect to. Most schemes designate
protocols that have a default port number. Another port number
may optionally be supplied, in decimal, separated from the
host by a colon. If the port is omitted, the colon is as well.

url-path
The rest of the locator consists of data specific to the
scheme, and is known as the "url-path". It supplies the
details of how the specified resource can be accessed. Note
that the "/" between the host (or port) and the url-path is
NOT part of the url-path.

The url-path syntax depends on the scheme being used, as does the
manner in which it is interpreted.

 

[pharrington]

I am working on the FrontEnd of a website and need to make a RESTful
HTTP POST to a Backend system which checks for the presence of an API
KEY and a SESSSION KEY in the request URL as part of the security model.

Eg: To make a HTTP POST request, the URL should be of the format:
http://#{API_KEY}@#{BACKEND_HOST}:#{BACKEND_PORT}/#{PATH_TO_RESOURCE}?session_key=#{SESSION_KEY}

Now the problem I am facing is how do I specify the API_KEY in the URL
using the following code:

********************************************************************
server_addr = "http://#{API_KEY}@#{BACKEND_HOST_AND_PORT}"
api_path =
"#{BACKEND_HOST_PATH}/users/#{session[:user_id]}/contact_groups.xml?session_key=#{session[:session_key]}"

url = URI.parse(server_addr + api_path)
request =
Net::HTTP:ost.new(url.path+"?session_key=#{session[:session_key]}")

request.body = "<?xml version='1.0'
encoding='UTF-8'><contact_groups>SOME NESTED DATA GOES
HERE</contact_groups>"
response = Net::HTTP.start(url.host, url.port) {|http|
http.request(request)}
********************************************************************

When using the exact same code as above, the Backend responds back with
a HTTP 401 - !ruby/object:Net::HTTPUnauthorized error code.

And if I change the last line to include the API KEY as shown below:

response = Net::HTTP.start(url.user + "@" + url.host, url.port) {|http|
http.request(request)}

Ruby gives a"getaddrinfo: nodename nor servname provided, or not known"
error.

What is the right way to make this post to the Backend. Any response in
this regard will be greatly appreciated.

Thanks,

Since there doesn't seem to be alot of ri documentation about
Net::HTTP:ost and I'm too lazy to look at the source,
Net::HTTP.start looks like its invoked like this:

-------------------------------------------------------
Net::HTTP::start
Net::HTTP::start(address, port = nil, p_addr = nil, p_port = nil,
p_user = nil, p_pass = nil) {|+http+| ...}
------------------------------------------------------------------------
creates a new Net::HTTP object and opens its TCP connection and
HTTP session. If the optional block is given, the newly created
Net::HTTP object is passed to it and closed when the block
finishes. In this case, the return value of this method is the
return value of the block. If no block is given, the return value
of this method is the newly created Net::HTTP object itself, and
the caller is responsible for closing it upon completion.


so instead of loading the apikey and port and all into a single
string, have you tried using the parameters to the Net::HTTP.start
method as such?

Eitherway, you might just be better of using curb for anything HTTP
related.

 

[Engine Yard]

The parameters in Net::HTTP::start(address, port = nil, p_addr = nil,
p_port = nil, p_user = nil, p_pass = nil) {|+http+| ...} are:

address = SERVER_ADDRESS
PORT = SERVER_PORT
P_ADDR = PROXY_SERVER_ADDRESS
P_PORT = PROXY_SERVER_PORT
P_USER = PROXY_USER
P_PASS = PROXY_USER_PASSWORD


I am not really sure about what a proxy server and a proxy user are, but
here is what happened in my case.

The URL that my backend expected was of the format:
http://[email protected]:80/selection/books/andy-grove.xml?session_key=32klfd3n
where '1234' was the API_KEY that was used to authenticate the calling
application.

If you look at it from a more general perspective this is similar to
http:user_nameassword@DOMAIN_NAME/PATH_TO_RESOURCE/RESPONSE_FORMAT?QUERY_PARAMS

Now I just had to substitute the user_name field with the API_KEY which
I was unable to do using: Net::HTTP::start(address, port = nil, p_addr =
nil, p_port = nil, p_user = nil, p_pass = nil) {|+http+| ...}

since the user and password here refer to the proxy user and the proxy
password.

But I found a simpler solution to this problem:

1. Set the 'Authorization' request header to 'Basic' and specify the
base64 encoded 'API_KEY + :' as the username value and leave the
password field empty.

This will basically form a url of the form:
http://encode64(API_KEY
+':')@SERVER_IP_ADDRESS/PATH_TO_RESOURCE/BLAH_BLAH_BLAH

To set the authorization header you can use:
a) Net::HTTP.Post.basic_auth(username, password)
b) Net::HTTP.Post.add_field('Authorization', 'Basic usernameassword)
(NOT EXACTLY SURE IF THE SYNTAX IS RIGHT)

Here is the link to ruby's documentation:
http://ruby-doc.org/stdlib/libdoc/net/http/rdoc/index.html