Problems using client certificates with net/https

[Jim Clark]

I have a very basic script to access a SSL encrypted site that uses
client certificates. Problem is that I continue to get 401 unauthorized
error even though the certificate works in a browser and I've verified
that the username/password used in the script to be the exact same
values that work in a browser.

Here is the script:

#!c:/ruby/bin/ruby.exe
CERT_FILE = "c:/certs/jim_nopw.pem"
require 'net/https'
https = Net::HTTP.new('some_website.com', 443)
https.use_ssl = true
https.cert = OpenSSL::X509::Certificate.new( File.read(CERT_FILE) )
https.key = OpenSSL:Key::RSA.new( File.read(CERT_FILE) )
https.verify_mode = OpenSSL::SSL::VERIFY_NONE
https.read_timeout = 120
https.start do |https|
request = Net::HTTP::Get.new('/default.asp')
request.basic_auth 'DOMAIN\username', 'password'
response = https.request(request)
response.value
puts response.body
end

This is the error output:
401 "Unauthorized"
C:/ruby/lib/ruby/1.8/net/http.rb:2106:in `error!'
C:/ruby/lib/ruby/1.8/net/http.rb:2115:in `value'
C:/MyApps/Ruby/webchecker2.rb:19
C:/ruby/lib/ruby/1.8/net/http.rb:547:in `start'
C:/MyApps/Ruby/webchecker2.rb:13
C:/ruby/lib/ruby/1.8/net/http.rb:2106:in `error!': 401 "Unauthorized"
(Net::HTTPServerException)
from C:/ruby/lib/ruby/1.8/net/http.rb:2115:in `value'
from C:/MyApps/Ruby/webchecker2.rb:19
from C:/ruby/lib/ruby/1.8/net/http.rb:547:in `start'
from C:/MyApps/Ruby/webchecker2.rb:13

Getting back to the certificate, here is how I manipulated it from a
browser to what I am feeding my Ruby script:

- Export the certificate from IE7 to c:\certs\jim.pfx
- Convert the certificate from PKCS12 format to non-password protected
PEM format according to OpenSSL howto at
http://www.madboa.com/geek/openssl/#cert-pkcs12:
C:\OpenSSL\bin>openssl pkcs12 -in c:\certs\jim.pfx -out
c:\certs\jim_nopw.pem -nodes
Enter Import Password:
MAC verified OK

C:\OpenSSL\bin>type c:\certs\jim_nopw.pem
Bag Attributes
localKeyID: 01 00 00 00
friendlyName: {65C0E138-9765-4B64-A591-ADC0CB302B31}
Microsoft CSP Name: Microsoft Enhanced Cryptographic Provider v1.0
Key Attributes
X509v3 Key Usage: 10
-----BEGIN RSA PRIVATE KEY-----
MIICWwIBAAKBgQDGHTOVdo3SDvysWTh1OTinMWSjQhqpPcxO24KbDdyWQml7iInv
... lines deleted and characters changed to protect my private key
Rf7cP02FJmftyH3D6hiK0Pzjv4a7d1vwWqj3naIB4Q==
-----END RSA PRIVATE KEY-----
Bag Attributes
localKeyID: 01 00 00 00
subject=/C=US/O=Organization/OU=ECA/OU=ORC/OU=Company/CN=Clark.James.ORC1000099999.ID
issuer=/C=US/O=Organization/OU=ECA/OU=Certification Authorities/CN=ORC ECA
-----BEGIN CERTIFICATE-----
MIIEPjCCA6egAwIBAgICeYQwDQYJKoZIhvcNQWEFBQAwazELMAkG2ErEBhMCVVMx
... lines deleted and characters changed to alter certificate
0/fVUv7Lx56ceXkQ7hDGiMXv32TWIp6CO+fkQkQhZ7wevA==
-----END CERTIFICATE-----

C:\OpenSSL\bin>


Anyone have any thoughts on what else I should be doing?

My environment:
Windows XP sp2
ruby 1.8.6 (2007-09-24 patchlevel 111) [i386-mswin32]

Thanks in advance,
Jim

 

[yermej]

I have a very basic script to access a SSL encrypted site that uses
client certificates. Problem is that I continue to get 401 unauthorized
error even though the certificate works in a browser and I've verified
that the username/password used in the script to be the exact same
values that work in a browser.

Here is the script:
<...snip...>

Anyone have any thoughts on what else I should be doing?

My environment:
Windows XP sp2
ruby 1.8.6 (2007-09-24 patchlevel 111) [i386-mswin32]

Thanks in advance,
Jim

The SSL portion looks right. Are you certain the server is using basic
authentication? The error doesn't look like any errors I've gotten
when having client cert problems.

 

[Daniel Sheppard]

I have a very basic script to access a SSL encrypted site that uses=20

client certificates. Problem is that I continue to get 401=20
unauthorized=20
https.start do |https|
request =3D Net::HTTP::Get.new('/default.asp')
request.basic_auth 'DOMAIN\username', 'password'
response =3D https.request(request)
response.value
puts response.body
end
=20
This is the error output:
401 "Unauthorized"

Given the fact that you're username looks like a windows domain user
name, I'm guessing that the server is using NTLM authentication and not
basic authentication.

Dan.

 

[Jim Clark]

Given the fact that you're username looks like a windows domain user
name, I'm guessing that the server is using NTLM authentication and not
basic authentication

As Homer would say, "D'oh!". I checked the web server and it is indeed
running "Integrated Windows Authentication". Thank you for pointing out
what was staring me in the face.

To get NTLM authentication working with net/https, I found
http://code.google.com/p/ruby-httpclient/wiki/Home and thought this
would be a good option. Reading up on it would seem all I would need to
do is apply the patch files to ntlm.rb and open-uri.rb and change my
authentication line from:

request.basic_auth 'DOMAIN\username', 'password'
to
request.ntlm_auth 'DOMAIN\username', 'password'

No dice though... http.rb still has no clue about NTLM authentication. I
started to modify http.rb to handle NTLM authentication based on the
example in rubyntlm that uses sockets but found myself getting far
deeper into NTLM auth than I wanted to for what should be a fairly short
and simple project.

So, being lazy and pressed for time, I took another look and found the
httpclient-2.1.2 library. However, documentation is sparse so does
anyone have any good examples of it in use?

Thanks in advance!

-Jim

 

[Daniel Sheppard]

=20

To get NTLM authentication working with net/https, I found=20
http://code.google.com/p/ruby-httpclient/wiki/Home and thought this=20
would be a good option. Reading up on it would seem all I=20
would need to=20
do is apply the patch files to ntlm.rb and open-uri.rb and change my=20
authentication line from:
=20
request.basic_auth 'DOMAIN\username', 'password'
to
request.ntlm_auth 'DOMAIN\username', 'password'

Alternatively, if you just want to get it working, you might want
to look at proxying your requests through
http://ntlmaps.sourceforge.net/

Dan.