SSLException when running as JWS/JNLP rather than in NetBeans

[casperbang]

I'm getting a nasty internal SSLException when talking to a Servlet
that is proxied behind an SSH tunnel, when called from a deployed (and
signed) Web Start Application:

Caused by: javax.net.ssl.SSLException: Received fatal alert:
unexpected_message
at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Unknown
Source)
at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Unknown
Source)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.recvAlert(Unknown
Source)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(Unknown
Source)
at
com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(Unknown
Source)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(Unknown
Source)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(Unknown
Source)
at sun.net.www.protocol.https.HttpsClient.afterConnect(Unknown
Source)
at
sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(Unknown
Source)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(Unknown
Source)
at com.brunata.httprmi.HttpRmiProxy.invokeRemote(HttpRmiProxy.java:
136)
at com.brunata.httprmi.HttpRmiProxy.invoke(HttpRmiProxy.java:202)

Oddly, there were never any problems when running from NetBeans/Ant up
against the SSL endpoint. What is the difference between these two
scenarios that could be playing games with me (different security
manager, other providers)?
I've seen lots of posts regarding SSL client-server communication in
the past, but I am fairly certain that these were due to older (1.2)
JSE version which did not handle SSL and so not really relevant any
longer. Comments or suggestions much appreciated.

/Casper

PS: My environment is Sun JSE6.0, using URLConnection from client
(http://httprmi.googlecode.com/svn/trunk/HttpRmi/src/com/brunata/
httprmi/HttpRmiProxy.java) to HttpServlet (http://
httprmi.googlecode.com/svn/trunk/HttpRmi/src/com/brunata/httprmi/
HttpRmiServlet.java) on a
Tomcat 6.0.14.

 

[casperbang]

I've examined the difference between environment and system properties
between the two. Obviously no security manager is interfering when
running in NetBeans, because output from the deployed version has a
bunch of other stuff:

deployment.user.security.trusted.certs: C:\Documents and Settings
\Casper\Application Data\Sun\Java\Deployment\security\trusted.certs
deployment.javaws.viewer.bounds: 480,223,720,360
sun.java.launcher: SUN_STANDARD
sun.management.compiler: HotSpot Client Compiler
deployment.security.TLSv1: true
deployment.cache.enabled: true
deployment.system.security.cacerts: C:\Program Files\Java
\jre1.6.0_02\lib\security\cacerts
java.runtime.version: 1.6.0_02-b06
deployment.security.notinca.warning: true
deployment.javaws.installURL: http://java.sun.com/products/autodl/j2se
deployment.javapi.lifecycle.exception: true
https.protocols: TLSv1,SSLv3
deployment.security.SSLv2Hello: false
deployment.security.askgrantdialog.notinca: true
deployment.proxy.override.hosts:
javaplugin.proxy.config.type: direct
deployment.security.clientauth.keystore.auto: true
deployment.user.security.trusted.clientauthcerts: C:\Documents and
Settings\Casper\Application Data\Sun\Java\Deployment\security
\trusted.clientcerts
sun.boot.library.path: C:\Program Files\Java\jre1.6.0_02\bin
deployment.user.cachedir: C:\Documents and Settings\Casper\Application
Data\Sun\Java\Deployment\cache
deployment.security.jsse.hostmismatch.warning: true
trustProxy: true
deployment.javaws.update.timeout: 1500
javawebstart.version: javaws-1.6.0_02
deployment.browser.vm.mozilla: true
sun.arch.data.model: 32
deployment.security.expired.warning: true
deployment.security.sandbox.jnlp.enhanced: true
deployment.log: true
sun.cpu.isalist:
deployment.repository.askdownloaddialog.show: true
deployment.javaws.associations: ASK_USER
deployment.javaws.shortcut: ASK_IF_HINTED
deployment.javaws.home.jnlp.url: http://java.sun.com/products/javawebstart
java.class.version: 50.0
deployment.cache.jarcompression: 0
deployment.javaws.ssv.enabled: true
deployment.security.sandbox.awtwarningwindow: true
deployment.trace: true
deployment.security.authenticator: true
java.vm.info: mixed mode, sharing
deployment.javaws.logFileName:
deployment.system.security.trusted.jssecerts: C:\Program Files\Java
\jre1.6.0_02\lib\security\trusted.jssecerts
deployment.security.validation.crl: false
deployment.user.security.trusted.cacerts: C:\Documents and Settings
\Casper\Application Data\Sun\Java\Deployment\security\trusted.cacerts
java.vm.version: 1.6.0_02-b06
http.auth.serializeRequests: true
deployment.security.validation.ocsp: false
deployment.user.security.trusted.jssecacerts: C:\Documents and Settings
\Casper\Application Data\Sun\Java\Deployment\security
\trusted.jssecacerts
deployment.javapi.trace.filename:
java.protocol.handler.pkgs: com.sun.javaws.net.protocol|
com.sun.deploy.net.protocol
deployment.system.security.trusted.clientauthcerts: C:\Program Files
\Java\jre1.6.0_02\lib\security\trusted.clientcerts
deployment.max.output.file.size: 10
deployment.system.security.trusted.certs: C:\Program Files\Java
\jre1.6.0_02\lib\security\trusted.certs
deployment.user.extdir: C:\Documents and Settings\Casper\Application
Data\Sun\Java\Deployment\ext
java.security.policy: file:C:\Program Files\Java\jre1.6.0_02\lib
\security\javaws.policy
deployment.security.askgrantdialog.show: true
deployment.user.security.policy:
file://C:/Documents%20and%20Settings/Casper/Application%20Data/Sun/Java/Deployment/security/java.policy
deployment.security.SSLv3: true
deployment.version: 6.0
deployment.proxy.type: 3
java.net.useSystemProxies: true
deployment.security.https.warning.show: false
deployment.javaws.autodownload: ALWAYS
deployment.max.output.files: 5
deployment.user.security.saved.credentials: C:\Documents and Settings
\Casper\Application Data\Sun\Java\Deployment\security\auth.dat
deployment.javaws.splash.index: C:\Documents and Settings\Casper
\Application Data\Sun\Java\Deployment\cache\6.0\splash\splash.xml
deployment.proxy.bypass.local: false
deployment.user.security.trusted.jssecerts: C:\Documents and Settings
\Casper\Application Data\Sun\Java\Deployment\security
\trusted.jssecerts
deployment.security.browser.keystore.use: true
deployment.proxy.same: false
deployment.system.security.jssecacerts: C:\Program Files\Java
\jre1.6.0_02\lib\security\jssecacerts
deployment.security.trusted.policy:

/Casper

 

[casperbang]

Hmm, it looks as though the problem only appear when I launch as a Web
Startable. Even a local launch.jnlp startup displays the errornous
behavior. Some kind of clash between the SSL proxy/frontend
certificate and my applications signed certificate (which are not the
same, are they required to be?) when running with the default Web
Start SecurityManager. The connection is set to do forwarding
(setInstanceFollowRedirects(true) and the active permission on the
connection is reported to be SSL_RSA_WITH_RC4_128_MD5.

/Casper

 

[casperbang]

Well I have exhausted all possebilities I am capable of. I think it's
a JRE bug, as I see no reason why running from JAR should be any
different than running from a JWS scenario (even with SecurityManager
turned deliberately off). It looked awfully similar to this bug:
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6514454
Though the 6514454 bug was just fixed and pushed out with the 1.6
update 3, it did not fix my issue so I will file a bug with Sun
regarding my issue.

/Casper