Is it possible there might be a problem with Kerberos configuration? Do you
see any errors in the server's system event log from Kerberos? If you
enable auditing of logon events, what type of logon processes the user when
it fails?
Typically, using an IP address in the URL will force a downgrade to NTLM.
That might be why things are working.
When using Kerberos auth, both the user and server are authenticated. The
server is authenticated via its servicePrincipalName. I bet you that the
SPN for the server name that you are using in the URL is associated with the
computer account for the server, not your custom domain user. As such, the
app pool itself cannot be authenticated.
Typically, the way to correct this is to move the SPN that is being used
from the computer account to the custom account. You can also fix this by
creating a different DNS name for the service (with an A record, not a
CNAME, as Kerberos will resolve CNAME back to the A record!) and set your
special service account to have that SPN.
HTH!
Joe K.