Digitally signing XML files

Discussion in 'Java' started by Roedy Green, Feb 7, 2006.

  1. Roedy Green

    Roedy Green Guest

    I wonder if there is a standard way to digitally sign XML files.

    1. where do you put the signature, the public key? Can you tack them
    on the end, embed them in special tags?

    2. what do you sign, the encoded bytes or the chars?

    3. how do you deal with transient white space that might disappear if
    someone tidied the file.

    4. how do you deal with platform-specific new line chars? do you
    treat them as if nl, do you transform the document first, do you
    preserve them?

    5. what about lead trail space on fields. It this removed first?
    --
    Canadian Mind Products, Roedy Green.
    http://mindprod.com Java custom programming, consulting and coaching.
     
    Roedy Green, Feb 7, 2006
    #1
    1. Advertising

  2. Roedy Green

    Tony Morris Guest

    "Roedy Green" <> wrote in
    message news:...
    > I wonder if there is a standard way to digitally sign XML files.


    Take a look at the W3C XML Signature spec.

    --
    Tony Morris
    http://tmorris.net/
     
    Tony Morris, Feb 7, 2006
    #2
    1. Advertising

  3. Roedy Green <> writes:

    > I wonder if there is a standard way to digitally sign XML files.


    There is, W3C has a specification for XML Signatures, see
    <http://www.w3.org/TR/xmldsig-core/>. Apache has a Java
    implementation as a part of the XML-Security project.

    > 1. where do you put the signature, the public key? Can you tack them
    > on the end, embed them in special tags?


    You put in a new element called Signature. This contains references
    to the signed parts of the document and digests for each, as well as
    the computed signature value. It can also contain information on the
    key used, either by including the public key or a reference to it.

    > 2. what do you sign, the encoded bytes or the chars?


    Crypto algorithms typically require their input to be bytes, so that's
    what you need to produce.

    > 3. how do you deal with transient white space that might disappear if
    > someone tidied the file.
    >
    > 4. how do you deal with platform-specific new line chars? do you
    > treat them as if nl, do you transform the document first, do you
    > preserve them?
    >
    > 5. what about lead trail space on fields. It this removed first?


    These are easiest to answer with a single word: canonicalization.
    There are a couple of W3C-specified canonicalization algorithms (see,
    e.g., <http://www.w3.org/TR/2001/REC-xml-c14n-20010315>). Such an
    algorithm takes XML and attempts to remove any incidentals so that the
    same output is produced even after some reasonable XML processing.

    To go to specifics, canonicalization does not permit you to alter
    whitespace content of an XML document, so your point 3 would create a
    different document, I cannot find anything that says newlines are
    converted to any other form, and in point 5, if by "field" you mean
    element content or attribute value, all whitespace is retained.

    --
    Jaakko Kangasharju, Helsinki Institute for Information Technology
    You don't have to be crazy to work here...and it doesn't help either
     
    Jaakko Kangasharju, Feb 7, 2006
    #3
  4. Roedy Green

    Roedy Green Guest

    On Tue, 07 Feb 2006 13:09:17 +0200, Jaakko Kangasharju
    <> wrote, quoted or indirectly quoted someone who said
    :

    >
    >These are easiest to answer with a single word: canonicalization.
    >There are a couple of W3C-specified canonicalization algorithms (see,
    >e.g., <http://www.w3.org/TR/2001/REC-xml-c14n-20010315>). Such an
    >algorithm takes XML and attempts to remove any incidentals so that the
    >same output is produced even after some reasonable XML processing.


    I was trying to make sense of the standard. Perhaps you have digested
    it .

    1. does it require you to manually embed a private ? public? key in
    the xml file or will it work with ordinary code-signing certs.

    2. it is up to you to compute the digests, or is that the function of
    the various algorithms you specify.

    3. How do they specify algorithms in a platform independent way? Are
    these java code?

    I am working on an idea to improve PAD file submission. These are
    descriptions of shareware. The submission sites work hard to
    discourage automated submission, trying to discourage spam. My idea to
    fix this revolves around programmers and distribution sites having
    digital certificates and everything having globally unique ids. To do
    it requires a scheme for adding digital signatures to PAD files which
    are a flavour of XML.

    --
    Canadian Mind Products, Roedy Green.
    http://mindprod.com Java custom programming, consulting and coaching.
     
    Roedy Green, Feb 7, 2006
    #4
  5. Roedy Green <> writes:

    > I was trying to make sense of the standard. Perhaps you have digested
    > it .


    I'm certainly not an expert, but I'll try.

    > 1. does it require you to manually embed a private ? public? key in
    > the xml file or will it work with ordinary code-signing certs.


    Sure, you can specify the key as a cert too.

    > 2. it is up to you to compute the digests, or is that the function of
    > the various algorithms you specify.


    I'm not quite sure what you are asking here. It's best to let
    whatever implementation of XML Signature you use to handle that.

    > 3. How do they specify algorithms in a platform independent way? Are
    > these java code?


    Algorithms have URIs as identifiers. The signature specification
    defines URIs for a few common algorithms.

    > I am working on an idea to improve PAD file submission. These are
    > descriptions of shareware. The submission sites work hard to
    > discourage automated submission, trying to discourage spam. My idea to
    > fix this revolves around programmers and distribution sites having
    > digital certificates and everything having globally unique ids. To do
    > it requires a scheme for adding digital signatures to PAD files which
    > are a flavour of XML.


    Are you sure you have to sign XML? XML Signature is a pretty heavy
    thing to do. Could you manage by just signing the whole PAD file or
    is it likely that the XML needs to be processed somehow between
    signing and verifying?

    --
    Jaakko Kangasharju, Helsinki Institute for Information Technology
    Do you cry alone, do you cough by yourself?
     
    Jaakko Kangasharju, Feb 7, 2006
    #5
  6. Roedy Green

    Oliver Wong Guest

    "Roedy Green" <> wrote in
    message news:...
    > On Tue, 07 Feb 2006 13:09:17 +0200, Jaakko Kangasharju
    > <> wrote, quoted or indirectly quoted someone who said
    > :
    >
    >>
    >>These are easiest to answer with a single word: canonicalization.
    >>There are a couple of W3C-specified canonicalization algorithms (see,
    >>e.g., <http://www.w3.org/TR/2001/REC-xml-c14n-20010315>). Such an
    >>algorithm takes XML and attempts to remove any incidentals so that the
    >>same output is produced even after some reasonable XML processing.

    >
    > I was trying to make sense of the standard. Perhaps you have digested
    > it .


    I don't know about this XML-Signature standard in particular (the
    document is too long for me to read it in its entirety), but I can give you
    general information about cryptography and signing.

    >
    > 1. does it require you to manually embed a private ? public? key in
    > the xml file or will it work with ordinary code-signing certs.


    Since you're signing the XML document, you probably want to distribute
    the XML document. You should never distribute your private key, so the
    private key would never be embedded in the XML document. If someone wishes
    to verify the signature, they need access to your public key. While you
    COULD embed the public key in the XML, there's no way for the person doing
    the verifying to know if the public key they've found in the XML document
    really belongs to you or not. So it's more common to register your public
    key with an agreed upon 3rd party certification provider.

    For the purpose of these PAD files, a shareware author could create an
    account with the distribution sites, and as part of the registration
    process, send the sites a public key. Then, whenever the PAD is submitted,
    the site could verify with their (locally stored) copy of the public key to
    ensure that the XML really is from the author.

    >
    > 2. it is up to you to compute the digests, or is that the function of
    > the various algorithms you specify.


    This may be implementation dependent. Elsewhere in this thread, someone
    mentioned Apache had an implementation for the XML-Signature standard. You
    should read the documentation (JavaDocs?) for that project to find out
    exactly what is your responsibility, and what their implementation will take
    care of.

    >
    > 3. How do they specify algorithms in a platform independent way? Are
    > these java code?


    I've seen specification in very rigorous English prose, in pseudo code,
    as a mathematical formula, as C code, and sometimes as some combination of
    the above. I'm guessing this XML-Signature standard doesn't actually specify
    any algorithms, but rather just says once you've got an XML document, and a
    signature, how can you embed one within the other, etc.

    >
    > I am working on an idea to improve PAD file submission. These are
    > descriptions of shareware. The submission sites work hard to
    > discourage automated submission, trying to discourage spam. My idea to
    > fix this revolves around programmers and distribution sites having
    > digital certificates and everything having globally unique ids. To do
    > it requires a scheme for adding digital signatures to PAD files which
    > are a flavour of XML.


    The central problem is ensuring that a given person cannot claim to be
    two different people (which simply isn't technologically possible right
    now). Even with digital signatures, a spammer could create thousands of
    public-private key pairs, and still send in junk submissions, and the sites
    would still have to filter through these submissions. Given an XML document
    with a signature from a public key you've never seen before, how can you
    tell if this is a one-off key made by a spammer, or a newbie's first
    legitimate shareware submission?

    - Oliver
     
    Oliver Wong, Feb 7, 2006
    #6
  7. Roedy Green

    Guest

    Hi Roedy, I've actually done this a lot: See below for an answer to
    your questions. First though, allow me to post some code. (The
    enveloped signature comments have since been fixed by the xmlsec
    project).

    public static boolean sign(Document doc, X509Certificate cert,
    PrivateKey privateKey, boolean debug) throws WSSecurityException
    {
    try
    {
    //Add SecurityHelper.class header to the SOAP message if it does
    not exist
    String soap_header = "http://schemas.xmlsoap.org/soap/envelope/";
    org.apache.xml.security.Init.init();

    /******************* XML SIGNATURE INIT ***********************
    Append the signature element to proper location before signing
    ***************************************************************/
    // Look for the SOAP header
    Element headerElement = null;
    NodeList nodes = doc.getElementsByTagNameNS (soap_header,
    "Header");
    //No nodes are expected to be found (length of zero) - add
    //header here.
    if(nodes.getLength() == 0)
    {
    headerElement = doc.createElementNS (soap_header, "Header");
    nodes = doc.getElementsByTagNameNS (soap_header, "Envelope");
    if(nodes != null)
    {
    Element envelopeElement = (Element)nodes.item(0);
    headerElement.setPrefix(envelopeElement.getPrefix());
    envelopeElement.appendChild(headerElement);
    }
    }
    else
    {
    //This shouldn't happen unless explicity done elsewhere
    Fwlog.debug(SecurityHelper.class, Fwlog.WI, "Unexpectedly Found
    " + nodes.getLength() + " SOAP Header elements... probably ok but not
    tested");
    headerElement = (Element)nodes.item(0);
    }

    // http://xml-security is the base-uri, which needs to be
    //unique within document
    XMLSignature sig = new XMLSignature(doc, "http://xml-security",
    XMLSignature.ALGO_ID_SIGNATURE_DSA);
    // Add SOAP Body to XML Signature
    headerElement.appendChild(sig.getElement());
    // Due to the bug in the Apache security lib, it does not allow
    // us to sign whole message and make "enveloped-signature"
    // transform - strictly part of the specification.
    // Only sign the body - IT IS NOT CONFORMED TO THE SPEC!!!!!!
    //
    // Neat trick: since the XMLSignature is actually a part of the
    // SOAP XML document, the SOAP body is referenced as a URI
    // fragment
    sig.addDocument("#Body");
    /******************* END XML SIGNATURE INIT
    *********************/

    /****************** SIGN THE FUCKER *******************/
    sig.addKeyInfo(cert);
    sig.addKeyInfo(cert.getPublicKey());
    // Sign the XML Signature document with our private key
    sig.sign(privateKey);
    /****************** FUCKER SIGNED *********************/
    Fwlog.debug(SecurityHelper.class, Fwlog.WI, "SecurityHelper::sign
    -- tudo bem...");
    return true;
    }
    catch (Exception e)
    {
    Fwlog.error(SecurityHelper.class, Fwlog.WI, "SecurityHelper::sign
    -- Exception: ");
    Fwlog.error(SecurityHelper.class, Fwlog.WI, e);
    throw new WSSecurityException("Could not digitally sign SOAP
    message", e);
    }
    }

    /**
    Verifica a assinatura do documento XML
    @param doc documento XML
    @return boolean true se tem succeso, false no contrĂ¡rio
    */
    public static boolean verify(Document doc) throws WSSecurityException
    {
    try
    {
    org.apache.xml.security.Init.init();

    // must match baseURI
    String baseURI = "http://xml-security";
    CachedXPathAPI xpathAPI = new CachedXPathAPI();
    Element nsctx = doc.createElement("nsctx");
    nsctx.setAttribute("xmlns:ds", Constants.SignatureSpecNS);

    Element signatureElem = (Element) xpathAPI.selectSingleNode(doc,
    "//ds:Signature", nsctx);
    // Check to make sure that the document claims to have been
    signed
    if (null == signatureElem)
    {
    throw new IllegalStateException ("SOAP Document not digitally
    signed - missing element: //ds:Signature");
    }

    XMLSignature sig = new XMLSignature (signatureElem, baseURI);
    boolean verify = sig.checkSignatureValue
    (sig.getKeyInfo().getPublicKey());
    if (true == verify)
    {
    Fwlog.debug(SecurityHelper.class, Fwlog.WI,
    "SecurityHelper::verify -- tudo bem, returning true...");
    return true;
    }
    }
    catch (Exception e)
    {
    Fwlog.error(SecurityHelper.class, Fwlog.WI,
    "SecurityHelper::verify -- Exception: ");
    Fwlog.error(SecurityHelper.class, Fwlog.WI, e);
    throw new WSSecurityException("Could not verify digitally signed
    SOAP message", e);
    }

    // signature verification failed -
    //do not forward request to SOAP Service.
    return false;
    }

    See below:

    Roedy Green escreveu:

    > On Tue, 07 Feb 2006 13:09:17 +0200, Jaakko Kangasharju
    > <> wrote, quoted or indirectly quoted someone who said
    > :
    >
    > >
    > >These are easiest to answer with a single word: canonicalization.
    > >There are a couple of W3C-specified canonicalization algorithms (see,
    > >e.g., <http://www.w3.org/TR/2001/REC-xml-c14n-20010315>). Such an
    > >algorithm takes XML and attempts to remove any incidentals so that the
    > >same output is produced even after some reasonable XML processing.

    >
    > I was trying to make sense of the standard. Perhaps you have digested
    > it .
    >
    > 1. does it require you to manually embed a private ? public? key in
    > the xml file or will it work with ordinary code-signing certs.


    You embed the keys.

    >
    > 2. it is up to you to compute the digests, or is that the function of
    > the various algorithms you specify.
    >


    Its dependant on your arguments to 'keytool' for example, ie, how you
    create your PrivateKey.

    > 3. How do they specify algorithms in a platform independent way? Are
    > these java code?
    >


    You can use triple des, aes etc. There's a xmlsec dependency of 'bouncy
    hunter' - you might look there for more info.

    > I am working on an idea to improve PAD file submission. These are
    > descriptions of shareware. The submission sites work hard to
    > discourage automated submission, trying to discourage spam. My idea to
    > fix this revolves around programmers and distribution sites having
    > digital certificates and everything having globally unique ids. To do
    > it requires a scheme for adding digital signatures to PAD files which
    > are a flavour of XML.
    >


    For globally unique ids - don't forget about java.util.UUID

    Sounds kool, good luck!

    HTH,
    iksrazal
    http://www.braziloutsource.com/
     
    , Feb 7, 2006
    #7
  8. Roedy Green

    Roedy Green Guest

    On Tue, 07 Feb 2006 16:32:27 GMT, "Oliver Wong" <>
    wrote, quoted or indirectly quoted someone who said :

    > Since you're signing the XML document, you probably want to distribute
    >the XML document. You should never distribute your private key, so the
    >private key would never be embedded in the XML document.


    I looked as if you were supposed to do this, then presumably it would
    be removed after signing.

    I want to spank folks so often who write specs. They seem ever so
    intent on appearing learned with no interest in explaining what is
    ACTUALLY happening.

    --
    Canadian Mind Products, Roedy Green.
    http://mindprod.com Java custom programming, consulting and coaching.
     
    Roedy Green, Feb 8, 2006
    #8
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. O S
    Replies:
    0
    Views:
    740
  2. Replies:
    1
    Views:
    544
    Daniel Pitts
    Feb 6, 2008
  3. Replies:
    1
    Views:
    361
    red floyd
    Feb 7, 2008
  4. haxier

    Digitally sign PDF files

    haxier, Aug 11, 2008, in forum: Python
    Replies:
    4
    Views:
    663
    John Krukoff
    Aug 15, 2008
  5. Adam Tauno Williams
    Replies:
    2
    Views:
    887
    Stefan Behnel
    Dec 30, 2010
Loading...

Share This Page