Data Interfacing

Discussion in 'Java' started by sandygs, May 20, 2005.

  1. sandygs

    sandygs Guest

    I am having to interface data from different databases in my
    org to other orgs (so far within the company).

    I have Servlets taking the request and returning the data (adding
    extra values in certain cases).
    The existing users are expecting either XML or delimited or positional
    formats.
    I had written formatters to do this job about several yrs ago.
    Since we are getting more users asking data in different flavors to fit
    their new or legacy apps, I was thinking of rewriting our
    formatting classes now than later before it can clutter.

    My approach is to use Jaxp to create a DOM structures from the
    data retrieved, add an attribute like "PAD_LEN" for each data
    element, pass the document to the Formatter, which...
    - Uses DOM Transformers to build xml format for xml clients.
    - Custom Delimiter Transformer1: Walks through the document and appends
    data and delimiters to the resultbuffer in case of delimited clients.
    - Custom Positional Transformer2: Walks through the document and pads
    trailing spaces for fixed-positional format users.

    Though this will take a lot of work to re-write the existing queries, I
    think it would be worth a while before it gets even bulkier.
    I thought this approach will also help if we migrate towards soap
    structure.

    I am curious to know if there are better approaches for such a
    scenario. I read Biztalk is capable of something like this... but we
    dont want to use any third-party tools to do this job for sure, just
    because it gives us more control over the formats.

    Any suggestions appreciated. Thnks.
     
    sandygs, May 20, 2005
    #1
    1. Advertising

  2. sandygs

    Guest

    On 2005-05-20, sandygs <> wrote:
    > I am having to interface data from different databases in my
    > org to other orgs (so far within the company).
    >
    > I have Servlets taking the request and returning the data (adding
    > extra values in certain cases).
    > The existing users are expecting either XML or delimited or positional
    > formats.
    > I had written formatters to do this job about several yrs ago.
    > Since we are getting more users asking data in different flavors to fit
    > their new or legacy apps, I was thinking of rewriting our
    > formatting classes now than later before it can clutter.
    >
    > My approach is to use Jaxp to create a DOM structures from the
    > data retrieved, add an attribute like "PAD_LEN" for each data
    > element, pass the document to the Formatter, which...
    > - Uses DOM Transformers to build xml format for xml clients.
    > - Custom Delimiter Transformer1: Walks through the document and appends
    > data and delimiters to the resultbuffer in case of delimited clients.
    > - Custom Positional Transformer2: Walks through the document and pads
    > trailing spaces for fixed-positional format users.
    >
    > Though this will take a lot of work to re-write the existing queries, I
    > think it would be worth a while before it gets even bulkier.
    > I thought this approach will also help if we migrate towards soap
    > structure.
    >
    > I am curious to know if there are better approaches for such a
    > scenario. I read Biztalk is capable of something like this... but we
    > dont want to use any third-party tools to do this job for sure, just
    > because it gives us more control over the formats.
    >
    > Any suggestions appreciated. Thnks.


    I'm using a pipeline engine I wrote, called Relay to do this sort of
    thing. The transforms you mention can be written in XSLT, or of they
    queries are very, very large, you can write a SAX filter, which would be
    more efficent. The pipeline engine I've written is designed to make it
    simple to write custom transformers, and have them link in correctly, and
    also cache the resutls. I'm buried in work, or else I'd be working on an
    open source release.

    There are similar engines out there, like Cocoon and Oberon OXF.

    In any case, if you're not familiar with XSLT this is would definately be
    an application. Try looking through this book.

    http://aha.homelinux.com/bindee/work/refbooks/xml/jxslt/index.htm

    Cheers.

    --
    Alan Gutierrez -
    - http://engrm.com/blogometer/index.html
    - http://engrm.com/blogometer/rss.2.0.xml
     
    , May 20, 2005
    #2
    1. Advertising

  3. sandygs

    sandygs Guest

    Thnks alan, for the suggestion. It sounds like the way to go using
    XSLT.
    Either way I guess I am on the right track, gather data into a DOM
    structure and let the transformers worry about format-conversions...,
    right?
    And when I get to learn XSLT, all I/someone will need to change is my
    custom transformers to use XSLT for formatting... right?

    Thanks.


    wrote:
    > On 2005-05-20, sandygs <> wrote:
    > > I am having to interface data from different databases in my
    > > org to other orgs (so far within the company).
    > >
    > > I have Servlets taking the request and returning the data (adding
    > > extra values in certain cases).
    > > The existing users are expecting either XML or delimited or

    positional
    > > formats.
    > > I had written formatters to do this job about several yrs ago.
    > > Since we are getting more users asking data in different flavors to

    fit
    > > their new or legacy apps, I was thinking of rewriting our
    > > formatting classes now than later before it can clutter.
    > >
    > > My approach is to use Jaxp to create a DOM structures from the
    > > data retrieved, add an attribute like "PAD_LEN" for each data
    > > element, pass the document to the Formatter, which...
    > > - Uses DOM Transformers to build xml format for xml clients.
    > > - Custom Delimiter Transformer1: Walks through the document and

    appends
    > > data and delimiters to the resultbuffer in case of delimited

    clients.
    > > - Custom Positional Transformer2: Walks through the document and

    pads
    > > trailing spaces for fixed-positional format users.
    > >
    > > Though this will take a lot of work to re-write the existing

    queries, I
    > > think it would be worth a while before it gets even bulkier.
    > > I thought this approach will also help if we migrate towards soap
    > > structure.
    > >
    > > I am curious to know if there are better approaches for such a
    > > scenario. I read Biztalk is capable of something like this... but

    we
    > > dont want to use any third-party tools to do this job for sure,

    just
    > > because it gives us more control over the formats.
    > >
    > > Any suggestions appreciated. Thnks.

    >
    > I'm using a pipeline engine I wrote, called Relay to do this sort

    of
    > thing. The transforms you mention can be written in XSLT, or of

    they
    > queries are very, very large, you can write a SAX filter, which

    would be
    > more efficent. The pipeline engine I've written is designed to

    make it
    > simple to write custom transformers, and have them link in

    correctly, and
    > also cache the resutls. I'm buried in work, or else I'd be

    working on an
    > open source release.
    >
    > There are similar engines out there, like Cocoon and Oberon OXF.
    >
    > In any case, if you're not familiar with XSLT this is would

    definately be
    > an application. Try looking through this book.
    >
    > http://aha.homelinux.com/bindee/work/refbooks/xml/jxslt/index.htm
    >
    > Cheers.
    >
    > --
    > Alan Gutierrez -
    > - http://engrm.com/blogometer/index.html
    > - http://engrm.com/blogometer/rss.2.0.xml
     
    sandygs, May 20, 2005
    #3
  4. sandygs

    Guest

    On 2005-05-20, sandygs <> wrote:

    > Thnks alan, for the suggestion. It sounds like the way to go using
    > XSLT.


    > Either way I guess I am on the right track, gather data into a DOM
    > structure and let the transformers worry about format-conversions...,
    > right?


    Sure. Pull the XML document into a DOM. Then create a simple interace that
    take the DOM and emits a byte stream. Simple. It will buy you time to
    learn about XML pipelining and XSLT.

    > And when I get to learn XSLT, all I/someone will need to change is my
    > custom transformers to use XSLT for formatting... right?


    The transforms you described, to CSV, positional text, etc. They
    all exist as XSLT somewhere. There is a mailing list called xsl-list that
    you can Google. I'm on it. It is a very helpful community.

    Using XSLT as the langauge to describe the output probably your best bet.
    Look over that book I linked. You'll see that they discuss format
    conversion strategies.

    Cheers.

    --
    Alan Gutierrez -
    - http://engrm.com/blogometer/index.html
    - http://engrm.com/blogometer/rss.2.0.xml
     
    , May 20, 2005
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. SneakerNet
    Replies:
    0
    Views:
    678
    SneakerNet
    Sep 1, 2003
  2. Amontec Team, Laurent Gauch

    interfacing Chameleon POD

    Amontec Team, Laurent Gauch, Feb 5, 2004, in forum: VHDL
    Replies:
    0
    Views:
    628
    Amontec Team, Laurent Gauch
    Feb 5, 2004
  3. Gunit

    Interfacing to PCI

    Gunit, Aug 25, 2004, in forum: VHDL
    Replies:
    0
    Views:
    1,128
    Gunit
    Aug 25, 2004
  4. cka
    Replies:
    2
    Views:
    4,316
    pablo aimar
    Sep 11, 2004
  5. headware

    interfacing with data layer

    headware, May 14, 2004, in forum: ASP .Net
    Replies:
    2
    Views:
    482
    headware
    May 18, 2004
Loading...

Share This Page