Selecting Technology for a Data Filtering and Manipulation Project

Discussion in 'Java' started by sengsational, Aug 5, 2008.

  1. sengsational

    sengsational Guest

    New Project: SAP to i2 using Oracle (< sorry to do that to you in a
    Java forum).

    Last week I played with Oracle's JDeveloper a little bit. This week
    I've been playing with Eclipse DTP with Oracle's plug in. I'm sure
    I've only scratched the surface of possibilities, so thought I'd see
    if there were any ideas from the community (that's you!).

    The project is to manage and manipulate data between SAP's Sales and
    Distribution (for demand data), various i2 supply chain apps, and back
    to SAP Product Planning. Master data and transactional data. For
    those coders not into manufacturing, all of that doesn't really
    matter. The point is that between these commercial applications will
    be periodic batch jobs that involve large amounts of data which needs
    heavy manipulation sometimes, needs a significant amount of
    persistence (Oracle will be used), and will need many batch jobs to
    run in a coordinated way.

    One solution would be to write a bunch of PL/SQL procedures and drive
    those from an external batch scheduling system. But of course that
    produces spaghetti in the way of cursors, temp tables, sub-sub-sub
    selects that are impossible to diagnose, and most of all, doesn't
    employ Java!

    So the question is: What tool-set should be deployed to handle this
    project so that it can be treated like a real application, and not as
    a heap of steaming "stuff"? Is there a "clean" way to do this? There
    are so many projects running on Eclipse, I don't know if any actually
    fit this kind of project. Can an OR mapping idea be employed? BPEL
    maybe? Which tools should I look at?

    --Dale--
    sengsational, Aug 5, 2008
    #1
    1. Advertising

  2. sengsational wrote:
    > New Project: SAP to i2 using Oracle (< sorry to do that to you in a
    > Java forum).
    >
    > Last week I played with Oracle's JDeveloper a little bit. This week
    > I've been playing with Eclipse DTP with Oracle's plug in. I'm sure
    > I've only scratched the surface of possibilities, so thought I'd see
    > if there were any ideas from the community (that's you!).
    >
    > The project is to manage and manipulate data between SAP's Sales and
    > Distribution (for demand data), various i2 supply chain apps, and back
    > to SAP Product Planning. Master data and transactional data. For
    > those coders not into manufacturing, all of that doesn't really
    > matter. The point is that between these commercial applications will
    > be periodic batch jobs that involve large amounts of data which needs
    > heavy manipulation sometimes, needs a significant amount of
    > persistence (Oracle will be used), and will need many batch jobs to
    > run in a coordinated way.
    >
    > One solution would be to write a bunch of PL/SQL procedures and drive
    > those from an external batch scheduling system. But of course that
    > produces spaghetti in the way of cursors, temp tables, sub-sub-sub
    > selects that are impossible to diagnose, and most of all, doesn't
    > employ Java!
    >
    > So the question is: What tool-set should be deployed to handle this
    > project so that it can be treated like a real application, and not as
    > a heap of steaming "stuff"? Is there a "clean" way to do this? There
    > are so many projects running on Eclipse, I don't know if any actually
    > fit this kind of project. Can an OR mapping idea be employed? BPEL
    > maybe? Which tools should I look at?
    >
    > --Dale--

    Sounds like an data integration/ETL (extraction, transformation and
    loading) job for which something like Informatica or its competitors
    should be a good choice. There are open source java based tools which I
    haven't used but look into clover ETL and Pentaho data integration. And
    search for (open source?) data integration tools.
    Abhijat Vatsyayan, Aug 5, 2008
    #2
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. New User
    Replies:
    1
    Views:
    448
    Saravana [MVP]
    Nov 27, 2003
  2. =?Utf-8?B?U2hhd24gUmFtaXJleg==?=

    Custom Collection and Filtering Data

    =?Utf-8?B?U2hhd24gUmFtaXJleg==?=, Jun 30, 2006, in forum: ASP .Net
    Replies:
    3
    Views:
    3,092
    Karl Seguin [MVP]
    Jun 30, 2006
  3. Guest
    Replies:
    5
    Views:
    424
    Scott David Daniels
    Mar 30, 2007
  4. Replies:
    0
    Views:
    321
  5. Ryan Liu
    Replies:
    0
    Views:
    127
    Ryan Liu
    Dec 12, 2008
Loading...

Share This Page