Data munging, the choices available.

Discussion in 'Perl Misc' started by irishhacker@gmail.com, Apr 22, 2007.

  1. Guest

    What's the percentage of Perl users who use Perl for data munging
    (cleaning up data , data transformation, etc) on a fairly common
    occasion?
    Perl is particularly good at regular expressions, which is useful for
    some types of data munging.

    There are mainly three choices for data munging. Which choice is best
    depends on the type of data problem one has. There are many different
    types of data munging, both in degree of difficulty, and flavor.

    ALL-PURPOSE PROGRAMMING LANGUAGES
    obvious example: Perl

    SPECIALIZED PROGRAMMING LANGUAGES
    obvious example: SAS datastep (but extremely expensive) , also SPSS
    ( to get data ready for analysis, same thing)
    PSPP (GPL open source re-implementation of SPSS programming language,
    @ http://directory.fsf.org/math/stats )
    DAP (GPL open source re-implementation of SAS programming language, @
    http://directory.fsf.org/math/stats )
    vilno (GPL open source, another data transformation programming
    language and engine, @ http://code.google.com/p/vilno )

    GRAPHICAL USER INTERFACE
    Kettle ( http://kettle.pentaho.org )
    KETL, ( http://www.ketl.org ) and on and on.
    Particularly popular with the "T" part of "ETL" .
    ETL is always marketed as having a GUI front-end, no one ever mentions
    using an ETL programming language.
    If the complexity/quality of the data is not that bad, and hence the
    required munging is not too complicated, then a GUI product is good.
    But if Murphy's law strikes with the databases(if something can go
    wrong it will), programming languages provide more flexibily for bad
    situations.
     
    , Apr 22, 2007
    #1
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Guest

    Cookie munging problem

    Guest, Aug 1, 2003, in forum: ASP .Net
    Replies:
    5
    Views:
    1,363
    Seaside
    Aug 2, 2003
  2. Replies:
    2
    Views:
    1,311
  3. Timothy Grant

    py.test munging strings in asserts?

    Timothy Grant, Apr 21, 2006, in forum: Python
    Replies:
    1
    Views:
    314
    Johnny deBris
    Apr 21, 2006
  4. Chris Allen
    Replies:
    3
    Views:
    323
    Arnau Sanchez
    Aug 29, 2007
  5. cartercc

    An odd sort requirement - data munging

    cartercc, Nov 4, 2008, in forum: Perl Misc
    Replies:
    7
    Views:
    117
    Uri Guttman
    Nov 4, 2008
Loading...

Share This Page