Sequential analysis in perl?

Discussion in 'Perl Misc' started by D Cameron, May 24, 2004.

  1. D Cameron

    D Cameron Guest

    I just started writing perl a couple of weeks ago when I walked on to a
    project that was using perl to extract data from a raw data file. It has
    been a dream for text processing, as advertised. But after the data is
    extracted I want to do some statistical tests of the sequential analysis
    variety, and I'm not sure if perl is the best way to go. I was hoping
    some people here could offer some opinions.

    The questions I want to answer are along the lines of showing that a
    Markov chain is of order r, and not order r-1; looking at individual
    cells of a contingency table to see which ones are significant; and
    calculating the likelihood ratio chi-square (LRX^2).

    The calculations involve a lot of matrix manipulation which I'm
    currently doing with 2D arrays and lots of for(i;i<n;i++) loops. I've
    searched around CPAN a bit but the Statistics and Algorithm namespaces
    didn't offer what I was looking for. After getting great results from
    Algorithm::Cluster I was hoping I'd find something similar for
    sequential analysis.

    My perl knowledge is limited, I'm only around chapter 4 of the Camel
    book. The entertaining writing has me reading every page, but I was
    quite impressed to find I could do a
    %results{$picture}{$question}[$prev][$curr] structure without too many
    complaints.

    Basically I'm wondering:
    -Does the problem sound appropriate for perl, or should I go looking
    somewhere else?
    -Am I missing something about perl that would simplify my code
    significantly?
    -Can anyone suggest packages that would be useful that I've overlooked?

    In case my statistical jargon is unclear, I'll mention my reference here:
    Gottman & Roy (1990) Sequential Analysis: A guide for behavioral
    researchers. New York: Cambridge University Press.
     
    D Cameron, May 24, 2004
    #1
    1. Advertising

  2. D Cameron

    Anno Siegel Guest

    D Cameron <> wrote in comp.lang.perl.misc:
    > I just started writing perl a couple of weeks ago when I walked on to a
    > project that was using perl to extract data from a raw data file. It has
    > been a dream for text processing, as advertised. But after the data is
    > extracted I want to do some statistical tests of the sequential analysis
    > variety, and I'm not sure if perl is the best way to go. I was hoping
    > some people here could offer some opinions.
    >
    > The questions I want to answer are along the lines of showing that a
    > Markov chain is of order r, and not order r-1; looking at individual
    > cells of a contingency table to see which ones are significant; and
    > calculating the likelihood ratio chi-square (LRX^2).
    >
    > The calculations involve a lot of matrix manipulation which I'm
    > currently doing with 2D arrays and lots of for(i;i<n;i++) loops. I've


    [...]

    > -Am I missing something about perl that would simplify my code
    > significantly?
    > -Can anyone suggest packages that would be useful that I've overlooked?


    Take a look at PDL on CPAN.

    Anno
     
    Anno Siegel, May 24, 2004
    #2
    1. Advertising

  3. D Cameron

    Jim Keenan Guest

    Jim Keenan, May 24, 2004
    #3
  4. D Cameron <> wrote:

    > I just started writing perl a couple of weeks ago when I walked on
    > to a project that was using perl to extract data from a raw data
    > file. It has been a dream for text processing, as advertised. But
    > after the data is extracted I want to do some statistical tests of
    > the sequential analysis variety, and I'm not sure if perl is the
    > best way to go. I was hoping some people here could offer some
    > opinions.
    >
    > The questions I want to answer are along the lines of showing that
    > a Markov chain is of order r, and not order r-1; looking at
    > individual cells of a contingency table to see which ones are
    > significant; and calculating the likelihood ratio chi-square
    > (LRX^2).
    >
    > The calculations involve a lot of matrix manipulation which I'm
    > currently doing with 2D arrays and lots of for(i;i<n;i++) loops.
    > I've searched around CPAN a bit but the Statistics and Algorithm
    > namespaces didn't offer what I was looking for. After getting
    > great results from Algorithm::Cluster I was hoping I'd find
    > something similar for sequential analysis.


    You might take a look at the R language: http://www.r-project.org/

    It may be more appropriate for your purpose. There's no reason you
    can't use Perl to massage your data into a usable form and then use R
    as the analysis language.
     
    David K. Wall, May 24, 2004
    #4
  5. D Cameron

    D Cameron Guest

    Unfortunately I lot of the stuff google came up with are link farms.
    Maybe this is time to bust out Vivissimo. Anyway, I did find out about R
    and the MatLab packages through a similar search, which is what prompted
    me to start questioning perl altogether.

    Dave
     
    D Cameron, May 24, 2004
    #5
  6. D Cameron

    D Cameron Guest

    Thanks for the R suggestion! I'd run across a reference to it in one of
    the readmes for a Statistics:: module and was thinking it might be worth
    looking at. We had been hoping to get the process down to a one step
    batch sort of thing, but even two steps would be better than what they
    had before.

    Dave
     
    D Cameron, May 24, 2004
    #6
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. john

    Sequential Machines

    john, Oct 28, 2004, in forum: VHDL
    Replies:
    6
    Views:
    642
    rickman
    Nov 5, 2004
  2. Taras_96
    Replies:
    5
    Views:
    2,196
    Taras_96
    Apr 14, 2005
  3. john
    Replies:
    7
    Views:
    707
  4. =?iso-8859-1?b?Sulq6Q==?=

    [EXCEL] sequential =?iso-8859-1?q?acc=E8s=2E?=

    =?iso-8859-1?b?Sulq6Q==?=, Jan 23, 2004, in forum: Perl
    Replies:
    0
    Views:
    721
    =?iso-8859-1?b?Sulq6Q==?=
    Jan 23, 2004
  5. ssubbarayan
    Replies:
    5
    Views:
    2,367
    Dave Hansen
    Nov 3, 2009
Loading...

Share This Page