Sequential analysis in perl?

[D Cameron]

I just started writing perl a couple of weeks ago when I walked on to a
project that was using perl to extract data from a raw data file. It has
been a dream for text processing, as advertised. But after the data is
extracted I want to do some statistical tests of the sequential analysis
variety, and I'm not sure if perl is the best way to go. I was hoping
some people here could offer some opinions.

The questions I want to answer are along the lines of showing that a
Markov chain is of order r, and not order r-1; looking at individual
cells of a contingency table to see which ones are significant; and
calculating the likelihood ratio chi-square (LRX^2).

The calculations involve a lot of matrix manipulation which I'm
currently doing with 2D arrays and lots of for(i;i<n;i++) loops. I've
searched around CPAN a bit but the Statistics and Algorithm namespaces
didn't offer what I was looking for. After getting great results from
Algorithm::Cluster I was hoping I'd find something similar for
sequential analysis.

My perl knowledge is limited, I'm only around chapter 4 of the Camel
book. The entertaining writing has me reading every page, but I was
quite impressed to find I could do a
%results{$picture}{$question}[$prev][$curr] structure without too many
complaints.

Basically I'm wondering:
-Does the problem sound appropriate for perl, or should I go looking
somewhere else?
-Am I missing something about perl that would simplify my code
significantly?
-Can anyone suggest packages that would be useful that I've overlooked?

In case my statistical jargon is unclear, I'll mention my reference here:
Gottman & Roy (1990) Sequential Analysis: A guide for behavioral
researchers. New York: Cambridge University Press.

 

[Anno Siegel]

I just started writing perl a couple of weeks ago when I walked on to a
project that was using perl to extract data from a raw data file. It has
been a dream for text processing, as advertised. But after the data is
extracted I want to do some statistical tests of the sequential analysis
variety, and I'm not sure if perl is the best way to go. I was hoping
some people here could offer some opinions.

The questions I want to answer are along the lines of showing that a
Markov chain is of order r, and not order r-1; looking at individual
cells of a contingency table to see which ones are significant; and
calculating the likelihood ratio chi-square (LRX^2).

The calculations involve a lot of matrix manipulation which I'm
currently doing with 2D arrays and lots of for(i;i<n;i++) loops. I've
[...]

-Am I missing something about perl that would simplify my code
significantly?
-Can anyone suggest packages that would be useful that I've overlooked?

Take a look at PDL on CPAN.

Anno

 

[Jim Keenan]

[snip]

Are the links at this Google search for: Markov Perl "sequential
analysis" of any use?
http://www.google.com/search?as_q=M...s_occt=any&as_dt=i&as_sitesearch=&safe=images

Jim Keenan

 

[David K. Wall]

I just started writing perl a couple of weeks ago when I walked on
to a project that was using perl to extract data from a raw data
file. It has been a dream for text processing, as advertised. But
after the data is extracted I want to do some statistical tests of
the sequential analysis variety, and I'm not sure if perl is the
best way to go. I was hoping some people here could offer some
opinions.

The questions I want to answer are along the lines of showing that
a Markov chain is of order r, and not order r-1; looking at
individual cells of a contingency table to see which ones are
significant; and calculating the likelihood ratio chi-square
(LRX^2).

The calculations involve a lot of matrix manipulation which I'm
currently doing with 2D arrays and lots of for(i;i<n;i++) loops.
I've searched around CPAN a bit but the Statistics and Algorithm
namespaces didn't offer what I was looking for. After getting
great results from Algorithm::Cluster I was hoping I'd find
something similar for sequential analysis.

You might take a look at the R language: http://www.r-project.org/

It may be more appropriate for your purpose. There's no reason you
can't use Perl to massage your data into a usable form and then use R
as the analysis language.

 

[D Cameron]

Unfortunately I lot of the stuff google came up with are link farms.
Maybe this is time to bust out Vivissimo. Anyway, I did find out about R
and the MatLab packages through a similar search, which is what prompted
me to start questioning perl altogether.

Dave

 

[D Cameron]

Thanks for the R suggestion! I'd run across a reference to it in one of
the readmes for a Statistics:: module and was thinking it might be worth
looking at. We had been hoping to get the process down to a one step
batch sort of thing, but even two steps would be better than what they
had before.

Dave