Question about PDL

Discussion in 'Perl Misc' started by January Weiner, Feb 10, 2008.

  1. Hello,

    in my program, I am using a matrix which has usually a size of 2000x2000 or
    similar. It contains short integers. The matrix is filled in one by one in
    a loop over rows and columns.

    The only calculation that I am doing with this matrix is averaging
    over a window sliding along the diagonals. Finally, I need to access each
    matrix element one after another in a loop to calculate something.

    At first, I used a regular Perl matrix, constructed as [ [ .... ], [ ... ],
    ..... ]. This was quite slow and took a lot of memory (the memory footprint
    of my program grews up by roughly 50MB), so I googled and found PDL -- Perl
    Data Language.

    PDL is supposed to be much faster and to have a smaller memory footprint.

    While I can see the latter (the footprint is now negligible compared to the
    whole program), it is roughly three to four times slower than the regular
    perlish way. I am creating the matrix as follows [please bear with me -- I
    am not posting actual code, because it is rather complex, and you will see
    that answering my question doesn't require finding out whether my code is
    correct]:

    $matrix = short(zeroes($l1, $l2)) ;

    (where $l1 and $l2 are dimensions of the matrix), and accessing / setting
    the elements using at() and set():

    set( $matrix, $i, $j, $value ) ;
    $value = at( $matrix, $i, $j ) ;

    There is another way of doing it using PDL::NiceSlice, which uses
    constructs like $matrix->($i, $j), but I found it to be even slower.

    QUESTION: Is this a normal behaviour? Is it normal that a standard perlish
    matrix is few times faster than the PDL implementation? Or should I start
    finding out where I messed things up?

    Thank you in advance,

    January
     
    January Weiner, Feb 10, 2008
    #1
    1. Advertising

  2. January Weiner <> writes:

    > PDL is supposed to be much faster and to have a smaller memory footprint.
    >
    > While I can see the latter (the footprint is now negligible compared to the
    > whole program), it is roughly three to four times slower than the regular
    > perlish way. I am creating the matrix as follows [please bear with me -- I
    > am not posting actual code, because it is rather complex, and you will see
    > that answering my question doesn't require finding out whether my code is
    > correct]:
    >
    > $matrix = short(zeroes($l1, $l2)) ;
    >
    > (where $l1 and $l2 are dimensions of the matrix), and accessing / setting
    > the elements using at() and set():
    >
    > set( $matrix, $i, $j, $value ) ;
    > $value = at( $matrix, $i, $j ) ;
    >
    > There is another way of doing it using PDL::NiceSlice, which uses
    > constructs like $matrix->($i, $j), but I found it to be even slower.


    If I'm reading you correctly, all your calculations involve getting a
    value from the matrix, processing it in pure perl and then setting it
    back. That is not how you get good speed from PDL. You want to move as
    many calculations as you can to the PDL operators, especially those
    operators that can act on groups of values.

    I only have very limited knowledge of PDL, but it seems to me that you
    could probably do your averaging by using the PDL projection
    operators. See

    <http://www.johnlapeyre.com/pdl/pdldoc/newbook/node4.html#SECTION00440000000000000000>

    and the section immediately after that.

    --
    Joost Diepenmaat | blog: http://joost.zeekat.nl/ | work: http://zeekat.nl/
     
    Joost Diepenmaat, Feb 10, 2008
    #2
    1. Advertising

  3. Joost Diepenmaat <> wrote:
    > If I'm reading you correctly, all your calculations involve getting a
    > value from the matrix, processing it in pure perl and then setting it
    > back. That is not how you get good speed from PDL. You want to move as
    > many calculations as you can to the PDL operators, especially those
    > operators that can act on groups of values.


    Yes, I do it now. The operation that I needed was conv2d from PDL::Image2D,
    with a convolution matrix of the form zeroes($window,$window)->diagonal++.

    However, the other big advantage from PDL is for me the reduction of the
    memory footprint, which is substantial.

    I'm now stuck with the old conundrum: low memory footprint OR speed.

    j.

    > I only have very limited knowledge of PDL, but it seems to me that you
    > could probably do your averaging by using the PDL projection
    > operators. See


    > <http://www.johnlapeyre.com/pdl/pdldoc/newbook/node4.html#SECTION00440000000000000000>


    > and the section immediately after that.


    > --
    > Joost Diepenmaat | blog: http://joost.zeekat.nl/ | work: http://zeekat.nl/
     
    January Weiner, Feb 10, 2008
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Junmou Zhang
    Replies:
    0
    Views:
    1,424
    Junmou Zhang
    Jul 10, 2003
  2. Mark Ohlund

    PDL function call failing

    Mark Ohlund, May 2, 2004, in forum: Perl
    Replies:
    1
    Views:
    478
    Slaven Rezic
    May 2, 2004
  3. Jose Quesada
    Replies:
    1
    Views:
    133
    Joel VanderWerf
    Sep 20, 2003
  4. Jeff Silverman

    I am having problems installing PDL using CPAN

    Jeff Silverman, Nov 11, 2003, in forum: Perl Misc
    Replies:
    0
    Views:
    142
    Jeff Silverman
    Nov 11, 2003
  5. Keflavich
    Replies:
    0
    Views:
    129
    Keflavich
    Feb 15, 2007
Loading...

Share This Page