automating a perl installation on a cluster (or using non-standard nfs paths)

Discussion in 'Perl Misc' started by Rahul, Feb 6, 2009.

  1. Rahul

    Rahul Guest

    I am configure a perl code (check_lm_sensors; a nagios plugin) so that I
    can eventually roll it out on 200+ servers we have in our computer
    cluster. The problem is that there are several dependancies. eg. the CPAN
    modules Nagios::plugin, Class::Accessor, Config::Tiny, version.

    I already have one machine running where I used "perl -MCPAN -e 'install
    Math::Calc::Units'" etc. for each dependancy that the original code
    complained about. What's a better way of doing this so that I can
    automate the process?

    My approach:
    All the nodes have a common nfs mounter directory too (/usr/local/) so I
    thought of doing manual compiles once to this non-standard location by
    using " perl Makefile.PL PREFIX=/usr/local/lib/perl; make; make test;
    make install"

    Unfortunately then the next programs down the line don't seem to find the
    modules they need where they expect to find them. eg.

    ######error########
    Can't locate Math/Calc/Units/Convert.pm in @INC (@INC contains:
    /usr/lib64/perl5/site_perl/5.8.8/x86_64-linux-thread-multi
    /usr/lib64/perl5/site_perl/5.8.7/x86_64-linu
    #########################

    I tried " setenv PERL5LIB /usr/local/lib/perl" but many modules seem to
    be installed deep down this heirarchy. PERL5LIB based searches do not
    seem recursive?

    Example: /usr/lib/perl5/site_perl/5.8.8/Math/Calc/Units/Convert.pm

    So my $PERL5LIB does not help much unless I were to set each deep
    location in there. Besides, when I manually browsed I find non-perl-
    module files like "Grammar.y" that I am not sure if I need copied or not.

    What's the best way to handle these twin related problems: (1) Automating
    a multi-dependancy perl install to many servers (2) using non-standard
    paths.

    Any advice?


    --
    Rahul
     
    Rahul, Feb 6, 2009
    #1
    1. Advertising

  2. Re: automating a perl installation on a cluster (or using non-standardnfs paths)

    Rahul wrote:
    >
    > My approach:
    > All the nodes have a common nfs mounter directory too (/usr/local/) so I
    > thought of doing manual compiles once to this non-standard location by
    > using " perl Makefile.PL PREFIX=/usr/local/lib/perl; make; make test;
    > make install"


    I don't understand what the independent purposes of PREFIX and LIB are,
    but it seems you have set both of them, not just PREFIX, or things go
    wrong, in the way you described.

    > Unfortunately then the next programs down the line don't seem to find the
    > modules they need where they expect to find them. eg.
    >
    > ######error########
    > Can't locate Math/Calc/Units/Convert.pm in @INC (@INC contains:
    > /usr/lib64/perl5/site_perl/5.8.8/x86_64-linux-thread-multi
    > /usr/lib64/perl5/site_perl/5.8.7/x86_64-linu
    > #########################
    >
    > I tried " setenv PERL5LIB /usr/local/lib/perl" but many modules seem to
    > be installed deep down this heirarchy. PERL5LIB based searches do not
    > seem recursive?


    It is, at least in some circumstances, but the recursion is rules-based,
    not exhaustive. You can't just set PERL5LIB to be / and have it scour
    your entire system! you might need to set PERL5LIB to some thing like
    /usr/local/lib/perl/lib to have it work right.

    >
    > Example: /usr/lib/perl5/site_perl/5.8.8/Math/Calc/Units/Convert.pm


    Ah, what is that an example of? "/usr/local/lib/perl" ne
    "/usr/lib/perl5/site_perl"



    Xho
     
    Xho Jingleheimerschmidt, Feb 7, 2009
    #2
    1. Advertising

  3. Rahul

    Rahul Guest

    Re: automating a perl installation on a cluster (or using non-standard nfs paths)

    Sam <> wrote in
    news:-scan.com:

    > Use your Linux distribution's native software update and maintenance
    > tools.
    >
    > With RPM/yum based distros, for example, provided that you package
    > your Perl module properly, and define your dependencies, and you have
    > your standard yum repositories enabled, "yum install
    > perl-yourmodule.rpm" will automatically download and install all of
    > your dependencies, if necessary.


    Ah! Thanks Sam! I didn't realize yum maintained CPAN modules too. I thought
    "CPAN was the yum for perl!"

    >
    > The best way to handle non-standard paths is to eliminate them. Use
    > your standard Perl install, and use the built-in, tested tools, that
    > are part of your distribution.
    >


    I don't agree. Vut maybe I am wrong! Standardization has its avantages. But
    one needs to make a case b case exception sometimes. In my case if I go
    with the standardized paths it will mean 200 installs locally on each
    remote host. That's duplication of effort, netork bandwidth and hard disk
    space. Future updates will again have to be pushed to all 200 machines.
    Besides this way it is harder to maintain module consistancy rather than if
    I just install thinggs centrally.

    Thus for me the NFS mount to a custom dir approach does seem the better way
    to go. Feel free to point out flaws. Maybe I am wrong!

    --
    Rahul
     
    Rahul, Feb 8, 2009
    #3
  4. Rahul

    Rahul Guest

    Ben Morrow <> wrote in news:edq166-a0h.ln1
    @osiris.mauzo.dyndns.org:

    > Specifically, an entry /foo in PERL5LIB will also include
    > /foo/VERSION/ARCH, /foo/VERSION and /foo/ARCH where VERSION is something
    > like 5.8.8 and ARCH is something like i386-linux. It will also include
    > directories for any compatible previous versions that were found at
    > Configure time: you can get the list with perl -V:inc_version_list. This
    > is exactly the same logic as used by 'use lib'.
    >


    Thanks Ben. That explains it! When I looked up PERL5LIB online the
    explaination I found was too terse to explain that underlying logic. Now I
    know more!

    --
    Rahul
     
    Rahul, Feb 8, 2009
    #4
  5. Rahul

    Rahul Guest

    Ben Morrow <> wrote in news:edq166-a0h.ln1
    @osiris.mauzo.dyndns.org:

    > Recentish versions of MakeMaker (since 6.31) have an INSTALL_BASE
    > parameter which actually does the right thing: INSTALL_BASE=/usr/local
    > is probably what you want in this case. PREFIX is rather weird, in that
    > it tries to emulate the structure of your system perl installation,
    > rather than using a normal layout.
    >


    I tried installing ExtUtils-MakeMaker-6.48, the first step in all my
    dependancies. But I get this error when I try "make" that I can't really
    figure out.

    [root@star255 ExtUtils-MakeMaker-6.48]# perl Makefile.PL
    INSTALL_BASE=/usr/local
    Using included version of ExtUtils::Command (1.14) as it is newer than the
    installed version (1.09).
    Writing Makefile for ExtUtils::MakeMaker

    [root@star255 ExtUtils-MakeMaker-6.48]# make
    make: *** No rule to make target `/usr/lib64/perl5/5.8.8/x86_64-linux-
    thread-multi/CORE/config.h', needed by `Makefile'. Stop.


    What am I doing wrong here?

    --
    Rahul
     
    Rahul, Feb 9, 2009
    #5
  6. Rahul

    Rahul Guest

    Ben Morrow <> wrote in news:edq166-a0h.ln1
    @osiris.mauzo.dyndns.org:

    > Specifically, an entry /foo in PERL5LIB will also include
    > /foo/VERSION/ARCH, /foo/VERSION and /foo/ARCH where VERSION is something
    > like 5.8.8 and ARCH is something like i386-linux. It will also include
    > directories for any compatible previous versions that were found at
    > Configure time: you can get the list with perl -V:inc_version_list. This
    > is exactly the same logic as used by 'use lib'.
    >
    >


    I found this quite useful article about automating perl module installs. In
    case it helps anybody else in a similar situation:

    http://www.ibm.com/developerworks/opensource/library/l-depperl.html

    It does seem that deploying perl across a cluster of machines is quite a
    difficult job!

    --
    Rahul
     
    Rahul, Feb 10, 2009
    #6
  7. Rahul

    Rahul Guest

    Ben Morrow <> wrote in news:97f966-lu1.ln1
    @osiris.mauzo.dyndns.org:

    > Does that file exist? What does


    Thanks again Ben! locate on config.h does not throw any such file.

    > perl -V:archlib
    >
    > print? Does the file CORE/config.h exist in that directory?


    #>perl -V:archlib
    archlib='/usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi';

    ANd no config.h to be found once again!

    #>ls /usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi/CORE/
    libperl.so reentr.inc sperl.o


    > Do you have either of PERL5LIB or PERL_MM_OPT set in your environment
    > (or any other vars beginning with PERL)?


    Nope. No PERL vars set.

    export | grep PERL throws a null.




    --
    Rahul
     
    Rahul, Feb 11, 2009
    #7
  8. Rahul

    Rahul Guest

    Ben Morrow <> wrote in news:97f966-lu1.ln1
    @osiris.mauzo.dyndns.org:

    > If you don't have the perl headers installed, your OS may have split up
    > the perl distribution in a misguided effort to save space. You may need
    > to install some sort of perl-dev or perl-headers package to get them.
    >


    Oh! That was it.

    I needed a " yum install perl-devel.x86_64"

    Now for ExtUtils-MakeMaker-6.48 the make works!

    Strange! I'd never have figured this out by myself! Thanks for all your
    help Ben! Let's see if I can finally get perl to install modules to a non-
    default location!

    This seems like a very uphill battle!

    --
    Rahul
     
    Rahul, Feb 11, 2009
    #8
  9. Rahul

    Rahul Guest

    Ben Morrow <> wrote in
    news::

    > This is probably because you set PREFIX wrong. You've probably ended
    > up installing somewhere like
    > /usr/local/lib/perl/lib64/perl5/site_perl/5.8.8/x86_64-linux-thread-mul
    > ti, which is several levels deeper than you meant. If you use
    > INSTALL_BASE=/usr/local you should instead get modules in
    > /usr/local/lib/perl5/x86_64-linux-thread-multi, which
    > PERL5LIB=/usr/local/lib/perl5 will find just fine (see below).
    >



    Almost. I get them in:

    /usr/local/lib/perl5/ExtUtils/MM.pm

    So very close to the "correct" path that you predicted. I'm not sure if
    the absence of "x86_64-linux-thread-multi" is indicative of any trouble!

    --
    Rahul
     
    Rahul, Feb 11, 2009
    #9
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Nik
    Replies:
    2
    Views:
    390
    John Harrison
    Jul 16, 2004
  2. Quentin Crain
    Replies:
    0
    Views:
    277
    Quentin Crain
    May 5, 2005
  3. Steven Hansen
    Replies:
    7
    Views:
    183
    Ezra Zygmuntowicz
    Nov 29, 2006
  4. Alex F.

    Using perl over network (NFS)

    Alex F., Feb 20, 2004, in forum: Perl Misc
    Replies:
    4
    Views:
    132
    Brian McCauley
    Feb 25, 2004
  5. Replies:
    2
    Views:
    191
    Big and Blue
    Nov 2, 2006
Loading...

Share This Page