LWP::Simple getstore with absolute path not working

Discussion in 'Perl Misc' started by emrefan, Jun 14, 2008.

  1. emrefan

    emrefan Guest

    I am using the following line of sort in a batch file to fetch things
    off the network quickly and mostly I have success but if I need to
    indicate an absolute path in the URL, I always get a "not found"
    error. I have perl 5.8.x, URI.pm 1.36 and URI::URL.pm 5.03 on the XP
    system
    where did the test. Help!

    perl -MLWP::Simple -e "getstore( '%1', '%2' );"

    Oh, the exact URL that I tested it was something like this:
    ftp://user://dir-under-the-root/some-dir/some-file

    I thought the "//" before "dir-under-the-root" should have it working
    for me, but no.
     
    emrefan, Jun 14, 2008
    #1
    1. Advertising

  2. emrefan

    Ben Morrow Guest

    Quoth emrefan <>:
    > I am using the following line of sort in a batch file to fetch things
    > off the network quickly and mostly I have success but if I need to
    > indicate an absolute path in the URL, I always get a "not found"
    > error. I have perl 5.8.x, URI.pm 1.36 and URI::URL.pm 5.03 on the XP
    > system
    > where did the test. Help!
    >
    > perl -MLWP::Simple -e "getstore( '%1', '%2' );"
    >
    > Oh, the exact URL that I tested it was something like this:
    >
    > ftp://user://dir-under-the-root/some-dir/some-file
    >
    > I thought the "//" before "dir-under-the-root" should have it working
    > for me, but no.


    FTP urls don't work like that. The standard hack is

    ftp://host//dir-under-the-root/some-dir/some-file

    where %2f is a url-encoded '/'; but strictly speaking this requires your
    FTP server to support a chdir to '/dir-under-the-root' in one go, which
    is not required. Most do, however.

    Ben

    --
    We do not stop playing because we grow old;
    we grow old because we stop playing.
     
    Ben Morrow, Jun 14, 2008
    #2
    1. Advertising

  3. emrefan

    szr Guest

    Ben Morrow wrote:
    > Quoth emrefan <>:
    >> I am using the following line of sort in a batch file to fetch things
    >> off the network quickly and mostly I have success but if I need to
    >> indicate an absolute path in the URL, I always get a "not found"
    >> error. I have perl 5.8.x, URI.pm 1.36 and URI::URL.pm 5.03 on the XP
    >> system
    >> where did the test. Help!
    >>
    >> perl -MLWP::Simple -e "getstore( '%1', '%2' );"
    >>
    >> Oh, the exact URL that I tested it was something like this:
    >>
    >> ftp://user://dir-under-the-root/some-dir/some-file
    >>
    >> I thought the "//" before "dir-under-the-root" should have it working
    >> for me, but no.

    >
    > FTP urls don't work like that. The standard hack is


    Actually the url he gave is mostly correct (should be just one "/" after
    the host par:

    ftp://user:/dir-under-the-root/some-dir/some-file

    This should work just fine.

    You may want to test it in a browser and then in your script add checks
    to catch cases where adir or file doesn't exist.

    > ftp://host//dir-under-the-root/some-dir/some-file
    >
    > where %2f is a url-encoded '/'; but strictly speaking this requires
    > your FTP server to support a chdir to '/dir-under-the-root' in one
    > go, which is not required. Most do, however.



    Why would it need to do that? Give that url, it should be going to
    "/fdir-under-the-root/some-dir/some-file" in one go.

    --
    szr
     
    szr, Jun 15, 2008
    #3
  4. emrefan

    emrefan Guest

    On Jun 16, 5:55 am, "szr" <> wrote:
    > Ben Morrow wrote:
    > > Quoth emrefan <>:
    > >> I am using the following line of sort in a batch file to fetch things
    > >> off the network quickly and mostly I have success but if I need to
    > >> indicate an absolute path in the URL, I always get a "not found"
    > >> error.  I have perl 5.8.x, URI.pm 1.36 and URI::URL.pm 5.03 on the XP
    > >> system
    > >> where did the test. Help!

    >
    > >>                     perl -MLWP::Simple -e "getstore( '%1', '%2' );"

    >
    > >> Oh, the exact URL that I tested it was something like this:

    >
    > >> ftp://user://dir-under-the-root/some-dir/some-file

    >
    > >> I thought the "//" before "dir-under-the-root" should have it working
    > >> for me, but no.

    >
    > > FTP urls don't work like that. The standard hack is

    >
    > Actually the url he gave is mostly correct (should be just one "/" after
    > the host par:
    >
    > ftp://user:/dir-under-the-root/some-dir/some-file
    >
    > This should work just fine.
    >
    > You may want to test it in a browser and then in your script add checks
    > to catch cases where  adir or file doesn't exist.
    >
    > >    ftp://host//dir-under-the-root/some-dir/some-file

    >
    > > where %2f is a url-encoded '/'; but strictly speaking this requires
    > > your FTP server to support a chdir to '/dir-under-the-root' in one
    > > go, which is not required. Most do, however.

    >
    > Why would it need to do that? Give that url, it should be going to
    > "/fdir-under-the-root/some-dir/some-file" in one go.


    When a user logs in via ftp (including when the user is "anonymous"),
    he is placed in wherever is considered his home directory and if this
    home directory does not happen to be "/", then the URL with just a
    slash after the hostname is not going to work as intended. I tested
    that, had problems and so queried. With an "%2f" after the hostname,
    it worked. I reckond I'd call that a bug in URI.pm & co. but at least
    I now have a workaround.
     
    emrefan, Jun 16, 2008
    #4
  5. emrefan

    emrefan Guest

    On Jun 14, 11:53 pm, Ben Morrow <> wrote:
    > Quoth emrefan <>:
    >
    > > I am using the following line of sort in a batch file to fetch things
    > > off the network quickly and mostly I have success but if I need to
    > > indicate an absolute path in the URL, I always get a "not found"
    > > error.  I have perl 5.8.x, URI.pm 1.36 and URI::URL.pm 5.03 on the XP
    > > system
    > > where did the test. Help!

    >
    > >                     perl -MLWP::Simple -e "getstore('%1', '%2' );"

    >
    > > Oh, the exact URL that I tested it was something like this:

    >
    > > ftp://user://dir-under-the-root/some-dir/some-file

    >
    > > I thought the "//" before "dir-under-the-root" should have it working
    > > for me, but no.

    >
    > FTP urls don't work like that. The standard hack is
    >
    >    ftp://host//dir-under-the-root/some-dir/some-file
    >
    > where %2f is a url-encoded '/'; but strictly speaking this requires your
    > FTP server to support a chdir to '/dir-under-the-root' in one go, which
    > is not required. Most do, however.


    Thanks! It worked. I think I bumped into a bug in URI.pm & co, but at
    least I now have a workaround. :)
     
    emrefan, Jun 16, 2008
    #5
  6. emrefan

    szr Guest

    emrefan wrote:
    > On Jun 16, 5:55 am, "szr" <> wrote:
    >> Ben Morrow wrote:
    >>> Quoth emrefan <>:
    >>>> I am using the following line of sort in a batch file to fetch
    >>>> things off the network quickly and mostly I have success but if I
    >>>> need to indicate an absolute path in the URL, I always get a "not
    >>>> found" error. I have perl 5.8.x, URI.pm 1.36 and URI::URL.pm 5.03
    >>>> on the XP system
    >>>> where did the test. Help!

    >>
    >>>> perl -MLWP::Simple -e "getstore( '%1', '%2' );"

    >>
    >>>> Oh, the exact URL that I tested it was something like this:

    >>
    >>>> ftp://user://dir-under-the-root/some-dir/some-file

    >>
    >>>> I thought the "//" before "dir-under-the-root" should have it
    >>>> working for me, but no.

    >>
    >>> FTP urls don't work like that. The standard hack is

    >>
    >> Actually the url he gave is mostly correct (should be just one "/"
    >> after the host par:
    >>
    >> ftp://user:/dir-under-the-root/some-dir/some-file
    >>
    >> This should work just fine.
    >>
    >> You may want to test it in a browser and then in your script add
    >> checks to catch cases where adir or file doesn't exist.
    >>
    >>> ftp://host//dir-under-the-root/some-dir/some-file

    >>
    >>> where %2f is a url-encoded '/'; but strictly speaking this requires
    >>> your FTP server to support a chdir to '/dir-under-the-root' in one
    >>> go, which is not required. Most do, however.

    >>
    >> Why would it need to do that? Give that url, it should be going to
    >> "/fdir-under-the-root/some-dir/some-file" in one go.

    >
    > When a user logs in via ftp (including when the user is "anonymous"),
    > he is placed in wherever is considered his home directory


    Actually this is server/config (and client/config) dependant. For
    example on my personal system, if I login in as a run of the mill user,
    it drops me into my home (actually it chroots unless otherwise specified
    for that user in the ftpd config.)

    > and if this home directory does not happen to be "/", then the URL
    > with just a slash after the hostname is not going to work as intended.


    That's because it expects an absolute path. Depending on the ftp
    server's configuration, the ftp root that your login "sees" could be in
    fact the home dir itself (which is the case when some form of chroot'ing
    is used), or the true root of the system, in which case you referernce
    your home dir be something like /home/user, depending on the the
    platform/OS.

    If you want to access directory 'foo' that resides in your home dir, and
    your ftp's root / is chroot'ed to your home dir, they this should work:

    ftp://server/foo

    else, if you are not chrooted (where the ftp's root is the real root of
    the system), then

    ftp://server/home/johnsmith/foo

    is is what you want.

    Again, it depends on what / (root) for your ftp login is, which is
    dependant on the ftp server software and the platform it runs on.


    > I tested that, had problems and so queried. With an "%2f" after
    > the hostname, it worked. I reckond I'd call that a bug in
    > URI.pm & co. but at least I now have a workaround.


    When you login manually via ftp with the same account, and type "cd /"
    and they "pwd" or "ls", where do you see your self, in your home
    directory, or in the server's real root directory?

    Also, do you know what platform your ftp server is running on?

    --
    szr
     
    szr, Jun 16, 2008
    #6
  7. On 2008-06-16 06:34, emrefan <> wrote:
    > On Jun 16, 5:55 am, "szr" <> wrote:
    >> You may want to test it in a browser and then in your script add checks
    >> to catch cases where  adir or file doesn't exist.
    >>
    >> >    ftp://host//dir-under-the-root/some-dir/some-file

    >>
    >> > where %2f is a url-encoded '/'; but strictly speaking this requires
    >> > your FTP server to support a chdir to '/dir-under-the-root' in one
    >> > go, which is not required. Most do, however.


    It should support a chdir to / (if it has a directory called "/"), so
    you should be able to write
    ftp://host///dir-under-the-root/some-dir/some-file
    then.


    >> Why would it need to do that? Give that url, it should be going to
    >> "/fdir-under-the-root/some-dir/some-file" in one go.

    >
    > When a user logs in via ftp (including when the user is "anonymous"),
    > he is placed in wherever is considered his home directory and if this
    > home directory does not happen to be "/", then the URL with just a
    > slash after the hostname is not going to work as intended. I tested
    > that, had problems and so queried. With an "%2f" after the hostname,
    > it worked. I reckond I'd call that a bug in URI.pm & co. but at least
    > I now have a workaround.


    It's not a bug. That's how FTP URIs are supposed to work. To quote from
    RFC 1738:

    | The url-path of a FTP URL has the following syntax:
    |
    | <cwd1>/<cwd2>/.../<cwdN>/<name>;type=<typecode>
    |
    | Where <cwd1> through <cwdN> and <name> are (possibly encoded) strings
    | and <typecode> is one of the characters "a", "i", or "d".
    [...]
    | The url-path is interpreted as a series of FTP commands as follows:
    |
    | Each of the <cwd> elements is to be supplied, sequentially, as the
    | argument to a CWD (change working directory) command.
    |
    | If the typecode is "d", perform a NLST (name list) command with
    | <name> as the argument, and interpret the results as a file
    | directory listing.
    |
    | Otherwise, perform a TYPE command with <typecode> as the argument,
    | and then access the file whose name is <name> (for example, using
    | the RETR command.)

    So ftp://host//dir-under-the-root/some-dir/some-file means:

    Connect to host and log in as anonymous.
    cwd /dir-under-the-root
    cwd some-dir
    retr some-file

    Note that /dir-under-the-root is only a directory under the root on unix
    and window systems. Other OSs may have other conventions (once upon a
    time, FTP servers running VMS or IBM host OSs were common), and the host
    OS may not even have the concept of a "root directory".

    hp
     
    Peter J. Holzer, Jun 16, 2008
    #7
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. James Byrne
    Replies:
    3
    Views:
    565
    James Byrne
    Sep 14, 2010
  2. Dan
    Replies:
    3
    Views:
    134
    Brian Wakem
    Aug 19, 2005
  3. Larry Snider

    LWP::Simple Not Working

    Larry Snider, Dec 30, 2003, in forum: Perl Misc
    Replies:
    20
    Views:
    317
    Matt Garrish
    Jan 4, 2004
  4. Perl Mon
    Replies:
    4
    Views:
    230
    Perl Mon
    Jan 11, 2005
  5. jwcarlton

    LWP::Simple - relative or absolute path?

    jwcarlton, Jan 17, 2011, in forum: Perl Misc
    Replies:
    3
    Views:
    192
    Peter J. Holzer
    Jan 17, 2011
Loading...

Share This Page