LWP::Simple getstore with absolute path not working

Discussion in 'Perl Misc' started by emrefan, Jun 14, 2008.

  1. emrefan

    emrefan Guest

    I am using the following line of sort in a batch file to fetch things
    off the network quickly and mostly I have success but if I need to
    indicate an absolute path in the URL, I always get a "not found"
    error. I have perl 5.8.x, URI.pm 1.36 and URI::URL.pm 5.03 on the XP
    where did the test. Help!

    perl -MLWP::Simple -e "getstore( '%1', '%2' );"

    Oh, the exact URL that I tested it was something like this:

    I thought the "//" before "dir-under-the-root" should have it working
    for me, but no.
    emrefan, Jun 14, 2008
    1. Advertisements

  2. emrefan

    Ben Morrow Guest

    FTP urls don't work like that. The standard hack is


    where %2f is a url-encoded '/'; but strictly speaking this requires your
    FTP server to support a chdir to '/dir-under-the-root' in one go, which
    is not required. Most do, however.

    Ben Morrow, Jun 14, 2008
    1. Advertisements

  3. emrefan

    szr Guest

    Actually the url he gave is mostly correct (should be just one "/" after
    the host par:


    This should work just fine.

    You may want to test it in a browser and then in your script add checks
    to catch cases where adir or file doesn't exist.

    Why would it need to do that? Give that url, it should be going to
    "/fdir-under-the-root/some-dir/some-file" in one go.
    szr, Jun 15, 2008
  4. emrefan

    emrefan Guest

    When a user logs in via ftp (including when the user is "anonymous"),
    he is placed in wherever is considered his home directory and if this
    home directory does not happen to be "/", then the URL with just a
    slash after the hostname is not going to work as intended. I tested
    that, had problems and so queried. With an "%2f" after the hostname,
    it worked. I reckond I'd call that a bug in URI.pm & co. but at least
    I now have a workaround.
    emrefan, Jun 16, 2008
  5. emrefan

    emrefan Guest

    Thanks! It worked. I think I bumped into a bug in URI.pm & co, but at
    least I now have a workaround. :)
    emrefan, Jun 16, 2008
  6. emrefan

    szr Guest

    Actually this is server/config (and client/config) dependant. For
    example on my personal system, if I login in as a run of the mill user,
    it drops me into my home (actually it chroots unless otherwise specified
    for that user in the ftpd config.)
    That's because it expects an absolute path. Depending on the ftp
    server's configuration, the ftp root that your login "sees" could be in
    fact the home dir itself (which is the case when some form of chroot'ing
    is used), or the true root of the system, in which case you referernce
    your home dir be something like /home/user, depending on the the

    If you want to access directory 'foo' that resides in your home dir, and
    your ftp's root / is chroot'ed to your home dir, they this should work:


    else, if you are not chrooted (where the ftp's root is the real root of
    the system), then


    is is what you want.

    Again, it depends on what / (root) for your ftp login is, which is
    dependant on the ftp server software and the platform it runs on.

    When you login manually via ftp with the same account, and type "cd /"
    and they "pwd" or "ls", where do you see your self, in your home
    directory, or in the server's real root directory?

    Also, do you know what platform your ftp server is running on?
    szr, Jun 16, 2008
  7. It should support a chdir to / (if it has a directory called "/"), so
    you should be able to write

    It's not a bug. That's how FTP URIs are supposed to work. To quote from
    RFC 1738:

    | The url-path of a FTP URL has the following syntax:
    | <cwd1>/<cwd2>/.../<cwdN>/<name>;type=<typecode>
    | Where <cwd1> through <cwdN> and <name> are (possibly encoded) strings
    | and <typecode> is one of the characters "a", "i", or "d".
    | The url-path is interpreted as a series of FTP commands as follows:
    | Each of the <cwd> elements is to be supplied, sequentially, as the
    | argument to a CWD (change working directory) command.
    | If the typecode is "d", perform a NLST (name list) command with
    | <name> as the argument, and interpret the results as a file
    | directory listing.
    | Otherwise, perform a TYPE command with <typecode> as the argument,
    | and then access the file whose name is <name> (for example, using
    | the RETR command.)

    So ftp://host/%2fdir-under-the-root/some-dir/some-file means:

    Connect to host and log in as anonymous.
    cwd /dir-under-the-root
    cwd some-dir
    retr some-file

    Note that /dir-under-the-root is only a directory under the root on unix
    and window systems. Other OSs may have other conventions (once upon a
    time, FTP servers running VMS or IBM host OSs were common), and the host
    OS may not even have the concept of a "root directory".

    Peter J. Holzer, Jun 16, 2008
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.