slfjs

[kj]

I'm looking for the "moral equivalent" of the (fictitious) `openremote`
function below:

my $handle = openremote( 'http://some.domain.org/huge.tsv' ) or die $!;
while ( <$handle> ) {
chomp;
# etc.
# do stuff with $_
}
close $handle;

IOW, I'm looking for a way to open a read handle to a remote file
so that I can read from it *line-by-line*. (Typically this file
will be larger than I want to read all at once into memory. IOW,
I want to avoid solutions based on stuffing the value returned into
LWP::Simple::get into an IO::String.)

I'm sure this is really basic stuff, but I have not been able to
find it after a lot of searching.

TIA!

kj

 

[Jim Gibson]

I'm looking for the "moral equivalent" of the (fictitious) `openremote`
function below:

my $handle = openremote( 'http://some.domain.org/huge.tsv' ) or die $!;
while ( <$handle> ) {
chomp;
# etc.
# do stuff with $_
}
close $handle;

IOW, I'm looking for a way to open a read handle to a remote file
so that I can read from it *line-by-line*. (Typically this file
will be larger than I want to read all at once into memory. IOW,
I want to avoid solutions based on stuffing the value returned into
LWP::Simple::get into an IO::String.)

I'm sure this is really basic stuff, but I have not been able to
find it after a lot of searching.

You cannot read a file on a remote system line-by-line, barring NFS or
other file distribution system that mimics a local file system from a
remote system. The normal ways to read such a file are:

1. Download the remote file to your local system and read the copy.

2. Open a socket to the remote system and read the socket (but not
exactly a line-oriented protocol). This assume a cooperative socket
server on the remote system that will read the file and provide its
contents to you over the connected socket.

FTP and HTTP are two protocols that provide these services. They both
can be used from a Perl program. See the Net::FTP and LWP::UserAgent
modules. They both assume a corresponding service on the remote
computer.