treat array as a file

[Jeevan]

Hi,

I have some data which I am getting from a socket. I am currently
storing the
data in an array (so that future reading of the data will be fast as
it will
be in RAM instead of hard disk). Is there any way I can treat this
array as a file i.e. can I apply file operations (like fopen, fscanf
etc) on this array
without having to write the data into a physical file and then read
the file from the disk.

Thanks
Jeevan.

 

[Irrwahn Grausewitz]

(e-mail address removed) (Jeevan) wrote in

Hi,

I have some data which I am getting from a socket. I am currently
storing the
data in an array (so that future reading of the data will be fast as
it will
be in RAM instead of hard disk). Is there any way I can treat this
array as a file i.e. can I apply file operations (like fopen, fscanf
etc) on this array

No way. But why should one want to do file operations on an array?
As you said, it resides in RAM, so you've got random access anyway ...
and maintaining a read/write index to a position in an array is no
big deal, is it? If you prefer, you could easily write functions
similar to those you mentioned, working on arrays instead of files.
Hint: read about sscanf(), strto*(), str*chr(), etc.

without having to write the data into a physical file and then read
the file from the disk.

You can do this as well. It's like flying from Australia to Bali
via Norway.

Thanks
Jeevan.

Regards

Irrwahn.

 

[Eric Sosman]

Hi,

I have some data which I am getting from a socket. I am currently
storing the
data in an array (so that future reading of the data will be fast as
it will
be in RAM instead of hard disk). Is there any way I can treat this
array as a file i.e. can I apply file operations (like fopen, fscanf
etc) on this array
without having to write the data into a physical file and then read
the file from the disk.

You can apply fopen() to anything your file system
will recognize as a named file, but to nothing else. If
your system supports file names like "RAM:08112200-08113fff"
or "/proc/1234/as" you're all set. Otherwise, you'll need
to find a different approach.

 

[Kevin Bracey]

I have some data which I am getting from a socket. I am currently storing
the data in an array (so that future reading of the data will be fast as it
will be in RAM instead of hard disk). Is there any way I can treat this
array as a file i.e. can I apply file operations (like fopen, fscanf etc)
on this array without having to write the data into a physical file and
then read the file from the disk.

This is fairly frequently asked - what you're looking for is some sort of

FILE *fopenmem(void *ptr, size_t len, const char *mode);

call. It would be nice, but unfortunately it doesn't exist. At least not
in the ISO C standard. Kind of a shame, as most implementations would
find it fairly straightforward to implement by setting the internal buffer
pointers to the user's object.

Actually, that's given me an idea. How about using setvbuf to set the buffer
size to the size of the file? Then the C library should just read the whole
thing into the buffer and won't need to touch the disk again.

 

[Tom Zych]

This is fairly frequently asked - what you're looking for is some sort of

FILE *fopenmem(void *ptr, size_t len, const char *mode);

call. It would be nice, but unfortunately it doesn't exist. At least not
in the ISO C standard. Kind of a shame, as most implementations would
find it fairly straightforward to implement by setting the internal buffer
pointers to the user's object.

That would be a handy function at times. Anyone know if anyone has
implemented it in a library?

 

[Jeevan]

Thanks for the reply. Actually I am getting the data from a socket
which I am storing into an array. I found a round about method using
sockets and dup. The code is:

int duplicate = dup(sock);//sock is the socket descriptor
FILE *fp = fdopen(duplicate);

This worked in Unix but not in MS VC++ (dup is giving error). The
reason why I want to do file operations on data in array (or from
sockets) is that, there is a code which uses file operations (the data
to that code was in a file). Now I want to make the data coming from a
socket available to that code without making any changes to it.

Jeevan

 

[LibraryUser]

*** rude and evil top-posting fixed ***

Thanks for the reply. Actually I am getting the data from a socket
which I am storing into an array. I found a round about method using
sockets and dup. The code is:

int duplicate = dup(sock);//sock is the socket descriptor
FILE *fp = fdopen(duplicate);

This worked in Unix but not in MS VC++ (dup is giving error). The
reason why I want to do file operations on data in array (or from
sockets) is that, there is a code which uses file operations (the data
to that code was in a file). Now I want to make the data coming from a
socket available to that code without making any changes to it.

Please do not top post.

This problem is why your query is off-topic on c.l.c. Sockets,
particular compilers and operating systems are not mentioned in
the C standard, and thus such things are inherently non-portable.

Any attempts to answer such questions here almost invariably end
up giving mis-information in some form or other. Thus the
queries should be addressed to newsgroups dealing with your
particular system(s).

 

[Kevin Bracey]

In message <[email protected]>

This problem is why your query is off-topic on c.l.c. Sockets,
particular compilers and operating systems are not mentioned in
the C standard, and thus such things are inherently non-portable.

For God's sake, his query wasn't in the slightest off-topic. His original
question didn't involve sockets, compilers or operating systems. What's wrong
with you people? Have you just programmed some sort of automated script
to say every single query is off-topic if certain keywords occur in a thread?
Is it a sin to explain WHY he wants to do something, and what the background
to the query is? If he didn't people would just ask why anyway.

His original query was "how can I treat an array as a file". Last time I
checked, C was advanced enough to include both of those two hip new computing
concepts, so it shouldn't be too much of a heinous crime to ask in
comp.lang.c whether they can be interworked.

The answer is, unfortunately, there is no portable way of doing this in ISO C
(so by merely asking whether there was, you are automatically off-topic, and
I'm probably off-topic by telling you there isn't).

This is odd, as most implementations of the stdio library probably do this
internally to implement sprintf(); it's natural and easy given the buffering
requirements of stdio. It would make sense to expose this internal working
portably via some sort of

FILE *fmemopen(void *ptr, size_t len, const char *mode);

sprintf() could then be implemented as

int sprintf(char *buf, const char *fmt, ...)
{
FILE *f = fmemopen(buf, SIZE_MAX, "w");
va_list ap = va_start(ap, fmt);
int result = vfprintf(f, fmt, ap);
va_end(ap);
fclose(f);
}

Maybe the original poster would like to propose this as an addition to
<stdio.h> in comp.std.c.

As ISO C doesn't (yet) provide what you're looking for, your next port of
call should probably be a POSIX newsgroup to see if you can find a
POSIX-standard method, maybe involving connecting to the s*cket directly, and
then beyond that platform-specific newsgroups.

Chuck Falconer, on vacation.

Some vacation, sitting in a library telling poor innocents asking sensible
questions that they're off-topic. I suggest you take a vacation from your
vacation. Seriously.

 

[Glen Herrmannsfeldt]

Hi,

I have some data which I am getting from a socket. I am currently
storing the
data in an array (so that future reading of the data will be fast as
it will
be in RAM instead of hard disk). Is there any way I can treat this
array as a file i.e. can I apply file operations (like fopen, fscanf
etc) on this array
without having to write the data into a physical file and then read
the file from the disk.

There are some OS that supply this ability, but it is not standard. It
might be something like /dev/mem. One OS that I used to know that ran on a
machine with VME bus supplied device drivers for different VME address
spaces. (VME bus allows, for example separate decoding of a 24 bit address
space and a 32 bit address space. While they may overlap, that is not
required.) A device such as /dev/vme32d32 would allow 32 bit data cycles on
a 32 bit VME address space.

To use this feature, you would have to find where your array is in memory,
and fseek() to that place.

Again, completely non-portable.

There is also the non-portable mmap() that allows one to treat a disk file
as an array. The two features can even be combined!

-- glen

 

[Dave Thompson]

On 4 Sep 2003 19:50:09 -0700, (e-mail address removed) (Jeevan) wrote:

Please don't top-post. Context lost as a result.

Thanks for the reply. Actually I am getting the data from a socket
which I am storing into an array. I found a round about method using
sockets and dup. The code is:

int duplicate = dup(sock);//sock is the socket descriptor
FILE *fp = fdopen(duplicate);

This worked in Unix but not in MS VC++ (dup is giving error). <snip>

<offtopic=system-specific> In Unix a socket descriptor *is* a file
descriptor. You can also use read() and write() instead of recv() and
send(); this can be considered either convenient or obfuscatory. Note
that dup'ing means that fclose'ing the FILE* leaves the (low-level)
socket open, *but* unless it's at EOF (and hence useless) stdio may
well have read ahead for buffering (probably making it useless).

In Windows socket handles are not file handles/identifiers, and can
only be used with winsock routines. I don't think there's any way to
do what you want in MS' runtime (MSVCRT.DLL etc.). Well, except
spawning threads which copy the socket input to a pipe whose other end
is used as a file and/or vice versa to output, or perhaps spawning a
proceess to do so if the socket can be opened there. Or somewhat less
unstandardly, copy socket input to a real but temporary file which is
used as input and/or direct output to such a file which you then copy
to socket output. </>

<offtopic=C++> If it's possible to get or change the code you want to
call to be C++ (which of course MS VC++ also supports) *and use C++
I/O instead of C I/O (FILEs)*, you can easily write (or find) a class
that wraps a socket into a streambuf. </>

- David.Thompson1 at worldnet.att.net

 

[Glen Herrmannsfeldt]

On 4 Sep 2003 19:50:09 -0700, (e-mail address removed) (Jeevan) wrote:

Please don't top-post. Context lost as a result.


<offtopic=system-specific> In Unix a socket descriptor *is* a file
descriptor. You can also use read() and write() instead of recv() and
send(); this can be considered either convenient or obfuscatory. Note
that dup'ing means that fclose'ing the FILE* leaves the (low-level)
socket open, *but* unless it's at EOF (and hence useless) stdio may
well have read ahead for buffering (probably making it useless).

In Windows socket handles are not file handles/identifiers, and can
only be used with winsock routines. I don't think there's any way to
do what you want in MS' runtime (MSVCRT.DLL etc.).

I thought, but I never tried it, that on NT, and OS based on NT, that file
handles and socket descriptors were interchangable, but not on Win95 based
OS. To maintain compatablility for both, they use different calls. I
thought that is what the books I have said, but I never needed to try it.

-- glen