how to read a file newly generated in real time?

[webinfinite]

I am starting a new thread for this topic since previous problem has
been solved.

I have code like this:

#include <stdio.h>


int main(){

FILE *fp;

while(1){
fp = fopen("foo.txt", "r"); ;
if (fp != NULL)
break;
else{
printf("Waiting for the file\n");
}
}


printf("I am out.\n");
fclose(fp);
return 0;
}

foo.txt is generated after this code starts to execute. My problem
here is that even after foo.txt has been generated, the code is still
in a tight loop printing:

Waiting for the file
Waiting for the file
Waiting for the file
Waiting for the file
Waiting for the file
Waiting for the file

Until long after the file is generated, the code comes out. Is there
any solution to force the code to find out if this file has been
generated?

Thank you.

 

[Jens Thoms Toerring]

I am starting a new thread for this topic since previous problem has
been solved.

I have code like this:

#include <stdio.h>

int main(){
FILE *fp;
while(1){
fp = fopen("foo.txt", "r"); ;
if (fp != NULL)
break;
else{
printf("Waiting for the file\n");
}
}
printf("I am out.\n");
fclose(fp);
return 0;
}

foo.txt is generated after this code starts to execute. My problem
here is that even after foo.txt has been generated, the code is still
in a tight loop printing:

Waiting for the file
Waiting for the file
Waiting for the file
Waiting for the file
Waiting for the file
Waiting for the file

Until long after the file is generated, the code comes out. Is there
any solution to force the code to find out if this file has been
generated?

What's a "long time"? If you're in such a tight loop you can get
hundreds of such lines of output is half a second (and maybe your
terminal can't keep up and takes a long time to just output all
those lines).

But perhaps the other application that creates the file creates
it with write-only permissions and only changes that once it's
done with writing everything it wants to the file? Of course
this is pure guesswork since there is no information to go by
except what you write, but it might be useful to check (with
probably system-specific functions, e.g. if you're running
UNIX you could check what errno tells you about the reasons)
why fopen() fails for that long.

Regards, Jens

 

[webinfinite]

What's a "long time"? If you're in such a tight loop you can get
hundreds of such lines of output is half a second (and maybe your
terminal can't keep up and takes a long time to just output all
those lines).

But perhaps the other application that creates the file creates
it with write-only permissions and only changes that once it's
done with writing everything it wants to the file? Of course
this is pure guesswork since there is no information to go by
except what you write, but it might be useful to check (with
probably system-specific functions, e.g. if you're running
UNIX you could check what errno tells you about the reasons)
why fopen() fails for that long.

Regards, Jens
--
\ Jens Thoms Toerring ___ (e-mail address removed)
\__________________________ http://toerring.de- -

- -

It takes the code more than 10 seconds to get the file. I would like
to find a solution that when you generate the file, it will be
captured by the code immediately. Is there a way? Thanks.

 

[webinfinite]

It takes the code more than 10 seconds to get the file. I would like
to find a solution that when you generate the file, it will be
captured by the code immediately. Is there a way? Thanks.- -

- -

I timed the code, the code will take 43 second to pick up the new
generated file. Anything can make this process quick?

 

[Jens Thoms Toerring]

It takes the code more than 10 seconds to get the file. I would like
to find a solution that when you generate the file, it will be
captured by the code immediately. Is there a way? Thanks.

There's nothing in your program that would keep it form doing so
unless there are additional circumstances that would make fopen()
fail. One of those could be that the other program opens the file
with permissions that don't allow your program to open it for
reading - but you're the only one who can check for that. To
repeat: in your program fopen() should return successfully once
the file exists and it's permitted to open the file for reading.
One of these two conditions must not be satisfied for the 10
seconds you're writing about and only you have the possibility
to find out what it is - therefore I did recommend to resort to
system-specific methods (the C standard only requires that
fopen() returns NULL on failure but does not specify any methods
to obtain further informations) to determine why fopen() did fail,
those methods may tell you that the file either doesn't exist
when you assume it does or that you for some time don't have the
permissions to open it for reading.

Regards, Jens

 

[webinfinite]

There's nothing in your program that would keep it form doing so
unless there are additional circumstances that would make fopen()
fail. One of those could be that the other program opens the file
with permissions that don't allow your program to open it for
reading - but you're the only one who can check for that. To
repeat: in your program fopen() should return successfully once
the file exists and it's permitted to open the file for reading.
One of these two conditions must not be satisfied for the 10
seconds you're writing about and only you have the possibility
to find out what it is - therefore I did recommend to resort to
system-specific methods (the C standard only requires that
fopen() returns NULL on failure but does not specify any methods
to obtain further informations) to determine why fopen() did fail,
those methods may tell you that the file either doesn't exist
when you assume it does or that you for some time don't have the
permissions to open it for reading.

Regards, Jens
--
\ Jens Thoms Toerring ___ (e-mail address removed)
\__________________________ http://toerring.de- -

- -

The way I am doing this is very simiple. Under the same directory of
the executable,there is a file named "boo.txt". I start the code in
one terminal. Then in the other terminal, I just type: "mv boo.txt
foo.txt".

I was hoping the code capture this immediately, but it didn't. It
takes more than 40 seconds to take this change. It takes too long. I
was wondering if there is any efficient way to reduce this 40 seconds
to say maybe 5 seconds?

 

[Francine.Neary]

The way I am doing this is very simiple. Under the same directory of
the executable,there is a file named "boo.txt". I start the code in
one terminal. Then in the other terminal, I just type: "mv boo.txt
foo.txt".

I was hoping the code capture this immediately, but it didn't. It
takes more than 40 seconds to take this change. It takes too long. I
was wondering if there is any efficient way to reduce this 40 seconds
to say maybe 5 seconds?

<OT>
As you're using *nix, you could try looking at FAM (http://oss.sgi.com/
projects/fam/)
</OT>

 

[Jens Thoms Toerring]

The way I am doing this is very simiple. Under the same directory of
the executable,there is a file named "boo.txt". I start the code in
one terminal. Then in the other terminal, I just type: "mv boo.txt
foo.txt".

I was hoping the code capture this immediately, but it didn't. It
takes more than 40 seconds to take this change. It takes too long. I
was wondering if there is any efficient way to reduce this 40 seconds
to say maybe 5 seconds?

This can only be a system-specific problem and, as far as I can
see, it's definitely not related to the C language. I don't know
what your system is so I can't even tell you where to ask (the
commands you mention look a bit like UNIX, but for example under
Linux I couldn't reproduce anything similar to what you write,
once I rename the file to foo.txt your program stops immediately).
So all I can recommend is to take this to a group that deals with
the details of the system you are using since that problem is un-
likely to be related in any way to C. If it's a UNIX-like system
you could go to comp.unix.programmer but please write explicitely
what kind of UNIX you're running, this can make a lot of a diffe-
rence.
Regards, Jens

 

[Al Balmer]

On Thu, 16 Aug 2007 22:09:12 -0000, "(e-mail address removed)"

(This is not really on topic here)

Until long after the file is generated, the code comes out.

Of course it does. The tight loop has filled all the buffers the
system can provide, the buffer in the printer, etc, and it will keep
printing until they empty. In fact, if there's no more buffer space,
it will probably suspend your program and resume it to print the
remaining hundreds or thousands of lines.

Is there
any solution to force the code to find out if this file has been
generated?

Of course, but the best solution is probably dependent on your system.
Ask in an appropriate newsgroup.

In the meantime, you can do some things. Put a delay in the loop so
that you print an entire line before checking the file again. If you
can't wait that long, check the file more often but don't print as
often. You can do that by either checking elapsed time and doing the
print only when (say) two seconds have elapsed, or by printing only
every Nth time around the loop, N determined by experiment.

You can combine these things, too. Maybe you can put a one second
delay in the loop, and print the message every 10 iterations, for
example.

 

[Keith Thompson]

I am starting a new thread for this topic since previous problem has
been solved.

I have code like this:

#include <stdio.h>


int main(){

FILE *fp;

while(1){
fp = fopen("foo.txt", "r"); ;
if (fp != NULL)
break;
else{
printf("Waiting for the file\n");
}
}


printf("I am out.\n");
fclose(fp);
return 0;
}

foo.txt is generated after this code starts to execute. My problem
here is that even after foo.txt has been generated, the code is still
in a tight loop printing:

Waiting for the file
Waiting for the file
Waiting for the file
Waiting for the file
Waiting for the file
Waiting for the file

Until long after the file is generated, the code comes out. Is there
any solution to force the code to find out if this file has been
generated?

Like others here, I suspect that printing "Waiting for the file" in a
tight loop is the real problem.

Try this version of your program. If the problem goes away, then it
was probably caused by output buffering. If it doesn't, then it's
probably something system-specific; try comp.unix.programmer (and be
prepared to offer more details about your system).

#include <stdio.h>
int main(void)
{
unsigned long count = 0;
FILE *fp;
while (1) {
fp = fopen("foo.txt", "r");
if (fp != NULL) {
break;
}
else {
count ++;
}
}
printf("Done, count = %ld\n", count);
fclose(fp);
return 0;
}

 

[Richard Tobin]

man 1 dnotify

"No entry for dnotify in section 1 of the manual"

-- Richard

 

[Edwin van den Oetelaar]

I am starting a new thread for this topic since previous problem has
been solved.

[snip]


Until long after the file is generated, the code comes out. Is there
any solution to force the code to find out if this file has been
generated?

Thank you.

man 1 dnotify

do not busywait, EVER !

 

[Edwin van den Oetelaar]

"No entry for dnotify in section 1 of the manual"

-- Richard

I am sorry, maybe the OP is more lucky.
http://www.google.nl/search?q=man+1+dnotify for everyone else then.

;-)

 

[Kenneth Brody]

:
[...]

while(1){
fp = fopen("foo.txt", "r"); ;
if (fp != NULL)
break;
else{
printf("Waiting for the file\n");
}
} [...]
foo.txt is generated after this code starts to execute. My problem
here is that even after foo.txt has been generated, the code is still
in a tight loop printing:

Waiting for the file
Waiting for the file
Waiting for the file
Waiting for the file
Waiting for the file
Waiting for the file

Until long after the file is generated, the code comes out. Is there
any solution to force the code to find out if this file has been
generated?

How long is "long after"?

Your program may be running in such a tight loop that it is sending
output faster than the output device can handle, and it takes that
long to simply finish displaying the already-printf'ed messages.

Add some code (outside the scope of clc) to slow it down. Perhaps
add a 1/10th second delay at the end of the loop.

Or only print your message after some time has elapsed. You can
use time(), and only print if the value is different from the last
call to time().

For example, here is some (untested!) code:

if ( ClockHasChanged() )
printf("Waiting for the file\n");

...

int ClockHasChanged()
{
static time_t prev = 0;
time_t now = time(NULL);

if ( prev != now )
{
prev = now;
return 1;
}

return 0;
}

--
+-------------------------+--------------------+-----------------------+
| Kenneth J. Brody | www.hvcomputer.com | #include |
| kenbrody/at\spamcop.net | www.fptech.com | <std_disclaimer.h> |
+-------------------------+--------------------+-----------------------+
Don't e-mail me at: <mailto:[email protected]>

 

[Dik T. Winter]

....
> It takes the code more than 10 seconds to get the file. I would like
> to find a solution that when you generate the file, it will be
> captured by the code immediately. Is there a way? Thanks.

How do you *know* that it takes ten seconds? That there is still
ten seconds of output going along the screen? But that is also
possible due to buffering. Your screen cannot cope with the speed
with which those lines are generated.

 

[Dik T. Winter]

> The way I am doing this is very simiple. Under the same directory of
> the executable,there is a file named "boo.txt". I start the code in
> one terminal. Then in the other terminal, I just type: "mv boo.txt
> foo.txt".
>
> I was hoping the code capture this immediately, but it didn't. It
> takes more than 40 seconds to take this change. It takes too long. I
> was wondering if there is any efficient way to reduce this 40 seconds
> to say maybe 5 seconds?

Skip the printing of the line "Waiting for file", and see that it
will happen very much faster.