Tomcat can't 'see' new files

[HightowerC]

Hello all.

This might be slightly off topic for this forum, but I'm betting
there's some of you out there with some experience with Apache Tomcat.

Here's my problem.

I have an applet that is used for scanning document images. When the
applet loads, it loads a multipage tiff file, and via a different
command line based application, splits the pages in to single page
tiff files with names in the format of pageaa.tif, pageab.tif, etc.

My problem is that when the pages get 'split' apart, Tomcat doesn't
seem to recognize that the new files exist. i.e. If i point my broswer
at http://localhost:8080/myapp/images/pageaa.tiff I get a HTTP 404
error. If I wait a few minutes and try again, then I no longer get the
error.

Then, if I add another page on the end of the list (ie. pageac.tif)
Tomcat gives me a 404 error for just that page. I can get to pageaa
and pageab, but not pageac. Again, if I wait a few minutes, I can
eventually get to pageac without the 404 error.

I'm thinking that tomcat is just taking a while in recognizing that
the new tiff files (single pages) exist. Is there anything I can do?
Am I on the right track?

 

[HightowerC]

I'm thinking that tomcat is just taking a while in recognizing that
the new tiff files (single pages) exist. Is there anything I can do?
Am I on the right track?

I forgot to note that I have set the cachingAllowed=false and
cacheTTL=500 in the server.xml file.

Any other ideas/theories?

 

[Dave Miller]

Hello all.

This might be slightly off topic for this forum, but I'm betting
there's some of you out there with some experience with Apache Tomcat.

Here's my problem.

I have an applet that is used for scanning document images. When the
applet loads, it loads a multipage tiff file, and via a different
command line based application, splits the pages in to single page
tiff files with names in the format of pageaa.tif, pageab.tif, etc.

My problem is that when the pages get 'split' apart, Tomcat doesn't
seem to recognize that the new files exist. i.e. If i point my broswer
at http://localhost:8080/myapp/images/pageaa.tiff I get a HTTP 404
error. If I wait a few minutes and try again, then I no longer get the
error.

Then, if I add another page on the end of the list (ie. pageac.tif)
Tomcat gives me a 404 error for just that page. I can get to pageaa
and pageab, but not pageac. Again, if I wait a few minutes, I can
eventually get to pageac without the 404 error.

I'm thinking that tomcat is just taking a while in recognizing that
the new tiff files (single pages) exist. Is there anything I can do?
Am I on the right track?

The applet is running on the client and Tomcat is running on the server,
on which side is the "different command line based application" being
run and to where is it saving the resulting files?

 

[HightowerC]

The applet is running on the client and Tomcat is running on the server,
on which side is the "different command line based application" being
run and to where is it saving the resulting files?

--
Dave Miller
Java Web Hostinghttp://www.cheap-jsp-hosting.com/- Hide quoted text -

- Show quoted text -

The command line app that splits the files is being run on the server,
and it is saving the resulting files on the server.

 

[Dave Miller]

The command line app that splits the files is being run on the server,
and it is saving the resulting files on the server.

The only thing that I can think of is that the app that splits the files
could be not releasing it's lock on the newly created file. So long as
the file is accessible to Tomcat it should immediately serve it as
available.

 

[HightowerC]

The only thing that I can think of is that the app that splits the files
  could be not releasing it's lock on the newly created file. So long as
the file is accessible to Tomcat it should immediately serve it as
available.

Thanks for the reply.

Some new strange behavior to report. I was testing this applet again
yesterday afternoon, and it split the multipage tiff (8 pages) into
the single page tiff files (pageaa, pageab, ..., pageah).
I could see/navigate to all pages except for pageah.tiff. I then added
another singe page tiff (pageag.tiff), and could navigate to that one,
but still not pageah. I just kept getting the 404 error. The
last_updated timestamp on pageah was the same as on pageaa through
pageag, but for some reason, even after several minutes, I still could
not navigate to this ONE page.

I'm fairly certain that the 7 page version (pageaa-pageag) was cached
somewhere (either in tomcat, or the browser, or maybe even windows
networking somehow caches files?) and that's why tomcat refused to
serve up the last page (pageah).

Any other ideas?

 

[Dave Miller]

Thanks for the reply.

Some new strange behavior to report. I was testing this applet again
yesterday afternoon, and it split the multipage tiff (8 pages) into
the single page tiff files (pageaa, pageab, ..., pageah).
I could see/navigate to all pages except for pageah.tiff. I then added
another singe page tiff (pageag.tiff), and could navigate to that one,
but still not pageah. I just kept getting the 404 error. The
last_updated timestamp on pageah was the same as on pageaa through
pageag, but for some reason, even after several minutes, I still could
not navigate to this ONE page.

I'm fairly certain that the 7 page version (pageaa-pageag) was cached
somewhere (either in tomcat, or the browser, or maybe even windows
networking somehow caches files?) and that's why tomcat refused to
serve up the last page (pageah).

Any other ideas?

Cache might explain not seeing a new version of an existing file but not
a 404.

It could be that Tomcat has a restriction that it will only retry a 404
call after x seconds have elapsed, but that would explain seconds not
minutes.

As the app is creating a new file, watch the directory and try opening
the new file in a locally based graphics program. As soon as it's
viewable in the graphics program (but not before) try hitting it with a
new browser instance (not via the applet). I'm pretty sure that Tomcat
will serve it - if so you've got a code problem not a container problem.

If Tomcat doesn't immediately serve it, post how many seconds it takes
between the time the graphics program displays it and Tomcat serves it.

If Tomcat does serve it, post the code that writes the file to disk.

 

[HightowerC]

Cache might explain not seeing a new version of an existing file but not
a 404.

It could be that Tomcat has a restriction that it will only retry a 404
call after x seconds have elapsed, but that would explain seconds not
minutes.

As the app is creating a new file, watch the directory and try opening
the new file in a locally based graphics program. As soon as it's
viewable in the graphics program (but not before) try hitting it with a
new browser instance (not via the applet). I'm pretty sure that Tomcat
will serve it - if so you've got a code problem not a container problem.

If Tomcat doesn't immediately serve it, post how many seconds it takes
between the time the graphics program displays it and Tomcat serves it.

If Tomcat does serve it, post the code that writes the file to disk.

--
Dave Miller
Java Web Hostinghttp://www.cheap-jsp-hosting.com/- Hide quoted text -

- Show quoted text -

Thanks Dave. I'll try that and get back to you.

 

[Michael Jung]

Any other ideas?

I have the same problem (which doesn't really make it more on-topic
when I generate temporary files. Tomcat doesn't serve them. I haven't
made any elaborate time measurements, but I can contribute that it
happens on Linux (both client and server) and I looked at the files
with ls and cat during the problem time and I tried various browsers;
I'm pretty sure it's a Tomcat problem.

If you happen to find a solution elswhere, I'd be grateful, if you'd
tell me.

Michael

 

[Dave Miller]

I have the same problem (which doesn't really make it more on-topic
when I generate temporary files. Tomcat doesn't serve them. I haven't
made any elaborate time measurements, but I can contribute that it
happens on Linux (both client and server) and I looked at the files
with ls and cat during the problem time and I tried various browsers;
I'm pretty sure it's a Tomcat problem.

ls is merely listing the directory and cat will take whatever is on disk
at that point in time. Neither tell you whether the file is readable by
Tomcat or another app.

If you can open or use the file in its native app and Tomcat still won't
serve it, it's a container problem. Otherwise, it would seem unlikely
that Tomcat is the culprit.

What's more likely is that the stream writing the file is not closing
properly and Tomcat is respecting its lock.

 

[Michael Jung]

ls is merely listing the directory and cat will take whatever is on
disk at that point in time. Neither tell you whether the file is
readable by Tomcat or another app.

I don't see the difference between cat and "another app". It was a
new file and I checked the permissions; they were okay.

If you can open or use the file in its native app and Tomcat still
won't serve it, it's a container problem. Otherwise, it would seem
unlikely that Tomcat is the culprit.

What is a file's native app? Do you ask, whether I can reopen the
file in the same thread, where I closed it?

This is the code:

try {
long t = System.currentTimeMillis();
Writer w = new BufferedWriter(new FileWriter(myDir + t + ".html"));
String f = ...
w.write(f);
w.flush();
w.close();
}
catch (IOException e) {
e.printStackTrace();
}

What I will do then is ls in myDir and paste the <t>.html into a
browser -> not served. (BTW, repeated calls to the above will create
new temporary files. The first one is still not served.)

What's more likely is that the stream writing the file is not closing
properly and Tomcat is respecting its lock.

With the code above that would imply a bug in java file handling. (Or
do I have to keep a handle on the FileWriter and close it explicitely
as well? That's counter intuitive.) I haven't experienced the problem
outside of Tomcat though, i.e. usually opening the file from another
process or thread within the process. In fact, I have written a small
http server because of this exact problem (and it works). It doesn't
serve "jsp" yet though:-(

Michael

 

[John B. Matthews]

[...]

This is the code:

try {
long t = System.currentTimeMillis();
Writer w = new BufferedWriter(
new FileWriter(myDir + t + ".html"));
String f = ...
w.write(f);
w.flush();
w.close();
}
catch (IOException e) {
e.printStackTrace();
}

What I will do then is ls in myDir and paste the <t>.html into a
browser -> not served. (BTW, repeated calls to the above will
create new temporary files. The first one is still not served.)

Is it possible for this code to be called fast enough to create
duplicate names? I don't quite see how it could be a problem here, but
I've seen it in other contexts on a sufficiently fast machine or a
sufficiently low resolution timer[1]. Perhaps you might try
System.nanoTime() and see?

[1] <http://mindprod.com/jgloss/time.html>

 

[Michael Jung]

[...]

This is the code:

try {
long t = System.currentTimeMillis();
Writer w = new BufferedWriter(
new FileWriter(myDir + t + ".html"));
String f = ...
w.write(f);
w.flush();
w.close();
}
catch (IOException e) {
e.printStackTrace();
}

What I will do then is ls in myDir and paste the <t>.html into a
browser -> not served. (BTW, repeated calls to the above will
create new temporary files. The first one is still not served.)

Is it possible for this code to be called fast enough to create
duplicate names? I don't quite see how it could be a problem here, but
I've seen it in other contexts on a sufficiently fast machine or a
sufficiently low resolution timer[1]. Perhaps you might try
System.nanoTime() and see?

Not a problem in this context. (The proper way to do this is
File.createTempFile, I know.)

Michael

 

[Dave Miller]

This is the code:

try {
long t = System.currentTimeMillis();
Writer w = new BufferedWriter(new FileWriter(myDir + t + ".html"));
String f = ...
w.write(f);
w.flush();
w.close();
}
catch (IOException e) {
e.printStackTrace();
}

What I will do then is ls in myDir and paste the <t>.html into a
browser -> not served. (BTW, repeated calls to the above will create
new temporary files. The first one is still not served.)

Put the below (obviously with corrected path info) on your Tomcat as
test.jsp. If you point your browser to test.jsp you should end up at
test_file.jsp with "Hello World!" on your screen. If not, post the results.

<%@ page import="java.util.*, java.io.*,java.lang.*, java.sql.*" %>
<%
File file = new File("[path_to_webapps]/test_file.jsp");
try{
FileOutputStream outStream = new FileOutputStream(file);
outStream.write("Hello World!".getBytes());
outStream.close();
}catch(Exception ex){
}
response.sendRedirect("test_file.jsp");

%>

 

[John B. Matthews]

[...]

Not a problem in this context. (The proper way to do this is
File.createTempFile, I know.)

I tried to reproduce the effect you describe using the versions and code
below. At frequencies > 100 Hz (period < 10 ms), the Timer's period fell
below the task's execution time, and the usual delay in the first task
caused subsequent executions to "bunch up", as described in the API [1];
but I could always browse the first document. It's probably an
irrelevant artifact of Timer, but it may shed light. I enabled directory
listings in web.xml to see the growing list more easily.

See also Dave Miller's suggestion in this thread.

$ ./bin/catalina.sh version
....
Server version: Apache Tomcat/6.0.18
Server built: Jul 22 2008 02:00:36
Server number: 6.0.18.0
OS Name: Mac OS X
OS Version: 10.5.6
Architecture: ppc
JVM Version: 1.5.0_16-b06-284
JVM Vendor: Apple Inc.

<code>
import java.io.BufferedWriter;
import java.io.FileWriter;
import java.io.IOException;
import java.io.Writer;
import java.util.Date;
import java.util.Timer;
import java.util.TimerTask;

/**
* @author John B. Matthews
*/
public class TomcatTest {

private static final int MAX = 100;
private static final String myDir =
"/Users/Shared/tomcat/webapps/examples/temp/";
private static volatile int count;
private static final Timer timer = new Timer();
private static final TimerTask task = new TimerTask() {
public void run() {
count++;
try {
long t = System.currentTimeMillis();
Writer w = new BufferedWriter(
new FileWriter(myDir + t + ".html"));
String f = ""
+ "<html>"
+ "<body>"
+ "Ã la recherche du temps perdu: "
+ count + " " + new Date(t)
+ "</body>" + "</html>";
w.write(f);
w.flush();
w.close();
} catch (IOException e) {
e.printStackTrace();
}
if (count == MAX) timer.cancel();
}
};

public static void main(String[] args) {
timer.scheduleAtFixedRate(task, 1000L, 100L);
}
}
</code>

[1] <http://java.sun.com/javase/6/docs/api/java/util/Timer.html>

 

[divination]

Were you able to solve this? I'm facing exactly the same problem - it takes Tomcat6 (on WinXP) about 5 mins to recognize any newly added static files.