File watching utility written in Java?

Discussion in 'Java' started by Bomb Diggy, Jun 29, 2003.

  1. Bomb Diggy

    Bomb Diggy Guest

    Hi,

    I need a tool that will recognise new files in a directory (of an ftp
    server) and then run some other program - say my java program which
    will process the new file. Is there a shareware/freeware tool
    available?

    Thanks.
    Bomb Diggy, Jun 29, 2003
    #1
    1. Advertising

  2. Implementing this functionality in *Java*, would not kill your machine!?

    Bomb Diggy wrote:
    > Hi,
    >
    > I need a tool that will recognise new files in a directory (of an ftp
    > server) and then run some other program - say my java program which
    > will process the new file. Is there a shareware/freeware tool
    > available?
    >
    > Thanks.
    Ahmed Moustafa, Jun 29, 2003
    #2
    1. Advertising

  3. Ahmed Moustafa:

    >Yes, but, the JVM will take a lot of the memory and the CPU,


    A couple of MB. Not exactly a problem these days.

    >that's in
    >addition to the disk IO every second or something?


    Checking a directory every couple of seconds is negligible.

    However, there is a more elegant approach (see my other posting).

    Regards,
    Marco
    --
    Please reply in the newsgroup, not by email!
    Java programming tips: http://jiu.sourceforge.net/javatips.html
    Other Java pages: http://www.geocities.com/marcoschmidt.geo/java.html
    Marco Schmidt, Jun 30, 2003
    #3
  4. Bomb Diggy:

    >I need a tool that will recognise new files in a directory (of an ftp
    >server) and then run some other program - say my java program which
    >will process the new file. Is there a shareware/freeware tool
    >available?


    Using JNI, you can use the Win32 API's notification service to be told
    about directory content changes:
    <http://groups.google.com/groups?selm=dq6fevspvqahsl70a5qublnqgnbnmtqo7t%404ax.com>.

    Regards,
    Marco
    --
    Please reply in the newsgroup, not by email!
    Java programming tips: http://jiu.sourceforge.net/javatips.html
    Other Java pages: http://www.geocities.com/marcoschmidt.geo/java.html
    Marco Schmidt, Jun 30, 2003
    #4
  5. >>>Implementing this functionality in *Java*, would not kill your machine!?
    >>
    >>
    >> Of course not - all it needs is a Thread that wakes up every few seconds
    >> and uses the java.io.File class to get the current list of files, comparing
    >> the
    >> timestamps to the time of the previous scan.

    >
    >
    > You can actually save a step; on most OS's, when a file is
    > created the timestamp of the directory is updated, so, if
    > all you want is new files, you can just have a thread that
    > poles the timestamp of the directory.


    Is reading a directory timestamp considered a disk IO?
    Ahmed Moustafa, Jun 30, 2003
    #5
  6. Bomb Diggy

    Drew Volpe Guest

    Last time we met, Ahmed Moustafa <> had said:
    > William Brogden wrote:
    >
    > >>Implementing this functionality in *Java*, would not kill your machine!?

    > >
    > >
    > > Of course not - all it needs is a Thread that wakes up every few seconds
    > > and uses the java.io.File class to get the current list of files, comparing
    > > the
    > > timestamps to the time of the previous scan.

    >
    > Yes, but, the JVM will take a lot of the memory and the CPU, that's in
    > addition to the disk IO every second or something?


    No. If written in any reasonable manner, it should take ~1 MB of RAM
    and have no noticeable CPU or disk hit; all you're doing is getting the
    timestamp of 1 directory once a second. Nothing resource intensive about
    that.


    Let's test this...

    To prove it to myself, I wrote the program below which does what the
    user would like, except that it only lists the dir contents instead
    of running a program on a file. When I run it on my machine, which
    is slow by today's standards (400Mhz PII, 384 MB of RAM, 5400 rpm ide
    hd), the program takes up 1.6 MB of RAM and the CPU for its process
    never goes above 0.0. Even when I up the polling to 100 times a
    second, the CPU goes to 0.19% every now and then, and then back to 0.0.

    So, actually, Java seems like a good language for writing programs
    like this. It took me ~2 minutes to code this up and, while the
    RAM used is probably more than it would be in other languages,
    1.6 MB is very reasonable.


    ---
    import java.io.*;

    public class TestPoller extends Thread {

    public void run() {

    File directory = new File("/home/drew/stuff/javatemp/testpoller");
    long currentLastModified = 0;
    while (true) {
    if (directory.lastModified() > currentLastModified) {
    currentLastModified = directory.lastModified();
    String[] files = directory.list();
    System.out.println("Directory contents changed. Now:");
    for (int i=0; i < files.length; i++) {
    System.out.println(" " + files);
    }
    }
    try {
    this.sleep(1000);
    }
    catch (InterruptedException e) {
    System.out.println("Interrupted!");
    }
    }
    }

    public static void main(String[] args) throws Exception {
    TestPoller tp = new TestPoller();
    tp.setPriority(Thread.MIN_PRIORITY);
    tp.start();

    }
    }
    ---


    --
    --------------------------------------------------------------------------
    The geographical center of Boston is in Roxbury. Due north of the
    center we find the South End. This is not to be confused with South
    Boston which lies directly east from the South End. North of the South
    End is East Boston and southwest of East Boston is the North End.

    Drew Volpe, mylastname at hcs o harvard o edu
    Drew Volpe, Jun 30, 2003
    #6
  7. Bomb Diggy

    Drew Volpe Guest

    Last time we met, Ahmed Moustafa <> had said:

    > > You can actually save a step; on most OS's, when a file is
    > > created the timestamp of the directory is updated, so, if
    > > all you want is new files, you can just have a thread that
    > > poles the timestamp of the directory.

    >
    > Is reading a directory timestamp considered a disk IO?


    It has to read from disk, so yes. But it's only reading
    a timestamp, so it's very little overhead.


    dv

    --
    --------------------------------------------------------------------------
    The geographical center of Boston is in Roxbury. Due north of the
    center we find the South End. This is not to be confused with South
    Boston which lies directly east from the South End. North of the South
    End is East Boston and southwest of East Boston is the North End.

    Drew Volpe, mylastname at hcs o harvard o edu
    Drew Volpe, Jun 30, 2003
    #7
  8. Bomb Diggy

    Jon A. Cruz Guest

    Drew Volpe wrote:
    > You can actually save a step; on most OS's, when a file is
    > created the timestamp of the directory is updated, so, if
    > all you want is new files, you can just have a thread that
    > poles the timestamp of the directory.
    >


    Actually... maybe not.

    On all the Windows OS's it's also a function of the filesystem in use.
    Jon A. Cruz, Jun 30, 2003
    #8
  9. Bomb Diggy

    Drew Volpe Guest

    Last time we met, Jon A. Cruz <> had said:
    > Drew Volpe wrote:
    > > You can actually save a step; on most OS's, when a file is
    > > created the timestamp of the directory is updated, so, if
    > > all you want is new files, you can just have a thread that
    > > poles the timestamp of the directory.
    > >

    >
    > Actually... maybe not.
    >
    > On all the Windows OS's it's also a function of the filesystem in use.


    it seems to work for NTFS (w/ win2k). Which does it not work on ?


    dv

    --
    --------------------------------------------------------------------------
    The geographical center of Boston is in Roxbury. Due north of the
    center we find the South End. This is not to be confused with South
    Boston which lies directly east from the South End. North of the South
    End is East Boston and southwest of East Boston is the North End.

    Drew Volpe, mylastname at hcs o harvard o edu
    Drew Volpe, Jun 30, 2003
    #9
  10. Drew Volpe:

    >It has to read from disk, so yes.


    Not necessarily. The information may well be in the file system cache
    in memory, so even that small I/O may not be necessary. In fact, after
    the timestamp has been retrieved I'm relatively sure that it is kept
    in memory and stays there, esp. given that it is queried so often.

    Regards,
    Marco
    --
    Please reply in the newsgroup, not by email!
    Java programming tips: http://jiu.sourceforge.net/javatips.html
    Other Java pages: http://www.geocities.com/marcoschmidt.geo/java.html
    Marco Schmidt, Jul 1, 2003
    #10
  11. Bomb Diggy

    Van Glass Guest

    You could do this by routinely connecting to the FTP server and
    comparing the list of files in the directory to the last time you
    connected. Try using the FTP component included in iNet Factory and
    available for download at http://www.jscape.com/inetfactory/index.html

    In particular look at the Ftp#getDirListing method which returns an
    Enumeration of FtpFile.

    Marco Schmidt <> wrote in message news:<>...
    > Bomb Diggy:
    >
    > >I need a tool that will recognise new files in a directory (of an ftp
    > >server) and then run some other program - say my java program which
    > >will process the new file. Is there a shareware/freeware tool
    > >available?

    >
    > Using JNI, you can use the Win32 API's notification service to be told
    > about directory content changes:
    > <http://groups.google.com/groups?selm=dq6fevspvqahsl70a5qublnqgnbnmtqo7t%404ax.com>.
    >
    > Regards,
    > Marco
    Van Glass, Jul 2, 2003
    #11
  12. I wrote a java service called DeployServer that reads a database for a
    set of directories to scan and specific files to look for. It reads
    the timestamps adn compares it to the one in memory. If it is newer the
    file is deployed (moved to a new location or deployed via 9ias and then
    the timestamp updated.
    Ivan S Kirkpatrick, Jul 3, 2003
    #12
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Steve C. Orr, MCSD
    Replies:
    1
    Views:
    556
    reaway lee
    Aug 24, 2003
  2. Luis Esteban Valencia Muñoz
    Replies:
    3
    Views:
    1,406
    Scott Allen
    Jun 4, 2005
  3. aljosa

    file parsing/watching + qt

    aljosa, May 4, 2005, in forum: Python
    Replies:
    2
    Views:
    328
    Swaroop C H
    May 4, 2005
  4. Benjamin D. LeMasurier

    watching a file

    Benjamin D. LeMasurier, Oct 1, 2004, in forum: C Programming
    Replies:
    5
    Views:
    302
    dandelion
    Oct 4, 2004
  5. KaiWen
    Replies:
    102
    Views:
    2,715
    Jorgen Grahn
    Sep 15, 2011
Loading...

Share This Page