Storing preprocessed data over executions

[Hendrik Maryns]

Hi,

I have a simple program that queries a ‘tree bank’. This is a text file
containing descriptions of linguistic trees. To make the querying
efficient, the data is proprocessed into a binary format. This is
stored somewhere on disk, and used by other parts of the program.

At this point, the binary data is stored in the place where the original
file is, with another file ending. This solution is unsatisfactory,
since the directory in question might not be writable. So I want to
store the preprocessed tree banks somewhere in a directory for my
program. Furthermore, I’d like to have access to them in a subsequent
execution of the program, since this preprocessing step is quite
intensive and there is no need to do it each time. Also, some other
file are generated during execution and they can be put there as well;
and I want to have a ‘cleanup’ option to delete them.

How is this handled generally in Java?

In Linux, one would make a directory .<programname> in the user’s home
directory and store data there. There is no such thing as a workspace
where this data can be stored, like Eclipse does.

I could of course ask the user to choose a directory at startup, but
that is sub-optimal.

What I am trying now is to create a temp dir and store stuff there, but
that does not persist between executions, since a new temp dir is
created each time.

TIA, H.
--
Hendrik Maryns
http://tcl.sfs.uni-tuebingen.de/~hendrik/
==================
http://aouw.org
Ask smart questions, get good answers:
http://www.catb.org/~esr/faqs/smart-questions.html


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.4-svn0 (GNU/Linux)
Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org

iD8DBQFH9iIAe+7xMGD3itQRAnhsAJ9SfoQpn+UX7MCicyXonyYcSzzbxACfW/Y+
y99PBqVrmI2UUgPchqkP7w0=
=Jusm
-----END PGP SIGNATURE-----

 

[RedGrittyBrick]

Hi,

I have a simple program that queries a ‘tree bank’. This is a text file
containing descriptions of linguistic trees. To make the querying
efficient, the data is proprocessed into a binary format. This is
stored somewhere on disk, and used by other parts of the program.

At this point, the binary data is stored in the place where the original
file is, with another file ending. This solution is unsatisfactory,
since the directory in question might not be writable. So I want to
store the preprocessed tree banks somewhere in a directory for my
program. Furthermore, I’d like to have access to them in a subsequent
execution of the program, since this preprocessing step is quite
intensive and there is no need to do it each time. Also, some other
file are generated during execution and they can be put there as well;
and I want to have a ‘cleanup’ option to delete them.

How is this handled generally in Java?

In Linux, one would make a directory .<programname> in the user’s home
directory and store data there. There is no such thing as a workspace
where this data can be stored, like Eclipse does.

I could of course ask the user to choose a directory at startup, but
that is sub-optimal.

What I am trying now is to create a temp dir and store stuff there, but
that does not persist between executions, since a new temp dir is
created each time.

Here's what I do to persist application-specific settings:

if ("Windows XP".equals(System.getProperty("os.name"))) {
propertiesPath = System.getProperty("user.home")
+ System.getProperty("file.separator")
+ "Application Data"
+ System.getProperty("file.separator")
+ developerName;

boolean success = (new File(propertiesPath)).mkdir();
...
propertiesFileName = propertiesPath
+ System.getProperty("file.separator")
+ appName + ".properties";
} else {
propertiesPath = System.getProperty("user.home");
propertiesFileName = propertiesPath
+ System.getProperty("file.separator")
+ "." + appName + ".properties";
}

But obviously this needs some attention

 

[Hendrik Maryns]

RedGrittyBrick schreef:

Here's what I do to persist application-specific settings:

if ("Windows XP".equals(System.getProperty("os.name"))) {
propertiesPath = System.getProperty("user.home")
+ System.getProperty("file.separator")
+ "Application Data"
+ System.getProperty("file.separator")
+ developerName;

boolean success = (new File(propertiesPath)).mkdir();
...
propertiesFileName = propertiesPath
+ System.getProperty("file.separator")
+ appName + ".properties";
} else {
propertiesPath = System.getProperty("user.home");
propertiesFileName = propertiesPath
+ System.getProperty("file.separator")
+ "." + appName + ".properties";
}

But obviously this needs some attention

Thanks, this indeed sort of what it’s supposed to be, I guess. But it
is kind of a hack. Isn’t there a standard way to do this? I suppose OSs
would need to support this.

Ok, the change for Vista is obvious (I’ll have to look into its file
organisation, though).

Thanks a bunch, H.

--
Hendrik Maryns
http://tcl.sfs.uni-tuebingen.de/~hendrik/
==================
http://aouw.org
Ask smart questions, get good answers:
http://www.catb.org/~esr/faqs/smart-questions.html


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.4-svn0 (GNU/Linux)
Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org

iD8DBQFH9kA6e+7xMGD3itQRAk6XAJ9t3fA2bKBOB/fZktL6grOI9yUtUACfUrnt
b7Cmh4cxRvy7R2rYpIBlef0=
=p8t9
-----END PGP SIGNATURE-----

 

[Roedy Green]

So I want to
store the preprocessed tree banks somewhere in a directory for my
program.

The easy way it to use Preferences for small amounts of data and the
Java Web Start mechanism to nail you down some persistent file space
for larger files.

See http://mindprod.com/jgloss/preferences.html
http://mindprod.com/jgloss/javawebstart.html

Other than that you could just sniff about on the drives looking for
lots of free space, and create a dir with a unique name. I would
consider this naughty.
see http://mindprod.com/jgloss/file.html

You can ask the user with a JFileChooser.
See http://mindprod.com/jgloss/jfilechooser.html
perhaps making an intelligent default .

 

[Roedy Green]

How is this handled generally in Java?

for five techniques see:
http://mindprod.com/jgloss/file.html#PERMANENTFILES

 

[Mark Space]

Thanks, this indeed sort of what it’s supposed to be, I guess. But it
is kind of a hack. Isn’t there a standard way to do this? I suppose OSs
would need to support this.

I have several programs I use that put .files in my (Windows Vista) home
directory, it works fine for me as a user. Why not just do the same for
both windows and unix? The code I quoted above looks great to me for both.

 

[Hendrik Maryns]

Mark Space schreef:

I have several programs I use that put .files in my (Windows Vista) home
directory, it works fine for me as a user. Why not just do the same for
both windows and unix? The code I quoted above looks great to me for both.

Because that is not the convention on Windows. I’d rather not impose
Linux conventions on Windoze users. Though I try to stop people from
using Windows, I will not do this by nagging them.

H.
--
Hendrik Maryns
http://tcl.sfs.uni-tuebingen.de/~hendrik/
==================
http://aouw.org
Ask smart questions, get good answers:
http://www.catb.org/~esr/faqs/smart-questions.html


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.4-svn0 (GNU/Linux)
Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org

iD8DBQFH+3q8e+7xMGD3itQRAiKqAJ9ENK6PqHYQEfHCTvvnws5t1f0nPgCfbHDd
ZEkbMqWFL/0YwWB+cd1QBqA=
=KbgP
-----END PGP SIGNATURE-----

 

[Hendrik Maryns]

Roedy Green schreef:

The easy way it to use Preferences for small amounts of data and the
Java Web Start mechanism to nail you down some persistent file space
for larger files.

See http://mindprod.com/jgloss/preferences.html
http://mindprod.com/jgloss/javawebstart.html

Other than that you could just sniff about on the drives looking for
lots of free space, and create a dir with a unique name. I would
consider this naughty.
see http://mindprod.com/jgloss/file.html

You can ask the user with a JFileChooser.
See http://mindprod.com/jgloss/jfilechooser.html
perhaps making an intelligent default .

Thank you Roedy for your suggestions, I now use a combination of
Preferences to store the given directory over sessions, and ask for it
the first time.

Though still not fine-tuned, the code now is as follows, improvement
suggestions welcome:

/**
* Initialize the base file directory. It is guaranteed to exist
* after this method has finished. First, the user preferences are
* queried whether a directory has been chosen in some earlier
* execution. If not, the user is asked for a directory in a file
* chooser. TODO: find better place automatically, e.g.
* ~/.MonaSearch in Linux, <User home>/Application Data/MonaSearch
* in WinXP
*
* @post The base file directory exists. | getBaseFileDir().exists()
* @throws HeadlessException
*/
private void initBaseFileDir() throws HeadlessException {
final String suggestedDirName = System.getProperty("user.home")
+ System.getProperty("file.separator") + ".MonaSearch";
final String saveDirectory = this.getUserPrefs().get("SAVEDIR",
null);
// null for default and check below
if (saveDirectory == null) {
final File suggestedDir = new File(suggestedDirName);
// ask the user to provide a directory where to store tree banks
// and stuff
final JFileChooser chooser = new JFileChooser();
chooser.setFileSelectionMode(JFileChooser.DIRECTORIES_ONLY);
chooser.setSelectedFile(suggestedDir);
chooser
.setDialogTitle("Please select a directory where preprocessed"
+ " treebanks and other files should be stored.");
// presumably, the user will want to choose a hidden file, e.g.
// ‘~/.MonaSearch’ on Linux.
chooser.setFileHidingEnabled(false);
final int result = chooser.showOpenDialog(this.frame);
switch (result) {
case JFileChooser.APPROVE_OPTION:
this.baseFileDir = chooser.getSelectedFile();
// TODO: what happens if user selects file by typing it in?
// save directory for next time
this.getUserPrefs().put("SAVEDIR",
this.baseFileDir.getAbsolutePath());
break;
case JFileChooser.CANCEL_OPTION:
case JFileChooser.ERROR_OPTION:
// TODO: warning, exit?
this.baseFileDir = suggestedDir;
break;
default:
}
} else {
this.baseFileDir = new File(saveDirectory);
}
// This is important, since even if a file is ‘chosen’, it could
// have been typed in the JFileChooser’s text field.
if (!this.baseFileDir.exists()) {
this.baseFileDir.mkdir();
}
}

Cheers, H.
--
Hendrik Maryns
http://tcl.sfs.uni-tuebingen.de/~hendrik/
==================
http://aouw.org
Ask smart questions, get good answers:
http://www.catb.org/~esr/faqs/smart-questions.html


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.4-svn0 (GNU/Linux)
Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org

iD8DBQFH+31he+7xMGD3itQRAjYlAJ9gkiZRV/JDg2+t0c5vMSvo4CfrigCfcUEf
J5a9LqXuFwg9d5+S27gXgW4=
=woSN
-----END PGP SIGNATURE-----