A gem for handling temporary file(s)?

[Albert Schlef]

I'm writing a program that needs to generate two or three temporary
files.

(Specifically: my program runs a shell command and I need to pass the
shell command a path to a non-existing file which it will dump data to.)

Is there a 'gem' that manages these things? Preferably it should remove
the files when the script finishes or whatever.

Of course, if Ruby supports this built-in that's fine too (I did 'ri
File' and 'ri FileUtils' but founds nothing).

 

[Paul Harrington]

I'm writing a program that needs to generate two or three temporary
files.

(Specifically: my program runs a shell command and I need to pass the
shell command a path to a non-existing file which it will dump data to.)

Is there a 'gem' that manages these things? Preferably it should remove
the files when the script finishes or whatever.

Of course, if Ruby supports this built-in that's fine too (I did 'ri
File' and 'ri FileUtils' but founds nothing).

ri Tempfile

that'll get you started

 

[Albert Schlef]

I'm writing a program that needs to generate two or three temporary
files.

(Specifically: my program runs a shell command and I need to pass the
shell command a path to a non-existing file which it will dump data to.)

[...]
ri Tempfile

that'll get you started

Thanks! I didn't know about Tempfile.

Though I have a little problem: Tempfile let me *open* a new temporary
file. But I just need to generate a temporary file *name*, which I'll
pass to a shell command.

I guess I'll immediately close the handle Tempfile.new() returns and
pass its path() to the shell command.

 

[Robert Klemme]

I'm writing a program that needs to generate two or three temporary
files.

(Specifically: my program runs a shell command and I need to pass the
shell command a path to a non-existing file which it will dump data to.)

[...]
ri Tempfile

that'll get you started

Thanks! I didn't know about Tempfile.

Though I have a little problem: Tempfile let me *open* a new temporary
file. But I just need to generate a temporary file *name*, which I'll
pass to a shell command.

You could use a dirty hack and abuse a private method:

irb(main):005:0> Tempfile.open('/tmp') {|tf| p tf,
tf.sendmake_tmpname,'a','o')}
#<File:/tmp/tmp20100303-4173-rdy9aq-0>
"a20100303-4173-kvk0d3-o"
=> [#<File:/tmp/tmp20100303-4173-rdy9aq-0 (closed)>,
"a20100303-4173-kvk0d3-o"]
irb(main):006:0>

I guess I'll immediately close the handle Tempfile.new() returns and
pass its path() to the shell command.

What do you want the external program to do with the tempfile?

Kind regards

robert

 

[botp]

I'm writing a program that needs to generate two or three temporary
files.

(Specifically: my program runs a shell command and I need to pass the
shell command a path to a non-existing file which it will dump data to.)

man mktemp
man tempfile

Is there a 'gem' that manages these things? Preferably it should remove
the files when the script finishes or whatever.

it is builtin in ruby. but in this case, you'd better do it all in ruby...

best regards -botp

 

[Albert Schlef]

You could use a dirty hack and abuse a private method:
tf.sendmake_tmpname,'a','o')}

Thanks.

Anyway, it turned out that was my smallest problems. I ended up writing
a wrapper class that remembers a set of related Tempfile object (or else
the files get deleted too soon for me).

 

[Caleb Clausen]

Thanks! I didn't know about Tempfile.

Though I have a little problem: Tempfile let me *open* a new temporary
file. But I just need to generate a temporary file *name*, which I'll
pass to a shell command.

I'm not expert enough to be certain about this, but by doing this
you'll be creating a tempfile race condition security hole in your
program. I think the same goes for Robert's suggestion as well. There
may be a way to do it securely... but it's probably tricky. One
advantage of Tempfile (and similar facilities in other languages) is
that it avoids this subtle and nassty security hole. But you have to
use it the way it wants to be used, otherwise you defeat the security.
This is why you're better off rewriting this external command in ruby,
if that's possible. Or rewriting your ruby script to make it an
integral part of the external program.

None of this may actually matter in your case... but you're the only
one with enough information to make that judgment.

 

[Daniel Berger]

)

man mktemp
man tempfile


it is builtin in ruby. but in this case, you'd better do it all in ruby..=

 

[Caleb Clausen]

Bah. Use file-temp.

gem install file-temp

I can't get file-temp to install at the moment, (for reasons unrelated
to file-temp, apparently). Does file-temp allow one to create
temporary _directories_? That's a feature I've often missed in the
past.

 

[Robert Klemme]

I'm not expert enough to be certain about this, but by doing this
you'll be creating a tempfile race condition security hole in your
program. I think the same goes for Robert's suggestion as well.

Do you mean there is a robustness issue or a security issue? I don't
see a security issue here. Robustness would only be at risk if the file
name generation algorithm is bad. What else am I missing?

There
may be a way to do it securely... but it's probably tricky. One
advantage of Tempfile (and similar facilities in other languages) is
that it avoids this subtle and nassty security hole. But you have to
use it the way it wants to be used, otherwise you defeat the security.
This is why you're better off rewriting this external command in ruby,
if that's possible. Or rewriting your ruby script to make it an
integral part of the external program.

None of this may actually matter in your case... but you're the only
one with enough information to make that judgment.

Albert still did not disclose what the external program should do with
the temporary file. We do not even know whether it is an option to
rewrite the external program.

Kind regards

robert

 

[Caleb Clausen]

Do you mean there is a robustness issue or a security issue? I don't
see a security issue here. Robustness would only be at risk if the file
name generation algorithm is bad. What else am I missing?

It's a security issue. See:
http://www.dwheeler.com/secure-programs/Secure-Programs-HOWTO/avoid-race.html
and scroll down to "7.10.1.2. Temporary Files"

As I implied above, I may not know what I'm talking about here, but
I'm fairly sure. Also, I didn't contemplate the snippet you
contributed closely; perhaps it avoids the race condition in some
clever way that I'm unaware of.

Albert still did not disclose what the external program should do with
the temporary file. We do not even know whether it is an option to
rewrite the external program.

Yes, indeed. Which is why I said "if that's possible".

 

[Caleb Clausen]

There
may be a way to do it securely... but it's probably tricky.

I _think_ that if you create a temporary file with Tempfile, and then
DON'T CLOSE IT, you can safely pass the tempfile's name to an external
command. (I'm assuming that the tempfile is an output from the
external command.... if it's an input, none of my security anxieties
apply. I think.) Only close the tempfile once the external command has
finished (and you've read out of it whatever information you need).
This will fail if the external command balks at writing to an already
existing file.

Alternatively, you could put your temp file in $HOME/tmp rather than
the system-wide /tmp, which is another way to sidestep the race. I'm
pretty sure. If you go this way, Tempfile is useless to you, tho.

 

[Tanaka Akira]

Alternatively, you could put your temp file in $HOME/tmp rather than
the system-wide /tmp, which is another way to sidestep the race. I'm
pretty sure. If you go this way, Tempfile is useless to you, tho.

Another way is Dir.mktmpdir in tmpdir.

require 'tmpdir'
Dir.mktmpdir {|d|
tmppath = File.join(d, "foo")
... use tmppath ...
}

 

[Albert Schlef]

one with enough information to make that judgment.

Albert still did not disclose what the external program should do with
the temporary file [...]

Thank you all, this task is behind me already.

If it interests you, then the external command I'm using is 'dot' (from
the graphviz package).

The command I'm using is something like this:

dot "%s" -Tgif -o "%s" -Tsvg -o "%s"

(I need three temporary files, in this example. 'dot' reads the first
file and writes images to the next two files.)

Smart guys would say that instead, on POSIX systems, I could write the
command as...

dot /dev/fd/10 -Tgif -o /dev/fd/11" -Tsvg -o /dev/fd/12

...but then I'd need to read simultaneously from fd/11 and fd/12 because
I'm not sure which is written first. But I need my program to work on
Windows too and I'm not sure it supports this "sophistication".

I'm not expert enough to be certain about this, but by doing this
you'll be creating a tempfile race condition [...]

I don't care much about robustness because the command is run from an
interactive application by a single user and I'm using the files
immediately. There isn't much a chance for a race condition.

I _think_ that if you create a temporary file with Tempfile, and then
DON'T CLOSE IT, you can safely pass the tempfile's name to an external
command.

For some reason, the files aren't always deleted (well, it seems they're
never deleted). I don't know why. At first I suspected 'dot' re-creates
the files, but when I do "ls -i" I see that the inodes haven't changed.
But I can live with this bug. Maybe I could add some code to the
"destructor" of my class to explicitly delete these files. Does Ruby
support "destructors"?

 

[Robert Klemme]

one with enough information to make that judgment.

Albert still did not disclose what the external program should do with
the temporary file [...]

Thank you all, this task is behind me already.

Considering your last comment about files not being deleted I'd say
it's not done yet.

If it interests you, then the external command I'm using is 'dot' (from
the graphviz package).

The command I'm using is something like this:

=A0dot "%s" -Tgif -o "%s" -Tsvg -o "%s"

(I need three temporary files, in this example. 'dot' reads the first
file and writes images to the next two files.)

Smart guys would say that instead, on POSIX systems, I could write the
command as...

=A0dot /dev/fd/10 -Tgif -o /dev/fd/11" -Tsvg -o /dev/fd/12

...but then I'd need to read simultaneously from fd/11 and fd/12 because
I'm not sure which is written first. But I need my program to work on
Windows too and I'm not sure it supports this "sophistication".

Yeah, probably not a good idea to go into that direction.

Btw, do you need to read those files while they are written or is it
sufficient to wait for process termination (e.e. use system) before
you access them?

I'm not expert enough to be certain about this, but by doing this
you'll be creating a tempfile race condition [...]

I don't care much about robustness because the command is run from an
interactive application by a single user and I'm using the files
immediately. There isn't much a chance for a race condition.

I _think_ that if you create a temporary file with Tempfile, and then
DON'T CLOSE IT, you can safely pass the tempfile's name to an external
command.

For some reason, the files aren't always deleted (well, it seems they're
never deleted). I don't know why. At first I suspected 'dot' re-creates
the files, but when I do "ls -i" I see that the inodes haven't changed.
But I can live with this bug. Maybe I could add some code to the
"destructor" of my class to explicitly delete these files. Does Ruby
support "destructors"?

We have

begin
...
ensure
...
end

The usual way to do this is to write a method. See
http://blog.rubybestpractices.com/posts/rklemme/002_Writing_Block_Methods.h=
tml

Cheers

robert

--=20
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/

 

[Caleb Clausen]

I'm not expert enough to be certain about this, but by doing this
you'll be creating a tempfile race condition [...]

I don't care much about robustness because the command is run from an
interactive application by a single user and I'm using the files
immediately. There isn't much a chance for a race condition.

It's not (just) a robustness issue, but a security issue. Even if
there's not much chance for a race condition, that can create a very
large security hole. I'd suggest you should take a look at the link I
posted in reply to Robert above. I just want to make sure you're
aware.

 

[Daniel Berger]

I can't get file-temp to install at the moment, (for reasons unrelated
to file-temp, apparently). Does file-temp allow one to create
temporary _directories_? That's a feature I've often missed in the
past.

No, but there's Dir.mktmpdir.

Regards,

Dan

 

[Caleb Clausen]

No, but there's Dir.mktmpdir.

Thanks to you and to Tanaka Akira for pointing this out, since I was
not aware of it. I'll definitely remember it next time I need that.

 

[Albert Schlef]

dot ... -Tgif -o ... -Tsvg -o ...

[...]
Btw, do you need to read those files while they are written or is it
sufficient to wait for process termination (e.e. use system) before
you access them?

(I read the files after termination (I'm using system()).)

We have

begin
...
ensure
...
end

The usual way to do this is to write a method. See
http://blog.rubybestpractices.com/posts/rklemme/002_Writing_Block_Methods.html

Thanks. I know this scheme (and the article), but for some reason it
didn't come up to my mind to use it. You just gave me a nice idea. I
need to read more Ruby code to train myself. The problem is that I love
to program so I prefer writing to reading :-(

It's not (just) a robustness issue, but a security issue. Even if
there's not much chance for a race condition, that can create a very
large security hole. I'd suggest you should take a look at the link I
posted in reply to Robert above. I just want to make sure you're
aware.

You mean a privacy issue. Yes, I fixed that. Thanks.

 

[Robert Klemme]

It's a security issue. See:
http://www.dwheeler.com/secure-programs/Secure-Programs-HOWTO/avoid-race.= html
and scroll down to "7.10.1.2. Temporary Files"

As I implied above, I may not know what I'm talking about here, but
I'm fairly sure. Also, I didn't contemplate the snippet you
contributed closely; perhaps it avoids the race condition in some
clever way that I'm unaware of.

Thanks for the link. If I understand all this right this cannot be
fixed as long as a) there are two processes involved or b) the second
process (dot) cannot be made inherit the file descriptor. An
alternative approach would avoid a shared directory like /tmp and
write the output in $HOME/.../somewhere - which might be slower
because /tmp is often mounted in RAM. Maybe it helps to create a
directory in /tmp which is owned by and only accessible to $USER; then
create the tempfiles in that directory.

Yes, indeed. Which is why I said "if that's possible".

Yep. I mainly included the remark to poke Albert so he would
eventually give the information. ;-)

Kind regards

robert

--=20
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/