Sandboxing eval'd code

[eastcoastcoder]

I'm working on a web app with complicated and ever changing business
rules.

On any given day, a new rule may need to be introduced - quickly.
There's not enough commonality between them to factor out and just make
configuration changes. So the developers are always adding and
modifying, which of course makes reliability and testing difficult.

I thought of the following solution: New business rules can be coded in
Ruby and serialized into the database, along with configuration options
as to when they apply. (The rules look at a bunch of things and return
either continue or abort - so making hooks for them is easy. They
should never modify *anything* - just return a value.)
The app will load them and run them in a sandbox, catching all
exceptions and also timeing them out if they fail to return.
Developers can add/modify these rules easily, without touching the core
app, and, should one have a bug, although it may give the wrong result,
it won't bring the rest of the system down.

One concern I have is that I know that eval'd code can modify class
definitions, and access other objects via ObjectSpace. Is there anyway
to eval code so that it can't change Classes and the like? A true
Sandbox.

I'm not familiar enough with $SAFE to know what it can do (although
I've heard it is not reliable). http://tryruby.hobix.com/ probably
does this, but source doesn't seem available.
http://approximity.com/cgi-bin/rubybuch_wiki/wpage.rb?nd=214 looks
relevant, but I can't figure it out.

In general, comments about this greatly appreciated.

 

[Erik Veenstra]

class String
def de_inspect
Thread.new do
$SAFE = 4

de_inspect_unsafe
end.value
end

def de_inspect_unsafe # But fast...
eval(self, Module.new.module_eval{binding})
end

def evil_eval # Don't use it !!!
eval(self)
end
end

def foo ; p [self, 1] ; end

foo

"def foo ; p [self, 2] ; end".evil_eval

foo

"def foo ; p [self, 3] ; end".de_inspect_unsafe

foo

"def foo ; p [self, 4] ; end".de_inspect # Will fail...

foo

 

[m4dc4p]

Could you elaborate a little more on what you are showing here?
Specifically, why is it called "de_inspect_unsafe" and what is the
array in "p [self, 1]", "p [self, 2]" etc for?

I have also wondered about this issue myself and would like to see what
people have done. I'd love to see why's "try ruby" code but that is
probably a pipe dream ...

 

[Erik Veenstra]

class String
def de_inspect
Thread.new do
$SAFE = 4

de_inspect_unsafe
end.value
end

def de_inspect_unsafe # But fast...
eval(self, Module.new.module_eval{binding})
end

def evil_eval # Don't use it !!!
eval(self)
end
end

def foo ; p [self, 1] ; end

foo

"def foo ; p [self, 2] ; end".evil_eval

foo

"def foo ; p [self, 3] ; end".de_inspect_unsafe

foo

"def foo ; p [self, 4] ; end".de_inspect # Will fail...

foo

 

[konsu]

hello,

i have been looking for a solution to almost the same problem. but have not
found anything yet except for a solution that uses threads. just like this
code. which i have doubts about because creating threads in a web
application does not seem to scale too much.

konstantin

 

[Michael Schilling]

Hello everyone,

I'm fairly new to ruby and wanted to write a very simple cgi program,
which uploads a file and returns a converted file. Because it is so easy
(and I want to learn about ruby) I tried to do it with the standard
cgi.rb library. But I can't get the multipart_form (I need it for the
file upload) to work right. Everything works fine using the normal form
method, but with the multipart_form I always get an EOF Error 'no
content body' in the apache log. As soon as there is any input from an
multipart_form, an error occurs at: checkfile = CGI.new

Below you'll find my code (I boiled it down to the important stuff) and the
corresponding part of my apache2 error_log. I searched the web, several
mailing lists and the ruby docs, but couldn't find an answer. I also
asked on #ruby-lang in freenode, but no one there knew the cgi.rb
library very well.

Any help, on why this server error is happening, is very much
appreciated.

Bye,
Michael <-- who hopes, that this is the right list for the question



My code:
------------------------------------------------------------------
#!/usr/bin/ruby -w

require "cgi"

# make_form create a very simple html form
def make_form
make_form = CGI.new("html4")
make_form.out {
make_form.html {
make_form.body { "\n" +
# make_form.form{ "\n" + # everything works using the normal form
make_form.multipart_form{ "\n" + # but not with multipart_form to upload a file
make_form.file_field("filename",40,500000) + "\n" +
make_form.br +
make_form.submit("test it") + "\n"
}
}
}
}
end

# answer should show the filename of the uploaded file
def answer(file_in)

answer = CGI.new("html4")
answer.out {
answer.html {
answer.body{ "\n" +
answer.p{ "This should be the filename:" + file_in + "\n" } +
answer.br
}
}
}
end



begin
checkfile = CGI.new
parameter = checkfile['filename']
# parameter = checkfile.params['filename'][0] # this doesn't work either

rescue
answer("ERROR: " + $!.to_s) # send eror message to browser if possible
end


# create form if no data was send to the script, otherwise answer
if parameter == ""
make_form()
else
answer(parameter.to_s)
end
------------------------------------------------------------------


Apache log:
------------------------------------------------------------------
[Wed Feb 08 21:08:57 2006] [error] [client 191.168.1.1] /usr/lib/ruby/1.8/cgi.rb:979:in `read_multipart', referer: http://localhost/cgi-bin/multiform-test.rb
[Wed Feb 08 21:08:57 2006] [error] [client 191.168.1.1] : , referer: http://localhost/cgi-bin/multiform-test.rb
[Wed Feb 08 21:08:57 2006] [error] [client 191.168.1.1] no content body, referer: http://localhost/cgi-bin/multiform-test.rb
[Wed Feb 08 21:08:57 2006] [error] [client 191.168.1.1] (, referer: http://localhost/cgi-bin/multiform-test.rb
[Wed Feb 08 21:08:57 2006] [error] [client 191.168.1.1] EOFError, referer: http://localhost/cgi-bin/multiform-test.rb
[Wed Feb 08 21:08:57 2006] [error] [client 191.168.1.1] ), referer: http://localhost/cgi-bin/multiform-test.rb
[Wed Feb 08 21:08:57 2006] [error] [client 191.168.1.1] \tfrom /usr/lib/ruby/1.8/cgi.rb:1104:in `initialize_query', referer: http://localhost/cgi-bin/multiform-test.rb
[Wed Feb 08 21:08:57 2006] [error] [client 191.168.1.1] \tfrom /usr/lib/ruby/1.8/cgi.rb:2270:in `initialize', referer: http://localhost/cgi-bin/multiform-test.rb
[Wed Feb 08 21:08:57 2006] [error] [client 191.168.1.1] \tfrom /home/httpd/cgi-bin/multiform-test.rb:25:in `answer', referer: http://localhost/cgi-bin/multiform-test.rb
[Wed Feb 08 21:08:57 2006] [error] [client 191.168.1.1] \tfrom /home/httpd/cgi-bin/multiform-test.rb:52, referer: http://localhost/cgi-bin/multiform-test.rb
[Wed Feb 08 21:08:57 2006] [error] [client 191.168.1.1] Premature end of script headers: multiform-test.rb, referer: http://localhost/cgi-bin/multiform-test.rb
------------------------------------------------------------------



--
Michael Schilling

eMail : (e-mail address removed)
URL : http://glcu.sf.net/


"Change my name I remain the same." - Moloko

 

[Erik Veenstra]

Could you elaborate a little more on what you are showing
here?

I'm showing 3 ways of doing what you want to achieve, in
increasing order of safety. (Although the definitions of the
methods are in decreasing order of safety...) That' all.

Specifically, why is it called "de_inspect_unsafe"

Because it's not as safe as "de_inspect". And I want to trigger
some alarm bells in case one want to use it...

and what is the array in "p [self, 1]", "p [self, 2]" etc
for?

If you don't get this, well, get any other string of Ruby code.
It's not important.

I have also wondered about this issue myself and would like
to see what people have done. I'd love to see why's "try
ruby" code but that is probably a pipe dream ...

Just ask him...

gegroet,
Erik V. - http://www.erikveen.dds.nl/

 

[konsu]

hello,

try running your script from the command line. but set the url and method
environment variables first. on windows i do this:


set REQUEST_URI=/path
set REQUEST_METHOD=GET
ruby muscript.cgi .


this will allow you to avoid dealing with apache for a while, and will show
more errors too.

konstantin

Hello everyone,

I'm fairly new to ruby and wanted to write a very simple cgi program,
which uploads a file and returns a converted file. Because it is so easy
(and I want to learn about ruby) I tried to do it with the standard
cgi.rb library. But I can't get the multipart_form (I need it for the
file upload) to work right. Everything works fine using the normal form
method, but with the multipart_form I always get an EOF Error 'no
content body' in the apache log. As soon as there is any input from an
multipart_form, an error occurs at: checkfile = CGI.new

Below you'll find my code (I boiled it down to the important stuff) and
the
corresponding part of my apache2 error_log. I searched the web, several
mailing lists and the ruby docs, but couldn't find an answer. I also
asked on #ruby-lang in freenode, but no one there knew the cgi.rb
library very well.

Any help, on why this server error is happening, is very much
appreciated.

Bye,
Michael <-- who hopes, that this is the right list for the question



My code:
------------------------------------------------------------------
#!/usr/bin/ruby -w

require "cgi"

# make_form create a very simple html form
def make_form
make_form = CGI.new("html4")
make_form.out {
make_form.html {
make_form.body { "\n" +
# make_form.form{ "\n" + # everything works using the normal
form
make_form.multipart_form{ "\n" + # but not with multipart_form
to upload a file
make_form.file_field("filename",40,500000) + "\n" +
make_form.br +
make_form.submit("test it") + "\n"
}
}
}
}
end

# answer should show the filename of the uploaded file
def answer(file_in)

answer = CGI.new("html4")
answer.out {
answer.html {
answer.body{ "\n" +
answer.p{ "This should be the filename:" + file_in + "\n" } +
answer.br
}
}
}
end



begin
checkfile = CGI.new
parameter = checkfile['filename']
# parameter = checkfile.params['filename'][0] # this doesn't work either

rescue
answer("ERROR: " + $!.to_s) # send eror message to browser if possible
end


# create form if no data was send to the script, otherwise answer
if parameter == ""
make_form()
else
answer(parameter.to_s)
end
------------------------------------------------------------------


Apache log:
------------------------------------------------------------------
[Wed Feb 08 21:08:57 2006] [error] [client 191.168.1.1]
/usr/lib/ruby/1.8/cgi.rb:979:in `read_multipart', referer:
http://localhost/cgi-bin/multiform-test.rb
[Wed Feb 08 21:08:57 2006] [error] [client 191.168.1.1] : , referer:
http://localhost/cgi-bin/multiform-test.rb
[Wed Feb 08 21:08:57 2006] [error] [client 191.168.1.1] no content body,
referer: http://localhost/cgi-bin/multiform-test.rb
[Wed Feb 08 21:08:57 2006] [error] [client 191.168.1.1] (, referer:
http://localhost/cgi-bin/multiform-test.rb
[Wed Feb 08 21:08:57 2006] [error] [client 191.168.1.1] EOFError, referer:
http://localhost/cgi-bin/multiform-test.rb
[Wed Feb 08 21:08:57 2006] [error] [client 191.168.1.1] ), referer:
http://localhost/cgi-bin/multiform-test.rb
[Wed Feb 08 21:08:57 2006] [error] [client 191.168.1.1] \tfrom
/usr/lib/ruby/1.8/cgi.rb:1104:in `initialize_query', referer:
http://localhost/cgi-bin/multiform-test.rb
[Wed Feb 08 21:08:57 2006] [error] [client 191.168.1.1] \tfrom
/usr/lib/ruby/1.8/cgi.rb:2270:in `initialize', referer:
http://localhost/cgi-bin/multiform-test.rb
[Wed Feb 08 21:08:57 2006] [error] [client 191.168.1.1] \tfrom
/home/httpd/cgi-bin/multiform-test.rb:25:in `answer', referer:
http://localhost/cgi-bin/multiform-test.rb
[Wed Feb 08 21:08:57 2006] [error] [client 191.168.1.1] \tfrom
/home/httpd/cgi-bin/multiform-test.rb:52, referer:
http://localhost/cgi-bin/multiform-test.rb
[Wed Feb 08 21:08:57 2006] [error] [client 191.168.1.1] Premature end of
script headers: multiform-test.rb, referer:
http://localhost/cgi-bin/multiform-test.rb
------------------------------------------------------------------



--
Michael Schilling

eMail : (e-mail address removed)
URL : http://glcu.sf.net/


"Change my name I remain the same." - Moloko

 

[Bill Kelly]

i have been looking for a solution to almost the same problem. but have not
found anything yet except for a solution that uses threads. just like this
code. which i have doubts about because creating threads in a web
application does not seem to scale too much.

time ruby -e '100000.times { Thread.new { }.join }'

real 0m1.130s
user 0m1.124s
sys 0m0.008s



Regards,

Bill

 

[konsu]

could you elaborate a bit? are these numbers good or bad? what about memory
consumption? may be i am missing something important about ruby threads, but
my impression was that spawning a thread on every request is bad.

konstantin

 

[Michael Schilling]

Hi,

thank you for the suggestions. I set the variables as you suggested
(with export REQUEST_URI=/path in Linux) but always only get the form. I
don't exactly know how a string from the multipart_form looks like, so
I couldn't really test the behaviour.

When I use the normal form method, and do not set the variables as you
suggested, I get the expected result and no error from
'myscript.rb filename=something'. With the multipart_form I also get no
error, but also no output -> only the form.

I also thought, that this problem might be a misconfiguration of apache,
but a guy on #ruby-lang told me, that it is very unlikely when the
normal form method works fine.

So I can't reproduce the error without using the browser, but don't get
the expected result either.


Thanks for any help.


Bye, Michael




konsu wrote on thursday the 9th of February 2006:

try running your script from the command line. but set the url and method
environment variables first. on windows i do this:


set REQUEST_URI=/path
set REQUEST_METHOD=GET
ruby muscript.cgi .

this will allow you to avoid dealing with apache for a while, and will show
more errors too.

konstantin

Hello everyone,

I'm fairly new to ruby and wanted to write a very simple cgi program,
which uploads a file and returns a converted file. Because it is so easy
(and I want to learn about ruby) I tried to do it with the standard
cgi.rb library. But I can't get the multipart_form (I need it for the
file upload) to work right. Everything works fine using the normal form
method, but with the multipart_form I always get an EOF Error 'no
content body' in the apache log. As soon as there is any input from an
multipart_form, an error occurs at: checkfile = CGI.new

Below you'll find my code (I boiled it down to the important stuff) and
the
corresponding part of my apache2 error_log. I searched the web, several
mailing lists and the ruby docs, but couldn't find an answer. I also
asked on #ruby-lang in freenode, but no one there knew the cgi.rb
library very well.

Any help, on why this server error is happening, is very much
appreciated.

Bye,
Michael <-- who hopes, that this is the right list for the question



My code:
------------------------------------------------------------------
#!/usr/bin/ruby -w

require "cgi"

# make_form create a very simple html form
def make_form
make_form = CGI.new("html4")
make_form.out {
make_form.html {
make_form.body { "\n" +
# make_form.form{ "\n" + # everything works using the normal
form
make_form.multipart_form{ "\n" + # but not with multipart_form
to upload a file
make_form.file_field("filename",40,500000) + "\n" +
make_form.br +
make_form.submit("test it") + "\n"
}
}
}
}
end

# answer should show the filename of the uploaded file
def answer(file_in)

answer = CGI.new("html4")
answer.out {
answer.html {
answer.body{ "\n" +
answer.p{ "This should be the filename:" + file_in + "\n" } +
answer.br
}
}
}
end



begin
checkfile = CGI.new
parameter = checkfile['filename']
# parameter = checkfile.params['filename'][0] # this doesn't work either

rescue
answer("ERROR: " + $!.to_s) # send eror message to browser if possible
end


# create form if no data was send to the script, otherwise answer
if parameter == ""
make_form()
else
answer(parameter.to_s)
end
------------------------------------------------------------------


Apache log:
------------------------------------------------------------------
[Wed Feb 08 21:08:57 2006] [error] [client 191.168.1.1]
/usr/lib/ruby/1.8/cgi.rb:979:in `read_multipart', referer:
http://localhost/cgi-bin/multiform-test.rb
[Wed Feb 08 21:08:57 2006] [error] [client 191.168.1.1] : , referer:
http://localhost/cgi-bin/multiform-test.rb
[Wed Feb 08 21:08:57 2006] [error] [client 191.168.1.1] no content body,
referer: http://localhost/cgi-bin/multiform-test.rb
[Wed Feb 08 21:08:57 2006] [error] [client 191.168.1.1] (, referer:
http://localhost/cgi-bin/multiform-test.rb
[Wed Feb 08 21:08:57 2006] [error] [client 191.168.1.1] EOFError, referer:
http://localhost/cgi-bin/multiform-test.rb
[Wed Feb 08 21:08:57 2006] [error] [client 191.168.1.1] ), referer:
http://localhost/cgi-bin/multiform-test.rb
[Wed Feb 08 21:08:57 2006] [error] [client 191.168.1.1] \tfrom
/usr/lib/ruby/1.8/cgi.rb:1104:in `initialize_query', referer:
http://localhost/cgi-bin/multiform-test.rb
[Wed Feb 08 21:08:57 2006] [error] [client 191.168.1.1] \tfrom
/usr/lib/ruby/1.8/cgi.rb:2270:in `initialize', referer:
http://localhost/cgi-bin/multiform-test.rb
[Wed Feb 08 21:08:57 2006] [error] [client 191.168.1.1] \tfrom
/home/httpd/cgi-bin/multiform-test.rb:25:in `answer', referer:
http://localhost/cgi-bin/multiform-test.rb
[Wed Feb 08 21:08:57 2006] [error] [client 191.168.1.1] \tfrom
/home/httpd/cgi-bin/multiform-test.rb:52, referer:
http://localhost/cgi-bin/multiform-test.rb
[Wed Feb 08 21:08:57 2006] [error] [client 191.168.1.1] Premature end of
script headers: multiform-test.rb, referer:
http://localhost/cgi-bin/multiform-test.rb
------------------------------------------------------------------



--
Michael Schilling

eMail : (e-mail address removed)
URL : http://glcu.sf.net/


"Change my name I remain the same." - Moloko

 

[Erik Veenstra]

$ time ruby -e '(1..100000).collect{Thread.new{sleep 1}}.collect{|t| t.join}'

real 0m42.583s
user 0m37.693s
sys 0m1.828s

$ time ruby -e '(1..100000).collect{Thread.new{}.join}'

real 0m4.683s
user 0m3.984s
sys 0m0.245s

In the latter situation, the threads are not living side-by-side.
The second thread is started when the first one has finished.

The first example starts a lot of threads concurrently.

gegroet,
Erik V. -- http://www.erikveen.dds.nl/

 

[Bill Kelly]

could you elaborate a bit? are these numbers good or bad? what about memory
consumption? may be i am missing something important about ruby threads, but
my impression was that spawning a thread on every request is bad.

Hi, well, being able to create, execute, and join 100,000 threads
in about one second seemed to me to indicate that Ruby has pretty
low overhead for creating threads. (2.8GHz Pentium Xeon)

How many requests per second does your server need to handle?

I don't know if there's an absolute answer to whether creating
a thread per request is good or bad. It just looks like it doesn't
take very much time. (I don't think it takes much memory either.)

Unless your server is handling thousands of requests per second,
it looks like the overhead for creating a thread is fairly
insignificant. (Remember, Ruby currently has "green" threads,
so these are not actual Operating System threads being created.)


Regards,

Bill

 

[konsu]

i was just suggesting to mimic the web server's behaviour and simulate form
submission using environment variables since cgi relies just on these
variables and nothing more. to eliminate any apache misconfiguration. in my
earlier response i suggested to set REQUEST_METHOD=GET but to thest the form
submission you really need REQUEST_METHOD=POST and also you need to set the
corresponding variables that carry the request body. i do not remember the
names. the source for cgi.rb in the ruby lib directory might help you.

konstantin

Hi,

thank you for the suggestions. I set the variables as you suggested
(with export REQUEST_URI=/path in Linux) but always only get the form. I
don't exactly know how a string from the multipart_form looks like, so
I couldn't really test the behaviour.

When I use the normal form method, and do not set the variables as you
suggested, I get the expected result and no error from
'myscript.rb filename=something'. With the multipart_form I also get no
error, but also no output -> only the form.

I also thought, that this problem might be a misconfiguration of apache,
but a guy on #ruby-lang told me, that it is very unlikely when the
normal form method works fine.

So I can't reproduce the error without using the browser, but don't get
the expected result either.


Thanks for any help.


Bye, Michael




konsu wrote on thursday the 9th of February 2006:

try running your script from the command line. but set the url and method
environment variables first. on windows i do this:


set REQUEST_URI=/path
set REQUEST_METHOD=GET
ruby muscript.cgi .

this will allow you to avoid dealing with apache for a while, and will
show
more errors too.

konstantin

Hello everyone,

I'm fairly new to ruby and wanted to write a very simple cgi program,
which uploads a file and returns a converted file. Because it is so
easy
(and I want to learn about ruby) I tried to do it with the standard
cgi.rb library. But I can't get the multipart_form (I need it for the
file upload) to work right. Everything works fine using the normal form
method, but with the multipart_form I always get an EOF Error 'no
content body' in the apache log. As soon as there is any input from an
multipart_form, an error occurs at: checkfile = CGI.new

Below you'll find my code (I boiled it down to the important stuff) and
the
corresponding part of my apache2 error_log. I searched the web, several
mailing lists and the ruby docs, but couldn't find an answer. I also
asked on #ruby-lang in freenode, but no one there knew the cgi.rb
library very well.

Any help, on why this server error is happening, is very much
appreciated.

Bye,
Michael <-- who hopes, that this is the right list for the question



My code:
------------------------------------------------------------------
#!/usr/bin/ruby -w

require "cgi"

# make_form create a very simple html form
def make_form
make_form = CGI.new("html4")
make_form.out {
make_form.html {
make_form.body { "\n" +
# make_form.form{ "\n" + # everything works using the normal
form
make_form.multipart_form{ "\n" + # but not with
multipart_form
to upload a file
make_form.file_field("filename",40,500000) + "\n" +
make_form.br +
make_form.submit("test it") + "\n"
}
}
}
}
end

# answer should show the filename of the uploaded file
def answer(file_in)

answer = CGI.new("html4")
answer.out {
answer.html {
answer.body{ "\n" +
answer.p{ "This should be the filename:" + file_in + "\n" } +
answer.br
}
}
}
end



begin
checkfile = CGI.new
parameter = checkfile['filename']
# parameter = checkfile.params['filename'][0] # this doesn't work
either

rescue
answer("ERROR: " + $!.to_s) # send eror message to browser if possible
end


# create form if no data was send to the script, otherwise answer
if parameter == ""
make_form()
else
answer(parameter.to_s)
end
------------------------------------------------------------------


Apache log:
------------------------------------------------------------------
[Wed Feb 08 21:08:57 2006] [error] [client 191.168.1.1]
/usr/lib/ruby/1.8/cgi.rb:979:in `read_multipart', referer:
http://localhost/cgi-bin/multiform-test.rb
[Wed Feb 08 21:08:57 2006] [error] [client 191.168.1.1] : , referer:
http://localhost/cgi-bin/multiform-test.rb
[Wed Feb 08 21:08:57 2006] [error] [client 191.168.1.1] no content
body,
referer: http://localhost/cgi-bin/multiform-test.rb
[Wed Feb 08 21:08:57 2006] [error] [client 191.168.1.1] (, referer:
http://localhost/cgi-bin/multiform-test.rb
[Wed Feb 08 21:08:57 2006] [error] [client 191.168.1.1] EOFError,
referer:
http://localhost/cgi-bin/multiform-test.rb
[Wed Feb 08 21:08:57 2006] [error] [client 191.168.1.1] ), referer:
http://localhost/cgi-bin/multiform-test.rb
[Wed Feb 08 21:08:57 2006] [error] [client 191.168.1.1] \tfrom
/usr/lib/ruby/1.8/cgi.rb:1104:in `initialize_query', referer:
http://localhost/cgi-bin/multiform-test.rb
[Wed Feb 08 21:08:57 2006] [error] [client 191.168.1.1] \tfrom
/usr/lib/ruby/1.8/cgi.rb:2270:in `initialize', referer:
http://localhost/cgi-bin/multiform-test.rb
[Wed Feb 08 21:08:57 2006] [error] [client 191.168.1.1] \tfrom
/home/httpd/cgi-bin/multiform-test.rb:25:in `answer', referer:
http://localhost/cgi-bin/multiform-test.rb
[Wed Feb 08 21:08:57 2006] [error] [client 191.168.1.1] \tfrom
/home/httpd/cgi-bin/multiform-test.rb:52, referer:
http://localhost/cgi-bin/multiform-test.rb
[Wed Feb 08 21:08:57 2006] [error] [client 191.168.1.1] Premature end
of
script headers: multiform-test.rb, referer:
http://localhost/cgi-bin/multiform-test.rb
------------------------------------------------------------------



--
Michael Schilling

eMail : (e-mail address removed)
URL : http://glcu.sf.net/


"Change my name I remain the same." - Moloko

 

[why the lucky stiff]

One concern I have is that I know that eval'd code can modify class
definitions, and access other objects via ObjectSpace. Is there anyway
to eval code so that it can't change Classes and the like? A true
Sandbox.

Try Ruby uses fork. I setup the basic environment and then fork new
interpreters. I then remove all the constants and methods that could be
used maliciously. remove_const and remove_method, that's the secret.
However, there is some sticky business involved in trying to close up
the `ancestors' hole. I don't yet have an answer that doesn't involve a
hacked Ruby.

I don't use $SAFE because I didn't want the intrepid student to ever see
the word `taint' in her's session. Catch my drift? (Give it a sec,
it's drifting down to you right now.)

I don't use threads because I want the student to have all the power to
screw up her's environment without troubling other students. In
addition, I can use FreeBSD's process management to control individual
sessions. To prevent fork bombs and endless loops.

But I do use threads inside each session. One thread for an IRB
process, one thread for passing data through the FastCGI socket.

_why

 

[Michael Schilling]

Hello again,

as I couldn't reproduce the described multipart_form error (see
description below) directly on the command-line I looked into the cgi.rb
source and found the chunk of code that raises the exception:

Original code in cgi.rb
----------------------------------------------------
def read_multipart(boundary, content_length)
params = Hash.new([])
boundary = "--" + boundary
buf = ""
bufsize = 10 * 1024

# start multipart/form-data
stdinput.binmode if defined? stdinput.binmode
boundary_size = boundary.size + EOL.size
content_length -= boundary_size
status = stdinput.read(boundary_size)
if nil == status
raise EOFError, "no content body"
elsif boundary + EOL != status
raise EOFError, "bad content body"
end
---------------------------------------------------

After playing around with this part of the code, I have found more
questions than answers. When I negate the if clause before the exception
(see code below), the expression is still true (even so, it was false
for my program, before the change in cgi.rb), and the status variable
will even be written into the apache logs with something, that looks
right (see below for the logs - in the line with 'no content body'):

So whatever I do, I can't get rid of the exception. Something seems to
be very wrong here, but I have no clue, what it could be. Is this some
kind of bug in cgi.rb?

Maybe I should try a different approach, but I like a good challenge ;-)
Any more help is very much appreciated!


Bye, Michael


Changed code in cgi.rb
---------------------------------------------------
def read_multipart(boundary, content_length)
params = Hash.new([])
boundary = "--" + boundary
buf = ""
bufsize = 10 * 1024

# start multipart/form-data
stdinput.binmode if defined? stdinput.binmode
boundary_size = boundary.size + EOL.size
content_length -= boundary_size
status = stdinput.read(boundary_size)
if nil != status # set this to _not_ equal
raise EOFError, "no content body. Status: " + status.to_s # and add here the status as a string
elsif boundary + EOL != status
raise EOFError, "bad content body"
end
---------------------------------------------------

apache error log with original cgi.rb code:
-----------------------------------------------------------
[Fri Feb 10 03:56:33 2006] [error] [client 129.70.170.3] /usr/lib/ruby/1.8/cgi.rb:979:in `read_multipart', referer: http://calvin.physik.uni-bielefeld.de/cgi-bin/pdf2eps.rb
[Fri Feb 10 03:56:33 2006] [error] [client 129.70.170.3] : , referer: http://calvin.physik.uni-bielefeld.de/cgi-bin/pdf2eps.rb
[Fri Feb 10 03:56:33 2006] [error] [client 129.70.170.3] no content body. Status: ;, referer: http://calvin.physik.uni-bielefeld.de/cgi-bin/pdf2eps.rb
[Fri Feb 10 03:56:33 2006] [error] [client 129.70.170.3] (, referer: http://calvin.physik.uni-bielefeld.de/cgi-bin/pdf2eps.rb
[Fri Feb 10 03:56:33 2006] [error] [client 129.70.170.3] EOFError, referer: http://calvin.physik.uni-bielefeld.de/cgi-bin/pdf2eps.rb
-----------------------------------------------------------

apache error log with changed cgi.rb code:
-----------------------------------------------------------
[Fri Feb 10 03:57:02 2006] [error] [client 129.70.170.3] /usr/lib/ruby/1.8/cgi.rb:979:in `read_multipart', referer: http://calvin.physik.uni-bielefeld.de/cgi-bin/pdf2eps.rb
[Fri Feb 10 03:57:02 2006] [error] [client 129.70.170.3] : , referer: http://calvin.physik.uni-bielefeld.de/cgi-bin/pdf2eps.rb
[Fri Feb 10 03:57:02 2006] [error] [client 129.70.170.3] no content body. Status: Content-Disposition: form-data; name="sowas";, referer: http://calvin.physik.uni-bielefeld.de/cgi-bin/pdf2eps.rb
[Fri Feb 10 03:57:02 2006] [error] [client 129.70.170.3] (, referer: http://calvin.physik.uni-bielefeld.de/cgi-bin/pdf2eps.rb
[Fri Feb 10 03:57:02 2006] [error] [client 129.70.170.3] EOFError, referer: http://calvin.physik.uni-bielefeld.de/cgi-bin/pdf2eps.rb
-----------------------------------------------------------




------------- ORIGINAL MAILS ------------------------------------

konsu wrote on Thursday the 9th of February 2006:

i was just suggesting to mimic the web server's behaviour and simulate form
submission using environment variables since cgi relies just on these
variables and nothing more. to eliminate any apache misconfiguration. in my
earlier response i suggested to set REQUEST_METHOD=GET but to thest the form
submission you really need REQUEST_METHOD=POST and also you need to set the
corresponding variables that carry the request body. i do not remember the
names. the source for cgi.rb in the ruby lib directory might help you.

konstantin

Hi,

thank you for the suggestions. I set the variables as you suggested
(with export REQUEST_URI=/path in Linux) but always only get the form. I
don't exactly know how a string from the multipart_form looks like, so
I couldn't really test the behaviour.

When I use the normal form method, and do not set the variables as you
suggested, I get the expected result and no error from
'myscript.rb filename=something'. With the multipart_form I also get no
error, but also no output -> only the form.

I also thought, that this problem might be a misconfiguration of apache,
but a guy on #ruby-lang told me, that it is very unlikely when the
normal form method works fine.

So I can't reproduce the error without using the browser, but don't get
the expected result either.


Thanks for any help.


Bye, Michael




konsu wrote on thursday the 9th of February 2006:

try running your script from the command line. but set the url and method
environment variables first. on windows i do this:


set REQUEST_URI=/path
set REQUEST_METHOD=GET
ruby muscript.cgi .

this will allow you to avoid dealing with apache for a while, and will
show
more errors too.

konstantin


Hello everyone,

I'm fairly new to ruby and wanted to write a very simple cgi program,
which uploads a file and returns a converted file. Because it is so
easy
(and I want to learn about ruby) I tried to do it with the standard
cgi.rb library. But I can't get the multipart_form (I need it for the
file upload) to work right. Everything works fine using the normal form
method, but with the multipart_form I always get an EOF Error 'no
content body' in the apache log. As soon as there is any input from an
multipart_form, an error occurs at: checkfile = CGI.new

Below you'll find my code (I boiled it down to the important stuff) and
the
corresponding part of my apache2 error_log. I searched the web, several
mailing lists and the ruby docs, but couldn't find an answer. I also
asked on #ruby-lang in freenode, but no one there knew the cgi.rb
library very well.

Any help, on why this server error is happening, is very much
appreciated.

Bye,
Michael <-- who hopes, that this is the right list for the question



My code:
------------------------------------------------------------------
#!/usr/bin/ruby -w

require "cgi"

# make_form create a very simple html form
def make_form
make_form = CGI.new("html4")
make_form.out {
make_form.html {
make_form.body { "\n" +
# make_form.form{ "\n" + # everything works using the normal
form
make_form.multipart_form{ "\n" + # but not with
multipart_form
to upload a file
make_form.file_field("filename",40,500000) + "\n" +
make_form.br +
make_form.submit("test it") + "\n"
}
}
}
}
end

# answer should show the filename of the uploaded file
def answer(file_in)

answer = CGI.new("html4")
answer.out {
answer.html {
answer.body{ "\n" +
answer.p{ "This should be the filename:" + file_in + "\n" } +
answer.br
}
}
}
end



begin
checkfile = CGI.new
parameter = checkfile['filename']
# parameter = checkfile.params['filename'][0] # this doesn't work
either

rescue
answer("ERROR: " + $!.to_s) # send eror message to browser if possible
end


# create form if no data was send to the script, otherwise answer
if parameter == ""
make_form()
else
answer(parameter.to_s)
end
------------------------------------------------------------------


Apache log:
------------------------------------------------------------------
[Wed Feb 08 21:08:57 2006] [error] [client 191.168.1.1]
/usr/lib/ruby/1.8/cgi.rb:979:in `read_multipart', referer:
http://localhost/cgi-bin/multiform-test.rb
[Wed Feb 08 21:08:57 2006] [error] [client 191.168.1.1] : , referer:
http://localhost/cgi-bin/multiform-test.rb
[Wed Feb 08 21:08:57 2006] [error] [client 191.168.1.1] no content
body,
referer: http://localhost/cgi-bin/multiform-test.rb
[Wed Feb 08 21:08:57 2006] [error] [client 191.168.1.1] (, referer:
http://localhost/cgi-bin/multiform-test.rb
[Wed Feb 08 21:08:57 2006] [error] [client 191.168.1.1] EOFError,
referer:
http://localhost/cgi-bin/multiform-test.rb
[Wed Feb 08 21:08:57 2006] [error] [client 191.168.1.1] ), referer:
http://localhost/cgi-bin/multiform-test.rb
[Wed Feb 08 21:08:57 2006] [error] [client 191.168.1.1] \tfrom
/usr/lib/ruby/1.8/cgi.rb:1104:in `initialize_query', referer:
http://localhost/cgi-bin/multiform-test.rb
[Wed Feb 08 21:08:57 2006] [error] [client 191.168.1.1] \tfrom
/usr/lib/ruby/1.8/cgi.rb:2270:in `initialize', referer:
http://localhost/cgi-bin/multiform-test.rb
[Wed Feb 08 21:08:57 2006] [error] [client 191.168.1.1] \tfrom
/home/httpd/cgi-bin/multiform-test.rb:25:in `answer', referer:
http://localhost/cgi-bin/multiform-test.rb
[Wed Feb 08 21:08:57 2006] [error] [client 191.168.1.1] \tfrom
/home/httpd/cgi-bin/multiform-test.rb:52, referer:
http://localhost/cgi-bin/multiform-test.rb
[Wed Feb 08 21:08:57 2006] [error] [client 191.168.1.1] Premature end
of
script headers: multiform-test.rb, referer:
http://localhost/cgi-bin/multiform-test.rb
------------------------------------------------------------------



--
Michael Schilling

eMail : (e-mail address removed)
URL : http://glcu.sf.net/


"Change my name I remain the same." - Moloko

 

[ara.t.howard]

Hello again,

as I couldn't reproduce the described multipart_form error (see
description below) directly on the command-line I looked into the cgi.rb
source and found the chunk of code that raises the exception:

the problem is in your source i'm afraid. you can call CGI.new more than once
in a script : each call attempts to parse ENV and STDIN according to the cgi
spec. check out the docs for details. the other issue is that if, and only
if, you create a form using multipar the cgi params are either StringIO or
Tempfile objects. this is a good thing. consider a 10GB file upload. you
need a way to determine this. following is one way:


harp:~> cat a.rb

#! /usr/local/bin/ruby

require "cgi"

class Upload
attr_accessor 'cgi'

def initialize
self.cgi = CGI::new 'html4'
file = get 'file'
file ? answer : make_form
end

def make_form
cgi.out {
cgi.html {
cgi.body { "\n" +
cgi.multipart_form{ "\n" +
cgi.file_field("file",40,500000) + "\n" +
cgi.br +
cgi.submit("test it") + "\n"
}
}
}
}
end

def answer
cgi.out {
cgi.html {
cgi.body{ "\n" +
cgi.p{ "This should be the filename : " + param('file').original_filename + "\n" } +
cgi.br
}
}
}
end

def get key
value = param key
value = value.read.strip if value.respond_to?('read')
value
end

def param key
cgi.params[key].first
end
end

Upload::new if $0 == __FILE__


this should run with ruby 1.8 up.

inlined below is a totally simple upload script i sometimes use for personal
stuff. it allows upload/download and some other stuff.

regards.

-a

--
happiness is not something ready-made. it comes from your own actions.
- h.h. the 14th dali lama



#! /usr/local/bin/ruby

require 'cgi'
require 'tempfile'
require 'ftools'
require 'uri'

cgi = CGI::new
user = ENV['REMOTE_USER'] || 'anonymous'
headers = {}
content = ''

begin
mode = cgi.params['m'].first || 'index'
mode = mode.read.strip if mode.respond_to? 'read'

case mode
when /index|^$/

files = Dir['**/*'].select{|f| f !~ %r/html|cgi/}
dirname = File::dirname cgi.script_name

files.sort!{|a,b| File::stat(b).mtime <=> File::stat(a).mtime}

table_rows =
files.collect do |f|
stat = File::stat f
uri = 'http://' << cgi.host << File::join(dirname, f)
# <a href=#{ URI::escape f } title="right click -> save as">download</a>
<<-html
<tr valign=top aligh=left>
<td>
#{ CGI::escapeHTML f }
</td>
<td>
#{ CGI::escapeHTML stat.mtime.to_s }
</td>
<td>
#{ CGI::escapeHTML stat.size.to_s }
</td>
<td>
<a href=#{ uri } title="right click -> save (page) as">#{ uri }</a>
</td>
</tr>
html
end

table =
<<-html
<table width=100%>
<tr valign=top aligh=left>
<td> <b><u> file </b></u> </td>
<td> <b><u> modification time </b></u> </td>
<td> <b><u> size </b></u> </td>
<td> <b><u> download </b></u> </td>
</tr>
#{ table_rows }
</table>
html

content = <<-html
<html>
<head>
<title>#{ user }::files</title>
</head>
<body>
<h3>#{ user }::files</h3>

<table cellpadding=1 align=right width=100%>
<tr align=right>
<td>
<i>to download - right click -> save as</i>
</td>
</tr>
</table>

<hr>

<hr>
</table>
<hr>
#{ table }
<hr>
<h3><a href=#{ cgi.script_name }?m=upload>upload files</a></h3>
</body>
</html>
html

when /upload/
buttons = []
8.times do
buttons << <<-html
<input size=100 name=file type=file>
<hr width=30% align=left>
html
end
content = <<-html
<html>
<head>
<title>#{ user }::upload</title>
</head>
<body>
<h3>#{ user }::upload</h3>
<hr>
<form action=#{ cgi.script_name } method=POST enctype=multipart/form-data>
<input size=100 name=file type=file>
<hr width=30% align=left>
#{ buttons }

<input type=submit value=upload>
<hr width=30% align=left>

<input type=reset value=clear>
<hr width=30% align=left>

<input type=hidden name=m value=put>
<form>
<hr>
<h3><a href=#{ cgi.script_name }?m=index>index</a></h3>
</body>
</html>
html

when /put/
uploaded = []
tempfiles = cgi.params['file']
tempfiles.each do |tempfile|
org_path = tempfile.original_filename

local = "#{ org_path }"
local.gsub! %r|[/\\]|,'/'
local.gsub! %r|\s+|,'_'
local.downcase!

next if local =~ /^\s*$/

begin
File::mkpath(File::dirname(local)) rescue nil
if tempfile.path
File::copy tempfile.path, local
else
open(local,'w'){|f| f.write tempfile.read}
end
File::chmod 0777, local
uploaded << [org_path, true]
rescue
raise
raise tempfile.methods.inspect
uploaded << [org_path, false]
end
end

table_rows =
uploaded.collect do |u|
f, status = u
s = (status ? 'success' : 'failed')
<<-html
<tr valign=top aligh=left>
<td>
#{ CGI::escapeHTML f.inspect }
</td>
<td>
#{ s }
</td>
</tr>
html
end

table =
<<-html
<table width=100%>
<tr valign=top aligh=left>
<td> <b><u> file </b></u> </td>
<td> <b><u> status </b></u> </td>
</tr>
#{ table_rows }
</table>
html

content = <<-html
<html>
<head>
<title>#{ user }::uploaded</title>
</head>
<body>
<h3>#{ user }::uploaded</h3>
</table>
<hr>
#{ table }
<hr>
<h3><a href=#{ cgi.script_name }?m=index>index</a></h3>
</body>
</html>
html


else
content = "#{ cgi.inspect }"
content << '<hr>'
content << cgi.params.inspect
content << '<hr>'
content << cgi.params.map{|key,value| [key.inspect, value.map{|v| [v.inspect, v.class]}.inspect]}.inspect
end

rescue Exception => e
content = "<b>#{ e.message } - (#{ e.class })</b><hr>#{ e.backtrace.join '<br>' }"
ensure
cgi.out{ content.to_s }
end

 

[Michael Schilling]

Hi,

the problem is in your source i'm afraid. you can call CGI.new more than
once
in a script : each call attempts to parse ENV and STDIN according to the cgi
spec.

Thank you very much, for pointing to the error and for sending two very
nice solutions! It works great now!
Sometimes I'm so far down in the libraries, that I can't see the whole
picture anymore.


Bye, Michael


--
Michael Schilling

eMail : (e-mail address removed)
URL : http://glcu.sf.net/


"Change my name I remain the same." - Moloko