Discarding unwanted filetypes when uploading

[millrace]

Hi Guys

I'm uploading images from an html page to its server - using the html
upload. I only want to accept jpg and jpeg filetypes. I can prevent any
file whose full name does not end in one of these two easily enough,
but I also want to prevent other filetypes that have been renamed to
jpg or jpeg - which will otherwise just be uploaded with my current
name-based test.

Any ideas?

Thanks, Mike

 

[robic0]

On 25 Oct 2005 22:23:50 -0700, (e-mail address removed) wrote:
Well, you may want to try the "html" newsgroup (this is Perl).
Just curious, what do you mean by
"but I also want to prevent other filetypes that have been renamed to
jpg or jpeg"? Assuming the logic is on the client side, if you want to
prevent files that are not of the graphic file extensions stated,
there is licensed software that knows header and bodies on these
types. If you have a thin client, just html, and you want to block,
then the browser has to have that ability. If you put down a thick
client, with the licensed software in the form of an Active-x then
that would block, but your not doing that. The alternative is
to examine the image server side before you store it.

 

[usenet]

but I also want to prevent other filetypes that have been renamed to
jpg or jpeg - which will otherwise just be uploaded...

If the server is to prevent the upload, it must have some basis to make
this decision. When the improperly named file is on the CLIENT, the
server cannot see it and has no way to tell if it's really a JPEG.

You will have to accept the upload and THEN test its validity (you can
use Image::TestJPG from CPAN:
http://search.cpan.org/~jhudge/Image-TestJPG-0.9/TestJPG.pm, which was
written specifically for this purpose). Simply unlink the file (and
threaten the user, etc) if it doesn't pass the test.

If you're worried about DOS attacks where somebody may try to take down
your server by uploading gigabytes of garbage then you should approach
this concern in a different manner.

 

[millrace]

I don't think I was very clear. The 'html-upload' I referred to is just
the web-page form input. The Perl script then takes that received post
data, checks for legitimate file endings, and moves the file to a
storage directory. I think what I want to do is to examine the image
server-side, as I think you figured out. At the moment I can rename any
file's type before uploading it and the Perl script will still treat it
as a jpeg and so store it - which I don't want

 

[millrace]

I think that TestJPG sounds pretty good to me. That's excellent, thanks
both of you

 

[Gunnar Hjalmarsson]

I'm uploading images from an html page to its server - using the html
upload. I only want to accept jpg and jpeg filetypes. I can prevent any
file whose full name does not end in one of these two easily enough,
but I also want to prevent other filetypes that have been renamed to
jpg or jpeg - which will otherwise just be uploaded with my current
name-based test.

Any ideas?

Since I just did something similar, this is how I did it:

require Image::Magick;
my $image = Image::Magick->new;

my %formats = (
jpg => 'image/jpeg',
jpe => 'image/jpeg',
jpeg => 'image/jpeg',
png => 'image/png',
gif => 'image/gif',
);

my ($ext) = $file =~ /\.(\w+)$/;
$ext = lc $ext;

unless ( $formats{$ext or ''} ) {
error('The file doesn't have an expected file extension');
} elsif ( $image->Read($file) or
$image->Get('MIME') ne $formats{$ext} ) {
error('Unable to verify that the file is of the type' .
" "$formats{$ext}"");
}

As you can see, I wanted to test for more mime types than 'image/jpeg'.
In your case, Image::TestJPG may be sufficient (didn't know it existed
before David mentioned it).

 

[Scott Bryce]

In your case, Image::TestJPG may be sufficient (didn't know it existed
before David mentioned it).

Neither did I. When I needed this functionality, I rolled my own.

use strict;
use warnings;

my $filename = 'some_image_file.jpg';

print 'Yes' if is_image_JPG($filename);


sub is_image_JPG
{
my $file_name = shift;

open my $JPGFILE, '<', $file_name or die "Cannot open $filename -- $!";
binmode $JPGFILE;
read ($JPGFILE, my $string, 4);
close $JPGFILE;

return 1 if $string eq pack('C4', 0xFF, 0xD8, 0xFF, 0xE0);

return 0;
}

 

[Gunnar Hjalmarsson]

Neither did I. When I needed this functionality, I rolled my own.

read ($JPGFILE, my $string, 4);
close $JPGFILE;

return 1 if $string eq pack('C4', 0xFF, 0xD8, 0xFF, 0xE0);

I see that you only check the first four characters. Guess that leaves
room for false positives, e.g. somebody who deliberately wants to send
something bad that looks like a .jpg file could easily pass your test.
Or am I paranoid?

 

[A. Sinan Unur]

I see that you only check the first four characters. Guess that leaves
room for false positives, e.g. somebody who deliberately wants to send
something bad that looks like a .jpg file could easily pass your test.
Or am I paranoid?

No you are not. The method is next to useless. The only way to make sure
what you have is a JPEG image is to allow the upload to succeed, and
check if it indeed is.

For user friendliness, one can warn the person if the file does not meet
minimal criteria for it to be a JPEG image, however, that would not mean
squat when potentially hostile behavior has to be taken into account.

To prevent DOS, limit individual file size as a first precaution.

Then, check if the file is valid JPEG once it is on the server.

Sinan

 

[Scott Bryce]

No you are not. The method is next to useless.

I looks as though I'm not nearly paranoid enough. I appreciate the feedback.

 

[xhoster]

I see that you only check the first four characters. Guess that leaves
room for false positives, e.g. somebody who deliberately wants to send
something bad that looks like a .jpg file could easily pass your test.
Or am I paranoid?

Depends on what you are worried about. If "bad" means that it is a
malicious effort to waste storage, then people can send perfectly valid,
but useless, jpgs and waste storage that way. No way around that using
validity checks. If "bad" means people accidentally load things other than
jpgs, then Scotts solution should be fine. If "bad" means someone sends a
carefully crafted files which exploit holes in some jpg renders to do bad
things, then it is not clear (to me) that Image::TestJPG could detect all
(or even most) such efforts.


Xho

 

[Gunnar Hjalmarsson]

Virus scan files before doing anything else with new files.
Test those files with methods mentioned in this thread.

You sound a little more "paranoid" than me. ;-)

Anybody who knows if it might be dangerous to attempt to read a file
with a library such as Image::Magick? (That's the method I suggested,
but it doesn't involve virus scanning.)

 

[Gunnar Hjalmarsson]

Depends on what you are worried about. If "bad" means that it is a
malicious effort to waste storage, then people can send perfectly valid,
but useless, jpgs and waste storage that way. No way around that using
validity checks. If "bad" means people accidentally load things other than
jpgs, then Scotts solution should be fine. If "bad" means someone sends a
carefully crafted files which exploit holes in some jpg renders to do bad
things, then it is not clear (to me) that Image::TestJPG could detect all
(or even most) such efforts.

I was thinking of the latter, and the method I suggested involves
reading the file with Image::Magick, but I don't really know what IM
checks for.

 

[RedGrittyBrick]

You sound a little more "paranoid" than me. ;-)

Anybody who knows if it might be dangerous to attempt to read a file
with a library such as Image::Magick? (That's the method I suggested,
but it doesn't involve virus scanning.)

The vulnerability lies in GDIPlus.dll. AFAIK this buffer-overflow
vulnerability was fixed last year (2004).

Since ImageMagick is cross platform I'd guess it doesn't rely on MS
libraries anyway.

 

[Gunnar Hjalmarsson]

The vulnerability lies in GDIPlus.dll. AFAIK this buffer-overflow
vulnerability was fixed last year (2004).

Since ImageMagick is cross platform I'd guess it doesn't rely on MS
libraries anyway.

At least not on my Linux box... Thanks!