Get the entire file in a variable - error

[martijn]

H!

I'm using a database and now I want to compress a file and put it into
the database.

So I'm using gzip because php can open the gzip file's.
The only problem is saving the file into the database.

The function below does this:
- gzip the file [oke]
- get all the bytes with tst.getvalue() [error]
I only get the first line.

I have the same problem when I try it with file.open(), .read().

"how to get all the binary data in a variable to put that in a database
LONG field?"

Thank's

def compressFILE(sid,filedata):
tst = StringIO.StringIO()

#tmp
zip = gzip.GzipFile(str(sid)+'.gz','w',5,tst)
zip.write(filedata)

#put every byte in a database
print tst.getvalue()

zip.close()
tst.close()

return

 

[Jeff Epler]

It's not clear to me what you mean by "the first line" (gzip does not
output a file composed of lines, its output is byte-oriented).

Printing tst.getvalue() is probably not a very useful thing to do, since
it won't do anything useful when the output is a terminal, and it will
add an extra newline if you are redirecting to a file.

At least when close()ing the GzipFile before looking at the StringIO
instance's value, I get some bytes that gunzip just fine, giving the
original string.

Here's my interactive session:... Python 2.2.2 (#1, Feb 24 2003, 19:13:11)
... [GCC 3.2.2 20030222 (Red Hat Linux 3.2.2-4)] on linux2
... Type "help", "copyright", "credits" or "license" for more information.
... """)Python 2.2.2 (#1, Feb 24 2003, 19:13:11)
[GCC 3.2.2 20030222 (Red Hat Linux 3.2.2-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
I don't know anything about your database or its "LONG field".
Depending on the database software there could be additional problems
with embedded NULs, for instance.

Jeff

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.1 (GNU/Linux)

iD8DBQFCXoThJd01MZaTXX0RAgLqAJ9OZ6OEpJwB3wVUq5IlV2SFx2Zh7gCggdR4
N4njFJTpuOJ37GOuVhF8nRU=
=f5At
-----END PGP SIGNATURE-----

 

[martijn]

I mean it like this.

I must have a variable that includes a file (in this case a .gz file)
for putting that in a database. (never null)

Thanks,

 

[Pokerface]

What kind of DB? Maybe the field type you're hunting for is a BLOB? Or
perhaps you need to be formatting the string differently than it's
entered (like php's addslashes() or something)?

 

[Larry Bates]

In the future it REALLY helps if you post the code that
you used to open and read the file. Otherwise we have
to "guess" what you may of done.

That said, I'll try to guess:

1) Your code shows that you instantiate GzipFile class with
mode "w". I don't know what your data looks like but
normally you would want to use "wb" to handle binary
data.

2) You also need to do the same thing when you try to read
the file:

fp=file(filename, 'rb')

When you don't the first read will read until it encounters
binary bytest that look like a EOF and quit. When you open
with "b" mode it reads the number of bytes that the
filesytem has stored for the file.

Hope info helps.

Larry Bates

 

[John Machin]

H!

I'm using a database and now I want to compress a file and put it into
the database.

So I'm using gzip because php can open the gzip file's.
The only problem is saving the file into the database.

The function below does this:
- gzip the file [oke]
- get all the bytes with tst.getvalue() [error]
I only get the first line.

I have the same problem when I try it with file.open(), .read().

"how to get all the binary data in a variable to put that in a database
LONG field?"

Thank's

def compressFILE(sid,filedata):
tst = StringIO.StringIO()

#tmp
zip = gzip.GzipFile(str(sid)+'.gz','w',5,tst)

According to the docs, it will convert your 'w' into 'wb'. That
shouldn't be your problem. However just because I'm paranoid doesn't
mean someone isn't out to get you, so change it to 'wb' anyway. It's
an extremely good habit to put the 'b' on whenever you are reading or
writing binary data.

zip.write(filedata)

#put every byte in a database
print tst.getvalue()

Here's your problem, being over-eager. Compressors that allow you to
feed them their input a spoonful at a time will maintain a large
amount of "state" and won't finish the job until you call their
close() method. Just like if you have a process writing a file to
disk, a second process trying to read the file before the first
process closes it is not in general guaranteed to see *any* of the
file's content, let alone all of it.

zip.close()
tst.close()

OK, now you can inspect the contents of "tst", write it to your
database, and then (and only then) worry about BLOBs and embedded
nulls and other nasties.

HTH,

John