Sending binary data over CGI

[Walter Huf]

Okay, I'm busy making a wonderful cgi program, and everything is working
wonderfully so far. I'm using the standard cgi library and the Cookie
library and they have been a huge help. Thanks to those who have created
them!
However, if I try to send binary data, like jpgs, through cgi, the data
gets corrupted in a very small manner. Whenever my program sends the
character 10, it gets converted to character 13+character 10. I found out
this only happens when I send data to stdout. If I send the data to a
standard file object, it works fine. However, with the stdout, the problem
arises.
Here is the relevant part of the code:
---Begin
from sys import stdout
def senddownloaddata(file): #file is a pointer to an open file
line=file.read(1) #file pointer opening and closing are
while len(line)==1: # handled outside of function
stdout.write(line)
line=file.read(1)
stdout.write(line)
---End
Now the code that copies files:
---Begin
curdir="C:\windows\desktop"
temp=file(curdir+"/bach.it","rb",0)
out=file(curdir+"/bach2.it","wb",0)
data=temp.read(1)
while len(data)==1:
out.write(data)
data=temp.read(1)
temp.close()
out.close()
---End

I'm using Windows 98 SE with an Apache 2.0.44 Win32 server. That all works
fine. The first line of the interactive script says:
PythonWin 2.3.2 (#49, Nov 13 2003, 10:34:54) [MSC v.1200 32 bit (Intel)] on
win32.
I'm assuming that means I have Python 2.3.2. Everything else in the program
works very nice, except for the data corruption.

Please help me with this problem. It has me stumped.
Thanks in advance!

--Walter Huf--
(e-mail address removed)

 

[Peter Otten]

However, if I try to send binary data, like jpgs, through cgi, the data
gets corrupted in a very small manner. Whenever my program sends the
character 10, it gets converted to character 13+character 10. I found out
this only happens when I send data to stdout. If I send the data to a
standard file object, it works fine. However, with the stdout, the problem
arises.

Have you tried groups.google.com? Searching c.l.py for "stdout binary"
should suffice. For now:

http://mail.python.org/pipermail/python-list/2003-May/161515.html
http://mail.python.org/pipermail/python-list/2003-May/161319.html

Peter

 

[Tim Roberts]

Okay, I'm busy making a wonderful cgi program, and everything is working
wonderfully so far. I'm using the standard cgi library and the Cookie
library and they have been a huge help. Thanks to those who have created
them!
However, if I try to send binary data, like jpgs, through cgi, the data
gets corrupted in a very small manner. Whenever my program sends the
character 10, it gets converted to character 13+character 10. I found out
this only happens when I send data to stdout. If I send the data to a
standard file object, it works fine. However, with the stdout, the problem
arises.
...
Please help me with this problem. It has me stumped.

Really? This is such a common problem that most Windows Python programmers
encounter it very early in their experiments.

The issue, of course, is that stdout is opened as a text file, not as a
binary file, and we all know that the LF to CR-LF conversion you describe
is part of the normal processing of a text file in Windows.

If you must send binary data through stdout, do this before you start to
write:

import os
import msvcrt
...
msvcrt.setmode( stdout.fileno(), os.O_BINARY )

 

[Walter Huf]

Really? This is such a common problem that most Windows Python
programmers encounter it very early in their experiments.

The issue, of course, is that stdout is opened as a text file, not as
a binary file, and we all know that the LF to CR-LF conversion you
describe is part of the normal processing of a text file in Windows.

If you must send binary data through stdout, do this before you start
to write:

import os
import msvcrt
...
msvcrt.setmode( stdout.fileno(), os.O_BINARY )

Awesome, thanks! I'm quite new at Python. I've done a lot in Visual Basic,
and even written a couple cgiprogs in basic, but my server on the
internet(http://hufman.cobalty.com) is a linux server, and, Micro$oft being
Micro$oft, there is no visual basic compiler or interpreter for linux. I
started learning Python last Sunday, so I wouldn't know anything about the
conversions. Well, I knew about the newline conversions, I think, but I
didn't know how to get around it. I couldn't find anything about it in the
help. Thank you though!

--Walter Huf--
(e-mail address removed)