problem with sockets and transferring binary files

[thebiggestbangtheory]

Dear all,
I am a python newbie. I am now progressing to writing a
network app in python to learn more about it. I have a client and a
server in python. The client sends a msg to the server asking it to
tar a binary .dbxml file and then send it over to it. The steps are:
from the
1. Client sends msg to server
2. Server tars up binary file and Encrypts it using ezpycrypto
3. Server sends it to client
4. Client receives file and decrypts it and untars it

Surprisingly, the sha1 hash of the encrypted data before it is sent
from server is different from the encrypted file data received by the
client. I post some code below to provide more info about what I am
doing.

#client after sending initial msg
#get server replywhich sends back the .dbxml file
myHost = '127.0.0.1'
myPort = 20001

        s1 = socket(AF_INET, SOCK_STREAM)    # create a TCP socket
        s1.bind((myHost, myPort))            # bind it to the server
port
        s1.listen(1)                         # allow 1 simultaneous
                                      # pending connections

        connection, address = s1.accept() # connection is a new socket
        while 1:
          data = connection.recv(10000000) # receive up to 10000000
bytes
          if data:
            info=decryptfile(data)

            #we have recieved a compressed .tar.gz file
            #write out info to a file
            buf = open(currentpath+'received-container.tar.gz', 'w')
            buf.write(info)
            buf.close()

            #testing code: must be removed
            os.system('sudo sha1sum '+currentpath+'received-xml-
container.tar.gz')

            #uncompress
            os.system('sudo tar -xvzf '+currentpath+'received-xml-
container.tar.gz')

#the server after receiving the msg from client
#dump the binary file
        os.system('sudo cp '+'/'+path+'1.binary '+currentpath
+'1.binary')

        #compress the file
        os.system('sudo tar -cvzf '+currentpath+'1.bin.tar.gz
'+currentpath+'1.binary')

        #encrypt the file specified in command line
        cipher=encryptfile(secretkey, currentpath+'1.bin.tar.gz')

        #send it over to sender
        send_to_sender(cipher, senderaddress)

        #testing code: needs to be removed
        buf = open(currentpath+'cipher', 'w')
        buf.write(cipher)
        buf.close()
        os.system('sudo sha1sum '+currentpath+'cipher')

#function code
def send_to_sender(cipher, servername):
    PORT = 20001
    BUFF = 10000000
    clientSocket = socket(AF_INET, SOCK_STREAM)
    HOST = servername
    ADDR = (HOST, PORT)
    clientSocket.connect(ADDR)
    clientSocket.send(cipher)
    clientSocket.close()

What I see at the client side is

7105b60d3167f2424d9e2806c49cca86c00577ba received-container.tar.gz

gzip: stdin: unexpected end of file
tar: Unexpected EOF in archive
tar: Unexpected EOF in archive
tar: Error is not recoverable: exiting now
da39a3ee5e6b4b0d3255bfef95601890afd80709 received-container.tar.gz

gzip: stdin: unexpected end of file
tar: Child returned status 1
tar: Error exit delayed from previous errors

It seems two file pieces are received! but why? The buffer size is big
enough to accept the whole thing in one go. The binary file is about
460K in size.

Any pointers, comments will be greatly appreciated .

Thanks

 

[Gabriel Genellina]

I am a python newbie. I am now progressing to writing a
network app in python to learn more about it. [...]
Surprisingly, the sha1 hash of the encrypted data before it is sent
from server is different from the encrypted file data received by the
client.

data = connection.recv(10000000) # receive up to 10000000
bytes

recv returns *up* *to* 10000000 bytes: maybe only one byte.
You'll have to keep recv'ing the pieces until you get an empty string
(when client closes the connection), or send the file size first and then
the actual data.

buf = open(currentpath+'received-container.tar.gz', 'w')
buf.write(info)
buf.close()

If you're on Linux it doesn't matter, but the 'w' above should be 'wb' for
a binary file.

def send_to_sender(cipher, servername):
...
clientSocket.send(cipher)

Similar to recv above: send() might not send the whole string at once (try
sendall instead).

 

[thebiggestbangtheory]

           I am a python newbie. I am now progressing to writing a
network app in python to learn more about it. [...]
Surprisingly, the sha1 hash of the encrypted data before it is sent
from server is different  from the encrypted file data received by the
client.
          data = connection.recv(10000000) # receive up to 10000000
bytes

recv returns *up* *to* 10000000 bytes: maybe only one byte.
You'll have to keep recv'ing the pieces until you get an empty string  
(when client closes the connection), or send the file size first and then  
the actual data.

            buf = open(currentpath+'received-container.tar.gz', 'w')
            buf.write(info)
            buf.close()

If you're on Linux it doesn't matter, but the 'w' above should be 'wb' for  
a binary file.

def send_to_sender(cipher, servername):
    ...
    clientSocket.send(cipher)

Similar to recv above: send() might not send the whole string at once (try  
sendall instead).

Thanks! very helpful