does not work on freeBSD but works on linux, and windows

[John fabiani]

Hi,
this is a little strange. All I'm trying to do is ftp a 'dbf' file - but
the file is corrupt after the transfer.

1. the freeBSD 4.4 is a couple of years old using:

Python 2.1.1 (#1, Sep 13 2001, 18:12:15)
[GCC 2.95.3 20010315 (release) [FreeBSD]] on freebsd4
Type "copyright", "credits" or "license" for more information.

2. below is a simple ftp program

import ftplib
import sys
remote=ftplib.FTP(host='myIpAddress',user='username',passwd='password')
mohpac07=open("/usr/home/spx/servplus/outbounds/mohpac07.dbf",'rb')
remote.set_debuglevel(2)
remote.set_pasv(0)
remote.storbinary("STOR mohpac07.dbf",mohpac07,8192)
remote.close
remote.quit

3. below is the output.

*cmd* 'TYPE I'
*put* 'TYPE I\r\n'
*get* '200 Switching to Binary mode.\r\n'
*resp* '200 Switching to Binary mode.'
*cmd* 'PORT 190,10,10,2,14,194'
*put* 'PORT 190,10,10,2,14,194\r\n'
*get* '200 PORT command successful. Consider using PASV.\r\n'
*resp* '200 PORT command successful. Consider using PASV.'
*cmd* 'STOR mohpac07.dbf'
*put* 'STOR mohpac07.dbf\r\n'
*get* '150 Ok to send data.\r\n'
*resp* '150 Ok to send data.'
*get* '226 File receive OK.\r\n'
*resp* '226 File receive OK.'


I then test the dbf file by attemping to open the file with foxpro and
the file is corrupt (not a table). The strange thing the same code
works from XP, and Linux (both are python 2.3). I don't know much about
freeBSD and I'm not sure if I can install 2.3.

Anybody no why this is happening?

TIA
John

 

[David Bolen]

2. below is a simple ftp program

import ftplib
import sys
remote=ftplib.FTP(host='myIpAddress',user='username',passwd='password')
mohpac07=open("/usr/home/spx/servplus/outbounds/mohpac07.dbf",'rb')
remote.set_debuglevel(2)
remote.set_pasv(0)
remote.storbinary("STOR mohpac07.dbf",mohpac07,8192)
remote.close
remote.quit

Just a shot in the dark, but if your last two lines are accurate,
you're simply referencing the close and quit names within the ftplib
object, but aren't actually calling them. Also, quit() includes the
close operation, so I think you'd want that to end with:

remote.quit()

Note that your log never shows the Goodbye during a clean exit from
the server.

It's possible that the platform differences you are seeing has to do
with differences in buffering at the application and/or protocol stack
layer and perhaps how stranded data is handled during process/socket
shutdown. It's possible that the final portion of the data related to
the STOR is never getting transmitted to the remote ftp server.

-- David

 

[Oliver Fromme]

> 1. the freeBSD 4.4 is a couple of years old using:
> Python 2.1.1 (#1, Sep 13 2001, 18:12:15)

FreeBSD 4.4 is quite old, and I would strongly recommend
upgrading, especially if the machine is connected to a
network. The latest stable release of FreeBSD is 4.10.
It also comes with the latest version of Python (2.3.4),
both as a ready-made binary package and as a "port" for
easy compilation.

I suggest you upgrade that machine. I'm sure the problem
won't persist.

Best regards
Oliver

 

[John fabiani]

FreeBSD 4.4 is quite old, and I would strongly recommend
upgrading, especially if the machine is connected to a
network. The latest stable release of FreeBSD is 4.10.
It also comes with the latest version of Python (2.3.4),
both as a ready-made binary package and as a "port" for
easy compilation.

I suggest you upgrade that machine. I'm sure the problem
won't persist.

Best regards
Oliver

Thanks for your reply. Can freeBSD support python 2.3? And if it does
how do I install it.
John

 

[Oliver Fromme]

> Thanks for your reply. Can freeBSD support python 2.3?

Yes, it's a standard package in FreeBSD 4.10.

> And if it does
> how do I install it.

First you should remove your old version of python, in order
to avoid any potential pitfalls. Use the pkg_delete command
for that purpose.

To install the precompiled binary package for FreeBSD 4.10,
perform this command as root:

pkg_add -r python

If you prefer to fetch the source code and compile it your-
self, type these commands as root (provided that you have
installed the ports collection, which really is strongly
recommended):

cd /usr/ports/lang/python
make install
make clean

For more details on the pkg_add(1) command, please refer to
its manual page. For more information on the FreeBSD ports
collection, please refer to the appropriate chapter in the
Handbook.

Best regards
Oliver

 

[Till Plewe]

....

Thanks for your reply. Can freeBSD support python 2.3?
yes

And if
it does how do I install it.

1) cd /usr/ports/lang/python; make install clean

or install portupgrade first

2) cd /usr/ports/sysutils/portupgrade; make install clean
portinstall python;

but to upgrade freebsd you should have a look at the handbook
at www.freebsd.org.

- Till

 

[Donn Cave]

Quoth Oliver Fromme <[email protected]>:
|> 1. the freeBSD 4.4 is a couple of years old using:
|> Python 2.1.1 (#1, Sep 13 2001, 18:12:15)
|
| FreeBSD 4.4 is quite old, and I would strongly recommend
| upgrading, especially if the machine is connected to a
| network. The latest stable release of FreeBSD is 4.10.
| It also comes with the latest version of Python (2.3.4),
| both as a ready-made binary package and as a "port" for
| easy compilation.
|
| I suggest you upgrade that machine. I'm sure the problem
| won't persist.

Wow, I'm surprised to hear that. I will have to go back
and read the problem again, but it sounds like a very
ordinary application in terms of operating system support.
Not something that would expose a bug in FreeBSD 4.4.
What makes you think it's an OS bug?

Donn Cave, (e-mail address removed)

 

[Oliver Fromme]

> Quoth Oliver Fromme <[email protected]>:
> | [...]
> | I suggest you upgrade that machine. I'm sure the problem
> | won't persist.
>
> Wow, I'm surprised to hear that. I will have to go back
> and read the problem again, but it sounds like a very
> ordinary application in terms of operating system support.
> Not something that would expose a bug in FreeBSD 4.4.
> What makes you think it's an OS bug?

By saying "upgrade" I meant both the OS and Python.
From his description, it seemed to be some kind of data
corruption, which could very well be an OS issue or a
Python issue (in libftp, for example). His application
code seemed to be correct, as far as I can tell.

That's all speculation of course, but even if the problem
persists with an up-to-date OS and Python, the update was
certainly not useless.

Best regards
Oliver

 

[John fabiani]

Quoth Oliver Fromme <[email protected]>:
| [...]
| I suggest you upgrade that machine. I'm sure the problem
| won't persist.

Wow, I'm surprised to hear that. I will have to go back
and read the problem again, but it sounds like a very
ordinary application in terms of operating system support.
Not something that would expose a bug in FreeBSD 4.4.
What makes you think it's an OS bug?

By saying "upgrade" I meant both the OS and Python.
From his description, it seemed to be some kind of data
corruption, which could very well be an OS issue or a
Python issue (in libftp, for example). His application
code seemed to be correct, as far as I can tell.

That's all speculation of course, but even if the problem
persists with an up-to-date OS and Python, the update was
certainly not useless.

Best regards
Oliver

I can't really update the OS because the main app on the server is using
SCO foxpro 2.6. This means that the OS is using the special lib's
reqired for SCO (xenix,unix libs) to run foxpro 2.6.

Anyway I know very little about FreeBSD. In fact I'll have to learn how
to upgrade Python 2.1 to Python 2.3. Of course this assumes that
problem has something to do with Python and not the under lying SO. BTW
FTP works without any issues - at least manually. Also the code I
provided is what is realy run on the computer.

Question can I just update the ftplib module? I'm thinking that might
be easier than attempt to update all of it to 2.3.

John

 

[Donn Cave]

I can't really update the OS because the main app on the server is using
SCO foxpro 2.6. This means that the OS is using the special lib's
reqired for SCO (xenix,unix libs) to run foxpro 2.6.

Anyway I know very little about FreeBSD. In fact I'll have to learn how
to upgrade Python 2.1 to Python 2.3. Of course this assumes that
problem has something to do with Python and not the under lying SO. BTW
FTP works without any issues - at least manually. Also the code I
provided is what is realy run on the computer.

Question can I just update the ftplib module? I'm thinking that might
be easier than attempt to update all of it to 2.3.

It wouldn't hurt to try, and sure wouldn't hurt to look at
what has changed - if anything.

Before embarking on a more complicated upgrade, it would
probably be worth a further look at your software. If you
missed it, go back and find David Bolen's response to your
post. If your ftp function isn't already a stand-alone program,
break it out into a stand-alone program and see if it works any
different. Look at the result file and see if you can see
specifically what's different - "it's corrupted" isn't much
to go on. BSD will work differently in some subtle ways than
Linux, and that probably won't change from FreeBSD 4.4 to 4.10 -
not all differences from Linux and Windows are bugs.

Donn Cave, (e-mail address removed)

 

[Reed L. O'Brien]

Quoth Oliver Fromme <[email protected]>:
| [...]
| I suggest you upgrade that machine. I'm sure the problem
| won't persist.
Wow, I'm surprised to hear that. I will have to go back
and read the problem again, but it sounds like a very
ordinary application in terms of operating system support.
Not something that would expose a bug in FreeBSD 4.4.
What makes you think it's an OS bug?

By saying "upgrade" I meant both the OS and Python.
From his description, it seemed to be some kind of data
corruption, which could very well be an OS issue or a
Python issue (in libftp, for example). His application
code seemed to be correct, as far as I can tell.

That's all speculation of course, but even if the problem
persists with an up-to-date OS and Python, the update was
certainly not useless.

Best regards
Oliver

I can't really update the OS because the main app on the server is
using SCO foxpro 2.6. This means that the OS is using the special
lib's reqired for SCO (xenix,unix libs) to run foxpro 2.6.

Anyway I know very little about FreeBSD. In fact I'll have to learn
how to upgrade Python 2.1 to Python 2.3. Of course this assumes that
problem has something to do with Python and not the under lying SO.
BTW FTP works without any issues - at least manually. Also the code I
provided is what is realy run on the computer.

Question can I just update the ftplib module? I'm thinking that might
be easier than attempt to update all of it to 2.3.

John

Have you a test box?
download the ports tree
get cvsup and portupgrade and use them
or build python yourself
I don't beleive the SCO libs have been removed, but can't say for certain.
Set up a test box.

 

[John fabiani]

It wouldn't hurt to try, and sure wouldn't hurt to look at
what has changed - if anything.

Before embarking on a more complicated upgrade, it would
probably be worth a further look at your software. If you
missed it, go back and find David Bolen's response to your
post. If your ftp function isn't already a stand-alone program,
break it out into a stand-alone program and see if it works any
different. Look at the result file and see if you can see
specifically what's different - "it's corrupted" isn't much
to go on. BSD will work differently in some subtle ways than
Linux, and that probably won't change from FreeBSD 4.4 to 4.10 -
not all differences from Linux and Windows are bugs.

Donn Cave, (e-mail address removed)

When I say corrupt I'm saying the program Foxpro can't open the file.
In this case the file is data file in the DBF format. The program as
posted is being used - it is not part of any other larger program. The
only thing I changed was the ip address and login info. Using the same
program (no changes) works from Linux (SUSE 8.0, SUSE 9.1 64) and
windows XP. Of course it may be that freeBSD requires a different way
of writing the program but others have reported that the program works
for different version of freeBSD (but no one has reported that freeBSD
4.4 works). If you are willing to help we can talk on the phone and
discuss it further. Just email at jfabiani at yolo.com. Again thanks
John

 

[Tuure Laurinolli]

I then test the dbf file by attemping to open the file with foxpro and

the file is corrupt (not a table). The strange thing the same code
works from XP, and Linux (both are python 2.3). I don't know much about
freeBSD and I'm not sure if I can install 2.3.

Sure you can install 2.3. The ports tree(collection of third party
software) is pretty much separate from the rest of the OS and is usually
kept up to date even on older OS versions as not doing so would result
in huge security holes. On FreeBSD updating through minor versions
should also be painless, so upgrading to 4.10 shouldn't be a big deal.
Running the SCO binary(?) should be handled by the kernel transparently.

The Handbook at freebsd.org is really helpful in learning the system.

Anyway, back to the problem, is the file of correct size? If it is, you
could try to look at it in raw format(with a hex editor most probably,
hexedit port in FreeBSD) to see if you can figure out what is garbled.

 

[David Bolen]

When I say corrupt I'm saying the program Foxpro can't open the
file. In this case the file is data file in the DBF format. The
program as posted is being used - it is not part of any other larger
program. (...)

Have you tried fixing the code itself as I suggested earlier (e.g., to
actually close the FTP connection cleanly)? If it is preventing the
final chunk of data from being transmitted across the socket, that
could easily show up as a corruption of the file from Foxpro's
perspective.

While it might not change anything, it's certainly an easier option to
try compared to some of the other items being discussed in this
thread, not to mention it's probably a bug in the code in any event.

-- David

 

[Donn Cave]

When I say corrupt I'm saying the program Foxpro can't open the file.
In this case the file is data file in the DBF format.

But what "physical" difference do you see in the file? Is it
shorter? Run "cmp" on a bad one and a good version of what
should be the same file ... do differences appear early in the
file, or late? Is the first difference on a block boundary
(divisible by 512?) That's the kind of analysis I'm talking
about.

Of course it may be that freeBSD requires a different way
of writing the program but others have reported that the program works
for different version of freeBSD (but no one has reported that freeBSD
4.4 works). If you are willing to help we can talk on the phone and
discuss it further. Just email at jfabiani at yolo.com. Again thanks
John

I'm sure someone on this list would be interested in taking
this on for a modest consulting fee. Maybe more if your
consultant has to install his or her own 4.4, but that
shouldn't be too hard (BSDs will happily install on a
single BIOS partition.)

Donn Cave, (e-mail address removed)