[newbie] trying socket as a replacement for nc

[Jean Dubois]

I have an ethernet-rs232 adapter which allows me to connect to a measurement instrument by means of netcat on a linux system.
e.g. entering nc 10.128.59.63 7000
allows me to enter e.g.
*IDN?
after which I get an identification string of the measurement instrument back.
I thought I could accomplish the same using the python module "socket"
and tried out the sample program below which doesn't work however:
#!/usr/bin/env python

"""
A simple echo client
"""
import socket
host = '10.128.59.63'
port = 7000
size = 10
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((host,port))
s.send('*IDN?')
data = s.recv(size)
s.close()
print 'Received:', data

Can anyone here tell me how to do it properly?
thanks in advance
jean

 

[Conor Hughes]

I have an ethernet-rs232 adapter which allows me to connect to a
measurement instrument by means of netcat on a linux system.
e.g. entering nc 10.128.59.63 7000
allows me to enter e.g.
*IDN?
after which I get an identification string of the measurement
instrument back.
I thought I could accomplish the same using the python module "socket"
and tried out the sample program below which doesn't work however:

In what way does it not work? Do you not get any data? Do you get the
wrong data? Does your program block at a point which you do not
understand?

Probably more to the point, are you sure you are sending exactly the
same data as you did with netcat? netcat running with stdin a terminal
sends data line-by-line, and includes the newline in the data that it
sends. You didn't send a newline in your example.

 

[Jean Dubois]

In what way does it not work? Do you not get any data? Do you get the

wrong data? Does your program block at a point which you do not

understand?



Probably more to the point, are you sure you are sending exactly the

same data as you did with netcat? netcat running with stdin a terminal

sends data line-by-line, and includes the newline in the data that it

sends. You didn't send a newline in your example.

Thanks for the reply, I changed the line you mentioned to
s.send('*IDN?\n')
Rerunning the program then shows me as response the first time
Received:
The measurement instrument gives a beep indicating it receives something
which is however not recognized als normal input

Running the script a second time, the program hangs.. and I have to reboot the adapter

kind regards,
jean

 

[Chris Angelico]

Thanks for the reply, I changed the line you mentioned to
s.send('*IDN?\n')

See if there's a newline issue - you might need \r\n here.

ChrisA

 

[Jean Dubois]

I have an ethernet-rs232 adapter which allows me to connect to a measurement instrument by means of netcat on a linux system.


e.g. entering nc 10.128.59.63 7000

allows me to enter e.g.

*IDN?

after which I get an identification string of the measurement instrument back.

I thought I could accomplish the same using the python module "socket"

and tried out the sample program below which doesn't work however:



Sockets reserve the right to split one socket.send() into multiple socket..recv()'s on the other end of the communication, or to aggregate multiple socket.send()'s into a single socket.recv() - pretty much any way the relevant IP stacks and communications equipment feel like for the sake of performance or reliability.


The confusing thing about this is, it won't be done on every transmission- in fact, it'll probably happen rather seldom unless you're on a heavy loaded network or have some MTU issues (see Path MTU Discovery, and bear in mind that paths can change during a TCP session).  But writing your code assuming it will never happen is a bad idea.



For this reason, I wrote http://stromberg.dnsalias.org/~strombrg/bufsock.html , which abstracts away these complications, and actually makes things pretty simple.  There are examples on the web page.



HTH

Dear Dan,
Could you copy paste here the code for your function I have to add to my "program"?

thanks in advance
jean

 

[Jean Dubois]

See if there's a newline issue - you might need \r\n here.



ChrisA

I changed it as you suggested to:

s.send('*IDN?\r\n')

unfortunately this doesn't change the result, first run give as response:
Received:
second run makes the program hang and the adapter needs rebooting

kind regards,
jean

 

[Grant Edwards]

Sockets reserve the right to split one socket.send() into multiple
socket.recv()'s on the other end of the communication, or to aggregate
multiple socket.send()'s into a single socket.recv() - pretty much any way
the relevant IP stacks and communications equipment feel like for the sake
of performance or reliability.

Just to be pedantic: _TCP_ sockets reserver that right. UDP sockets
do not, and do in fact guarantee that each message is discrete. [It
appears that the OP is undoubtedly using TCP sockets.]

The confusing thing about this is, it won't be done on every
transmission - in fact, it'll probably happen rather seldom unless
you're on a heavy loaded network or have some MTU issues (see Path
MTU Discovery, and bear in mind that paths can change during a TCP
session). But writing your code assuming it will never happen is a
bad idea.

And it _will_ fail someday in some odd circumstance when, for example,
some customer is be using it via a dial-up PPP connection, or there is
a satellite link in the path, or there's a flakey router somewhere,
or...

 

[Chris Angelico]

And it _will_ fail someday in some odd circumstance when, for example,
some customer is be using it via a dial-up PPP connection, or there is
a satellite link in the path, or there's a flakey router somewhere,
or...

Or you write, write, write in quick succession. Nagle's Algorithm
means that if the first one hasn't yet been acknowledged, the second
one will be delayed, which means it'll probably be combined with the
third and sent when the first one's ACK comes through. Stream sockets
guarantee a stream of bytes; datagram sockets guarantee datagrams.
Weird how that goes, isn't it?

ChrisA

 

[Mark Lawrence]

you probably need to use something like wireshark to see what is actually
happening and compare it to a good connection in the normal way.

You've sent this twice old chap, you've a configuration issue somewhere
I'd guess

 

[Dan Stromberg]

Dear Dan,
Could you copy paste here the code for your function I have to add to my "program"?

This is untested, but it should be something like the following:

#!/usr/bin/env python

"""
A simple echo client
"""
import socket as socket_mod
import bufsock as bufsock_mod
host = '10.128.59.63'
port = 7000
size = 10
socket = socket_mod.socket(socket.AF_INET, socket.SOCK_STREAM)
socket.connect((host,port))
bufsock = bufsock_mod.bufsock(socket)
bufsock.send('*IDN?')
data = bufsock.recv(size)
bufsock.close()
print 'Received:', data

You might look over
http://stackoverflow.com/questions/...ication-server-using-python/19918706#19918706
for a more complete example.

 

[Dan Stromberg]

Sockets reserve the right to split one socket.send() into multiple
socket.recv()'s on the other end of the communication, or to aggregate
multiple socket.send()'s into a single socket.recv() - pretty much any way
the relevant IP stacks and communications equipment feel like for the sake
of performance or reliability.

Just to be pedantic: _TCP_ sockets reserver that right. UDP sockets
do not, and do in fact guarantee that each message is discrete. [It
appears that the OP is undoubtedly using TCP sockets.]

I haven't done a lot of UDP, but are you pretty sure UDP can't at
least fragment large packets? What's a router or switch to do if the
Path MTU isn't large enough for an original packet?

http://www.gamedev.net/topic/343577-fragmented-udp-packets/

 

[Chris Angelico]

Sockets reserve the right to split one socket.send() into multiple
socket.recv()'s on the other end of the communication, or to aggregate
multiple socket.send()'s into a single socket.recv() - pretty much any way
the relevant IP stacks and communications equipment feel like for the sake
of performance or reliability.

Just to be pedantic: _TCP_ sockets reserver that right. UDP sockets
do not, and do in fact guarantee that each message is discrete. [It
appears that the OP is undoubtedly using TCP sockets.]

I haven't done a lot of UDP, but are you pretty sure UDP can't at
least fragment large packets? What's a router or switch to do if the
Path MTU isn't large enough for an original packet?

http://www.gamedev.net/topic/343577-fragmented-udp-packets/

I'm no expert on this (mostly I do TCP, or UDP with fairly small
packets), but the packet should be reassembled at the far end. When
your application comes to receive it, it'll receive the entire UDP
packet as a whole.

UDP fragmentation has several problems. First, if any fragment is
lost, it won't be retransmitted (as TCP will), so the whole datagram
is lost. And secondly, if you stream data across the network in a
series of packets just a little too large to fit, each one will get
split in two and you'll end up with twice as many packets going out,
ergo abysmal performance. With TCP, there's the chance that the sender
and receiver can between them figure out what packet size to use (cf
path MTU discovery), but that won't happen with UDP unless the
application consciously does it. So it's something to be cautious of
in terms of performance, but if you want to send large UDP packets
because they make sense, just go ahead and do it.

Now, if you want reliability AND datagrams, it's a lot easier to add
boundaries to a TCP stream (sentinel or length prefixes) than to add
reliability to UDP...

ChrisA

 

[Dave Angel]

I haven't done a lot of UDP, but are you pretty sure UDP can't at
least fragment large packets? What's a router or switch to do if the
Path MTU isn't large enough for an original packet?

A router doesn't talk udp or tcp, it deals with ip packets, which it
IS permitted to fragment.

Conversely the udp layer in the os doesn't care about MTU.

 

[Jean Dubois]

Op donderdag 12 december 2013 22:23:22 UTC+1 schreef Dan Stromberg:

This is untested, but it should be something like the following:



#!/usr/bin/env python



"""

A simple echo client

"""

import socket as socket_mod

import bufsock as bufsock_mod

host = '10.128.59.63'

port = 7000

size = 10

socket = socket_mod.socket(socket.AF_INET, socket.SOCK_STREAM)

socket.connect((host,port))

bufsock = bufsock_mod.bufsock(socket)

bufsock.send('*IDN?')

data = bufsock.recv(size)

bufsock.close()

print 'Received:', data



You might look over

http://stackoverflow.com/questions/...ication-server-using-python/19918706#19918706

for a more complete example.

Thank you very much for the example, the only trouble I'm having now is installing the bufsock module:
wget http://dcs.nac.uci.edu/~strombrg/bufsock.tar.gz
results in The requested URL /~strombrg/bufsock.tar.gz was not found on this server.
Could you supply me the necessary installation instructions?

kind regards,
jean
p.s. I'm using Linux/Kubuntu 11.04

 

[Dan Stromberg]

Op donderdag 12 december 2013 22:23:22 UTC+1 schreef Dan Stromberg:

Thank you very much for the example, the only trouble I'm having now is installing the bufsock module:
wget http://dcs.nac.uci.edu/~strombrg/bufsock.tar.gz
results in The requested URL /~strombrg/bufsock.tar.gz was not found on this server.
Could you supply me the necessary installation instructions?

That's an old link. It's now at
http://stromberg.dnsalias.org/~strombrg/bufsock.html

HTH

 

[Mark Lawrence]

kind regards,
jean
p.s. I'm using Linux/Kubuntu 11.04

Would you please read and action this
https://wiki.python.org/moin/GoogleGroupsPython to prevent us seeing the
double line spacing that accompanied the above, thanks.

 

[Jean Dubois]

Op vrijdag 13 december 2013 04:32:30 UTC+1 schreef Dan Stromberg:

That's an old link. It's now at

http://stromberg.dnsalias.org/~strombrg/bufsock.html



HTH

I surfed to the new download-link (http://stromberg.dnsalias.org/svn/bufsock/) but I don't see any instructions how to download or install
bufsock.py, I see it has something to do with svn which I don't know how to handle. Could you help me with that too?

thanks in advance
jean


http://stromberg.dnsalias.org/svn/bufsock/

 

[Jean Dubois]

Op vrijdag 13 december 2013 04:32:30 UTC+1 schreef Dan Stromberg:

That's an old link. It's now at

http://stromberg.dnsalias.org/~strombrg/bufsock.html



HTH

I surfed to the new download-link (http://stromberg.dnsalias.org/svn/bufsock/) but I don't see any instructions how to download or install
bufsock.py, I see it has something to do with svn which I don't know how to handle. Could you help me with that too?

thanks in advance
jean


http://stromberg.dnsalias.org/svn/bufsock/

 

[Jean Dubois]

Op vrijdag 13 december 2013 09:35:18 UTC+1 schreef Mark Lawrence:

Would you please read and action this
https://wiki.python.org/moin/GoogleGroupsPython to prevent us seeing the
double line spacing that accompanied the above, thanks.

--

Mark Lawrence

Dear Mark,
I'm sorry for the inconvenience my postings may have caused. I now have
followed
the instructions on the link you mentioned and installed the plugin en
python-script.

hope it worked (I saw the text light up yellow when pressing the edit-key a second time). A small suggestion from a newbie: it would perhaps be possible
to make the script check itself whether pyhon2 or python3 should be used?

thanks for having patience with me
kind regards,
jean

 

[Grant Edwards]

Sockets reserve the right to split one socket.send() into multiple
socket.recv()'s on the other end of the communication, or to aggregate
multiple socket.send()'s into a single socket.recv() - pretty much any way
the relevant IP stacks and communications equipment feel like for the sake
of performance or reliability.

Just to be pedantic: _TCP_ sockets reserver that right. UDP sockets
do not, and do in fact guarantee that each message is discrete. [It
appears that the OP is undoubtedly using TCP sockets.]

I haven't done a lot of UDP, but are you pretty sure UDP can't at
least fragment large packets? What's a router or switch to do if the
Path MTU isn't large enough for an original packet?

http://www.gamedev.net/topic/343577-fragmented-udp-packets/

You're conflating IP datagrams and Ethernet packets. The IP stack can
fragment an IP datagram into multiple Ethernet packets which are then
reassembled by the receiving IP stack into a single datagram before
being passed up to the next layer (in this case, UDP).

Did you read the thread you pointed to? Your question was answerd by
posting #4 in the thread you cited:

1) Yes, packets will be fragmented at the network layer (IP), but this
is something you do not have to worry about since the network
layer will reassemble the fragments before passing them back up
to the transport layer (UDP). UDP garentees preserved message
boundaries, so you never have to worry about only receiving a
packet fragment :~).


A few other references:

http://tools.ietf.org/html/rfc791

1.1. Motivation

[...] The internet protocol provides for transmitting blocks of data
called datagrams from sources to destinations, [...] The internet
protocol also provides for fragmentation and reassembly of long
datagrams, if necessary, for transmission through "small packet"
networks.

[...]

1.4 Operation

[...]

The internet modules use fields in the internet header to fragment
and reassemble internet datagrams when necessary for transmission
through "small packet" networks.

[...]

From http://en.wikipedia.org/wiki/IP_fragmentation

If a receiving host receives a fragmented IP packet, it has to
reassemble the datagram and pass it to the higher protocol layer.
Reassembly is intended to happen in the receiving host but in
practice it may be done by an intermediate router, for example,
network address translation may need to re-assemble fragments in
order to translate data streams, e.g. the FTP control protocol, as
described in RFC 2993