is there a problem on this simple code

[jrlen balane]

basically what the code does is transmit data to a hardware and then
receive data that the hardware will transmit.

import serial
import string
import time
from struct import *


ser = serial.Serial()

ser.baudrate = 9600
ser.port = 0
ser
ser.close()
ser.open()

command = 67
message_no = 1
total_data = 2

item = 12000 #to warm-up the hardware
hexed = hex(item)
data_hi, data_lo = divmod(item, 0x100)
checksum = -(data_hi + data_lo + 0x46) & 0xff
ser.write(pack('6B', command, message_no, total_data, data_lo,
data_hi, checksum)) #serial transmit protocol
time.sleep(1)

item = 10000
no = 0
for item in range(10000, 30001, 250):
no = no +1
hexed = hex(item)
data_hi, data_lo = divmod(item, 0x100)
checksum = -(data_hi + data_lo + 0x46) & 0xff
ser.write(pack('6B', command, message_no, total_data, data_lo,
data_hi, checksum))
data = ser.read(10)
(command, msg_no, no_databyte, temp1, temp2, pyra1, pyra2,
voltage, current, checksum) = unpack('10B', data) #serial receive
protocol
print no, command, msg_no, no_databyte, temp1, temp2, pyra1,
pyra2, voltage, current, checksum
time.sleep(1)

ser.close()


===========================
and here is some result after running the program on Idle Python 2.3
(enthought ed.)

1 70 168 6 0 0 0 0 0 0 12
2 70 2 6 0 0 0 0 0 0 178
3 70 3 6 0 0 0 0 0 0 177
4 70 4 6 0 0 0 0 0 0 176
5 70 5 6 0 0 0 0 0 0 175
6 70 6 6 0 0 0 0 0 0 174
7 70 7 6 0 0 0 0 0 0 173
8 70 8 6 0 0 0 0 0 0 172
9 70 9 6 0 0 0 0 0 0 171
10 70 10 6 0 0 0 0 0 0 170
11 70 11 6 0 0 0 0 0 0 169
12 70 12 6 0 0 0 0 0 0 168
13 70 13 6 0 0 0 0 0 0 167
14 70 14 6 0 0 0 0 0 0 166
15 70 15 6 0 0 0 0 0 0 165

==========================

the result i am expecting is for the data bytes (bytes 4-9) to change
its value since i am able to control the sensors which the data were
based. i am just asking for your opinion if there is something wrong
with this program, otherwise, it could be a hardware problem.

thanks in advance.

 

[Harlin Seritt]

hah, this code is anything but simple...

 

[jrlen balane]

@sir harlin
so you are saying that there is nothing wrong in this simple program.

 

[mensanator]

basically what the code does is transmit data to a hardware and then
receive data that the hardware will transmit.

import serial
import string
import time
from struct import *


ser = serial.Serial()

ser.baudrate = 9600
ser.port = 0
ser
ser.close()
ser.open()

command = 67
message_no = 1
total_data = 2

item = 12000 #to warm-up the hardware
hexed = hex(item)
data_hi, data_lo = divmod(item, 0x100)
checksum = -(data_hi + data_lo + 0x46) & 0xff
ser.write(pack('6B', command, message_no, total_data, data_lo,
data_hi, checksum)) #serial transmit protocol
time.sleep(1)

item = 10000
no = 0
for item in range(10000, 30001, 250):
no = no +1
hexed = hex(item)
data_hi, data_lo = divmod(item, 0x100)
checksum = -(data_hi + data_lo + 0x46) & 0xff
ser.write(pack('6B', command, message_no, total_data, data_lo,
data_hi, checksum))

What actually gets transmitted is "C\x01\x02\x10'\x83".
That's 18 bytes. Is the command supposed to be the ASCII
characters \x01 or a single byte whose value is 1?

 

[John Machin]

jrlen balane *TOP-POSTED*:

@sir harlin
so you are saying that there is nothing wrong in this simple program.

No, he is saying that it is not simple IHHO. Neither are your
explanations IMHO. What does "(enthought ed.)" mean??

Some observations:

The variable message_no is set to 1 initially, but thereafter is set to
the message_no returned from the hardware -- 168 the first time. This
is at best very confusing. I'd suggest two separate variables,
to_hw_msg_no and from_hw_msg_no. Does the hardware reject a message
with a number that is not ((previous msg no + 1) % 256)?

You have a similar problem with "command"; initially 67 but then is set
to what comes back from the hardware i.e. 70. Again confusing, maybe
wrong, use 2 variables to_hw_command and from_hw_command.

Some suggestions:

(1) read the manual for the hardware interface

(2) look at all the variables you have; what are they for? E.g. "hexed"
is computed but never used. Clean out the irrelevant stuff and it may
become simple enough for delicate folk like Harlin to be able to read
it

(3) are you sure you have the check-sum right? Using a hard-coded 0x46
(= 67 + 1 + 2) is dangerous! Recall command was 67 but then changes to
70! What does the manual say happens if the check-sum fails? It would
be sensible to compute the check-sum from the *ACTUAL* data that is
being sent: check_sum = 256 - (to_hw_command + to_hw_msg_no +
total_data + data_lo + data_hi) & 0xff -- but see next point.

(4) If the program is going to become more complicated, you should get
some structure into it. For example a function
bytes_to_hardware(command, msg_no, list_or_tuple_of_ints) would be a
good idea -- it would compute the data-length and the check_sum and do
the struct.pack(), ONCE.

Then you could have another function (say)
!def item_to_hardware(command, msg_no, item):
! bytes_to_hardware(command, msg_no, divmod(1tem, 256))

(5) Not only are the data values not changing, they appear to be all
zero; what does this mean?

(6) You are not validating the checksum coming back from the hardware;
the values printed out appear to be OK -- that is, they sum to zero
modulo 256; check the manual to see if this is the correct expectation!
-- but the point of having a CHECKsum is that you should CHECK it. It
may not be OK tomorrow.

(7) What does command=70 coming back from the hardware mean? Are there
"commands" that mean (a) invalid command received (b) failure in
message_number protocol (c) check_sum error in received message? If so,
augment your program to check for these. If not, or the h/w doesn't
respond appropriately when you send it a bad message, drop it down the
great white chute in your bathroom

 

[John Machin]

What actually gets transmitted is "C\x01\x02\x10'\x83".

No, that's repr(What actually gets transmitted)

That's 18 bytes. Is the command supposed to be the ASCII
characters \x01 or a single byte whose value is 1?

For a start, according to the OP's code, the command ('C' a.k.a. 67) is
first. The 1 is a meant to be a message number.

Secondly, the hardware smells like it's got an 8080 or 6502 inside. The
likelihood that it groks Python/C string representation is minimal.
Folk just don't send 18 bytes at 9600 bps when 6 bytes will do.

 

[Bengt Richter]

No, that's repr(What actually gets transmitted)

If so, that's 6 bytes, not 18:

>>> "C\x01\x02\x10'\x83" "C\x01\x02\x10'\x83"
>>> list("C\x01\x02\x10'\x83") ['C', '\x01', '\x02', '\x10', "'", '\x83']
>>> len("C\x01\x02\x10'\x83")

6

OTOH,

>>> list(r"C\x01\x02\x10'\x83") ['C', '\\', 'x', '0', '1', '\\', 'x', '0', '2', '\\', 'x', '1', '0', "'", '\\', 'x', '8', '3']
>>> len(r"C\x01\x02\x10'\x83")

18

For a start, according to the OP's code, the command ('C' a.k.a. 67) is
first. The 1 is a meant to be a message number.

Secondly, the hardware smells like it's got an 8080 or 6502 inside. The
likelihood that it groks Python/C string representation is minimal.
Folk just don't send 18 bytes at 9600 bps when 6 bytes will do.

Regards,
Bengt Richter

 

[John Machin]

No, that's repr(What actually gets transmitted)

If so, that's 6 bytes, not 18:

"C\x01\x02\x10'\x83" "C\x01\x02\x10'\x83"
list("C\x01\x02\x10'\x83") ['C', '\x01', '\x02', '\x10', "'", '\x83']
len("C\x01\x02\x10'\x83")

6

OTOH,

list(r"C\x01\x02\x10'\x83")

['C', '\\', 'x', '0', '1', '\\', 'x', '0', '2', '\\', 'x', '1', '0', "'", '\\', 'x', '8', '3']

18


For a start, according to the OP's code, the command ('C' a.k.a. 67) is
first. The 1 is a meant to be a message number.

Secondly, the hardware smells like it's got an 8080 or 6502 inside. The
likelihood that it groks Python/C string representation is minimal.
Folk just don't send 18 bytes at 9600 bps when 6 bytes will do.

Regards,
Bengt Richter

The number of bytes transmitted is 6. However the length of the visual
representation of what was sent is 18. Mensator was confused by this,
as was apparent from his/her question "Is the command supposed to be
the ASCII characters \x01 or a single byte whose value is 1?". I tried
to explain this to him/her. However it's not apparent whether your post
is part of the problem or part of the solution. Enlightenment, please.

 

[mensanator]

No, that's repr(What actually gets transmitted)

Drat, I always get burned by that.

For a start, according to the OP's code, the command ('C' a.k.a. 67) is
first. The 1 is a meant to be a message number.

Yeah, that's what I meant.

Secondly, the hardware smells like it's got an 8080 or 6502 inside. The
likelihood that it groks Python/C string representation is minimal.

That's what I was thinking. And maybe the replies he's seeing
from the hardware is an error message because it doesn't
understand what it's seeing (which could be due to other things).
A datascope would come in handy for this situation.

 

[Bengt Richter]

(e-mail address removed) wrote:

What actually gets transmitted is "C\x01\x02\x10'\x83".

No, that's repr(What actually gets transmitted)

If so, that's 6 bytes, not 18:

"C\x01\x02\x10'\x83" "C\x01\x02\x10'\x83"
list("C\x01\x02\x10'\x83")

['C', '\x01', '\x02', '\x10', "'", '\x83']

len("C\x01\x02\x10'\x83") 6

OTOH,
list(r"C\x01\x02\x10'\x83")

['C', '\\', 'x', '0', '1', '\\', 'x', '0', '2', '\\', 'x', '1', '0', "'", '\\', 'x', '8', '3']

len(r"C\x01\x02\x10'\x83") 18


That's 18 bytes. Is the command supposed to be the ASCII
characters \x01 or a single byte whose value is 1?

For a start, according to the OP's code, the command ('C' a.k.a. 67) is
first. The 1 is a meant to be a message number.

Secondly, the hardware smells like it's got an 8080 or 6502 inside. The
likelihood that it groks Python/C string representation is minimal.
Folk just don't send 18 bytes at 9600 bps when 6 bytes will do.

Regards,
Bengt Richter

The number of bytes transmitted is 6. However the length of the visual
representation of what was sent is 18. Mensator was confused by this,
as was apparent from his/her question "Is the command supposed to be
the ASCII characters \x01 or a single byte whose value is 1?". I tried
to explain this to him/her. However it's not apparent whether your post
is part of the problem or part of the solution. Enlightenment, please.

Sorry for jumping in with a largely irrelevant comment. I didn't look
at the code, just sought to illustrate the 6/18 thing further, in a kneejerk reaction.
Though BTW FWIW the visual sequence of glyphs representing the data was more a str output
than repr, I guess:
"C\x01\x02\x10'\x83"

Sorry, no enlightenment ;-)

Regards,
Bengt Richter

 

[jrlen balane]

the hardware is a school project that uses a microcontroller for "light dimming"
the message command "67" will tell the microcontroller (PIC16F877) to
do a command (to control the intensity of a lamp)
the message command "70" should tell the GUI that the microcontroller
has started transmitting.
the message sent by the GUI is different from the message sent by the
microcontroller
(but i guess sir John is right, i think i should use different
variable for the data transmitted and data received)
i am currently developing this part of the project which implements
the serial communication
i've fixed the command and total_data since i know beforehand that
this will be its value.
to simplify the program, i have also fixed the message_no since the
microcontroller will still accept the transmitted data as long as the
checksum == 0.

anymore suggestion???...
==================================================
command = 67
message_no = 1
total_data = 2
item=10000
for item in range(10000, 30001, 250):
ser.open()
data_hi, data_lo = divmod(item, 0x100)
checksum = -(data_hi + data_lo + 0x46) & 0xff
ser.write(pack('6B', command, message_no, total_data, data_lo,
data_hi, checksum))
data = ser.read(10)
(command, msg_no, no_databyte, temp1, temp2, pyra1, pyra2,
voltage, current, checksum) = unpack('10B', data) #serial receive
protocol
print command, msg_no, no_databyte, temp1, temp2, pyra1, pyra2,
voltage, current, checksum
ser.flushInput()
ser.close()

===================================
i've rewritten the code and deleted some unnecessary entries, now the
problem is :
21 6 64 64 192 0 0 0 175 70
70 2 6 64 64 192 0 0 0 114
70 11 6 64 64 192 0 0 0 105
0 0 104 70 2 6 64 64 192 0
70 2 6 64 64 192 0 0 0 114
128 128 103 70 2 6 64 64 192 0
70 2 6 64 64 192 0 0 0 114
16 208 246 70 2 6 64 64 192 0
70 2 6 64 64 192 0 0 0 114

===================================
the received data does not always start with the command "70",
is this ok??? since i can always search first for the command "70"
before i read the remaining 9 bytes, then calculate first for the
checksum before finally accepting the received data.

am i making sense here?! please help me...

 

[jrlen balane]

this is not working, what is wrong with this code?? what it "should"
do is find first the command "70" then read the remaining 9 bytes once
the command was found:

rx_data1=0
while (rx_data1 != 0x46):
rx_data1 = ser.read(1)
(rx_command) = unpack('1B', rx_data1)

rc_data2=ser.read(9)
(rx_msg_no, rx_no_databyte, temp1, temp2, pyra1, pyra2, voltage,
current, rx_checksum) = unpack('9B', data)
print rx_command, rx_msg_no, rx_no_databyte, temp1, temp2, pyra1,
pyra2, voltage, current, rx_checksum
===================================

this replaces this chunk from the original code:
data = ser.read(10)
(command, msg_no, no_databyte, temp1, temp2, pyra1, pyra2,
voltage, current, checksum) = unpack('10B', data) #serial receive
protocol
print command, msg_no, no_databyte, temp1, temp2, pyra1, pyra2,
voltage, current, checksum

===================================

thanks again

 

[Dennis Lee Bieber]

to simplify the program, i have also fixed the message_no since the
microcontroller will still accept the transmitted data as long as the
checksum == 0.

Ah, but DOES the checksum = 0?

I notice you haven't been testing the received data checksum, or
you would have noticed that it sums ALL bytes from the command
introducer to just before the checksum value.

From the middle of your data listing:

70 2 6 64 64 192 0 0 0 114

That would seem to imply that the checksum you should be sending
ALSO incorporates all bytes. This time, from your code...

checksum = -(data_hi + data_lo + 0x46) & 0xff
ser.write(pack('6B', command, message_no, total_data, data_lo,
data_hi, checksum))

You are calculating a checksum that is not including the
message_no and total_data fields. What happens if you use:

checksum = -(command + message_no, + total_data +
data_lo, data_hi) & 0xFF

21 6 64 64 192 0 0 0 175 70
70 2 6 64 64 192 0 0 0 114
70 11 6 64 64 192 0 0 0 105
0 0 104 70 2 6 64 64 192 0
70 2 6 64 64 192 0 0 0 114

the received data does not always start with the command "70",
is this ok??? since i can always search first for the command "70"

Ah, but /can/ you search for the first 70... Notice how your
first line of data ENDED with a 70. If you treat that as the
command/response introducer, and then read 9 more bytes, you will be
reading
70 message number
2 data length
6 data1
64 data2
64 ?
192 ?
0 ?
0 ?
0 checksum

Off-hand, I'd have to wonder if you are losing bytes during your
processing -- is there any active hardware handshaking on that serial
port (9600bps is getting rather fast for no handshaking).

70 11 6 64 64 192 0 0 0 105
0 0 104 70 2 6 64 64 192 0
70 2 6 64 64 192 0 0 0 114

Note how the first line I've quoted is a valid response with message
number 11. Then, look at the next line -- "0 0 104", if you assume the
104 is a checksum, it would be consistent with a message string of "70
12 6 64 64 129 0 0 0 104". Message number of 12, an incrementing
sequence, but you've lost 7 bytes. You then have seven bytes that start
a response, but are missing the last three, before getting a full line
with message number 2 (which seems to be rcvd#+1, if you are sending a
constant 1).

before i read the remaining 9 bytes, then calculate first for the
checksum before finally accepting the received data.

am i making sense here?! please help me...

I have the feeling you should modularize down to a function to
retrieve a response that makes use of the data length... In pseudo-code
(since I don't know the serial module in use):

def getPacket():
data = []
checksum = 0
(message_code, sequence_no, data_length) = serial.read(3)
checksum = message_code + sequence_no + data_length
for d in range(data_length):
data.append(serial.read(1))
checksum += data[d]
checksum += serial.read(1)
if (-checksum & 0xFF) != 0:
#bad packet -- raise an exception?
return (message_code, sequence_no, data_length, data)
#don't need to return checksum, it has been tested already

For completeness, a send probably should be created also...

def sendPacket(message_code, sequence_no, data=None):
#don't need data_length, as that would be computed from the
#length of data itself; 0 for [] or None, otherwise len(data)


If you are losing data from lack of handshaking and data
overrun, you may need to make the getPacket routine into a thread (and
maybe another thread for sending), so it can just do continuous reading,
and for every valid response sequence put it onto a Queue, where the
main thread can extract it when it is ready to process the contents.


--

 

[Dennis Lee Bieber]

this is not working, what is wrong with this code?? what it "should"

Insufficient data... What is it /doing/ that makes you say it
"is not working".

do is find first the command "70" then read the remaining 9 bytes once
the command was found:

See my prior response, where you had a line ending a 70,
followed by a valid line starting with 70... If you were reading for the
first 70, and then took 9 more bytes, you were reading garbage -- the
code is doing exactly what you asked for.

rx_data1=0
while (rx_data1 != 0x46):
rx_data1 = ser.read(1)
(rx_command) = unpack('1B', rx_data1)

Isn't this unpack rather redundant -- assuming ser.read(1) only
reads one byte, then rx_data1 and rx_command would be identical.

rc_data2=ser.read(9)
(rx_msg_no, rx_no_databyte, temp1, temp2, pyra1, pyra2, voltage,
current, rx_checksum) = unpack('9B', data)
print rx_command, rx_msg_no, rx_no_databyte, temp1, temp2, pyra1,
pyra2, voltage, current, rx_checksum

Are your data arguments in the correct order? On your previous
message, the slots for voltage and current were coming back as 0, 0
while temp1 and temp2 had non-zero values.

--

 

[John Machin]

the hardware is a school project that uses a microcontroller for "light dimming"
the message command "67" will tell the microcontroller (PIC16F877) to
do a command (to control the intensity of a lamp)
the message command "70" should tell the GUI that the microcontroller
has started transmitting.
the message sent by the GUI is different from the message sent by the
microcontroller
(but i guess sir John is right, i think i should use different
variable for the data transmitted and data received)
i am currently developing this part of the project which implements
the serial communication
i've fixed the command and total_data since i know beforehand that
this will be its value.
to simplify the program, i have also fixed the message_no since the
microcontroller will still accept the transmitted data as long as the
checksum == 0.

(1) But it's NOT zero after the first time around! (2) How do you know
what it will accept? Guessing or reading the manual? If the checksum is
not zero, then what happens? Please quote the exact section from the
manual.

anymore suggestion???...

YES: show the whole program; you have left out the initialisation part.

==================================================
command = 67
message_no = 1
total_data = 2
item=10000
for item in range(10000, 30001, 250):
ser.open()
data_hi, data_lo = divmod(item, 0x100)
checksum = -(data_hi + data_lo + 0x46) & 0xff
ser.write(pack('6B', command, message_no, total_data, data_lo,
data_hi, checksum))
data = ser.read(10)
(command, msg_no, no_databyte, temp1, temp2, pyra1, pyra2,
voltage, current, checksum) = unpack('10B', data) #serial receive
protocol
print command, msg_no, no_databyte, temp1, temp2, pyra1, pyra2,
voltage, current, checksum
ser.flushInput()
ser.close()

===================================
i've rewritten the code and deleted some unnecessary entries, now the
problem is :
21 6 64 64 192 0 0 0 175 70
70 2 6 64 64 192 0 0 0 114
70 11 6 64 64 192 0 0 0 105
0 0 104 70 2 6 64 64 192 0
70 2 6 64 64 192 0 0 0 114
128 128 103 70 2 6 64 64 192 0
70 2 6 64 64 192 0 0 0 114
16 208 246 70 2 6 64 64 192 0
70 2 6 64 64 192 0 0 0 114

===================================
the received data does not always start with the command "70",
is this ok??? since i can always search first for the command "70"
before i read the remaining 9 bytes, then calculate first for the
checksum before finally accepting the received data.

am i making sense here?! please help me...

As Dennis has told you, searching is pointless. You are losing data
now. Before you weren't losing data. Until you fix that problem,
fiddling with anything else is pointless. You have stopped doing
sleep() -- why? You are now doing ser.open() and ser.flushInput() and
ser.close() on *each* time around the loop -- why? I'd strongly suggest
you put these back the way they were. THEN do what I told you to do:
use a separate tx_command and rx_command. The first time through the
loop, command is set to 70 from the received data, and then the second
time around, you SEND 70, not 67!!! Fix that, and then show us what you
get. DON'T try searching for a 70. Don't thrash around trying multiple
changes at once -- make changes one at a time so you can see the
effects.

Do you have a part number for the manual which describes the 67 and 70
"commands" and the check-sum? Is the manual on the internet anywhere?

 

[jrlen balane]

the assembly program for the microcontroller is created by a
classmate. he based the protocol for the serial program from a
protocol he found in the internet. unfortunately, i can't find the
fpdf file, guess i'll just ask him later when he comes back.

 

[Peter Hansen]

Sorry for jumping in with a largely irrelevant comment. I didn't look
at the code, just sought to illustrate the 6/18 thing further, in a kneejerk reaction.
Though BTW FWIW the visual sequence of glyphs representing the data was more a str output
than repr, I guess:

"C\x01\x02\x10'\x83"

Actually, both of those have an additional repr() call
courtesy of the Python interactive console. The output
of str() on that string is unprintable, but the above
representation has already been repr()ed by Python for
consumption by fragile hyoo-mans...

-Peter

 

[Peter Hansen]

Isn't this unpack rather redundant -- assuming ser.read(1) only
reads one byte, then rx_data1 and rx_command would be identical.

Brain fart... unpack converts the raw bytes to integer
values with the "B" format character. You're thinking
of what would happen with a "1c" format.

-Peter

 

[Jan Rienyer Gadil]

@ sir Peter
so you mean that it is correct (at least on the unpack() part)

when i run this program on IDLE , Python 2.3 (enthought edition),
nothing is outputted on the shell, until i decide to close the shell,
wherein it tells me if i would like to kill a process...

import serial
import string
import time
from struct import *

ser = serial.Serial()

ser.baudrate = 9600
ser.port = 0
ser
ser.close()
ser.open()

command = 67
message_no = 1
total_data = 2

item = 10000

for item in range(10000, 30001, 250):
data_hi, data_lo = divmod(item, 0x100)
checksum = -(data_hi + data_lo + 0x46) & 0xff
ser.write(pack('6B', command, message_no, total_data, data_lo,
data_hi, checksum))

rx_data1=0
while (rx_data1 != 0x46):
rx_data1 = ser.read(1)
(rx_command) = unpack('1B', rx_data1)

rx_data2=ser.read(9)
(rx_msg_no, rx_no_databyte, temp1, temp2, pyra1, pyra2, voltage,
current, rx_checksum) = unpack('9B', data)
print rx_command, rx_msg_no, rx_no_databyte, temp1, temp2, pyra1,
pyra2, voltage, current, rx_checksum

ser.close()

 

[Bengt Richter]

Actually, both of those have an additional repr() call
courtesy of the Python interactive console. The output
of str() on that string is unprintable, but the above
representation has already been repr()ed by Python for
consumption by fragile hyoo-mans... Ok,

"C\x01\x02\x10'\x83"

But note that _no_ str-type string is printable at all until you assume
that it is an encoding identifying a glyph sequence and you (re)encode/interpret
for a device (virtual or not) that has a font and can present the font
information visually to your eyes (or via some modulation of sensible
environment for other senses) ;-)

Regards,
Bengt Richter