How do I converted a null (0) terminated string to a Python string?

Discussion in 'Python' started by Michael, Sep 14, 2006.

  1. Michael

    Michael Guest

    Hi All,

    I've received (via UDP) a null terminated string and need to convert it
    into a Python string. Can anyone tell me how this is done? If it helps,
    I know the number of characters in the string.

    Thanks,
    M. McDonnell
     
    Michael, Sep 14, 2006
    #1
    1. Advertising

  2. Michael

    Steve Holden Guest

    Re: How do I converted a null (0) terminated string to a Pythonstring?

    Michael wrote:
    > Hi All,
    >
    > I've received (via UDP) a null terminated string and need to convert it
    > into a Python string. Can anyone tell me how this is done? If it helps,
    > I know the number of characters in the string.
    >
    > Thanks,
    > M. McDonnell
    >

    Have you received this string in Python or in C? If the former, then
    just throw away the last character of the string you've received and
    you're done!

    s = s[:-1]

    regards
    Steve
    --
    Steve Holden +44 150 684 7255 +1 800 494 3119
    Holden Web LLC/Ltd http://www.holdenweb.com
    Skype: holdenweb http://holdenweb.blogspot.com
    Recent Ramblings http://del.icio.us/steve.holden
     
    Steve Holden, Sep 14, 2006
    #2
    1. Advertising

  3. Michael

    John Machin Guest

    Michael wrote:
    > Hi All,
    >
    > I've received (via UDP) a null terminated string and need to convert it
    > into a Python string. Can anyone tell me how this is done? If it helps,
    > I know the number of characters in the string.
    >


    I think you mean NUL, not null.

    What have you received it into, if it's not a Python string?

    You probably need/want this:

    if strg[-1] == "\0":
    strg = strg[:-1]
    alternatively:
    strg = strg.rstrip("\0") # requires Python 2.2.2 or later

    It's possible you may be talking about a fixed length string which
    contains useful_stuff + "\0" + padding -- in that case you need

    strg = strg.split("\0")[0] # grab upto (but not including) the first
    NUL (if any)

    If you're not sure what you've got, print repr(the_input_string)

    HTH,
    John
     
    John Machin, Sep 14, 2006
    #3
  4. Michael

    Michael Guest

    Thank you very much for your responses. To answer some of the
    questions... Yes, I am in Python receiving a C language 0 terminated
    string that was sent to my Python program in a UDP packet (which is how
    I know the count). Are your responses still correct given this
    clarification?

    Thanks much,
    MDM

    John Machin wrote:
    > Michael wrote:
    > > Hi All,
    > >
    > > I've received (via UDP) a null terminated string and need to convert it
    > > into a Python string. Can anyone tell me how this is done? If it helps,
    > > I know the number of characters in the string.
    > >

    >
    > I think you mean NUL, not null.
    >
    > What have you received it into, if it's not a Python string?
    >
    > You probably need/want this:
    >
    > if strg[-1] == "\0":
    > strg = strg[:-1]
    > alternatively:
    > strg = strg.rstrip("\0") # requires Python 2.2.2 or later
    >
    > It's possible you may be talking about a fixed length string which
    > contains useful_stuff + "\0" + padding -- in that case you need
    >
    > strg = strg.split("\0")[0] # grab upto (but not including) the first
    > NUL (if any)
    >
    > If you're not sure what you've got, print repr(the_input_string)
    >
    > HTH,
    > John
     
    Michael, Sep 14, 2006
    #4
  5. Michael

    John Machin Guest

    Michael top-posted [corrected]:
    > John Machin wrote:
    > > Michael wrote:
    > > > Hi All,
    > > >
    > > > I've received (via UDP) a null terminated string and need to convert it
    > > > into a Python string. Can anyone tell me how this is done? If it helps,
    > > > I know the number of characters in the string.
    > > >

    > >
    > > I think you mean NUL, not null.
    > >
    > > What have you received it into, if it's not a Python string?
    > >
    > > You probably need/want this:
    > >
    > > if strg[-1] == "\0":
    > > strg = strg[:-1]
    > > alternatively:
    > > strg = strg.rstrip("\0") # requires Python 2.2.2 or later
    > >
    > > It's possible you may be talking about a fixed length string which
    > > contains useful_stuff + "\0" + padding -- in that case you need
    > >
    > > strg = strg.split("\0")[0] # grab upto (but not including) the first
    > > NUL (if any)
    > >
    > > If you're not sure what you've got, print repr(the_input_string)
    > >
    > > HTH,
    > > John

    > Thank you very much for your responses. To answer some of the
    > questions... Yes, I am in Python receiving a C language 0 terminated
    > string that was sent to my Python program in a UDP packet (which is how
    > I know the count). Are your responses still correct given this
    > clarification?


    My responses are correct. Your "clarification" indicates to me that you
    are going by what you are told, not by inspection of (several instances
    of) the packet contents, using repr(). It's up to you whether you want
    to be skeptical about the packet contents or not. I certainly wouldn't
    be throwing the last byte away without checking that it was in fact a
    NUL.

    Cheers,
    John
     
    John Machin, Sep 14, 2006
    #5
  6. Michael

    Michael Guest

    John,

    Thanks for your reply. Just wondering... how are Python strings
    formatted? Evidently they're not 0 terminated.

    Thanks again,
    MDM

    John Machin wrote:
    > Michael top-posted [corrected]:
    > > John Machin wrote:
    > > > Michael wrote:
    > > > > Hi All,
    > > > >
    > > > > I've received (via UDP) a null terminated string and need to convert it
    > > > > into a Python string. Can anyone tell me how this is done? If it helps,
    > > > > I know the number of characters in the string.
    > > > >
    > > >
    > > > I think you mean NUL, not null.
    > > >
    > > > What have you received it into, if it's not a Python string?
    > > >
    > > > You probably need/want this:
    > > >
    > > > if strg[-1] == "\0":
    > > > strg = strg[:-1]
    > > > alternatively:
    > > > strg = strg.rstrip("\0") # requires Python 2.2.2 or later
    > > >
    > > > It's possible you may be talking about a fixed length string which
    > > > contains useful_stuff + "\0" + padding -- in that case you need
    > > >
    > > > strg = strg.split("\0")[0] # grab upto (but not including) the first
    > > > NUL (if any)
    > > >
    > > > If you're not sure what you've got, print repr(the_input_string)
    > > >
    > > > HTH,
    > > > John

    > > Thank you very much for your responses. To answer some of the
    > > questions... Yes, I am in Python receiving a C language 0 terminated
    > > string that was sent to my Python program in a UDP packet (which is how
    > > I know the count). Are your responses still correct given this
    > > clarification?

    >
    > My responses are correct. Your "clarification" indicates to me that you
    > are going by what you are told, not by inspection of (several instances
    > of) the packet contents, using repr(). It's up to you whether you want
    > to be skeptical about the packet contents or not. I certainly wouldn't
    > be throwing the last byte away without checking that it was in fact a
    > NUL.
    >
    > Cheers,
    > John
     
    Michael, Sep 14, 2006
    #6
  7. Michael

    John Machin Guest

    Michael top-posted [again]:
    >
    > John Machin wrote:
    > > Michael top-posted [corrected]:
    > > > John Machin wrote:
    > > > > Michael wrote:
    > > > > > Hi All,
    > > > > >
    > > > > > I've received (via UDP) a null terminated string and need to convert it
    > > > > > into a Python string. Can anyone tell me how this is done? If it helps,
    > > > > > I know the number of characters in the string.
    > > > > >
    > > > >
    > > > > I think you mean NUL, not null.
    > > > >
    > > > > What have you received it into, if it's not a Python string?
    > > > >
    > > > > You probably need/want this:
    > > > >
    > > > > if strg[-1] == "\0":
    > > > > strg = strg[:-1]
    > > > > alternatively:
    > > > > strg = strg.rstrip("\0") # requires Python 2.2.2 or later
    > > > >
    > > > > It's possible you may be talking about a fixed length string which
    > > > > contains useful_stuff + "\0" + padding -- in that case you need
    > > > >
    > > > > strg = strg.split("\0")[0] # grab upto (but not including) the first
    > > > > NUL (if any)
    > > > >
    > > > > If you're not sure what you've got, print repr(the_input_string)
    > > > >
    > > > > HTH,
    > > > > John
    > > > Thank you very much for your responses. To answer some of the
    > > > questions... Yes, I am in Python receiving a C language 0 terminated
    > > > string that was sent to my Python program in a UDP packet (which is how
    > > > I know the count). Are your responses still correct given this
    > > > clarification?

    > >
    > > My responses are correct. Your "clarification" indicates to me that you
    > > are going by what you are told, not by inspection of (several instances
    > > of) the packet contents, using repr(). It's up to you whether you want
    > > to be skeptical about the packet contents or not. I certainly wouldn't
    > > be throwing the last byte away without checking that it was in fact a
    > > NUL.
    > >
    > > Cheers,
    > > John

    > John,
    >
    > Thanks for your reply. Just wondering... how are Python strings
    > formatted? Evidently they're not 0 terminated.


    A Python string is an object. The details of the internal storage may
    vary between implementations. CPython has 8-bit str objects and 16-bit
    or 32-bit Unicode objects. In IronPython, (str is Unicode) is true, and
    they are 16 bits. In any case the object knows its own length without
    having to scan for a terminator. Thus, a string can contain NULs.

    Having said all that, the CPython str implementation does have an
    additional byte at the end; this is set to zero and is not counted in
    the length. However you never see that and don't really need to know
    unless you are writing an extension module in C -- it's handy to know
    that you don't have to append a NUL if you want to call a C library
    function.

    Cheers,
    John
     
    John Machin, Sep 14, 2006
    #7
  8. Re: How do I converted a null (0) terminated string to a Pythonstring?

    Michael wrote:

    > Thanks for your reply. Just wondering... how are Python strings
    > formatted? Evidently they're not 0 terminated.


    have you tried *printing* the thing you got via UDP?

    to get a programmer-friendly representation of an arbitrary object, use

    print repr(obj)

    (where obj is your string, in this case).

    </F>
     
    Fredrik Lundh, Sep 14, 2006
    #8
  9. Michael

    John Machin Guest

    Fredrik Lundh wrote:
    > Michael wrote:
    >
    > > Thanks for your reply. Just wondering... how are Python strings
    > > formatted? Evidently they're not 0 terminated.

    >
    > have you tried *printing* the thing you got via UDP?
    >
    > to get a programmer-friendly representation of an arbitrary object, use
    >
    > print repr(obj)
    >
    > (where obj is your string, in this case).
    >


    Probably not; there was no indication after the two messages where I
    mentioned repr :)
     
    John Machin, Sep 14, 2006
    #9
  10. Michael

    Michael Guest

    John,

    Since I'm new to Python, I'm having trouble understanding what this
    means (see below). Would appreciate any help.

    if strg[-1] == "\0":
    strg = strg[:-1]

    Thanks,
    MDM

    John Machin wrote:
    > Fredrik Lundh wrote:
    > > Michael wrote:
    > >
    > > > Thanks for your reply. Just wondering... how are Python strings
    > > > formatted? Evidently they're not 0 terminated.

    > >
    > > have you tried *printing* the thing you got via UDP?
    > >
    > > to get a programmer-friendly representation of an arbitrary object, use
    > >
    > > print repr(obj)
    > >
    > > (where obj is your string, in this case).
    > >

    >
    > Probably not; there was no indication after the two messages where I
    > mentioned repr :)
     
    Michael, Sep 14, 2006
    #10
  11. Michael

    John Machin Guest

    Michael wrote:
    > John,
    >
    > Since I'm new to Python, I'm having trouble understanding what this
    > means (see below). Would appreciate any help.
    >
    > if strg[-1] == "\0":


    If the last (i.e index -1) byte in the string equals ASCII NUL:

    > strg = strg[:-1]


    then take a slice of the string from the start up to but not including
    the last byte and assign that to "strg"

    In other words, if the last byte of strg is NUL, throw it away.

    The truly paranoid would code that as
    if strg and strg[-1] etc etc
    so that it wouldn't blow up if strg is empty -- strange things can
    happen when you are reading other people's data :)

    Perhaps you should work through the tutorial; all the above concepts
    are treated in this section:
    http://docs.python.org/tut/node5.html#SECTION005120000000000000000

    HTH,
    John
     
    John Machin, Sep 14, 2006
    #11
  12. In <>, John Machin
    wrote:

    > In other words, if the last byte of strg is NUL, throw it away.
    >
    > The truly paranoid would code that as
    > if strg and strg[-1] etc etc
    > so that it wouldn't blow up if strg is empty -- strange things can
    > happen when you are reading other people's data :)


    I would spell it:

    if strg.endswith('\0'):
    strg = strg[:-1]

    Ciao,
    Marc 'BlackJack' Rintsch
     
    Marc 'BlackJack' Rintsch, Sep 14, 2006
    #12
  13. "Marc 'BlackJack' Rintsch" <> wrote in message
    news:p...

    > I would spell it:
    >
    > if strg.endswith('\0'):
    > strg = strg[:-1]


    I would just go with: strg = strg.rstrip('\0')
     
    Richard Brodie, Sep 14, 2006
    #13
  14. Michael

    Michael Guest

    I guess, I still don't see how this will work. I'm receiving a C
    zero-terminated string in my Python program as a 1K byte block (UDP
    datagram). If the string sent was "abc", then what I receive in Python
    is <a><b><c><0><garbage><garbage>...<last_garbage_byte>. How is Python
    going to know where in this 1K byte block the end of the string is? It
    seems that what I need to do is tell Python that the string ends at
    zero-relative index 3. What am I missing here?

    Marc 'BlackJack' Rintsch wrote:
    > In <>, John Machin
    > wrote:
    >
    > > In other words, if the last byte of strg is NUL, throw it away.
    > >
    > > The truly paranoid would code that as
    > > if strg and strg[-1] etc etc
    > > so that it wouldn't blow up if strg is empty -- strange things can
    > > happen when you are reading other people's data :)

    >
    > I would spell it:
    >
    > if strg.endswith('\0'):
    > strg = strg[:-1]
    >
    > Ciao,
    > Marc 'BlackJack' Rintsch
     
    Michael, Sep 14, 2006
    #14
  15. Michael

    Robert Kern Guest

    Re: How do I converted a null (0) terminated string to a Pythonstring?

    Michael wrote:
    > I guess, I still don't see how this will work. I'm receiving a C
    > zero-terminated string in my Python program as a 1K byte block (UDP
    > datagram). If the string sent was "abc", then what I receive in Python
    > is <a><b><c><0><garbage><garbage>...<last_garbage_byte>. How is Python
    > going to know where in this 1K byte block the end of the string is? It
    > seems that what I need to do is tell Python that the string ends at
    > zero-relative index 3. What am I missing here?


    Nothing. This is what I would do:


    In [34]: s
    Out[34]: 'abc\x00garbage'

    In [35]: s.split('\x00', 1)[0]
    Out[35]: 'abc'

    In [36]: s.split?
    Type: builtin_function_or_method
    Base Class: <type 'builtin_function_or_method'>
    String Form: <built-in method split of str object at 0x6ada2c8>
    Namespace: Interactive
    Docstring:
    S.split([sep [,maxsplit]]) -> list of strings

    Return a list of the words in the string S, using sep as the
    delimiter string. If maxsplit is given, at most maxsplit
    splits are done. If sep is not specified or is None, any
    whitespace string is a separator.


    Using the maxsplit argument saves split from having to do unnecessary work
    splitting the garbage portion if there are nulls there, too.

    --
    Robert Kern

    "I have come to believe that the whole world is an enigma, a harmless enigma
    that is made terrible by our own mad attempt to interpret it as though it had
    an underlying truth."
    -- Umberto Eco
     
    Robert Kern, Sep 14, 2006
    #15
  16. Michael

    Robert Kern Guest

    Re: How do I converted a null (0) terminated string to a Pythonstring?

    Robert Kern wrote:
    > Michael wrote:
    >> I guess, I still don't see how this will work. I'm receiving a C
    >> zero-terminated string in my Python program as a 1K byte block (UDP
    >> datagram). If the string sent was "abc", then what I receive in Python
    >> is <a><b><c><0><garbage><garbage>...<last_garbage_byte>. How is Python
    >> going to know where in this 1K byte block the end of the string is? It
    >> seems that what I need to do is tell Python that the string ends at
    >> zero-relative index 3. What am I missing here?

    >
    > Nothing. This is what I would do:
    >
    >
    > In [34]: s
    > Out[34]: 'abc\x00garbage'
    >
    > In [35]: s.split('\x00', 1)[0]
    > Out[35]: 'abc'


    And I see that this is the advice that John Machin already gave you. Shame on me
    for not reading the thread before replying.

    --
    Robert Kern

    "I have come to believe that the whole world is an enigma, a harmless enigma
    that is made terrible by our own mad attempt to interpret it as though it had
    an underlying truth."
    -- Umberto Eco
     
    Robert Kern, Sep 14, 2006
    #16
  17. Re: How do I converted a null (0) terminated string to a Pythonstring?

    Marc 'BlackJack' Rintsch wrote:

    > I would spell it:
    >
    > if strg.endswith('\0'):
    > strg = strg[:-1]


    unless you're in a hurry; startswith and endswith are horribly
    inefficient compared to ordinary indexing/slicing.

    (as I've pointed out elsewhere, even "s[:len(t)] == t" is usually faster
    than "s.startswith(t)" for short prefixes, where "short" is a lot longer
    than you may think).

    </F>
     
    Fredrik Lundh, Sep 14, 2006
    #17
  18. Re: How do I converted a null (0) terminated string to aPython string?

    At Thursday 14/9/2006 11:45, Michael wrote:

    >Since I'm new to Python, I'm having trouble understanding what this
    >means (see below). Would appreciate any help.
    >
    >if strg[-1] == "\0":
    > strg = strg[:-1]


    The Python Tutorial will answer your questions, it is enlightning and
    easy to follow.



    Gabriel Genellina
    Softlab SRL





    __________________________________________________
    Preguntá. Respondé. Descubrí.
    Todo lo que querías saber, y lo que ni imaginabas,
    está en Yahoo! Respuestas (Beta).
    ¡Probalo ya!
    http://www.yahoo.com.ar/respuestas
     
    Gabriel Genellina, Sep 14, 2006
    #18
  19. Michael

    John Machin Guest

    Michael wrote:
    > I guess, I still don't see how this will work. I'm receiving a C
    > zero-terminated string in my Python program as a 1K byte block (UDP
    > datagram). If the string sent was "abc", then what I receive in Python
    > is <a><b><c><0><garbage><garbage>...<last_garbage_byte>. How is Python
    > going to know where in this 1K byte block the end of the string is? It
    > seems that what I need to do is tell Python that the string ends at
    > zero-relative index 3. What am I missing here?
    >


    You are missing this, contained in the third message in this thread
    i.e. my first reply to you:
    """
    It's possible you may be talking about a fixed length string which
    contains useful_stuff + "\0" + padding -- in that case you need

    strg = strg.split("\0")[0] # grab upto (but not including) the first
    NUL (if any)
    """

    Please note that Robert Kern's solution is better than the above (as he
    says, it won't waste time splitting up the garbage, the mind-boggling
    relative size of which I hadn't catered for).

    You are also missing the frequent (3 at last count) exhortations to:
    print repr(your_string)
    which is much more precise than verbal descriptions.

    Cheers,
    John
     
    John Machin, Sep 14, 2006
    #19
  20. Michael

    Michael Guest

    Robert,

    Thanks to you and everyone else for the help. The "s.split('\x00',
    1)[0] " solved the problem.

    Thanks again,
    MDM

    Robert Kern wrote:
    > Michael wrote:
    > > I guess, I still don't see how this will work. I'm receiving a C
    > > zero-terminated string in my Python program as a 1K byte block (UDP
    > > datagram). If the string sent was "abc", then what I receive in Python
    > > is <a><b><c><0><garbage><garbage>...<last_garbage_byte>. How is Python
    > > going to know where in this 1K byte block the end of the string is? It
    > > seems that what I need to do is tell Python that the string ends at
    > > zero-relative index 3. What am I missing here?

    >
    > Nothing. This is what I would do:
    >
    >
    > In [34]: s
    > Out[34]: 'abc\x00garbage'
    >
    > In [35]: s.split('\x00', 1)[0]
    > Out[35]: 'abc'
    >
    > In [36]: s.split?
    > Type: builtin_function_or_method
    > Base Class: <type 'builtin_function_or_method'>
    > String Form: <built-in method split of str object at 0x6ada2c8>
    > Namespace: Interactive
    > Docstring:
    > S.split([sep [,maxsplit]]) -> list of strings
    >
    > Return a list of the words in the string S, using sep as the
    > delimiter string. If maxsplit is given, at most maxsplit
    > splits are done. If sep is not specified or is None, any
    > whitespace string is a separator.
    >
    >
    > Using the maxsplit argument saves split from having to do unnecessary work
    > splitting the garbage portion if there are nulls there, too.
    >
    > --
    > Robert Kern
    >
    > "I have come to believe that the whole world is an enigma, a harmless enigma
    > that is made terrible by our own mad attempt to interpret it as though it had
    > an underlying truth."
    > -- Umberto Eco
     
    Michael, Sep 15, 2006
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    0
    Views:
    490
  2. semut
    Replies:
    7
    Views:
    331
    Thad Smith
    Dec 5, 2006
  3. Raghavendra Mahuli

    Non Null Terminated string

    Raghavendra Mahuli, Sep 15, 2005, in forum: C++
    Replies:
    4
    Views:
    332
    Richard Herring
    Sep 15, 2005
  4. semut
    Replies:
    12
    Views:
    569
  5. ssylee
    Replies:
    4
    Views:
    521
    CBFalconer
    Aug 12, 2008
Loading...

Share This Page