unicode memory usage

Discussion in 'Python' started by Gary Robinson, Sep 3, 2003.

  1. We have an application which involves storing a lot of strings in RAM. It
    would be most convenient to use Unicode strings, but I am wary of doubling
    memory usage. My fear is based on the idea that unicode strings may take two
    bytes per character in order to accomodate non-ascii characters.

    But I don't know whether that's actually how Python strings work internally.

    So, my question: Do unicode strings in Python take substantially more memory
    than classic python strings or not, assuming the strings are generally 99%
    ASCII characters (but not 100%)?


    --Gary

    --
    Putting http://wecanstopspam.org in your email helps it pass through
    overzealous spam filters.

    Gary Robinson
    CEO
    Transpose, LLC

    207-942-3463
    http://www.transpose.com
    http://radio.weblogs.com/0101454
    Gary Robinson, Sep 3, 2003
    #1
    1. Advertising

  2. Gary Robinson wrote:

    > But I don't know whether that's actually how Python strings work internally.


    Python Unicode objects use normally 2 bytes per character, unless Python
    is built in UCS-4 mode, in which case they use 4 bytes per character.

    > So, my question: Do unicode strings in Python take substantially more memory
    > than classic python strings or not, assuming the strings are generally 99%
    > ASCII characters (but not 100%)?


    Yes; you can expect that 99% of the storage for characters are null
    bytes, then. Whether this is substantial depends on the total amount of
    storage that you need for string objects, compared to the storage needed
    for other things, or the storage available.

    Regards,
    Martin
    =?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=, Sep 4, 2003
    #2
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. metfan
    Replies:
    2
    Views:
    4,825
    Robert Olofsson
    Oct 21, 2003
  2. hvt
    Replies:
    0
    Views:
    1,183
  3. hvt
    Replies:
    0
    Views:
    1,440
  4. Krist
    Replies:
    8
    Views:
    6,296
    Arne Vajhøj
    Feb 10, 2010
  5. MrsEntity
    Replies:
    20
    Views:
    441
Loading...

Share This Page