sqlite3 and UTF-8

Discussion in 'Python' started by Ale Ghelfi, Dec 7, 2010.

  1. Ale Ghelfi

    Ale Ghelfi Guest

    I try to connect a database sqlite by sqlite3, but return an error.

    >>> rowset = cur.fetchall()

    Traceback (most recent call last):
    File "<pyshell#20>", line 1, in <module>
    rowset = cur.fetchall()
    OperationalError: Could not decode to UTF-8 column 'DATO' with text
    'Document n°10'

    What's happend? thank you
    Ale Ghelfi, Dec 7, 2010
    #1
    1. Advertising

  2. Ale Ghelfi

    Peter Otten Guest

    Ale Ghelfi wrote:

    > I try to connect a database sqlite by sqlite3, but return an error.
    >
    > >>> rowset = cur.fetchall()

    > Traceback (most recent call last):
    > File "<pyshell#20>", line 1, in <module>
    > rowset = cur.fetchall()
    > OperationalError: Could not decode to UTF-8 column 'DATO' with text
    > 'Document n°10'
    >
    > What's happend? thank you


    How did you enter the data into the database? If it was with a script under
    your control modify it to feed unicode instead of str to the database.

    Otherwise, if you know the encoding used in the database maybe setting
    Connection.text_factory

    actual_encoding = ... # whatever
    def decode(s):
    return s.decode(actual_encoding)

    db = sqlite3.connect(...)
    db.text_factory = decode


    helps (untested). See also

    http://docs.python.org/library/sqlite3.html#sqlite3.Connection.text_factory

    Peter
    Peter Otten, Dec 7, 2010
    #2
    1. Advertising

  3. Ale Ghelfi

    Ale Ghelfi Guest

    i try this :

    > actual_encoding = ... # whatever
    > def decode(s):
    > return s.decode(actual_encoding)
    >
    > db = sqlite3.connect(...)
    > db.text_factory = decode


    but now the error is :

    >>> rowset = cur.fetchall()

    Traceback (most recent call last):
    File "<pyshell#40>", line 1, in <module>
    rowset = cur.fetchall()
    File "<pyshell#34>", line 2, in decode
    return s.decode(enc)
    File "/usr/lib/python2.6/encodings/utf_8.py", line 16, in decode
    return codecs.utf_8_decode(input, errors, True)
    UnicodeDecodeError: 'utf8' codec can't decode byte 0xb0 in position 33:
    invalid start byte
    Ale Ghelfi, Dec 7, 2010
    #3
  4. Ale Ghelfi

    Peter Otten Guest

    Ale Ghelfi wrote:

    > i try this :
    >
    >> actual_encoding = ... # whatever
    >> def decode(s):
    >> return s.decode(actual_encoding)
    >>
    >> db = sqlite3.connect(...)
    >> db.text_factory = decode

    >
    > but now the error is :
    >
    > >>> rowset = cur.fetchall()

    > Traceback (most recent call last):
    > File "<pyshell#40>", line 1, in <module>
    > rowset = cur.fetchall()
    > File "<pyshell#34>", line 2, in decode
    > return s.decode(enc)
    > File "/usr/lib/python2.6/encodings/utf_8.py", line 16, in decode
    > return codecs.utf_8_decode(input, errors, True)
    > UnicodeDecodeError: 'utf8' codec can't decode byte 0xb0 in position 33:
    > invalid start byte


    So you specified

    actual_encoding = "UTF-8"

    ? That's pointless because UTF-8 is the default, and you've already seen
    that failing. You can set

    db.text_factory = str

    but you'll probably run into problems with that later, e. g. when you try to
    display the retrieved data.

    So again, what data do you expect to find in the column(s) you can't decode
    properly?

    Peter
    Peter Otten, Dec 7, 2010
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Filipe Sousa
    Replies:
    2
    Views:
    696
    Filipe Sousa
    Sep 6, 2007
  2. jmfauth
    Replies:
    4
    Views:
    308
    jmfauth
    Oct 13, 2010
  3. Grzegorz ¦liwiñski
    Replies:
    2
    Views:
    940
    Grzegorz ¦liwiñski
    Jan 19, 2011
  4. Jeffrey 'jf' Lim
    Replies:
    5
    Views:
    476
    Jeffrey 'jf' Lim
    Apr 9, 2007
  5. SunSw0rd
    Replies:
    4
    Views:
    249
    SunSw0rd
    Jul 2, 2009
Loading...

Share This Page