sqlite3 and UTF-8

[Ale Ghelfi]

I try to connect a database sqlite by sqlite3, but return an error.
Traceback (most recent call last):
File "<pyshell#20>", line 1, in <module>
rowset = cur.fetchall()
OperationalError: Could not decode to UTF-8 column 'DATO' with text
'Document n°10'

What's happend? thank you

 

[Peter Otten]

I try to connect a database sqlite by sqlite3, but return an error.

Traceback (most recent call last):
File "<pyshell#20>", line 1, in <module>
rowset = cur.fetchall()
OperationalError: Could not decode to UTF-8 column 'DATO' with text
'Document n°10'

What's happend? thank you

How did you enter the data into the database? If it was with a script under
your control modify it to feed unicode instead of str to the database.

Otherwise, if you know the encoding used in the database maybe setting
Connection.text_factory

actual_encoding = ... # whatever
def decode(s):
return s.decode(actual_encoding)

db = sqlite3.connect(...)
db.text_factory = decode


helps (untested). See also

http://docs.python.org/library/sqlite3.html#sqlite3.Connection.text_factory

Peter

 

[Ale Ghelfi]

i try this :

actual_encoding = ... # whatever
def decode(s):
return s.decode(actual_encoding)

db = sqlite3.connect(...)
db.text_factory = decode

but now the error is :
Traceback (most recent call last):
File "<pyshell#40>", line 1, in <module>
rowset = cur.fetchall()
File "<pyshell#34>", line 2, in decode
return s.decode(enc)
File "/usr/lib/python2.6/encodings/utf_8.py", line 16, in decode
return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xb0 in position 33:
invalid start byte

 

[Peter Otten]

i try this :


but now the error is :

Traceback (most recent call last):
File "<pyshell#40>", line 1, in <module>
rowset = cur.fetchall()
File "<pyshell#34>", line 2, in decode
return s.decode(enc)
File "/usr/lib/python2.6/encodings/utf_8.py", line 16, in decode
return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xb0 in position 33:
invalid start byte

So you specified

actual_encoding = "UTF-8"

? That's pointless because UTF-8 is the default, and you've already seen
that failing. You can set

db.text_factory = str

but you'll probably run into problems with that later, e. g. when you try to
display the retrieved data.

So again, what data do you expect to find in the column(s) you can't decode
properly?

Peter