Py 3: How to switch application to Unicode strings?

[Gnarlodious]

I am using Python 3, getting an error from SQLite:

sqlite3.ProgrammingError: You must not use 8-bit bytestrings unless
you use a text_factory that can interpret 8-bit bytestrings (like
text_factory = str). It is highly recommended that you instead just
switch your application to Unicode strings.

So... how do I switch to Unicode? I thought I was doing it when I put

# coding:utf-8

at the start of my script.

-- Gnarlie

 

[Mark Tolonen]

# coding:utf-8

All that does is mean that the script itself is encoded as utf8.

Actually it means that the user has declared that the source file is encoded
in utf-8. A common source of errors is that the source file is *not*
encoded in utf-8. Make sure to save the source file in the encoding
declared.

-Mark

 

[Gnarlodious]

Well, Python 3 is supposed to be all Unicode by default. I shouldn't
even need to say
# coding:UTF-8

And, the file is saved as Unicode.

There are many mentions of this error found by Google, but none seen
to clearly say what the problem is or how to fix it.

FYI, the problem line says:

cursor.execute('insert into Data values
(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)', frameTuple)

and one of the strings in the tuple contains a character like 'ñ'.
I have a version of the SQLite editor that works as expected in a
browser, I don't know why.

-- Gnarlie

 

[Arnaud Delobelle]

Well, Python 3 is supposed to be all Unicode by default. I shouldn't
even need to say
# coding:UTF-8

And, the file is saved as Unicode.

When a filed is saved, shouldn't it be in a specific encoding? I don't
see how you can save your file 'as unicode'. You should save your file
with UTF-8 encoding.

HTH

 

[Mark Tolonen]

Well, Python 3 is supposed to be all Unicode by default. I shouldn't
even need to say
# coding:UTF-8

Yes, in Python 3, an absence of a 'coding' line assumes UTF-8.

And, the file is saved as Unicode.

There is no such thing as "saved as Unicode". Unicode is not an encoding.
For example, 'ñ' is the Unicode codepoint 241. This can be stored in a file
in a number of ways. UTF-8 is the two bytes 0xc3 0xB1. UTF-16LE is 0xF1
0x00. UTF-16BE is 0x00 0xF1. latin-1 is the single byte 0xF1. If your
editor saves your file in the encoding "latin-1", and you don't use a coding
line to declare it, Python 3 will throw an error if it finds a non-UTF-8
byte sequence in the file.

There are many mentions of this error found by Google, but none seen
to clearly say what the problem is or how to fix it.

FYI, the problem line says:

cursor.execute('insert into Data values
(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)', frameTuple)

Is frameTuple a byte string or Unicode string. print(repr(frameTuple)).

and one of the strings in the tuple contains a character like 'ñ'.
I have a version of the SQLite editor that works as expected in a
browser, I don't know why.

Post the simplest, complete source code that exhibits the problem.

-Mark

 

[Gnarlodious]

Thanks for the help, but I am going to skip this problem because I
don't need Unicode characters in a script anyway.

-- Gnarlie