Encodign issue in Python 3.3.1 (once again)

Discussion in 'Python' started by Íßêïò Ãêñ33ê, May 26, 2013.

  1. This is the code that although correct becaus it works with englisg(standARD ASCII letters) it wont with Greek:

    if( log ):
    name = log
    # print specific client header info
    cur.execute('''SELECT hits, money FROM clients WHERE name = %s''', (name,) )
    data = cur.fetchone()
    =======================

    The following is the live output of: tail -F /usr/local/apache/logs/error_log &


    [Sun May 26 19:24:04 2013] [error] [client 46.12.46.11] Error in sys.excepthook:, referer: http://superhost.gr/cgi-bin/pelatologio.py
    [Sun May 26 19:24:04 2013] [error] [client 46.12.46.11] UnicodeEncodeError: 'ascii' codec can't encode characters in position 2050-2056: ordinal not in range(128), referer: http://superhost.gr/cgi-bin/pelatologio.py
    [Sun May 26 19:24:04 2013] [error] [client 46.12.46.11] , referer: http://superhost.gr/cgi-bin/pelatologio.py
    [Sun May 26 19:24:04 2013] [error] [client 46.12.46.11] Original exception was:, referer: http://superhost.gr/cgi-bin/pelatologio.py
    [Sun May 26 19:24:04 2013] [error] [client 46.12.46.11] Traceback (most recent call last):, referer: http://superhost.gr/cgi-bin/pelatologio.py
    [Sun May 26 19:24:04 2013] [error] [client 46.12.46.11] File "pelatologio.py", line 122, in <module>, referer: http://superhost.gr/cgi-bin/pelatologio.py
    [Sun May 26 19:24:04 2013] [error] [client 46.12.46.11] cur.execute('''SELECT hits, money FROM clients WHERE name = %s''', (name,) ), referer: http://superhost.gr/cgi-bin/pelatologio.py
    [Sun May 26 19:24:04 2013] [error] [client 46.12.46.11] File "/opt/python3/lib/python3.3/site-packages/pymysql/cursors.py", line 108, in execute, referer: http://superhost.gr/cgi-bin/pelatologio.py
    [Sun May 26 19:24:04 2013] [error] [client 46.12.46.11] query = query.encode(charset), referer: http://superhost.gr/cgi-bin/pelatologio.py
    [Sun May 26 19:24:04 2013] [error] [client 46.12.46.11] UnicodeEncodeError: 'latin-1' codec can't encode characters in position 46-52: ordinal not in range(256), referer: http://superhost.gr/cgi-bin/pelatologio.py


    I can udnerstand that this is an encoding issue but i dont knwo how to fix this.
    please help.

    The search works only for english letters as expected but my cleints have Greek name and surnames.
     
    Íßêïò Ãêñ33ê, May 26, 2013
    #1
    1. Advertisements

  2. Any idea how to correct this encoding issue?
     
    Íßêïò Ãêñ33ê, May 26, 2013
    #2
    1. Advertisements

  3. This appears to be an issue with your mysql database and how you set it
    up.. It is using charset='latin-1' whereas you need it to consistently
    use charset='utf8' both for storing strings and for retrieval. Check the
    mysql manual or ask on a mysql list for how to do that. I have no idea
    so don't ask me, or send mail to my emall address.
     
    Terry Jan Reedy, May 26, 2013
    #3
  4. No thi is not a mysql issue becaus ei have this line above for storing and retrieval form database.

    con = pymysql.connect( db = 'metrites', host = 'localhost', user = 'me', passwd = 'somepass', init_command='SET NAMES UTF8' )
     
    Íßêïò Ãêñ33ê, May 26, 2013
    #4
  5. Research MySQL and UTF-8, then see how your command is actually
    flawed. Then research ESR and asking questions the smart way, and see
    how your post is also flawed.

    ChrisA
     
    Chris Angelico, May 26, 2013
    #5
  6. No wonder the Greek economy is so screwed up.
     
    Mark Lawrence, May 26, 2013
    #6
  7. ----------------------------------------
    LOL LOL LOL LOL LOL LOL HAHAHAHAHA!!!!
     
    Carlos Nepomuceno, May 26, 2013
    #7
  8. Do you think the Greek economy would be improved by Python coming with
    PostgreSQL bindings? That'd push a lot of people onto a better
    database engine than MySQL.

    ChrisA
     
    Chris Angelico, May 26, 2013
    #8
  9. Even the best economies would be improved by Python coming with
    PostgreSQL bindings, but if and only if the user knew how to use them :)
     
    Mark Lawrence, May 26, 2013
    #9
  10. But iu have it set up for 'utf-8' as seen in this statement.

    con = pymysql.connect( db = 'metrites', host = 'localhost', user =
    'me', passwd = 'somepass', charset='utf-8', init_command='SET NAMES UTF8' )

    Yoiu mean i shoudl chnag eit to greek isoo= (iso-8859-7)

    but then i store english names and greek names as well, so it has to be a utf-8.
     
    Íßêïò Ãêñ33ê, May 27, 2013
    #10
  11. I cant solve this plz help!
     
    Íßêïò Ãêñ33ê, May 27, 2013
    #11
  12. That might not help... see below.
    Right. UTF-8 is the encoding you want.

    I think your problem is that when the MySQL database was created, it was
    not created using the UTF-8 mode, but rather latin-1 (despite your
    init_command which has nothing to do with database creation). You'll
    want to recreate the database and make double sure you've selected the
    right character set and collation:

    http://dev.mysql.com/doc/refman/5.0/en/charset-applications.html

    At least that should be your first step in debugging this problem.

    If the database is indeed utf-8, then you can move on to debugging why
    the python code is choosing latin-1 to encode your query string.

    That's my two-cents for today.
     
    Michael Torrie, May 27, 2013
    #12
  13. Sure, all you need do is get your cheque book out. Also have you ever
    heard the expression "patience is a virtue"?
     
    Mark Lawrence, May 27, 2013
    #13
  14. Ôç ÄåõôÝñá, 27 ÌáÀïõ 2013 5:45:25 ì.ì. UTC+3, ï ÷ñÞóôçò Mark Lawrence Ýãñáøå:
    Well, if i'am gonna pay someone and i will at some point because i want my script to be also re-written using a web framework(Django) it will certainly be someone that can actually provide a so;lution instead of aksing for cheque books.
     
    Íßêïò Ãêñ33ê, May 27, 2013
    #14
  15. So you want to pay someone who won't ask for money?

    It may be sordid, but you're going to have to discuss money at some
    point if you're serious about paying someone.

    Oh, and you may want to hire a typist, too. At the moment, your posts
    make you appear not to care about the job.

    ChrisA
     
    Chris Angelico, May 27, 2013
    #15
  16. Ôç ÄåõôÝñá, 27 ÌáÀïõ 2013 6:52:32 ì.ì. UTC+3, ï ÷ñÞóôçò Chris Angelico Ýãñáøå:
    I always make typos when iam typing fast.
    And yes at some point i will pay a person from this groups to alter my scripts to utilize web frameworks.
    To answer your question yes, i will be paying a person from this group thathave been providing me solutions to previous questions without laughing atmy expense and without requiring money frome to do so.
    I have 2-3 people in mind.
    I have managed to turn ALL my python 2.6 scripts to python 3.3.1 except this last problem i have to deal with.

    http://superhost.gr/?page=pelatologio.py works partiall but only displaysinfo when i click on person names in English, not with Greek names.

    If someone want to give me a hand with this, please do.
     
    Íßêïò Ãêñ33ê, May 27, 2013
    #16
  17. I have checked the database through phpMyAdmin and it is indeed UTF-8.

    I have no idea why python 3.3.1 chooses to work with latin-iso only....
     
    Íßêïò Ãêñ33ê, May 27, 2013
    #17
  18. Yes, but then you don't go back and correct them before hitting Send,
    which is what makes you appear to care very little. Typos in real-time
    chat are understandable (though even on MUDs, most people tend to
    broadcast less typos than your posts have, because even there there's
    time to edit before hitting Enter), but with email, it's very easy to
    spend 60 seconds glancing over your post before you send it. You're
    using gmail, so chances are all you need to do is use a web browser
    with a spellchecker and you'll catch all of the typos; I use Google
    Chrome, and it has the now-prevalent red squiggly lines under all
    suspected errors. (It's not perfect, of course; quite a few
    correctly-spelled words are mistakenly flagged. But it's still pretty
    effective at catching typos.)

    You're effectively asking people to put in a few minutes' work,
    sometimes quite a few minutes, to help you. Is it too much to hope
    that you'll spend one more minute on your posts?

    This is something that's utterly trivial, yet a window to your mind.
    It's like boarding an airliner and finding coffee stains on the
    flip-down trays - the plane flies just fine with those stains, yet the
    impression they give is that the crew don't care about engine
    maintenance either.

    ChrisA
     
    Chris Angelico, May 27, 2013
    #18
  19. Ôç Ôñßôç, 28 ÌáÀïõ 2013 1:18:06 ð.ì. UTC+3, ï÷ñÞóôçò Chris Angelico Ýãñáøå:
    No it is not, you are right, i should be more careful when typing and spendsome time to correct before i send my responses. I promise i will not overlook this as well as the blank line google imposes.

    As fo my initial question any comments-remarks?
     
    Íßêïò Ãêñ33ê, May 28, 2013
    #19
  20. Íßêïò Ãêñ33ê

    rusi Guest

    You say you should be more careful to correct your typing and
    'promise' to not overlook this.
    Your next line has a 'fo' instead of a 'for'
     
    rusi, May 28, 2013
    #20
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.