UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb6 in position

[Îίκος]

Hello although my code is correct just today i saw this in the error_log

[Sun Sep 29 07:44:43 2013] [error] [client 173.245.49.126] Traceback
(most recent call last):
[Sun Sep 29 07:44:43 2013] [error] [client 173.245.49.126] File
"/home/nikos/public_html/cgi-bin/metrites.py", line 30, in <module>
[Sun Sep 29 07:44:43 2013] [error] [client 173.245.49.126] host =
socket.gethostbyaddr( ipval ) [0]
[Sun Sep 29 07:44:43 2013] [error] [client 173.245.49.126]
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb6 in position 0:
invalid start byte
[Sun Sep 29 07:44:43 2013] [error] [client 173.245.49.126] Premature end
of script headers: metrites.py


Which should never had happened since i use the following code:


ipval = ( os.environ.get('HTTP_CF_CONNECTING_IP') or
os.environ.get('REMOTE_ADDR', "Cannot Resolve") )
city = "Άγνωστη Πόλη"
host = "Άγνωστη ΠÏοέλευση"
try:
gi = pygeoip.GeoIP('/usr/local/share/GeoIPCity.dat')
city = gi.time_zone_by_addr( ipval )
host = socket.gethostbyaddr( ipval ) [0]
except socket.gaierror as e:
print( "metrites.py => (%s): " % lastvisit, repr( sys.exc_info() ),
file=open('/tmp/err.out', 'w') )

So vene if somethign might have gone wrong with the 'host' assignment,
'host' should have defaulted to "Άγνωστη ΠÏοέλευση"

why the utf-8 error?

 

[Îίκος]

Στις 29/9/2013 10:49 πμ, ο/η Îίκος έγÏαψε:

Hello although my code is correct just today i saw this in the error_log

[Sun Sep 29 07:44:43 2013] [error] [client 173.245.49.126] Traceback
(most recent call last):
[Sun Sep 29 07:44:43 2013] [error] [client 173.245.49.126] File
"/home/nikos/public_html/cgi-bin/metrites.py", line 30, in <module>
[Sun Sep 29 07:44:43 2013] [error] [client 173.245.49.126] host =
socket.gethostbyaddr( ipval ) [0]
[Sun Sep 29 07:44:43 2013] [error] [client 173.245.49.126]
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb6 in position 0:
invalid start byte
[Sun Sep 29 07:44:43 2013] [error] [client 173.245.49.126] Premature end
of script headers: metrites.py


Which should never had happened since i use the following code:


ipval = ( os.environ.get('HTTP_CF_CONNECTING_IP') or
os.environ.get('REMOTE_ADDR', "Cannot Resolve") )
city = "Άγνωστη Πόλη"
host = "Άγνωστη ΠÏοέλευση"
try:
gi = pygeoip.GeoIP('/usr/local/share/GeoIPCity.dat')
city = gi.time_zone_by_addr( ipval )
host = socket.gethostbyaddr( ipval ) [0]
except socket.gaierror as e:
print( "metrites.py => (%s): " % lastvisit, repr( sys.exc_info() ),
file=open('/tmp/err.out', 'w') )

So vene if somethign might have gone wrong with the 'host' assignment,
'host' should have defaulted to "Άγνωστη ΠÏοέλευση"

why the utf-8 error?

Since this is not a socket.gaierror it wants:

except Exception as e: ?

i just tried it and now it works.

but then agian why the

Re: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb6 in position
0: invalid start byte

when it comes to dtermine the 'host' ?

 

[Chris Angelico]

Hello although my code is correct just today i saw this in the error_log

You retain this ridiculous arrogance that your code is somehow
"correct", despite the problems you continually face. Do not these
problems prove that your code is NOT correct?

ChrisA

 

[Îίκος]

Στις 29/9/2013 10:53 πμ, ο/η Chris Angelico έγÏαψε:

You retain this ridiculous arrogance that your code is somehow
"correct", despite the problems you continually face. Do not these
problems prove that your code is NOT correct?

ChrisA

You fail to understand that these code i now use was written with the
help of regulars here and yes its correct.


As i said in my previous post i already identified that the problem was
at the error handling.


Since this is not a socket.gaierror it wants:

except Exception as e: ?

i just tried it and now it works.

but then again why the: UnicodeDecodeError: 'utf-8' codec can't
decode byte 0xb6 in position 0: invalid start byte

when it comes to determine the 'host' ?

 

[Chris Angelico]

Re: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb6 in position 0:
invalid start byte

Something's trying to decode a stream of bytes as UTF-8, and it's not
UTF-8. Work out what in your code is bytes and what is strings, and do
your own conversions.

http://www.joelonsoftware.com/articles/Unicode.html

READ IT. Do not write another line of code until you actually
understand what he's saying there.

ChrisA

 

[Îίκος]

Στις 29/9/2013 11:00 πμ, ο/η Chris Angelico έγÏαψε:

Something's trying to decode a stream of bytes as UTF-8, and it's not
UTF-8. Work out what in your code is bytes and what is strings, and do
your own conversions.

http://www.joelonsoftware.com/articles/Unicode.html

READ IT. Do not write another line of code until you actually
understand what he's saying there.

ChrisA

okey i will, but isnt this just weird:

How come all these daysthe followinf line work as expected:

host = socket.gethostbyaddr( ipval ) [0]

and only just today just happened to output:
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb6 in position 0:
invalid start byte erros?

Nothign got added in my script. This is the only line that tried to
determine the host.

 

[Chris Angelico]

How come all these daysthe followinf line work as expected:


host = socket.gethostbyaddr( ipval ) [0]

and only just today just happened to output:

Maybe the ipval is different. Maybe something else is causing the
error. I wonder how, in Python, you would find out which one it is?
Wouldn't it be nice if Python had a function - statement in Python 2 -
that told you what something was.

ChrisA

 

[Îίκος]

Στις 29/9/2013 11:07 πμ, ο/η Chris Angelico έγÏαψε:

How come all these daysthe followinf line work as expected:


host = socket.gethostbyaddr( ipval ) [0]

and only just today just happened to output:

Maybe the ipval is different. Maybe something else is causing the
error. I wonder how, in Python, you would find out which one it is?
Wouldn't it be nice if Python had a function - statement in Python 2 -
that told you what something was.

The snippet is this:

ipval = ( os.environ.get('HTTP_CF_CONNECTING_IP') or
os.environ.get('REMOTE_ADDR', "Cannot Resolve") )
city = "Άγνωστη Πόλη"
host = "Άγνωστη ΠÏοέλευση"
try:
gi = pygeoip.GeoIP('/usr/local/share/GeoIPCity.dat')
city = gi.time_zone_by_addr( ipval )
host = socket.gethostbyaddr( ipval ) [0]
except Exception as e:
print( "metrites.py => (%s): " % lastvisit, repr( sys.exc_info() ),
file=open('/tmp/err.out', 'w') )


ipval is suposed to get just an ip address.
ip addresses doesnt change its just number seperated by dots.
the same provider i uses yesterday i continue to use today, i just
restrated the router and got a new ip address and after the new address
got assignes to me.

unicode decore error related to host started to happen.
Can someone explain this or have a way to troubleshoot it because this
error is bugging me for several months now but not all the time just in
cases liek miens today.

 

[Chris Angelico]

ipval is suposed to get just an ip address.
ip addresses doesnt change its just number seperated by dots.

That's one way an IP address can be rendered. It's not the only one.
There's an entire category of IP addresses that's usually written with
colons rather than dots. However, I reiterate: How do you KNOW what's
in ipval? I wonder if there's an easy way for you to find out...

ChrisA

 

[Îίκος]

Στις 29/9/2013 11:23 πμ, ο/η Chris Angelico έγÏαψε:

That's one way an IP address can be rendered. It's not the only one.
There's an entire category of IP addresses that's usually written with
colons rather than dots. However, I reiterate: How do you KNOW what's
in ipval? I wonder if there's an easy way for you to find out...

ChrisA

yes it is, i take it you mean to just print the damn varibale to see
what on eath is actually outputs back....

 

[Îίκος]

Στις 29/9/2013 11:30 πμ, ο/η Îίκος έγÏαψε:

Στις 29/9/2013 11:23 πμ, ο/η Chris Angelico έγÏαψε:
yes it is, i take it you mean to just print the damn varibale to see
what on eath is actually outputs back....

Here, as exactly as i though of it:

print( ipval ) returned 46.198.177.249

just a noirmal IPv4 ip address.

which leaved the questions of:

[Sun Sep 29 07:44:43 2013] [error] [client 173.245.49.126] Traceback
(most recent call last):
[Sun Sep 29 07:44:43 2013] [error] [client 173.245.49.126] File
"/home/nikos/public_html/cgi-bin/metrites.py", line 30, in <module>
[Sun Sep 29 07:44:43 2013] [error] [client 173.245.49.126] host =
socket.gethostbyaddr( ipval ) [0]
[Sun Sep 29 07:44:43 2013] [error] [client 173.245.49.126]
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb6 in position 0:
invalid start byte
[Sun Sep 29 07:44:43 2013] [error] [client 173.245.49.126] Premature end
of script headers: metrites.py


What on earth it cant decode?

 

[Steven D'Aprano]

Στις 29/9/2013 11:30 πμ, ο/η Îίκος έγÏαψε:


Here, as exactly as i though of it:

print( ipval ) returned 46.198.177.249

.............................^^^^^^^^^^^^^^

But the error comes from:

[Sun Sep 29 07:44:43 2013] [error] [client 173.245.49.126]

..............................................^^^^^^^^^^^^^^


Given that the error occurred hours ago, I don't think you'll be able to
diagnose why it happened unless it happens again.


Another thought comes to mind... unless you have changed your habits, I
recall that you edit the source code live. If that is the case, it is
possible that the error actually occurred in a completely unrelated line,
if you happened to be editing the file at the moment the error occurred.

 

[Îίκος]

Στις 29/9/2013 12:45 μμ, ο/η Steven D'Aprano έγÏαψε:

Στις 29/9/2013 11:30 πμ, ο/η Îίκος έγÏαψε:


Here, as exactly as i though of it:

print( ipval ) returned 46.198.177.249

............................^^^^^^^^^^^^^^

But the error comes from:

[Sun Sep 29 07:44:43 2013] [error] [client 173.245.49.126]

.............................................^^^^^^^^^^^^^^


Given that the error occurred hours ago, I don't think you'll be able to
diagnose why it happened unless it happens again.


Another thought comes to mind... unless you have changed your habits, I
recall that you edit the source code live. If that is the case, it is
possible that the error actually occurred in a completely unrelated line,
if you happened to be editing the file at the moment the error occurred.

You remembr correctly, i'am still editing the source code live to
reflect instant changes.

But....at that moment and all mornign long i wasnt altering the code at
all when this error occured and i just tried it at the moment as iam
typing it this by alterting the

except Exception as e: ===> except socket.gaierror as e:
which cannot handle unicore realted errors and the exact same error
appeared with my ip address involved at the error log.

so, the question that arises again and reamins is how come 'host' vars
gets a values most of the times and sometimes it just outputs back
unicode related erros.

I must tell you that i had this problem also months ago, but since i
couldnt able to solve it i left it as it was, but it just started to
make things more and more hectic.

 

[Ned Batchelder]

Στις 29/9/2013 11:00 πμ, ο/η Chris Angelico έγÏαψε:

Something's trying to decode a stream of bytes as UTF-8, and it's not
UTF-8. Work out what in your code is bytes and what is strings, and do
your own conversions.

http://www.joelonsoftware.com/articles/Unicode.html

READ IT. Do not write another line of code until you actually
understand what he's saying there.

ChrisA

okey i will, but isnt this just weird:

How come all these daysthe followinf line work as expected:

host = socket.gethostbyaddr( ipval ) [0]

and only just today just happened to output:
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb6 in position 0:
invalid start byte erros?

Nothign got added in my script. This is the only line that tried to
determine the host.

This is the nature of Unicode pain in Python 2 (Python 3 has a different
kind!). This may help you understand what's going on:
http://nedbatchelder.com/text/unipain.html

--Ned.

 

[Dave Angel]

On 29/9/2013 06:24, Íßêïò wrote:

except Exception as e: ===> except socket.gaierror as e:
which cannot handle unicore realted errors and the exact same error
appeared with my ip address involved at the error log.

so, the question that arises again and reamins is how come 'host' vars
gets a values most of the times and sometimes it just outputs back
unicode related erros.

I must tell you that i had this problem also months ago, but since i
couldnt able to solve it i left it as it was, but it just started to
make things more and more hectic.

it would be interesting to see a more complete stack trace. But I'd
guess that in the process of looking up the ip address at the dns
servers, somebody is producing a non-ascii byte string that isn't
encoded in utf-8. It might conceivably be in your /etc/hosts file as
well, assuming gethostbyname() looks there when the internet query
fails.

 

[Îίκος]

Στις 29/9/2013 2:11 μμ, ο/η Dave Angel έγÏαψε:

On 29/9/2013 06:24, Îίκος wrote:



it would be interesting to see a more complete stack trace. But I'd
guess that in the process of looking up the ip address at the dns
servers, somebody is producing a non-ascii byte string that isn't
encoded in utf-8. It might conceivably be in your /etc/hosts file as
well, assuming gethostbyname() looks there when the internet query
fails.

Thank you for being willing to look this further.

root@secure [~]# cat /etc/hosts
127.0.0.1 localhost.localdomain localhost
84.200.17.58 secure.superhost.gr secure
root@secure [~]#

Just tell me what else file contents or error_log output you want me to
show you from my server and i will.

Here is my domain's DNS Zone Entry in case it will helpo with something:

let me show you how superhost.gr entry appears via WHM in the Zoen section.

Code:
; cPanel first:11.38.2.7 (update_time):1380448502
Cpanel::ZoneFile::VERSION:1.3 hostname:secure.superhost.gr latest:11.38.2.7
; Zone file for superhost.gr
$TTL 14400
superhost.gr. 86400 IN SOA ns1.superhost.gr. nikos.gr33k.gmail.com. (
2013092903 ;Serial Number
86400 ;refresh
7200 ;retry
3600000 ;expire
86400 )

superhost.gr. 86400 IN NS ns1.superhost.gr.
superhost.gr. 86400 IN NS ns2.superhost.gr.


superhost.gr. 14400 IN A 84.200.17.58

localhost 14400 IN A 127.0.0.1

superhost.gr. 14400 IN MX 0 superhost.gr.

mail 14400 IN CNAME superhost.gr.
www 14400 IN CNAME superhost.gr.
ftp 14400 IN CNAME superhost.gr.



As for the mail iam afrid it outputs this:
Code:
Warning MX CNAME Check WARNING: CNAME was returned for the following MX
records:
mail.superhost.gr
The CNAME(s) that were returned are listed above. This is not ok per the
RFCs and can cause problems including mail being lost!
Error MX A request returns CNAME WARNING: MX records points to a CNAME.
CNAMEs are not allowed in MX records, according to RFC974, RFC1034
3.6.2, RFC1912 2.4, and RFC2181 10.3. The problem MX record(s) are:
mail.superhost.gr points to ['superhost.gr']
This can cause problems
Do i need to chnage the mx record in cloudflare so mx records point to
somethign else being ???????

Also my webpage works as

www.superhost.gr but not as superhost.gr

I have removed my webiste from cloulflare to make thigns simpler and now
in order to viw mywebiste i must give
http://www.superhost.gr

if i give plain http://superhost.gr it doesnt load at all.

The only way is http://www.superhost.gr but no link on my webpage works
when clicked upon.

iam getting this: http://superhost.gr/cgi-sys/defaultwebpage.cgi

Clearly these are DNS and Mail issues.

i have asked PaulVPS which is my provider and they dont f***** care to
help resolve those issues, and i have paid them upo until May 2014.

I just dotn want to lsoe other customers

 

[Dave Angel]

Thank you for being willing to look this further.

Willing, but probably not able. I think I know a lot about the
language, and less about the libraries. I know very little about the
administration side of internet use. The reference to /etc/hosts is
only a guess, as I said.

 

[Îίκος]

Στις 29/9/2013 2:46 μμ, ο/η Dave Angel έγÏαψε:

Willing, but probably not able. I think I know a lot about the
language, and less about the libraries. I know very little about the
administration side of internet use. The reference to /etc/hosts is
only a guess, as I said.

Can you please point me to a direction that someone will be able to help
me with this since the provider doesn't care to do so?

 

[Chris “Kwpolska†Warrick]

Στις 29/9/2013 2:46 μμ, ο/η Dave Angel έγÏαψε:
Can you please point me to a direction that someone will be able to help me
with this since the provider doesn't care to do so?

I can point you to “find a sysadmin that will work for you and fix
your problems for moneyâ€. Where can you find one? That’s not a
question for me. I suggest looking around Greek websites, as someone
speaking the same language as you could help you better.

 

[Chris Angelico]

I can point you to “find a sysadmin that will work for you and fix
your problems for money”. Where can you find one? That’s not a
question for me. I suggest looking around Greek websites, as someone
speaking the same language as you could help you better.

To be more specific: Greek job-posting web sites. Or maybe general
job-posting web sites will let you search specifically for people who
speak Greek. Either way, you're looking for someone to hire, here.
Maybe you want a contract position, or maybe salaried, but either way,
that's what you're needing.

ChrisA