UnicodeEncode Error ?

R

Rama Vadakattu

While doing the below

1) fetch html page
2) extract title using BeatifulSoup
3) Save into the database.

iam getting the below error (at the bottom).

Few observations which i had:
1) type of variable which holds this title is <class
'BeautifulSoup.NavigableString'>
2) iam getting this problem when the title has character whose
ordinal value > 128

I searched in google and tried various suggestions but none of them is
working.And also i couldnot able to understand the rootcause for such
a kind of behaviour.

What will be the root cause for such a kind of problem?Any hint on how
to resolve this error?

~~~
Error
----------------------------------------------------------------------------------------------------------
Traceback (most recent call last):
File "fetchcron.py", line 11, in <module>
cmanager.triggerfetchFeed()
File "/home/rama/djangoprojects/socialreader/views.py", line 40, in
triggerfetchFeed
self.fetchFeed(furl)
File "/home/rama/djangoprojects/socialreader/views.py", line 102, in
fetchFeed
ne.save()
File "/usr/lib/python2.5/site-packages/django/db/models/base.py",
line 311, in save
self.save_base(force_insert=force_insert,
force_update=force_update)
File "/usr/lib/python2.5/site-packages/django/db/models/base.py",
line 383, in save_base
result = manager._insert(values, return_id=update_pk)
File "/usr/lib/python2.5/site-packages/django/db/models/manager.py",
line 138, in _insert
return insert_query(self.model, values, **kwargs)
File "/usr/lib/python2.5/site-packages/django/db/models/query.py",
line 894, in insert_query
return query.execute_sql(return_id)
File "/usr/lib/python2.5/site-packages/django/db/models/sql/
subqueries.py", line 309, in execute_sql
cursor = super(InsertQuery, self).execute_sql(None)
File "/usr/lib/python2.5/site-packages/django/db/models/sql/
query.py", line 1734, in execute_sql
cursor.execute(sql, params)
File "/usr/lib/python2.5/site-packages/django/db/backends/util.py",
line 19, in execute
return self.cursor.execute(sql, params)
File "/usr/lib/python2.5/site-packages/django/db/backends/mysql/
base.py", line 83, in execute
return self.cursor.execute(query, args)
File "/var/lib/python-support/python2.5/MySQLdb/cursors.py", line
151, in execute
query = query % db.literal(args)
File "/var/lib/python-support/python2.5/MySQLdb/connections.py",
line 247, in literal
return self.escape(o, self.encoders)
File "/var/lib/python-support/python2.5/MySQLdb/connections.py",
line 180, in string_literal
return db.string_literal(obj)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xc9' in
position 1: ordinal not in range(128)
-------------------------------------------------------------------------------------------------------------------------------
 
L

Lie Ryan

Rama said:
While doing the below

1) fetch html page
2) extract title using BeatifulSoup
3) Save into the database.

iam getting the below error (at the bottom).

Few observations which i had:
1) type of variable which holds this title is <class
'BeautifulSoup.NavigableString'>
2) iam getting this problem when the title has character whose
ordinal value > 128

I searched in google and tried various suggestions but none of them is
working.And also i couldnot able to understand the rootcause for such
a kind of behaviour.

What's the title of the feed. I guess it contains non-ASCII character.
Try encoding the title with a proper encoding that can handle the
title's character set (utf-8 is a good choice), then save it again to
the database.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,484
Members
44,903
Latest member
orderPeak8CBDGummies

Latest Threads

Top