indentation messing up my tuple?

[localpricemaps]

i have the following code which is used to create a tuple of food and
drink. if the page i am trying to scrape has a total of 10 food/drink
items that i end up getting a nice list of 10 food/drink items in my
text file BUT they are all a repeat of the first item so i end up
getting a text file that looks like this:

shrimp, coke
shrimp, coke
shrimp, coke

instead of being

shrimp, coke
hamburger, oj

here is my code:

for row in bs('div', {'style' : 'both'}):
data=[]

for incident in bs('h3', {'class' : 'name'}):
foodlist = []
for oText in incident.fetchText( oRE):
foodlist.append(oText.strip() + "','")
food = ''.join(foodlist)



for incident in bs('span', {'class' : 'drink'}):
drink = incident.findNextSibling('a', {'class': 'nojs'})
drinklist = []
for oText in drink.fetchText( oRE):
drinklist.append(oText.strip() + "','")
drink = ''.join(drinklist)


tuple = (food + drink + "\n")
data.append(tuple)
f = open("test.txt", 'a')
f.write ( ''.join( tuple ) )

 

[Steven D'Aprano]

here is my code:

for row in bs('div', {'style' : 'both'}):

What is bs? Is it a secret?

data=[]

for incident in bs('h3', {'class' : 'name'}):
foodlist = []
for oText in incident.fetchText( oRE):
foodlist.append(oText.strip() + "','")
food = ''.join(foodlist)

Put a "print food" statement at the end of this line to see what you have.

for incident in bs('span', {'class' : 'drink'}):
drink = incident.findNextSibling('a', {'class': 'nojs'}) drinklist =
[]
for oText in drink.fetchText( oRE):
drinklist.append(oText.strip() + "','")
drink = ''.join(drinklist)

Put a "print drink" statement at the end of this line to see what you have.

tuple = (food + drink + "\n")
data.append(tuple)

This seems awfully pointless. At the beginning of every loop, you set data
to the empty list. After a while you append one tuple to it. But you don't
use data again, and it just gets reset to the empty list at the beginning
of the next loop.

What is the purpose of data?

f = open("test.txt", 'a')
f.write ( ''.join( tuple ) )

You are re-opening the file every single time around the loop. You should
either do this:

f = open("test.txt", "w")
for row in bs(...):
# processing
f.write(one line only)
f.close()

or do this:

data = []
for row in bs(...):
# processing
# accumulate everything you want in data
f.writelines(data)
f.close()

 

[bonono]

What is bs? Is it a secret?

seems to be beautiful soup, in this context.

 

[localpricemaps]

sorry i forgot to add in the code for my tuple which is at the very end

tuple = (food+ drink + "\n")
data.append(tuple)
f = open("froogle.sql", 'a')
f.write ( ''.join( tuple )

 

[localpricemaps]

sorry i left out my tuple which is at the end of my code

tuple = (food + drink + "\n")
data.append(tuple)
f = open("froogle.sql", 'a')
f.write ( ''.join( tuple )

 

[infidel]

tuple is the name of the built-in type, so it's not a very good idea to
reassign it to something else.

(food + drink + '\n') is not a tuple, (food + drink + '\n',) is

There's no reason to use tuples here, just do this:

data.append(food + drink)
f.write('\n'.join(data))

 

[localpricemaps]

i am using a tuple because i am building lists. if i just use (food +
drink) then while drink is unique food remains the same do i get this:

(burger, coke)
(burger, 7up)
(burger, sprite)

 

[infidel]

i am using a tuple because i am building lists.

I don't understand

if i just use (food +
drink) then while drink is unique food remains the same do i get this:

(burger, coke)
(burger, 7up)
(burger, sprite)

I don't understand what you're saying here.


food and drink are both strings. adding them together gives you a new
string. putting parentheses around a string does not give you a tuple,
you need that magic comma to get python to recognize the expression as
a tuple.

As I said:


And since all you're doing with the data list is joining into a single
string, I still don't see where you need any tuples.

 

[localpricemaps]

the, the issue is that the last loop adds the last value of everything
to the data array