problem with global var

[Bruno Ferreira]

Hi,

I wrote a very simple python program to generate a sorted list of
lines from a squid access log file.

Here is a simplified version:

##################################
1 logfile = open ("squid_access.log", "r")
2 topsquid = [["0", "0", "0", "0", "0", "0", "0"]]
3
4 def add_sorted (list):
5 for i in range(50):
6 if int(list[4]) > int(topsquid[4]):
7 topsquid.insert(i,list)
8 break
8 # Max len = 50
10 if len(topsquid) > 50:
11 topsquid = topsquid[0:50]
12
13 while True:
14 logline = logfile.readline()
15 linefields = logline.split()
16
17 if logline != "":
18 add_sorted (linefields)
19 else:
20 break
21
22 for i in range (len(topsquid)):
23 print topsquid[4]
####################################

When I execute the program _without_ the lines 10 and 11:

10 if len(topsquid) > 50:
11 topsquid = topsquid[0:50]

it runs perfectly.

But if I execute the program _with_ those lines, this exception is thrown:

bruno@ts:~$ python topsquid.py
Traceback (most recent call last):
File "topsquid.py", line 20, in <module>
add_sorted (linefields)
File "topsquid.py", line 6, in add_sorted
if int(list[4]) > int(topsquid[4]):
UnboundLocalError: local variable 'topsquid' referenced before assignment


Note that now the error shown is not related with the lines 10 and 11,
but wiht a line prior to them.

Any hints?

 

[wes]

Hi,

I wrote a very simple python program to generate a sorted list of
lines from a squid access log file.

Here is a simplified version:

##################################
1 logfile = open ("squid_access.log", "r")
2 topsquid = [["0", "0", "0", "0", "0", "0", "0"]]
3
4 def add_sorted (list): global topsquid
5 for i in range(50):
6 if int(list[4]) > int(topsquid[4]):
7 topsquid.insert(i,list)
8 break
8 # Max len = 50
10 if len(topsquid) > 50:
11 topsquid = topsquid[0:50]
12
13 while True:
14 logline = logfile.readline()
15 linefields = logline.split()
16
17 if logline != "":
18 add_sorted (linefields)
19 else:
20 break
21
22 for i in range (len(topsquid)):
23 print topsquid[4]
####################################

When I execute the program _without_ the lines 10 and 11:

10 if len(topsquid) > 50:
11 topsquid = topsquid[0:50]

it runs perfectly.

But if I execute the program _with_ those lines, this exception is thrown:

bruno@ts:~$ python topsquid.py
Traceback (most recent call last):
File "topsquid.py", line 20, in <module>
add_sorted (linefields)
File "topsquid.py", line 6, in add_sorted
if int(list[4]) > int(topsquid[4]):
UnboundLocalError: local variable 'topsquid' referenced before assignment


Note that now the error shown is not related with the lines 10 and 11,
but wiht a line prior to them.

Any hints?

Try line 4 add.

 

[Diez B. Roggisch]

Hi,

I wrote a very simple python program to generate a sorted list of
lines from a squid access log file.

Here is a simplified version:

##################################
1 logfile = open ("squid_access.log", "r")
2 topsquid = [["0", "0", "0", "0", "0", "0", "0"]]
3
4 def add_sorted (list):
5 for i in range(50):
6 if int(list[4]) > int(topsquid[4]):
7 topsquid.insert(i,list)
8 break
8 # Max len = 50
10 if len(topsquid) > 50:
11 topsquid = topsquid[0:50]
12
13 while True:
14 logline = logfile.readline()
15 linefields = logline.split()
16
17 if logline != "":
18 add_sorted (linefields)
19 else:
20 break
21
22 for i in range (len(topsquid)):
23 print topsquid[4]
####################################

When I execute the program _without_ the lines 10 and 11:

10 if len(topsquid) > 50:
11 topsquid = topsquid[0:50]

it runs perfectly.

But if I execute the program _with_ those lines, this exception is thrown:

bruno@ts:~$ python topsquid.py
Traceback (most recent call last):
File "topsquid.py", line 20, in <module>
add_sorted (linefields)
File "topsquid.py", line 6, in add_sorted
if int(list[4]) > int(topsquid[4]):
UnboundLocalError: local variable 'topsquid' referenced before assignment


Note that now the error shown is not related with the lines 10 and 11,
but wiht a line prior to them.

Any hints?

Use

def add_sorted(list):
global topsquid
...

to make topsquid a global variable to add_sorted. Otherwise python sees that
it gets referred by in the if-statement before assigning to it, thus
resulting in the error you see.

The reason for this is that a (limited) static analysis of python-code is
performed to determine which variables are local to a function and which
not. The criteria essentially is the appearance on the left-hand-side of an
expression makes a variable (or name) local to that function. Which makes
it require the explicit global declaration.

Apart from that there are quite a few things worth mentioning in your code:

- don't shadow built-in names like list

- it's superfluous to do

for i in xrange(len(some_list)):
.. some_list ..

as you do, unless you need the index. Instead do

for element in some_list:
... element ...

If you need an index, do

for i, element in enumerate(some_list):
...

- don't use range, use xrange if you don't need a list but rather
want to enumerate indices.

- the while-loop is superfluous as well, just do

for line in logfile:
...

or if your python is older do

for line in logfile.xreadlines():
...

Diez

 

[Steven D'Aprano]

Hi,

I wrote a very simple python program to generate a sorted list of lines
from a squid access log file.

Here is a simplified version:

##################################
1 logfile = open ("squid_access.log", "r")
2 topsquid = [["0", "0", "0", "0", "0", "0", "0"]]

[snip]


Others have already solved the immediate problem, but a much better
design would be to avoid using a global variable in the first place.



def add_sorted(alist, data):
"""Add figures from alist to collated data and return data."""
# do your processing here...
return data



topsquid=[["0", "0", "0", "0", "0", "0", "0"]]
for line in logfile:
linefields = logline.split()
topsquid = add_sorted(linefields, topsquid)

 

[Peter Otten]

I wrote a very simple python program to generate a sorted list of
lines from a squid access log file.

Now that your immediate problem is solved it's time to look at the heapq
module. It solves the problem of finding the N largest items in a list
much more efficiently. I think the following does the same as your code:

import heapq

def key(record):
return int(record[4])

logfile = open("squid_access.log", "r")
records = (line.split() for line in logfile)
topsquid = heapq.nlargest(50, records, key=key)

for record in topsquid:
print record[4]

Peter

 

[Bruno Ferreira]

Hello all,

Amazing

The program is working properly now, the code is much better and I
learned a bit more Python.

Thank you all, guys.

Bruno.

I wrote a very simple python program to generate a sorted list of
lines from a squid access log file.

Now that your immediate problem is solved it's time to look at the heapq
module. It solves the problem of finding the N largest items in a list
much more efficiently. I think the following does the same as your code:

import heapq

def key(record):
return int(record[4])

logfile = open("squid_access.log", "r")
records = (line.split() for line in logfile)
topsquid = heapq.nlargest(50, records, key=key)

for record in topsquid:
print record[4]

Peter