loop over list and process into groups

S

Sneaky Wombat

[ {'vlan_or_intf': 'VLAN2021'},
{'vlan_or_intf': 'Interface'},
{'vlan_or_intf': 'Po1'},
{'vlan_or_intf': 'Po306'},
{'vlan_or_intf': 'VLAN2022'},
{'vlan_or_intf': 'Interface'},
{'vlan_or_intf': 'Gi7/33'},
{'vlan_or_intf': 'Po1'},
{'vlan_or_intf': 'Po306'},
{'vlan_or_intf': 'VLAN2051'},
{'vlan_or_intf': 'Interface'},
{'vlan_or_intf': 'Gi9/6'},
{'vlan_or_intf': 'VLAN2052'},
{'vlan_or_intf': 'Interface'},
{'vlan_or_intf': 'Gi9/6'},]

I want it to be converted to:

[{'2021':['Po1','Po306']},{'2022':['Gi7/33','Po1','Po306']},etc etc]

I was going to write a def to loop through and look for certain pre-
compiled regexs, and then put them in a new dictionary and append to a
list, but I'm having trouble thinking of a good way to capture each
dictionary. Each dictionary will have a key that is the vlan and the
value will be a list of interfaces that participate in that vlan.
Each list will be variable, many containing only one interface and
some containing many interfaces.

I thought about using itertools, but i only use that for fixed data.
I don't know of a good way to loop over variably sized data. I was
wondering if anyone had any ideas about a good way to convert this
list or dictionary into the right format that I need. The solution I
come up with will most likely be ugly and error prone, so I thought
i'd ask this python list while I work. Hopefully I learn a better way
to solve this problem.

Thanks!

I also have the data in a list,

[ 'VLAN4065',
'Interface',
'Gi9/6',
'Po2',
'Po3',
'Po306',
'VLAN4068',
'Interface',
'Gi9/6',
'VLAN4069',
'Interface',
'Gi9/6',]
 
M

mk

Sneaky said:
I was going to write a def to loop through and look for certain pre-
compiled regexs, and then put them in a new dictionary and append to a
list,
Click to expand...

regexes are overkill in this case I think.
[ 'VLAN4065',
'Interface',
'Gi9/6',
'Po2',
'Po3',
'Po306',
'VLAN4068',
'Interface',
'Gi9/6',
'VLAN4069',
'Interface',
'Gi9/6',]
Click to expand...

Why not construct an intermediate dictionary?

elems = [ 'VLAN4065',
'Interface',
'Gi9/6',
'Po2',
'Po3',
'Po306',
'VLAN4068',
'Interface',
'Gi9/6',
'VLAN4069',
'Interface',
'Gi9/6',]

def makeintermdict(elems):
vd = {}
vlan = None
for el in elems:
if el.startswith('VLAN'):
vlan = el.replace('VLAN','')
elif el == 'Interface':
vd[vlan] = []
else:
vd[vlan].append(el)
return vd

def makelist(interm):
finlist = []
for k in interm.keys():
finlist.append({k:interm[k]})
return finlist

if __name__ == "__main__":
intermediate = makeintermdict(elems)
print intermediate
finlist = makelist(intermediate)
print 'final', finlist


{'4068': ['Gi9/6'], '4069': ['Gi9/6'], '4065': ['Gi9/6', 'Po2', 'Po3',
'Po306']}
final [{'4068': ['Gi9/6']}, {'4069': ['Gi9/6']}, {'4065': ['Gi9/6',
'Po2', 'Po3', 'Po306']}]

I hope this is not your homework. :)

Regards,
mk
 
S

Sneaky Wombat

Sneaky said:
I was going to write a def to loop through and look for certain pre-
compiled regexs, and then put them in a new dictionary and append to a
list,
Click to expand...

regexes are overkill in this case I think.
[ 'VLAN4065',
 'Interface',
 'Gi9/6',
 'Po2',
 'Po3',
 'Po306',
 'VLAN4068',
 'Interface',
 'Gi9/6',
 'VLAN4069',
 'Interface',
 'Gi9/6',]
Click to expand...

Why not construct an intermediate dictionary?

elems = [ 'VLAN4065',
  'Interface',
  'Gi9/6',
  'Po2',
  'Po3',
  'Po306',
  'VLAN4068',
  'Interface',
  'Gi9/6',
  'VLAN4069',
  'Interface',
  'Gi9/6',]

def makeintermdict(elems):
     vd = {}
     vlan = None
     for el in elems:
         if el.startswith('VLAN'):
             vlan = el.replace('VLAN','')
         elif el == 'Interface':
             vd[vlan] = []
         else:
             vd[vlan].append(el)
     return vd

def makelist(interm):
     finlist = []
     for k in interm.keys():
         finlist.append({k:interm[k]})
     return finlist

if __name__ == "__main__":
     intermediate = makeintermdict(elems)
     print intermediate
     finlist = makelist(intermediate)
     print 'final', finlist

{'4068': ['Gi9/6'], '4069': ['Gi9/6'], '4065': ['Gi9/6', 'Po2', 'Po3',
'Po306']}
final [{'4068': ['Gi9/6']}, {'4069': ['Gi9/6']}, {'4065': ['Gi9/6',
'Po2', 'Po3', 'Po306']}]

I hope this is not your homework. :)

Regards,
mk
Click to expand...

Thanks mk,

My approach was a lot more complex than yours, but your's is better.
I like itertools and was using islice to create tuples for (start,end)
string slicing. Too much work though. Thanks!

-j
 
L

lbolla

[ {'vlan_or_intf': 'VLAN2021'},
 {'vlan_or_intf': 'Interface'},
 {'vlan_or_intf': 'Po1'},
 {'vlan_or_intf': 'Po306'},
 {'vlan_or_intf': 'VLAN2022'},
 {'vlan_or_intf': 'Interface'},
 {'vlan_or_intf': 'Gi7/33'},
 {'vlan_or_intf': 'Po1'},
 {'vlan_or_intf': 'Po306'},
 {'vlan_or_intf': 'VLAN2051'},
 {'vlan_or_intf': 'Interface'},
 {'vlan_or_intf': 'Gi9/6'},
 {'vlan_or_intf': 'VLAN2052'},
 {'vlan_or_intf': 'Interface'},
 {'vlan_or_intf': 'Gi9/6'},]

I want it to be converted to:

[{'2021':['Po1','Po306']},{'2022':['Gi7/33','Po1','Po306']},etc etc]

I was going to write a def to loop through and look for certain pre-
compiled regexs, and then put them in a new dictionary and append to a
list, but I'm having trouble thinking of a good way to capture each
dictionary.  Each dictionary will have a key that is the vlan and the
value will be a list of interfaces that participate in that vlan.
Each list will be variable, many containing only one interface and
some containing many interfaces.

I thought about using itertools, but i only use that for fixed data.
I don't know of a good way to loop over variably sized data.  I was
wondering if anyone had any ideas about a good way to convert this
list or dictionary into the right format that I need.  The solution I
come up with will most likely be ugly and error prone, so I thought
i'd ask this python list while I work.  Hopefully I learn a better way
to solve this problem.

Thanks!

I also have the data in a list,

[ 'VLAN4065',
 'Interface',
 'Gi9/6',
 'Po2',
 'Po3',
 'Po306',
 'VLAN4068',
 'Interface',
 'Gi9/6',
 'VLAN4069',
 'Interface',
 'Gi9/6',]
Click to expand...



===================================

from itertools import groupby

data = \
[ {'vlan_or_intf': 'VLAN2021'},
{'vlan_or_intf': 'Interface'},
{'vlan_or_intf': 'Po1'},
{'vlan_or_intf': 'Po306'},
{'vlan_or_intf': 'VLAN2022'},
{'vlan_or_intf': 'Interface'},
{'vlan_or_intf': 'Gi7/33'},
{'vlan_or_intf': 'Po1'},
{'vlan_or_intf': 'Po306'},
{'vlan_or_intf': 'VLAN2051'},
{'vlan_or_intf': 'Interface'},
{'vlan_or_intf': 'Gi9/6'},
{'vlan_or_intf': 'VLAN2052'},
{'vlan_or_intf': 'Interface'},
{'vlan_or_intf': 'Gi9/6'},]

def clean_up(lst):
return [d.values()[0] for d in data if d.values()[0] != 'Interface']

out = {}
for k, g in groupby(clean_up(data) , key=lambda s:
s.startswith('VLAN')):
if k:
key = list(g)[0].replace('VLAN','')
else:
out[key] = list(g)

print out
===================================

hth,
L.
 
N

nn

lbolla said:
[ {'vlan_or_intf': 'VLAN2021'},
 {'vlan_or_intf': 'Interface'},
 {'vlan_or_intf': 'Po1'},
 {'vlan_or_intf': 'Po306'},
 {'vlan_or_intf': 'VLAN2022'},
 {'vlan_or_intf': 'Interface'},
 {'vlan_or_intf': 'Gi7/33'},
 {'vlan_or_intf': 'Po1'},
 {'vlan_or_intf': 'Po306'},
 {'vlan_or_intf': 'VLAN2051'},
 {'vlan_or_intf': 'Interface'},
 {'vlan_or_intf': 'Gi9/6'},
 {'vlan_or_intf': 'VLAN2052'},
 {'vlan_or_intf': 'Interface'},
 {'vlan_or_intf': 'Gi9/6'},]

I want it to be converted to:

[{'2021':['Po1','Po306']},{'2022':['Gi7/33','Po1','Po306']},etc etc]

I was going to write a def to loop through and look for certain pre-
compiled regexs, and then put them in a new dictionary and append to a
list, but I'm having trouble thinking of a good way to capture each
dictionary.  Each dictionary will have a key that is the vlan and the
value will be a list of interfaces that participate in that vlan.
Each list will be variable, many containing only one interface and
some containing many interfaces.

I thought about using itertools, but i only use that for fixed data.
I don't know of a good way to loop over variably sized data.  I was
wondering if anyone had any ideas about a good way to convert this
list or dictionary into the right format that I need.  The solution I
come up with will most likely be ugly and error prone, so I thought
i'd ask this python list while I work.  Hopefully I learn a better way
to solve this problem.

Thanks!

I also have the data in a list,

[ 'VLAN4065',
 'Interface',
 'Gi9/6',
 'Po2',
 'Po3',
 'Po306',
 'VLAN4068',
 'Interface',
 'Gi9/6',
 'VLAN4069',
 'Interface',
 'Gi9/6',]
Click to expand...



===================================

from itertools import groupby

data = \
[ {'vlan_or_intf': 'VLAN2021'},
{'vlan_or_intf': 'Interface'},
{'vlan_or_intf': 'Po1'},
{'vlan_or_intf': 'Po306'},
{'vlan_or_intf': 'VLAN2022'},
{'vlan_or_intf': 'Interface'},
{'vlan_or_intf': 'Gi7/33'},
{'vlan_or_intf': 'Po1'},
{'vlan_or_intf': 'Po306'},
{'vlan_or_intf': 'VLAN2051'},
{'vlan_or_intf': 'Interface'},
{'vlan_or_intf': 'Gi9/6'},
{'vlan_or_intf': 'VLAN2052'},
{'vlan_or_intf': 'Interface'},
{'vlan_or_intf': 'Gi9/6'},]

def clean_up(lst):
return [d.values()[0] for d in data if d.values()[0] != 'Interface']

out = {}
for k, g in groupby(clean_up(data) , key=lambda s:
s.startswith('VLAN')):
if k:
key = list(g)[0].replace('VLAN','')
else:
out[key] = list(g)

print out
===================================

hth,
L.
Click to expand...

Good use of groupby. Here is what I ended up coming up:

from itertools import groupby
laninfo=[ 'VLAN4065',
'Interface',
'Gi9/6',
'Po2',
'Po3',
'Po306',
'VLAN4068',
'Interface',
'Gi9/6',
'VLAN4069',
'Interface',
'Gi9/6',]

def splitgrp(s, f=[False]):
f[0]^=s.startswith('VLAN')
return f[0]
lanlst=(list(g) for k,g in groupby(laninfo,key=splitgrp))
out={item[0][4:]:item[2:] for item in lanlst}
print(out)
 
D

Dennis Lee Bieber

[ {'vlan_or_intf': 'VLAN2021'},
{'vlan_or_intf': 'Interface'},
{'vlan_or_intf': 'Po1'},
{'vlan_or_intf': 'Po306'},
{'vlan_or_intf': 'VLAN2022'},
{'vlan_or_intf': 'Interface'},
{'vlan_or_intf': 'Gi7/33'},
{'vlan_or_intf': 'Po1'},
{'vlan_or_intf': 'Po306'},
{'vlan_or_intf': 'VLAN2051'},
{'vlan_or_intf': 'Interface'},
{'vlan_or_intf': 'Gi9/6'},
{'vlan_or_intf': 'VLAN2052'},
{'vlan_or_intf': 'Interface'},
{'vlan_or_intf': 'Gi9/6'},]
Click to expand...
That is an interesting data set...

It is a list of ONE ELEMENT dictionaries...
I want it to be converted to:

[{'2021':['Po1','Po306']},{'2022':['Gi7/33','Po1','Po306']},etc etc]
Click to expand...

And so is your specified output... How do you intend to use this
data structure? Compared to a dictionary with multiple entries:

{ "2021" : ["P01", "Po306"],
"2022" : ["Gi7/33", "Po1", "Po306"],
.... }

which can be used via keys: converted["2022"] would retrieve the list of
whatever's...
I also have the data in a list,

[ 'VLAN4065',
'Interface',
'Gi9/6',
'Po2',
'Po3',
'Po306',
'VLAN4068',
'Interface',
'Gi9/6',
'VLAN4069',
'Interface',
'Gi9/6',]
Click to expand...

Offhand, the "Interface" entries are superfluous. You appear to be
looking for items starting with VLAN, stripping off the VLAN and using
the remaining numbers as the "name", and then collecting all entries up
to the next "VLAN...", ignoring the "Interface" item.

-=-=-=-=-=-=-=-=-=-=-=-=-
INDATA = [ "VLAN4065",
"Interface",
"Gi9/6",
"Po2",
"Po3",
"Po306",
"VLAN4068",
"Interface",
"Gi9/6",
"VLAN4069",
"Interface",
"Gi9/6" ]

outDict = {}

key = None
for item in INDATA:
if item.startswith("VLAN"):
if key:
outDict[key] = items
key = item[4:]
items = []
elif item == "Interface":
pass
else:
if key: #if no key, can't use data
items.append(item)
if key: #handle last set
outDict[key] = items

from pprint import pprint
pprint(outDict)
-=-=-=-=-=-=-=-=-=-=-=-=-
{'4065': ['Gi9/6', 'Po2', 'Po3', 'Po306'],
'4068': ['Gi9/6'],
'4069': ['Gi9/6']}
-=-=-=-=-=-=-=-=-=-=-=-=-
 
P

Paul Rubin

lbolla said:
for k, g in groupby(clean_up(data) , key=lambda s: s.startswith('VLAN')):
if k:
key = list(g)[0].replace('VLAN','')
Click to expand...

This is the nicest solution, I think. Mine was more cumbersome.
 
M

mk

Sneaky said:
[ 'VLAN4065',
'Interface',
'Gi9/6',
'Po2',
'Po3',
'Po306',
'VLAN4068',
'Interface',
'Gi9/6',
'VLAN4069',
'Interface',
'Gi9/6',]
Click to expand...

Hey, I just invented a cute ;-) two-liner using list comprehensions:

# alist = list above

tmp, dk = [], {}
[(x.startswith('VLAN') and (dk.setdefault(x,[]) or tmp.append(x))) or
(not x.startswith('VLAN') and dk[tmp[-1]].append(x)) for x in alist
if x != 'Interface']

No need to use a nuke like itertools to kill a fly. ;-)

Regards,
mk
 
N

nn

mk said:
Sneaky said:
[ 'VLAN4065',
'Interface',
'Gi9/6',
'Po2',
'Po3',
'Po306',
'VLAN4068',
'Interface',
'Gi9/6',
'VLAN4069',
'Interface',
'Gi9/6',]
Click to expand...

Hey, I just invented a cute ;-) two-liner using list comprehensions:

# alist = list above

tmp, dk = [], {}
[(x.startswith('VLAN') and (dk.setdefault(x,[]) or tmp.append(x))) or
(not x.startswith('VLAN') and dk[tmp[-1]].append(x)) for x in alist
if x != 'Interface']

No need to use a nuke like itertools to kill a fly. ;-)

Regards,
mk
Click to expand...

Oh my! You could have at least used some "if else" to make it a little
bit easier on the eyes :)

[(dk.setdefault(x,[]) or tmp.append(x))
if x.startswith('VLAN')
else dk[tmp[-1]].append(x)
for x in alist
if x != 'Interface']
 
L

lbolla

Sneaky said:
[ 'VLAN4065',
 'Interface',
 'Gi9/6',
 'Po2',
 'Po3',
 'Po306',
 'VLAN4068',
 'Interface',
 'Gi9/6',
 'VLAN4069',
 'Interface',
 'Gi9/6',]
Click to expand...

Hey, I just invented a cute ;-) two-liner using list comprehensions:

# alist = list above

tmp, dk = [], {}
[(x.startswith('VLAN') and (dk.setdefault(x,[]) or tmp.append(x))) or
(not x.startswith('VLAN') and dk[tmp[-1]].append(x))    for x in alist
if x != 'Interface']

No need to use a nuke like itertools to kill a fly. ;-)

Regards,
mk
Click to expand...

It looks like Perl ;-)
 
M

mk

nn said:
Oh my! You could have at least used some "if else" to make it a little
bit easier on the eyes :)
Click to expand...

That's my entry into """'Obfuscated' "Python" '"''code''"'
'"contest"'""" and I'm proud of it. ;-)

Regards,
mk
 
M

mk

lbolla said:
It looks like Perl ;-)
Click to expand...

A positive proof that you can write perl code in Python. I, for
instance, have had my brain warped by C and tend to write C-like code in
Python. That's only half a joke, sadly. I'm trying to change my habits
but it's hard.

Regards,
mk
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Similar Threads

While Loop Freezing? 1
Infinite loop problem 1
Break up list into groups 19
Reading File Into 2D List 2
Loop over regexp groups 8
Looping through a list of dictionaries 2
regular expression extracting groups 3
Minimum Total Difficulty 0

Members online

No members online now.
Total: 25 (members: 0, guests: 25)
Robots: 374

Forum statistics

Threads
473,755
Messages
2,569,537
Members
45,020
Latest member
GenesisGai

Latest Threads

Top