"filtered view" upon lists?

[Wildemar Wildenburger]

Hi there

I don't know how else to call what I'm currently implementing: An object
that behaves like a list but doesn't store it's own items but rather
pulls them from a larger list (if they match a certain criterion).
Changes to the filter are instantly reflected in the underlying list.
Clear enough?

Ok, so I figured that this is generic enough to be found in some
standard module already (I've had this often enough: Painfully
implementing s/th and then finding it in the libs some weeks later.).

Any pointers?

c.u.
wildemar

 

[Jorge Godoy]

I don't know how else to call what I'm currently implementing: An object that
behaves like a list but doesn't store it's own items but rather pulls them
from a larger list (if they match a certain criterion).
Changes to the filter are instantly reflected in the underlying list.
Clear enough?

It looks like you're implementing a callable to me. This is a method that
returns results based on some input -- here your original list and filter.

Then you'll use this method wherever you need that filtered list.

Ok, so I figured that this is generic enough to be found in some standard
module already (I've had this often enough: Painfully implementing s/th and
then finding it in the libs some weeks later.).

I don't believe it is generic. Nobody knows your data specs or filtering
needs.

Any pointers?

Use of list comprehension might make it easier to code this:


def myCallable(my_list, filter):
filtered_list = [(item) for item in my_list if filter(item)]
return filtered_list


Example of full code:
.... filtered_list = [(item) for item in list if filter(item)]
.... return filtered_list
....

myCallable(test_list, filter) [0, 2, 4, 6, 8]
for item in myCallable(test_list, filter):

.... print "See? I'm", item
....
See? I'm 0
See? I'm 2
See? I'm 4
See? I'm 6
See? I'm 8

 

[Paul McGuire]

I don't know how else to call what I'm currently implementing: An object
that
behaves like a list but doesn't store it's own items but rather pulls
them
from a larger list (if they match a certain criterion).
Changes to the filter are instantly reflected in the underlying list.
Clear enough?

It looks like you're implementing a callable to me. This is a method that
returns results based on some input -- here your original list and filter.

Then you'll use this method wherever you need that filtered list.

Ok, so I figured that this is generic enough to be found in some standard
module already (I've had this often enough: Painfully implementing s/th
and
then finding it in the libs some weeks later.).

I don't believe it is generic. Nobody knows your data specs or filtering
needs.

Any pointers?

Use of list comprehension might make it easier to code this:


def myCallable(my_list, filter):
filtered_list = [(item) for item in my_list if filter(item)]
return filtered_list


Example of full code:
... filtered_list = [(item) for item in list if filter(item)]
... return filtered_list
...

myCallable(test_list, filter) [0, 2, 4, 6, 8]
for item in myCallable(test_list, filter):

... print "See? I'm", item
...
See? I'm 0
See? I'm 2
See? I'm 4
See? I'm 6
See? I'm 8

Functionally-speaking, there is the filter built-in that does this same
thing:

data = range(10)
odds = filter(lambda x : x % 2, data)
evens = filter(lambda x : x % 2 != 1, data)
odds [1, 3, 5, 7, 9]
evens

[0, 2, 4, 6, 8]


-- Paul

 

[Wildemar Wildenburger]

>
> It looks like you're implementing a callable to me. This is a method that
> returns results based on some input -- here your original list and filter.
> Then you'll use this method wherever you need that filtered list.
>

Ok, so I'm not clear enough .
I don't just want to extract certain elements from a list, I want an
object that looks like a list, however all changes made to that object
are automagically reflected in the original list. (I guess that is one
of those 'if it's hard to explain, ...' cases.)

I should have included an example right away ... here goes:

# I have a list
l = [1, 2, 3, 4, 5, 6, 7]

# I then want to create a Filter instance
# (Filter beeing a *class* implemented by me)
# where isEven() returns True whenever an item of l
# should be included in f (in this case, even numbers).
# (I'm asking if something like this exists in the libs
# or elsewhere)
f = Filter(l, isEven)

# The desired behavior now goes something like this:
f

>>> [2, 4, 6]

del f[1]
f

>>> [2, 6] l
>>> [1, 2, 3, 5, 6, 7]

f.append(77)
f

>>> [2, 6, 77]

# 77 being intentionally uneven
l

>>> [1, 2, 3, 5, 6, 7, 77]

# could be [1, 2, 3, 5, 6, 77, 7] as well
# I don't care here

# and so forth ...

I think SQL views are the direct analog.

> I don't believe it is generic. Nobody knows your data specs or filtering
> needs.

Hence the additional function in the Filter constructor . You suggest
the same thing below, so that is obviously no problem.

> Use of list comprehension might make it easier to code this:
>
> <snip elaborate example>

I sort of wanted to avoid these. Though my lists shouldn't terribly
long, so performance is not an issue so much. I simply want to avoid
having two datasets that I have to sync. Major pain, I believe.


Coding all that really is quite straight forward, but it is a lot of
mule-work. I hoped (no, I still hope) that there might be somethin like
that around already.

A bit clearer now?

bye
wildemar

 

[Steve Holden]

It looks like you're implementing a callable to me. This is a method that
returns results based on some input -- here your original list and filter.
Then you'll use this method wherever you need that filtered list.

Ok, so I'm not clear enough .
I don't just want to extract certain elements from a list, I want an
object that looks like a list, however all changes made to that object
are automagically reflected in the original list. (I guess that is one
of those 'if it's hard to explain, ...' cases.)

I should have included an example right away ... here goes:

# I have a list
l = [1, 2, 3, 4, 5, 6, 7]

# I then want to create a Filter instance
# (Filter beeing a *class* implemented by me)
# where isEven() returns True whenever an item of l
# should be included in f (in this case, even numbers).
# (I'm asking if something like this exists in the libs
# or elsewhere)
f = Filter(l, isEven)

# The desired behavior now goes something like this:
f

[2, 4, 6]

del f[1]
f

[2, 6] l
[1, 2, 3, 5, 6, 7]

f.append(77)
f

[2, 6, 77]

# 77 being intentionally uneven
l

[1, 2, 3, 5, 6, 7, 77]

# could be [1, 2, 3, 5, 6, 77, 7] as well
# I don't care here

# and so forth ...

I think SQL views are the direct analog.

I don't think they are. If you add a non-conforming row to a SQL view it
doesn't appear int he view. If you modify a row in an updateable view so
it no longer confirms with the view's conditions it disappears from the
view.

This is a classic cause of user perplexity - they update a record
without being aware that they are using a view and suddenly it "disappears".

Also you don't say whether changes to l are supposed to be reflected in
f in the same way that changes to f are reflected in l.

It might be better if you were to explain your use case: what is this
beast supposed to be for?

Hence the additional function in the Filter constructor . You suggest
the same thing below, so that is obviously no problem.

I sort of wanted to avoid these. Though my lists shouldn't terribly
long, so performance is not an issue so much. I simply want to avoid
having two datasets that I have to sync. Major pain, I believe.


Coding all that really is quite straight forward, but it is a lot of
mule-work. I hoped (no, I still hope) that there might be somethin like
that around already.

A bit clearer now?

Hardly at all. Frankly it doesn't seem as though you really know what
the specifications are yourself, so you can hardly expect us to divine
them by use of our psychic powers. Besides which, psychic powers cost
extra ;-)

So give us that use case so we get a bit more insight into why you even
see the need for such a thing and how you want it ti behave. Or is all
this just theoretical?

regards
Steve

 

[bearophileHUGS]

Wildemar Wildenburger:

I sort of wanted to avoid these. Though my lists shouldn't terribly
long, so performance is not an issue so much. I simply want to avoid
having two datasets that I have to sync. Major pain, I believe.

Note that if you just have to scan the "view list", then you can use
the lazy:

itertool.ifilter(predicate, iterable)

Bye,
bearophile

 

[Wildemar Wildenburger]

> I don't think they are. If you add a non-conforming row to a SQL view

it doesn't appear int he view. If you modify a row in an updateable view
so it no longer confirms with the view's conditions it disappears from
the view.

Me and my wording again: I typed 'direct analog' but meant 'somewhat
analog'.

> Also you don't say whether changes to l are supposed to be reflected

in f in the same way that changes to f are reflected in l.

Right. Yes, the synchronicity (a word?) is meant to be in both directions.

> It might be better if you were to explain your use case: what is this

beast supposed to be for?

I have a database of interconnected objects. Each object has a list
tuples; these tuples hold references to other objects and details about
the type of connection.
The 'views' I am imagining are sub-lists of this one list, based on the
connection types. Order is important here, that's why I'm not using sets.
BTW: I have gone the way of nested lists before and found it too rigid
and complicated to use.

>Hardly at all. Frankly it doesn't seem as though you really know what
> the specifications are yourself, so you can hardly expect us to

divine > them by use of our psychic powers. Besides which, psychic
powers cost > extra ;-)

Right again. I'm not sure what I want yet as I'm still exploring ideas.
If something /like/ the above had existed (which by now I'm gathering is
not the case) I'd just adapted my ideas to the specs given.

> So give us that use case so we get a bit more insight into why you

even see the need for such a thing and how you want it ti behave. Or is
all this just theoretical?

No no, its very much real. Just not as fleshed out as it might be ---
I'm sort of an explorative coder; I usually don't know how something is
going to work until it works.

Thx for the effort
wildemar

 

[Wildemar Wildenburger]

Note that if you just have to scan the "view list", then you can use
the lazy:

itertool.ifilter(predicate, iterable)

Yeah, had found that already; thanks anyway.
This function is actually the very reason I'm even asking this: 'If the
std libs solve this problem for, maybe they solve the next one as well'
I thought.
Well, can't always win, can you?


wildemar

 

[Steve Holden]

Me and my wording again: I typed 'direct analog' but meant 'somewhat
analog'.


Right. Yes, the synchronicity (a word?) is meant to be in both directions.


I have a database of interconnected objects. Each object has a list
tuples; these tuples hold references to other objects and details about
the type of connection.
The 'views' I am imagining are sub-lists of this one list, based on the
connection types. Order is important here, that's why I'm not using sets.
BTW: I have gone the way of nested lists before and found it too rigid
and complicated to use.



Right again. I'm not sure what I want yet as I'm still exploring ideas.
If something /like/ the above had existed (which by now I'm gathering is
not the case) I'd just adapted my ideas to the specs given.



No no, its very much real. Just not as fleshed out as it might be ---
I'm sort of an explorative coder; I usually don't know how something is
going to work until it works.

That's all very well, but you might want to Google for "YAGNI". You
still haven't given us mcuh of a clue about the real need.

Thx for the effort Happy to (tray to) help.

regards
Steve

 

[Wildemar Wildenburger]

That's all very well, but you might want to Google for "YAGNI".

You still haven't given us mcuh of a clue about the real need.

There is no "real need". I'm trying things, looking what works out
eventually. I don't do that for a living (if I did, I'd be starved by now).
Or do you mean the real use case? Because that I did do. There really
isn't anything more to it.

Ok, maybe there is (though I still think it boils down to what I have
said before):
I have objects, called Ideas. Ideas have IDs, used as primary keys in a
berkeley database. I want to connect them with connections that have a
'type', may or may not have a 'direction' and may or may not have a
'context'. I want to avoid storing the connections as separate database
objects (lets for now just accept that).

So I decided to store connections as lists of tuples inside Ideas:
[(ID, type, direction, context), (ID, type, direction, context), ...]
Those I want to Filter.

I'll gladly take any advice on how to do it better; mind you however
that the only data structure the db is supposed to deal with is the 'Idea'.

Again: I find this hard to explain, so I should probably rethink the
whole thing.

wildemar

 

[Raymond Hettinger]

[Wildemar Wildenburger]

I don't know how else to call what I'm currently implementing: An object
that behaves like a list but doesn't store it's own items but rather
pulls them from a larger list (if they match a certain criterion).
Changes to the filter are instantly reflected in the underlying list.
Clear enough?

Here's some code to try out:

import UserList

class ListView(UserList.UserList):
@property
def data(self):
return filter(self._filter_function, self._actual_list)
def __init__(self, actual_list, filter_function=None):
self._actual_list = actual_list
self._filter_function = filter_function

### Example calls
a = range(10)
b = ListView(a, lambda x: x%2==0)
print list(b), len(b), b[2], b[:3]
a[:] = range(10,20)
print list(b), len(b), b[2], b[:3]



Raymond Hettinger