rewrite for achieving speedup

[Johnny Blonde]

Hello Group!

I really tried hard for two hours to rewrite the following expression
(python 2.4):
--------------------------
teilnehmer = []
for r in Reisen.select(AND(Reisen.q.RESVON <= datum, Reisen.q.RESBIS

= datum)):

for g in r.BUCHUNGEN:
for t in g.aktiveTeilnehmer:
teilnehmer.append(t)
--------------------------

to something like
--------------------------
teilnehmer = [x for x in ........]
--------------------------

Reisen is a SQLObject class, Reisen.select(...), aktiveTeilnehmer and
BUCHUNGEN all are of the type SelectResults.

unfortunately i just can´t figure it out to make it work.
i hope someone maybe can help me?

I hope to gain performance by rewriting it...

Thanks a lot for your help!

Regards, Frank

 

[Steve Holden]

Hello Group!

I really tried hard for two hours to rewrite the following expression
(python 2.4):
--------------------------
teilnehmer = []
for r in Reisen.select(AND(Reisen.q.RESVON <= datum, Reisen.q.RESBIS

= datum)):

for g in r.BUCHUNGEN:
for t in g.aktiveTeilnehmer:
teilnehmer.append(t)
--------------------------

to something like
--------------------------
teilnehmer = [x for x in ........]
--------------------------

Reisen is a SQLObject class, Reisen.select(...), aktiveTeilnehmer and
BUCHUNGEN all are of the type SelectResults.

unfortunately i just can´t figure it out to make it work.
i hope someone maybe can help me?

I hope to gain performance by rewriting it...

Thanks a lot for your help!

>>> lt = [[[1,2,3], [2,3,4]], [[3,4,5], [4,5,6]]]
>>> lf = [c for a in lt for b in a for c in b]
>>> lf [1, 2, 3, 2, 3, 4, 3, 4, 5, 4, 5, 6]
>>>

Untested:

teilnehmer = [t for r in Reisen.select(AND(Reisen.q.RESVON <= datum,
reisen.q.RESBIS >= datum)) for g in r.BUCHUNGEN for t in g.aktiveTeilnehmer]

regards
Steve

 

[Steven Bethard]

Hello Group!

I really tried hard for two hours to rewrite the following expression
(python 2.4):
--------------------------
teilnehmer = []
for r in Reisen.select(AND(Reisen.q.RESVON <= datum, Reisen.q.RESBIS

= datum)):

for g in r.BUCHUNGEN:
for t in g.aktiveTeilnehmer:
teilnehmer.append(t)
--------------------------

to something like
--------------------------
teilnehmer = [x for x in ........]
--------------------------

Reisen is a SQLObject class, Reisen.select(...), aktiveTeilnehmer and
BUCHUNGEN all are of the type SelectResults.

unfortunately i just can´t figure it out to make it work.
i hope someone maybe can help me?

I hope to gain performance by rewriting it...

Thanks a lot for your help!

lt = [[[1,2,3], [2,3,4]], [[3,4,5], [4,5,6]]]
lf = [c for a in lt for b in a for c in b]
lf [1, 2, 3, 2, 3, 4, 3, 4, 5, 4, 5, 6]

Untested:

teilnehmer = [t for r in Reisen.select(AND(Reisen.q.RESVON <= datum,
reisen.q.RESBIS >= datum)) for g in r.BUCHUNGEN for t in
g.aktiveTeilnehmer]

Note also that you can probably get most of the speedup above by binding
the append method to a function-local name::

teilnehmer = []
append = teilnehmer.append
for r in Reisen.select(...):
for g in r.BUCHUNGEN:
for t in g.aktiveTeilnehmer:
append(t)

That's pretty much all a list comprehension is doing anyway.

STeVe

 

[Johnny Blonde]

thanks steve h.,

works like this just perfectly!

steve b.:
for the next time if i cannot figure it out i will just do it like
this!

thanks a lot guys,
Frank

 

[Jun.Jin.act+group.python]

Hello Group!
I really tried hard for two hours to rewrite the following expression
(python 2.4):
--------------------------
teilnehmer = []
for r in Reisen.select(AND(Reisen.q.RESVON <= datum, Reisen.q.RESBIS
= datum)):
for g in r.BUCHUNGEN:
for t in g.aktiveTeilnehmer:
teilnehmer.append(t)
--------------------------
to something like
--------------------------
teilnehmer = [x for x in ........]
--------------------------
Reisen is a SQLObject class, Reisen.select(...), aktiveTeilnehmer and
BUCHUNGEN all are of the type SelectResults.
unfortunately i just can´t figure it out to make it work.
i hope someone maybe can help me?
I hope to gain performance by rewriting it...
Thanks a lot for your help!
lt = [[[1,2,3], [2,3,4]], [[3,4,5], [4,5,6]]]
lf = [c for a in lt for b in a for c in b]
lf

[1, 2, 3, 2, 3, 4, 3, 4, 5, 4, 5, 6]

Untested:

teilnehmer = [t for r in Reisen.select(AND(Reisen.q.RESVON <= datum,
reisen.q.RESBIS >= datum)) for g in r.BUCHUNGEN for t in
g.aktiveTeilnehmer]

Note also that you can probably get most of the speedup above by binding
the append method to a function-local name::

teilnehmer = []
append = teilnehmer.append
for r in Reisen.select(...):
for g in r.BUCHUNGEN:
for t in g.aktiveTeilnehmer:
append(t)

That's pretty much all a list comprehension is doing anyway.

STeVe

hi steve,
why would binding to a function-local name speeds up performance?
sorry for asking such a simple question. many thanks.

 

[Alex Martelli]

why would binding to a function-local name speeds up performance?

Like any other constant-hoisting, pulling the lookup out of the loop
speeds things up because otherwise Python must repeat the lookup each
time through the loop (Python doesn't _know_ that, for example, zip.zap
is always the same bound method object -- without potentially deep
analysis, it can't rule out that looking up zap on zip has side effects
and the like, so when you tell it to look it up, it looks it up, no ifs,
no buts).

You can measure the effect with -mtimeit ...:

brain:~ alex$ python -mtimeit -s'def f(L):

for x in xrange(1000): L.append' 'f([])'

1000 loops, best of 3: 215 usec per loop

brain:~ alex$ python -mtimeit -s'def g(L):

ap = L.append
for x in xrange(1000): ap' 'g([])'

10000 loops, best of 3: 79.1 usec per loop

note that in each function f and g I'm doing only the lookup, not the
call to the method thus looked up.

Of course, this depends on the fact that for a function to use a local
variable takes very little time indeed (since the compiler identifies
which variables are local, and transforms any use of their names into a
rapid indexing of a "local variables array", at the time the def
statement executes -- so by the time the function gets called, i.e.,
that its body executes, that execution takes advantage of this
optimization automatically performed by the Python compiler).


Alex