J
Jeffrey Barish
I have a python module that contains an assignment statement binding a long
list of things to a name:
list_of_things = [thing1, thing2, ...]
where each thing instantiates class Thing when executed. I send this
statement through a socket to a remote module that executes it. The
problem is that it takes too long to send the entire statement.
Accordingly, I am thinking of a solution that sends the list in chunks.
The client then executes each chunk and reassembles the complete list.
Thus, I would first send something like:
chunk = [thing1, thing2, ... thing10]
and then
chunk = [thing11, thing12, ... thing20]
until all things have been transmitted. At the client end, I would execute
each chunk statement and then do
list_of_things.append(chunk)
The point of this solution is that I can start doing useful work in the
client as soon as I receive the first chunk, and the others can arrive in
the background and be available by the time I need them.
One way I could implement this solution is to execute the statement for the
entire list_of_things in the server and then create chunk = [...]
statements with the lists filled using the repr of the class. I believe
that this solution will work, but it seems a shame to execute the
list_of_things statement in the server rather than have the server stupidly
handle strings (because executing the statement takes time and because the
server currently doesn't understand "Thing"). Should I investigate using a
parser to carve up the list_of_things = [...] statement? Basically, I just
need to identify each Thing(...) expression and then count out some number
of them. I should be able to do that using re. Or perhaps I should write
my own parser using Python string operations as all I need to do is count
out some number of Things delimited by "Thing(" at one end and "),\nThing("
at the other (or ")]\n" at the end of the list). Did I just answer my own
question?
Of course, I need to balance the complexity of any alternative solution
against simply executing the statement on the server.
list of things to a name:
list_of_things = [thing1, thing2, ...]
where each thing instantiates class Thing when executed. I send this
statement through a socket to a remote module that executes it. The
problem is that it takes too long to send the entire statement.
Accordingly, I am thinking of a solution that sends the list in chunks.
The client then executes each chunk and reassembles the complete list.
Thus, I would first send something like:
chunk = [thing1, thing2, ... thing10]
and then
chunk = [thing11, thing12, ... thing20]
until all things have been transmitted. At the client end, I would execute
each chunk statement and then do
list_of_things.append(chunk)
The point of this solution is that I can start doing useful work in the
client as soon as I receive the first chunk, and the others can arrive in
the background and be available by the time I need them.
One way I could implement this solution is to execute the statement for the
entire list_of_things in the server and then create chunk = [...]
statements with the lists filled using the repr of the class. I believe
that this solution will work, but it seems a shame to execute the
list_of_things statement in the server rather than have the server stupidly
handle strings (because executing the statement takes time and because the
server currently doesn't understand "Thing"). Should I investigate using a
parser to carve up the list_of_things = [...] statement? Basically, I just
need to identify each Thing(...) expression and then count out some number
of them. I should be able to do that using re. Or perhaps I should write
my own parser using Python string operations as all I need to do is count
out some number of Things delimited by "Thing(" at one end and "),\nThing("
at the other (or ")]\n" at the end of the list). Did I just answer my own
question?
Of course, I need to balance the complexity of any alternative solution
against simply executing the statement on the server.