Best architecture for proxy?

[Andrew Warkentin]

I am going to write a general-purpose modular proxy in Python. It will
consist of a simple core and several modules for things like filtering
and caching. I am not sure whether it is better to use multithreading,
or to use an event-driven networking library like Twisted or Medusa/
Asyncore. Which would be the better architecture to use?

 

[Bjoern Schliessmann]

I am going to write a general-purpose modular proxy in Python. It
will consist of a simple core and several modules for things like
filtering and caching. I am not sure whether it is better to use
multithreading, or to use an event-driven networking library like
Twisted or Medusa/ Asyncore. Which would be the better
architecture to use?

I'd definitely use an event-driven approach with Twisted.

Generally, multithreading is less performant than multiplexing. High
performance servers mostly use a combination of both, though.

Regards,


Björn

 

[Steve Holden]

I'd definitely use an event-driven approach with Twisted.

Generally, multithreading is less performant than multiplexing. High
performance servers mostly use a combination of both, though.

Converselt I'd recommend Medusa - not necessarily because it's "better",
but becuase I know it better. There's also a nice general-purpose proxy
program (though I'd be surprised if Twisted didn't also have one).

regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
--------------- Asciimercial ------------------
Get on the web: Blog, lens and tag the Internet
Many services currently offer free registration
----------- Thank You for Reading -------------

 

[Andrew Warkentin]

Converselt I'd recommend Medusa - not necessarily because it's "better",
but becuase I know it better. There's also a nice general-purpose proxy
program (though I'd be surprised if Twisted didn't also have one).

Would an event-driven proxy be able to handle multiple connections
with large numbers of possibly CPU-bound filters? I use The
Proxomitron (and would like to write my own proxy that can use the
same filter sets, but follows the Unix philosophy) and some of the
filters appear to be CPU-bound, because they cause The Proxomitron to
hog the CPU (although that might just be a Proxomitron design flaw or
something). Wouldn't CPU-bound filters only allow one connection to be
filtered at a time? On the Medusa site, it said that an event-driven
architecture only works for I/O-bound programs.

 

[Steve Holden]

Would an event-driven proxy be able to handle multiple connections
with large numbers of possibly CPU-bound filters? I use The
Proxomitron (and would like to write my own proxy that can use the
same filter sets, but follows the Unix philosophy) and some of the
filters appear to be CPU-bound, because they cause The Proxomitron to
hog the CPU (although that might just be a Proxomitron design flaw or
something). Wouldn't CPU-bound filters only allow one connection to be
filtered at a time? On the Medusa site, it said that an event-driven
architecture only works for I/O-bound programs.

Well since Medusa doesn't thread or fork you are correct, it's likely
that connection requests could be rejected if the response processors
take a significant amount of real and CPU time.

regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
--------------- Asciimercial ------------------
Get on the web: Blog, lens and tag the Internet
Many services currently offer free registration
----------- Thank You for Reading -------------