1 Million users.. I can't Scale!!

[Paul Rubin]

Currently, the content is generated and a number of SMS per user are
generated. I'll have to measure this more accurately but a cursory
glance indicated that we're generting approximately 1000 sms per
second. (I'm sure this can't be right.. the parser\generator should be
faster than that

Don't be sure. Try some profiling, and maybe Psyco, C extensions,
etc. Python is many things but it's not terribly fast at
interpreting. It sounds like you have more bottlenecks than just the
python app though, starting with the SMS gateway issue that you just
mentioned. If you get more gateway bandwidth and your application can
split across multiple boxes, then the simple short-term fix is buy
more boxes. Longer term you need to re-examine the sw architecture
and possibly make changes.

 

[yoda]

Thanks for the whitepapers and incredibly useful advice. I'm beginning
to get a picture of what I should be thinking about and implementing to
achieve this kind of scalability. Before I go down any particular
route here's a synopsis of the application.

1)User requests are received only during subscription. We currently
don't have any problems with this because subscription rates increase
along a sigmoid curve.

2)Once a user subscribes we begin to send them content as it becomes
available.

3)The content is sports data. Content generation is dependent on the
day. On days when there's a lot of action, we can generate up to 20
separate items in a second every 10 minutes.

4)The content is event driven e.g. a goal is scored. It is therefore
imperative that we send the content to the subscribers within a period
of 5 minutes or less.

There is a difference between one million users each who make one request once a >month, and one million users who are each hammering the system with ten >requests a second. Number of users on its own is a meaningless indicator of >requirements.

Quite true and this lack of clarity was a mistake on my part. Requests
from users do not really become a significant part of this equation
because, as described above, once a user subscribes the onus is upon us
to generate messages throughout a given period determined by the number
of updates a user has subscribed to receive.

5)Currently, hardware is a constraint (we're a startup and can't afford
high end servers). I would prefer a solution that doesn't have to
result in any changes to the hardware stack. For now, let's assume that
hardware is not part of the equation and every optimization has to be
software based. (except the beautiful network optimizations suggested)

 

[Aahz]

Sure, multiple machines are probably the right approach for the OP; I
didn't mean to disagree with that. I just don't think they are "the
only practical way for a multi-process application to scale beyond a few
processors," like you said. For many (most?) applications in need of
serious scalability, multi-processor servers are preferable. IBM has
eServers available with up to 64 processors each, and Sun sells E25Ks
with 72 processors apiece. I like to work on those sorts of machine
when possible. Of course, they're not right for every application,
especially since they're so expensive.

Do these use shared memory?

 

[Jeff Schwab]

Do these use shared memory?

Yes.

 

[phil]

Quite true and this lack of clarity was a mistake on my part. Requests
from users do not really become a significant part of this equation
because, as described above, once a user subscribes the onus is upon us
to generate messages throughout a given period determined by the number
of updates a user has subscribed to receive.

So you are trying to SEND a million times several packets every
5 minutes?
No way Python is the bottleneck in that volume.
I have a POS app in Python that handles 10,000 packets
per SECOND including a MySQL lookup.
You have a bottleneck, but its not Python.

 

[Christos Georgiou]

For many (most?) applications in need of
serious scalability, multi-processor servers are preferable. IBM has
eServers available with up to 64 processors each, and Sun sells E25Ks
with 72 processors apiece.

SGI offers modular single-image Itanium2 servers of up to 512 CPU at the
moment:

http://www.sgi.com/products/servers/altix/configs.html

And NASA have clustered 20 of these machines to create a 10240 CPU
cluster...

I like to work on those sorts of machine
when possible. Of course, they're not right for every application,
especially since they're so expensive.

And expensive they are

 

[maxoutrage]

How are you actually sending messages to the SMSC?

If you are directly connected - IE using SMPP or UCP then I would
imagine that there is a bottle neck at the SMSC. Large SMSC systems
in the US typically deliver upto 1000 sm/s with larger systems
delivering
2000+ sm/s - From the throughput you require I can see this could be
a problem. I can imagine the carrier would not be happy for your
application
to dominate the SMSC and may impose throttling on your application.

What have you agreed with the carrier ragarding your connection. With
a high volume application it is important to verify the network
bandwidth
available and round trip delays. Also, verify the the windowing
parameters
for the SMPP/UCP connection and the maximum number of connections you
are
allowed to make.

Are you connecting to a multi-node SMSC? Then make separate connections
to each node.

Peter.