Distribution

Discussion in 'Python' started by prince.pangeni, Mar 20, 2012.

  1. Hi all,
    I am doing a simulation project using Python. In my project, I want
    to use some short of distribution to generate requests to a server.
    The request should have two distributions. One for request arrival
    rate (should be poisson) and another for request mix (i.e. out of the
    total requests defined in request arrival rate, how many requests are
    of which type).
    Example: Suppose the request rate is - 90 req/sec (generated using
    poisson distribution) at time t and we have 3 types of requests (i.e.
    r1, r2, r2). The request mix distribution output should be similar to:
    {r1 : 50 , r2 : 30 , r3 : 10} (i.e. out of 90 requests - 50 are of r1
    type, 30 are of r2 type and 10 are of r3 type).
    As I an new to python distribution module, I am not getting how to
    code this situation. Please help me out for the same.

    Thanks in advance

    Prince
    prince.pangeni, Mar 20, 2012
    #1
    1. Advertising

  2. prince.pangeni

    Peter Otten Guest

    prince.pangeni wrote:

    > Hi all,
    > I am doing a simulation project using Python. In my project, I want
    > to use some short of distribution to generate requests to a server.
    > The request should have two distributions. One for request arrival
    > rate (should be poisson) and another for request mix (i.e. out of the
    > total requests defined in request arrival rate, how many requests are
    > of which type).
    > Example: Suppose the request rate is - 90 req/sec (generated using
    > poisson distribution) at time t and we have 3 types of requests (i.e.
    > r1, r2, r2). The request mix distribution output should be similar to:
    > {r1 : 50 , r2 : 30 , r3 : 10} (i.e. out of 90 requests - 50 are of r1
    > type, 30 are of r2 type and 10 are of r3 type).
    > As I an new to python distribution module, I am not getting how to
    > code this situation. Please help me out for the same.


    You don't say what distribution module you're talking of, and I guess I'm
    not the only one who'd need to know that detail.

    However, with sufficient resolution and duration the naive approach sketched
    below might be good enough.

    # untested
    DURATION = 3600 # run for one hour
    RATE = 90 # requests/sec
    RESOLUTION = 1000 # one msec

    requests = ([r1]*50 + [r2]*30 + [r3]*10)
    time_slots = [0]*(RESOLUTION*DURATION)
    times = range(RESOLUTION*DURATION)

    for _ in range(DURATION*RATE):
    time_slots[random.choice(times)] += 1

    for time, count in enumerate(time_slots):
    for _ in range(count):
    issue_request_at(random.choice(requests), time)
    Peter Otten, Mar 20, 2012
    #2
    1. Advertising

  3. prince.pangeni

    Robert Kern Guest

    On 3/20/12 11:21 AM, Ben Finney wrote:
    > "prince.pangeni"<> writes:
    >
    >> I am doing a simulation project using Python. In my project, I want
    >> to use some short of distribution to generate requests to a server.

    >
    > What is a distribution? That term already means something in Python
    > jargon, and it doesn't match the rest of your use case.
    >
    > So what do you mean by “distribution� Maybe we can find a less
    > confusing term.


    Judging from the context, he means a probability distribution.

    --
    Robert Kern

    "I have come to believe that the whole world is an enigma, a harmless enigma
    that is made terrible by our own mad attempt to interpret it as though it had
    an underlying truth."
    -- Umberto Eco
    Robert Kern, Mar 20, 2012
    #3
  4. prince.pangeni

    Robert Kern Guest

    On 3/20/12 4:31 AM, prince.pangeni wrote:
    > Hi all,
    > I am doing a simulation project using Python. In my project, I want
    > to use some short of distribution to generate requests to a server.
    > The request should have two distributions. One for request arrival
    > rate (should be poisson) and another for request mix (i.e. out of the
    > total requests defined in request arrival rate, how many requests are
    > of which type).
    > Example: Suppose the request rate is - 90 req/sec (generated using
    > poisson distribution)


    Just a note on terminology to be sure we're clear: a Poisson *distribution*
    models the number of arrivals in a given time period if the events are from a
    Poisson *process* with a given mean rate. To model the inter-event arrival
    times, you use an exponential distribution. If you want to handle events
    individually in your simulation, you will need to use the exponential
    distribution to figure out the exact times for each. If you are handling all of
    the events in each second "in bulk" without regard to the exact times or
    ordering within that second, then you can use a Poisson distribution.

    > at time t and we have 3 types of requests (i.e.
    > r1, r2, r2). The request mix distribution output should be similar to:
    > {r1 : 50 , r2 : 30 , r3 : 10} (i.e. out of 90 requests - 50 are of r1
    > type, 30 are of r2 type and 10 are of r3 type).
    > As I an new to python distribution module, I am not getting how to
    > code this situation. Please help me out for the same.


    I am going to assume that you want to handle each event independently. A basic
    strategy is to keep a time variable starting at 0 and use a while loop until the
    time reaches the end of the simulation time. Increment it using a draw from the
    exponential distribution each loop. Each iteration of the loop is an event. To
    determine the kind of event, you will need to draw from a weighted discrete
    distribution. What you want to do here is to do a cumulative sum of the weights,
    draw a uniform number from 0 to the total sum, then use bisect to find the item
    that matches.

    import bisect
    import random


    # Use a seeded PRNG for repeatability. Use the methods on the Random
    # object rather than the functions in the random module.
    prng = random.Random(1234567890)

    avg_rate = 90.0 # reqs/sec

    kind_weights = [50.0, 30.0, 10.0]
    kind_cumsum = [sum(kind_weights[:i+1]) for i in range(len(kind_weights))]
    kind_max = kind_cumsum[-1]

    max_time = 10.0 # sec
    t = 0.0 # sec
    events = [] # (t, kind)
    while t < max_time:
    dt = prng.expovariate(avg_rate)
    u = prng.uniform(0.0, kind_max)
    kind = bisect.bisect_left(kind_cumsum, u)
    events.append((t, kind))
    t += dt


    --
    Robert Kern

    "I have come to believe that the whole world is an enigma, a harmless enigma
    that is made terrible by our own mad attempt to interpret it as though it had
    an underlying truth."
    -- Umberto Eco
    Robert Kern, Mar 20, 2012
    #4
  5. On Tue, 20 Mar 2012 12:00:50 +0000, Robert Kern <>
    declaimed the following in gmane.comp.python.general:

    > On 3/20/12 11:21 AM, Ben Finney wrote:
    > > "prince.pangeni"<> writes:
    > >
    > >> I am doing a simulation project using Python. In my project, I want
    > >> to use some short of distribution to generate requests to a server.

    > >
    > > What is a distribution? That term already means something in Python
    > > jargon, and it doesn't match the rest of your use case.
    > >
    > > So what do you mean by “distribution”? Maybe we can find a less
    > > confusing term.

    >
    > Judging from the context, he means a probability distribution.
    >

    Futhermore, he explicitly mentions Poisson later in the text.

    That should just be a case of finding the mathematical definition of
    the distribution and coding something using random (since the random
    module has gaussian/normal, but not Poisson -- maybe numpy/simpy?) to
    obtain the time variant.

    I suspect the real question is how to handle the weighted selection
    of r1-r3. And for such a relatively short sample set (90 entries), the
    chunked list shown previously, with randint to index into it is viable.

    The alternative, more generalized (in a way), would be to use a
    sorted list of the probabilities (or sums)... (pseudo-code)

    probs = [ (numR1, r1),
    (numR1 + numR2, r2),
    (numR1 + numR2 + numR3, r3) ]
    #obviously one is unlikely to actually create constants for the numRs
    #more likely to create in line
    probs.sort()

    ran = random.randint(numR1 + numR2 + numR3)
    for (p, r) in probs:
    if ran > p: continue
    theR = r
    --
    Wulfraed Dennis Lee Bieber AF6VN
    HTTP://wlfraed.home.netcom.com/
    Dennis Lee Bieber, Mar 20, 2012
    #5
  6. Il 20/03/2012 12:21, Ben Finney ha scritto:
    > "prince.pangeni"<> writes:
    >
    >> I am doing a simulation project using Python. In my project, I want
    >> to use some short of distribution to generate requests to a server.


    I guess scipy is also available in plain python (didn't check), but the
    following works with Sage :

    ----------------------------------------------------------------------
    | Sage Version 4.8, Release Date: 2012-01-20 |
    | Type notebook() for the GUI, and license() for information. |
    ----------------------------------------------------------------------
    sage: from scipy import stats
    sage: X=stats.poisson.rvs
    sage: X(4)
    5
    sage: X(4)
    2
    sage: X(4)
    3


    Hope it helps
    Laurent
    Laurent Claessens, Mar 20, 2012
    #6
  7. On Tue, 20 Mar 2012 11:29:37 -0400, Dennis Lee Bieber
    <> declaimed the following in
    gmane.comp.python.general:


    > module has gaussian/normal, but not Poisson -- maybe numpy/simpy?) to
    > obtain the time variant.
    >

    Whoops -- scipy, not simpy


    --
    Wulfraed Dennis Lee Bieber AF6VN
    HTTP://wlfraed.home.netcom.com/
    Dennis Lee Bieber, Mar 20, 2012
    #7
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Oliver Denzel

    Re: C# windows app distribution thru web

    Oliver Denzel, Jul 7, 2003, in forum: ASP .Net
    Replies:
    0
    Views:
    1,117
    Oliver Denzel
    Jul 7, 2003
  2. Edwin E. Smith

    MSDE database distribution?

    Edwin E. Smith, Aug 17, 2005, in forum: ASP .Net
    Replies:
    2
    Views:
    712
    Edwin E. Smith
    Aug 18, 2005
  3. Dan Nordquist
    Replies:
    0
    Views:
    746
    Dan Nordquist
    Jul 11, 2003
  4. cade

    msde database distribution

    cade, Nov 17, 2003, in forum: ASP .Net
    Replies:
    2
    Views:
    354
    Cowboy \(Gregory A. Beamer\)
    Nov 17, 2003
  5. Arnaud
    Replies:
    0
    Views:
    329
    Arnaud
    Dec 9, 2003
Loading...

Share This Page