python shutting down sloooooooowly/tuning dictionaries

Discussion in 'Python' started by Till Plewe, Apr 21, 2004.

  1. Till Plewe

    Till Plewe Guest

    Is there a way to speed up killing python from within a python program?
    Sometimes shutting down takes more than 10 times as much time as the actual
    running of the program.

    The programs are fairly simple (searching/organizing large boardgame
    databases) but use a lot of memory (1-6GB). The memory is mostly used
    for simple structures like trees or relations. Typically there will be
    a few large dictionaries and many small dictionaries/sets/arrays/lists.

    Since the entire database is too large I typically run the same
    program 10-100 times on smaller parts. The annoying bit is that the
    time for finishing one instance of a program before starting the next
    instance can take a lot of time. I have started using a shell script to
    check whether certain files have been written and then kill the
    program from the os to speed up everything. But this solution is way
    too ugly. There should be a better way, but I don't seem to be able to
    find one.

    A second question. Is it possible to control the resizing of
    dictionaries? It seems that resizing always doubles the size, but for
    dictionaries using more than half of the remaining memory than this
    is a problem. I would rather have a slightly too full dictionaries in
    main memory than have a sparse dictionary partially swapped out.

    I would like to be able to restrict the maximum growth of each resize
    operation to around 100MB or so.

    TIA.

    - Till
     
    Till Plewe, Apr 21, 2004
    #1
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. EW
    Replies:
    6
    Views:
    1,102
  2. Replies:
    3
    Views:
    1,051
    bruno at modulix
    May 16, 2006
  3. Jack
    Replies:
    2
    Views:
    1,605
  4. lysdexia
    Replies:
    6
    Views:
    530
    John Machin
    Dec 2, 2007
  5. Brandon
    Replies:
    12
    Views:
    504
    Brandon
    Aug 15, 2008
Loading...

Share This Page