Sharing Data in Python

Discussion in 'Python' started by raunakgup90@gmail.com, May 11, 2012.

  1. Guest

    I have some Pickled data, which is stored on disk, and it is about 100 MB in size.

    When my python program is executed, the picked data is loaded using the cPickle module, and all that works fine.

    If I execute the python multiple times using python main.py for example, each python process will load the same data multiple times, which is the correct behaviour.

    How can I make it so, all new python process share this data, so it is only loaded a single time into memory?

    asked the same question on SO, but could not get any constructive responses.. http://stackoverflow.com/questions/10550870/sharing-data-in-python/10551845
    , May 11, 2012
    #1
    1. Advertising

  2. Miki Tebeka Guest

    > How can I make it so, all new python process share this data, so it is only loaded a single time into memory?
    You can have one process as server and client ask for parts of data.
    You might be able to do something smart with mmap but I can't think of a way.

    I Linux systems, if you first load the data and then fork, the OS will keep all the read only data shared.
    Miki Tebeka, May 12, 2012
    #2
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Jason
    Replies:
    1
    Views:
    707
  2. Dennis Gavrilov
    Replies:
    1
    Views:
    1,438
    Dennis Gavrilov
    Jul 24, 2003
  3. PeteZ

    Inter-Application Data sharing

    PeteZ, Oct 31, 2003, in forum: ASP .Net
    Replies:
    2
    Views:
    317
    Guest
    Oct 31, 2003
  4. Darren Oakey

    Sharing Session data across two projects

    Darren Oakey, Dec 19, 2003, in forum: ASP .Net
    Replies:
    0
    Views:
    409
    Darren Oakey
    Dec 19, 2003
  5. Mervin Williams

    Sharing data across web forms?

    Mervin Williams, Apr 9, 2004, in forum: ASP .Net
    Replies:
    2
    Views:
    467
    Cor Ligthert
    Apr 9, 2004
Loading...

Share This Page