sharing complex data structures between threads in mod_perl2

Discussion in 'Perl' started by Dennis Gavrilov, Jul 8, 2003.

  1. Hi, All!

    I have two questions: strategic and technical.
    Technical one first:
    I need to share an array of objects (implemented as hashes, having
    references to other objects and hashes, sharing done after blessing)
    between all of the mod_perl2 threads. The structure can grow quite big
    - tenths of thousands of array elements. It can grow as system
    operates (not possible to construct at apache startup). Sharing array
    is OK, but inserting mixed structures of hash refs (sometimes blessed)
    afterward into it is problematic.
    I've written a small routine that attempts to do a deep copy of
    arbitrary data structure in order to accomplish the objective. The
    problem is that threads::shared::share (or routine logic) seems to
    replace original references. Hence circular reference handling code
    does not help. Anyway routine does not work for more elaborated
    structures.
    Maybe there are some better approaches to the task, which I'm missing
    out. Code of the routine is underneath.

    Strategic one: I'm dealing with ex-CGI application, that has to run
    under mod_perl 2. The main bottleneck is authorization system, which
    is highly configurable and allows to define permissions up to a single
    attribute of a particular row. It works nice under mod_perl 1. Changes
    to object structure are automatically reflected in authorization in
    both database and application server memory. The part in "application
    server memory" speeds up the whole thing greatly. Now under 2nd
    mod_perl every thread maintains its own idea of current state of
    authorization information, which is sometimes way out of date.
    I see currently several options of addressing the issue:
    1. Insert everywhere quite expensive database checks and bring
    authorization info of every thread up-to-date as needed - more
    development and some performance trade-offs
    2. Flush authorization info stored in every thread after some changes
    to reflected object structure - little development, but significant
    performance tradeoff :)
    3. Share the whole thing between all of the threads - great memory and
    performance gains, seemingly simple development, lack of stable
    platform
    Although giving the best benefits the third option has already turned
    out to be a two day long nightmare.
    I would greatly appreciate any opinions, ideas or comments on the two
    questions above.

    Thanks for reading it this far,
    Dennis

    sub share_struct {
    my $struct = shift;
    my $level = shift;
    my $map = shift || {};
    return unless ($] > 5.006001);
    require threads;
    require threads::shared;
    import threads;
    import threads::shared;
    my $type = ref $struct;
    if (exists $map->{$struct}) {
    return $struct;
    } elsif ($type) {
    $map->{$struct} = $struct; }
    die "Structure complexity exceeds maximum supported level" if
    ($level>5000);
    if ($type) {
    if ($type eq 'ARRAY') {
    my @arr_copy = @{$struct};
    for (my $i=0; $i<=$#arr_copy; $i++) {
    $arr_copy[$i] = ${&share_struct(\$arr_copy[$i], $level++,
    $map)}; }
    share($struct);
    @{$struct} = @arr_copy;
    } elsif ($type eq 'REF' or $type eq 'SCALAR') {
    if (ref $struct) {
    &share_struct($$struct, $level++, $map); share($struct);
    }
    } else { my %hash_copy = %{$struct};
    foreach my $key (keys %hash_copy) {
    $hash_copy{$key} = ${&share_struct(\$hash_copy{$key}, $level++,
    $map)};
    }
    share($struct);
    %{$struct} = %hash_copy;
    }
    }
    return $struct;
    }
     
    Dennis Gavrilov, Jul 8, 2003
    #1
    1. Advertising

  2. I've decided that sharing the entire structure is too much of a
    project for now and came up with a compromise solution. I need to have
    a flag variable shared between all of the mod_perl2 threads. Even this
    does not seem to work. I declare couple of shared variables and set
    their value to current time when flag has to be raised. Every other
    thread should check its local variable with last update time and
    update the structure if time in global variable is greater than one in
    local, setting local variable to update start time.
    Although code gives no error, supposedly shared varibales appear to
    have different values in every thread.
    I do it like this:

    use threads;
    use threads::shared;
    my $tree_flush_t_global : shared;

    sub mark_tree_dirty {
    $tree_flush_t_global = time;
    $tree_flush_t_local = $tree_flush_t_global;
    }

    sub self {
    ....
    if ($tree_flush_t_global > $tree_flush_t_local) {
    ....
    }

    I just can't get what I am doing wrong. Logic is so straightforward
    and has been verified so many times that I can not imagine having
    programing or design error.
    I'd really appreciate any comment or idea, except for switching to
    Java, for example, that already has well established threads support.
    Those I've heard already :)
     
    Dennis Gavrilov, Jul 24, 2003
    #2
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Ram P. Dash

    Sharing DLL between threads

    Ram P. Dash, May 27, 2005, in forum: ASP .Net
    Replies:
    6
    Views:
    610
    Ziga Jakhel
    May 31, 2005
  2. Rick Giuly
    Replies:
    3
    Views:
    375
    Chris Uppal
    Nov 13, 2004
  3. Replies:
    3
    Views:
    658
    jan V
    Sep 14, 2005
  4. Alfonso Morra
    Replies:
    11
    Views:
    721
    Emmanuel Delahaye
    Sep 24, 2005
  5. Fergus McMenemie
    Replies:
    2
    Views:
    85
    Fergus McMenemie
    Jul 24, 2006
Loading...

Share This Page