Access to perl structures from C programs

[boris.ivanov]

Dear All

I just wonder if u know proper solution.
I have perl software which keeps hashes. This hashes I gonna dump on
disk or in shared memory. Then I need to access these hashes and
strings in read/write mode from C programs.
Sort of Windows registry. Its hould be fast and transactional safe.

I did in memory drive and dump perl structures in XML format but found
it kinda slow. XML perl dumper wasnt really fast. So now Im thinking
about make C library to parse/compile perl hashes. Any1 heard about C
Lib which can work with perl in memory data?

Thanks

Boris

 

[xhoster]

Dear All

I just wonder if u know proper solution.
I have perl software which keeps hashes. This hashes I gonna dump on
disk or in shared memory. Then I need to access these hashes and
strings in read/write mode from C programs.
Sort of Windows registry. Its hould be fast and transactional safe.

What exactly do you mean by "transactional safe"? Safe for transactions
involving mutiple changes in one Atomic block? Durable? Concurrent?
Isolated? Safe for multiple processes on machine? Multiple threads in
one process? Multiple processes on multiple machines?

I did in memory drive and dump perl structures in XML format but found
it kinda slow. XML perl dumper wasnt really fast. So now Im thinking
about make C library to parse/compile perl hashes. Any1 heard about C
Lib which can work with perl in memory data?

You mean like XS, or embedded perl? perldoc perlxs, perldoc perlembed.

Xho

 

[boris.ivanov]

Thanks for reply

I mean this suppose to be multitasking environment x writers and y
readers on the same host. Some of processes will run under root
permissions some under nobody. PC isolated from inet, so security not a
question.

I wish to have reader/writer of simple structure like

%d = (
"a" => "1",
"b" => "kjhlkjhlkjh"
);

keys and values both treated as strings.
So set/get/unset operations suppose to be atomic per key. I suppose to
keep this in shared memory. Dont think socket solution will be fast
enough.

I know that xs good for writing perl modules using C/C++ I have
opposite task. C/C++ programs which should read/write perl data.

I checked perlembed its sound great but looks like not far away from
lunching perl pass arguments and pass back output to C programs.

Thanks
Boris

 

[xhoster]

Thanks for reply

I mean this suppose to be multitasking environment x writers and y
readers on the same host. Some of processes will run under root
permissions some under nobody. PC isolated from inet, so security not a
question.

I wish to have reader/writer of simple structure like

%d = (
"a" => "1",
"b" => "kjhlkjhlkjh"
);

keys and values both treated as strings.
So set/get/unset operations suppose to be atomic per key. \

Have you considered using some kind of dedicated RDBMS (mysql,
postgres, oracle, etc.) to handle this part?

I suppose to
keep this in shared memory. Dont think socket solution will be fast
enough.

If speed is of such utmost concern, then I think you are going about this
wrong. A Perlish data structure is most likely going to have a Perlish
performance, even if accessed from C. You should instead seek a C-ish data
structure.

I know that xs good for writing perl modules using C/C++ I have
opposite task. C/C++ programs which should read/write perl data.

I think you want the wrong thing. Have C/C++ programs which read/write
C/C++ data structures. If you need to give Perl shared access to those
same data structures (which may be a mistake--If Perl is too slow to use
for the main body of the work, then you probably don't want the lumbering
beast locking elements of the data and holding the locks for the duration
of those too-slow Perl operation, interfering with the faster C programs)
then use XS to implement Perl access to the underlying C-ish structures.

Also note that Perl doesn't have good ways to put its own data structures
into shared memory. Modules exist, but they are generally not very natural
or efficient. So I think that trying to use Perl data structures in shared
memory and have C access in ways more efficient than Perl can access them
itself is somewhat of a fool's errand.

I checked perlembed its sound great but looks like not far away from
lunching perl pass arguments and pass back output to C programs.

Well, that doesn't really seem that far away from what you want

Xho

 

[boris.ivanov]

You have a point. Idea with MySQL not bad (using in memory tables since
I dont need to keep data between restarts).

Idea of simplifying perl data good as well. I will transform set/get to
be string only.

Thanks for reply.

BR
Boris