How to use threads to parallelise 4 different DB connections?

[anju]

Hi ,

My script is making connection to the four databases one by one and
writing the output to a single excelsheet. But there being huge
records, its taking too much time ( 4 hours), which is not acceptable.

So I need to synchronise the process so that the connection is made to
all the 4 DBs at one time and they run parallely.

I tried using Perl threads/ ithreads for the same but its giving
problem, like handles can't be shared , use clone() method etc. etc.

Can you please advice how can I do the same?

Please reply ASAP.

Many Thanks in advance.

Regards,
Anju

 

[Nick of course]

Hi ,

My script is making connection to the four databases one by one and
writing the output to a single excelsheet. But there being huge
records, its taking too much time ( 4 hours), which is not acceptable.

So I need to synchronise the process so that the connection is made to
all the 4 DBs at one time and they run parallely.

I tried using Perl threads/ ithreads for the same but its giving
problem, like handles can't be shared , use clone() method etc. etc.

Can you please advice how can I do the same?

Please reply ASAP.

Many Thanks in advance.

Regards,
Anju

I'm assuming that the excessive time taken is due to database access
and/or intermediate processing - there is no way it can take 4 hours to
write to a spreadsheet; they just don't get that big. Use four
separate processes (not threads) running in parallel that write to
intermediate files (maybe use Storable for efficiency) then a final
process to write the spreadsheet. As a solution, it is a lot simpler
and more robust than using threads.

 

[xhoster]

Hi ,

My script is making connection to the four databases one by one and
writing the output to a single excelsheet. But there being huge
records, its taking too much time ( 4 hours), which is not acceptable.

How can one Excel spreadsheet hold 4 hours worth of records? That just
doesn't sound plausible for a decently tuned query. This suggests you
probably should spend more time tuning your database/sql and less time
tuning your perl.

So I need to synchronise the process so that the connection is made to
all the 4 DBs at one time and they run parallely.

Why not just use 4 programs which run simultanously and dump into temp
files, and a 5th program which combines these files into one final one?

I tried using Perl threads/ ithreads for the same but its giving
problem, like handles can't be shared , use clone() method etc. etc.

If you showed me real code with the real errors it produces, maybe I could
help. If you don't, the chances are much less that I can.

Can you please advice how can I do the same?

Please reply ASAP.

We are not paid consultants. Treating us as if we were is likely to
engender ill-will.

Xho

 

[Dr.Ruud]

anju schreef:

Please reply ASAP.

Right after your donation:
http://donate.perl-foundation.org/index.pl?node=Fund+Drive+Details&selfund=2
http://donate.perl-foundation.org/index.pl?node=Contribution FAQ