SoapExtensions, DIME Attachment and Pipes

O

Oleg

Does anybody know a good C# replacement for
java.io.PipedInputStream, PipedOutputStream?

The reason I'm asking is sometimes I need transfer really
amount of huge data (DataSet) using web services.
At the same time this data can be compressed up to 50
times using #ZipLib library.
I have tried SoapExtension & DIME Attachment methods.
They both work pretty well, except consuming too much
memory on both client and server sides.

I currently use this approach (DIME Attachment):
1. Create DataSet and populate it with data.
2. Create compressed memory stream [gzipStream = new
GZipOutputStream(new MemoryStream(1024))] (i.e.
MemoryStream "decorated by" GZipOutputStream)
2. Serialize the DataSet into the stream [ds.WriteXml
(gzipStream)]
3. Use gzipStream as input stream for DimeAttachment
[DimeAttachment dimeAttachment = new DimeAttachment
("application/x-gzip",
TypeFormatEnum.MediaType,
memoryStream)]

I'm not sure about implementation of dimeAttachment, but
I hope it writes data
from dimeAttachment stream directly (i.e. without
buffering whole stream in memory!) and asynchronously
into "network stream".

Things are much worse with SoapExtension...
(I can describe if somebody really interested :)

But if we can use PipedInputStream, PipedOutputStream it
will work like this:
1. Got populated DataSet.
2. Create connected PipedOutputStream & PipedInputStream.
3. Create new thread (T) and give it DataSet &
PipedOutputStream
4. When thread (T) runs, it starts writing DataSet into
PipedOutputStream "decorated" by GZipOutputStream.
5. Use PipedInputStream as input stream for DimeAttachment

The difference with previous scenario is that
we never have whole serialized & zipped DataSet in memory.
Instead we have only small part of it hold in a buffer
used by piped streams.
PipedOutputStream.write call will block when the buffer
is full.
PipedInputStream.read call will block when the buffer is
empty.

So, does anybody know a good C# replacement for
java.io.PipedInputStream, PipedOutputStream? :)
Sincerely,
Oleg

P.S. I know what asynchronous IO is. I.e. I think I
know! :)
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,483
Members
44,903
Latest member
orderPeak8CBDGummies

Latest Threads

Top