[BOUNTY] Pure Ruby Zlib::GzipWriter

[Daniel Berger]

Hi,

There's a project out there called "zliby" [1] which partially
implements the zlib library. Unfortunately, only the Zlib::GzipReader
class has been implemented so far. I'd like to have a pure Ruby
implementation of the Zlib::GzipWriter class.

I've put in a feature request [2] but I'd like to see it expedited. In
order to expedite this, I'm offering $100 to anyone who can implement
it. In order to qualify for the bounty, your Zlib::GzipWriter class
must meet the following criteria:

* It must be pure Ruby. No extensions. No FFI.
* It must have tests (and pass them)
* It must match the current Zlib::GzipWriter interface
* It must be cross platform
* It must be able to generate a proper gem file from a gemspec (the
acid test)
* It must be contributed back to the zliby project
* It must be done within 60 days of this post

If these conditions are met, you will receive your bounty via PayPal.

What's my motivation? Mainly MS Windows. I run a hand built version of
Ruby, and getting the zlib wrapper to build with MS VC++ is such a
colossal PITA that I'd like to avoid it completely. Beyond that,
however, is that it eliminates a third party dependency (zlib) and
gives me the ability to create gems. At the moment I can only read
them.

Any takers? Any matchers?

Regards,

Dan

[1] http:/www.rubyforge.org/projects/zliby
[2] http://rubyforge.org/tracker/index.php?func=detail&aid=23239&group_id=5769&atid=22371

 

[rogerdpack]

I've put in a feature request [2] but I'd like to see it expedited. In

order to expedite this, I'm offering $100 to anyone who can implement
it. In order to qualify for the bounty, your Zlib::GzipWriter class
must meet the following criteria:

That would indeed be convenient.
Would be nice to someday no longer have the dependency of rubygems on
zlib.
http://groups.google.com/group/comp.lang.ruby/browse_thread/thread/a6966f6fc1424f61
Then there'd be one less setup hurdle for ruby.
I'm a watcher for now though
-=R

 

[Charles Oliver Nutter]

I've put in a feature request [2] but I'd like to see it expedited. In
order to expedite this, I'm offering $100 to anyone who can implement
it. In order to qualify for the bounty, your Zlib::GzipWriter class
must meet the following criteria:

That would indeed be convenient.
Would be nice to someday no longer have the dependency of rubygems on
zlib.
http://groups.google.com/group/comp.lang.ruby/browse_thread/thread/a6966f6fc1424f61
Then there'd be one less setup hurdle for ruby.
I'm a watcher for now though

Of course you could use JRuby, which doesn't have such issues with zlib
and Windows...

- Charlie

 

[Daniel Berger]

I've put in a feature request [2] but I'd like to see it expedited. In
order to expedite this, I'm offering $100 to anyone who can implement
it. In order to qualify for the bounty, your Zlib::GzipWriter class
must meet the following criteria:

That would indeed be convenient.
Would be nice to someday no longer have the dependency of rubygems on
zlib.
http://groups.google.com/group/comp.lang.ruby/browse_thread/thread/a6...
Then there'd be one less setup hurdle for ruby.
I'm a watcher for now though

Of course you could use JRuby, which doesn't have such issues with zlib
and Windows...

I thought JRuby used FFI. No? If not, it should just be matter of
porting the code. But if so, you still have two third party
dependencies - FFI and zlib.dll.

Regards,

Dan

 

[Charles Oliver Nutter]

I thought JRuby used FFI. No? If not, it should just be matter of
porting the code. But if so, you still have two third party
dependencies - FFI and zlib.dll.

The JVM includes zlib and obviously has a very nice version for Windows.
We just wrap the JVM's zlib-based compression classes. But that doesn't
really solve your original problem since you want to run a handbuilt C
Ruby...

- Charlie

 

[The Higgs bozo]

What's my motivation? Mainly MS Windows. I run a hand built version of
Ruby, and getting the zlib wrapper to build with MS VC++ is such a
colossal PITA that I'd like to avoid it completely. Beyond that,
however, is that it eliminates a third party dependency (zlib) and
gives me the ability to create gems. At the moment I can only read
them.

As a cheap shortcut, could you store a base64-encoded zlib dll in a .rb
file, decode it, then call it with DL or FFI?

 

[Daniel Berger]

As a cheap shortcut, could you store a base64-encoded zlib dll in a .rb
file, decode it, then call it with DL or FFI?

I'm not really sure how to do that, and it's not really what I'm
looking for I'm afraid.

Regards,

Dan

 

[The Higgs bozo]

I'm not really sure how to do that, and it's not really what I'm
looking for I'm afraid.

You didn't know how, or didn't understand? DL is part of the standard
library. You could carry around the zlib dll right inside an .rb file,
and use DL to call the zlib function(s) you needed.

I understand you were looking for a general-purpose solution, but in the
meantime this does what you need, albeit gymnastically. Namely, zlib on
Windows without the requirement of building a ruby extension.

 

[Daniel Berger]

You didn't know how, or didn't understand? =A0DL is part of the standard
library. =A0You could carry around the zlib dll right inside an .rb file,
and use DL to call the zlib function(s) you needed.

Just to clarify for purposes of the bounty, no DL, and no distributed
dll's.

I understand you were looking for a general-purpose solution, but in the
meantime this does what you need, albeit gymnastically. =A0Namely, zlib o= n
Windows without the requirement of building a ruby extension.

I really do appreciate the suggestion but it's not what I'm looking
for.

Regards,

Dan

 

[Joel VanderWerf]

library. You could carry around the zlib dll right inside an .rb file,
and use DL to call the zlib function(s) you needed.

Out of curiosity, how do you suggest doing this? Put the dll in the DATA
section, write to a tmp file, and load? Or is there some magic?

 

[Roger Pack]

What's my motivation? Mainly MS Windows. I run a hand built version of

Ruby, and getting the zlib wrapper to build with MS VC++ is such a
colossal PITA that I'd like to avoid it completely. Beyond that,
however, is that it eliminates a third party dependency (zlib) and
gives me the ability to create gems. At the moment I can only read
them.

One thing I use to help me compile on doze is Lavena's rubyinstaller
[I'm sure you've seen it..].
Anyway I'll match $50
-=r

 

[Daniel Berger]

What's my motivation? Mainly MS Windows. I run a hand built version of
Ruby, and getting the zlib wrapper to build with MS VC++ is such a
colossal PITA that I'd like to avoid it completely. Beyond that,
however, is that it eliminates a third party dependency (zlib) and
gives me the ability to create gems. At the moment I can only read
them.

One thing I use to help me compile on doze is Lavena's rubyinstaller
[I'm sure you've seen it..].
Anyway I'll match $50

Excellent Roger, thank you.

By the way, MS Windows is not the only reason removing the external
dependency on zlib would be a good thing:

http://rubyforge.org/tracker/index.php?func=3Ddetail&aid=3D23354&group_id=
=3D126&atid=3D575

Regards,

Dan

 

[Daniel Berger]

Hi,

There's a project out there called "zliby" [1] which partially
implements the zlib library. Unfortunately, only the Zlib::GzipReader
class has been implemented so far. I'd like to have a pure Ruby
implementation of the Zlib::GzipWriterclass.

I've put in a feature request [2] but I'd like to see it expedited. In
order to expedite this, I'm offering $100 to anyone who can implement
it. In order to qualify for the bounty, your Zlib::GzipWriterclass
must meet the following criteria:

* It must be pure Ruby. No extensions. No FFI.
* It must have tests (and pass them)
* It must match the current Zlib::GzipWriterinterface
* It must be cross platform
* It must be able to generate a proper gem file from a gemspec (the
acid test)
* It must be contributed back to the zliby project
* It must be done within 60 days of this post

If these conditions are met, you will receive your bounty via PayPal.

What's my motivation? Mainly MS Windows. I run a hand built version of
Ruby, and getting the zlib wrapper to build with MS VC++ is such a
colossal PITA that I'd like to avoid it completely. Beyond that,
however, is that it eliminates a third party dependency (zlib) and
gives me the ability to create gems. At the moment I can only read
them.

Yet more ammunition:

http://www.misuse.org/science/2007/09/07/ruby-zlib-buffer-error-on-windows/

Regards,

Dan

 

[Roger Pack]

Yet more ammunition:

http://www.misuse.org/science/2007/09/07/ruby-zlib-buffer-error-on-windows/

Interesting. I wonder if it's somehow related to
http://redmine.ruby-lang.org/issues/show/776
-=r

 

[Daniel Berger]

Interesting. =A0I wonder if it's somehow related to http://redmine.ruby-l=

ang.org/issues/show/776

It probably is, but there's nothing we can do about it since zlib
isn't developed separately as a gem, and it would still require that
someone build a binary.

If we had a pure Ruby version available as a gem on the other hand...



Regards,

Dan

 

[Charles L.]

If we had a pure Ruby version available as a gem on the other hand...



Regards,

Dan

I'm most of the way done, just missing dynamic huffman trees, and more
interface similarity with zlib. Need to change the specs from zliby
though as the algorithm I'm using is not identical to zlib and gives
different (but still valid) results.

For example, zlib's lz77 phase doesn't seem to behave optimally wrt
repeats:
=> true

(using CZlib for inflation there as I've not integrated it with the
zliby code yet).

 

[Heesob Park]

Hi,

If we had a pure Ruby version available as a gem on the other hand...

I finished translation and testing of the pure ruby version.
The ruby version is exact same api interface and functionality with
the zlib c source code.
It is also compatible with ruby 1.8.x and 1.9.x and includes
examble.rb and minizip.rb for testing.

You can download at http://121.78.227.9/rbzlib/

I am working on translation of ruby library zlib.c to pure ruby code.

Regards,

Park Heesob

 

[Daniel Berger]

Hi,


I finished translation and testing of the pure ruby version.
The ruby version is exact same api interface and functionality with
the zlib c source code.
It is also compatible with ruby 1.8.x and 1.9.x and includes
examble.rb and minizip.rb for testing.

You can download at http://121.78.227.9/rbzlib/
Awesome!

I am working on translation of ruby library zlib.c to pure ruby code.

Looks like you and Charles L. are in a race.

Regards,

Dan

 

[Daniel Berger]

I'm most of the way done, just missing dynamic huffman trees, and more
interface similarity with zlib. Need to change the specs from zliby
though as the algorithm I'm using is not identical to zlib and gives
different (but still valid) results.

For example, zlib's lz77 phase doesn't seem to behave optimally wrt
repeats:

=> true

(using CZlib for inflation there as I've not integrated it with the
zliby code yet).

Sounds like you're on track. But, you're in a race with Park Heesob.

Dan

 

[Charles L.]

Sounds like you're on track. But, you're in a race with Park Heesob.

Dan

Heh. Somehow I think he's got it covered. That reimplementation of
zlib in ruby is actually pretty cool. I'm curious Park, as to how much
you did by hand and how much you were able to automate it...?

I've put the deflate code i wrote on pastie - http://pastie.org/355367

It's about 600 lines, but needs cleaning up. Will probably be ~500 when
I've added in dynamic trees and cleaned it up a bit.

Prelim benchmarks show the current bottleneck to be the LZ77
implementation which is interesting. I think for it to perform
acceptably, a lot more of the code needs to become more idiomatic,
higher-level ruby.