[ANN] LibInject 0.1.0: Small Pieces Smooshed Together

[Francis Hwang]

I've just released the first version of LibInject, which is a developer
tool for injecting external dependencies into a Ruby file. More
specifically, it finds require statements that point to files that
aren't in the standard library, expands those files, and drops the raw
text into the original file.

http://libinject.rubyforge.org/

Why would somebody do this instead of just using a plain old require
statement? One reason might be because you want to write a one-file
script that can also have external dependencies. I wrote LibInject for
use with FeedBlender, which is intended as a one-file script aimed at
Ruby newbies who might not even know how to install from .tar.gz files
or from RubyGems. With LibInject, I can have external dependencies in
FeedBlender, use a Rake task to inject those libraries into
feedblender.rb, and then distribute the end result.

For example, let's say you've three files, a.rb, b.rb, and c.rb. a.rb
requires b.rb and c.rb, and b.rb requires c.rb:

a.rb:

require 'b'
require 'c'

class A; end

b.rb:

require 'c'

class B; end

c.rb:

class C; end

If you wanted to distribute a version of a.rb without any external
dependencies, this is how you'd run LibInject:

require 'libinject'

puts LibInject.lib_inject( File.open( 'a.rb' ) )

# We could also pass LibInject.lib_inject a String, like so:
# contents = File.open( 'a.rb' ) do |f| f.gets( nil ); end
# puts LibInject.lib_inject( contents )

And this is what you'll get:

#
v---------v---------v---------v---------v---------v---------v---------v
# LibInject: begin 'b' library injection
#
v---------v---------v---------v---------v---------v---------v---------v


#
v---------v---------v---------v---------v---------v---------v---------v
# LibInject: begin 'c' library injection
#
v---------v---------v---------v---------v---------v---------v---------v

class C; end

#
^---------^---------^---------^---------^---------^---------^---------^
# LibInject: end 'c' library injection
#
^---------^---------^---------^---------^---------^---------^---------^


class B; end

#
^---------^---------^---------^---------^---------^---------^---------^
# LibInject: end 'b' library injection
#
^---------^---------^---------^---------^---------^---------^---------^



class A; end

The code looks sort of messy, but now it's all in one file.

Anyway, it's a strange little script, and I'm curious as to whether
anybody else will get some use out of it. Let me know if you do!

F.

 

[Jim Weirich]

I've just released the first version of LibInject, which is a developer
tool for injecting external dependencies into a Ruby file. More
specifically, it finds require statements that point to files that
aren't in the standard library, expands those files, and drops the raw
text into the original file.

http://libinject.rubyforge.org/

Why would somebody do this instead of just using a plain old require
statement? One reason might be because you want to write a one-file
script that can also have external dependencies. I wrote LibInject for
use with FeedBlender, which is intended as a one-file script aimed at
Ruby newbies who might not even know how to install from .tar.gz files
or from RubyGems. With LibInject, I can have external dependencies in
FeedBlender, use a Rake task to inject those libraries into
feedblender.rb, and then distribute the end result.

For example, let's say you've three files, a.rb, b.rb, and c.rb. a.rb
requires b.rb and c.rb, and b.rb requires c.rb:

a.rb:

require 'b'
require 'c'

class A; end

b.rb:

require 'c'

class B; end

c.rb:

class C; end

If you wanted to distribute a version of a.rb without any external
dependencies, this is how you'd run LibInject:

require 'libinject'

puts LibInject.lib_inject( File.open( 'a.rb' ) )

# We could also pass LibInject.lib_inject a String, like so:
# contents = File.open( 'a.rb' ) do |f| f.gets( nil ); end
# puts LibInject.lib_inject( contents )

And this is what you'll get:

#
v---------v---------v---------v---------v---------v---------v---------v
# LibInject: begin 'b' library injection
#
v---------v---------v---------v---------v---------v---------v---------v


#
v---------v---------v---------v---------v---------v---------v---------v
# LibInject: begin 'c' library injection
#
v---------v---------v---------v---------v---------v---------v---------v

class C; end

#
^---------^---------^---------^---------^---------^---------^---------^
# LibInject: end 'c' library injection
#
^---------^---------^---------^---------^---------^---------^---------^


class B; end

#
^---------^---------^---------^---------^---------^---------^---------^
# LibInject: end 'b' library injection
#
^---------^---------^---------^---------^---------^---------^---------^



class A; end

The code looks sort of messy, but now it's all in one file.

Anyway, it's a strange little script, and I'm curious as to whether
anybody else will get some use out of it. Let me know if you do!

F.

Cool! Does it handle things like this:

class Command
def create_option_parser
require 'optparse'
@parser = OptionParser.new
...
end
end

 

[Francis Hwang]

Cool! Does it handle things like this:

class Command
def create_option_parser
require 'optparse'
@parser = OptionParser.new
...
end
end

Nope. It really only handles the simplest, most common case, of the
require statement being on its own line outside of any class/method
definitions. I could try to make it smart enough to pull require
statements out of class & method definitions, but I'm not certain if
it'd be worth the effort. Are there actually many libs out there that
use requires this way? I personally try to avoid just 'cause I think it
makes the code look messier, but obviously that's sort of just a
personal style thing.

f.

 

[Austin Ziegler]

=20
Nope. It really only handles the simplest, most common case, of the
require statement being on its own line outside of any class/method
definitions. I could try to make it smart enough to pull require
statements out of class & method definitions, but I'm not certain if
it'd be worth the effort. Are there actually many libs out there that
use requires this way? I personally try to avoid just 'cause I think it
makes the code look messier, but obviously that's sort of just a
personal style thing.

I generally have all of my includes outside of methods, but there are
definite times when it is advantageous to do that in a method (e.g.,
when your code requires something 'big' and you want to delay loading
as long as possible). PDF::Writer *sort-of* does that with REXML, only
requiring it if on a Unix/Linux platform.

-austin
--=20
Austin Ziegler * (e-mail address removed)
* Alternate: (e-mail address removed)

 

[Jim Weirich]

Francis Hwang said:

Nope. It really only handles the simplest, most common case, of the
require statement being on its own line outside of any class/method
definitions. I could try to make it smart enough to pull require
statements out of class & method definitions, but I'm not certain if
it'd be worth the effort. Are there actually many libs out there that
use requires this way? I personally try to avoid just 'cause I think it
makes the code look messier, but obviously that's sort of just a
personal style thing.

My personal style (in general) is to put all the requires at the top leve=
l
at the beginning. However in RubyGems we use the above idiom often to
avoid requiring things that are not needed. The optparse is a bad exampl=
e
because it is always (eventually) needed, but things like the compression
libraries or the network http libraries are only required for certain
variations of the gem command. By requiring them inline at the place the=
y
are used means they are not loaded when they are not needed.

--=20
-- Jim Weirich (e-mail address removed) http://onestepback.org

 

[Francis Hwang]

Francis Hwang said:

My personal style (in general) is to put all the requires at the top level
at the beginning. However in RubyGems we use the above idiom often to
avoid requiring things that are not needed. The optparse is a bad example
because it is always (eventually) needed, but things like the compression
libraries or the network http libraries are only required for certain
variations of the gem command. By requiring them inline at the place they
are used means they are not loaded when they are not needed.

Thanks for the input, Jim & Austin. Okay, I'll put a fix for this in my
queue.

f.

 

[Joel VanderWerf]

--------------040300080408080201000206
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

I've just released the first version of LibInject, which is a
developer tool for injecting external dependencies into a Ruby file.
More specifically, it finds require statements that point to files
that aren't in the standard library, expands those files, and drops
the raw text into the original file.

http://libinject.rubyforge.org/

That's nice. Has anybody thought about a custom #require that looks in
the DATA of the main file, and if it finds a table of filename=>offsets
there, loads that file from between those offsets in DATA, else
falls back to Kernel#require ?

Anyway, here's a little proof of concept, called darb (DATA-Archived
RuBy script). Note that this solves a different problem than Francis's
LibInject: you provide list of files, and it generates an archive from
which those files can be loaded on demand. Francis's solution finds the
files for you, but they are all loaded automatically. (Also, note that
darb doesn't unpack the archive, as tar2rubyscript does. Everything
stays in the original archive file.)

What does darb mean? According to http://www.bartleby.com/68/2/1602.html:

darb is an Americanism probably nearly obsolete today, a slang word
from the 1920s meaning "something or someone very handsome, valuable,
attractive, or otherwise excellent."

No comment!

--
vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407




--------------040300080408080201000206
Content-Type: text/plain;
name="darb"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
filename="darb"

#!/usr/bin/env ruby

if ARGV.delete("-h")
puts <<-HELP; exit
Usage: darb main.rb feature1 feature2 feature3=path/to/file ... > script.rb

DARB == DATA-Archived RuBy script.

Writes a darb archive to standard out.

The main.rb code is executed when the archive is executed. Darb
simply inserts this code into the archive, after one initial line
to set up a customized #require. This main part of the archive can
be freely edited after generating the archive. After the __END__
line, the archive contains the #require implementation, an offset
table, and the library files themselves. Darb makes no effort to
separate the embedded files or make them easily extractable

Each of the features can be #require-d from the main.rb file, and
will be loaded as needed from the DATA segment of the archive. As
with Kernel#require, feature names are added to $LOADED_FEATURES
and only loaded once per feature name. #require falls back to the
original Kernel#require if the feature is not found in the archive.

Feature syntax:

- the .rb is optional

- other extensions are pemitted, but .so is not supported

- the 'feature3=path/to/file' argument means that darb should
copy the file from the given path into the archive, and load
it in response to "require 'feature3'".

Error messages are reported correctly in terms of the original
file and line.

Limitations: main.rb cannot have its own __DATA__ section.

Side effects: Darb defines the private instance method
Kernel#empty_binding, which simply returns a new binding with
no local variables.

AUTHOR : Joel VanderWerf, (e-mail address removed)
VERSION : 0.1
LICENSE : ruby license (credit appreciated)
HELP
end

main = ARGV.shift

puts %{eval(DATA.inject("") {|s,l| break s if /^__END__$/=~l; s<<l})}
puts
puts File.read(main)
puts "__END__"
puts <<EOS
Kernel.module_eval do
table = {
EOS

offset = 0
files = []

table_lines = ARGV.map do |feature|
if /=/ =~ feature
feature, file = feature.split("=").map {|f|f.strip}
else
feature, file = feature, feature
end

feature = feature.dup
feature.slice! /\.rb$/

file += ".rb" unless /\.\w+$/ =~ file
file = File.expand_path(file)
files << file

len = File.size(file)
line = %{ "#{feature}" => [#{offset}, #{len}]}
offset += len
line
end

puts table_lines.join(",\n")

puts <<EOS
}
start_pos = DATA.pos

meth = instance_methodrequire) # Thanks, batsman!

def empty_binding
binding
end
private :empty_binding

define_methodrequire) do |feature|
feature = feature.dup
feature.slice! /\.rb$/
k, (pos,len) = table.find {|k,v|k==feature}
if k
if $LOADED_FEATURES.include? feature
false
else
DATA.seek start_pos + pos
str = DATA.read(len)
eval str, empty_binding, feature
$LOADED_FEATURES << feature
true
end
else
meth.bind(self).call(feature)
end
end
end

__END__
EOS

files.each do |file|
puts File.read(file)
end

--------------040300080408080201000206
Content-Type: text/plain;
name="transcript.txt"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
filename="transcript.txt"

================== main.rb
require 'foo'
require 'foo'
require 'bar'
require 'zap'
require 'yaml'

p foo
p bar
p zap
p YAML

================== foo.rb
def foo
"foo"
end

puts "loading foo once!"

# add something to the foo.rb top-level binding -- see bar.rb
x = 1
================== bar.rb
def bar
"bar"
end

p self # ==> main

# bar.rb gets a different top-level binding from foo.rb,
# as it should
raise if defined?(x)
================== path/to/some-file.rb
def zap
"zap"
end
==================

$ darb main.rb foo.rb bar zap=path/to/some-file.rb >arc.rb
$ ruby arc.rb
loading foo once!
main
"foo"
"bar"
"zap"
YAML

--------------040300080408080201000206--

 

[Florian Frank]

My personal style (in general) is to put all the requires at the top level
at the beginning. However in RubyGems we use the above idiom often to
avoid requiring things that are not needed.

You can have both, if you do
autload :OptionParser, 'optparse'
at the beginning. 'optparse' is only loaded, when the OptionParser
constant is accessed and doesn't exist.

It's also possible to do something like

module AllMyStuff
module A
autoload :MyClass, 'all_my_stuff/a/my_class'
end
end

and

a = AllMyStuff::A::MyClass.new

for more complicated setups, because autoload is a method of Kernel and
Module.

 

[Jim Weirich]

You can have both, if you do
=C2=A0 autload :OptionParser, 'optparse'

Cool! I never stumbled across autoload before. I'm going to give it a t=
ry.

--=20
-- Jim Weirich (e-mail address removed) http://onestepback.org