Introducing myself and my interest in ruby

[Larry Felton Johnson]

This is just a note introducing myself to the list, and
explaining what I hope to get from the list.

I'm a software systems engineer at Georgia State University.
That basically translates to Unix Systems Administrator, though
my specialty is programming systems utilities. We run Solaris.

Most of my scripts are written in perl, I have one pretty large
scale project I wrote and maintain in php, and for the forseeable
future I'll probably still be primarily working in perl (most of
the people in our group know at least a little perl, so if I'm
not available the code is not inaccessable to my workgroup).

About a year ago I began looking at ruby, really liked it. It seemed
clean, easy to read, and had all the usual capabilities which attracted
me to perl to begin with. I didn't pursue it at the time because perl
was adequate for my needs. About two weeks ago I started dabbling in ruby
again, and have decided to write those utilities not likely to be
maintained by anyone else (in other words scripts for my own use)
in ruby so I'll get a more rounded experience in the language.

The typical script I'll be writing is the usual administrators utility
where one opens a file (including program output in that category) loops
through the file a line at a time doing regex matches, and does appropriate
stuff based on those matches.

I look forward to browsing this list and asking questions from time to
time. ruby strikes me as a very good scripting language.

Larry

 

[Ara.T.Howard]

This is just a note introducing myself to the list, and
explaining what I hope to get from the list.
howdy

I'm a software systems engineer at Georgia State University.
That basically translates to Unix Systems Administrator, though
my specialty is programming systems utilities. We run Solaris.

Most of my scripts are written in perl, I have one pretty large
scale project I wrote and maintain in php, and for the forseeable
future I'll probably still be primarily working in perl (most of
the people in our group know at least a little perl, so if I'm
not available the code is not inaccessable to my workgroup).

About a year ago I began looking at ruby, really liked it. It seemed
clean, easy to read, and had all the usual capabilities which attracted
me to perl to begin with. I didn't pursue it at the time because perl
was adequate for my needs. About two weeks ago I started dabbling in ruby
again, and have decided to write those utilities not likely to be
maintained by anyone else (in other words scripts for my own use)
in ruby so I'll get a more rounded experience in the language.

this is how i started. after three days i gave up perl altogether. after a
week i gave up c++ altogether. after a month or two i gave up c except to
write ruby extensions. after a few more months i gave up idl for image
processing and use classes i wrote using mmap that are simpler and faster.

The typical script I'll be writing is the usual administrators utility
where one opens a file (including program output in that category) loops
through the file a line at a time doing regex matches, and does appropriate
stuff based on those matches.

I look forward to browsing this list and asking questions from time to
time. ruby strikes me as a very good scripting language.

don't limit yourself to scripting tasks! i'm currenlty designing an entire
near-realtime satelite image processing system in ruby!

Larry

hope you'll find the group useful - it's full of really bright and helpful
people

-a
--
===============================================================================
| EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
| PHONE :: 303.497.6469
| ADDRESS :: E/GC2 325 Broadway, Boulder, CO 80305-3328
| URL :: http://www.ngdc.noaa.gov/stp/
| TRY :: for l in ruby perl;do $l -e "print \"\x3a\x2d\x29\x0a\"";done
===============================================================================

 

[Simon Strandgaard]

howdy

Yes, welcome to Ruby.


[snip]

this is how i started. after three days i gave up perl altogether. after a
week i gave up c++ altogether. after a month or two i gave up c except to
write ruby extensions. after a few more months i gave up idl for image
processing and use classes i wrote using mmap that are simpler and faster.

Almost the same story here. After being using C++ for many years, I now
use Ruby for everything. I think/hope I have become a better programmer,
because of several things:
* I have become more productive.. the compile-step is gone.
* Ruby is more readable than c++.. I can quicker navigate through the code.
* there is shorter from thought to implementation.. because of less typing.
* what is dirty in C++, can be implemented really nice in Ruby.
* irb.. instant feedback is great.
* test-first development.

 

[Hal Fulton]

don't limit yourself to scripting tasks! i'm currenlty designing an entire
near-realtime satelite image processing system in ruby!

Ara,

That's pretty cool. What can you tell us about this without
violating some kind of confidentiality policy?

Also, there's a "real world Ruby" page in the wiki somewhere.
This might make a nice addition.

Hal

 

[Ara.T.Howard]

Ara,

That's pretty cool. What can you tell us about this without violating some
kind of confidentiality policy?

ha! check out the url in my sig for general info...

i can't elaborate much at the moment as i'm slammed with a deadline but the
_design_ so far is:

- various job classes utilizing image processing classes that use guy's mmap
module (considering some stuff using narray). our algorithims are all
custom so no packaged lib would work. eg. we aren't doing an normal image
ops - we go through pixels and scanlines applying custom algorithims to
them to detect stuff (nighttime lights of the world). this is the
scientist's bag - i avoid it and consider only values 0-255 and issues of
massive i/o. typical files are 100mb each and explode into about 1.5 GB
out output. we currently have programs that _require_ 2.5mb of ram. it's
absolutely ridiculous..

- an entire system built around sge (sun grid engine) and seriously heavy
duty netapps and mass storage units - lots of data


essentially we process incoming satelite data and archive it. in addition
certain clients subscribe to have us do custom data processing/delivery for
them. the current system - isn't - it's a massive pile of scripts that call
scripts that call scripts...

1024.times{ puts "scripts that call scripts" }

....

i jest not.


the central problems are

- the processing is the 'science'. eg. it changes almost daily
- the volumes of data and cpu requirements present all sorts of challenges
- setting a 'subscription' (custom processing for a customer) requires a
ground-up approach. there is _zero_ reusable code currently
- the logging is non-existent
- we can't use a database (politics)
- our sysads are horrible

my current work has been

- figure out a persistence strategy. i wanted to use pstore for alot of it
but couldn't since it didn't support locking over nfs. see

http://raa.ruby-lang.org/list.rhtml?name=posixlock

- configurable work flows see

http://raa.ruby-lang.org/list.rhtml?name=flow

- controlling various external process (idl, c program, fortran, etc) in a
way that allows fine control over output see

http://raa.ruby-lang.org/list.rhtml?name=session

- quite a few in-house packages to deal with various data formats and do
various calculations see

http://www.codeforpeople.com/lib/ruby/dmsp/
http://www.codeforpeople.com/lib/ruby/envi/
http://www.codeforpeople.com/lib/ruby/solpos/
http://www.codeforpeople.com/lib/ruby/stpjob/

all the above is a literal snapshot of what i'm doing - as is normal around
here i haven't had much time for docs... sorry.


at the exact moment i'm working on a configurable set of jobs that generates
fire products using nighttime satelite imagery. it's basically a testbed for
what the near-realtime system might be but i am still very much in the
'enumerate the problem hotspots and check them off' phase. i still haven't
been able to check them all off so my 'design' ( a strong term ) is very much
in flux - suggestions (espcially in form of solved.tar.gz) are welcome! ;-)


a year ago i designed a bi-temporal database emulator (ruby classes) and a web
interface to it. that system was almost all ruby and is in production here
are noaa. see

http://www.fsl.noaa.gov/publications/forum/feb2003/2_03_MMeta.html
http://raa.ruby-lang.org/list.rhtml?name=btpgsql

and specifically

http://www.codeforpeople.com/lib/ruby/btpgsql/btpgsql-0.2.4/doc/

for quite a long discussion of the database part. the web interface is,
unfortunately, viewable only from the intranet. it's based on fastcgi and
postgresql though.

Also, there's a "real world Ruby" page in the wiki somewhere. This might
make a nice addition.

when time allows...

back to work...

cheers.

-a


--
===============================================================================
| EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
| PHONE :: 303.497.6469
| ADDRESS :: E/GC2 325 Broadway, Boulder, CO 80305-3328
| URL :: http://www.ngdc.noaa.gov/stp/
| TRY :: for l in ruby perl;do $l -e "print \"\x3a\x2d\x29\x0a\"";done
===============================================================================

 

[Robert]

ha! check out the url in my sig for general info...

i can't elaborate much at the moment as i'm slammed with a deadline but the
_design_ so far is:

- various job classes utilizing image processing classes that use guy's mmap
module (considering some stuff using narray). our algorithims are all
custom so no packaged lib would work. eg. we aren't doing an normal image
ops - we go through pixels and scanlines applying custom algorithims to
them to detect stuff (nighttime lights of the world). this is the
scientist's bag - i avoid it and consider only values 0-255 and issues of
massive i/o. typical files are 100mb each and explode into about 1.5 GB
out output. we currently have programs that _require_ 2.5mb of ram. it's
absolutely ridiculous..

- an entire system built around sge (sun grid engine) and seriously heavy
duty netapps and mass storage units - lots of data


essentially we process incoming satelite data and archive it. in addition
certain clients subscribe to have us do custom data processing/delivery for
them. the current system - isn't - it's a massive pile of scripts that call
scripts that call scripts...

1024.times{ puts "scripts that call scripts" }

...

i jest not.


the central problems are

- the processing is the 'science'. eg. it changes almost daily
- the volumes of data and cpu requirements present all sorts of challenges
- setting a 'subscription' (custom processing for a customer) requires a
ground-up approach. there is _zero_ reusable code currently
- the logging is non-existent
- we can't use a database (politics)
- our sysads are horrible

my current work has been

- figure out a persistence strategy. i wanted to use pstore for alot of it
but couldn't since it didn't support locking over nfs. see

http://raa.ruby-lang.org/list.rhtml?name=posixlock

- configurable work flows see

http://raa.ruby-lang.org/list.rhtml?name=flow

- controlling various external process (idl, c program, fortran, etc) in a
way that allows fine control over output see

http://raa.ruby-lang.org/list.rhtml?name=session

- quite a few in-house packages to deal with various data formats and do
various calculations see

http://www.codeforpeople.com/lib/ruby/dmsp/
http://www.codeforpeople.com/lib/ruby/envi/
http://www.codeforpeople.com/lib/ruby/solpos/
http://www.codeforpeople.com/lib/ruby/stpjob/

all the above is a literal snapshot of what i'm doing - as is normal around
here i haven't had much time for docs... sorry.


at the exact moment i'm working on a configurable set of jobs that generates
fire products using nighttime satelite imagery. it's basically a testbed for
what the near-realtime system might be but i am still very much in the
'enumerate the problem hotspots and check them off' phase. i still haven't
been able to check them all off so my 'design' ( a strong term ) is very much
in flux - suggestions (espcially in form of solved.tar.gz) are welcome! ;-)


a year ago i designed a bi-temporal database emulator (ruby classes) and a web
interface to it. that system was almost all ruby and is in production here
are noaa. see

http://www.fsl.noaa.gov/publications/forum/feb2003/2_03_MMeta.html
http://raa.ruby-lang.org/list.rhtml?name=btpgsql

and specifically

http://www.codeforpeople.com/lib/ruby/btpgsql/btpgsql-0.2.4/doc/

for quite a long discussion of the database part. the web interface is,
unfortunately, viewable only from the intranet. it's based on fastcgi and
postgresql though.





when time allows...

back to work...

cheers.

-a

I didn't understand half of it. But that is what makes it way cool!

 

[Glenn M. Lewis]

Hi Larry! Thanks for the introduction and welcome!
Perl was my favorite language for over a decade, with C and then later C++ being
the languages I developed in for runtime speed and for work requirements.

About three years ago I ran across an article in Dr. Dobb's Journal that immediately
hooked me on Ruby, and I have not turned back to Perl since that time.

Using SWIG (http://www.swig.org/), I have developed runtime-critical algorithms in
C++ and SWIG'd the interface so that I can totally access and control them from Ruby.

And just this last month, I have come across a compiled language that I believe will
completely replace C and C++ for me: The D Language (http://www.digitalmars.com/d/).
It even has a SWIG module for it so that I can easily access D from Ruby!

Now if I could only get our programmers at work to all switch from Perl and Tcl to Ruby
and from C++ to D !!!

-- Glenn

 

[Ara.T.Howard]

Hi Larry! Thanks for the introduction and welcome!
Perl was my favorite language for over a decade, with C and then later C++ being
the languages I developed in for runtime speed and for work requirements.

About three years ago I ran across an article in Dr. Dobb's Journal that immediately
hooked me on Ruby, and I have not turned back to Perl since that time.

Using SWIG (http://www.swig.org/), I have developed runtime-critical algorithms in
C++ and SWIG'd the interface so that I can totally access and control them from Ruby.

And just this last month, I have come across a compiled language that I believe will
completely replace C and C++ for me: The D Language (http://www.digitalmars.com/d/).
It even has a SWIG module for it so that I can easily access D from Ruby!

Now if I could only get our programmers at work to all switch from Perl and Tcl to Ruby
and from C++ to D !!!

wow - D looks very cool indeed. thanks for the pointer.

-a

-- Glenn

--
===============================================================================
| EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
| PHONE :: 303.497.6469
| ADDRESS :: E/GC2 325 Broadway, Boulder, CO 80305-3328
| URL :: http://www.ngdc.noaa.gov/stp/
| TRY :: for l in ruby perl;do $l -e "print \"\x3a\x2d\x29\x0a\"";done
===============================================================================

 

[Joel VanderWerf]

And just this last month, I have come across a compiled language that I
believe will
completely replace C and C++ for me: The D Language
(http://www.digitalmars.com/d/).
It even has a SWIG module for it so that I can easily access D from Ruby!

Very nice. LOTY 2004 anyone

How well does swigging D code into ruby work? I didn't see anything
about it on the swig site. Do you have a link?

The D links page has link for using swig "to generate code for D" but
that sounds like swigging C/C++ code into D...

http://www.digitalmars.com/d/dlinks.html

 

[Mark Hubbart]

And just this last month, I have come across a compiled language that
I believe will
completely replace C and C++ for me: The D Language
(http://www.digitalmars.com/d/).
It even has a SWIG module for it so that I can easily access D from
Ruby!

Which leads to a question that has been bouncing around in my mind for
several weeks now: What obstacles are there against being able to
compile Ruby? I would guess that eval() would be one, but other than
that, would it be feasible to create a compilable subset of Ruby?

--Mark

 

[gabriele renzi]

il Fri, 27 Feb 2004 17:59:07 +0900, Mark Hubbart <[email protected]> ha
scritto::

Which leads to a question that has been bouncing around in my mind for
several weeks now: What obstacles are there against being able to
compile Ruby? I would guess that eval() would be one, but other than
that, would it be feasible to create a compilable subset of Ruby?

IIRC lisp has eval() too, and there are plenty of compiler out ther

I bet you could write a compiler from ruby to at&t asm and let GAS do
the rest

 

[Robert Klemme]

il Fri, 27 Feb 2004 17:59:07 +0900, Mark Hubbart <[email protected]> ha
scritto::

The question is, how attractive would this subset be? I mean, there's a
lot of code around that relys on beeing able to use eval and friends.

IIRC lisp has eval() too, and there are plenty of compiler out ther

But not all of them compile to native. GNU CLISP compiles to byte code
AFAIK. Dunno what the native compiling compilers support, basically
you'll have to include the compiler in the runtime. And looking at the
high efforts done at Sun and MS to enhance their VM's, that shows that
it's not too easy to be able to come up with such a system that does work
correctly *and* is fast.

I bet you could write a compiler from ruby to at&t asm and let GAS do
the rest

Hm...

robert

 

[Mark Hubbart]

The question is, how attractive would this subset be? I mean, there's
a
lot of code around that relys on beeing able to use eval and friends.

I understand that. The attraction would not be that you can just
compile any previously written code, but that you can code something
*new* using the subset, and compile it. The compilable subset (call it
cRuby) wouldn't be competing with Ruby, but C, C++, etc. For most
things, one would still use good 'ol Ruby

What cRuby *would* be useful for:
- Writing shared libraries that people programming in other languages
could use
- Creating multi-platform apps that don't require a Ruby installation
- Avoiding C/C++ at all costs

Some projects simply *require* using a compilable language. I have not
yet found a compilable language that is easy and fun to use. cRuby
could fill the gap there.

But not all of them compile to native. GNU CLISP compiles to byte code
AFAIK.

From the CLISP manpage:
"Invoked with -c, the specified lisp files are compiled to a bytecode
that can be executed more efficiently."

From my (very limited) experience with it, it doesn't seem that lisp
has an eval statement, so much as lisp *is* an eval statement

Dunno what the native compiling compilers support, basically
you'll have to include the compiler in the runtime.

Which is the reason for the language subset - within the subset, there
would be no need for including the runtime; it could just be compiled
to machine code.

Back when I was using perl, I checked out their Perl->C project. It
could translate perl code into c code. Problem is, it was trying to
make it so that *any* valid perl code, including evals, could be
translated. A simple "hello world" would get translated into a few
thousand lines of C code.

--Mark

 

[Jim Weirich]

Mark Hubbart said:

Which is the reason for the language subset - within the subset, there
would be no need for including the runtime; it could just be compiled
to machine code.

I believe that this approach was used with Squeak Smalltalk. They wrote
the basic Smalltalk system in a subset of smalltalk that could be easily
translated to C. (See the "Brief History" section at
http://www.squeak.org/about/index.html).

 

[gabriele renzi]

il Sat, 28 Feb 2004 02:32:13 +0900, Mark Hubbart <[email protected]> ha
scritto::

What cRuby *would* be useful for:
- Writing shared libraries that people programming in other languages
could use
- Creating multi-platform apps that don't require a Ruby installation
- Avoiding C/C++ at all costs

first and last sounds much like pyrex[1].

[1]http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/version/Doc/About.html

 

[Mark Hubbart]

What cRuby *would* be useful for:
- Writing shared libraries that people programming in other languages
could use
- Creating multi-platform apps that don't require a Ruby installation
- Avoiding C/C++ at all costs

first and last sounds much like pyrex[1].

[1]http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/version/Doc/
About.html

Indeed, it does I wasn't thinking so much of translation to C, but
that would probably be much easier, and would be *much* better than not
having anything. And, it could pave the way for modifying a C compiler
to support the subset. (I hope I'm not being completely naive

If there *was* a compiler, then it would be perfect for projects like
Rubyx and ROS. It would make it so that more people would be able to
participate, since they could leverage their Ruby skills towards the
project.

--Mark

 

[Hal Fulton]

Indeed, it does I wasn't thinking so much of translation to C, but
that would probably be much easier, and would be *much* better than not
having anything. And, it could pave the way for modifying a C compiler
to support the subset. (I hope I'm not being completely naive

If there *was* a compiler, then it would be perfect for projects like
Rubyx and ROS. It would make it so that more people would be able to
participate, since they could leverage their Ruby skills towards the
project.

Mark,

I like the way you're thinking. I believe the principal reason a Ruby
compiler (or C translator) doesn't exist is simply that no one has
written one.

That being the case, go for it.

Of course, there are a few VM approaches out there (mostly experimental,
I think). There is also an rb2c which may still be in the RAA. (I
believe it was written for Ruby 1.4.x).

Of course, some aspects of such a project would be easier than others.



Cheers,
Hal

 

[Tobias Nurmiranta]

Hi,

From my (very limited) experience with it, it doesn't seem that lisp
has an eval statement, so much as lisp *is* an eval statement

Lisp has always had an eval statement, ever since 1960 when the first
paper on Lisp[1] was published.

CL-USER(6): (eval '(defun f (x) (+ x 1)))
F
CL-USER(7): (f 3)
4

[1] http://www-formal.stanford.edu/jmc/recursive.html
, Tobias

 

[Mark Hubbart]

From my (very limited) experience with it, it doesn't seem that lisp
has an eval statement, so much as lisp *is* an eval statement

Lisp has always had an eval statement, ever since 1960 when the first
paper on Lisp[1] was published.

actually, I was meaning that as a joke. I haven't ever gotten beyond
the very basics with lisp. To me, the "code is data is code" thing
makes me feel as if every statement is eval'ed... ("Which part's the
code, and which part's the data?" "Run it and figure it out.") Please
excuse a novice

--Mark

 

[Robert Klemme]

From my (very limited) experience with it, it doesn't seem that lisp
has an eval statement, so much as lisp *is* an eval statement

Lisp has always had an eval statement, ever since 1960 when the first
paper on Lisp[1] was published.

actually, I was meaning that as a joke. I haven't ever gotten beyond
the very basics with lisp. To me, the "code is data is code" thing
makes me feel as if every statement is eval'ed... ("Which part's the
code, and which part's the data?" "Run it and figure it out.") Please
excuse a novice

And because Lispers felt not confortable with the simple distinction of
code, data and eval they added macros.

robert