[ANN] arrayfields-3.0.0

Discussion in 'Ruby' started by Ara.T.Howard, Jun 29, 2004.

  1. Ara.T.Howard

    Ara.T.Howard Guest

    URLS:

    http://raa.ruby-lang.org/project/arrayfields/
    http://www.codeforpeople.com/lib/ruby/arrayfields/

    SYNOPSIS

    allow keyword access to arrays:

    require 'arrayfields'

    fields = 'name', 'age'
    row = [ 'bob', 30 ]

    row.fields = fields

    row[ 'name' ] #=> 'bob'
    row.indices 'name', 'age' #=> [ 'bob', 30 ]

    assigning to un-named fields appends:

    stack = []
    stack.fields = %w(zero one)
    stack['zero'] = 'zero'
    stack['one'] = 'one'
    stack #=> [ 'zero', 'one' ]

    *very* useful for database work

    relation = pgconn.query sql
    relation.size #=> 65536

    # yikes! do we really want to re-construct a hash for for each tuple when
    # we already have Arrays?

    fields = %w(ssn name position)
    table.each{|tuple| tuple.fields = fields}

    table[34578]['ssn'] #=> 574865032

    LIST OF OVERRIDDEN METHODS

    Array#[]
    Array#[]=
    Array#at
    Array#delete_at
    Array#fill
    Array#values_at
    Array#indices
    Array#indexes
    Array#slice
    Array#slice!

    LIST OF NEW Array METHODS

    Array#fields=
    Array#each_with_field

    DOCS/USAGE/SAMPLE

    lib/arrayfields.rb
    test/arrayfields.rb

    AUTHOR



    -a
    --
    ===============================================================================
    | EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
    | PHONE :: 303.497.6469
    | A flower falls, even though we love it;
    | and a weed grows, even though we do not love it.
    | --Dogen
    ===============================================================================
     
    Ara.T.Howard, Jun 29, 2004
    #1
    1. Advertising

  2. Ara.T.Howard

    Joao Pedrosa Guest

    Hi,

    Cool, I'm gonna give it a whirl.

    Thanks a lot,
    Joao

    --- "Ara.T.Howard" <> wrote:
    > URLS:
    >
    > http://raa.ruby-lang.org/project/arrayfields/
    >
    > http://www.codeforpeople.com/lib/ruby/arrayfields/
    >
    > SYNOPSIS
    >
    > allow keyword access to arrays:
    >
    > require 'arrayfields'
    >
    > fields = 'name', 'age'
    > row = [ 'bob', 30 ]
    >
    > row.fields = fields
    >
    > row[ 'name' ] #=> 'bob'
    > row.indices 'name', 'age' #=> [ 'bob', 30 ]





    __________________________________
    Do you Yahoo!?
    Yahoo! Mail Address AutoComplete - You start. We finish.
    http://promotions.yahoo.com/new_mail
     
    Joao Pedrosa, Jun 29, 2004
    #2
    1. Advertising

  3. Ara.T.Howard wrote:
    > URLS:
    >
    > http://raa.ruby-lang.org/project/arrayfields/
    > http://www.codeforpeople.com/lib/ruby/arrayfields/
    >
    > SYNOPSIS
    >
    > allow keyword access to arrays:
    >
    > require 'arrayfields'
    >
    > fields = 'name', 'age'
    > row = [ 'bob', 30 ]
    >
    > row.fields = fields
    >
    > row[ 'name' ] #=> 'bob'
    > row.indices 'name', 'age' #=> [ 'bob', 30 ]


    Hm, sorta like something I've been using:

    ---
    module AccessibleIndex
    def index_accessor(h)
    h.each do |sym, idx|
    define_method sym do
    self[idx]
    end
    define_method "#{sym}=" do |val|
    self[idx] = val
    end
    end
    end
    end

    a = [0,1,2,3,4,5]
    class << a
    extend AccessibleIndex
    index_accessor :x => 3
    end

    a.x = "three"
    p a # ==> [0, 1, 2, "three", 4, 5]
    ---

    But this approach doesn't help with values_at and the like.
     
    Joel VanderWerf, Jun 29, 2004
    #3
  4. Ara.T.Howard

    Ara.T.Howard Guest

    On Wed, 30 Jun 2004, Joel VanderWerf wrote:

    > ---
    > module AccessibleIndex
    > def index_accessor(h)
    > h.each do |sym, idx|
    > define_method sym do
    > self[idx]
    > end
    > define_method "#{sym}=" do |val|
    > self[idx] = val
    > end
    > end
    > end
    > end
    >
    > a = [0,1,2,3,4,5]
    > class << a
    > extend AccessibleIndex
    > index_accessor :x => 3
    > end
    >
    > a.x = "three"
    > p a # ==> [0, 1, 2, "three", 4, 5]
    > ---
    >
    > But this approach doesn't help with values_at and the like.


    cool. your way is perhaps safer. ;-)

    the reason i wrote this in the first place was that i was doing alot of
    postgresql work and it was returning these huge result sets that i would then
    write code that looked like:

    if tuple[17] =~ pat or tuple[9].to_i < 42
    ...
    end

    and i was reading my own code thinking - huh? so then i started doing:

    res = pgconn.exec sql

    tuples = res.result
    fields = res.fields

    tuples =
    tuples.collect do |tuple|
    h = {}
    fields.each do |field|
    h[field] = tuple.shift
    end
    h
    end

    this is NOT good if tuple.size == 42000!! finally i wrote the arrayfields
    module. it still requires work:

    res = pgconn.exec sql

    tuples = res.result
    fields = res.fields

    tuples.each{|t| t.fields = fields}


    but now all 42000 tuples SHARE a copy of fields for doing their lookups - i
    still have to iterate over all of em - but i don't need to create any new
    objects and my code new reads like:

    if tuple['name'] =~ pat or tuple['age'].to_i < 42
    ...
    end

    which is a lot nicer.

    in any case it's just one of those things that i ended up using alot. i'd
    like to re-write it in C but don't have the time right now...


    cheers.


    -a
    --
    ===============================================================================
    | EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
    | PHONE :: 303.497.6469
    | A flower falls, even though we love it;
    | and a weed grows, even though we do not love it.
    | --Dogen
    ===============================================================================
     
    Ara.T.Howard, Jun 29, 2004
    #4
  5. Ara.T.Howard

    Ben Giddings Guest

    Ara.T.Howard wrote:
    > if tuple[17] =~ pat or tuple[9].to_i < 42
    > ...
    > end

    ...
    > if tuple['name'] =~ pat or tuple['age'].to_i < 42
    > ...
    > end


    Is there any reason you didn't just use either constants or variables like:

    if tuple[NAME] =~ pat or tuple[AGE].to_i < 42
    ...
    end

    or if your indices weren't constant, set them from a DB query and then do:

    if tuple[name] =~ pat or tuple[age].to_i < 42
    ...
    end

    arrayfields looks great, and makes things much easier, but it just seems
    odd to me that you'd approach it the way you did.

    Ben
     
    Ben Giddings, Jun 29, 2004
    #5
  6. Ara.T.Howard

    Ara.T.Howard Guest

    On Wed, 30 Jun 2004, Ben Giddings wrote:

    > Is there any reason you didn't just use either constants or variables like:


    yes.

    constants:

    can't be reused - obviously - so

    tuple_a[NAME]

    tuple_b[NAME]

    won't work of NAME is different for both

    this also doesn't work when the relation is modified. in general there is
    no guaruntee on the order of fields returned to this is never a good
    option.


    variables:

    i like to do this

    name, age, ssn = tuple.values_at 'name', 'age', 'ssn'

    so i already used up those var names - not that i couldn't use 'name_idx',
    etc. but this starts getting messy fast if you are dealing for four or
    five tables with 10-20 fields each.

    lastly there is one nice feature of using the arrayfields method: say you are
    generating a report from a relation using something like you are suggesting

    res = pgconn.exec sql
    tuples, fields = res.tuples, res.fields

    name, age, ssn = %w(name age ssn).map{|f| fields.index f}

    YAML::dump(
    tuples.map do |t|
    Hash[
    'name' => tuple[name],
    'age' => tuple[age],
    'ssn' => tuple[ssn],
    ]
    end
    )

    and later the relation is modified to now have a new column you'll need to
    modify the code. whereas if you use the fields this code will work even when
    the table is modified

    res = pgconn.exec sql
    tuples, fields = res.tuples, res.fields

    YAML::dump(
    tuples.map do |t|
    t.fields = fields
    fields.inject({}){|h,f| h[f] = t[f]; h}
    end
    )

    of course you could write the code to have this feature either way - my point
    is just that when you extract the indexes into named vars you limit the way
    your code will work as the database add/drops fields... i was originally also
    doing alot of this sort of thing:


    def where_clause tuple, op = ' or '
    clauses =
    tuple.fields.map do |field|
    "#{ field } = '#{ tuple[field] }'"
    end

    clauses.join op
    end

    which just started to get painful when the tuple didn't know it's own field
    names... you can imagine doing alot of this stuff if you have to map fields
    to var names and pass this mapping into each sql generation method...


    cheers.

    -a
    --
    ===============================================================================
    | EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
    | PHONE :: 303.497.6469
    | A flower falls, even though we love it;
    | and a weed grows, even though we do not love it.
    | --Dogen
    ===============================================================================
     
    Ara.T.Howard, Jun 29, 2004
    #6
  7. Ara.T.Howard

    Kirk Haines Guest

    On Wed, 30 Jun 2004 06:37:56 +0900, Ara.T.Howard wrote

    > and i was reading my own code thinking - huh? so then i started doing:
    >
    > res = pgconn.exec sql
    >
    > tuples = res.result
    > fields = res.fields
    >
    > tuples =
    > tuples.collect do |tuple|
    > h = {}
    > fields.each do |field|
    > h[field] = tuple.shift
    > end
    > h
    > end
    >
    > this is NOT good if tuple.size == 42000!! finally i wrote the arrayfields
    > module. it still requires work:
    >
    > res = pgconn.exec sql
    >
    > tuples = res.result
    > fields = res.fields
    >
    > tuples.each{|t| t.fields = fields}
    >
    > but now all 42000 tuples SHARE a copy of fields for doing their
    > lookups - i still have to iterate over all of em - but i don't need
    > to create any new objects and my code new reads like:
    >
    > if tuple['name'] =~ pat or tuple['age'].to_i < 42
    > ...
    > end
    >
    > which is a lot nicer.


    Okay, admittedly, sometimes I'm too dense for my own good, but I'm
    confused. It looks to me like, internally, each array is using a hash to
    keep track of the fields to indices mapping, and that hash is not shared
    amongst arrays with the same set of fields. So you end up using more memory
    using arrayfields for something like database result sets than if you just
    used hashes.

    I put together a few simple little test programs to try it out, using
    arrayfields and using hashes to store similar sets of data, and in practice
    it looks like arrayfields uses almost 2x the amount of RAM for a given data
    set, as well. Am I missing something, here?


    Thanks,

    Kirk Haines
     
    Kirk Haines, Jun 29, 2004
    #7
  8. Ara.T.Howard

    Ara.T.Howard Guest

    On Wed, 30 Jun 2004, Kirk Haines wrote:

    > Okay, admittedly, sometimes I'm too dense for my own good, but I'm
    > confused. It looks to me like, internally, each array is using a hash to
    > keep track of the fields to indices mapping, and that hash is not shared
    > amongst arrays with the same set of fields. So you end up using more memory
    > using arrayfields for something like database result sets than if you just
    > used hashes.
    >
    > I put together a few simple little test programs to try it out, using
    > arrayfields and using hashes to store similar sets of data, and in practice
    > it looks like arrayfields uses almost 2x the amount of RAM for a given data
    > set, as well. Am I missing something, here?
    >
    >
    > Thanks,
    >
    > Kirk Haines


    not as dense as i am ;-) you are correct about the ram. i 'fixed' myself
    into this when i was making some updates for testing - let me have a look at
    it and post a fix....

    -a
    --
    ===============================================================================
    | EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
    | PHONE :: 303.497.6469
    | A flower falls, even though we love it;
    | and a weed grows, even though we do not love it.
    | --Dogen
    ===============================================================================
     
    Ara.T.Howard, Jun 30, 2004
    #8
  9. Ara.T.Howard

    Guest

    Hi,

    At Wed, 30 Jun 2004 05:02:58 +0900,
    Ara.T.Howard wrote in [ruby-talk:104864]:
    > http://raa.ruby-lang.org/project/arrayfields/
    > http://www.codeforpeople.com/lib/ruby/arrayfields/


    $ ruby-1.8 -r arrayfields.rb -e '[][:x]'
    -e:1:in `[]': Symbol as array index (TypeError)
    from -e:1
    $ grep VERSION arrayfields.rb
    VERSION = '3.1.0'

    `and' and `or' operators have same precedence.

    BTW, you couldn't use Struct?

    tuples =
    fields = Struct.new(nil, res.fields)

    tuples = res.collect {|tuple| fields.new(*tuple)}

    --
    Nobu Nakada
     
    , Jun 30, 2004
    #9
  10. Ara.T.Howard

    Guest

    Hi,

    At Wed, 30 Jun 2004 10:52:28 +0900,
    wrote in [ruby-talk:104888]:
    > $ ruby-1.8 -r arrayfields.rb -e '[][:x]'
    > -e:1:in `[]': Symbol as array index (TypeError)
    > from -e:1


    Sorry, this is not a problem, I was a stupid.

    --
    Nobu Nakada
     
    , Jun 30, 2004
    #10
  11. Ara.T.Howard

    Ara.T.Howard Guest

    On Wed, 30 Jun 2004 wrote:

    > Hi,
    >
    > At Wed, 30 Jun 2004 05:02:58 +0900,
    > Ara.T.Howard wrote in [ruby-talk:104864]:
    >> http://raa.ruby-lang.org/project/arrayfields/
    >> http://www.codeforpeople.com/lib/ruby/arrayfields/

    >



    > $ ruby-1.8 -r arrayfields.rb -e '[][:x]'
    > -e:1:in `[]': Symbol as array index (TypeError)
    > from -e:1
    > $ grep VERSION arrayfields.rb
    > VERSION = '3.1.0'
    >
    > `and' and `or' operators have same precedence.


    the instance only is extended, and only then after instance.fields= has been
    called. eg.

    ~ > ruby -r arrayfields-3.0.0 -e 'a = [42]; a.fields = [:answer]; p a[:answer]'
    42

    ~ > ruby -r arrayfields-3.1.0 -e 'a = [42]; a.fields = [:answer]; p a[:answer]'
    42

    so, in the normal case only the method Array#fields= is added when you

    require 'arrayfields'

    i didn't want to polute the namespace any more than needed.


    > BTW, you couldn't use Struct?
    >
    > tuples =
    > fields = Struct.new(nil, res.fields)
    >
    > tuples = res.collect {|tuple| fields.new(*tuple)}


    i suppose i could... but there the would have a lot of object create on large
    result sets no?

    regards.

    -a
    --
    ===============================================================================
    | EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
    | PHONE :: 303.497.6469
    | A flower falls, even though we love it;
    | and a weed grows, even though we do not love it.
    | --Dogen
    ===============================================================================
     
    Ara.T.Howard, Jun 30, 2004
    #11
  12. Ara.T.Howard

    Guest

    Hi,

    At Wed, 30 Jun 2004 11:22:54 +0900,
    Ara.T.Howard wrote in [ruby-talk:104892]:
    > > BTW, you couldn't use Struct?
    > >
    > > tuples =
    > > fields = Struct.new(nil, res.fields)
    > >
    > > tuples = res.collect {|tuple| fields.new(*tuple)}

    >
    > i suppose i could... but there the would have a lot of object create on large
    > result sets no?


    I don't like to hold such huge dataset in one time, so I'd use
    PGresult#each instead as possible.

    --
    Nobu Nakada
     
    , Jun 30, 2004
    #12
  13. Ara.T.Howard

    Sean O'Dell Guest

    Ara, any chance of getting arrayfields to not modify the Array methods but
    instead simply add an accessor to Array which returns an object that
    encapsulates this new functionality?

    So, instead of saying:

    row = ['bob', 30]
    row.fields = ['name', 'age']
    row['name'] #=> 'bob'

    you would say something like:

    row = ['bob', 30]
    row.arrayfields.fields = ['name', 'age']
    row.arrayfields['name'] #=> 'bob'

    My thinking is that if arrayfields functionality were contained in its own
    object that linked back to a parent array, you could create arrayfield
    objects and attach them to other "array-like" objects, such as database rows
    and so on.

    But also, it might prevent problems that arise later from taking over Array's
    methods where people want to use both arrayfields, and regular Array
    functionality.

    Just an idea.

    Sean O'Dell
     
    Sean O'Dell, Jun 30, 2004
    #13
  14. Ara.T.Howard

    Ara.T.Howard Guest

    On Thu, 1 Jul 2004, Sean O'Dell wrote:

    > Ara, any chance of getting arrayfields to not modify the Array methods but
    > instead simply add an accessor to Array which returns an object that
    > encapsulates this new functionality?
    >
    > So, instead of saying:
    >
    > row = ['bob', 30]
    > row.fields = ['name', 'age']
    > row['name'] #=> 'bob'
    >
    > you would say something like:
    >
    > row = ['bob', 30]
    > row.arrayfields.fields = ['name', 'age']
    > row.arrayfields['name'] #=> 'bob'
    >
    > My thinking is that if arrayfields functionality were contained in its own
    > object that linked back to a parent array, you could create arrayfield
    > objects and attach them to other "array-like" objects, such as database rows
    > and so on.
    >
    > But also, it might prevent problems that arise later from taking over Array's
    > methods where people want to use both arrayfields, and regular Array
    > functionality.


    well - theoretically you lose zero Array functionality since ArrayFields
    methods are used if, and only if, the idx paramter is a String or Symbol which
    would never be done with a normal Array.

    that being said:

    jib:~/shared > cat a.rb
    require 'arrayfields'

    class FieldedArray
    attr :fieldset
    def initialize fields, array
    @a = array
    self.fields = fields
    end
    def method_missing(meth, *args, &block)
    @a.send(meth, *args, &block)
    end
    def fields= fields
    extend ArrayFields unless defined? @fieldset

    @fieldset =
    if ArrayFields::FieldSet === fields
    fields
    else
    ArrayFields::FieldSet.new fields
    end
    end
    attr_reader :fieldset
    def fields
    @fieldset and @fieldset.fields
    end
    end

    fa = FieldedArray.new %w(zero one two), [0, 1, 2]
    fa.each_with_field{|e, f| puts "#{ f } : #{ e }"}

    jib:~/shared > ruby a.rb
    zero : 0
    one : 1
    two : 2


    i can't bloody believe that worked - go matz!

    what'ya think of adding the above to 3.3.0 (along with gem spec)?

    -a
    --
    ===============================================================================
    | EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
    | PHONE :: 303.497.6469
    | A flower falls, even though we love it;
    | and a weed grows, even though we do not love it.
    | --Dogen
    ===============================================================================
     
    Ara.T.Howard, Jun 30, 2004
    #14
  15. Ara.T.Howard

    Sean O'Dell Guest

    On Wednesday 30 June 2004 12:32, Ara.T.Howard wrote:
    > On Thu, 1 Jul 2004, Sean O'Dell wrote:
    > > Ara, any chance of getting arrayfields to not modify the Array methods
    > > but instead simply add an accessor to Array which returns an object that
    > > encapsulates this new functionality?
    > >
    > > So, instead of saying:
    > >
    > > row = ['bob', 30]
    > > row.fields = ['name', 'age']
    > > row['name'] #=> 'bob'
    > >
    > > you would say something like:
    > >
    > > row = ['bob', 30]
    > > row.arrayfields.fields = ['name', 'age']
    > > row.arrayfields['name'] #=> 'bob'
    > >
    > > My thinking is that if arrayfields functionality were contained in its
    > > own object that linked back to a parent array, you could create
    > > arrayfield objects and attach them to other "array-like" objects, such as
    > > database rows and so on.
    > >
    > > But also, it might prevent problems that arise later from taking over
    > > Array's methods where people want to use both arrayfields, and regular
    > > Array functionality.

    >
    > well - theoretically you lose zero Array functionality since ArrayFields
    > methods are used if, and only if, the idx paramter is a String or Symbol
    > which would never be done with a normal Array.
    >
    > that being said:
    >
    > jib:~/shared > cat a.rb
    > require 'arrayfields'
    >
    > class FieldedArray
    > attr :fieldset
    > def initialize fields, array
    > @a = array
    > self.fields = fields
    > end
    > def method_missing(meth, *args, &block)
    > @a.send(meth, *args, &block)
    > end
    > def fields= fields
    > extend ArrayFields unless defined? @fieldset
    >
    > @fieldset =
    > if ArrayFields::FieldSet === fields
    > fields
    > else
    > ArrayFields::FieldSet.new fields
    > end
    > end
    > attr_reader :fieldset
    > def fields
    > @fieldset and @fieldset.fields
    > end
    > end
    >
    > fa = FieldedArray.new %w(zero one two), [0, 1, 2]
    > fa.each_with_field{|e, f| puts "#{ f } : #{ e }"}
    >
    > jib:~/shared > ruby a.rb
    > zero : 0
    > one : 1
    > two : 2
    >
    >
    > i can't bloody believe that worked - go matz!
    >
    > what'ya think of adding the above to 3.3.0 (along with gem spec)?


    I like your solution above where none of the arrayfields code goes directly
    into the Array class. Benefits: no overhead for Arrays that don't use the
    arrayfields feature, and the ability to work with other objects that aren't
    Array's but which have array-like functionality.

    Sean O'Dell
     
    Sean O'Dell, Jun 30, 2004
    #15
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Ara.T.Howard
    Replies:
    2
    Views:
    151
    Ara.T.Howard
    Jun 30, 2004
  2. Ara.T.Howard

    [ANN] arrayfields-3.3.0

    Ara.T.Howard, Jul 1, 2004, in forum: Ruby
    Replies:
    3
    Views:
    119
    Ara.T.Howard
    Jul 1, 2004
  3. Ara.T.Howard

    [ANN] arrayfields-3.4.0

    Ara.T.Howard, Sep 4, 2004, in forum: Ruby
    Replies:
    3
    Views:
    104
    Joao Pedrosa
    Sep 6, 2004
  4. Replies:
    0
    Views:
    108
  5. Replies:
    0
    Views:
    84
Loading...

Share This Page