Array of Hashes in an array of hashes - Complicated!

[Matt Brooks]

I have an unique problem that I can't solve. I am sorry this is long,
but the details are needed to solve it I think.

The reason I have to use an array of hashes that sometimes has an array
of a hashes in it is as follows.

I have a variably repeating block of binary data with variable repeating
fields within it; which I was previously simply decoding using
bit-structs and also using unpack to get variably repeating fields, and
then just printing the data to the screen. Now, I need to save all
those values into hash keys, so they are accessible later, after they
have been printed to the screen.


**Problem: The entire variable length data structure repeats a variable
amount of times(known before-hand in a variable at the beginning),
inside this data structure are some variables occurring variable amounts
of times(also known right before the repeating of the fields occurs).

For a simple outline of variables I have that need to be stored instead
of simply displaying/printing them as I parse the incoming data.... of
course this masks some other complications I will address later, but the
short example should suffice.

num_signals #how many times this entire structure repeats(varies
each time)
signal_id #regular variable easily saved in hash
signal_freq #regular variable easily saved in hash
num_times_next_two_repeat #next two fields repeat as a pair this #
times
parameter_name #again this repeats with the below as a pair N times
parameter_value #again this repeats with the above as a pair N times
signal_bandwidth #regular variable easily saved in hash


***Quick summary of how I was printing/displaying this complicated mess
to the screen: All these variables are available to me using bit-struct
fields and then using unpack to loop through the cases where there are a
variable number of times a pair of variables repeat within the data. I
previously would just print the regualar fields using the inspect method
of bitstruct to print each individual variable, but is easily accessible
using this: <nameofbitstructobject>.nameofbitstructfield

Then I would display an appended string of the variable fields by using
unpack to unpack the parameter_name and parameter_value and append it to
a string for displaying later, then unpack the next parameter_name and
parameter_value and append it to the same string, etc... then finally
after all variable repeating fields were done, I would display the
string to the screen. This worked fine for just displaying to the user,
now I need them saved uniquely(they repeat and have the same name, can't
have two hash keys with same name, ie. part of problem).



So, now I want each value to be in a hash, and each key name has to be
unique. I WISH I WISH I could simply increment a hash key symbol name
(I don't think it is possible). I wish I could just add to my hash as I
have been doing for the other non-repeating fields and just increment
the hash key names like parameter_name0 and parameter_value0, then
parameter_name1 and parameter_value1. But since this occurs a variable
amount of times I can't hardcode the variable names.

Since I don't think that is possible, I thought I could create, another
array that holds hashes of the 2 repeating fields, ie. array.length
would equal num_times_next_two_repeat.


I have previously created the top array called, signal_data:

signal_data = Array.new(num_signals, 0)

Then before each new iteration for each signal the following runs:

signal_data[signal_index] = Hash.new

Therefore to save the first repeating field "I think" it would be
something like

signal_data[signal_index][:num_times_repeated_array][repeated_field_index][arameter_name] = parameter_value

****Couple problems That I don't understand how to fix:

1st of all, should this work at all?

How do I declare the second array?
Is it like this?

:num_times_repeated_array = Array.new(num_times_next_two_repeat, 0)

I am confused about the colon used in the hash key and the colon at the
beginning of the name of the array itself, does the name of the array
have the colon?


Would you be able to get to a 2nd repetition of the repeated field like
this?

puts signal_data[signal_index]['num_times_repeated_array'][1]['parameter_name']

*****NEXT PROBLEM if the above step is able to work:
So if that indeed that accesses that repeated field, here is the next
problem, how can I get that repeated field to be the hash key of the
next repeated field, which would be the hash value of that key? Crazy I
know!!!

I want this embedded second array to only hold one key and value pair
per array index, ie. the parameter_name to be the hash key and the
parameter_value to be the hash value for each index in the array. And
the length of that array of course would be = to that variable
num_times_next_two_repeat.




Thank you very much for whatever help you can offer, I know it is long.
But I can not figure it out. I'll be waiting, wish I could say
patiently but probably not, haha.
-Matt

 

[Matt Brooks]

Therefore to save the first repeating field "I think" it would be

something like

signal_data[signal_index][:num_times_repeated_array][repeated_field_index][arameter_name] = parameter_value

Two things, I meant for the above to save parameter_name for now, later
on, i want it to save the parameter_value to a hash key of the
parameter_name...

signal_data[signal_index][:num_times_repeated_array][repeated_field_index][arameter_name] = parameter_name

Also I guess I forgot to declare the hashes for the second array:

num_times_repeated_array[repeated_field_index] = Hash.new

Then I could save stuff to it???

signal_data[signal_index][:num_times_repeated_array][repeated_field_index][arameter_name] = parameter_name

 

[John W Higgins]

[Note: parts of this message were removed to make it a legal post.]

Morning Matt,

I have an unique problem that I can't solve. I am sorry this is long,
but the details are needed to solve it I think.

The reason I have to use an array of hashes that sometimes has an array
of a hashes in it is as follows.

I don't have any specific solution here for you other then to suggest that
you might be very much limiting yourself by thinking only in terms of Arrays
and Hashes. This sounds like something where a nice object would be more
appropriate. It might take more "legwork" to develop a good layout here but
it would seem that you would be much better off.

num_signals #how many times this entire structure repeats(varies
each time)
signal_id #regular variable easily saved in hash
signal_freq #regular variable easily saved in hash
num_times_next_two_repeat #next two fields repeat as a pair this #
times
parameter_name #again this repeats with the below as a pair N times
parameter_value #again this repeats with the above as a pair N times
signal_bandwidth #regular variable easily saved in hash

Something like a Signal object with id, freq, and bandwidth as
attr_accessors along with parameters as a hash stored within the object
might be a nice starting point.

You could then have an array of Signal objects that would store the
individual signals.

John

 

[Matt Brooks]

Hi John,
Thanks for the input, how might I create a variable number of objects
with different object names though?

if the following while loop happens the same object is created over and
over again. I loose say the other 30 signal objects i have made. As I
was saying before how would you increment a variable name or object name
in this implementation?

while index < num_signals
sig_1 = Signal.new
....
....
end


Also, I would need a way to overwrite and kill all the objects every 10
seconds or so, when all the signals update with different information.
Is that possible?

How might I keep track of all the signal objects created and all their
unique names and then kill them, so that I can then make a variable
number of new signal objects that contain a different number of
variables possibly, and definitely different data.

Thanks!
Matt

 

[John W Higgins]

[Note: parts of this message were removed to make it a legal post.]

Matt,

Hi John,
...


while index < num_signals
sig_1 = Signal.new
....
....
end

I would create an array of the objects. It's not a matter of throwing away
arrays or hashes but better organization of your data into objects rather
then layers of data in hashes and array will just make things easier.

For example

sig_array = Array.new
while index < num_signals
sig = Signal.new
sig.id = ...
...
sig_array << sig
end

Also, I would need a way to overwrite and kill all the objects every 10
seconds or so, when all the signals update with different information.
Is that possible?

Just clear the array when you want to drop everything

sig_array.clear

John

 

[7stud --]

Hi John,
Thanks for the input, how might I create a variable number of objects
with different object names though?

if the following while loop happens the same object is created over and
over again. I loose say the other 30 signal objects i have made. As I
was saying before how would you increment a variable name or object name
in this implementation?

while index < num_signals
sig_1 = Signal.new
....
....
end

You wouldn't.

signals = []

while index < num_signals
signals << Signal.new
end

Also, I would need a way to overwrite and kill all the objects every 10
seconds or so, when all the signals update with different information.
Is that possible?

signals = []

 

[Aldric Giacomoni]

+1 on object creation

There's a Ruby tutorial somewhere on the interblag about creating a
mini-adventure game. It begins like this:

def Thing
attr_accessor :name, :description
end

And everything starts from there.

def Weapon < Thing
end

def Sword < Weapon
end

etc etc. You probably don't need that much breaking down, but it seems
you do need to break it down some ;-)
The basic way to think about it is : if I can't explain it in three
sentences or less, I have to break it down.

If you REALLY REALLY REALLY wanted to create some kind of iterating
name, you could always do things like
iterator = 0
array = []
while iterator < 30
array << "a#{iterator}".to_sym
iterator += 1
end

But that's really dirty...

 

[David A. Black]

Hi --

+1 on object creation

There's a Ruby tutorial somewhere on the interblag about creating a
mini-adventure game. It begins like this:

def Thing
attr_accessor :name, :description
end

And everything starts from there.

def Weapon < Thing
end

def Sword < Weapon
end

etc etc. You probably don't need that much breaking down, but it seems
you do need to break it down some ;-)
The basic way to think about it is : if I can't explain it in three
sentences or less, I have to break it down.

If you REALLY REALLY REALLY wanted to create some kind of iterating
name, you could always do things like
iterator = 0
array = []
while iterator < 30
array << "a#{iterator}".to_sym
iterator += 1
end

But that's really dirty...

array = (0...30).map {|i| :"a#{i}" }

(Probably irrelevant to the thread but I couldn't resist


David

 

[Matt Brooks]

Thank you for everyone's input, I am thinking about it all right now,
and hope to have a combined solution tomorrow.

I have to figure out how to make a Signal Object, i can't see how I am
going to get around having this messed up hash problem even if I make a
signal object. because of the repeating fields within the data.

I think having objects only changes the fact of instead of having an
array of hashes at my top level, i have an array of objects that have
hashes inside of them anyway... I have hundreds of variables for each
signal, and I can't make this signal object have 200 local variables, I
think it would be dirty to type
attr_reader: for all the hundreds variables...

Plus then only some of them get defined each time...

Am I thinking about this correctly? i have seen that tutorial, i think
actually in my book here by Peter Cooper or at least something similar,
although I am not seeing how it relates fully.

Thanks!!!
Matt

 

[Rob Biedenharn]

Thank you for everyone's input, I am thinking about it all right now,
and hope to have a combined solution tomorrow.

I have to figure out how to make a Signal Object, i can't see how I am
going to get around having this messed up hash problem even if I
make a
signal object. because of the repeating fields within the data.

I think having objects only changes the fact of instead of having an
array of hashes at my top level, i have an array of objects that have
hashes inside of them anyway... I have hundreds of variables for each
signal, and I can't make this signal object have 200 local
variables, I
think it would be dirty to type
attr_reader: for all the hundreds variables...

Consider what a Hash is: a special mapping of keys to values with the
restriction that a key can only appear once. Why not represent the
data not as a {'key'=>'value'} but as ['key','value']? Unless the
performance optimization of the Hash lookup is truly important. You
can then use things like Array#assoc to get the pair back.

Plus then only some of them get defined each time...

Am I thinking about this correctly? i have seen that tutorial, i
think
actually in my book here by Peter Cooper or at least something
similar,
although I am not seeing how it relates fully.

Thanks!!!
Matt
--

You get the ordering (which is what comes to my mind when you say
"repeating fields") of an array, but your have pairs rather than
simple scalar values.

-Rob

Rob Biedenharn http://agileconsultingllc.com
(e-mail address removed)

 

[John W Higgins]

[Note: parts of this message were removed to make it a legal post.]

Matt,

I have to figure out how to make a Signal Object, i can't see how I am
going to get around having this messed up hash problem even if I make a
signal object. because of the repeating fields within the data.

class Signal
....
end

x = Signal.new

I think having objects only changes the fact of instead of having an
array of hashes at my top level, i have an array of objects that have
hashes inside of them anyway... I have hundreds of variables for each
signal, and I can't make this signal object have 200 local variables, I
think it would be dirty to type
attr_reader: for all the hundreds variables...

It's difficult to come up with suggestions if we only have partial
information - you never mentioned that you had 200 or so possible variables
for these items - not bitching - it's just difficult to be of assistance
when one thinks you are looking at 4-5 variables and suddenly it grows 50
times.

The first advantage of the object is that you can approach things one piece
at a time and get better control over each piece - everything is well
defined and you aren't dealing with as many abstract concepts as arrays of
hash of arrays.......

So what you appear to have is the following - a single Signal has hundreds
of variables of which some of them have multiple values for the same Signal.
Lets start piecing together a class then (no ruby available so this might
not work exactly as typed.

(Gist located here http://gist.github.com/187725)

class SignalVariable
attr_reader :value

def add(val)
#we switch from a raw string to an array if this is the second
time we've added
if @value
#add the new value to the array
#it looks strange but it doesn't matter how many times you have
added an item
#it's easier then an if x then y else z concept to me
@value = [@value, val].flatten
else
@value = val
end
end
end

class Signal
def initialize()
#setup the hash and have it create the variable object for every new item
@variables = Hash.new{ |h,k| h[k] = SignalVariable.new }
end

def add_variable(name, value)
@variables[name].add(value)
end

def method_missing(name, *args)
#this will allow a call to Signal.xxx to return xxx automatically
#no need to define every variable
return @variables[name].value if @variables.key?(name)
super
end
end


This would allow you to just enter each variable as you came to it -
regardless of whether it was repeated or not. The return values would be
either the value or an array of values depending on whether or not something
was repeated.

I'm sure some folks will think this is a foolish option - it's just the way
I would approach it......

John

 

[Matt Brooks]

I'm sure some folks will think this is a foolish option - it's just the
way
I would approach it......

John

Hey John,

This is great. Not foolish at all. I think this will work great for
this, and I have already started implementing it and it looks to work
good so far.

All I am doing is adding to an array each signal object I create, as
7stud mentioned, and then adding to each object till the next signal is
reached.

I do have a quick question, I thought the way method_missing worked, was
you could call that method using any method name, so I tried it for heck
of it just because I never used it, i did blahparameter_name), and it
complained that there was no blah method, which I thought would cause
the method_missing method to run. I changed the method to be
get_val(name, *args) and it worked.

What is the role of the *args as an input? Does this allow me to ask for
an array of parameters at once? Or is something to do with the
missing_method thing?

Thanks again,
Matt

 

[John W Higgins]

[Note: parts of this message were removed to make it a legal post.]

Morning Matt,

...

I do have a quick question, I thought the way method_missing worked, was
you could call that method using any method name, so I tried it for heck
of it just because I never used it, i did blahparameter_name), and it
complained that there was no blah method, which I thought would cause
the method_missing method to run. I changed the method to be
get_val(name, *args) and it worked.

Probably poor wording on my part in terms of the parameters names for
method_missing. This might look a little better

def method_missing(method_name, *args)
#this will allow a call to Signal.xxx to return xxx automatically

#no need to define every variable
return @variables[method_name].value if @variables.key?(method_name)
super
end

What method_missing does is give you the name of the method as the first
parameter and then the *args is just an array I believe that holds any
number of variables passed in. It allows method_missing to handle things
like

blah(a, b, c, d, e) as easily as blah(a) as blah()

In each of those cases method_missing would be called with "blah" for the
method_name and *args would contain whatever was in the ().

So in this case if you wanted the freq parameter you could call something
like

x = Signal.new
x.add_variable('freq', 10)
x.freq #returns 10

Method missing will still fire an unknown method error if you use a
parameter name that doesn't exist - that's what the call to super does in
the method_missing function. It checks it's list of parameters for what you
want and then just treats it as a normal bad method call.

Always remember that pp is a great help to see things as they exist in the
code. If you wanted to check out what method_missing was getting an option
would be to add require 'pp' to the top of your code and then modify
method_missing to look something like this.

def method_missing(method_name, *args)
#this will allow a call to Signal.xxx to return xxx automatically

pp 'Method missing got the following', method_name, *args
#no need to define every variable
return @variables[method_name].value if @variables.key?(method_name)
super
end


Hope that clears it up a little.

John

 

[Matt Brooks]

Thanks John, for the method_missing explanation, that makes more sense
now, and it works, but only if I do the following: notice :signal_index

@signals = []
attr_reader :signals
@signals << Signal.new
@signals[signal_index].add_variablesignal_index, signal_index)
framework.write_log("Signal
Index:#{framework.binary.signals[0].signal_index}")


if I do: notice 'signal_index'

@signals[signal_index].add_variable('signal_index', signal_index)
framework.write_log("Signal
Index:#{framework.binary.signals[0].signal_index}")

it says no method error.

1.) So : or ' ', one is symbol one is string, i didn't think that would
make a difference? I'm not sure if that is a side affect of me having
all these objects buried, like I have a framework object that makes a
binary object that then houses the functions you gave me. And I am
accessing those variables from a ruby script that is outside of all that
mess, which is why i have a hierarchy of object calls....



2.) Is there a way to make the 'signal_index' a variable? I would like
to be able to store the contents of a variable as the name that is then
associated with a value.

like this possibly? I want the contents of parameter_name to be the
label or name associated with the contents of the variable
parameter_value
@signals[signal_index].add_variable(#{parameter_name}, parameter_value)




3.) Also, If I wanted to print all key value pairs in the @variables, in
an understandable format just to prove and see that they are all there,
how would you do that, without first knowing which variables were stored
that time around.

Would I iterate through @variables ? for all the names, and through
@values for all the values? I guess I don't fully understand how the
embedded array of repeating fields is arranged and where it is located
within @values and @variables. So how would you also handle printing
that embedded second array of repeating values and names?



Sorry I am new to Ruby, and I do appreciate your help.
-Matt

 

[Matt Brooks]

2.) Is there a way to make the 'signal_index' a variable? I would like
to be able to store the contents of a variable as the name that is then
associated with a value.

like this possibly? I want the contents of parameter_name to be the
label or name associated with the contents of the variable
parameter_value
@signals[signal_index].add_variable(#{parameter_name}, parameter_value)

nevermind on #2, I got that to work!

@signals[signal_index].add_variable(parameter_name.to_sym,
parameter_value)

 

[John W Higgins]

[Note: parts of this message were removed to make it a legal post.]

Matt,

3.) Also, If I wanted to print all key value pairs in the @variables, in
an understandable format just to prove and see that they are all there,
how would you do that, without first knowing which variables were stored
that time around.

Would I iterate through @variables ? for all the names, and through
@values for all the values? I guess I don't fully understand how the
embedded array of repeating fields is arranged and where it is located
within @values and @variables. So how would you also handle printing
that embedded second array of repeating values and names?

If this is just for testing and you just want the info on the command line
then you could just go with something like

require 'pp'

and then inside the class add

def show_variables()
pp @variables #This gives one blob of data
#or
@variables.each{ |var| pp var } #This will break each variable onto a line
with [ key, value ] symantics
end

This will give you a "nice" output of what the data is inside the variable
list.

John

 

[Matt Brooks]

This will give you a "nice" output of what the data is inside the
variable
list.

John

Awesome, I appreciate your help!