Variable references in yaml files

[dara]

I have been trying to include variable references in a yaml file. I
have a feeling it's not possible to do this, but I thought I'd ask
here before giving up.

Here is my ruby script:
require 'yaml'
hash = YAML.load_file('sampl.yml')

fourth_member = 'Joan'

hash.each_pair do |key, value|
puts "Value for key \"#{key}\" is: \"#{value}\""
end

puts "\nFrom within script, \"fourth\" is: #{fourth_member}"

The contents of sampl.yml are:
:first: 'John'
:second: 'Jane'
:third: 'Jack'
:fourth: "#{fourth_member}"

The output of the script is:

Value for key "first" is: "John"
Value for key "second" is: "Jane"
Value for key "third" is: "Jack"
Value for key "fourth" is: "#{fourth_member}"

From within script, "fourth" is: Joan

I want the fourth line of the output to read:
"Value for key "fourth" is: "Joan""

I have done much googling and tried every possible permutation of
escape characters I could think of, with no success. Can anyone
either:
1) Show me a way to do this (much preferred ).
2) Confirm this can't be done (would at least save me wasting further
time on it).

Thanks,

-Dara

 

[David Masover]

The contents of sampl.yml are:
:first: 'John'
:second: 'Jane'
:third: 'Jack'
:fourth: "#{fourth_member}"

In other words, you have a local variable fourth_member which you want to
appear there?

Can anyone
either:
1) Show me a way to do this (much preferred ).
2) Confirm this can't be done (would at least save me wasting further
time on it).

I can do both.

I doubt very much that Yaml itself supports this, and it would be very
dangerous and scary if it does.

However, you could easily do something like this yourself. The quick-and-
dirty, dangerous way would probably go something like this:

hash.each_pair do |key, value|
hash[key] = eval "\"#{value}\""
end

It should be obvious why this is dangerous, though -- that yaml file can now
contain arbitrary code, probably not what you want. But you get the idea --
it's easy enough to build some kind of template system. Here's a safer way --
first, don't use a local variable, use a hash of variables you want to make
available to the script:

yaml_variables = {:fourth_member => 'Joan'}

Then it's a simple matter of substitution:

hash.each_value do |string|
yaml_variables.each_pair do |key, value|
string.gsub! "\#{#{key}}", value
end
end

Note that you're not constrained to the #{} syntax. You can make up your own.

This still has some flaws -- for example, if you have something like this:

yaml_variables = {:foo => '#{bar}', :bar => 'baz'}

Depending what order you iterate through the yaml_variables hash, you might
get either the string '#{bar}' (probably what you wanted), or the string
'baz', replacing any occurrence of '#{foo}'.

There are other problems -- I'm assuming your yaml file is a single, flat hash
-- but I'm sure you can come up with something better.

Also, it might be helpful to know what you're actually doing with this. It's
quite possible you don't need anything nearly this complex, and Yaml does have
some substitution of its own that might be handy -- though that's within a
file, it doesn't pull values out of your script. Also, if you're just trying
to define a default value, leave the value nil in the Yaml file, and merge it
into a hash of default values.

 

[dara]

Thanks for the detailed response, David.

What I'm actually doing: creating hashes of "input + expected results"
for test scenarios. I have 200+ test scenarios to run, and want to run
them all through one script. The hard part is not all the expected
results are static (e.g. today's date is an expected result). The hash
is a little more complex than my example (one more layer of key-value
nesting) but nothing too tricky.

So, I need a way to create some of the expected results on the fly,
while most of the expected results are static, predictable strings. I
was hoping to put all the input and expected results (including on-the-
fly stuff) in a single yaml file for each scenario and then iterate on
the yaml files, but it seems that's not do-able. So I'll have to have
add special handling.

Thanks for your suggestions. I think I'm going to go with something
similar to your second suggestion. Instead of strings for the on-the-
fly expected results, I will make them symbols (ruby yaml allows
this). Then, in my script, I will have a substitution hash which is
keyed by the symbol, and has as its value, the on-the-fly expected
result. Hard to describe it in words, but the example below
illustrates it (I hope).

I guess my example would have been clearer if I'd shown something on-
the-fly. I've added a time variable to my example. I think I'll
implement my solution like so:

require 'yaml'
hash =3D YAML.load_file('sampl.yml')
substitution_hash =3D {:fourth_member =3D> 'Joan',
:time =3D> Time.new.strftime("%m/%d/%y")}
hash.each_pair do |key, value|
value =3D substitution_hash[value] if value.class =3D=3D Symbol
puts "Value for key \"#{key}\" is: \"#{value}\""
end


sampl.yml now looks like:

:first: 'John'
:second: 'Jane'
:third: 'Jack'
:fourth: :fourth_member
:today: :time

Sample output:
Value for key "first" is: "John"
Value for key "second" is: "Jane"
Value for key "third" is: "Jack"
Value for key "today" is: "12/03/09"
Value for key "fourth" is: "Joan"


Thus, I can set my expected result for "today" on-the-fly.

If you think I'm making a bad decision doing it this way, please let
me know.

Thanks,

-Dara

The contents of sampl.yml are:
:first: 'John'
:second: 'Jane'
:third: 'Jack'
:fourth: "#{fourth_member}"

In other words, you have a local variable fourth_member which you want to
appear there?

Can anyone
either:
1) Show me a way to do this (much preferred ).
2) Confirm this can't be done (would at least save me wasting further
time on it).

I can do both.

I doubt very much that Yaml itself supports this, and it would be very
dangerous and scary if it does.

However, you could easily do something like this yourself. The quick-and-
dirty, dangerous way would probably go something like this:

hash.each_pair do |key, value|
=A0 hash[key] =3D eval "\"#{value}\""
end

It should be obvious why this is dangerous, though -- that yaml file can = now
contain arbitrary code, probably not what you want. But you get the idea = --
it's easy enough to build some kind of template system. Here's a safer wa= y --
first, don't use a local variable, use a hash of variables you want to ma= ke
available to the script:

yaml_variables =3D {:fourth_member =3D> 'Joan'}

Then it's a simple matter of substitution:

hash.each_value do |string|
=A0 yaml_variables.each_pair do |key, value|
=A0 =A0 string.gsub! "\#{#{key}}", value
=A0 end
end

Note that you're not constrained to the #{} syntax. You can make up your = own.

This still has some flaws -- for example, if you have something like this= :

yaml_variables =3D {:foo =3D> '#{bar}', :bar =3D> 'baz'}

Depending what order you iterate through the yaml_variables hash, you mig= ht
get either the string '#{bar}' (probably what you wanted), or the string
'baz', replacing any occurrence of '#{foo}'.

There are other problems -- I'm assuming your yaml file is a single, flat= hash
-- but I'm sure you can come up with something better.

Also, it might be helpful to know what you're actually doing with this. I= t's
quite possible you don't need anything nearly this complex, and Yaml does= have
some substitution of its own that might be handy -- though that's within = a
file, it doesn't pull values out of your script. Also, if you're just try= ing
to define a default value, leave the value nil in the Yaml file, and merg= e it
into a hash of default values.

 

[dara]

Actually, thinking just a little further, your suggestion of merging
hashes is probably the way to go. Similar to below, but instead of the
convoluted substitution hash, I can just merge a hash of on-the-fly
expected results with the hard-coded ones.

So the solution doesn't really scratch my itch of hoping to keep all
the expected results in one place, but I think I straightened out my
thinking and learned something in the process. Thanks!

The contents of sampl.yml are:
:first: 'John'
:second: 'Jane'
:third: 'Jack'
:fourth: "#{fourth_member}"

In other words, you have a local variable fourth_member which you want to
appear there?

Can anyone
either:
1) Show me a way to do this (much preferred ).
2) Confirm this can't be done (would at least save me wasting further
time on it).

I can do both.

I doubt very much that Yaml itself supports this, and it would be very
dangerous and scary if it does.

However, you could easily do something like this yourself. The quick-and-
dirty, dangerous way would probably go something like this:

hash.each_pair do |key, value|
=A0 hash[key] =3D eval "\"#{value}\""
end

It should be obvious why this is dangerous, though -- that yaml file can = now
contain arbitrary code, probably not what you want. But you get the idea = --
it's easy enough to build some kind of template system. Here's a safer wa= y --
first, don't use a local variable, use a hash of variables you want to ma= ke
available to the script:

yaml_variables =3D {:fourth_member =3D> 'Joan'}

Then it's a simple matter of substitution:

hash.each_value do |string|
=A0 yaml_variables.each_pair do |key, value|
=A0 =A0 string.gsub! "\#{#{key}}", value
=A0 end
end

Note that you're not constrained to the #{} syntax. You can make up your = own.

This still has some flaws -- for example, if you have something like this= :

yaml_variables =3D {:foo =3D> '#{bar}', :bar =3D> 'baz'}

Depending what order you iterate through the yaml_variables hash, you mig= ht
get either the string '#{bar}' (probably what you wanted), or the string
'baz', replacing any occurrence of '#{foo}'.

There are other problems -- I'm assuming your yaml file is a single, flat= hash
-- but I'm sure you can come up with something better.

Also, it might be helpful to know what you're actually doing with this. I= t's
quite possible you don't need anything nearly this complex, and Yaml does= have
some substitution of its own that might be handy -- though that's within = a
file, it doesn't pull values out of your script. Also, if you're just try= ing
to define a default value, leave the value nil in the Yaml file, and merg= e it
into a hash of default values.

 

[Marnen Laibow-Koser]

Actually, thinking just a little further, your suggestion of merging
hashes is probably the way to go. Similar to below, but instead of the
convoluted substitution hash, I can just merge a hash of on-the-fly
expected results with the hard-coded ones.

So the solution doesn't really scratch my itch of hoping to keep all
the expected results in one place, but I think I straightened out my
thinking and learned something in the process. Thanks!

Two ideas:

1. Rails preprocesses some of its YAML configuration files with ERb.
You might try that.

2. If you need dynamic results, perhaps YAML is not the right tool.
Perhaps Ruby, possibly with a test library, would be better.

Best,
-- 
Marnen Laibow-Koser
http://www.marnen.org
(e-mail address removed)

 

[David Masover]

Well, two complaints about this. One, you're top posting, but that's a matter
of taste...

And two:

value = substitution_hash[value] if value.class == Symbol

I doubt it matters here, but I'd do:

if value.kind_of?(Symbol)

I like to duck-type when I can, and when I can't, this is still closer than
checking the actual class. (kind_of? checks inheritance -- it would return
true if it's an instance of a child class of Symbol, or if Symbol was a module
that it included/extended from, or if it's a class which overrides #kind_of?
to lie and say it's a Symbol.)

And yes, hash merging is probably the simpler solution.