yaml format

[Charles L. Snyder]

Hi

Pardon the long post:
I am playing around with one of the ruby quiz problems (stock
portfolio) - trying to modify it to do some other things. I am reading
in data from a csv file, getting the stock symbols from it, and using
the symbols via Yahoo finance to get data, which I output into a .yaml
format. Everything is fine, except that I get a screwy yaml output, and
can't figure out where the extra " and \ are coming from.

Here is the ruby code:

#!/usr/local/bin/ruby

# Stock data via: http://www.gummy-stuff.org/Yahoo-data.htm

require 'open-uri'
require 'yaml'
require 'ostruct'
require 'csv'

# myfile = 'mystocks_final.yaml' # change this for your file
data = ["s","n","p","p2"] #symbols to check on yahoo

# stock_struct = Struct::new( "StockStruct", :name, :shares, :symbol,
:date, :ass_class)
# get the data from the yaml file
portfolio = YAML.load( open('my_db4.yaml').read )

stocks = []
portfolio.each {|r| stocks.push(r.symbol)} # stick the symbols into the
stock array
data = data.map { |tag| tag[/\w+/] }.join

# put in a delete from array for those stocks with incorrect or
unobtainable symbols
bad_symbols = ["CREF Bond","CREF
Equity","8401765","CREFAnn","8401463","CREF Global
Eq","8806422","CASH", "CREF Stock", "CREF Equity IN"]
stocks.delete_if {|x| bad_symbols.include?(x)}
stocks = stocks.join("+")
stock_struct = Struct::new( "StockStruct", :symbol, :name, rice,
:date, :shares, :ass_class, :ass_subclass)
w = []
open "http://finance.yahoo.com/d/quotes.csv?s=#{stocks}&f=#{data}" do
|m|
m.each do |n|
date_now = Time.now.localtime.strftime("%Y-%m-%d")
n = n.split(/,/)
w.push(stock_struct.new(n[0], n[1], n[2],date_now, 5, "Domestic
Equity","Precious Metal"))
end
end

open("my_yaml_output.yaml","w"){|e| e << w.to_yaml}


Here is a sample of the my_yaml_output file:

---
- !ruby/struct:StockStruct
symbol: "'\"AAPL\"'"
name: "\"APPLE COMPUTER\""
price: "60.76"
date: "2006-06-09"
shares: 5
ass_class: Domestic Equity
ass_subclass: Precious Metal
- !ruby/struct:StockStruct
symbol: "'\"BGEIX\"'"
name: "\"AMERICAN CENTURY \""
price: "17.79"
date: "2006-06-09"
shares: 5
ass_class: Domestic Equity
ass_subclass: Precious Metal


what I want is: (get rid of all the extra " and \)

---
- !ruby/struct:StockStruct
symbol: AAPL
name: APPLE COMPUTER
price: 60.76
date: "2006-06-09"
shares: 5
ass_class: Domestic Equity
ass_subclass: Precious Metal
- !ruby/struct:StockStruct
symbol: BGEIX
name: AMERICAN CENTURY
price: 17.79
date: "2006-06-09"
shares: 5
ass_class: Domestic Equity
ass_subclass: Precious Metal


Note - when i try a
puts n
puts n.class

in the line just above this one : w.push(stock_struct.new(n[0], n[1],
n[2],date_now, 5, "Domestic Equity","Precious Metal"))
I get
"AAPL"
string
"BGEIX"
string

so I don't see why feeding in strings gives me all this weird stuff ?

PS Ruby 1.82 , Win XP

Thanks in advance

 

[coachhilton]

Are you certain Yahoo is not giving you back quotes strings?

Ken

Hi

Pardon the long post:
I am playing around with one of the ruby quiz problems (stock
portfolio) - trying to modify it to do some other things. I am reading
in data from a csv file, getting the stock symbols from it, and using
the symbols via Yahoo finance to get data, which I output into a .yaml
format. Everything is fine, except that I get a screwy yaml output, and
can't figure out where the extra " and \ are coming from.

Here is the ruby code:

#!/usr/local/bin/ruby

# Stock data via: http://www.gummy-stuff.org/Yahoo-data.htm

require 'open-uri'
require 'yaml'
require 'ostruct'
require 'csv'

# myfile = 'mystocks_final.yaml' # change this for your file
data = ["s","n","p","p2"] #symbols to check on yahoo

# stock_struct = Struct::new( "StockStruct", :name, :shares, :symbol,
:date, :ass_class)
# get the data from the yaml file
portfolio = YAML.load( open('my_db4.yaml').read )

stocks = []
portfolio.each {|r| stocks.push(r.symbol)} # stick the symbols into the
stock array
data = data.map { |tag| tag[/\w+/] }.join

# put in a delete from array for those stocks with incorrect or
unobtainable symbols
bad_symbols = ["CREF Bond","CREF
Equity","8401765","CREFAnn","8401463","CREF Global
Eq","8806422","CASH", "CREF Stock", "CREF Equity IN"]
stocks.delete_if {|x| bad_symbols.include?(x)}
stocks = stocks.join("+")
stock_struct = Struct::new( "StockStruct", :symbol, :name, rice,
:date, :shares, :ass_class, :ass_subclass)
w = []
open "http://finance.yahoo.com/d/quotes.csv?s=#{stocks}&f=#{data}" do
|m|
m.each do |n|
date_now = Time.now.localtime.strftime("%Y-%m-%d")
n = n.split(/,/)
w.push(stock_struct.new(n[0], n[1], n[2],date_now, 5, "Domestic
Equity","Precious Metal"))
end
end

open("my_yaml_output.yaml","w"){|e| e << w.to_yaml}


Here is a sample of the my_yaml_output file:

---
- !ruby/struct:StockStruct
symbol: "'\"AAPL\"'"
name: "\"APPLE COMPUTER\""
price: "60.76"
date: "2006-06-09"
shares: 5
ass_class: Domestic Equity
ass_subclass: Precious Metal
- !ruby/struct:StockStruct
symbol: "'\"BGEIX\"'"
name: "\"AMERICAN CENTURY \""
price: "17.79"
date: "2006-06-09"
shares: 5
ass_class: Domestic Equity
ass_subclass: Precious Metal


what I want is: (get rid of all the extra " and \)

---
- !ruby/struct:StockStruct
symbol: AAPL
name: APPLE COMPUTER
price: 60.76
date: "2006-06-09"
shares: 5
ass_class: Domestic Equity
ass_subclass: Precious Metal
- !ruby/struct:StockStruct
symbol: BGEIX
name: AMERICAN CENTURY
price: 17.79
date: "2006-06-09"
shares: 5
ass_class: Domestic Equity
ass_subclass: Precious Metal


Note - when i try a
puts n
puts n.class

in the line just above this one : w.push(stock_struct.new(n[0], n[1],
n[2],date_now, 5, "Domestic Equity","Precious Metal"))
I get
"AAPL"
string
"BGEIX"
string

so I don't see why feeding in strings gives me all this weird stuff ?

PS Ruby 1.82 , Win XP

Thanks in advance

 

[Charles L. Snyder]

Are you certain Yahoo is not giving you back quotes strings?

I don't think that is the problem, because I tried this:

Note - when i try a
puts n
puts n.class

in the line just above this one : w.push(stock_struct.new(n[0], n[1],
n[2],date_now, 5, "Domestic Equity","Precious Metal"))
I get
"AAPL"
string
"BGEIX"
string

 

[Ross Bamford]

Are you certain Yahoo is not giving you back quotes strings?

I don't think that is the problem, because I tried this:

Note - when i try a
puts n
puts n.class

in the line just above this one : w.push(stock_struct.new(n[0], n[1],
n[2],date_now, 5, "Domestic Equity","Precious Metal"))
I get
"AAPL"
string
"BGEIX"
string

But those strings *do* contain quotes - puts translates \ stuff for
output. If you did p n rather than puts n, You'd see:

"\"AAPL\""
string
"\"BGEIX\""
string

(p uses inspect, showing the actual form of the string). The backslashed
quote here is the quote that becomes visible in your puts output.

One easy way I can see to fix this, since you're already requiring CSV in
your script, is to change this line:

n = n.split(/,/)

to this:

n = CSV.parse_line(n)

and let CSV worry about correctly parsing out the quoted fields.

 

[Charles L. Snyder]

(p uses inspect, showing the actual form of the string). The backslashed
quote here is the quote that becomes visible in your puts output.

One easy way I can see to fix this, since you're already requiring CSV in
your script, is to change this line:

n = n.split(/,/)

to this:

n = CSV.parse_line(n)

and let CSV worry about correctly parsing out the quoted fields.

Thanks -

I tried that, Ross - now my_yaml_output.yaml looks like this:

---
- !ruby/struct:StockStruct
symbol: !str:CSV::Cell AAPL
name: !str:CSV::Cell APPLE COMPUTER
price: !str:CSV::Cell 60.76
date: "2006-06-09"
shares: 5
ass_class: Domestic Equity
ass_subclass: Precious Metal
- !ruby/struct:StockStruct
symbol: !str:CSV::Cell BGEIX
name: !str:CSV::Cell "AMERICAN CENTURY "
price: !str:CSV::Cell 17.79
date: "2006-06-09"
shares: 5
ass_class: Domestic Equity
ass_subclass: Precious Metal

and when a place

p n[0]

in the same spot, I get

"AAPL"
"BGEIX"

- good old normal looking strings...
(this time with ruby 1.84 on another win xp machine)

? -v puzzled

Thanks for your suggestions !

 

[Ross Bamford]

One easy way I can see to fix this, since you're already requiring CSV
in
your script, is to change this line:

n = n.split(/,/)

to this:

n = CSV.parse_line(n)

and let CSV worry about correctly parsing out the quoted fields.

I tried that, Ross - now my_yaml_output.yaml looks like this:

---
- !ruby/struct:StockStruct
symbol: !str:CSV::Cell AAPL
name: !str:CSV::Cell APPLE COMPUTER
price: !str:CSV::Cell 60.76
[...snipped...]

and when a place

p n[0]

in the same spot, I get

"AAPL"
"BGEIX"

- good old normal looking strings...
(this time with ruby 1.84 on another win xp machine)

? -v puzzled

This is just because CSV uses it's own class (CVS::Cell instances) to
represent cells in the parsed string. These mostly act as the data they
hold but if you want to force it to use strings, you could just map the
output:

n = CSV.parse_line(n).map! { |s| s.to_s }

Alternatively, looking at the data Yahoo sends over I guess you should be
okay with a bit of regex foo, e.g:

n = n.split(/,/).map! { |s| s.gsub(/^['"]?(.*?)['"]?$/) { $1 } }

(only lightly tested).

 

[Charles L. Snyder]

Both of these suggestions work perfectly.


Thanks for your help !!!