T
treefrog
Hi,
I'm trying to work out how to tackle a particular problem in rails.
I have some data. A lot of data (well not NSA style a lot of data, but
a lot for me: 1Gb at the moment, more to come).
I'm building a ruby application to process and view this data, and have
been working round using rails as a front end - do the heavy lifting
using some maths libraries, then use rails to display results, etc.
The problem that I have is as follows: The natural organisation is that
the data consists of DataSets, each DataSet is a group of related time
series (Series), and each series consists fo a set of observations,
each Observation having a set of parameters.
So far, so good. The logical layout of the data is something like
DataSet has_many Series
Series has_many Observation
The problem is that I need to be able to do processing of the data
offline at a later point, and one of the thngs I am doing is developing
new ways of dealing with the data. So I need to apply a transformation
to each Observation. The transformed data hasn't lost anything, and it
still comes in Series which belong to a dataset.
One option would be
DataSet has_many Series
DataSet has many TransformedSeries
Series has_many Observation
TransformedSeries has many TransformedObservation
but the problem is that every time I add a new transformation, I need
to change my DB schema, etc, and add new code in to DataSet - which I
personally believe is quite ugly!
What I would really like is for the Series to have a field
(series_type) which contains the name of the table to look up the
Observation in. So, depending on the value in the series_type field,
Series would perform has_many operations using the Observation table or
the TransformedObservation table (or whatever other table).
Is there any easy way to achieve this without directly hacking
has_many?
Best regards,
Treefrog
I'm trying to work out how to tackle a particular problem in rails.
I have some data. A lot of data (well not NSA style a lot of data, but
a lot for me: 1Gb at the moment, more to come).
I'm building a ruby application to process and view this data, and have
been working round using rails as a front end - do the heavy lifting
using some maths libraries, then use rails to display results, etc.
The problem that I have is as follows: The natural organisation is that
the data consists of DataSets, each DataSet is a group of related time
series (Series), and each series consists fo a set of observations,
each Observation having a set of parameters.
So far, so good. The logical layout of the data is something like
DataSet has_many Series
Series has_many Observation
The problem is that I need to be able to do processing of the data
offline at a later point, and one of the thngs I am doing is developing
new ways of dealing with the data. So I need to apply a transformation
to each Observation. The transformed data hasn't lost anything, and it
still comes in Series which belong to a dataset.
One option would be
DataSet has_many Series
DataSet has many TransformedSeries
Series has_many Observation
TransformedSeries has many TransformedObservation
but the problem is that every time I add a new transformation, I need
to change my DB schema, etc, and add new code in to DataSet - which I
personally believe is quite ugly!
What I would really like is for the Series to have a field
(series_type) which contains the name of the table to look up the
Observation in. So, depending on the value in the series_type field,
Series would perform has_many operations using the Observation table or
the TransformedObservation table (or whatever other table).
Is there any easy way to achieve this without directly hacking
has_many?
Best regards,
Treefrog