building an object index - data structure question

[Raj]

Hi all,

I have a requirement to be able to search a huge list of java objects
(constructed from a database resultset) given the value of certain
member variables. I could achieve the same by having n separate
HashMaps, one for each variable member I want to include in the
search, with the hashmap key being the member value and the hashmap
value being the object itself. Is there a better data structure I can
use? Any better way to do this? Would a treemap be more efficient for
searching than a HashMap? If this were in the database, an index could
be used. What would be the java equivalent?

Thanks in advance

Greetings
-Raj

 

[Michael Borgwardt]

I have a requirement to be able to search a huge list of java objects
(constructed from a database resultset) given the value of certain
member variables. I could achieve the same by having n separate
HashMaps, one for each variable member I want to include in the
search, with the hashmap key being the member value and the hashmap
value being the object itself. Is there a better data structure I can
use? Any better way to do this?

The best way would be to include the search criteria in the DB query
in the first place. THAT'S WHAT A DB IS FOR!!!

Would a treemap be more efficient for
searching than a HashMap?
No.

If this were in the database, an index could
be used. What would be the java equivalent?

Your original idea with n HashMaps is the exact equivalent. Note that
a DB index also only allows searching on one field and needs to be
created separately (with the corresponding overhead) for each field.

 

[Raj]

The best way would be to include the search criteria in the DB query

in the first place. THAT'S WHAT A DB IS FOR!!!

I have to look up this object map several hundred / thousand times for
every time this program is executed (which runs in a batch fashion)
and so I need this data structure in memory so as to save needless
database calls. and besides, making changes to the stored procedures
returning the data is not an option in the current development
environment due to political and other reasons.

Thanks
-Raj

 

[Michael Borgwardt]

I have to look up this object map several hundred / thousand times for
every time this program is executed (which runs in a batch fashion)
and so I need this data structure in memory so as to save needless
database calls.

Are you sure these DB calls really are "needless", i.e. would be slower
than doing work that a DB is optimized for in your own code? I find that
assumption highly dubious. Actually, If the work consists of simple
lookups on fields, then it should be possible to get the entire result
with a DB query whose resultset just contains the data you're interested
in. But...

and besides, making changes to the stored procedures
returning the data is not an option in the current development
environment due to political and other reasons.

....of course it's possible that artificial restrictions prevent a sane
solution.

 

[Sudsy]

Raj wrote:

Are you sure these DB calls really are "needless", i.e. would be slower
than doing work that a DB is optimized for in your own code? I find that
assumption highly dubious. Actually, If the work consists of simple
lookups on fields, then it should be possible to get the entire result
with a DB query whose resultset just contains the data you're interested
in. But...

...of course it's possible that artificial restrictions prevent a sane
solution.

How about a combination of the two? Store the objects (I get the impression
that they're large, hence no desire to keep fetching from DB) and then
perform lookups which merely return the primary key of the records of
interest?
SELECT * FROM table and put into a HashMap where the primary key of the
record is the key for the HashMap and the value is a value-object
containing the record details.
SELECT pk FROM table WHERE condition and use the pk (primary key) values
in the ResultSet in HashMap#get.
A tad complex but it would seem like a fair division of labour.
Then again, without a more complete description of the situation this is
merely conjecture.