Strategy help for java search engine weighting

[Tom Dyess]

I'm converting a search engine from Delphi to Java and would like some help
with my strategy. This is for the oraclepower.com website.

BACKGROUND:
I have a table of links that I need to match up with keywords the user
enters. If the user enters the keywords "oracle" and "reports" I query the
database for all link titles and descriptions that exist in the table
something like this

SELECT col1..coln
FROM LINK_LIST
WHERE title like '%keyword%' or subject like '%keyword%'

I then pull the recordset into what is called a client data set (Delphi),
which is a
recordset "in memory" instead of one directly attached to a database or XML
file. I
then use an algorithm to weight each result based on where and how many
times it appears in title and link, then sort by weight, then display the
resource to the web page.

QUESTION:

Do the java libraries have an object that can easily be loaded from a
ResultSet, have a weight column added to it then sort itself, then examine
the columns from the memory object; or do I have to create this object
myself?

If I have to create this myself, what objects will help me reduce the coding
/ designing (ie other than ArrayList or some generic collection). Should
this be done with SAX and an in-memory XML object (no file needed, and would
slow things down)? Should I create a collection that loads itself from a
ResultSet including metadata and can sort by a particular column, set of
columns? I know how I would do it in Delphi, but need some insight into
various java solutions.

Thanks,
Tom Dyess
OraclePower.com

 

[Chris]

I'm converting a search engine from Delphi to Java and would like some
help

with my strategy. This is for the oraclepower.com website.

BACKGROUND:
I have a table of links that I need to match up with keywords the user
enters. If the user enters the keywords "oracle" and "reports" I query the
database for all link titles and descriptions that exist in the table
something like this

SELECT col1..coln
FROM LINK_LIST
WHERE title like '%keyword%' or subject like '%keyword%'

I then pull the recordset into what is called a client data set (Delphi),
which is a
recordset "in memory" instead of one directly attached to a database or XML
file. I
then use an algorithm to weight each result based on where and how many
times it appears in title and link, then sort by weight, then display the
resource to the web page.

QUESTION:

Do the java libraries have an object that can easily be loaded from a
ResultSet, have a weight column added to it then sort itself, then examine
the columns from the memory object; or do I have to create this object
myself?

If I have to create this myself, what objects will help me reduce the coding
/ designing (ie other than ArrayList or some generic collection). Should
this be done with SAX and an in-memory XML object (no file needed, and would
slow things down)? Should I create a collection that loads itself from a
ResultSet including metadata and can sort by a particular column, set of
columns? I know how I would do it in Delphi, but need some insight into
various java solutions.

First, let me recommend you reconsider the make/buy decision. Buying a
search engine written in Java can save you a lot of time. Dieselpoint
http://dieselpoint.com is a good choice for this kind of thing.

But if you want to do it yourself, creating a class is not hard. Try this:

public class Result implements Comparable {
// add getter and setters for these fields
private int weight;
private String title;
private String text;
private String whatever;

public int compareTo(Object o) {
return ((Result)o).weight - weight;
}
}

Use the Result class thusly:

ArrayList list = new ArrayList();
ResultSet rs = (get your result set here)
while (rs.next()) {
Result result = new Result();
result.setTitle(rs.getString(title)):
... etc ..

// perform weight calcs here

result.setWeight(weight);
list.add(result);
}

Object [] array = list.toArray();
java.util.Arrays.sort(array);

The code above isn't tested, but you get the idea: the secret is to
implement the Comparable interface, and then take advantage of the
Arrays.sort() method, which depends on it.

 

[Tom Dyess]

Re Buying v. Making: I wrote the original (Delphi/ISAPI) and it works fine.
The exercise of converting this to java is to give me a reason to learn java
on a realistic project more than anything. Lol.

Ya, I was thinking of making something generic similar to various Delphi
classes. That way I can do something like Obj.SortByField(String field),
Obj.FieldAsString(String fieldname), next(), prev(),
Obj.LoadFromResultSet(ResultSet rs), etc. I was thinking of creating objects
with an aggregated ArrayList (rows) of another aggregated ArrayList (fields)
in which the custom "Field" objects would store the metadata.

It seems that it's a little too lowlevel, not that that's a problem, but I'd
hate to do it if something similar was already done by classes already
defined in the java distro.

Tom

I'm converting a search engine from Delphi to Java and would like some help
with my strategy. This is for the oraclepower.com website.

BACKGROUND:
I have a table of links that I need to match up with keywords the user
enters. If the user enters the keywords "oracle" and "reports" I query
the
database for all link titles and descriptions that exist in the table
something like this

SELECT col1..coln
FROM LINK_LIST
WHERE title like '%keyword%' or subject like '%keyword%'

I then pull the recordset into what is called a client data set (Delphi),
which is a
recordset "in memory" instead of one directly attached to a database or XML
file. I
then use an algorithm to weight each result based on where and how many
times it appears in title and link, then sort by weight, then display the
resource to the web page.

QUESTION:

Do the java libraries have an object that can easily be loaded from a
ResultSet, have a weight column added to it then sort itself, then
examine
the columns from the memory object; or do I have to create this object
myself?

If I have to create this myself, what objects will help me reduce the coding
/ designing (ie other than ArrayList or some generic collection). Should
this be done with SAX and an in-memory XML object (no file needed, and would
slow things down)? Should I create a collection that loads itself from a
ResultSet including metadata and can sort by a particular column, set of
columns? I know how I would do it in Delphi, but need some insight into
various java solutions.

First, let me recommend you reconsider the make/buy decision. Buying a
search engine written in Java can save you a lot of time. Dieselpoint
http://dieselpoint.com is a good choice for this kind of thing.

But if you want to do it yourself, creating a class is not hard. Try this:

public class Result implements Comparable {
// add getter and setters for these fields
private int weight;
private String title;
private String text;
private String whatever;

public int compareTo(Object o) {
return ((Result)o).weight - weight;
}
}

Use the Result class thusly:

ArrayList list = new ArrayList();
ResultSet rs = (get your result set here)
while (rs.next()) {
Result result = new Result();
result.setTitle(rs.getString(title)):
... etc ..

// perform weight calcs here

result.setWeight(weight);
list.add(result);
}

Object [] array = list.toArray();
java.util.Arrays.sort(array);

The code above isn't tested, but you get the idea: the secret is to
implement the Comparable interface, and then take advantage of the
Arrays.sort() method, which depends on it.