sorting large data sets

[Simon]

Hi,

I have a table I am populating with data from a DB, and have it
sortable on the client side. When the table exceeds around 300 rows,
I start running into problems with rendering and sorting it. I
started out using Prototype to help with the sorting, but it appears
that this results in a lot of calls to _getElementsByXPath, which are
taking up a large percentage of the rendering time. I believe this
call is occurring as the DOM is being parsed.

What is the general/most accepted manner of dealing with large data
sets (I am expecting some to grow into the thousands) in a table so
that it can be sorted client side? I am thinking of building a
javascript (JSON) object containing the data, and sorting based on
this. Anybody have any thoughts on this approach, or any suggestions
for other ways to approach this?

Thanks,

Simon

 

[Matt Kruse]

What is the general/most accepted manner of dealing with large data
sets (I am expecting some to grow into the thousands) in a table so
that it can be sorted client side?

There are a number of things to consider and a lot of small
optimizations you can put in place to quickly do client-side table
sorting. Especially in IE, loop optimizations and other things make a
big difference.

Check out my table sorting library here:
http://www.JavascriptToolbox.com/lib/table/

And is extremely fast and has a lot of functionality that you can
enable just by putting classes on your table.

I am working on an update that will fix a few minor things and add
more, but the version that is there is working quite well for me in
several production projects.

Matt Kruse

 

[Simon]

This looks pretty promising, thanks. Are there ways to extend the
sorting? The table I am working on has other data types (buttons,
coloured divs) that I would need to sort on.

Simon

 

[Matt Kruse]

This looks pretty promising, thanks. Are there ways to extend the
sorting? The table I am working on has other data types (buttons,
coloured divs) that I would need to sort on.

Yes, but it's not documented (yet).

If you want to define a custom sorting function, check out the Sort
object where you can define more functions.

If you want to define how a node in a cell is converted into a
sortable value, check out the nodeValue object, where you can map
elements (BUTTON, DIV, etc) to custom functions that will convert the
node into some text to sort on.

You are, of course, welcome to modify and customize the code as you
wish to work with your project.

Matt Kruse

 

[Dr J R Stockton]

In comp.lang.javascript message <[email protected]
oglegroups.com>, Mon, 5 Nov 2007 16:23:03, Simon

I have a table I am populating with data from a DB, and have it
sortable on the client side. When the table exceeds around 300 rows,
I start running into problems with rendering and sorting it. I
started out using Prototype to help with the sorting, but it appears
that this results in a lot of calls to _getElementsByXPath, which are
taking up a large percentage of the rendering time. I believe this
call is occurring as the DOM is being parsed.

ISTM that you should not sort the Table /in situ/; you should read the
DB data into an array-like structure, sort that, and then write it into
the Table. Only if the Table is directly editable (e.g. by having

What is the general/most accepted manner of dealing with large data
sets (I am expecting some to grow into the thousands) in a table so
that it can be sorted client side? I am thinking of building a
javascript (JSON) object containing the data, and sorting based on
this.

Yes, do that.

Almost certainly, with that amount of data, you should if practical use
Array.sort() rather than Array.sort(Function) - requirement, that each
element of the data can be pre-processed to a string form which will
sort as desired, and regenerated from that form. Otherwise, replace
each element by {E:element, K:key) and write a simple comparison
function that compares keys.

You may not need to de-pre-process in full.

<URL:http://www.merlyn.demon.co.uk/js-order.htm> has a simple speed
demo for the effects of pre-processing. Using keys is slow, but could
be advantageous for repeated sorting.

It's a good idea to read the newsgroup c.l.j and its FAQ. See below.