sorting large data sets

Discussion in 'Javascript' started by Simon, Nov 5, 2007.

  1. Simon

    Simon Guest

    Hi,

    I have a table I am populating with data from a DB, and have it
    sortable on the client side. When the table exceeds around 300 rows,
    I start running into problems with rendering and sorting it. I
    started out using Prototype to help with the sorting, but it appears
    that this results in a lot of calls to _getElementsByXPath, which are
    taking up a large percentage of the rendering time. I believe this
    call is occurring as the DOM is being parsed.

    What is the general/most accepted manner of dealing with large data
    sets (I am expecting some to grow into the thousands) in a table so
    that it can be sorted client side? I am thinking of building a
    javascript (JSON) object containing the data, and sorting based on
    this. Anybody have any thoughts on this approach, or any suggestions
    for other ways to approach this?

    Thanks,

    Simon
     
    Simon, Nov 5, 2007
    #1
    1. Advertising

  2. Simon

    Matt Kruse Guest

    On Nov 5, 10:23 am, Simon <> wrote:
    > What is the general/most accepted manner of dealing with large data
    > sets (I am expecting some to grow into the thousands) in a table so
    > that it can be sorted client side?


    There are a number of things to consider and a lot of small
    optimizations you can put in place to quickly do client-side table
    sorting. Especially in IE, loop optimizations and other things make a
    big difference.

    Check out my table sorting library here:
    http://www.JavascriptToolbox.com/lib/table/

    And is extremely fast and has a lot of functionality that you can
    enable just by putting classes on your table.

    I am working on an update that will fix a few minor things and add
    more, but the version that is there is working quite well for me in
    several production projects.

    Matt Kruse
     
    Matt Kruse, Nov 5, 2007
    #2
    1. Advertising

  3. Simon

    Simon Guest

    This looks pretty promising, thanks. Are there ways to extend the
    sorting? The table I am working on has other data types (buttons,
    coloured divs) that I would need to sort on.

    Simon


    On Nov 5, 11:47 am, Matt Kruse <> wrote:
    > On Nov 5, 10:23 am, Simon <> wrote:
    >
    > > What is the general/most accepted manner of dealing with large data
    > > sets (I am expecting some to grow into the thousands) in a table so
    > > that it can be sorted client side?

    >
    > There are a number of things to consider and a lot of small
    > optimizations you can put in place to quickly do client-side table
    > sorting. Especially in IE, loop optimizations and other things make a
    > big difference.
    >
    > Check out my table sorting library here:http://www.JavascriptToolbox.com/lib/table/
    >
    > And is extremely fast and has a lot of functionality that you can
    > enable just by putting classes on your table.
    >
    > I am working on an update that will fix a few minor things and add
    > more, but the version that is there is working quite well for me in
    > several production projects.
    >
    > Matt Kruse
     
    Simon, Nov 5, 2007
    #3
  4. Simon

    Matt Kruse Guest

    On Nov 5, 11:06 am, Simon <> wrote:
    > This looks pretty promising, thanks. Are there ways to extend the
    > sorting? The table I am working on has other data types (buttons,
    > coloured divs) that I would need to sort on.


    Yes, but it's not documented (yet).

    If you want to define a custom sorting function, check out the Sort
    object where you can define more functions.

    If you want to define how a node in a cell is converted into a
    sortable value, check out the nodeValue object, where you can map
    elements (BUTTON, DIV, etc) to custom functions that will convert the
    node into some text to sort on.

    You are, of course, welcome to modify and customize the code as you
    wish to work with your project.

    Matt Kruse
     
    Matt Kruse, Nov 5, 2007
    #4
  5. In comp.lang.javascript message <
    oglegroups.com>, Mon, 5 Nov 2007 16:23:03, Simon
    <> posted:

    >I have a table I am populating with data from a DB, and have it
    >sortable on the client side. When the table exceeds around 300 rows,
    >I start running into problems with rendering and sorting it. I
    >started out using Prototype to help with the sorting, but it appears
    >that this results in a lot of calls to _getElementsByXPath, which are
    >taking up a large percentage of the rendering time. I believe this
    >call is occurring as the DOM is being parsed.


    ISTM that you should not sort the Table /in situ/; you should read the
    DB data into an array-like structure, sort that, and then write it into
    the Table. Only if the Table is directly editable (e.g. by having
    <input type=text> in cells) should there be any need for code to read
    the Table.

    >What is the general/most accepted manner of dealing with large data
    >sets (I am expecting some to grow into the thousands) in a table so
    >that it can be sorted client side? I am thinking of building a
    >javascript (JSON) object containing the data, and sorting based on
    >this.


    Yes, do that.

    Almost certainly, with that amount of data, you should if practical use
    Array.sort() rather than Array.sort(Function) - requirement, that each
    element of the data can be pre-processed to a string form which will
    sort as desired, and regenerated from that form. Otherwise, replace
    each element by {E:element, K:key) and write a simple comparison
    function that compares keys.

    You may not need to de-pre-process in full.

    <URL:http://www.merlyn.demon.co.uk/js-order.htm> has a simple speed
    demo for the effects of pre-processing. Using keys is slow, but could
    be advantageous for repeated sorting.

    It's a good idea to read the newsgroup c.l.j and its FAQ. See below.

    --
    (c) John Stockton, Surrey, UK. ?@merlyn.demon.co.uk Turnpike v6.05 IE 6
    news:comp.lang.javascript FAQ <URL:http://www.jibbering.com/faq/index.html>.
    <URL:http://www.merlyn.demon.co.uk/js-index.htm> jscr maths, dates, sources.
    <URL:http://www.merlyn.demon.co.uk/> TP/BP/Delphi/jscr/&c, FAQ items, links.
     
    Dr J R Stockton, Nov 5, 2007
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Eric Lilja
    Replies:
    9
    Views:
    369
    Old Wolf
    May 26, 2005
  2. nish
    Replies:
    1
    Views:
    333
  3. CMOS
    Replies:
    15
    Views:
    479
    James Kanze
    May 17, 2007
  4. Patrick  Sullivan
    Replies:
    6
    Views:
    286
    Carl Banks
    Sep 27, 2008
  5. Martin Pirker

    handling large data sets

    Martin Pirker, Dec 8, 2003, in forum: Ruby
    Replies:
    10
    Views:
    241
    Ian Hobson
    Dec 9, 2003
Loading...

Share This Page