How do they search ?

Discussion in 'Java' started by gk, Sep 25, 2005.

  1. gk

    gk Guest

    have you seen ebay ? they have a search option at the top . you can
    search almost anything in that search box. may be you could search
    DVD,Books,CD etc almost anything. or they have a single database table
    where all items are pushed ?


    Its a very powerful search .

    QUESTION is how do they search ?


    i am assuming they have a separate datbase table for DVD, they have
    Database table for CDS, they have database table for Books, they have
    database table for mobile phone etc etc.they have all individual tables
    for each catregory . that means they have hundreds of tables in thier
    database for each item category.


    Now , how do they going to search when you search for an item ?
    as a user can type anything he wish to search . how they are going to
    search ?

    can you tell what is the architecture for which a good search could be
    performed ?


    P.S : how do i track my posts ? today when i came back i did not find
    my old posts . it has been lost !!! does usenet dont have any facility
    so that i could save my posts ?
    gk, Sep 25, 2005
    #1
    1. Advertising

  2. gk

    Roedy Green Guest

    On 25 Sep 2005 09:08:04 -0700, "gk" <> wrote or quoted
    :

    >i am assuming they have a separate datbase table for DVD, they have
    >Database table for CDS, they have database table for Books, they have
    >database table for mobile phone etc etc.they have all individual tables
    >for each catregory . that means they have hundreds of tables in thier
    >database for each item category.


    They most likely do it with a giant combined index. there are several
    ways they could implement it: It is not that outlandish. Have ever
    seen "Books In Print"? You can pick it up in two hands. There are
    dictionaries fatter.

    1. they split the giant master index up alphabetically into 10 pieces
    and put each piece on a different server. The request is farmed out
    for each word in the search and then combined.

    2. they split the index up by type . e.g.. books, DVD, electronics and
    run each on a different server.

    If you are amazed how ebay/Amazon works, think about google which
    indexes the entire web.
    --
    Canadian Mind Products, Roedy Green.
    http://mindprod.com Again taking new Java programming contracts.
    Roedy Green, Sep 25, 2005
    #2
    1. Advertising

  3. gk

    Alan Krueger Guest

    gk wrote:
    > i am assuming they have a separate datbase table for DVD, they have
    > Database table for CDS, they have database table for Books, they have
    > database table for mobile phone etc etc.they have all individual tables
    > for each catregory . that means they have hundreds of tables in thier
    > database for each item category.


    Probably not, since you don't actually have to specify a single
    normalized type for anything you want to sell. Their categories are
    attributes of a listing, not the underlying type of the item. If they
    use a relational database, they most likely have all items going into a
    table for items for sale, a separate table for the categories, and a
    join table between these.
    Alan Krueger, Sep 25, 2005
    #3
  4. gk

    gk Guest

    Roedy Green wrote:
    > On 25 Sep 2005 09:08:04 -0700, "gk" <> wrote or quoted
    > :
    >
    > >i am assuming they have a separate datbase table for DVD, they have
    > >Database table for CDS, they have database table for Books, they have
    > >database table for mobile phone etc etc.they have all individual tables
    > >for each catregory . that means they have hundreds of tables in thier
    > >database for each item category.

    >
    > They most likely do it with a giant combined index. there are several
    > ways they could implement it: It is not that outlandish. Have ever
    > seen "Books In Print"? You can pick it up in two hands. There are
    > dictionaries fatter.
    >
    > 1. they split the giant master index up alphabetically into 10 pieces
    > and put each piece on a different server. The request is farmed out
    > for each word in the search and then combined.
    >
    > 2. they split the index up by type . e.g.. books, DVD, electronics and
    > run each on a different server.
    >
    > If you are amazed how ebay/Amazon works, think about google which
    > indexes the entire web.
    > --
    > Canadian Mind Products, Roedy Green.
    > http://mindprod.com Again taking new Java programming contracts.




    hi sir, i could not get you. whats that giant master index and blah
    blah . everthinhg is going out of my head. its very difficult. i am
    interested to know this. will you please tell a link or web page where
    i could get the things as like you are mentioning . its very interesting
    gk, Sep 26, 2005
    #4
  5. gk

    gk Guest

    Alan Krueger wrote:
    > gk wrote:
    > > i am assuming they have a separate datbase table for DVD, they have
    > > Database table for CDS, they have database table for Books, they have
    > > database table for mobile phone etc etc.they have all individual tables
    > > for each catregory . that means they have hundreds of tables in thier
    > > database for each item category.

    >
    > Probably not, since you don't actually have to specify a single
    > normalized type for anything you want to sell. Their categories are
    > attributes of a listing, not the underlying type of the item. If they
    > use a relational database, they most likely have all items going into a
    > table for items for sale, a separate table for the categories, and a
    > join table between these.



    will you please tell me your plan ? you mean , they have a table


    ebay_item_to_sale
    -------------------
    cds-James bond
    cds-michael jackson
    java book--bruce eckal



    category_table
    ---------------
    col: CD , DVD,Books,jewellary .


    something like this ?

    pleaase clarify more .


    one thing we must remember that an user is a whimsical one. anything
    he can input ! and he must get a search result.


    ohh, dont tell me about Google indexing. they are smart and big guys .
    i want to make a small web application .
    gk, Sep 26, 2005
    #5
  6. gk

    Roedy Green Guest

    On 25 Sep 2005 23:15:11 -0700, "gk" <> wrote or quoted
    :

    >hi sir, i could not get you. whats that giant master index and blah
    >blah . everthinhg is going out of my head. its very difficult. i am
    >interested to know this. will you please tell a link or web page where
    >i could get the things as like you are mentioning . its very interesting


    There is one search engine who publishes its code called Lucene. Have
    a look at a search engine called Dogpile. It has no databases of its
    own. It works by spawning requests to other databases. See
    http://mindprod.com/jgloss/searchengines.html
    --
    Canadian Mind Products, Roedy Green.
    http://mindprod.com Again taking new Java programming contracts.
    Roedy Green, Sep 26, 2005
    #6
  7. gk

    gk Guest

    sir, i dont understand why you are pointing to me to the web search ! i
    dont want to go for web search.

    i want in web site there will be a search box so, that user can type
    his/her interested products and get the result from my DB .

    this is quiet similar like ebay . ebay does not search from all over
    the web . it searches from its DB. some data entry operator puts those
    data in thire DB .

    so, question is ,
    how do they organise their DB so that an whimsical user could access
    anything whatever he likes from the DB.

    question is :

    what should be the table design ?

    and

    what should be the SQL query to fetch the data . of course, "LIKE"
    keyword in SQL query wont be of much help in this case.is not it ?


    i just need the flow how this thing is done. now adays many of the
    websites are doing this same thing.




    Roedy Green wrote:
    > On 25 Sep 2005 23:15:11 -0700, "gk" <> wrote or quoted
    > :
    >
    > >hi sir, i could not get you. whats that giant master index and blah
    > >blah . everthinhg is going out of my head. its very difficult. i am
    > >interested to know this. will you please tell a link or web page where
    > >i could get the things as like you are mentioning . its very interesting

    >
    > There is one search engine who publishes its code called Lucene. Have
    > a look at a search engine called Dogpile. It has no databases of its
    > own. It works by spawning requests to other databases. See
    > http://mindprod.com/jgloss/searchengines.html
    > --
    > Canadian Mind Products, Roedy Green.
    > http://mindprod.com Again taking new Java programming contracts.
    gk, Sep 26, 2005
    #7
  8. gk

    gk Guest

    How do i save my posts ? its getting lost . how do i track my posts ?
    there is no options like "My Topics" ?















    gk wrote:
    > sir, i dont understand why you are pointing to me to the web search ! i
    > dont want to go for web search.
    >
    > i want in web site there will be a search box so, that user can type
    > his/her interested products and get the result from my DB .
    >
    > this is quiet similar like ebay . ebay does not search from all over
    > the web . it searches from its DB. some data entry operator puts those
    > data in thire DB .
    >
    > so, question is ,
    > how do they organise their DB so that an whimsical user could access
    > anything whatever he likes from the DB.
    >
    > question is :
    >
    > what should be the table design ?
    >
    > and
    >
    > what should be the SQL query to fetch the data . of course, "LIKE"
    > keyword in SQL query wont be of much help in this case.is not it ?
    >
    >
    > i just need the flow how this thing is done. now adays many of the
    > websites are doing this same thing.
    >
    >
    >
    >
    > Roedy Green wrote:
    > > On 25 Sep 2005 23:15:11 -0700, "gk" <> wrote or quoted
    > > :
    > >
    > > >hi sir, i could not get you. whats that giant master index and blah
    > > >blah . everthinhg is going out of my head. its very difficult. i am
    > > >interested to know this. will you please tell a link or web page where
    > > >i could get the things as like you are mentioning . its very interesting

    > >
    > > There is one search engine who publishes its code called Lucene. Have
    > > a look at a search engine called Dogpile. It has no databases of its
    > > own. It works by spawning requests to other databases. See
    > > http://mindprod.com/jgloss/searchengines.html
    > > --
    > > Canadian Mind Products, Roedy Green.
    > > http://mindprod.com Again taking new Java programming contracts.
    gk, Sep 26, 2005
    #8
  9. gk wrote:
    > How do i save my posts ? its getting lost . how do i track my posts ?
    > there is no options like "My Topics" ?


    Please trim redundant text from your replies and consult
    the user documentation of your web interface[1] re the other stuff.

    [1] <http://groups.google.com/intl/en/googlegroups/about.html>

    A tip though, is that most posts are visible in one of many
    archives of the group[2], which you should be reading generally,
    rather than asking specific questions and expecting detailed
    personalised help, as you seem to be doing.

    [2] <http://groups.google.com/group/comp.lang.java.programmer>

    Also note that there is nothing in your latest post that
    is 'on-topic' for comp.lang.java.programmer.
    Andrew Thompson, Sep 26, 2005
    #9
  10. gk

    Roedy Green Guest

    On 26 Sep 2005 00:15:11 -0700, "gk" <> wrote or quoted
    :

    >sir, i dont understand why you are pointing to me to the web search ! i
    >dont want to go for web search.
    >
    >i want in web site there will be a search box so, that user can type
    >his/her interested products and get the result from my DB .


    The difference between what you are doing and what web search engines
    do, or Amazon does for books, or ebay does for dancing pandas is the
    content of your index and the scale. You were asking general
    questions about searching.

    For your particular project, you need to do some basic study of
    databases, servlets etc. See http://mindprod.com/jgloss/sql.html
    http://mindprod.com/jgloss/jbdc.html
    http://mindprod.com/jgloss/servlet.html
    http://mindprod.com/jgloss/cgi.html

    For a small site, just create a relational database of your products
    and compose SQL queries from the contents of the forms. You don't
    need to even think about how it all works. That is the SQL engine's
    problem. Only if you are trying to pull off something on a massive
    scale do you have to start thinking about multiple servers and custom
    indexing techniques.
    --
    Canadian Mind Products, Roedy Green.
    http://mindprod.com Again taking new Java programming contracts.
    Roedy Green, Sep 26, 2005
    #10
  11. gk

    Roedy Green Guest

    On 26 Sep 2005 00:24:13 -0700, "gk" <> wrote or quoted
    :

    >How do i save my posts ? its getting lost . how do i track my posts ?
    >there is no options like "My Topics" ?


    see http://mindprod.com/jgloss/newsreader.html

    --
    Canadian Mind Products, Roedy Green.
    http://mindprod.com Again taking new Java programming contracts.
    Roedy Green, Sep 26, 2005
    #11
  12. gk

    gk Guest

    that means i can not save my posts ! if the day after tomorrow i came
    back to read my old post i wont find it . ohh, is this the beauty of
    usenet :(
    gk, Sep 26, 2005
    #12
  13. gk wrote:
    > that means i can not save my posts !


    You must have read 'that' wrong, since you came to an utterly
    incorrect conclusion.

    Now that I have seemingly convinced you to trim *most*
    of what you are replying to, I'll ask you to leave just
    a little so readers can understand what you are replying
    to - much like I do here.

    >...if the day after tomorrow i came
    > back to read my old post i wont find it .


    Tell me - do you know how to 'save a web page'?
    If you could open that web page later, and it contained
    the text of your post, would that be what you need?

    > ohh, is this the beauty of usenet :(


    A lot of tools (like usenet) work a lot better for you
    after you get a 'few tricks up your sleeve' to help
    sort, track and organise posts. E.G. Usenet is a very
    useful source of archived information, I mean, searching
    information from 'saved posts'...
    Andrew Thompson, Sep 27, 2005
    #13
  14. gk

    Roedy Green Guest

    On 26 Sep 2005 07:18:10 -0700, "gk" <> wrote or quoted
    :

    >that means i can not save my posts ! if the day after tomorrow i came
    >back to read my old post i wont find it . ohh, is this the beauty of
    >usenet :(


    Let me repeat my advice.

    Get a newsreader. Then you can keep your posts, and anyone else's as
    long as you like.

    See http://mindprod.com/jgloss/newsreader.html

    Also groups.google.ca keeps archives.
    --
    Canadian Mind Products, Roedy Green.
    http://mindprod.com Again taking new Java programming contracts.
    Roedy Green, Sep 27, 2005
    #14
  15. gk

    gk Guest

    thanks for the responses.

    sir, i dont understand what u are talking about. i just want to save
    my posts and till now, i did not find any solution.

    couple of things , i am guessing from your comments

    1) do u want me to save the web page in my Hard Disk?

    2) do you want to me to copy the url of my post in a Notepad and save
    the Notepad ? so that later on i could use the url and open that post
    in the browser ?

    3) do you want me to bookmark my posts ?


    BUT ALL OF THESE ARE VERY VERY BAD PRACTICE

    yes, i saw the link

    http://mindprod.com/jgloss/newsreader.html

    but there are browsers and tools . i use firefox,ie,
    opera,netscape,mozila and plenty of browsers.
    but that does not give me the facility i was talking about.


    if you are using a forum, they provide a option like "My Posts"

    if you are using a yahoo group, they provide a option like posts in
    email. so most updated e-mail will reach to you.

    why usenet is so exceptional and unconvenient ? OF COURSE, its good for
    archieving but that does not mean, i can not track my own posts .


    I am very much upset because i am losing my posts. and i dont want to
    save those STATIC html pages in hard disk

    thank you






    i dont know what / how u keep track your posts
    gk, Sep 27, 2005
    #15
  16. gk

    Dag Sunde Guest

    "gk" <> wrote in message
    news:...
    >
    > thanks for the responses.
    >
    > sir, i dont understand what u are talking about. i just want to save
    > my posts and till now, i did not find any solution.
    >
    > couple of things , i am guessing from your comments
    >
    > 1) do u want me to save the web page in my Hard Disk?
    >
    > 2) do you want to me to copy the url of my post in a Notepad and save
    > the Notepad ? so that later on i could use the url and open that post
    > in the browser ?
    >
    > 3) do you want me to bookmark my posts ?
    >
    >
    > BUT ALL OF THESE ARE VERY VERY BAD PRACTICE


    No, you have misundestood all that he said.

    >
    > yes, i saw the link
    >
    > http://mindprod.com/jgloss/newsreader.html
    >
    > but there are browsers and tools . i use firefox,ie,
    > opera,netscape,mozila and plenty of browsers.
    > but that does not give me the facility i was talking about.
    >


    Yes, they do!

    He is NOT talking about BROWSERS at all!
    I think his point is:
    Stop using a BROWSER as a NEWSREADER.

    Use a Newsreader!
    NOTE!
    * A NewsReader have NOTHING to do with browsers.
    * A NewsReader is a separate application installed
    on your computer.
    * Some browsers come bundled with a newsreader, but
    that is not the browser itself.


    >
    > if you are using a forum, they provide a option like "My Posts"
    >


    That is HTML-pages viewed in a browser!

    > if you are using a yahoo group, they provide a option like posts in
    > email. so most updated e-mail will reach to you.


    That is HTML-pages viewed in a browser!
    You can use a NewsReader instead!

    >
    > why usenet is so exceptional and unconvenient ? OF COURSE, its good for
    > archieving but that does not mean, i can not track my own posts .


    That is because you use a browser-based interface to some web-server
    hosting archived newsgroups. That is HTML-pages viewed in a browser!

    You can use a NewsReader instead to read news from a News-Server!

    > I am very much upset because i am losing my posts. and i dont want to
    > save those STATIC html pages in hard disk


    If HTML is involved at all when you read newsgroups, you're doing
    something wrong

    --
    Dag.
    Dag Sunde, Sep 27, 2005
    #16
  17. gk

    Roedy Green Guest

    On 26 Sep 2005 22:35:13 -0700, "gk" <> wrote or quoted
    :

    >http://mindprod.com/jgloss/newsreader.html
    >
    >but there are browsers and tools . i use firefox,ie,
    >opera,netscape,mozila and plenty of browsers.
    >but that does not give me the facility i was talking about.


    A newsreader is a tool, just like a browser or a compiler.

    It stores copies of posts on your local hard disk. It fetches posts
    for you. It organises the posts in threads and lets you select what
    you want to read and who you want to filter out.

    It is much faster than trying to compose or read with a browser.
    It is as specialised tool for dealing with newsgroups.
    --
    Canadian Mind Products, Roedy Green.
    http://mindprod.com Again taking new Java programming contracts.
    Roedy Green, Sep 27, 2005
    #17
  18. gk

    Roedy Green Guest

    On 26 Sep 2005 22:35:13 -0700, "gk" <> wrote or quoted
    :

    >I am very much upset because i am losing my posts. and i dont want to
    >save those STATIC html pages in hard disk


    You keep asking questions then discarding the answers thinking you
    know more than the person who answered your question.

    For the third time. The answer to your problem is to download a
    newsreader. After you have done that and experimented for a few days,
    THEN come back and complain about my advice.
    ..
    see http://mindprod.com/jgloss/newsreader.html

    You don't even know what a newsreader is and yet you are pontificating
    to us why they are inadequate.
    --
    Canadian Mind Products, Roedy Green.
    http://mindprod.com Again taking new Java programming contracts.
    Roedy Green, Sep 27, 2005
    #18
  19. gk

    Virgil Green Guest

    gk wrote:

    <big snip>

    > why usenet is so exceptional


    <snip to intentionally remove context>

    Because only exceptional people are truly qualified to use usenet

    --
    Virgil
    Virgil Green, Sep 27, 2005
    #19
  20. gk

    gk Guest


    >If HTML is involved at all when you read newsgroups, you're doing
    >something wrong


    ok. it was my mistake. actually, i use firefox browser and then come to
    google and search my posts to read. so, i told it was inconvenient.


    >You keep asking questions then discarding the answers thinking you
    >know more than the person who answered your question.


    you are mistaken sir. all, i told in my posts was my findings. did not
    i mention my doubts in points ? i saw the link you provided and
    become confused and mentioned doubts in my posts.


    >You can use a NewsReader instead to read news from a News-Server!


    it was not known to me before . I have NOT usued usenet before . and i
    am inexperienced in this regard .may be, you are using usenet for a
    long time . so, it was easy for you but not to me.



    >>Let me repeat my advice.
    >>Get a newsreader. Then you can keep your posts, and anyone else's as
    >>long as you like.


    OK. thank you.
    gk, Sep 28, 2005
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Asfand Yar Qazi
    Replies:
    4
    Views:
    396
    Asfand Yar Qazi
    Nov 12, 2004
  2. Martin M.
    Replies:
    4
    Views:
    341
    Simon Brunning
    Dec 15, 2005
  3. Dave Rudolf
    Replies:
    1
    Views:
    300
    Kai-Uwe Bux
    May 17, 2006
  4. Abby Lee
    Replies:
    5
    Views:
    396
    Abby Lee
    Aug 2, 2004
  5. David A. Black
    Replies:
    2
    Views:
    223
    Tim Hunter
    Aug 19, 2004
Loading...

Share This Page