Creating search engine in Python Good or Bad [in performance]

Discussion in 'Python' started by Clement, Apr 19, 2007.

  1. Clement

    Clement Guest

    I am now creating IR engine in python which has its own database
    implementation . For nearly 2GB files it works fast. Can i expect the
    same speed when my database goes large. Else i have to chose other
    language[c/c++] for the speed.............
    Please tell me the solution................................
     
    Clement, Apr 19, 2007
    #1
    1. Advertising

  2. Clement

    Larry Bates Guest

    Clement wrote:
    > I am now creating IR engine in python which has its own database
    > implementation . For nearly 2GB files it works fast. Can i expect the
    > same speed when my database goes large. Else i have to chose other
    > language[c/c++] for the speed.............
    > Please tell me the solution................................
    >


    Your speed will be good if your database design is good. We don't have
    a crystal ball out here to see into your database design. It is also
    not clear what the bottlenecks to "speed" would be for you. If it is
    disk I/O or LAN bandwidth C won't help at all.

    -Larry
     
    Larry Bates, Apr 19, 2007
    #2
    1. Advertising

  3. Clement

    azrael Guest

    That's right. It all depends on your design. When you do a select you
    could brute force it to to return the data, but the speed comes from
    elegant designing and programing skills. Using C++ or even Better C
    would probably boost your database model. Try to sort the data in the
    database so you can efficient use Hash-es


    On Apr 19, 8:46 pm, Larry Bates <> wrote:
    > Clement wrote:
    > > I am now creating IR engine in python which has its own database
    > > implementation . For nearly 2GB files it works fast. Can i expect the
    > > same speed when my database goes large. Else i have to chose other
    > > language[c/c++] for the speed.............
    > > Please tell me the solution................................

    >
    > Your speed will be good if your database design is good. We don't have
    > a crystal ball out here to see into your database design. It is also
    > not clear what the bottlenecks to "speed" would be for you. If it is
    > disk I/O or LAN bandwidth C won't help at all.
    >
    > -Larry
     
    azrael, Apr 19, 2007
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Eric Anderson

    Bad Transform or Bad Engine?

    Eric Anderson, Oct 4, 2005, in forum: XML
    Replies:
    1
    Views:
    378
    Peter Flynn
    Oct 5, 2005
  2. Sasha
    Replies:
    3
    Views:
    591
    Sasha
    May 22, 2007
  3. pandi
    Replies:
    5
    Views:
    455
    pandi
    Dec 14, 2009
  4. rantingrick
    Replies:
    44
    Views:
    1,226
    Peter Pearson
    Jul 13, 2010
  5. Jimmy Tran

    creating search engine

    Jimmy Tran, Sep 10, 2003, in forum: ASP General
    Replies:
    2
    Views:
    106
Loading...

Share This Page