seaching a list...

Discussion in 'Python' started by bruce, Aug 10, 2006.

  1. bruce

    bruce Guest

    hi...

    i'm playing with a test sample. i have somethhing like:
    dog = mysql_get(.....)
    .
    .
    .

    such that 'dog' will be an 'AxB' array of data from the tbls

    furher in the test app, i'm going to have a list, foo:
    foo = 'a','b','c','d'

    i'm trying to determine what's the fastest way of searching through the
    'dog' array/list of information for the 'foo' list.

    should i essentially make dog into A lists, where each list is B elements,
    or should it somehow combine all the elements/items in 'dog' into one large
    list, and then search through that for the 'foo' list...

    also, in searching through google, i haven't come across the list.search
    function.. so just how do i search a list to see if it contains a sublist...

    my real problem involves figuring out how to reduce the number of hits to
    the db/tbl...

    thanks

    ps. if this is confusing, i could provide psuedo-code to make it easier to
    see...
    bruce, Aug 10, 2006
    #1
    1. Advertising

  2. bruce

    Simon Forman Guest

    bruce wrote:
    > hi...
    >
    > i'm playing with a test sample. i have somethhing like:
    > dog = mysql_get(.....)
    > .
    > .
    > .
    >
    > such that 'dog' will be an 'AxB' array of data from the tbls


    What's an "'AxB' array", do you mean a list of lists? If not, what
    kind of object do you mean and what methods does it have?

    >
    > furher in the test app, i'm going to have a list, foo:
    > foo = 'a','b','c','d'


    That's a tuple, not a list.

    >
    > i'm trying to determine what's the fastest way of searching through the
    > 'dog' array/list of information for the 'foo' list.
    >
    > should i essentially make dog into A lists, where each list is B elements,
    > or should it somehow combine all the elements/items in 'dog' into one large
    > list, and then search through that for the 'foo' list...


    This really depends on what kind of searching you want to do, and what
    kind of results you want.

    If dog is a list of lists, and foo might occur as one of the sublists,
    then you could do something like this:

    try:
    i = dog.index(foo)
    except ValueError:
    i = -1

    >
    > also, in searching through google, i haven't come across the list.search
    > function..


    That's because it doesn't exist.

    > so just how do i search a list to see if it contains a sublist...


    One thing that can do this is the difflib.SequenceMatcher() class.
    Read this: http://docs.python.org/lib/sequence-matcher.html

    |>> from difflib import SequenceMatcher
    |>> N = range(10)
    |>> M = range(3, 6)
    |>> s = SequenceMatcher(a=N, b=M)
    |>> c = len(M)
    |>> [i for i, j, n in s.get_matching_blocks() if n == c]
    [3]


    There may be better ways.

    >
    > my real problem involves figuring out how to reduce the number of hits to
    > the db/tbl...


    What?

    >
    > thanks
    >
    > ps. if this is confusing, i could provide psuedo-code to make it easier to
    > see...


    Yes, please.


    Peace,
    ~Simon
    Simon Forman, Aug 10, 2006
    #2
    1. Advertising

  3. bruce

    Larry Bates Guest

    bruce wrote:
    > hi...
    >
    > i'm playing with a test sample. i have somethhing like:
    > dog = mysql_get(.....)
    > .
    > .
    > .
    >
    > such that 'dog' will be an 'AxB' array of data from the tbls
    >
    > furher in the test app, i'm going to have a list, foo:
    > foo = 'a','b','c','d'
    >
    > i'm trying to determine what's the fastest way of searching through the
    > 'dog' array/list of information for the 'foo' list.
    >
    > should i essentially make dog into A lists, where each list is B elements,
    > or should it somehow combine all the elements/items in 'dog' into one large
    > list, and then search through that for the 'foo' list...
    >
    > also, in searching through google, i haven't come across the list.search
    > function.. so just how do i search a list to see if it contains a sublist...
    >
    > my real problem involves figuring out how to reduce the number of hits to
    > the db/tbl...
    >
    > thanks
    >
    > ps. if this is confusing, i could provide psuedo-code to make it easier to
    > see...
    >
    >
    >
    >

    You should use the database for what it is good at storing and searching
    through data. Don't read all the data from a table and search through it.
    Rather, create indexes on the table so that you can locate the data quickly
    in the database by passing in something you are looking for and let the
    database do the searching. I can promise you this will almost always be
    faster and more flexible. Something like:

    Assume the columns are called rownumber, c1, c2, c3, c4 and the table is
    indexed on c1, c2, c3, and c4. This will happen almost instantly no matter
    how many rows you are searching for.

    select rownumber from database_table
    where c1="a" and c2="b" and c3="c" and c5="d"

    It takes one "hit" to the db/table to return the rowindex that matches.

    -Larry Bates
    Larry Bates, Aug 11, 2006
    #3
  4. bruce

    John Machin Guest

    bruce wrote:
    > hi larry...
    >
    > thanks for the reply...
    >
    > the issue i'm having is that i'm going to have to compare multiple rows of
    > information to the information in the db. so essentially i'd have to do a
    > hit to the db, for each row of information i want to compare if i did it
    > your way... (which was what i had thought about)
    >
    > the issue of doing the string/list compare/search is that i can get
    > everything from the db with one call... i can then iterate through memory
    > for each of my row information that i'm searching to see if it exists in the
    > db...
    >
    > memory searches should be faster than the network overhead, and the
    > associated multiple db calls...


    (1) Don't top-post.
    (2) Do as asked: supply some pseudo-code for comparing a row of query
    information with a row extracted from your database -- whether you are
    comparing on one/some/all columns and what type of comparison are vital
    pieces of information. How many query rows? How many database rows?
    John Machin, Aug 11, 2006
    #4
  5. At Thursday 10/8/2006 21:54, bruce wrote:

    >the issue i'm having is that i'm going to have to compare multiple rows of
    >information to the information in the db. so essentially i'd have to do a
    >hit to the db, for each row of information i want to compare if i did it
    >your way... (which was what i had thought about)
    >
    >the issue of doing the string/list compare/search is that i can get
    >everything from the db with one call... i can then iterate through memory
    >for each of my row information that i'm searching to see if it exists in the
    >db...
    >
    >memory searches should be faster than the network overhead, and the
    >associated multiple db calls...


    should... are you sure?
    How many rows on the database? how many rows to compare? network overhead?
    Do some timing/performance tests to evaluate that. Things aren't
    always as you expect.



    Gabriel Genellina
    Softlab SRL





    __________________________________________________
    Preguntá. Respondé. Descubrí.
    Todo lo que querías saber, y lo que ni imaginabas,
    está en Yahoo! Respuestas (Beta).
    ¡Probalo ya!
    http://www.yahoo.com.ar/respuestas
    Gabriel Genellina, Aug 11, 2006
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. imran
    Replies:
    2
    Views:
    12,797
    Rhino
    Mar 3, 2005
  2. LittlePython

    Seaching Active Directory via ADO

    LittlePython, Feb 16, 2006, in forum: Python
    Replies:
    10
    Views:
    979
    LittlePython
    Feb 19, 2006
  3. ruds

    seaching a file

    ruds, Jul 13, 2007, in forum: Java
    Replies:
    6
    Views:
    362
    Andrew Thompson
    Jul 14, 2007
  4. KK

    Seaching the data

    KK, Jun 21, 2008, in forum: Java
    Replies:
    2
    Views:
    309
    Arne Vajhøj
    Jun 21, 2008
  5. hgbso

    seaching a txt file

    hgbso, Sep 6, 2008, in forum: C Programming
    Replies:
    0
    Views:
    316
    hgbso
    Sep 6, 2008
Loading...

Share This Page