Is it possible to pass CSV Reader Object As Argument to anotherPython File ???

Discussion in 'Python' started by bansi, Jan 26, 2011.

  1. bansi

    bansi Guest

    I have following two python scripts
    -namelookupWrapper.py
    -namelookup.py


    The namelookupWrapper.py takes input of "memberId", "memberName" from
    CLI and has following code snippet

    idf = sys.argv[1]
    namef = sys.argv[2]
    real_script = "C:\\Splunk\\etc\\apps\\search\\bin\\namelookup.py"
    r = csv.reader(sys.stdin)
    os.execv(python_executable, [ python_executable, real_script ] +
    sys.argv[1:] )



    Wondering how would i pass csv reader object "r" as an argument using
    os.execv() to another python script i.e. namelookup.py
     
    bansi, Jan 26, 2011
    #1
    1. Advertising

  2. On 1/26/2011 7:51 AM bansi said...
    > I have following two python scripts
    > -namelookupWrapper.py
    > -namelookup.py
    >
    >
    > The namelookupWrapper.py takes input of "memberId", "memberName" from
    > CLI and has following code snippet
    >
    > idf = sys.argv[1]
    > namef = sys.argv[2]
    > real_script = "C:\\Splunk\\etc\\apps\\search\\bin\\namelookup.py"
    > r = csv.reader(sys.stdin)
    > os.execv(python_executable, [ python_executable, real_script ] +
    > sys.argv[1:] )
    >
    >
    >
    > Wondering how would i pass csv reader object "r" as an argument using
    > os.execv() to another python script i.e. namelookup.py
    >



    I suspect you're on the wrong path. You probably want to import
    namelookup within namelooupWrapper to use the functions it defines.

    Consider:

    [root@fcfw2 src]# cat > test1.py

    def say(what): print what

    [root@fcfw2 src]# cat > test2.py

    #!/usr/local/bin/python
    import sys
    from test1 import say
    say(sys.argv[1])

    [root@fcfw2 src]# chmod a+x test2.py

    [root@fcfw2 src]# ./test2.py hello
    hello


    HTH,

    Emile
     
    Emile van Sebille, Jan 26, 2011
    #2
    1. Advertising

  3. bansi

    bansi Guest

    On Jan 26, 11:30 am, Emile van Sebille <> wrote:
    > On 1/26/2011 7:51 AM bansi said...
    >
    >
    >
    >
    >
    > > I have following two python scripts
    > > -namelookupWrapper.py
    > > -namelookup.py

    >
    > > The namelookupWrapper.py takes input of "memberId", "memberName" from
    > > CLI and has following code snippet

    >
    > > idf = sys.argv[1]
    > > namef = sys.argv[2]
    > > real_script = "C:\\Splunk\\etc\\apps\\search\\bin\\namelookup.py"
    > > r = csv.reader(sys.stdin)
    > > os.execv(python_executable, [ python_executable, real_script ] +
    > > sys.argv[1:] )

    >
    > > Wondering how would i pass csv reader object "r" as an argument using
    > > os.execv() to another python script i.e. namelookup.py

    >
    > I suspect you're on the wrong path.  You probably want to import
    > namelookup within namelooupWrapper to use the functions it defines.
    >
    > Consider:
    >
    > [root@fcfw2 src]# cat > test1.py
    >
    > def say(what): print what
    >
    > [root@fcfw2 src]# cat > test2.py
    >
    > #!/usr/local/bin/python
    > import sys
    > from test1 import say
    > say(sys.argv[1])
    >
    > [root@fcfw2 src]# chmod a+x test2.py
    >
    > [root@fcfw2 src]# ./test2.py hello
    > hello
    >
    > HTH,
    >
    > Emile- Hide quoted text -
    >
    > - Show quoted text -


    Emile,
    Thanks for quick response. I am not sure if "import
    namelookup within namelooupWrapper" helps because they are two
    independent scripts which has to be executed in sequence.
    First namelookupWrapper.py running under Python 2.6 accept arguments
    from stdin and uses csv reader object to read it i.e.
    r=csv.reader(sys.stdin)

    And then it has to pass csv reader object to another python script
    namelookup.py running under Python 2.7 because it uses pyodbc to
    connect to database and iterates thru reader object

    Any better ideas/suggestions will be greatly appreciated
     
    bansi, Jan 26, 2011
    #3
  4. bansi

    Chris Rebert Guest

    On Wed, Jan 26, 2011 at 7:51 AM, bansi <> wrote:
    > I have following two python scripts
    > -namelookupWrapper.py
    > -namelookup.py
    >
    >
    > The namelookupWrapper.py takes input of "memberId", "memberName" from
    > CLI and has following code snippet
    >
    > idf = sys.argv[1]
    > namef = sys.argv[2]
    > real_script = "C:\\Splunk\\etc\\apps\\search\\bin\\namelookup.py"
    > r = csv.reader(sys.stdin)
    > os.execv(python_executable, [ python_executable, real_script ] +
    > sys.argv[1:] )
    >
    > Wondering how would i pass csv reader object "r" as an argument using
    > os.execv() to another python script i.e. namelookup.py


    It's not possible to pass Python objects between processes in such a
    manner. Given that "independent" scripts can't directly take objects
    as input anyway, I doubt the two scripts are truly independent from
    each other. I would therefore concur with van Sebille that you should
    just rewrite them so that one script imports from the other rather
    than spawning the other. It should not be too hard to port the Python
    2.6 script to Python 2.7 (or vice-versa if necessary).

    Cheers,
    Chris
    --
    http://blog.rebertia.com
     
    Chris Rebert, Jan 26, 2011
    #4
  5. bansi

    bansi Guest

    On Jan 26, 1:31 pm, Chris Rebert <> wrote:
    > On Wed, Jan 26, 2011 at 7:51 AM, bansi <> wrote:
    > > I have following two python scripts
    > > -namelookupWrapper.py
    > > -namelookup.py

    >
    > > The namelookupWrapper.py takes input of "memberId", "memberName" from
    > > CLI and has following code snippet

    >
    > > idf = sys.argv[1]
    > > namef = sys.argv[2]
    > > real_script = "C:\\Splunk\\etc\\apps\\search\\bin\\namelookup.py"
    > > r = csv.reader(sys.stdin)
    > > os.execv(python_executable, [ python_executable, real_script ] +
    > > sys.argv[1:] )

    >
    > > Wondering how would i pass csv reader object "r" as an argument using
    > > os.execv() to another python script i.e. namelookup.py

    >
    > It's not possible to pass Python objects between processes in such a
    > manner. Given that "independent" scripts can't directly take objects
    > as input anyway, I doubt the two scripts are truly independent from
    > each other. I would therefore concur with van Sebille that you should
    > just rewrite them so that one script imports from the other rather
    > than spawning the other. It should not be too hard to port the Python
    > 2.6 script to Python 2.7 (or vice-versa if necessary).
    >
    > Cheers,
    > Chris
    > --http://blog.rebertia.com- Hide quoted text -
    >
    > - Show quoted text -


    Thanks Chris. Sorry for mis-communicating, the two python scripts are
    dependant in a way that
    namelookupWrapper.py needs to pass csv record object to another
    python script

    If thats not possible then please let me know how to do the workaround
    i didnt understood the import thing and not sure if it helps in my
    case

    Here are the details
    namelookupwrapper.py - takes input from stdin. Using csv reader object
    i iterate thru the input which looks like as shown below

    [MemberId, MemberName]
    [123, ]
    [456, ]
    [989, ]


    Now i have another script i.e. namelookup.py running under Python 2.7
    using pyodbc to retrieve Member Names from database for a given Member
    Id in namelooupWrapper.py

    So please let me know how to accomplish this
     
    bansi, Jan 26, 2011
    #5
  6. bansi

    bansi Guest

    On Jan 26, 4:36 pm, Ben Finney <> wrote:
    > bansi <> writes:
    > > Thanks Chris. Sorry for mis-communicating, the two python scripts are
    > > dependant in a way that namelookupWrapper.py needs to pass csv record
    > > object to another python script

    >
    > Why have you structured them that way, though? What constraint is
    > keeping you from doing the work in a single process, where the CSV
    > reader object can be shared?
    >
    > > If thats not possible then please let me know how to do the workaround
    > > i didnt understood the import thing and not sure if it helps in my
    > > case

    >
    > The problem as you've described it so far is best solved by having a
    > single process accessing the CSV reader object in memory. If that
    > doesn't suit your use case, you'll need to explain why not.
    >
    > --
    >  \       “To have the choice between proprietary software packages, is |
    >   `\      being able to choose your master. Freedom means not having a |
    > _o__)                        master.” —Richard M. Stallman, 2007-05-16 |
    > Ben Finney


    Thanks Ben for quick response. The constraint is in using third party
    tool called Splunk which has in-built Python 2.6 which doesnt support
    pyodbc on Windows 64 bit OS. Hence i have to install Python 2.7 for
    pyodbc.
    That means namelookupwrapper.py acts as a wrapper running under
    Splunk environment taking input from Splunk via stdin and storing it
    in csv reader object and then needs to call actual script
    namelookup.py under Python 2.7 for making connection to database

    Hope i have clarified a bit
     
    bansi, Jan 26, 2011
    #6
  7. bansi

    Ethan Furman Guest

    bansi wrote:
    > First namelookupWrapper.py running under Python 2.6 accept arguments
    > from stdin and uses csv reader object to read it i.e.
    > r=csv.reader(sys.stdin)
    >
    > And then it has to pass csv reader object to another python script
    > namelookup.py running under Python 2.7 because it uses pyodbc to
    > connect to database and iterates thru reader object


    Ben Finney wrote:
    > bansi <> writes:
    >
    >> Thanks Chris. Sorry for mis-communicating, the two python scripts are
    >> dependant in a way that namelookupWrapper.py needs to pass csv record
    >> object to another python script

    >
    > Why have you structured them that way, though? What constraint is
    > keeping you from doing the work in a single process, where the CSV
    > reader object can be shared?
    >
    >> If thats not possible then please let me know how to do the workaround
    >> i didnt understood the import thing and not sure if it helps in my
    >> case

    >
    > The problem as you've described it so far is best solved by having a
    > single process accessing the CSV reader object in memory. If that
    > doesn't suit your use case, you'll need to explain why not.
    >



    In other words, why can't you use Python 2.7 to accept input and
    generate a csv.reader?

    ~Ethan~
     
    Ethan Furman, Jan 26, 2011
    #7
  8. bansi

    bansi Guest

    On Jan 26, 6:25 pm, Ethan Furman <> wrote:
    > bansi wrote:
    >
    >  > First namelookupWrapper.py running under Python 2.6 accept arguments
    >  > from stdin and uses csv reader object to read it i.e.
    >  > r=csv.reader(sys.stdin)
    >  >
    >  > And then it has to pass csv reader object to another python script
    >  > namelookup.py running under Python 2.7 because it uses pyodbc to
    >  > connect to database and iterates thru reader object
    >
    >
    >
    >
    >
    > Ben Finney wrote:
    > > bansi <> writes:

    >
    > >> Thanks Chris. Sorry for mis-communicating, the two python scripts are
    > >> dependant in a way that namelookupWrapper.py needs to pass csv record
    > >> object to another python script

    >
    > > Why have you structured them that way, though? What constraint is
    > > keeping you from doing the work in a single process, where the CSV
    > > reader object can be shared?

    >
    > >> If thats not possible then please let me know how to do the workaround
    > >> i didnt understood the import thing and not sure if it helps in my
    > >> case

    >
    > > The problem as you've described it so far is best solved by having a
    > > single process accessing the CSV reader object in memory. If that
    > > doesn't suit your use case, you'll need to explain why not.

    >
    > In other words, why can't you use Python 2.7 to accept input and
    > generate a csv.reader?
    >
    > ~Ethan~- Hide quoted text -
    >
    > - Show quoted text -


    Ethan,
    The python script takes the input from Splunk (http://www.splunk.com/
    base/Documentation/) which supports only Python 2.6
    So the real constraint is Splunk supports only Python 2.6 .

    As you know Python 2.6 doesnt support or doesnt have pyodbc install
    for Windows 64 bit OS
    So i installed Python 2.7 and thereafter pyodbc install for Windows 64
    bit OS for Python 2.7
     
    bansi, Jan 27, 2011
    #8
  9. bansi

    MRAB Guest

    On 27/01/2011 00:57, bansi wrote:
    > On Jan 26, 6:25 pm, Ethan Furman<> wrote:
    >> bansi wrote:
    >>
    >> > First namelookupWrapper.py running under Python 2.6 accept arguments
    >> > from stdin and uses csv reader object to read it i.e.
    >> > r=csv.reader(sys.stdin)
    >> >
    >> > And then it has to pass csv reader object to another python script
    >> > namelookup.py running under Python 2.7 because it uses pyodbc to
    >> > connect to database and iterates thru reader object

    >>
    >>
    >>
    >>
    >>
    >> Ben Finney wrote:
    >>> bansi<> writes:

    >>
    >>>> Thanks Chris. Sorry for mis-communicating, the two python scripts are
    >>>> dependant in a way that namelookupWrapper.py needs to pass csv record
    >>>> object to another python script

    >>
    >>> Why have you structured them that way, though? What constraint is
    >>> keeping you from doing the work in a single process, where the CSV
    >>> reader object can be shared?

    >>
    >>>> If thats not possible then please let me know how to do the workaround
    >>>> i didnt understood the import thing and not sure if it helps in my
    >>>> case

    >>
    >>> The problem as you've described it so far is best solved by having a
    >>> single process accessing the CSV reader object in memory. If that
    >>> doesn't suit your use case, you'll need to explain why not.

    >>
    >> In other words, why can't you use Python 2.7 to accept input and
    >> generate a csv.reader?
    >>
    >> ~Ethan~- Hide quoted text -
    >>
    >> - Show quoted text -

    >
    > Ethan,
    > The python script takes the input from Splunk (http://www.splunk.com/
    > base/Documentation/) which supports only Python 2.6
    > So the real constraint is Splunk supports only Python 2.6 .
    >
    > As you know Python 2.6 doesnt support or doesnt have pyodbc install
    > for Windows 64 bit OS
    > So i installed Python 2.7 and thereafter pyodbc install for Windows 64
    > bit OS for Python 2.7


    Have you actually tried Splunk with Python 2.7? It might not work with
    versions which are earlier than Python 2.6, but that doesn't
    necessarily mean that it won't work with versions of Python 2 which are
    later than Python 2.6 (unless the documentation says that it must be
    Python 2.6).
     
    MRAB, Jan 27, 2011
    #9
  10. bansi

    bansi Guest

    On Jan 26, 8:31 pm, MRAB <> wrote:
    > On 27/01/2011 00:57, bansi wrote:
    >
    >
    >
    >
    >
    > > On Jan 26, 6:25 pm, Ethan Furman<>  wrote:
    > >> bansi wrote:

    >
    > >>   >  First namelookupWrapper.py running under Python 2.6 accept arguments
    > >>   >  from stdin and uses csv reader object to read it i.e.
    > >>   >  r=csv.reader(sys.stdin)

    >
    > >>   >  And then it has to pass csv reader object to another python script
    > >>   >  namelookup.py running under Python 2.7 because it uses pyodbc to
    > >>   >  connect to database and iterates thru reader object

    >
    > >> Ben Finney wrote:
    > >>> bansi<>  writes:

    >
    > >>>> Thanks Chris. Sorry for mis-communicating, the two python scripts are
    > >>>> dependant in a way that namelookupWrapper.py needs to pass csv record
    > >>>> object to another python script

    >
    > >>> Why have you structured them that way, though? What constraint is
    > >>> keeping you from doing the work in a single process, where the CSV
    > >>> reader object can be shared?

    >
    > >>>> If thats not possible then please let me know how to do the workaround
    > >>>> i didnt understood the import thing and not sure if it helps in my
    > >>>> case

    >
    > >>> The problem as you've described it so far is best solved by having a
    > >>> single process accessing the CSV reader object in memory. If that
    > >>> doesn't suit your use case, you'll need to explain why not.

    >
    > >> In other words, why can't you use Python 2.7 to accept input and
    > >> generate a csv.reader?

    >
    > >> ~Ethan~- Hide quoted text -

    >
    > >> - Show quoted text -

    >
    > > Ethan,
    > > The python script takes the input from Splunk (http://www.splunk.com/
    > > base/Documentation/) which supports only Python 2.6
    > > So the real constraint is Splunk supports only Python 2.6 .

    >
    > > As you know Python 2.6 doesnt support or doesnt have pyodbc install
    > > for Windows  64 bit OS
    > > So i installed Python 2.7 and thereafter pyodbc install for Windows 64
    > > bit OS for Python 2.7

    >
    > Have you actually tried Splunk with Python 2.7? It might not work with
    > versions which are earlier than Python 2.6, but that doesn't
    > necessarily mean that it won't work with versions of Python 2 which are
    > later than Python 2.6 (unless the documentation says that it must be
    > Python 2.6).- Hide quoted text -
    >
    > - Show quoted text -


    Splunk's latest version 4.1.6 doesn't support Python 2.7
    I tried the import trick but it didnt work because the real script
    which runs under Python 2.7 has import pyodbc so it results in
    following error

    c:\Splunk\etc\apps\search\bin>splunk cmd python namelookupWrapper.py
    memberId memberName < memberInput.csv
    Traceback (most recent call last):
    File "namelookupWrapper.py", line 3, in <module>
    import namelookup
    File "c:\Splunk\etc\apps\search\bin\namelookup.py", line 7, in
    <module>
    import pyodbc
    ImportError: DLL load failed: The specified module could not be found.

    Please let me know if i am missing something on import. If so please
    provide me with an example
     
    bansi, Jan 28, 2011
    #10
  11. bansi

    bansi Guest

    On Jan 28, 9:46 am, bansi <> wrote:
    > On Jan 26, 8:31 pm, MRAB <> wrote:
    >
    >
    >
    >
    >
    > > On 27/01/2011 00:57, bansi wrote:

    >
    > > > On Jan 26, 6:25 pm, Ethan Furman<>  wrote:
    > > >> bansi wrote:

    >
    > > >>   >  First namelookupWrapper.py running under Python 2.6 accept arguments
    > > >>   >  from stdin and uses csv reader object to read it i.e.
    > > >>   >  r=csv.reader(sys.stdin)

    >
    > > >>   >  And then it has to pass csv reader object to another python script
    > > >>   >  namelookup.py running under Python 2.7 because it uses pyodbc to
    > > >>   >  connect to database and iterates thru reader object

    >
    > > >> Ben Finney wrote:
    > > >>> bansi<>  writes:

    >
    > > >>>> Thanks Chris. Sorry for mis-communicating, the two python scripts are
    > > >>>> dependant in a way that namelookupWrapper.py needs to pass csv record
    > > >>>> object to another python script

    >
    > > >>> Why have you structured them that way, though? What constraint is
    > > >>> keeping you from doing the work in a single process, where the CSV
    > > >>> reader object can be shared?

    >
    > > >>>> If thats not possible then please let me know how to do the workaround
    > > >>>> i didnt understood the import thing and not sure if it helps in my
    > > >>>> case

    >
    > > >>> The problem as you've described it so far is best solved by having a
    > > >>> single process accessing the CSV reader object in memory. If that
    > > >>> doesn't suit your use case, you'll need to explain why not.

    >
    > > >> In other words, why can't you use Python 2.7 to accept input and
    > > >> generate a csv.reader?

    >
    > > >> ~Ethan~- Hide quoted text -

    >
    > > >> - Show quoted text -

    >
    > > > Ethan,
    > > > The python script takes the input from Splunk (http://www.splunk.com/
    > > > base/Documentation/) which supports only Python 2.6
    > > > So the real constraint is Splunk supports only Python 2.6 .

    >
    > > > As you know Python 2.6 doesnt support or doesnt have pyodbc install
    > > > for Windows  64 bit OS
    > > > So i installed Python 2.7 and thereafter pyodbc install for Windows 64
    > > > bit OS for Python 2.7

    >
    > > Have you actually tried Splunk with Python 2.7? It might not work with
    > > versions which are earlier than Python 2.6, but that doesn't
    > > necessarily mean that it won't work with versions of Python 2 which are
    > > later than Python 2.6 (unless the documentation says that it must be
    > > Python 2.6).- Hide quoted text -

    >
    > > - Show quoted text -

    >
    > Splunk's latest version 4.1.6 doesn't support Python 2.7
    > I tried the import trick but it didnt work because the real script
    > which runs under Python 2.7 has import pyodbc so it results in
    > following error
    >
    > c:\Splunk\etc\apps\search\bin>splunk cmd python namelookupWrapper.py
    > memberId memberName < memberInput.csv
    > Traceback (most recent call last):
    >   File "namelookupWrapper.py", line 3, in <module>
    >     import namelookup
    >   File "c:\Splunk\etc\apps\search\bin\namelookup.py", line 7, in
    > <module>
    >     import pyodbc
    > ImportError: DLL load failed: The specified module could not be found.
    >
    > Please let me know if i am missing something on import. If so please
    > provide me with an example- Hide quoted text -
    >
    > - Show quoted text -


    Here are some more details from my earlier posting. Please click the
    below link

    http://answers.splunk.com/questions/11145/its-getting-mysterious-to-make-the-lookup-script-work
     
    bansi, Jan 28, 2011
    #11
  12. On Fri, Jan 28, 2011 at 1:33 PM, bansi <> wrote:
    > On Jan 28, 9:46 am, bansi <> wrote:
    >> On Jan 26, 8:31 pm, MRAB <> wrote:
    >>
    >>
    >>
    >>
    >>
    >> > On 27/01/2011 00:57, bansi wrote:

    >>
    >> > > On Jan 26, 6:25 pm, Ethan Furman<>  wrote:
    >> > >> bansi wrote:

    >>
    >> > >>   >  First namelookupWrapper.py running under Python 2.6 accept arguments
    >> > >>   >  from stdin and uses csv reader object to read it i.e.
    >> > >>   >  r=csv.reader(sys.stdin)

    >>
    >> > >>   >  And then it has to pass csv reader object to another python script
    >> > >>   >  namelookup.py running under Python 2.7 because it uses pyodbc to
    >> > >>   >  connect to database and iterates thru reader object

    >>
    >> > >> Ben Finney wrote:
    >> > >>> bansi<>  writes:

    >>
    >> > >>>> Thanks Chris. Sorry for mis-communicating, the two python scripts are
    >> > >>>> dependant in a way that namelookupWrapper.py needs to pass csv record
    >> > >>>> object to another python script

    >>
    >> > >>> Why have you structured them that way, though? What constraint is
    >> > >>> keeping you from doing the work in a single process, where the CSV
    >> > >>> reader object can be shared?

    >>
    >> > >>>> If thats not possible then please let me know how to do the workaround
    >> > >>>> i didnt understood the import thing and not sure if it helps in my
    >> > >>>> case

    >>
    >> > >>> The problem as you've described it so far is best solved by having a
    >> > >>> single process accessing the CSV reader object in memory. If that
    >> > >>> doesn't suit your use case, you'll need to explain why not.

    >>
    >> > >> In other words, why can't you use Python 2.7 to accept input and
    >> > >> generate a csv.reader?

    >>
    >> > >> ~Ethan~- Hide quoted text -

    >>
    >> > >> - Show quoted text -

    >>
    >> > > Ethan,
    >> > > The python script takes the input from Splunk (http://www.splunk.com/
    >> > > base/Documentation/) which supports only Python 2.6
    >> > > So the real constraint is Splunk supports only Python 2.6 .

    >>
    >> > > As you know Python 2.6 doesnt support or doesnt have pyodbc install
    >> > > for Windows  64 bit OS
    >> > > So i installed Python 2.7 and thereafter pyodbc install for Windows 64
    >> > > bit OS for Python 2.7

    >>
    >> > Have you actually tried Splunk with Python 2.7? It might not work with
    >> > versions which are earlier than Python 2.6, but that doesn't
    >> > necessarily mean that it won't work with versions of Python 2 which are
    >> > later than Python 2.6 (unless the documentation says that it must be
    >> > Python 2.6).- Hide quoted text -

    >>
    >> > - Show quoted text -

    >>
    >> Splunk's latest version 4.1.6 doesn't support Python 2.7
    >> I tried the import trick but it didnt work because the real script
    >> which runs under Python 2.7 has import pyodbc so it results in
    >> following error
    >>
    >> c:\Splunk\etc\apps\search\bin>splunk cmd python namelookupWrapper.py
    >> memberId memberName < memberInput.csv
    >> Traceback (most recent call last):
    >>   File "namelookupWrapper.py", line 3, in <module>
    >>     import namelookup
    >>   File "c:\Splunk\etc\apps\search\bin\namelookup.py", line 7, in
    >> <module>
    >>     import pyodbc
    >> ImportError: DLL load failed: The specified module could not be found.
    >>
    >> Please let me know if i am missing something on import. If so please
    >> provide me with an example- Hide quoted text -
    >>
    >> - Show quoted text -

    >
    > Here are some more details from my earlier posting. Please click the
    > below link
    >
    > http://answers.splunk.com/questions/11145/its-getting-mysterious-to-make-the-lookup-script-work
    > --
    > http://mail.python.org/mailman/listinfo/python-list
    >


    Have you tried downloading the source for PyODBC and compiling it
    yourself? All you need to do is python setup.py install. My guess
    would be that it works just fine on 64-bit Python 2.6, they just never
    released a re-compiled version of it for that platform.
     
    Benjamin Kaplan, Jan 28, 2011
    #12
  13. bansi

    bansi Guest

    On Jan 28, 1:52 pm, Benjamin Kaplan <> wrote:
    > On Fri, Jan 28, 2011 at 1:33 PM, bansi <> wrote:
    > > On Jan 28, 9:46 am, bansi <> wrote:
    > >> On Jan 26, 8:31 pm, MRAB <> wrote:

    >
    > >> > On 27/01/2011 00:57, bansi wrote:

    >
    > >> > > On Jan 26, 6:25 pm, Ethan Furman<>  wrote:
    > >> > >> bansi wrote:

    >
    > >> > >>   >  First namelookupWrapper.py running under Python 2.6 accept arguments
    > >> > >>   >  from stdin and uses csv reader object to read it i.e.
    > >> > >>   >  r=csv.reader(sys.stdin)

    >
    > >> > >>   >  And then it has to pass csv reader object to another python script
    > >> > >>   >  namelookup.py running under Python 2.7 because it uses pyodbc to
    > >> > >>   >  connect to database and iterates thru reader object

    >
    > >> > >> Ben Finney wrote:
    > >> > >>> bansi<>  writes:

    >
    > >> > >>>> Thanks Chris. Sorry for mis-communicating, the two python scripts are
    > >> > >>>> dependant in a way that namelookupWrapper.py needs to pass csv record
    > >> > >>>> object to another python script

    >
    > >> > >>> Why have you structured them that way, though? What constraint is
    > >> > >>> keeping you from doing the work in a single process, where the CSV
    > >> > >>> reader object can be shared?

    >
    > >> > >>>> If thats not possible then please let me know how to do the workaround
    > >> > >>>> i didnt understood the import thing and not sure if it helps in my
    > >> > >>>> case

    >
    > >> > >>> The problem as you've described it so far is best solved by having a
    > >> > >>> single process accessing the CSV reader object in memory. If that
    > >> > >>> doesn't suit your use case, you'll need to explain why not.

    >
    > >> > >> In other words, why can't you use Python 2.7 to accept input and
    > >> > >> generate a csv.reader?

    >
    > >> > >> ~Ethan~- Hide quoted text -

    >
    > >> > >> - Show quoted text -

    >
    > >> > > Ethan,
    > >> > > The python script takes the input from Splunk (http://www.splunk.com/
    > >> > > base/Documentation/) which supports only Python 2.6
    > >> > > So the real constraint is Splunk supports only Python 2.6 .

    >
    > >> > > As you know Python 2.6 doesnt support or doesnt have pyodbc install
    > >> > > for Windows  64 bit OS
    > >> > > So i installed Python 2.7 and thereafter pyodbc install for Windows 64
    > >> > > bit OS for Python 2.7

    >
    > >> > Have you actually tried Splunk with Python 2.7? It might not work with
    > >> > versions which are earlier than Python 2.6, but that doesn't
    > >> > necessarily mean that it won't work with versions of Python 2 which are
    > >> > later than Python 2.6 (unless the documentation says that it must be
    > >> > Python 2.6).- Hide quoted text -

    >
    > >> > - Show quoted text -

    >
    > >> Splunk's latest version 4.1.6 doesn't support Python 2.7
    > >> I tried the import trick but it didnt work because the real script
    > >> which runs under Python 2.7 has import pyodbc so it results in
    > >> following error

    >
    > >> c:\Splunk\etc\apps\search\bin>splunk cmd python namelookupWrapper.py
    > >> memberId memberName < memberInput.csv
    > >> Traceback (most recent call last):
    > >>   File "namelookupWrapper.py", line 3, in <module>
    > >>     import namelookup
    > >>   File "c:\Splunk\etc\apps\search\bin\namelookup.py", line 7, in
    > >> <module>
    > >>     import pyodbc
    > >> ImportError: DLL load failed: The specified module could not be found.

    >
    > >> Please let me know if i am missing something on import. If so please
    > >> provide me with an example- Hide quoted text -

    >
    > >> - Show quoted text -

    >
    > > Here are some more details from my earlier posting. Please click the
    > > below link

    >
    > >http://answers.splunk.com/questions/11145/its-getting-mysterious-to-m...
    > > --
    > >http://mail.python.org/mailman/listinfo/python-list

    >
    > Have you tried downloading the source for PyODBC and compiling it
    > yourself? All you need to do is python setup.py install. My guess
    > would be that it works just fine on 64-bit Python 2.6, they just never
    > released a re-compiled version of it for that platform.- Hide quoted text -
    >
    > - Show quoted text -


    Thanks Benjamin. Please point me to the website from where i can
    download pyodbc for Windows 64 bit OS under Python 2.6 and
    installation instructions
     
    bansi, Jan 28, 2011
    #13
  14. On Fri, Jan 28, 2011 at 3:42 PM, bansi <> wrote:
    > On Jan 28, 1:52 pm, Benjamin Kaplan <> wrote:
    >> On Fri, Jan 28, 2011 at 1:33 PM, bansi <> wrote:
    >> > On Jan 28, 9:46 am, bansi <> wrote:
    >> >> On Jan 26, 8:31 pm, MRAB <> wrote:

    >>
    >> >> > On 27/01/2011 00:57, bansi wrote:

    >>
    >> >> > > On Jan 26, 6:25 pm, Ethan Furman<>  wrote:
    >> >> > >> bansi wrote:

    >>
    >> >> > >>   >  First namelookupWrapper.py running under Python 2.6 accept arguments
    >> >> > >>   >  from stdin and uses csv reader object to read it i.e.
    >> >> > >>   >  r=csv.reader(sys.stdin)

    >>
    >> >> > >>   >  And then it has to pass csv reader object to another python script
    >> >> > >>   >  namelookup.py running under Python 2.7 because it uses pyodbc to
    >> >> > >>   >  connect to database and iterates thru reader object

    >>
    >> >> > >> Ben Finney wrote:
    >> >> > >>> bansi<>  writes:

    >>
    >> >> > >>>> Thanks Chris. Sorry for mis-communicating, the two python scripts are
    >> >> > >>>> dependant in a way that namelookupWrapper.py needs to pass csv record
    >> >> > >>>> object to another python script

    >>
    >> >> > >>> Why have you structured them that way, though? What constraint is
    >> >> > >>> keeping you from doing the work in a single process, where the CSV
    >> >> > >>> reader object can be shared?

    >>
    >> >> > >>>> If thats not possible then please let me know how to do the workaround
    >> >> > >>>> i didnt understood the import thing and not sure if it helps in my
    >> >> > >>>> case

    >>
    >> >> > >>> The problem as you've described it so far is best solved by having a
    >> >> > >>> single process accessing the CSV reader object in memory. If that
    >> >> > >>> doesn't suit your use case, you'll need to explain why not.

    >>
    >> >> > >> In other words, why can't you use Python 2.7 to accept input and
    >> >> > >> generate a csv.reader?

    >>
    >> >> > >> ~Ethan~- Hide quoted text -

    >>
    >> >> > >> - Show quoted text -

    >>
    >> >> > > Ethan,
    >> >> > > The python script takes the input from Splunk (http://www.splunk.com/
    >> >> > > base/Documentation/) which supports only Python 2.6
    >> >> > > So the real constraint is Splunk supports only Python 2.6 .

    >>
    >> >> > > As you know Python 2.6 doesnt support or doesnt have pyodbc install
    >> >> > > for Windows  64 bit OS
    >> >> > > So i installed Python 2.7 and thereafter pyodbc install for Windows 64
    >> >> > > bit OS for Python 2.7

    >>
    >> >> > Have you actually tried Splunk with Python 2.7? It might not work with
    >> >> > versions which are earlier than Python 2.6, but that doesn't
    >> >> > necessarily mean that it won't work with versions of Python 2 which are
    >> >> > later than Python 2.6 (unless the documentation says that it must be
    >> >> > Python 2.6).- Hide quoted text -

    >>
    >> >> > - Show quoted text -

    >>
    >> >> Splunk's latest version 4.1.6 doesn't support Python 2.7
    >> >> I tried the import trick but it didnt work because the real script
    >> >> which runs under Python 2.7 has import pyodbc so it results in
    >> >> following error

    >>
    >> >> c:\Splunk\etc\apps\search\bin>splunk cmd python namelookupWrapper.py
    >> >> memberId memberName < memberInput.csv
    >> >> Traceback (most recent call last):
    >> >>   File "namelookupWrapper.py", line 3, in <module>
    >> >>     import namelookup
    >> >>   File "c:\Splunk\etc\apps\search\bin\namelookup.py", line 7, in
    >> >> <module>
    >> >>     import pyodbc
    >> >> ImportError: DLL load failed: The specified module could not be found..

    >>
    >> >> Please let me know if i am missing something on import. If so please
    >> >> provide me with an example- Hide quoted text -

    >>
    >> >> - Show quoted text -

    >>
    >> > Here are some more details from my earlier posting. Please click the
    >> > below link

    >>
    >> >http://answers.splunk.com/questions/11145/its-getting-mysterious-to-m....
    >> > --
    >> >http://mail.python.org/mailman/listinfo/python-list

    >>
    >> Have you tried downloading the source for PyODBC and compiling it
    >> yourself? All you need to do is python setup.py install. My guess
    >> would be that it works just fine on 64-bit Python 2.6, they just never
    >> released a re-compiled version of it for that platform.- Hide quoted text -
    >>
    >> - Show quoted text -

    >
    > Thanks Benjamin. Please point me to the website from where i can
    > download pyodbc for Windows 64 bit OS under Python 2.6 and
    > installation instructions
    > --


    You don't download it for 64-bit Windows with Python 2.6. You download
    the source code from the website and make the Python 2.6, 64-bit
    Windows version yourself.

    Download the source zip file and extract it. Then, open up the command
    prompt and use the "cd" command to change directories to that source
    folder. For instance, if the source code has been extracted to
    C:\pyodbc-2.1.8\, you type in "cd C:\pyodbc-2.1.8" and press enter.

    After that, you just run the build script which is already in there:

    C:\Python26\python26.exe setup.py install

    You'll need to have Visual C++ 2008 (not 2010) installed for this to
    work. You can get it for free from
    http://www.microsoft.com/express/Downloads/ if you don't already have
    it.


    > http://mail.python.org/mailman/listinfo/python-list
    >
     
    Benjamin Kaplan, Jan 28, 2011
    #14
  15. bansi

    bansi Guest

    On Jan 28, 4:22 pm, Benjamin Kaplan <> wrote:
    > On Fri, Jan 28, 2011 at 3:42 PM, bansi <> wrote:
    > > On Jan 28, 1:52 pm, Benjamin Kaplan <> wrote:
    > >> On Fri, Jan 28, 2011 at 1:33 PM, bansi <> wrote:
    > >> > On Jan 28, 9:46 am, bansi <> wrote:
    > >> >> On Jan 26, 8:31 pm, MRAB <> wrote:

    >
    > >> >> > On 27/01/2011 00:57, bansi wrote:

    >
    > >> >> > > On Jan 26, 6:25 pm, Ethan Furman<>  wrote:
    > >> >> > >> bansi wrote:

    >
    > >> >> > >>   >  First namelookupWrapper.py running under Python 2.6 accept arguments
    > >> >> > >>   >  from stdin and uses csv reader object to read it i.e.
    > >> >> > >>   >  r=csv.reader(sys.stdin)

    >
    > >> >> > >>   >  And then it has to pass csv reader object to another python script
    > >> >> > >>   >  namelookup.py running under Python 2.7 because it uses pyodbc to
    > >> >> > >>   >  connect to database and iterates thru reader object

    >
    > >> >> > >> Ben Finney wrote:
    > >> >> > >>> bansi<>  writes:

    >
    > >> >> > >>>> Thanks Chris. Sorry for mis-communicating, the two python scripts are
    > >> >> > >>>> dependant in a way that namelookupWrapper.py needs to pass csv record
    > >> >> > >>>> object to another python script

    >
    > >> >> > >>> Why have you structured them that way, though? What constraint is
    > >> >> > >>> keeping you from doing the work in a single process, where the CSV
    > >> >> > >>> reader object can be shared?

    >
    > >> >> > >>>> If thats not possible then please let me know how to do the workaround
    > >> >> > >>>> i didnt understood the import thing and not sure if it helps in my
    > >> >> > >>>> case

    >
    > >> >> > >>> The problem as you've described it so far is best solved by having a
    > >> >> > >>> single process accessing the CSV reader object in memory. If that
    > >> >> > >>> doesn't suit your use case, you'll need to explain why not.

    >
    > >> >> > >> In other words, why can't you use Python 2.7 to accept input and
    > >> >> > >> generate a csv.reader?

    >
    > >> >> > >> ~Ethan~- Hide quoted text -

    >
    > >> >> > >> - Show quoted text -

    >
    > >> >> > > Ethan,
    > >> >> > > The python script takes the input from Splunk (http://www.splunk.com/
    > >> >> > > base/Documentation/) which supports only Python 2.6
    > >> >> > > So the real constraint is Splunk supports only Python 2.6 .

    >
    > >> >> > > As you know Python 2.6 doesnt support or doesnt have pyodbc install
    > >> >> > > for Windows  64 bit OS
    > >> >> > > So i installed Python 2.7 and thereafter pyodbc install for Windows 64
    > >> >> > > bit OS for Python 2.7

    >
    > >> >> > Have you actually tried Splunk with Python 2.7? It might not work with
    > >> >> > versions which are earlier than Python 2.6, but that doesn't
    > >> >> > necessarily mean that it won't work with versions of Python 2 which are
    > >> >> > later than Python 2.6 (unless the documentation says that it must be
    > >> >> > Python 2.6).- Hide quoted text -

    >
    > >> >> > - Show quoted text -

    >
    > >> >> Splunk's latest version 4.1.6 doesn't support Python 2.7
    > >> >> I tried the import trick but it didnt work because the real script
    > >> >> which runs under Python 2.7 has import pyodbc so it results in
    > >> >> following error

    >
    > >> >> c:\Splunk\etc\apps\search\bin>splunk cmd python namelookupWrapper.py
    > >> >> memberId memberName < memberInput.csv
    > >> >> Traceback (most recent call last):
    > >> >>   File "namelookupWrapper.py", line 3, in <module>
    > >> >>     import namelookup
    > >> >>   File "c:\Splunk\etc\apps\search\bin\namelookup.py", line 7, in
    > >> >> <module>
    > >> >>     import pyodbc
    > >> >> ImportError: DLL load failed: The specified module could not be found.

    >
    > >> >> Please let me know if i am missing something on import. If so please
    > >> >> provide me with an example- Hide quoted text -

    >
    > >> >> - Show quoted text -

    >
    > >> > Here are some more details from my earlier posting. Please click the
    > >> > below link

    >
    > >> >http://answers.splunk.com/questions/11145/its-getting-mysterious-to-m....
    > >> > --
    > >> >http://mail.python.org/mailman/listinfo/python-list

    >
    > >> Have you tried downloading the source for PyODBC and compiling it
    > >> yourself? All you need to do is python setup.py install. My guess
    > >> would be that it works just fine on 64-bit Python 2.6, they just never
    > >> released a re-compiled version of it for that platform.- Hide quoted text -

    >
    > >> - Show quoted text -

    >
    > > Thanks Benjamin. Please point me to the website from where i can
    > > download pyodbc for Windows 64 bit OS under Python 2.6 and
    > > installation instructions
    > > --

    >
    > You don't download it for 64-bit Windows with Python 2.6. You download
    > the source code from the website and make the Python 2.6, 64-bit
    > Windows version yourself.
    >
    > Download the source zip file and extract it. Then, open up the command
    > prompt and use the "cd" command to change directories to that source
    > folder. For instance, if the source code has been extracted to
    > C:\pyodbc-2.1.8\, you type in "cd C:\pyodbc-2.1.8" and press enter.
    >
    > After that, you just run the build script which is already in there:
    >
    > C:\Python26\python26.exe setup.py install
    >
    > You'll need to have Visual C++ 2008 (not 2010) installed for this to
    > work. You can get it for free fromhttp://www.microsoft.com/express/Downloads/if you don't already have
    > it.
    >
    >
    >
    > >http://mail.python.org/mailman/listinfo/python-list- Hide quoted text -

    >
    > - Show quoted text -- Hide quoted text -
    >
    > - Show quoted text -


    Thanks Benjamin. Wondering why i need to Visual C++ 2008 . What it has
    to do with Python?
    Isn't it possible to implement your suggestion without installing
    Visual C++ 2008 .
     
    bansi, Jan 30, 2011
    #15
  16. bansi

    rusi Guest

    On Jan 30, 6:31 pm, bansi <> wrote:
    > On Jan 28, 4:22 pm, Benjamin Kaplan <> wrote:
    >
    > > You'll need to have Visual C++ 2008 (not 2010) installed for this to
    > > work. You can get it for free fromhttp://www.microsoft.com/express/Downloads/if
    > > you don't already have it.

    >
    >
    > Thanks Benjamin. Wondering why i need to Visual C++ 2008 . What it has
    > to do with Python?
    > Isn't it possible to implement your suggestion without installing
    > Visual C++ 2008 .



    http://code.google.com/p/pyodbc/wiki/Building#Windows
     
    rusi, Jan 30, 2011
    #16
  17. bansi

    rusi Guest

    On Jan 30, 10:35 pm, rusi <> wrote:
    > On Jan 30, 6:31 pm, bansi <> wrote:
    > > Isn't it possible to implement your suggestion without installing
    > > Visual C++ 2008 .

    >
    > http://code.google.com/p/pyodbc/wiki/Building#Windows


    Well... This is what the official site says...
    On second thoughts I wonder: Would it not be possible to compile python
    +pyodbc from source and use gcc/ming for that? Someone who knows more
    of the windows build process may say something about the issues
    involved.
     
    rusi, Jan 31, 2011
    #17
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. =?ISO-8859-2?Q?Martin_MOKREJ=A9?=
    Replies:
    22
    Views:
    604
    =?windows-1252?Q?Martin_MOKREJ=8A?=
    Mar 9, 2005
  2. Ben Kial
    Replies:
    1
    Views:
    718
    Eric Enright
    Nov 15, 2004
  3. Guilherme Grillo

    reader inside a reader

    Guilherme Grillo, Nov 7, 2007, in forum: ASP .Net
    Replies:
    5
    Views:
    545
    sloan
    Nov 7, 2007
  4. Tim
    Replies:
    1
    Views:
    344
    Peter Otten
    Jul 5, 2010
  5. Pokkai Dokkai
    Replies:
    1
    Views:
    249
    Hassan Schroeder
    Mar 24, 2008
Loading...

Share This Page