running python as a dameon

[peyman]

Hi

I have a Natural Language Processing (NLP) code written in python that
reads into memory a large training file and then given a sentence tags
it, using the training data. I want to put this NLP code on a server
which handles all incoming client http requests via PHP. What I want
to do is to provide the python NLP program as a service to any other
PHP/Java/Ruby process request. So the mapping is

http -> apache -> PHP/Java/Ruby/... -> Python NLP

I can not provide this service as a shell script because of the
inefficiencies of having to load into memory a large training data to
every service request. So what I want to do is to provide the NLP
service to other application processes as a python daemon

http -> apache -> PHP/Java/Ruby/... -> Python Dameon -> Python NLP

The daemon loads into memory the training data once. then every
service request event invokes the appropriate NLP code in the python
program. I've tried to use play around with twisted but am not making
too much of a headway. What I've done in the NLP code is to do this:

# filename: NLP.py
def parse(sentence):
structure=getentities(sentence)
print 'subject: \t',' '.join(structure[0])
print 'predicate: \t',' '.join(structure[1])
print 'TE: \t\t',' '.join(structure[2])

class TLogicClass():
def __init__(self,prop):
return parse(prop)

then in the dameon code done this

# filename: dameon.py
from twisted.application import service
import NLP

application=service.Application("nlp")
ParseService=NLP.TLogicClass("time flies like an arrow")
ParseService.setServiceParent(application)

but I get the following error when I run twistd -y daemon.py

I suspect I need to put twisted in the NLP.py but I don't know what I
need to do in order to get what I want to do which is:

to load into memory only once a large training data that can then be
queried by another (non-python) event based process (I need "reactor"
class?).

thank you in advance for your help

 

[Sean Davis]

Hi

I have a Natural Language Processing (NLP) code written in python that
reads into memory a large training file and then given a sentence tags
it, using the training data. I want to put this NLP code on a server
which handles all incoming client http requests via PHP. What I want
to do is to provide the python NLP program as a service to any other
PHP/Java/Ruby process request. So the mapping is

http -> apache -> PHP/Java/Ruby/... -> Python NLP

Why not use a simple CGI script or wsgi application? You could make
the service online and interactive and with the same application and
code make an XMLRPC web service. So, things would look more like:

http -> apache -> Python (running NLP and serving requests)

You can use apache to proxy requests to any one of a dozen or so
python-based webservers. You could also use mod_wsgi to interface
with a wsgi application.

Sean

I can not provide this service as a shell script because of the
inefficiencies of having to load into memory a large training data to
every service request. So what I want to do is to provide the NLP
service to other application processes as a python daemon

http -> apache -> PHP/Java/Ruby/... -> Python Dameon -> Python NLP

The daemon loads into memory the training data once. then every
service request event invokes the appropriate NLP code in the python
program. I've tried to use play around with twisted but am not making
too much of a headway. What I've done in the NLP code is to do this:

# filename: NLP.py
def parse(sentence):
    structure=getentities(sentence)
    print 'subject: \t',' '.join(structure[0])
    print 'predicate: \t',' '.join(structure[1])
    print 'TE: \t\t',' '.join(structure[2])

class TLogicClass():
    def __init__(self,prop):
        return parse(prop)

then in the dameon code done this

# filename: dameon.py
from twisted.application import service
import NLP

application=service.Application("nlp")
ParseService=NLP.TLogicClass("time flies like an arrow")
ParseService.setServiceParent(application)

but I get the following error when I run twistd -y daemon.py

I suspect I need to put twisted in the NLP.py but I don't know what I
need to do in order to get what I want to do which is:

to load into memory only once a large training data that can then be
queried by another (non-python) event based process  (I need "reactor"
class?).

thank you in advance for your help

 

[Michael Palmer]

Why not use a simple CGI script or wsgi application? You could make
the service online and interactive and with the same application and
code make an XMLRPC web service. So, things would look more like:

http -> apache -> Python (running NLP and serving requests)

You can use apache to proxy requests to any one of a dozen or so
python-based webservers. You could also use mod_wsgi to interface
with a wsgi application.

Sean

xmlrpc is the right idea, as it interfaces easily across languages.

 

[kaer]

xmlrpc is the right idea, as it interfaces easily across languages.

I just daemonize some utility script these week. For that I took one
of the scripts in the ASPN python cookbook : google those 3 words then
search python and you will find out your solution faster than I end up
this mail. It worked like a charm on my linux box.
BTW, I wouldn't complicate with xml stuffs if you don't need it.

Good luck.

 

[kaer]

I just daemonize some utility script these week. For that I took one
of the scripts in the ASPN python cookbook : google those 3 words then
search python and you will find out your solution faster than I end up
this mail. It worked like a charm on my linux box.
BTW, I wouldn't complicate with xml stuffs if you don't need it.

Good luck.

you obviously "search daemon" not "search python". Her is the link:
http://www.google.com/search?q=daem...occt=any&dt=i&sitesearch=code.activestate.com