python profiling for a XML parser program

[MacRules]

I have a python program doing XML data prasing and write the result to 2
data files; which will be loaded to MySQL.

I ran this.

$ python dealmaker.py
.... read data
loop through records
.... XML parsing
.... write to file1.dat
.... write to file2.date
done

Is there a python profiler just like for C program?
And tell me which functions or modules take a long time.

Can you show me URL or link on doing this task?

Thanks

 

[Diez B. Roggisch]

I have a python program doing XML data prasing and write the result to 2
data files; which will be loaded to MySQL.

I ran this.

$ python dealmaker.py
... read data
loop through records
... XML parsing
... write to file1.dat
... write to file2.date
done

Is there a python profiler just like for C program?
And tell me which functions or modules take a long time.

Can you show me URL or link on doing this task?

Google dead today?

http://www.google.com/search?q=pyth...s=org.mozilla:de-DE:official&client=firefox-a

First hit. It's build-in already.

Diez

 

[Paul Boddie]

Is there a python profiler just like for C program?
And tell me which functions or modules take a long time.

Can you show me URL or link on doing this task?

Having already looked at combining Python profilers with KCachegrind
(as suggested by Andrew Dalke at EuroPython 2007), I discovered the
following discussion about the tools required:

http://ddaa.net/blog/python/lsprof-calltree

I found the following script very convenient to generate profiler
output:

http://www.gnome.org/~johan/lsprofcalltree.py

You can then visualise the time spent by just passing the output file
to KCachegrind:

http://kcachegrind.sourceforge.net/html/Home.html

The map of time spent in each function is extremely useful, I find.

Paul

 

[MacRules]

Having already looked at combining Python profilers with KCachegrind
(as suggested by Andrew Dalke at EuroPython 2007), I discovered the
following discussion about the tools required:

http://ddaa.net/blog/python/lsprof-calltree

I found the following script very convenient to generate profiler
output:

http://www.gnome.org/~johan/lsprofcalltree.py

You can then visualise the time spent by just passing the output file
to KCachegrind:

http://kcachegrind.sourceforge.net/html/Home.html

The map of time spent in each function is extremely useful, I find.

Paul

Paul is better than Google.

Thanks for the info, I will take a look.

 

[Stefan Behnel]

I have a python program doing XML data prasing and write the result to 2
data files; which will be loaded to MySQL.
[...]
Is there a python profiler just like for C program?
And tell me which functions or modules take a long time.

In case you want to do this because you noticed performance problems with
the XML handling code, you might want to take a look at cElementTree (comes
with Py2.5) and/or lxml (separate package). They are the fastest XML
packages for Python, mostly compatible and very easy to use.

Stefan