Slowdown in Jython

[tac-tics]

I have an application written in jython which has to process a number
of records. It runs fine until it gets to about 666 records (and maybe
that's a sign), and then, it's performance and responsiveness goes down
the toilet. It looks like it's running out of memory and is being
forced to use extended memory, but I do not know enough about the
language to figure out where this is happening. It will eventually
finish the task, but the window stops responding, and it ends up taking
several hours (as opposed to several minutes as it should). I really
just wish I had a tool for polling the amount of memory Jython was
using at any given moment.

Does anyone have any strategy or advice for me?

 

[Gabriel Genellina]

I have an application written in jython which has to process a number
of records. It runs fine until it gets to about 666 records (and maybe
that's a sign), and then, it's performance and responsiveness goes down
the toilet. It looks like it's running out of memory and is being
forced to use extended memory, but I do not know enough about the

Only a standard advise: try not to load all the records together,
instead, if possible, process them one record at a time. This way
you're not bound by available memory.


--
Gabriel Genellina
Softlab SRL






__________________________________________________
Preguntá. Respondé. Descubrí.
Todo lo que querías saber, y lo que ni imaginabas,
está en Yahoo! Respuestas (Beta).
¡Probalo ya!
http://www.yahoo.com.ar/respuestas

 

[Kent Johnson]

I have an application written in jython which has to process a number
of records. It runs fine until it gets to about 666 records (and maybe
that's a sign), and then, it's performance and responsiveness goes down
the toilet. It looks like it's running out of memory and is being
forced to use extended memory, but I do not know enough about the
language to figure out where this is happening. It will eventually
finish the task, but the window stops responding, and it ends up taking
several hours (as opposed to several minutes as it should). I really
just wish I had a tool for polling the amount of memory Jython was
using at any given moment.

Does anyone have any strategy or advice for me?

You can find out how much memory Jython is using the same as you would
for any other application, e.g. Windows Task Manager or the equivalent.

Jython is a Java application and you can increase the max heap available
the same as for other java apps, using the command line switch -Xmx,
e.g. -Xmx512m to set the max heap to 512 megabytes.

Kent

 

[tac-tics]

Jython is a Java application

That was the intellectual leap I needed to solve the problem. I forgot
that I have total access to Java memory management. It turns out at the
point of slowdown, Java was continually running full GC, causing the
awful loss of performance. I figured out that I was not releasing a
very large chunk of memory right before the script, so I effectively
had a duplicate of every record in memory.

I'm still not sure why my memory usage is increasing during the script,
but with the removal of the duplicates in memory, it runs just fine
now. Problem solved for now.