1.8.2 - conituations memory leak fixed?

T

ts

W> Is this still an issue with the "official" release?

Nobody has given, yet, the proof that it exist a memory leak with
continuations.


Guy Decoux
 
E

Eric Hodel

--Apple-Mail-15--996546503
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset=US-ASCII; format=flowed


Am Mittwoch 19 Januar 2005 16:53 schrieb ts:

Sure, but the results might be the same ;-)

I don't see a callcc in there anywhere, so
callback_stream.with_callbacks_for could be doing other naughty things.
From personal experience with callcc, I would bet on something
referencing live objects over any memory leaks.

--
Eric Hodel - (e-mail address removed) - http://segment7.net
FEC2 57F1 D465 EB15 5D6E 7C11 332A 551C 796C 9F04

--Apple-Mail-15--996546503
content-type: application/pgp-signature; x-mac-type=70674453;
name=PGP.sig
content-description: This is a digitally signed message part
content-disposition: inline; filename=PGP.sig
content-transfer-encoding: 7bit

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (Darwin)

iD8DBQFB7qg4MypVHHlsnwQRAvVfAJ93kEziioJxwT5vwAOWk8s46LfEEACfdOQl
4zrtG/9+ofsy35YCUhGKZ84=
=WEUc
-----END PGP SIGNATURE-----

--Apple-Mail-15--996546503--
 
M

Michael Neumann

Am Mittwoch 19 Januar 2005 19:35 schrieb Eric Hodel:
I don't see a callcc in there anywhere, so
callback_stream.with_callbacks_for could be doing other naughty things.

Right. But it's the place where the callbacks are invoked. And the callback
that gets invoked actually uses callcc. It was very strange that, when I
change that line in some manner (introduce an assignment), the memory
consumption changes drastically.
From personal experience with callcc, I would bet on something
referencing live objects over any memory leaks.

I think the problem is due to the conservative GC. But of course, there might
also be some bugs in it (but then, memory consumption should be unbounded as
well, if I remove continuations, which I've done).

Regards,

Michael
 
M

Michael Neumann

Am Mittwoch 19 Januar 2005 20:41 schrieb itsme213:
If it is something other than a memory leak, and if it is locatable :)
would continuations in Wee become a 'recommended usage'?

Hm, there's still another issue. You can't marshal continuations in Ruby.
Don't know whether they'd become recommended ;-)

But it's so easy to leave them out or integrate them back into Wee, so I don't
think about that issue yet.

Regards,

Michael
 
J

jc

The last Ruby Weekly News has something that might be helpful here:

http://rubygarden.org/ruby?RubyNews/2005-01-03

Tanaka Akira posted a patch to record GC-related information:

* the total number of GC invocation
* the number of GC invocation, when an object is collected
* the location where the last GC is yielded
This patch might help you if you are interested in GC internals.
 
E

Eric Hodel

--Apple-Mail-33--976818807
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset=US-ASCII; format=flowed

Am Mittwoch 19 Januar 2005 19:35 schrieb Eric Hodel:

Right. But it's the place where the callbacks are invoked. And the
callback
that gets invoked actually uses callcc. It was very strange that, when
I
change that line in some manner (introduce an assignment), the memory
consumption changes drastically.

Due to the way Ruby saves the environment, the problem could be on the
line you show, or it could be where callcc is invoked. Without a
simple testcase, its difficult to track this problem down to a memory
leak, or a result of the conservative GC/environment feature:

def x
w = :stuff
proc do # w is free in this proc, so we don't need to save any
reference to it
# in the proc
puts "w: #{eval "w"}" # grab w out of the environment
end
end

x.call # prints w: stuff
# Because ruby keeps the entire local variable table around, we
can get to w
--
Eric Hodel - (e-mail address removed) - http://segment7.net
FEC2 57F1 D465 EB15 5D6E 7C11 332A 551C 796C 9F04

--Apple-Mail-33--976818807
content-type: application/pgp-signature; x-mac-type=70674453;
name=PGP.sig
content-description: This is a digitally signed message part
content-disposition: inline; filename=PGP.sig
content-transfer-encoding: 7bit

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (Darwin)

iD8DBQFB7vVIMypVHHlsnwQRAoO6AJwPG9SW27wpBmTLbxWUwi9GwZze9QCgpYuF
EEFhp1l2al1e0SESepfRiWs=
=zqsC
-----END PGP SIGNATURE-----

--Apple-Mail-33--976818807--
 
T

Tanaka Akira

jc said:
Tanaka Akira posted a patch to record GC-related information:

* the total number of GC invocation
* the number of GC invocation, when an object is collected
* the location where the last GC is yielded
This patch might help you if you are interested in GC internals.

It was not for memory leak. It was intended to investigate a wrongly
collected object. The memory leak is opposite situation: some objects
are not collected.

So, I updated the patch to investigate a live object.

The attached patch implements GC.trace_object which makes Ruby to
report why a registered object is not collected.

% ./ruby -ve '
k = nil
5.times {
callcc {|k| }
p k
GC.trace_object(k)
GC.start
}
'|&uniq
#<Continuation:0x401c75e8>
gc_mark(1): Scope (0x401d8d34) -> Data (0x401c75e8)
gc_mark(1): root (0x08078711,0x08078f4e) -> Data (0x401c75e8)
#<Continuation:0x401c78a4>
gc_mark(2): Scope (0x401d8d34) -> Data (0x401c78a4)
gc_mark(2): Data (0x401c78a4) -> Data (0x401c75e8)
gc_mark(2): root (0x08078711,0x08078f4e) -> Data (0x401c78a4)
#<Continuation:0x401c7804>
gc_mark(3): Scope (0x401d8d34) -> Data (0x401c7804)
gc_mark(3): Data (0x401c7804) -> Data (0x401c78a4)
gc_mark(3): Data (0x401c78a4) -> Data (0x401c75e8)
gc_mark(3): Data (0x401c7804) -> Data (0x401c78a4)
gc_mark(3): root (0x08078711,0x08078f4e) -> Data (0x401c7804)
#<Continuation:0x401c7728>
gc_mark(4): Scope (0x401d8d34) -> Data (0x401c7728)
gc_mark(4): Data (0x401c7728) -> Data (0x401c7804)
gc_mark(4): Data (0x401c7804) -> Data (0x401c78a4)
gc_mark(4): Data (0x401c78a4) -> Data (0x401c75e8)
gc_mark(4): Data (0x401c7804) -> Data (0x401c78a4)
gc_mark(4): Data (0x401c7728) -> Data (0x401c7804)
gc_mark(4): root (0x08078711,0x08078f4e) -> Data (0x401c7728)
#<Continuation:0x401c75ac>
gc_mark(5): Scope (0x401d8d34) -> Data (0x401c75ac)
gc_mark(5): Data (0x401c75ac) -> Data (0x401c7728)
gc_mark(5): Data (0x401c7728) -> Data (0x401c7804)
gc_mark(5): Data (0x401c7804) -> Data (0x401c78a4)
gc_mark(5): Data (0x401c78a4) -> Data (0x401c75e8)
gc_mark(5): Data (0x401c7804) -> Data (0x401c78a4)
gc_mark(5): Data (0x401c7728) -> Data (0x401c7804)
gc_mark(5): Data (0x401c75ac) -> Data (0x401c7728)
gc_mark(5): root (0x08078711,0x08078f4e) -> Data (0x401c75ac)
ruby 1.9.0 (2005-01-21) [i686-linux]

This mean that the Nth continuation refers the N-1th continuation.
So the continuations are never collected.

I'm not sure that it should be called "memory leak" or not. But it is
unpleasant behavior anyway. I have no idea to fix it portably,
though.

Index: error.c
===================================================================
RCS file: /src/ruby/error.c,v
retrieving revision 1.103
diff -u -p -r1.103 error.c
--- error.c 17 Nov 2004 02:27:37 -0000 1.103
+++ error.c 21 Jan 2005 11:29:24 -0000
@@ -244,6 +244,19 @@ static struct types {
{-1, 0}
};

+char *rb_object_structure_type(VALUE obj)
+{
+ struct types *type = builtin_types;
+ int t = TYPE(obj);
+
+ while (type->type >= 0) {
+ if (type->type == t) {
+ return type->name;
+ }
+ type++;
+ }
+}
+
void
rb_check_type(x, t)
VALUE x;
Index: eval.c
===================================================================
RCS file: /src/ruby/eval.c,v
retrieving revision 1.748
diff -u -p -r1.748 eval.c
--- eval.c 5 Jan 2005 03:49:50 -0000 1.748
+++ eval.c 21 Jan 2005 11:29:24 -0000
@@ -3865,7 +3865,7 @@ rb_eval(self, n)
break;

default:
- rb_bug("unknown node type %d", nd_type(node));
+ rb_bug("unknown node type %d (0x%lx)", nd_type(node), (long)node);
}
finish:
CHECK_INTS;
@@ -5741,8 +5741,8 @@ rb_call(klass, recv, mid, argc, argv, sc
struct cache_entry *ent;

if (!klass) {
- rb_raise(rb_eNotImpError, "method `%s' called on terminated object (0x%lx)",
- rb_id2name(mid), recv);
+ rb_bug("method `%s' called on terminated object (0x%lx)",
+ rb_id2name(mid), recv);
}
/* is it in the method cache? */
ent = cache + EXPR1(klass, mid);
Index: gc.c
===================================================================
RCS file: /src/ruby/gc.c,v
retrieving revision 1.195
diff -u -p -r1.195 gc.c
--- gc.c 7 Jan 2005 09:05:52 -0000 1.195
+++ gc.c 21 Jan 2005 11:29:24 -0000
@@ -284,6 +284,7 @@ typedef struct RVALUE {
struct {
unsigned long flags; /* always 0 for freed obj */
struct RVALUE *next;
+ long gc_count;
} free;
struct RBasic basic;
struct RObject object;
@@ -308,6 +309,7 @@ typedef struct RVALUE {
#endif
} RVALUE;

+static long gc_count = 0;
static RVALUE *freelist = 0;
static RVALUE *deferred_final_list = 0;

@@ -701,6 +703,28 @@ rb_gc_mark_maybe(obj)

#define GC_LEVEL_MAX 250

+static VALUE gc_mark_base = Qundef;
+static st_table *gc_mark_interest = NULL;
+
+static void gc_collected(RVALUE *p, long gc_count)
+{
+ VALUE v = (VALUE)p;
+ p->as.free.gc_count = gc_count;
+ if (gc_mark_interest && st_lookup(gc_mark_interest, v, NULL)) {
+ fprintf(stderr, "gc_collected(%ld): 0x%08lx\n", gc_count, v);
+ st_delete(gc_mark_interest, &v, NULL);
+ }
+}
+
+static VALUE
+gc_trace_object(VALUE self, VALUE obj)
+{
+ if (!gc_mark_interest)
+ gc_mark_interest = st_init_numtable();
+ st_insert(gc_mark_interest, obj, 1);
+ return obj;
+}
+
void
gc_mark(ptr, lev)
VALUE ptr;
@@ -708,6 +732,23 @@ gc_mark(ptr, lev)
{
register RVALUE *obj;

+ extern char *rb_object_structure_type(VALUE obj);
+ if (gc_mark_interest && st_lookup(gc_mark_interest, ptr, NULL) && gc_mark_base != ptr) {
+ if (gc_mark_base == Qundef) {
+ fprintf(stderr, "gc_mark(%ld): root (0x%08lx,0x%08lx) -> %s (0x%08lx)\n",
+ gc_count,
+ (unsigned long)__builtin_return_address(0),
+ (unsigned long)__builtin_return_address(1),
+ rb_object_structure_type(ptr), ptr);
+ }
+ else {
+ fprintf(stderr, "gc_mark(%ld): %s (0x%08lx) -> %s (0x%08lx)\n",
+ gc_count,
+ rb_object_structure_type(gc_mark_base), gc_mark_base,
+ rb_object_structure_type(ptr), ptr);
+ }
+ }
+
obj = RANY(ptr);
if (rb_special_const_p(ptr)) return; /* special const not marked */
if (obj->as.basic.flags == 0) return; /* free cell */
@@ -737,7 +778,7 @@ rb_gc_mark(ptr)
}

static void
-gc_mark_children(ptr, lev)
+gc_mark_children1(ptr, lev)
VALUE ptr;
int lev;
{
@@ -994,6 +1035,17 @@ gc_mark_children(ptr, lev)
}
}

+static void
+gc_mark_children(ptr, lev)
+ VALUE ptr;
+ int lev;
+{
+ VALUE saved_base = gc_mark_base;
+ gc_mark_base = ptr;
+ gc_mark_children1(ptr, lev);
+ gc_mark_base = saved_base;
+}
+
static void obj_free _((VALUE));

static void
@@ -1006,6 +1058,7 @@ finalize_list(p)
if (!FL_TEST(p, FL_SINGLETON)) { /* not freeing page */
p->as.free.flags = 0;
p->as.free.next = freelist;
+ gc_collected(p, gc_count);
freelist = p;
}
p = tmp;
@@ -1040,6 +1093,7 @@ gc_sweep()
unsigned long live = 0;

mark_source_filename(ruby_sourcefile);
+ if (source_filenames)
st_foreach(source_filenames, sweep_source_filename, 0);

freelist = 0;
@@ -1059,11 +1113,13 @@ gc_sweep()
if (need_call_final && FL_TEST(p, FL_FINALIZE)) {
p->as.free.flags = FL_MARK; /* remain marked */
p->as.free.next = final_list;
+ gc_collected(p, gc_count);
final_list = p;
}
else {
p->as.free.flags = 0;
p->as.free.next = freelist;
+ gc_collected(p, gc_count);
freelist = p;
}
n++;
@@ -1115,6 +1171,7 @@ rb_gc_force_recycle(p)
{
RANY(p)->as.free.flags = 0;
RANY(p)->as.free.next = freelist;
+ gc_collected(RANY(p), -1);
freelist = RANY(p);
}

@@ -1290,6 +1347,11 @@ int rb_setjmp (rb_jmp_buf);
#endif /* __human68k__ or DJGPP */
#endif /* __GNUC__ */

+#ifdef __GNUC__
+void *main_return_address;
+static void *last_gc_stacktrace[10];
+#endif
+
static void
garbage_collect()
{
@@ -1312,6 +1374,10 @@ garbage_collect()
if (during_gc) return;
during_gc++;

+ gc_count++;
+ if (gc_count < 0)
+ gc_count = 0;
+
init_mark_stack();

/* mark frame stack */
@@ -1401,6 +1467,21 @@ garbage_collect()
}
}
gc_sweep();
+
+#ifdef __GNUC__
+ memset(last_gc_stacktrace, 0, sizeof(last_gc_stacktrace));
+ if ((last_gc_stacktrace[0] = __builtin_return_address(0)) == main_return_address) goto gc_stacktrace_done;
+ if ((last_gc_stacktrace[1] = __builtin_return_address(1)) == main_return_address) goto gc_stacktrace_done;
+ if ((last_gc_stacktrace[2] = __builtin_return_address(2)) == main_return_address) goto gc_stacktrace_done;
+ if ((last_gc_stacktrace[3] = __builtin_return_address(3)) == main_return_address) goto gc_stacktrace_done;
+ if ((last_gc_stacktrace[4] = __builtin_return_address(4)) == main_return_address) goto gc_stacktrace_done;
+ if ((last_gc_stacktrace[5] = __builtin_return_address(5)) == main_return_address) goto gc_stacktrace_done;
+ if ((last_gc_stacktrace[6] = __builtin_return_address(6)) == main_return_address) goto gc_stacktrace_done;
+ if ((last_gc_stacktrace[7] = __builtin_return_address(7)) == main_return_address) goto gc_stacktrace_done;
+ if ((last_gc_stacktrace[8] = __builtin_return_address(8)) == main_return_address) goto gc_stacktrace_done;
+ if ((last_gc_stacktrace[9] = __builtin_return_address(9)) == main_return_address) goto gc_stacktrace_done;
+ gc_stacktrace_done: ;
+#endif
}

void
@@ -1427,6 +1508,20 @@ rb_gc_start()
return Qnil;
}

+VALUE
+numfree()
+{
+ RVALUE *f = freelist;
+ long num = 0;
+
+ while (f != 0) {
+ f = f->as.free.next;
+ num += 1;
+ }
+ printf("numfree:%ld\tmalloc_increase:%lu\tmalloc_limit:%lu\n", num, malloc_increase, malloc_limit);
+ return Qnil;
+}
+
void
ruby_set_stack_size(size)
size_t size;
@@ -1929,6 +2024,9 @@ Init_GC()
rb_define_module_function(rb_mObSpace, "undefine_finalizer", undefine_final, 1);

rb_define_module_function(rb_mObSpace, "_id2ref", id2ref, 1);
+
+ rb_define_module_function(rb_mObSpace, "numfree", numfree, 0);
+ rb_define_module_function(rb_mGC, "trace_object", gc_trace_object, 1);

rb_gc_register_address(&rb_mObSpace);
rb_global_variable(&finalizers);
Index: main.c
===================================================================
RCS file: /src/ruby/main.c,v
retrieving revision 1.13
diff -u -p -r1.13 main.c
--- main.c 23 Jun 2004 12:59:01 -0000 1.13
+++ main.c 21 Jan 2005 11:29:24 -0000
@@ -21,6 +21,10 @@
static void objcdummyfunction( void ) { objc_msgSend(); }
#endif

+#ifdef __GNUC__
+extern void *main_return_address;
+#endif
+
int
main(argc, argv, envp)
int argc;
@@ -31,6 +35,10 @@ main(argc, argv, envp)
#endif
#if defined(__MACOS__) && defined(__MWERKS__)
argc = ccommand(&argv);
+#endif
+
+#ifdef __GNUC__
+ main_return_address = __builtin_return_address(0);
#endif

ruby_init();
Index: variable.c
===================================================================
RCS file: /src/ruby/variable.c,v
retrieving revision 1.119
diff -u -p -r1.119 variable.c
--- variable.c 10 Jan 2005 14:07:53 -0000 1.119
+++ variable.c 21 Jan 2005 11:29:25 -0000
@@ -467,6 +467,7 @@ mark_global_entry(key, entry)
void
rb_gc_mark_global_tbl()
{
+ if (rb_global_tbl)
st_foreach_safe(rb_global_tbl, mark_global_entry, 0);
}
 
T

ts

T> +#ifdef __GNUC__

perhaps not a good idea

T> + memset(last_gc_stacktrace, 0, sizeof(last_gc_stacktrace));
T> + if ((last_gc_stacktrace[0] = __builtin_return_address(0)) == main_return_address) goto gc_stacktrace_done;
T> + if ((last_gc_stacktrace[1] = __builtin_return_address(1)) == main_return_address) goto gc_stacktrace_done;

uln% gdb --quiet ./miniruby
Using host libthread_db library "/lib64/libthread_db.so.1".
(gdb) r lib/fileutils.rb
Starting program: /opt/ts/ruby/r190/tmp/ruby/miniruby lib/fileutils.rb

Program received signal SIGSEGV, Segmentation fault.
0x000000000042dab6 in garbage_collect () at gc.c:1474
1474 if ((last_gc_stacktrace[1] = __builtin_return_address(1)) == main_return_address) goto gc_stacktrace_done;
(gdb) c
Continuing.
lib/fileutils.rb:1132: [BUG] Segmentation fault
ruby 1.9.0 (2005-01-21) [x86_64-linux]


Program received signal SIGABRT, Aborted.
0x0000002a95a35af9 in kill () from /lib64/libc.so.6
(gdb)



Guy Decoux
 
T

Tanaka Akira

ts said:
T> +#ifdef __GNUC__

perhaps not a good idea

Yes. Not for everyone.
0x000000000042dab6 in garbage_collect () at gc.c:1474
1474 if ((last_gc_stacktrace[1] = __builtin_return_address(1)) == main_return_address) goto gc_stacktrace_done;

Maybe gcc doesn't support __builtin_return_address(1) on IA64.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,484
Members
44,904
Latest member
HealthyVisionsCBDPrice

Latest Threads

Top