[ANN] Localmemcache-0.2.0: The beauty of memcached. For local data.Blazingly fast.

[Sven C. Koehler]

Hi,

I am happy to announce the next version of localmemcache (0.2.0), a
library that aims to provide an interface similar to memcached but for
accessing local data instead of remote data. It's based on mmap()'ed
shared memory for maximum speed.

* http://localmemcache.rubyforge.org/

Changes for 0.2.0:
* Localmemcache uses logging now: In case your application is terminated
while accessing the shared memory (eg by a kill -9), localmemcache is
now able to restore the integrity of your data.
* \0 character can now be used in keys and values
* The ruby binding now features a keys() method.
* Added a C API.

0.2.0 facts:
- lightweight: The core library is just about 1300 lines of C code
- fast: About 40% slower than Ruby's hash (or about 36 times
faster than a local memcached).

EXAMPLE
=======

require 'localmemcache'
$lm = LocalMemCache.new :namespace => "viewcounters"
$lm[:foo] = 1
$lm[:foo]
$lm.deletefoo)

INSTALL
=======

# gem install localmemcache

(In case rubyforge has not yet updated the mirrors, fetch the 0.2.0 gem
from here: http://github.com/sck/localmemcache/downloads and then do
# gem install localmemcache-0.2.0.gem )

CONTACT
=======

Please contact me with bugs, suggestions and patches at: schween + snafu # de

LINKS
=====

Localmemcache: http://localmemcache.rubyforge.org/
Rubyforge project: http://localmemcache.rubyforge.org/

Source code is hosted on github: http://github.com/sck/localmemcache/

Best,

Sven C. Koehler

afa5fe7a3a23930a50d0b8b4a1a55880 localmemcache-0.2.0.gem
050cab44c9a5b9cb6214779398c0cfb7 localmemcache-0.2.0.tar.gz

 

[ara.t.howard]

# gem install localmemcache

really nice idea. can't wait to try it out!

a @ http://codeforpeople.com/

 

[Bill Kelly]

I am happy to announce the next version of localmemcache (0.2.0), a
library that aims to provide an interface similar to memcached but for
accessing local data instead of remote data. It's based on mmap()'ed
shared memory for maximum speed.

Very nice!!

I've been daydreaming about a mmap'd hash table for a
couple years now -- and I wasn't even contemplating
having it able to be shared between processes, as it
appears you've done.

Your code appears nicely decoupled. It seems possible
if one were to write Windoze versions of lmc_lock.c and
lmc_shm.c , the rest of the code might just work on that
platform. (I may attempt this... someday...)


Anyway, thanks for releasing this interesting library!


Regards,

Bill

 

[Sven C. Koehler]

Very nice!!
Thanks!

Your code appears nicely decoupled. It seems possible
if one were to write Windoze versions of lmc_lock.c and
lmc_shm.c , the rest of the code might just work on that
platform. (I may attempt this... someday...)

Yeah, this should work. ;-)

-S.

 

[Michael Fellinger]

Hi,

I am happy to announce the next version of localmemcache (0.2.0), a
library that aims to provide an interface similar to memcached but for
accessing local data instead of remote data. It's based on mmap()'ed
shared memory for maximum speed.

This sounds great, can't wait to add it as a caching option to Ramaze.

* http://localmemcache.rubyforge.org/

Changes for 0.2.0:
* Localmemcache uses logging now: In case your application is
terminated while accessing the shared memory (eg by a kill -9),
localmemcache is now able to restore the integrity of your data.
* \0 character can now be used in keys and values
* The ruby binding now features a keys() method.
* Added a C API.

0.2.0 facts:
- lightweight: The core library is just about 1300 lines of C code
- fast: About 40% slower than Ruby's hash (or about 36 times
faster than a local memcached).

EXAMPLE
=======

require 'localmemcache'
$lm = LocalMemCache.new :namespace => "viewcounters"
$lm[:foo] = 1
$lm[:foo]
$lm.deletefoo)

I tried that, but $lm[:foo] always returned nil.
That's on ArchLinux, both 32 and 64bit.

 

[Rados³aw Bu³at]

require 'localmemcache'
$lm =3D LocalMemCache.new :namespace =3D> "viewcounters"
$lm[:foo] =3D 1
$lm[:foo]
$lm.deletefoo)

I tried that, but $lm[:foo] always returned nil.
That's on ArchLinux, both 32 and 64bit.

The same for me. My configuration: ubuntu 8.10 64bit, ruby 1.8.7.

--=20
Pozdrawiam

Rados=B3aw Bu=B3at
http://radarek.jogger.pl - m=F3j blog

 

[Sven C. Koehler]

require 'localmemcache'
$lm = LocalMemCache.new :namespace => "viewcounters"
$lm[:foo] = 1
$lm[:foo]
$lm.deletefoo)

I tried that, but $lm[:foo] always returned nil.
That's on ArchLinux, both 32 and 64bit.

Hmm... Can you see the namespace in /var/tmp/localmemcache/ ?
Can you see the key in $lm.keys()?
Does ``du -h /var/tmp/localmemcache/viewcounters.lmc'' show that there's
content in the file?

Best,

Sven

 

[Sven C. Koehler]

Hmm... Can you see the namespace in /var/tmp/localmemcache/ ?
Can you see the key in $lm.keys()?
Does ``du -h /var/tmp/localmemcache/viewcounters.lmc'' show that there's
content in the file?

This is how it looks on my machine:

| sck@comp ~/tmp $ uname -a
| Linux comp 2.6.23-gentoo-r3-b #6 SMP Sat Aug 23 08:57:32 UTC 2008
| x86_64 Intel(R) Xeon(R) CPU E5205 @ 1.86GHz GenuineIntel GNU/Linux
| sck@comp ~/tmp $ irb
| irb(main):001:0> require 'localmemcache'
| => true
| irb(main):002:0> $lm = LocalMemCache.new :namespace => "viewcounters"
| => #<LocalMemCache:0x2b5ed7ef0130>
| irb(main):003:0> $lm.keys
| => []
| irb(main):004:0> $lm[:foo]
| => nil
| irb(main):005:0> $lm[:foo] = 1
| => 1
| irb(main):006:0> $lm[:foo]
| => "1"
| irb(main):007:0> $lm.keys
| => ["foo"]
| irb(main):008:0> sck@comp ~/tmp $
| sck@comp ~/tmp $ ls -al /var/tmp/localmemcache/viewcounters.lmc
| -rwxr-xr-x 1 sck sck 1073741824 Apr 4 17:52
| /var/tmp/localmemcache/viewcounters.lmc
| sck@comp ~/tmp $ du -h /var/tmp/localmemcache/viewcounters.lmc
| 16K /var/tmp/localmemcache/viewcounters.lmc
| sck@comp ~/tmp $ irb
| irb(main):002:0> require 'localmemcache'
| => true
| irb(main):003:0> $lm = LocalMemCache.new :namespace => "viewcounters"
| => #<LocalMemCache:0x2b50be3e6978>
| irb(main):004:0> $lm.keys
| => ["foo"]
| irb(main):005:0>

-S.

 

[Rados³aw Bu³at]

Hmm... Can you see the namespace in /var/tmp/localmemcache/ ?

Yes
$ ls /var/tmp/localmemcache/
foo.lmc viewcounters.lmc

Can you see the key in $lm.keys()?

Yes, but they are duplicated after setting on the same key:

$lm =3D LocalMemCache.new :namespace =3D> "viewcounters"

$lm.keys =3D> []
$lm[:foo] =3D 1 =3D> 1
$lm[:foo] =3D> nil
$lm.keys =3D> ["foo"]
$lm[:foo] =3D 2 =3D> 2
$lm[:foo] =3D> nil
$lm.keys

=3D> ["foo", "foo"]

Does ``du -h /var/tmp/localmemcache/viewcounters.lmc'' show that there's
content in the file?

$ du -h /var/tmp/localmemcache/viewcounters.lmc
20K /var/tmp/localmemcache/viewcounters.lmc


--=20
Pozdrawiam

Rados=B3aw Bu=B3at
http://radarek.jogger.pl - m=F3j blog

 

[Matthias Reitinger]

This is how it looks on my machine:

I can reproduce the issue Michael and Radosław are having:

reima@marvin:~$ uname -a
Linux marvin 2.6.27-11-generic #1 SMP Thu Jan 29 19:24:39 UTC 2009 i686
GNU/Linux
reima@marvin:~$ ruby -v
ruby 1.8.7 (2008-08-11 patchlevel 72) [i486-linux]
reima@marvin:~$ irb

require 'localmemcache' => true
$lm = LocalMemCache.new :namespace => "viewcounters"

$lm.keys => []
$lm[:foo] => nil
$lm[:foo] = 1 => 1
$lm[:foo] => nil
$lm.keys => ["foo"]
puts `du -h /var/tmp/localmemcache/viewcounters.lmc`

16K /var/tmp/localmemcache/viewcounters.lmc
=> nil
And on a sidenote:

reima@marvin:~$ irb(irb):4: [BUG] Segmentation fault
ruby 1.8.7 (2008-08-11 patchlevel 72) [i486-linux]

Aborted
reima@marvin:~$

-Matthias

 

[Sven C. Koehler]

Ok, thanks to everyone who reported bugs!
I think I fixed this problem, so could anyone who had problems setting
keys run the following command to verify whether it works better now?

% git clone git://github.com/sck/localmemcache.git && \
cd localmemcache && rake sanity_test

The output at the end should look like this:

| LocalMemCache
| - should allow to set and query keys
| - should support the [] and []= operators
| - should allow deletion of keys
| - should return a list of keys
| - should support \0 in values and keys
| - should throw exception if pool is full
| - should support checking of namespaces
| - should support clearing of namespaces
|
| 8 specifications (8 requirements), 0 failures, 0 errors

(I haven't fixed the close bug yet.)

Best,

Sven

This is how it looks on my machine:

I can reproduce the issue Michael and Rados??aw are having:

reima@marvin:~$ uname -a
Linux marvin 2.6.27-11-generic #1 SMP Thu Jan 29 19:24:39 UTC 2009 i686
GNU/Linux
reima@marvin:~$ ruby -v
ruby 1.8.7 (2008-08-11 patchlevel 72) [i486-linux]
reima@marvin:~$ irb

require 'localmemcache' => true
$lm = LocalMemCache.new :namespace => "viewcounters"

$lm.keys => []
$lm[:foo] => nil
$lm[:foo] = 1 => 1
$lm[:foo] => nil
$lm.keys => ["foo"]
puts `du -h /var/tmp/localmemcache/viewcounters.lmc`

16K /var/tmp/localmemcache/viewcounters.lmc
=> nil
And on a sidenote:

reima@marvin:~$ irb(irb):4: [BUG] Segmentation fault
ruby 1.8.7 (2008-08-11 patchlevel 72) [i486-linux]

Aborted
reima@marvin:~$

-Matthias

 

[Rados³aw Bu³at]

Ok, thanks to everyone who reported bugs!
I think I fixed this problem, so could anyone who had problems setting
keys run the following command to verify whether it works better now?

% git clone git://github.com/sck/localmemcache.git && \
=A0 =A0 cd localmemcache && rake sanity_test

The output at the end should look like this:

| LocalMemCache
| - should allow to set and query keys
| - should support the [] and []=3D operators
| - should allow deletion of keys
| - should return a list of keys
| - should support \0 in values and keys
| - should throw exception if pool is full
| - should support checking of namespaces
| - should support clearing of namespaces
|
| 8 specifications (8 requirements), 0 failures, 0 errors

It works as you expected:

LocalMemCache
- should allow to set and query keys
- should support the [] and []=3D operators
- should allow deletion of keys
- should return a list of keys
- should support \0 in values and keys
- should throw exception if pool is full
- should support checking of namespaces
- should support clearing of namespaces

8 specifications (8 requirements), 0 failures, 0 errors
--=20
Pozdrawiam

Rados=B3aw Bu=B3at
http://radarek.jogger.pl - m=F3j blog

 

[Michael Fellinger]

Ok, thanks to everyone who reported bugs!
I think I fixed this problem, so could anyone who had problems setting
keys run the following command to verify whether it works better now?

% git clone git://github.com/sck/localmemcache.git && \
cd localmemcache && rake sanity_test

The output at the end should look like this:

| LocalMemCache
| - should allow to set and query keys
| - should support the [] and []= operators
| - should allow deletion of keys
| - should return a list of keys
| - should support \0 in values and keys
| - should throw exception if pool is full
| - should support checking of namespaces
| - should support clearing of namespaces
|
| 8 specifications (8 requirements), 0 failures, 0 errors

Aye, now it works as expected, thanks a lot. Will we get an updated release as
well?