Tuesday
Dec162008
[ANN] New Open Source Cache System
Tuesday, December 16, 2008 at 8:03AM
The SHOP.COM Cache System is now available at http://code.google.com/p/sccache/
The SHOP.COM Cache System is an object cache system that...
* is an in-process cache and external, shared Cache
* is horizontally scalable
* stores cached objects to disk
* supports associative keys
* is non-transactional
* can have any size key and any size data
* does auto-GC based on TTL
* is container and platform neutral
It was built in-house at SHOP.COM (by me) and has powered our website for years. We are open-sourcing it in the hope that it will be useful to others and to get some help in its maintenance.
This is our first open source attempt and we'd appreciate any help and comments.
The SHOP.COM Cache System is an object cache system that...
* is an in-process cache and external, shared Cache
* is horizontally scalable
* stores cached objects to disk
* supports associative keys
* is non-transactional
* can have any size key and any size data
* does auto-GC based on TTL
* is container and platform neutral
It was built in-house at SHOP.COM (by me) and has powered our website for years. We are open-sourcing it in the hope that it will be useful to others and to get some help in its maintenance.
This is our first open source attempt and we'd appreciate any help and comments.
Reader Comments (13)
How does this compare with memcached?
Interesting. What programming language is it written in?
With the horizontal scalability, is there any replication? Ie if one cache server dies, can the others take over?
Thanks for opening the code! Opensource is the way to go!
This looks pretty good, one question though, can I interact with it using PHP?
From what I am seeing, my application would need to be in Java to interact with it easily. Is there anyway for there to be some kind of front end wrapper so that we don't have to mix languages on our servers? I'd like to continue doing everything in PHP and maybe interact with this system via some kind of PHP to JAVA wrapper over HTTP.
I've got a bare-bones comparison table started at http://code.google.com/p/sccache/wiki/Comparison
>Interesting. What programming language is it written in?
100% Java
>With the horizontal scalability, is there any replication?
>Ie if one cache server dies, can the others take over?
You can mark some objects as "important" and they will be written to two cache server instances. So, if one goes down the object will still be available. However, sccache is not transactional and is not meant to replace a database.
>Is there anyway for there to be some kind of front end
>wrapper so that we don't have to mix languages on our servers?
When I get time I can do it. But, maybe some intrepid programmer can add it in :)
One other thing, is the cache timeout configurable? In our environment your solution would work perfectly (except for the java part, but we can get around that), but I'd like for the objects to never expire, except if we overwrite them.
It looks like it is set at 12 hours by default. Can we set it to forever?
Would have been nice if they stated that it was Java in the article somewhere. I had to click through to the project to see that it was Java.
>It looks like it is set at 12 hours by default. Can we set it to forever?
You could, but it affects how garbage collection works. The disk storage portion of the system does garbage collection based on maximum TTL. You could set it a huge number, but the cache files might grow very large.
How many objects would you be caching? Another option is to create two cache farms. One for the standard objects and another for the infinite objects. sccache is totally pluggable so you could plug a different disk storage implementation for the infinite objects.
FYI - there was a bug in the storage code. It's now fixed.
Issue:
http://code.google.com/p/sccache/issues/detail?id=1&can=1
Fix:
http://sccache.googlecode.com/files/sccache_0_4.zip