Thursday, September 22, 2011

My own NoSQL DB

15 days ago I read this article: http://blog.cubrid.org/dev-platform/nosql-benchmarking/ and got shock with the amazing performance, c'mmon! 21k operations per second? it's amazing...

After a long bath I started to wonder, how much performance will I achieve if I create my own NoSQL database, just an experiment for those rainy days, ok... here I am 1 week later after some hard work, I created my own NoSQL Database which is capable of:

1. Insert Records
2. Find records using the key (that's the benchmark)
3. Index the keys and whatever I want to index to find records later.
4. Split files to get indexes separated from data.
5. Created a layer to save the data using little endian (basically I copied the code from my other project d-jon)

Here's the test I did, with a modest machine and 8Gb of RAM:

1. 1M inserts of 1k (I know, the bench was over 50M, but I dont have free space on my machine right now)
2. A random search using random keys (300.662 searches)

And here are the stats:

the inserts took 23.48 secs, which means 42.584 operations per sec.
The searches took 20.3 secs, this means 14.763 operations per sec.

These results are preliminary, but I'm starting to think that I could beat MongoDB, Cassandra, and others,

I started my project because I was out of challenges, and it's turning to be a very attractive project, maybe this could be my next fulltime job.

A lot of work coming, but what the hell... djon started just like that and now have over a 3k of downloads and users all over the world.