Wednesday, May 12, 2010

MongoDB Bench Marking (Java) - I

Too see how fast the MongoDB for large scale number crunching stuff, I am fiddling with MongoDB (among the other alternatives). Idea is to see how fast data can be inserted in to MongoDB.

I wrote preliminary c++/java versions and trying do optimizations in java-driver.

So I generated simple schema with 6 Strings as key-value pairs  and 250 numbers in Array.

Test  setup: AMD Phenom II X4 965 , 4 GB DDR3 RAM, 512 GB SATA HDD, Fedora 12 Linux

Maximum speed that I could achieve in java-version ( driver 1.4)  is  1,000,000 records in 26.5 seconds  -- which turns around 37,735 records/sec

Total disk size for above data is 2,489 MB -- which turns about around 93.9 MB/sec

My HDD (SATA)  is  having rate of 120 MB/sec -- via unix's dd-command

Original  mongo-driver for java (both 1.4 and 2.0.rc.X releases) have some performance issues, after some hacking around 2 days, performance is reached to above level (3 times). Will shortly publish the changed code at mongodb-site.

Will shortly publish more code/benchmarking data :)