A first sight at Riak
It’s been a long time since I wanted to test RIAK, a “NoSQL” (what a fucking stupid word) database from the Basho team. It’s Dynamo-inspired, you can process data through “MapReduce” jobs on server side, and it’s content-agnostic. It can scale easily on numerous nodes and it aims at being strongly fault tolerant, so no node is particular in a riak cluster.
Have you ever heard about the CAP theorem ? In brief, it says that “In a distributed system, you can’t get a perfect Consistency, Availability and Partition tolerance at the same time”. It’s like that, it’s been proved, now you’ll have to live with that. Riak focuses on the A and P sides, which means a Riak cluster is not always perfectly Consistent (it’s not boolean, your consistency needs can be tuned). A first node can have some data, and you might have to wait a certain amount of time before this data is coherent with the node you query. Which is perfectly fine in most cases.
Install Riak
What’s fun with Riak is the scalability thing. If you want to test it, the wiki has a great tutorial about Building a Development Environment. Erlang was a bit too old on my Ubuntu 10.10 (R13B03 instead of required R13B04), so I had to get a fresh version from sources:
#!/bin/sh sudo aptitude install build-essential libncurses5-dev openssl libssl-dev [ -f otp_src_R13B04.tar.gz ] || wget http://erlang.org/download/otp_src_R13B04.tar.gz tar zxvf otp_src_R13B04.tar.gz cd otp_src_R13B04 ./configure && make && sudo make install
Then I installed Riak latest stable release (0.14.1). I could have taken current master branch too, but I wanted something stable for my first steps into :
#!/bin/sh [ -f riak-0.14.1.tar.gz ] || curl -O http://downloads.basho.com/riak/riak-0.14/riak-0.14.1.tar.gz tar zxvf riak-0.14.1.tar.gz mv riak-0.14.1 riak cd riak make all make devrel
It built me 3 nodes as expected, located in ./riak/dev folder. So let’s start the 3 nodes and cluster-ize them :
% cd riak/dev % ./dev1/bin/riak start % ./dev2/bin/riak start % ./dev3/bin/riak start % ./dev2/bin/riak-admin join dev1@127.0.0.1 % ./dev3/bin/riak-admin join dev1@127.0.0.1
Command line
We have 2 binaries :
% dev1/bin/riak
Usage: riak {start|stop|restart|reboot|ping|console|attach}
% dev1/bin/riak-admin
Usage: riak-admin { join | leave | backup | restore | test | status |
reip | js_reload | wait-for-service | ringready |
transfers | remove }
riak-admin allows us to see the parameters of a running instance through the status command, and especially the state of our
ring (= cluster of riak nodes) :
% dev1/bin/riak-admin status |grep ring_
ring_members : ['dev1@127.0.0.1','dev2@127.0.0.1','dev3@127.0.0.1']
ring_num_partitions : 64
ring_ownership : <<"[{'dev3@127.0.0.1',21},{'dev2@127.0.0.1',21},{'dev1@127.0.0.1',22}]">>
ring_creation_size : 64
And we can see if the ring is ready to serve requests :
% dev1/bin/riak-admin ringready
TRUE All nodes agree on the ring ['dev1@127.0.0.1','dev2@127.0.0.1',
'dev3@127.0.0.1']
We can even test if a node is able to read/write data :
% dev1/bin/riak-admin test =INFO REPORT==== 21-Dec-2010::18:35:06 === Successfully completed 1 read/write cycle to 'dev1@127.0.0.1'
NB: I didn’t manage to make this test fail, I don’t really know what it tests. Restricting read/write permissions on dev*/data/ doesn’t seem
to affect it, but I presume Riak buffers things in memory in such cases, so it would need some more tests…
HTTP Exploration
Riak has a full HTTP API, so it’s dead simple to dialog with your ring (from any of its nodes). It returns JSON, so it’s easy to parse (formatting is mine in the following examples).
You can organize data into buckets, which are containers for storing elements of the same type. You can get informations on a bucket
(existing or not) with :
% curl http://localhost:8091/riak/MyBucket
{
"props": {
"name": "MyBucket",
"n_val": 3,
"allow_mult": false,
"last_write_wins": false,
"precommit": [],
"postcommit": [],
"chash_keyfun": {
"mod": "riak_core_util",
"fun": "chash_std_keyfun"
},
"linkfun": {
"mod": "riak_kv_wm_link_walker",
"fun": "mapreduce_linkfun"
},
"old_vclock": 86400,
"young_vclock": 20,
"big_vclock": 50,
"small_vclock": 10,
"r": "quorum",
"w": "quorum",
"dw": "quorum",
"rw": "quorum"
}
}
If the bucket doesn’t exist, it will be created as soon as you insert an element in it.
You can get the list of all existing buckets with :
% curl http://localhost:8091/riak?buckets=true
{"buckets":["articles","rekon"]}
And get some general stats on the node you query (the result is quite similar to stats on a particular bucket) :
% curl http://localhost:8091/riak/stats
{
"props": {
"name": "stats",
"n_val": 3,
"allow_mult": false,
"last_write_wins": false,
"precommit": [],
"postcommit": [],
"chash_keyfun": {
"mod": "riak_core_util",
"fun": "chash_std_keyfun"
},
"linkfun": {
"mod": "riak_kv_wm_link_walker",
"fun": "mapreduce_linkfun"
},
"old_vclock": 86400,
"young_vclock": 20,
"big_vclock": 50,
"small_vclock": 10,
"r": "quorum",
"w": "quorum",
"dw": "quorum",
"rw": "quorum"
}
}
But I wanna haz some GUI !
Not happy with the command line ? Adam Hunter built Rekon, a data browser for your riak node, built as a riak app (only riak objects and javascript, yeah). Installation is pretty simple, in our case for our “dev1” node :
curl -s get.rekonapp.com | node=127.0.0.1:8091 sh
Then navigate to http://localhost:8091/riak/rekon/go, and tadam :
I let you play with it, it’s intuitive enough.
Conclusion
Riak is an amazing piece of software. In future posts, I’ll try to discuss how you can query it from a Rails app with the Ripple client library, and some other exciting things such as deployment and system administration things.