Skip to content

a distributed cluster for memcached with bi-directional replication

Notifications You must be signed in to change notification settings

downgoon/memcloud

Repository files navigation

memcloud

Join the chat at https://gitter.im/memcloud/Lobby

a distributed cluster for memcached with bi-directional replication. it was designed and developed on 2012, so some of technologies used in this project are not so fashion. But its reliability and manageability have been seen in production environments for many years.

architecture

memcloud is a distributed cluster for memcached with bi-directional replication. it consists of four components including mem-dns, mem-agent, mem-client and mem-alert.

by the way, a more lite version (memcloud-0.1.0 automan) is recommended if you just only want a bi-directional replicated memcached in case of data corruption.

main working flow

memcloud architecture

the mem-agent here mainly refers to a daemon of memcached instance with bi-directional replication (replication-testing is here) to its peer memcached.

mem-agent installation and registration

mem-agent

actually the mem-agent is a software package which contains at least two bash scripts named memcloud_install.sh and memcloud.sh. memcloud_install.sh is responsible for automatical installation and easy registration of memcached instance with bi-directional replication, and the other for starting memcached daemon.

mem-alert insight

mem-alert


how to start

the order of starting

  1. start mem-dns
  2. start mem-agent
  3. start mem-alert
  4. start mem-client

start mem-dns

download memcloud-dns-0.2.0.tar.gz, unpack it and start-memdns.sh.

$ wget https://github.com/downgoon/memcloud/releases/download/0.2.0/memcloud-dns-0.2.0.tar.gz
$ tar zxvf memcloud-dns-0.2.0.tar.gz
$ cd memcloud-dns-0.2.0
$ start-memdns.sh

NOTE

Before start-memdns.sh, please make sure its dependencies including mysqld and mongodb in the configuration are ready.

start mem-agent

download memcloud-agent-0.2.0.tar.gz, unpack it and memcloud-install.sh

$ wget https://github.com/downgoon/memcloud/releases/download/0.2.0/memcloud-agent-0.2.0.tar.gz
$ tar zxvf memcloud-agent-0.2.0.tar.gz
$ cd memcloud-agent-0.2.0
$ memcloud_install.sh

after installation, you can start it according to the chapter named Quick-Start#how-to-start-memcloud

start mem-alert

$ wget https://github.com/downgoon/memcloud/releases/download/0.2.0/memcloud-alert-0.2.0.tar.gz
$ tar zxvf memcloud-alert-0.2.0.tar.gz
$ cd memcloud-alert-0.2.0
$ bin/memalert start

NOTE

Before bin/memalert start, please make sure its dependency mongodb in the configuration is ready.

start mem-client

memcloud client sample code is as follows:

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import io.memcloud.client.MemCloudClient;

public class MemCloudClientDemo {

	private static final Logger LOG = LoggerFactory.getLogger(MemCloudClientDemo.class);

	public static void main(String[] args) throws Exception {

		System.setProperty("memcloud.memdns", "your-memcloud-dns-host.example.com");

    // appid allocated by mem-dns
		String appid = "10021";
		MemCloudClient mcc = MemCloudClient.buildDefault(appid);

		// set
		int i = 0;
		while (i < 10) {
			mcc.getMemcachedClient().set("k" + i, 60 * 5, "v" + i);
			i++;
		}

		// get

		i = 0;
		while (i < 10) {
			String value = mcc.getMemcachedClient().get("k" + i);
			LOG.info("value from memcloud: {}", value);
			i++;
		}
	}
}

add the dependency into pom.xml

<dependency>
	<groupId>io.memcloud</groupId>
	<artifactId>memcloud-client</artifactId>
	<version>0.2.0</version>
</dependency>

Some Snapshots

mem-dns sign-in

mem-dns sign-in

create an application

create an application

asking for memcached allocated to the given application

apply for memcached

application listing

application listing

authorize a requisition

authorize a requisition

health monitor

health monitor

bi-directional replication

bi-directional replication


how to build

$ git clone https://github.com/downgoon/memcloud.git
$ cd memcloud
$ mvn clean package