# Creating Redis Cluster with Docker and Compose

Hi readers! This is my first time writing a blog for my personal projects related to software engineering and computer science stuff. As for my first blog, I would like to write about setting up Redis cluster with docker. A little bit of background, I wrote this based on my previous struggle when improving my task at my company by using Redis. Before deploying the updated feature to the testing environment, of course we should test it locally. This is where I found the challenges:

1. There is little to no documentation about setting up redis locally at my current company
    
2. Everyone seems to have little knowledge about the implementation and I have been quite misguided on how to set the Redis cluster. Thankfully, by approaching the colleauge who set this up the first time on the code base (by using git blame) I can set the cluster successfully, but without using Docker
    

So based on those two points of background, I wrote this blog so you guys may not need to have the same struggle of switching back and forth between Redis docs, Docker docs, StackOverflow, and Slack 🙃.

## What is Redis

As the [web](https://redis.io/) says, Redis is an in-memory data store that is often used as a database, cache, streaming engine, and message broker. The data format on Redis is different than SQL or any other usual NoSQL databases because the data are saved in the Key-Value pair. The advantage is the value can store complex formats such as Java class containing attributes. Another advantage of using Redis is its speed on reading and writing data because it uses the memory. Just like having a HashMap variable in your Java code, instead the HashMap can be used system-wide by other services. More on the characteristics of Redis can be read on [its website](https://redis.io/docs/getting-started/faq/).

![Redis Logo source: https://www.google.com/imgres?imgurl=https%3A%2F%2Fupload.wikimedia.org%2Fwikipedia%2Fcommons%2Fthumb%2F6%2F64%2FLogo-redis.svg%2F2560px-Logo-redis.svg.png&imgrefurl=https%3A%2F%2Fcommons.wikimedia.org%2Fwiki%2FFile%3ALogo-redis.svg&tbnid=IFMpyGNQHT129M&vet=12ahUKEwiXuMqq47f8AhVX73MBHZ6ZCSgQMygAegUIARDJAQ..i&docid=iHABgCOnp6IM9M&w=2560&h=874&q=redis%20logo&ved=2ahUKEwiXuMqq47f8AhVX73MBHZ6ZCSgQMygAegUIARDJAQ](https://upload.wikimedia.org/wikipedia/commons/thumb/6/64/Logo-redis.svg/2560px-Logo-redis.svg.png align="left")

We may not need Redis at all in our daily use cases. But production-wise, when we are faced with heavy transactions and high-reliability requirements, that is when Redis would be quite useful.

## What is Docker Network and Why We Need It for Redis Cluster

I assume that you have already known or are familiar with Docker and container. If not, then you can read about Docker through the [documentation](https://docs.docker.com/). Or surf the internet about containers. A feature that we need in order to incorporate Redis into Docker is Docker Network. Docker Network is a feature that enables/prohibits a container from communicating with other containers or with the host. Docker network has several types of networks that we can use, such as bridge and host networks. For using Redis Cluster inside Docker, we need to use the network called *user-defined bridge* in Docker. To create it, we just need to execute this command. This network will be used in the next sections.

```bash
docker network create redis-cluster
```

The command above creates a user-defined network called `redis-cluster` . Or you can use another name for the network.

## The Code

There are 2 images that we need to pull first. Those images are:

1. Redis image, to run the individual Redis node
    
2. RedisInsight image, to run the **RedisInsight** as a visualizer to see what key exists in the database. Just like pgAdmin for PostgreSQL.
    

Execute the commands below

```bash
docker pull redis
docker pull redislabs/redisinsight
```

After that, we need to create a separate folder for each node. Redis recommends nodes with a minimum number of 6. So we will create those nodes step by step below.

1. Create six different folders. For convenience, I created the folder name with their respective port assignment
    
    ```bash
    mkdir 7000 7001 7002 7003 7004 7005
    ```
    
2. For each folder, create a file with the name `redis.conf` which contains the following
    
    ```bash
    port 7000
    cluster-enabled yes
    cluster-config-file nodes.conf
    cluster-node-timeout 5000
    appendonly yes
    bind 0.0.0.0
    ```
    
3. After creating all the `redis.conf` files for each folder, go back to the parent folder that contains these folders and open 6 separate terminals. And execute this command
    
    ```bash
    docker run -v `pwd`/7000:/redis --name node-1 -p 7000:7000 --network redis-cluster redis redis-server /redis/redis.conf
    ```
    
    *Note:*
    
    1. The `` `pwd`/7000 `` should be written according to the name of the folder. In this case, my folder is named the same as the port.
        
    2. Export the appropriate port for each node. If port inside the `redis.conf` is written as 7002, then expose port 7002.
        
    3. The container name for this node is `node-1` . You can custom it to another name, this should also be done to the other nodes for convenience.
        
4. Open the 7th terminal (I know that's a lot of terminals) and connect to one of the nodes and call `redis-cli` to create clusters. In my case, I connected to the `node-1` with port 7000.
    
    ```bash
    docker exec -it node-1 bash
    ```
    
    Redis will create the configuration automatically and show the prompt below. Type `yes` . After that, Redis will auto-create the configuration for the nodes
    
    ![](https://cdn.hashnode.com/res/hashnode/image/upload/v1673875780771/2b3902d5-2cd1-48f6-8954-0a921f5c8aa1.png align="center")
    
5. If no error, then the cluster is successfully created. Now on the same terminal, we will create another folder for RedisInsight, use that folder to persist the dump produced by RedisInsight and run the RedisInsight image connected to `redis-cluster` network
    
    ```bash
    mkdir redisinsight
    docker run -v `pwd`/redisinsight:/db -p 8001:8001 --network redis-cluster redislabs/redis-insight
    ```
    
6. After that open `localhost:8001` on your browser to see the RedisInsight is up and running. Input one of the node address and port. In this post, I am using the node with port 7000. Check all the cluster nodes listed on the RedisInsight. All the nodes that we created earlier should be visible. The step by step pictures can be seen below. Click "I already have a database"
    
    ![](https://cdn.hashnode.com/res/hashnode/image/upload/v1673876381530/8ccda489-6d07-4d08-ac43-fb3daed344c5.png align="center")
    
    Click "Connect to a Redis Database"
    
    ![](https://cdn.hashnode.com/res/hashnode/image/upload/v1673876390871/fc77a387-191c-4f70-b79c-35bc909b1252.png align="center")
    
    Input the host, port, and name. Since we are connected to a docker bridge network, we can refer the host with its host name instead of IP address
    
    ![](https://cdn.hashnode.com/res/hashnode/image/upload/v1673876463410/a745cd27-98df-4319-bfdd-53f125ad04b3.png align="center")
    
    Click "Select all". After that you can explore the redis insight by yourself.
    
    ![](https://cdn.hashnode.com/res/hashnode/image/upload/v1673876582266/682a7384-b2f2-4b13-ae46-5fa0bf557dc2.png align="center")
    
7. To check that the cluster is working properly, try to set a key by opening redis-cli on one of the node
    
    ```bash
    docker exec -it node-1 bash
    redis-cli -p 7000
    SET lalala lilili
    ```
    
8. After that, check the RedisInsight. The key-value that we have just set should also be visible
    

## Using Docker Compose to Simplify Things

> note: if you are executing the steps below with the same folders as above, Try to delete the files inside 7000, 7001,... folders except the `redis.conf` and delete all the previously created containers

All the steps above seem to be exhausting and repetitive. But I think it is quite important to understand the fundamental concepts. After we have understood it quite well, we can simplify those repetitive tasks by using Docker Compose. The `docker-compose.yml` file content can be seen here

%[https://gist.github.com/ilhamdcp/146dd1124bf431a6eaba73e8df8def32] 

After that, you can go to `localhost:8001` on your browser and fill in just like picture below

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1674053385100/8414b667-cef4-4323-8d96-560ebcc62cc4.png align="center")

And then, just like before, select all the nodes and continue

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1674053330513/28903a12-11bb-41b9-ab5c-4877514996ab.png align="center")

There are several things that you may want to take note about the `docker-compose.yml`:

1. Previously, the nodes are named `node-<number>` . Here on the docker compose, I change the node name to `redis-node-<number>`
    
2. Define the network using the configuration syntax below.
    
    ```yaml
    networks:
      redis-cluster-compose:
        driver: bridge
    ```
    
    I also distinguish the network name from before to avoid conflict. Since we use the bridge network, we can easily refer to other container that connected to the same network using its name directly instead of the IP address.
    
3. The services include the nodes and the Redisinsight. But, there is also one service configuration which can be seen below
    
    ```yaml
      redis-cluster-creator:
        image: redis:latest
        ports:
          - 6999:6999
        networks:
          - redis-cluster-compose
        command: redis-cli -p 7000 --cluster create redis-node-1:7000 redis-node-2:7001 redis-node-3:7002 redis-node-4:7003 redis-node-5:7004 redis-node-6:7005 --cluster-replicas 1 --cluster-yes
        depends_on:
          - redis-node-1
          - redis-node-2
          - redis-node-3
          - redis-node-4
          - redis-node-5
          - redis-node-6
    ```
    
    The purpose of this service is just to create the cluster by running the `redis-cli` command. You can use the `&&` by combining the `redis-cli` command after the `redis-server ./redis.conf` by adding `sh -c` or `bash -c` . [See on StackOverflow](https://stackoverflow.com/questions/30063907/docker-compose-how-to-execute-multiple-commands). I also added the `--cluster-yes` to auto-accept the prompt (noninteractive)
    

## Conclusion

Redis is an interesting tool that we can incorporate into a company's codebase or explore as our project. But, there are also challenges when using it. One of them is running a Redis cluster locally due to the long process of executing the individual nodes. That is where docker-compose comes to the rescue by configuring a script and we can use it multiple times. The other services that would want to use this Redis cluster must be connected to the docker container, meaning they also must be inside a Docker container. If you want to create a Redis cluster without a Docker or Docker Compose, that is also possible and you can create your custom bash script to automate it.