Kubernetes cluster migration using Velero

Migration is a natural phenomena

Velero (formerly known as Heptio Ark) is mainly used for backup and restore for kubernetes clusters, to handle any disaster scenarios.

It can also be used as a migration tool by using its native disaster management technique . This is bit tricky and complex technology but at the same time very simple for the kubernetes admins to do the migration from one cluster to another.

For more details on velero click here

Prerequisites:

i. Need to have two kubernetes clusters up and running, one new k8s (the destination cluster)where you want to migrate to and one old k8s cluster( the source cluster) from where you want all workloads to get migrated from including PV and PVCs with the data inside.

ii. Same storage class has to be created in the destination k8s cluster if you are using dynamic provision for your k8s storage solution.

iii. One VM which (preferably out of both the clusters), where your Minio will be running as a storage bucket (for your backup storage which you will take using velero from the old cluster/source cluster).

iv. Velero needs to be installed in both the clusters(source and destination k8s clusters).

Install minio :

Note: Here We have installed minio as docker container with host volume attached to it in /data , make sure you have the enough space to accommodate all the backup data from the old kubernetes/source cluster.

i. Run the below command to install the minio container:

docker run --name minio -p 9000:9000 -v data:/data minio/minio server /data

ii. To access the UI you can access it by <machine_IP>:9000

iii. set the password if you wish to change, that you can change from minio UI only default password for first time to login will be username: minioadmin & password: minioadmin

iv. create a bucket by clicking the “+” button, in our case we have created bucket name as “kubedemo”

Install Velero (repeat the same steps in both the clusters):

i. Download the latest releases of velero setup by from the github link. I have used the version 1.5.2 this can be download from here.

ii. Untar the downloaded velero package by running the command:

tar -xvzf velero-v1.5.2-linux-amd64.tar.gz

iii. move the velero executable to your local systems bin :

sudo mv velero-v1.5.2-linux-amd64/velero /usr/local/bin/

iv. check the velero version by running below command which will show you the version installed:

velero version
output:-
Client:
        Version: v1.5.2
        Git commit: e115e5a191b1fdb5d379b62a35916115e77124a4
Server:
        Version: v1.5.2

NOW YOU ARE GOOD TO GO!!!!!

v. create minio credential in some location in the master machine where you are running velero install command by using the below command: ( in our case we have created this in /root/minio.credential) :

cat <<EOF>> /root/minio.credentials
[default]
aws_access_key_id=minioadmin
aws_secret_access_key=minioadmin
EOF

vi. Now the final command to install the velero:

velero install --provider aws --plugins velero/velero-plugin-for-aws:v1.0.0 --bucket kubedemo --secret-file /root/minio.credentials --use-volume-snapshots=false --default-volumes-to-restic --backup-location-config region=minio,s3ForcePathStyle="true",s3Url=http://<minio_machine_ip>:9000 --use-restic

you can see the below pods will come up post the installation of Velero, where Velero will get deployed as deployment and will have one replica and restic will be deployed as daemon set :

[root@velro ~]# kubectl get po -n velero
NAME READY STATUS RESTARTS AGE
restic-2ww8r 1/1 Running 1 20d
restic-hkhrq 1/1 Running 1 26d
velero-7f847b4bc9-pl58d 1/1 Running 0 4d12h

Note: i. Here using the flag “--default-volumes-to-restic” will allow you to take auto backup of your PV,PVCs also with data, but this feature is enabled in 1.5.2 and above version of Velero.

ii. For the lower versions i.e.< 1.5.2 you need to annotate each pod with the volume mounts to take pod volume backup .

Backups and restore :

i. Run the backup command from the cluster which you are planning to migrate all the k8s workloads from (prefereably your older cluster/source cluster)

a. For velero>=1.5.2 version( which is our case here as the auto volume backup will come as part of the below command)

velero backup create myfirstbackup-bkp --include-namespaces my-namespace --wait

b. if you are using the velero < 1.5.2 (where the auto volume snapshot feature is not enabled) you need to run below two commands:

1. Fist annotate the pod with the volume you want to take backup of:
kubectl -n test-nginx annotate pod/my-nginx backup.velero.io/backup-volumes=myvolume-data 
2. Then run the backup command:
velero backup create mynginx-bkp --include-namespaces test-nginx --wait

ii. Run this command in the destination cluster/new k8s cluster where you want to migrate the resources to:

velero restore create myfirsrestore-rst --from-backup myfirstbackup-bkp --wait

Some useful commands :

i. To check the backups taken by velero:

velero backup get

ii. To check the restores done so far:

velero restore get

iii. to check the logs of failed backups/retores:

velero backup logs <backup-name>
velero restore logs <restore-name>

for more details and queries:-

Join us on Slack