Migrating a Linux-Server from one VPS to another over network

If you are interested in some background information — I moved this to the bottom. I will describe my surprisingly easy way to move my Linux (Ubuntu 16.04 LTS) server hosted on a VPS with all data and configuration (including all lets encrypt certificates for my https:-Websites) to a new machine (VPS from different vendor) with a significant different hardware profile! This was done completely over network requiring no physical access to neither source nor target. Additionally I never had to have detailed knowledge about setting up a machine at the hardware level. You just clone and change the few things which are not done automatically.

Clearly the first two steps can be used to backup/restore a VPS server to any location reachable via ssh (or any other target like nfs or smb clonezilla allows).

Prerequisites

You need

• VPS1 — source: My running, rather big server (Ubuntu 16.04 LTS) with LAMP installed, serving at least five important web sites and some utilities like Owncloud etc.
• VPS2 — auxiliary: A running, smaller server (Ubuntu 16.04 LTS) used only for Bigbluebutton with plenty of free space. Used to store the image. With prices being very low you could even rent a server just for this purpose.
• VPS3 — target: My newly bought machine, setup with (Ubuntu 16.04 LTS out of box).

The first two machines have been setup by me with a very basic setup, I used Digital Oceans very easy to follow and concise recipe. Most important: ssh access. Source and target, although from different providers, can be configured to boot into a rescue system able to run clonezilla which have been/could be e.g.

• grml on VPS1,
• Clonezilla Live on VPS2/VPS3

In both cases you have to use VNC to enter commands in this situation as your servers are not running — necessary login/access data should be provided by your vendor.

Step 1 — Create an image with Clonezilla

Shut down VPS1, boot from rescue (grml), start clonezilla and create a full image to a ssh-Destination (on VPS2).

This is the only crucial step: establish a ssh-connection to your intermediate/auxiliary server (here with the sudo user admin) to store the image data (here in /var/buxt). Login via ssh create some directory to store it:
ssh admin@mirror.klocker-mark.eu
sudo mkdir /var/buxt

Input this data to the questions asked from clonezilla which will mount this directory to /local/partimg, select your local disk (/dev/vda in my case) as source:

As you can see available disk space was not problem as it took about an hour to create a 60G image from my 160G server.

Step 2 — Restore image

The steps to restore are almost exactly the same as for saving an image. Repeat procedure from step 1 for VPS 3 to fire up the rescue system, start clonezilla, mount your auxiliary ssh-mounted folder and select the appropriate file to restore.

Restore too took about an hour. Clonezilla did most, but not all the magic with the target machine being somewhat different (e. g. different partition scheme using /dev/sda instead of /dev/vda). Most important the size of the disks had not been adjusted.

Step 3 — Adapt a few settings, reboot and enjoy

Enable Networking

Do this first: after this step everything can be done via ssh which is much more comfortable. Boot VPS3 after restoring. Using VNC I was able to login but in my case networking was not functional (so I could not ssh to it). This is easily repaired by finding which network devices are available/connected:
sudo ip addr
sudo ifconfig
After identifying that my new machine did not have an eth0 but instead an ens18, this is changed permanently by changing the according configuration file
sudo vi /etc/network/interfaces
and changing the lines regarding eth0 to
# The primary network interface
allow-hotplug ens18
iface ens18 inet dhcp
The command allow-hotplug instead of auto worked faster than auto.

Change hard coded IP adresses

There have been some places in my server with hard coded IPs from my previous installation, most obviously the hostname in /etc/hosts:
sudo vi /etc/hosts
Only a few others have been found over time, easily changed.

Adjust Partition Sizes (if necessary)

After cloning additional hard disk space — if available — is shown as free space but has to be configured to be used. Two scenarios are possible which are easily identified calling
sudo cfdisk

• Simple partitioning e. g. ext4 Filesystem — /dev/hda2 does not use all the space. Use this recipe — it worked great for me even for ext4.

• Logical volume managing. The steps are somewhat more complicated but this is the much more flexible solution. I followed the steps described here.

Step 4 — Point Domain to new Server

This is the time to actually switch servers! Go to your DNS administration panel and change IPs of all URLs concerned. This worked rather instantly for me but you have to be shure that the client where you are testing flushes its DNS cache to see the new IP (check simply py pinging). In Windows this is done via command line with
ipconfig /flushdns
and on my Mac with
sudo killall -HUP mDNSResponder

Refresh LetsEncrypt

I dont’t really know if that is a necessary step: I did it and it worked. Obviously it can do no harm:
sudo letsencrypt renew --dry-run

Conclusion

To be honest: I scheduled this transfer three times and only last week I actually did it. If I had known earlier how easy this would be I would have done this a year ago. Doing it that way (some time before deleting the old VPS) is absolutely stress-free, as storing/restoring the image can be done many times (if anything goes wrong).

Additionally snapshots available on my VPS make it even easier. Take one after each successful step and revert back if anything fails — in seconds.

Troubleshooting

Trying this recipe again for backing up a little server which I do not need now I ran into two problems which could be solved easily. Dont´t panic if you see these failure messages.

• Failure to find partition… is perhaps misleading. In my case the source disk was bigger than the remaining space on my target (a directory on my production server). Perhaps you can shrink your source (see here) or free up some space on target as I could. The size is taken from uncompressed total space: the target image, consisting of only compressed used space, will be considerably smaller.
• Errors on source disk… this can be solved easily without leaving clonezilla, you just have to check the option to run repair beforehand. I had to and bravely selected “automatic“ which worked flawlessly.

As stated above all this cloning is done on a shutdown VPS server for which you should have a recent snapshot.

History

Just for completeness here is some background information. Since more than three years I am using a self-configured VPS (virtual private server, named #1 here) for hosting my private blog, my professional blog (which you are reading right now) and my educational site consisting of Moodle/Mahara. Adding BigBlueButton required a second VPS (#2) which at that time was very affordable: €10 for 6 CPUs/12G RAM/200 G SSD. That would be just as much as my bill of electricity would be housing this server at home — and the power of such a server easily suffices to serve my applications.
Over time my first VPS was too small, I had to upgrade, especially disk space was not sufficient. This provider, although very reliable, asked much more for a similar VPS so I had to decide: stay with very limited resources or migrate to another provider. But migration seemed to be a very complicated and risky task and I would not allow my users downtimes of more than one day.
Needless to say that starting over with a new plain vanilla machine (Ubuntu 16.04 LTS) was no option as over time so many customizations or problem solving patches have been applied that it seemed impossible to do all that again.
So a quick search gave some hints how this could be done. I tried two of them, the way described here two times and this was so little effort that I really was amazed. What had I been waiting for?

Image from Colossus Cloud on Pixabay