Skip to main content

VDO

Updated Dec 31, 2021 ·

VDO​

VDO (Virtual Data Optimizer) operates as a block device virtualization layer that sits between the file system and physical storage. It optimizes storage by reducing data redundancy and compressing data before it's written to disk.

  • Used as a separate volume manager on top of which file systems will be created.
  • Identifies and eliminates duplicate data blocks, reducing storage requirements.
  • Compresses data before storage to save disk space.
  • Efficiently manages storage allocation, allocating space only as data is written.
  • Provides block-level storage optimization, compatible with various file systems.

VDO is implemented through a combination of software (kernel modules and utilities) and hardware capabilities (supporting systems with compatible hardware).

Provides thin-provisioned storage:

  • Uses a logical size of 10 times the physical size for VMs and containers.
  • Use a logical size of 3 times the physical size for object storage.

Lab Setup​

I'm performing this lab on an EC2 instance with multiple EBS volumes attached. I also used this same setup for the Stratis labs. To ensure everything is a fresh new block, we'll need to wipe out the block devices.

$ lsblk
NAME    MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
xvda    202:0    0  10G  0 disk
├─xvda1 202:1    0   1M  0 part
└─xvda2 202:2    0  10G  0 part /
xvdb    202:16   0  10G  0 disk

Remove any existing filesystem and wipe out the block device xvdb:

wipefs -a /dev/xvdb
blkid -p /dev/xvdb

Installation​

Install the necessary packages and reboot the system:

sudo yum install -y vdo kmod-kvdo
sudo reboot

Enable and start the VDO (Virtual Data Optimizer) service:

systemctl enable vdo
systemctl start vdo
systemctl status vdo

Create VDO devices​

note

Ensure underlying block devices are >4GIB

Create a VDO device named vdo1 using /dev/xvdb with a logical size of 10 terabytes:

$ vdo create --name=vdo1 --device=/dev/xvdb --vdoLogicalSize=10T
Creating VDO vdo1
      The VDO volume can address 6 GB in 3 data slabs, each 2 GB.
      It can grow to address at most 16 TB of physical storage in 8192 slabs.
      If a larger maximum size might be needed, use bigger slabs.
Starting VDO vdo1
Starting compression on VDO vdo1
VDO instance 0 volume is ready at /dev/mapper/vdo1

Format the VDO device vdo1 with XFS filesystem:

$ mkfs -t xfs -K /dev/mapper/vdo1
meta-data=/dev/mapper/vdo1       isize=512    agcount=10, agsize=268435455 blks
         =                       sectsz=4096  attr=2, projid32bit=1
         =                       crc=1        finobt=1, sparse=1, rmapbt=0
         =                       reflink=1
data     =                       bsize=4096   blocks=2684354550, imaxpct=5
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0, ftype=1
log      =internal log           bsize=4096   blocks=521728, version=2
         =                       sectsz=4096  sunit=1 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0

Wait for the system to register the new device name:

udevadm settle

Create a mount point for vdo1:

$ mkdir /mnt/diskvdo
$ ll /mnt/
total 0
drwxr-xr-x. 2 root root 6 Jan  2 15:39 diskfs
drwxr-xr-x. 2 root root 6 Jan  2 15:33 diskfs2
drwxr-xr-x. 2 root root 6 Jan  2 16:24 diskvdo

Add an entry to /etc/fstab for automatic mounting:

$ vim /etc/fstab

# VDO
/dev/mapper/vdo1                         /mnt/diskvdo             xfs     x-systemd.requires=vdo.service      0 0

Mount all filesystems defined in /etc/fstab:

mount -a

Verify that vdo1 is mounted correctly:

$ mount | grep vdo
/dev/mapper/vdo1 on /mnt/diskvdo type xfs (rw,relatime,seclabel,attr2,inode64,logbufs=8,logbsize=32k,noquota,x-systemd.requires=vdo.service)

vdostats​

Use vdostats to monitor and display statistics related to VDO (Virtual Data Optimizer) volumes.

$ vdostats
Device               1K-blocks      Used Available Use% Space saving%
/dev/mapper/vdo1      10485760   4202108   6283652  40%           99%
$ vdostats --human-readable
Device                    Size      Used Available Use% Space saving%
/dev/mapper/vdo1         10.0G      4.0G      6.0G  40%           99%

Removing VDO​

Before removing a VDO (Virtual Data Optimizer) volume, it's crucial to ensure that the volume is properly unmounted and deactivated to avoid data loss or corruption.

$ lsblk
NAME    MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
xvda    202:0    0  10G  0 disk
├─xvda1 202:1    0   1M  0 part
└─xvda2 202:2    0  10G  0 part /
xvdb    202:16   0  10G  0 disk
└─vdo1  253:0    0  10T  0 vdo  /mnt/diskvdo

Unmount the VDO device:

umount /mnt/diskvdo

Deactivate and remove the VDO device vdo1:

$ vdo deactivate --all
Deactivating VDO vdo1
$ vdo remove --name vdo1
Removing VDO vdo1
Stopping VDO vdo1

As an option, we can wipe out the whole block device.

wipefs -a /dev/xvdb
blkid -p /dev/xvdb