One of the prime job of Unix administrators will be extending and reducing the volumes/filesystems according to the application team requirement.In Veritas volume manages,we carry out such a tasks in online without un-mounting the filesystems. To increase or reduce the filesystem, you need to add or remove the disks from the diskgroup in vxvm. But in ZFS, once you have added the disk to the zpool,you can’t remove it unless it has valid vdev(virtual devices) in that zpool. So the bottom line is ,you can’t reduce the zpool size but it can be increased on fly by adding new LUNS or DISKS to the zpool.
In ZFS terminology ,volumes are nothing but ZFS dataset and dataset size can be increased or reduced on the fly without any problem.Check out Extending and reducing the ZFS datasets aka volumes (Task:3).
Topics:
5. Extending ZPOOL and zpool relayout(We are Here)
7. How to replace the failed Disk n ZFS ZPOOL
Task:1-Extending ZPOOL
First let see about extending stripped zpool.
Stripped ZPOOL:
1.Check the zpool status.
root@Unixarena-SOL11:~# zpool status oracle-S
pool: oracle-S
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
oracle-S ONLINE 0 0 0
c8t1d0 ONLINE 0 0 0
errors: No known data errors
root@Unixarena-SOL11:~# zpool list oracle-S
NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT
oracle-S 1.98G 118K 1.98G 0% 1.00x ONLINE -
root@Unixarena-SOL11:~#
2.Extend the zpool by adding a new LUN.
root@Unixarena-SOL11:~# zpool add oracle-S c8t2d0
root@Unixarena-SOL11:~# zpool status oracle-S
pool: oracle-S
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
oracle-S ONLINE 0 0 0
c8t1d0 ONLINE 0 0 0
c8t2d0 ONLINE 0 0 0
errors: No known data errors
3.Check the oracle-S zpool size now.It will be extended by 2GB.
root@Unixarena-SOL11:~# zpool list oracle-S
NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT
oracle-S 3.97G 138K 3.97G 0% 1.00x ONLINE -
root@Unixarena-SOL11:~#
Mirrored Zpool:
1.Check the zpool layout.
root@Unixarena-SOL11:~# zpool status oracle-M
pool: oracle-M
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
oracle-M ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
c8t1d0 ONLINE 0 0 0
c8t2d0 ONLINE 0 0 0
errors: No known data errors
root@Unixarena-SOL11:~#
2.Lets try to add disk without mirror on zpool oracle-M.
root@Unixarena-SOL11:~# zpool add oracle-M c8t3d0
vdev verification failed: use -f to override the following errors:
mismatched replication level: pool uses mirror and new vdev is disk
Unable to build pool from specified devices: invalid vdev configuration
root@Unixarena-SOL11:~#
So you can’t add disk without mirror copies in mirror zpool and it make sense.
3.Extend the mirror zpool with valid vdev’s.
root@Unixarena-SOL11:~# zpool add oracle-M mirror c8t3d0 c8t4d0
root@Unixarena-SOL11:~# zpool status oracle-M
pool: oracle-M
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
oracle-M ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
c8t1d0 ONLINE 0 0 0
c8t2d0 ONLINE 0 0 0
mirror-1 ONLINE 0 0 0
c8t3d0 ONLINE 0 0 0
c8t4d0 ONLINE 0 0 0
errors: No known data errors
root@Unixarena-SOL11:~# zpool list oracle-M
NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT
oracle-M 3.97G 92.5K 3.97G 0% 1.00x ONLINE -
root@Unixarena-SOL11:~#
That’s it .We have successfully extended mirror zpool.
RAIDZ Zpool:
1.Check the existing RAIDZ pool status.
root@Unixarena-SOL11:~# zpool status oracle-RZ
pool: oracle-RZ
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
oracle-RZ ONLINE 0 0 0
raidz1-0 ONLINE 0 0 0
c8t1d0 ONLINE 0 0 0
c8t2d0 ONLINE 0 0 0
c8t3d0 ONLINE 0 0 0
errors: No known data errors
root@Unixarena-SOL11:~# zpool list oracle-RZ
NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT
oracle-RZ 5.94G 176K 5.94G 0% 1.00x ONLINE -
root@Unixarena-SOL11:~#
2.Add a new LUN/Disk to the oracle-RZ to extend the zpool size.
root@Unixarena-SOL11:~# zpool add oracle-RZ raidz c8t4d0 c8t5d0 c8t6d0
3.Check the zpool status.
root@Unixarena-SOL11:~# zpool add oracle-RZ raidz c8t4d0 c8t5d0 c8t6d0
root@Unixarena-SOL11:~# zpool status oracle-RZ
pool: oracle-RZ
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
oracle-RZ ONLINE 0 0 0
raidz1-0 ONLINE 0 0 0
c8t1d0 ONLINE 0 0 0
c8t2d0 ONLINE 0 0 0
c8t3d0 ONLINE 0 0 0
raidz1-1 ONLINE 0 0 0
c8t4d0 ONLINE 0 0 0
c8t5d0 ONLINE 0 0 0
c8t6d0 ONLINE 0 0 0
errors: No known data errors
root@Unixarena-SOL11:~# zpool list oracle-RZ
NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT
oracle-RZ 11.9G 186K 11.9G 0% 1.00x ONLINE -
root@Unixarena-SOL11:~#
In the similar way you can extend the raidz2 and raidz3 zpools as well.
Task:2 ZFS- Zpool Re-layout
I don;t see any official oracle documents which talks about ZFS online zpool relayout. From My experience , i have find out the following ways to do the relayout .
Covert the stripe zpool to mirror zpool:
It can be done by just adding mirror to it.
1.List the stripe zpool.
root@Unixarena-SOL11:~# zpool status oracle-S
pool: oracle-S
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
oracle-S ONLINE 0 0 0
c8t1d0 ONLINE 0 0 0
errors: No known data errors
root@Unixarena-SOL11:~# zpool list oracle-S
NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT
oracle-S 1.98G 152K 1.98G 0% 1.00x ONLINE -
root@Unixarena-SOL11:~#
2.Convert the oracle-S pool layout to mirror using “zpool attach” command.
root@Unixarena-SOL11:~# zpool attach oracle-S c8t1d0 c8t2d0
root@Unixarena-SOL11:~# zpool status oracle-S
pool: oracle-S
state: ONLINE
scan: resilvered 86.5K in 0h0m with 0 errors on Wed Jul 24 13:16:46 2013
config:
NAME STATE READ WRITE CKSUM
oracle-S ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
c8t1d0 ONLINE 0 0 0
c8t2d0 ONLINE 0 0 0
errors: No known data errors
root@Unixarena-SOL11:~# zpool list oracle-S
NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT
oracle-S 1.98G 126K 1.98G 0% 1.00x ONLINE -
root@Unixarena-SOL11:~#
Covert the mirror zpool to striped zpool:
We can convert to striped zpool by just removing the mirror.
1.Check the zpool status .
root@Unixarena-SOL11:~# zpool status oracle-S
pool: oracle-S
state: ONLINE
scan: resilvered 86.5K in 0h0m with 0 errors on Wed Jul 24 13:16:46 2013
config:
NAME STATE READ WRITE CKSUM
oracle-S ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
c8t1d0 ONLINE 0 0 0
c8t2d0 ONLINE 0 0 0
errors: No known data errors
root@Unixarena-SOL11:~# zpool list oracle-S
NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT
oracle-S 1.98G 126K 1.98G 0% 1.00x ONLINE -
root@Unixarena-SOL11:~#
2.Convert the zpool to stripe layout using “zpool detach” command.
root@Unixarena-SOL11:~# zpool detach oracle-S c8t2d0
root@Unixarena-SOL11:~# zpool status oracle-S
pool: oracle-S
state: ONLINE
scan: resilvered 86.5K in 0h0m with 0 errors on Wed Jul 24 13:16:46 2013
config:
NAME STATE READ WRITE CKSUM
oracle-S ONLINE 0 0 0
c8t1d0 ONLINE 0 0 0
errors: No known data errors
root@Unixarena-SOL11:~# zpool list oracle-S
NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT
oracle-S 1.98G 115K 1.98G 0% 1.00x ONLINE -
root@Unixarena-SOL11:~#
Convert the mirror zpool to RaidZ:
There is no direct command to convert the mirror zpool to raidz.
1.Detach the mirror
root@Unixarena-SOL11:~# zpool status oracle-S
pool: oracle-S
state: ONLINE
scan: resilvered 86.5K in 0h0m with 0 errors on Wed Jul 24 13:16:46 2013
config:
NAME STATE READ WRITE CKSUM
oracle-S ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
c8t1d0 ONLINE 0 0 0
c8t2d0 ONLINE 0 0 0
errors: No known data errors
root@Unixarena-SOL11:~# zpool detach oracle-S c8t2d0
root@Unixarena-SOL11:~# zpool status oracle-S
pool: oracle-S
state: ONLINE
scan: resilvered 86.5K in 0h0m with 0 errors on Wed Jul 24 13:16:46 2013
config:
NAME STATE READ WRITE CKSUM
oracle-S ONLINE 0 0 0
c8t1d0 ONLINE 0 0 0
errors: No known data errors
root@Unixarena-SOL11:~#
2.Create a new RAIDZ zpool using the detached disk and two new disks.
root@Unixarena-SOL11:~# zpool create oracle-Z raidz c8t2d0
Unable to build pool from specified devices: invalid vdev specification: raidz requires at least 2 devices
root@Unixarena-SOL11:~# zpool create oracle-Z raidz c8t2d0 c8t3d0 c8t4d0
root@Unixarena-SOL11:~# zpool status oracle-Z
pool: oracle-Z
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
oracle-Z ONLINE 0 0 0
raidz1-0 ONLINE 0 0 0
c8t2d0 ONLINE 0 0 0
c8t3d0 ONLINE 0 0 0
c8t4d0 ONLINE 0 0 0
errors: No known data errors
root@Unixarena-SOL11:~#
3.Migrate the data from oracle-S pool to oracle-Z pool using snapshot.
root@Unixarena-SOL11:/oracle-S# df -h .
Filesystem Size Used Available Capacity Mounted on
oracle-S 2.0G 31K 2.0G 1% /oracle-S
root@Unixarena-SOL11:/oracle-S# ls -lrt
total 6
-rw-r--r-- 1 root root 0 Jul 24 13:34 1
-rw-r--r-- 1 root root 0 Jul 24 13:34 2
-rw-r--r-- 1 root root 0 Jul 24 13:34 3
-rw-r--r-- 1 root root 0 Jul 24 13:34 4
-rw-r--r-- 1 root root 0 Jul 24 13:34 5
-rw-r--r-- 1 root root 0 Jul 24 13:34 6
root@Unixarena-SOL11:/oracle-S#
root@Unixarena-SOL11:~# zfs snapshot oracle-S@25072013
root@Unixarena-SOL11:~# zfs send -R oracle-S@25072013 |zfs recv -v -F -d oracle-Z
receiving full stream of oracle-S@25072013 into oracle-Z@25072013
received 52.2KB stream in 1 seconds (52.2KB/sec)
root@Unixarena-SOL11:~#
4.Verify the data in oracle-Z pool.
root@Unixarena-SOL11:/oracle-Z# df -h .
Filesystem Size Used Available Capacity Mounted on
oracle-Z 2.0G 31K 2.0G 1% /oracle-Z
root@Unixarena-SOL11:/oracle-Z# ls -lrt
total 6
-rw-r--r-- 1 root root 0 Jul 24 13:34 1
-rw-r--r-- 1 root root 0 Jul 24 13:34 2
-rw-r--r-- 1 root root 0 Jul 24 13:34 3
-rw-r--r-- 1 root root 0 Jul 24 13:34 4
-rw-r--r-- 1 root root 0 Jul 24 13:34 5
-rw-r--r-- 1 root root 0 Jul 24 13:34 6
root@Unixarena-SOL11:/oracle-Z#
5.After verifying the data,destroy the oracle-S pool and release the LUN.
root@Unixarena-SOL11:~# zpool destroy oracle-S
root@Unixarena-SOL11:~#
The same way you can covert any zpool layout from one to another but it requires downtime for the applications.
Note:There is no way to reduce the zpool size as of now.If you added any LUN by mistake to the zpool ,then you need to destroy the zpool to take away the newly added LUN. So please be careful while adding LUN to ZFS storage pools.
Thank you for reading this article.Hope you liked it .