[关闭]
@saltyang 2018-08-10T10:00:14.000000Z 字数 5148 阅读 895

Vaai testing

未分类

VAAI Testing and Trouble-Shooting

  1. VAAI Block - SCSI Command

    • Atomic Test & Set (0x89 COMPARE AND WRITE, T10 SCSI command)
      • Lock one disk sector and update with it atomically.
    • Extended Copy (0x83 XCOPY, 0x84 Copy Results)
    • Write Same (Zero) (0x93 WRITE_SAME)
      • Eager zeroed thick VMDK
    • UNMAP
      • Dead Space Reclamation

  2. VAAI Block - NAS Primitives (Not supported)

    • Full File Clone
    • Fast File Clone / Native Snapshot Support
    • Extended Statistics
    • Reserve Space

  3. Check VAAI support

    • Enable SSH for ESXi host

      主机 > 配置 > 安全配置文件 > 服务 > 属性 > SSH 启动

    • Use esxcli to check VAAI support status

      > esxcli storage core device list -d eui.6665653462653938

      eui.6665653462653938
   Display Name: Potevio iSCSI Disk (eui.6665653462653938)
   Has Settable Display Name: true
   Size: 5120
   Device Type: Direct-Access
   Multipath Plugin: NMP
   Devfs Path: /vmfs/devices/disks/eui.6665653462653938
   Vendor: Potevio
   Model: PtBus
   Revision: 0500
   SCSI Level: 6
   Is Pseudo: false
   Status: degraded
   Is RDM Capable: true
   Is Local: false
   Is Removable: false
   Is SSD: false
   Is Offline: false
   Is Perennially Reserved: false
   Queue Full Sample Size: 0
   Queue Full Threshold: 0
   Thin Provisioning Status: yes
   Attached Filters:
   VAAI Status: supported
   Other UIDs: vml.01000000006665653462653938507442757320
   Is Local SAS Device: false
   Is Boot USB Device: false
   No of outstanding IOs with competing worlds: 32

      > esxcli storage core device vaai status get -d eui.6665653462653938

      eui.6665653462653938
   VAAI Plugin Name:
   ATS Status: supported
   Clone Status: supported
   Zero Status: supported
   Delete Status: supported

  4. Monitor VAAI related operations

    • Operation monitor

      > esxtop -d 3

      # 1. Select `u` for device view
# 2. Then `f` change which fields to show
# 3. Choose `o` and `p` for VAAI statistics

11:59:16pm up 66 days 23:11, 544 worlds, 9 VMs, 29 vCPUs; CPU load average: 0.17, 0.17, 0.17
DEVICE                               CLONE_RD CLONE_WR  CLONE_F MBC_RD/s MBC_WR/s      ATS ATSF     ZERO   ZERO_F MBZERO/s   DELETE
eui.6665653462653938                      513      513        0     0.00     0.00     2392    3     2174        1     0.00        0
mpx.vmhba1:C0:T0:L0                         0        0        0     0.00     0.00        0    0        0        0     0.00        0
naa.600508e000000000346e84af20e99600        0        0        0     0.00     0.00        0    0        0    25293     0.00        0
naa.600508e0000000003b4f94b413c8a609        0        0        0     0.00     0.00        0    0        0    14761     0.00        0
    • ATS/ATS_F
      Atomic Test & Set operations and failed ones
    • ZERO/ZERO_F
      Write Same (Zero) operations and failed ones
    • CLONE_RD/CLONE_WR/CLONE_F
      Extended Copy operations and failed ones
    • DELETE/DELETE_F
      UnMap operations and failed ones

  5. Testing Methodology / Scenarios

    • ATS
      • Frequent operations, no special scenarios
      • VM provision / Delete / Clone
    • Extended Copy
      • VM Clone in same datastore
    • Write Same
      • Provision eager zeroed think VMDK

    • UNMAP

      • VM Migration / Delete

      • Before migration
        ESXi > vmkfstools -Ph -v 1 /vmfs/volumes/vaai/

        VMFS-5.60 file system spanning 1 partitions.
File system label (if any): vaai
Mode: public ATS-only
Capacity 59.8 GB, 24.7 GB available, file block size 1 MB, max file size 64 TB
Volume Creation Time: Fri May  4 08:53:15 2018
Files (max/free): 130000/129971
Ptr Blocks (max/free): 64512/64462
Sub Blocks (max/free): 32000/31995
Secondary Ptr Blocks (max/free): 256/256
File Blocks (overcommit/used/overcommit %): 0/35918/0
Ptr Blocks  (overcommit/used/overcommit %): 0/50/0
Sub Blocks  (overcommit/used/overcommit %): 0/5/0
Volume Metadata size: 803962880
UUID: 5aec1f7b-7dfd8c80-1531-842b2b6fe984
Partitions spanned (on "lvm"):
        eui.6163386239626431:1
Is Native Snapshot Capable: YES
OBJLIB-LIB: ObjLib cleanup done.

        PtBus > ceph df # Check RBD pool space usage

      • After migration
        • ESXi > esxcli storage vmfs unmap -l vaai
        • ESXi > vmkfstools -Ph -v 1 /vmfs/volumes/vaai/
        • PtBus > ceph df # Check RBD pool space reclaim

  6. Issues

    • ESXi挂载iSCSI Target LUN，但LUN中已经有分区，需要删除分区

      • #> ssh root@192.168.1.245

      • > esxcfg-scsidevs -l

        eui.6665653462653938
Device Type: Direct-Access
Size: 5120 MB
Display Name: Potevio iSCSI Disk (eui.6665653462653938)
Multipath Plugin: NMP
Console Device: /vmfs/devices/disks/eui.6665653462653938
Devfs Path: /vmfs/devices/disks/eui.6665653462653938
Vendor: Potevio   Model: PtBus             Revis: 0500
SCSI Level: 6  Is Pseudo: false Status: degraded
Is RDM Capable: true  Is Removable: false
Is Local: false Is SSD: false
Other Names:
   vml.01000000006665653462653938507442757320
VAAI Status: supported
...

      • > partedUtil get /dev/disks/eui.6665653462653938

        652 255 63 10485760
1 2048 10474379 0 0
      • #> partedUtil delete /dev/disks/eui.6665653462653938 1

    • Kill VM When iSCSI disk is gone

      • > esxcli vm process list

        vaai-test
World ID: 15094214
Process ID: 0
VMX Cartel ID: 15094208
UUID: 42 2a f4 e7 26 b3 17 15-4c 94 ba df df 28 ae 82
Display Name: vaai-test
Config File: /vmfs/volumes/5ae3d691-27e4f6f0-a321-842b2b6fe984/vaai-test/vaai-test.vmx
      • #> esxcli vm process kill -t [soft|hard|force] -w [World ID]
      • #> esxcli vm process kill -t hard -w 15094214

  7. Referrence

    • https://blogs.vmware.com/vsphere/2012/04/vaai-thin-provisioning-block-reclaimunmap-in-action.html

    • Reclaiming Unused Storage Space

      When VMFS datastores reside on thin-provisioned LUNs, you can use ESXCLI commands to reclaim the unused logical blocks of a thin-provisioned LUN formatted with VMFS.
When you run the commands, you must specify the volume label (--volume-label) or the volume id (--volume-uuid) but you cannot specify both.
In each iteration, the command issues unmap commands to the number of file system blocks that are specified by the optional reclaim-unit argument, which defaults to 200. For newly created VMFS-5 file systems, the filesystem block size is always 1 MB. For VMFS-3 filesystems or VMFS-5 filesystems that were upgraded from VMFS-3, the filesystem block size could be one of 1, 2, 4, 8 MB.
The following examples illustrate how to use the command.
# esxcli storage vmfs unmap --volume-label datastore1 --reclaim-unit 100
# esxcli storage vmfs unmap -l datastore1 -n 100
# esxcli storage vmfs unmap --volume-uuid 515615fb-1e65c01d-b40f-001d096dbf97 --reclaim-unit 500
# esxcli storage vmfs unmap -u 515615fb-1e65c01d-b40f-001d096dbf97 -n 500

# esxcli storage vmfs unmap -l datastore1
# esxcli storage vmfs unmap -u 515615fb-1e65c01d-b40f-001d096dbf97
添加新批注
在作者公开此批注前，只有你和作者可见。
回复批注