HDS及服务器维护大全(9个ppt)6.ppt

1、如何架构高端存储系统,程志勇,主 题,Back End DKA , Raid Group and LDEV Front End CHA Work with host,IO的重要性,这是一个典型的OLTP的事务处理 : Email, DSS & Rich Media 会占用更多的IO资源 OLAP, CRM & ERP 会占用更多的CPU资源 减少IO的时间，提高IO的性能，将较大地提高应用的总体性能 注：这里并没有考虑网络传输占用的时间,HDS高端存储设备的结构,Cache pair,Cache pair(s),CHA portwork with host,“前端”,“后端”,DKA port

2、work with RAID,data is placed into cache & picked up by the other end,USP体系结构,Back End DKA Raid Group and LDEVRAID1 and RAID5,USP后端结构,B4-1,B4-18,B4-4,B4-6,B4-3,B4-5,B4-2,B4-8,B4-7,B4-10,B4-9,B4-15,B4-16,B4-17,B4-11,B4-13,B4-14,B4-12,18 x B4, each B4 = 4 x 16 RAID or 64 HDD,A pair of DKA first goes t

3、o 2 vertical B4, then horizontally to 2 additional B4 (except basic DKA),DKA Option,9900V 4xMP and 4x1G port,Cluster 1,Cluster 2,MP-PK,(SH281-B),MP-PK,(SH281-B),MP-PK,(SH281-B),MP-PK,(SH281-B),MP-PK (1MP),(SH281-B),MP-PK (1MP),(SH281-B),MP-PK (1MP),(SH281-B),MP-PK (1MP),(SH281-B),WP471-B,DKC-F460I-2

4、00,MP,-,PK,(SH350,-,A),MP,-,PK,(SH350,-,A),MP,-,PK,(SH350,-,A),MP,-,PK,(SH350,-,A),Cluster 1,Cluster 2,WP520,-,B,DKC,-,F510I,-,400,MP,-,PK (2MP),(SH350,-,A),MP,-,PK (2MP),(SH350,-,A),MP,-,PK (2MP),(SH350,-,A),MP,-,PK (2MP),(SH350,-,A),USP 8xMP and 8x2G port,DKA USP FCAL,HDU0,HDU1,HDU2,HDU3,HDU4,HDU5

5、,HDU6,HDU7,RAID1 2D+2D,RAID5 3D+1P,D,One random or sequential write from the server, store in cache,Two asynchronous writes to the mirrored disks.,RAID-1 2D+2D 写惩罚,Three random or sequential writes to cache require six IO to disk.,D,D,3,x,2,6,=,One random write from the server.,Read old data & old p

6、arity from disk. Two read IO,Write new data & new parity. Two write IO to disk,XOR new data, old data, old parity to create new parity,New D,New P,Old P,Old D,写没有命中的时候,Each random write means 4 IO. Three Insert writes to cache require 12 IO to/from disk.,RAID-5 3D+1P 随机写的写惩罚,3,x,4,12,=,XOR,XOR,写命中的时

7、候,RAID-5 3D+1P 随机写的写惩罚,One random write from the server. Old data already in cache,Read old parity from disk. One read IO,Write new data & new parity. Two write IO to disk,XOR new data, old data, old parity to create new parity,New D,New P,Old P,Old D,3,x,3,9,=,XOR,XOR,Each random write means 3 IO.

8、Three Insert writes to cache require 9 IO to/from disk.,D,P,D,D,Three sequential writes from the server.,XOR sequential writes to create Parity.,Four Writes to Disk.,Three sequential writes to cache require four IO to disk.,RAID-5 3D+1P 顺序写的写惩罚,3,+,1,4,=,XOR,XOR,RAID 类型的性能对比,RAID6 needs 6 IO instead

9、 of 4 in random write,4xHDD,8xHDD,8xHDD,8xHDD,LUN的分配要点,尽量不要把一个应用全压到一个RAID GROUP上，尽可能地分布在不同的RAID GROUP上 尽量避免顺序IO的应用与随机IO的应用分布在同一个RAID GROUP或LDEV上 如果联机事务处理的应用与批处理的应用共享RAID GROUP的时候，尽量避免它们在同一时间段工作 避免Shadow Image的原卷与目标卷在同一个RAID GROUP上,条带化的作用,1,2,3,P,4,5,P,6,LDEV1,LDEV2,LDEV3,RAID5 3D+1P,假设一个 3D+1P RAID5

10、 建立了 3 x 100G OPEN-V LDEV Stripe unit (chunk) is 64x8 = 512K, Stripe width is 3, Stripe size is 1536K 加入有一个3MB的文件，它会分成6部分 当读这个文件的时候，意味着有 “6”个 I/O 发送到 3硬盘上, 是并行的 这样做，肯定这个3MB的比文件放到同一个硬盘上，读速度快得多,1,1,2,2,3,3,4,4,RAID1 2D+2D,条带化就是尽可能地将数据分布到各个硬盘上 (1) 使用更多的硬盘放置数据 (2) 并行读写，提高性能,RAID5 的RAID GROUP间的条带化,Parity

11、 Group 1 (7D+1P),Parity Group 2 (7D+1P),LDEV#A,A-1,A-2,A-3,A-4,B-1,B-2,B-3,A-2,A-4,B-1,B-3,B-4,A-1,A-3,B-4,B-2,Parity Group 1+2 concatenation (7D+1P x 2),LDEV#B,2nd stripe (3.5M) goes to another PG),Front End CHA,USP前端板的位置,USP FC CHA,Always 1MP for 2 ports,8MP,4MP,Work with host,LUN是什么？在RAID GROUP的什

12、么位置,LDEV0=3GB,LDEV1,LDEV2,LDEV71,Volume = hdisk = PV = LUN = LDEV,LDEV = Logical DEVice,Logical Volume Manager,应用将数据存放在LV中，相关的LV在一起组成VG，LVM会将LV对应到PV，也就是说对应到真实的磁盘空间。 PV = LUN = LDEV LVM 的LV采用 striping 使数据分布到更多的Raid Group上，是提高IO性能的一个重要的方法 Some AIX Administrator still trying to allocate LV to outer edg

13、e, outer middle, center, inner middle, and inner edge of PV. Meaningless if the PV is RAID LDEV.,LVM 采用多大的Stripe Size才是最优化？,RAID-1 Stripe Size,No Parity Generation Two readable copies of data,stripe unit = 48K * 8 track = 384K stripe size = 384K * 2 Open-V: stripe unit = 64K * 8 track = 512K, stripe

14、 size = 512K * 2,Recall: OPEN-V RAID5 3D+1P storage striping spread 3M data to 3 HDD on one RAID Applying OS striping Assuming OS stripe unit is also 512K PV is LDEV1, LDEV11, LDEV21 Reading this 3M file means “6” I/O to 6 HDD, in parallel UNIX offers striping by PP size and also 4K to 128K chunk (A

15、IX 5.3 has 1M and up). Make sure you stripe across PV on different RAID GROUP Try to match OS stripe size with HDS stripe size OS stripe chunk/unit shall not be smaller than application I/O,Striping from Host,1,4,LDEV1,LDEV2,LDEV3,5,2,3,6,LDEV11,LDEV12,LDEV13,LDEV21,LDEV22,LDEV23,When OS Stripe size

16、 = 32K,PV1,1,2,3,4,5,6,7,8,9,LV,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,1,3,5,7,9,11,13,15,17,19,21,23,25,27,29,31,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,33,33,Open-V stripe is always 512K With OS stripe size = 32K What happen to read/write 32K of #21?,512K#2,512K#3

17、,PV2,512K#2,512K#3,1,2,3,4,9,10,11,12,17,18,19,20,25,26,27,28,5,6,7,8,13,14,15,16,21,22,23,24,29,30,31,32,1,2,3,4,5,6,7,8,9,LV,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,33,PV1,PV2,512K#2,512K#3,512K#2,512K#3,Open-V stripe is always 512K With OS stripe size = 128K What h

18、appen to read/write 32K of #21?,When OS Stripe size = 128K,1,2,3,4,9,10,11,12,5,6,7,8,13,14,15,16,21,22,23,24,29,30,31,32,LV,33,17,18,19,20,25,26,27,28,PV1,PV2,512K#2,512K#3,512K#2,512K#3,Open-V stripe is always 512K With OS stripe size = 512K What happen to read/write 32K of #21?,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,