솔라리스 Solaris 에서 Oracle ASM 디스크 Disk 인식이 안 될 경우

haisins

2년 ago

솔라리스 Solaris 에서 Oracle ASM 디스크 Disk 인식이 안 될 경우

실린더 번호 나 할당 영역을 체크해 봐야 합니다.

ASM 디스크 증설인 경우 prtvtoc 명령어로 기존에 할당된 asm 디스크 를 확인해서 동일하게 파티션 + 실린더가 구성 되어 받은 건지 확인 합니다.

신규 설치하여 처음 받은 ASM 디스크가 인식이 안되는 경우라면 512 바이트 이상 할당 받은 영역 과 실린더 3번 인지 확인 해야 합니다 .

——————————————————————————————————————————————————————————————————-

Partition # 2 is a standard partition and it SHOULD NOT be changed as it reflects the total capacity of the disk.

All other partitions are available for the configuration.

All this information for the partitions and other data is contained in a special structure stored within the first 512 bytes of the disk, on the first cylinders.

This structure is called VTOC (Volume Table of Contents). Actions like re-writting the VTOC will cause loosing the disk.

The output displayed above is a typical configuration, where slice 0 was formatted using the entire disk.

When using ASM and during the disk discovery, the previous configuration is the cause for missing disks or disks not been discovered.

The reason seems to be that having partitions that start at cylinder 0, the complete partition is interpreted as the VTOC area which is not used as a regular partition of the disk.

Grid Infra – ASM 11.2.0.3 bug on Solaris – slice 2 CANDIDATE

We recently ran into a bug (at least that how I look at it) that is introduced with ASM 11.2.0.3. As of this version,

ASM shows slice 2 (the complete disk, starting a cylinder 0 thus inlcuding the VTOC) as an CANDIDATE disk in ASM.

In previous versions of ASM (Grid Infra) only slices that started from cylinder 3 (or higher) where shown as CANDIDATE disks in ASM,

so you couldn’t accidently select slice 2 to be used for a diskgroup.

In ASM 11.2.0.3 it is possible to add the slice 2 (for example *c0t0d0s2) to an ASM diskgroup while another slice of the same disk,

starting at cylinder 3, is already part of the same or even another diskgroup. You can understand what a mess this gives!
Example partition table of solaris disk, where until 11.2.0.3 you would only see partition (slice) 0,

starting at cylinder 3 (see MOS note: ASM Does Not Discover Disk on Solaris [ID 368840.1]).

Current partition table (original):
Total disk cylinders available: 13651 + 2 (reserved cylinders)

Part      Tag    Flag     Cylinders         Size            Blocks
  0       root    wm       3 - 13650       49.98GB    (13648/0/0) 104816640
  1 unassigned    wu       0                0         (0/0/0)             0
  2     backup    wu       0 - 13650       49.99GB    (13651/0/0) 104839680
  3 unassigned    wm       0                0         (0/0/0)             0
  4 unassigned    wm       0                0         (0/0/0)             0
  5 unassigned    wm       0                0         (0/0/0)             0
  6 unassigned    wm       0                0         (0/0/0)             0
  7 unassigned    wm       0                0         (0/0/0)             0

Current partition table (original):

Total disk cylinders available: 13651 + 2 (reserved cylinders)

Part Tag Flag Cylinders Size Blocks

0 root wm 3 - 13650 49.98GB (13648/0/0) 104816640

1 unassigned wu 0 0 (0/0/0) 0

2 backup wu 0 - 13650 49.99GB (13651/0/0) 104839680

3 unassigned wm 0 0 (0/0/0) 0

4 unassigned wm 0 0 (0/0/0) 0

5 unassigned wm 0 0 (0/0/0) 0

6 unassigned wm 0 0 (0/0/0) 0

7 unassigned wm 0 0 (0/0/0) 0

ASM (11.2.0.3) discovers the following disks now. Take a good look at the disk number of the MEMBER disks and CANDIDATE disks:

SQL&gt; select path,header_status from v$asm_disk order by 1,2;

PATH                                               HEADER_STATUS
-------------------------------------------------- ---------------
/dev/rdsk/c3t60060E8005492800000049280000319Dd0s0  MEMBER
/dev/rdsk/c3t60060E8005492800000049280000319Dd0s2  CANDIDATE
/dev/rdsk/c3t60060E8005492800000049280000319Ed0s0  MEMBER
/dev/rdsk/c3t60060E8005492800000049280000319Ed0s2  CANDIDATE
/dev/rdsk/c3t60060E8005492800000049280000319Fd0s0  MEMBER
/dev/rdsk/c3t60060E8005492800000049280000319Fd0s2  CANDIDATE
/dev/rdsk/c3t60060E800549280000004928000031A0d0s0  MEMBER
/dev/rdsk/c3t60060E800549280000004928000031A0d0s2  CANDIDATE
/dev/rdsk/c3t60060E800549280000004928000031A6d0s0  MEMBER
/dev/rdsk/c3t60060E800549280000004928000031A6d0s2  CANDIDATE
/dev/rdsk/c3t60060E800549280000004928000031A7d0s0  MEMBER
/dev/rdsk/c3t60060E800549280000004928000031A7d0s2  CANDIDATE
/dev/rdsk/c3t60060E800549280000004928000031A8d0s0  MEMBER
/dev/rdsk/c3t60060E800549280000004928000031A8d0s2  CANDIDATE
/dev/rdsk/c3t60060E800549280000004928000031A9d0s0  MEMBER
/dev/rdsk/c3t60060E800549280000004928000031A9d0s2  CANDIDATE
/dev/rdsk/c3t60060E800549280000004928000031CBd0s0  MEMBER
/dev/rdsk/c3t60060E800549280000004928000031CBd0s2  CANDIDATE

SQL> select path,header_status from v$asm_disk order by 1,2;

PATH HEADER_STATUS

-------------------------------------------------- ---------------

/dev/rdsk/c3t60060E8005492800000049280000319Dd0s0 MEMBER

/dev/rdsk/c3t60060E8005492800000049280000319Dd0s2 CANDIDATE

/dev/rdsk/c3t60060E8005492800000049280000319Ed0s0 MEMBER

/dev/rdsk/c3t60060E8005492800000049280000319Ed0s2 CANDIDATE

/dev/rdsk/c3t60060E8005492800000049280000319Fd0s0 MEMBER

/dev/rdsk/c3t60060E8005492800000049280000319Fd0s2 CANDIDATE

/dev/rdsk/c3t60060E800549280000004928000031A0d0s0 MEMBER

/dev/rdsk/c3t60060E800549280000004928000031A0d0s2 CANDIDATE

/dev/rdsk/c3t60060E800549280000004928000031A6d0s0 MEMBER

/dev/rdsk/c3t60060E800549280000004928000031A6d0s2 CANDIDATE

/dev/rdsk/c3t60060E800549280000004928000031A7d0s0 MEMBER

/dev/rdsk/c3t60060E800549280000004928000031A7d0s2 CANDIDATE

/dev/rdsk/c3t60060E800549280000004928000031A8d0s0 MEMBER

/dev/rdsk/c3t60060E800549280000004928000031A8d0s2 CANDIDATE

/dev/rdsk/c3t60060E800549280000004928000031A9d0s0 MEMBER

/dev/rdsk/c3t60060E800549280000004928000031A9d0s2 CANDIDATE

/dev/rdsk/c3t60060E800549280000004928000031CBd0s0 MEMBER

/dev/rdsk/c3t60060E800549280000004928000031CBd0s2 CANDIDATE

I have created an SR for this problem, but until then you’ll have the following options:

Set the ASM_DISKSTRING parameter to /dev/rdsk/*s0 (or whatever slice you normally use)
Make sure the user grid does not have any privileges on the /dev/rdsk/*s2 devices.

Update 26-02-2013:
Oracle Support finally came back with an answer and a patch.

The most important thing is that they agree that this is actually a bug and created a bug (bug nr. 14577881) for it.

Support tells me that the problem is “really” fixed in Oracle 12.2 (when even 12.1 is not even released yet ;-)) but they have created a backport request for it.

I just got back that the backport is available as a one-off patch downloadable as patch 14577881 for 11.2.0.3.

오라클 MOS 에 있는 솔라리스 환경에서 ASM 디스크로 인식 시킬 경우 …

실린더 1 과 2번 은 피해야 합니다.

Modifying the partitions, starting few cylinders ahead (1 or 2 to avoid wasting space), will help ASM to discover the disk.

The following output displays the disk originally presented, but now with partition 0 starting at cylinder 3:

Part Tag Flag Cylinders Size Blocks 0 unassigned wm 3 - 51197 100.00GB (51198/0/0) 209707008 1 unassigned wm 0 0 (0/0/0) 0 2 backup wu 0 - 51197 100.00GB (51198/0/0) 209707008 3 unassigned wm 0 0 (0/0/0) 0 4 unassigned wm 0 0 (0/0/0) 0 5 unassigned wm 0 0 (0/0/0) 0 6 unassigned wm 0 0 (0/0/0) 0 7 unassigned wm 0 0 (0/0/0) 0

———————————————————————————————————————————————————————————————————————————————

ASM Does Not Discover Disk on Solaris (문서 ID 368840.1)

APPLIES TO:

Oracle Database – Enterprise Edition – Version 10.1.0.2 to 12.1.0.2 [Release 10.1 to 12.1]
Oracle Database Cloud Schema Service – Version N/A and later
Gen 1 Exadata Cloud at Customer (Oracle Exadata Database Cloud Machine) – Version N/A and later
Oracle Database Exadata Express Cloud Service – Version N/A and later
Oracle Cloud Infrastructure – Database Service – Version N/A and later
Oracle Solaris on SPARC (64-bit)
Solaris Operating System (SPARC 64-bit)
<br* *** <br=””>

SYMPTOMS

Using ASM on any  SOLARIS platform and not using a volume manager to manage the disks. Under this circumstances, the disks are identified under path /dev/rdsk.

The problem is the disks are not discovered when the instance is started.

Basic points that need to be reviewed:

* permissions of the device are set for the oracle owner, group and mode 0660.
* Parameter asm_diskstring is configured with a value that resolves the disks to be used.

Queries over v$asm_disk does not return rows or does not return all the expected disks.

CHANGES

CAUSE

In Solaris the disks have a specific number of partitions (7) that are configured in order to manipulate the disk.   format is the command used to review/configure the partitions.

Example:

Log in as root:
# format
Searching for disks…done

AVAILABLE DISK SELECTIONS:
0. c1t0d0 <SUN72G cyl 14087 alt 2 hd 24 sec 424>
/pci@1c,600000/scsi@2/sd@0,0
1. c1t1d0 <SUN72G cyl 14087 alt 2 hd 24 sec 424>
/pci@1c,600000/scsi@2/sd@1,0
2. c3t42d10 <SUN-StorEdge3510-411I cyl 51198 alt 2 hd 64 sec 64>
/pci@1d,700000/SUNW,qlc@1/fp@0,0/ssd@w226000c0ffa8796e,a
3. c3t42d11 <SUN-StorEdge3510-411I cyl 51198 alt 2 hd 64 sec 64>
/pci@1d,700000/SUNW,qlc@1/fp@0,0/ssd@w226000c0ffa8796e,b
4. c3t42d12 <SUN-StorEdge3510-411I cyl 51198 alt 2 hd 64 sec 64>
/pci@1d,700000/SUNW,qlc@1/fp@0,0/ssd@w226000c0ffa8796e,c
5. c3t42d13 <SUN-StorEdge3510-411I cyl 51198 alt 2 hd 64 sec 64>
/pci@1d,700000/SUNW,qlc@1/fp@0,0/ssd@w226000c0ffa8796e,d
6. c3t42d14 <SUN-StorEdge3510-411I cyl 51198 alt 2 hd 64 sec 64>
/pci@1d,700000/SUNW,qlc@1/fp@0,0/ssd@w226000c0ffa8796e,e
7. c4t42d10 <SUN-StorEdge3510-411I cyl 51198 alt 2 hd 64 sec 64>
/pci@1d,700000/SUNW,qlc@1,1/fp@0,0/ssd@w226000c0ffa8796e,a
8. c4t42d11 <SUN-StorEdge3510-411I cyl 51198 alt 2 hd 64 sec 64>
/pci@1d,700000/SUNW,qlc@1,1/fp@0,0/ssd@w226000c0ffa8796e,b
9. c4t42d12 <SUN-StorEdge3510-411I cyl 51198 alt 2 hd 64 sec 64>
/pci@1d,700000/SUNW,qlc@1,1/fp@0,0/ssd@w226000c0ffa8796e,c
10. c4t42d13 <SUN-StorEdge3510-411I cyl 51198 alt 2 hd 64 sec 64>
/pci@1d,700000/SUNW,qlc@1,1/fp@0,0/ssd@w226000c0ffa8796e,d
11. c4t42d14 <SUN-StorEdge3510-411I cyl 51198 alt 2 hd 64 sec 64>
/pci@1d,700000/SUNW,qlc@1,1/fp@0,0/ssd@w226000c0ffa8796e,e
Specify disk (enter its number): 2
selecting c3t42d10
[disk formatted]

You pick the disk you want by typing the number and now you will get this:

FORMAT MENU:
disk – select a disk
type – select (define) a disk type
partition – select (define) a partition table
current – describe the current disk
format – format and analyze the disk
fdisk – run the fdisk program
repair – repair a defective sector
show – translate a disk address
label – write label to the disk
analyze – surface analysis
defect – defect list management
backup – search for backup labels
verify – read and display labels
save – save new disk/partition definitions
volname – set 8-character volume name
! – execute , then return

then you type partition
then you type print

partition> p
Current partition table (original):
Total disk cylinders available: 51198 + 2 (reserved cylinders)

Part Tag Flag Cylinders Size Blocks
0 unassigned wm 0 – 51197 100.00GB (51198/0/0) 209707008
1 unassigned wm 0 0 (0/0/0) 0
2 backup wu 0 – 51197 100.00GB (51198/0/0) 209707008
3 unassigned wm 0 0 (0/0/0) 0
4 unassigned wm 0 0 (0/0/0) 0
5 unassigned wm 0 0 (0/0/0) 0
6 unassigned wm 0 0 (0/0/0) 0
7 unassigned wm 0 0 (0/0/0) 0

Partition # 2 is a standard partition and it SHOULD NOT be changed as it reflects the total capacity of the disk.

All other partitions are available for the configuration.

All this information for the partitions and other data is contained in a special structure stored within the first 512 bytes of the disk, on the first cylinders.

This structure is called VTOC  (Volume Table of Contents). Actions like re-writting the VTOC will cause loosing the disk.

The output displayed above is a typical configuration, where slice 0 was formatted using the entire disk.

When using ASM and during the disk discovery, the previous configuration is the cause for missing disks or disks not been discovered.

The reason seems to be that having partitions that start at cylinder 0, the complete partition is interpreted as the VTOC area which is not used as a regular partition of the disk.

SOLUTION

Modifying the partitions, starting few cylinders ahead (1 or 2 to avoid wasting space), will help ASM to discover the disk.

The following output displays the disk originally presented, but now with partition 0 starting at cylinder 3:

Part Tag Flag Cylinders Size Blocks 0 unassigned wm 3 - 51197 100.00GB (51198/0/0) 209707008 1 unassigned wm 0 0 (0/0/0) 0 2 backup wu 0 - 51197 100.00GB (51198/0/0) 209707008 3 unassigned wm 0 0 (0/0/0) 0 4 unassigned wm 0 0 (0/0/0) 0 5 unassigned wm 0 0 (0/0/0) 0 6 unassigned wm 0 0 (0/0/0) 0 7 unassigned wm 0 0 (0/0/0) 0

The disk configured was c3t42d10 and the partition selected was 0 (S0), which is reflected as /dev/rdsk/c3t42d10S0.

Having a valid value for parameter asm_disktring that resolves /dev/rdsk/c3t42d10S0 and running a query on v$asm_disk will now discover the disk and it will be available for ASM.

</br* ***>