I am constantly in new customer environments, setting up NetApp, Oracle RAC & ASM, NetApp SnapDrive, SnapManager for Oracle, etc on Linux…
Opinionated Rant: When ever possible, PLEASE use Oracle with Direct NFS or normal NFS on 10gb ethernet. Stop using Fiberchannel or iscsi. It is such a giant pain in the ass to use ASM. Really. Do yourself a favor and go NFS instead of ASM. You’ll thank me
While I know what I am doing in regards to storage and Oracle, most customers do not have a single point man (or woman) who knows all of these technologies and is the administrator to do day to day operations of them all. Also, when I have tried to teach the teams to do this, or if the DBA is in charge of all of these steps in a small environment, they typically get very confused when thinking about the relationship of the /dev/sd* and the multipath /dev/mapper/360* entries.
Typically, if a customer wanted to add a new LUN to an Oracle database server, the steps would look like this:
- DBA asks SAN admin for a LUN
- SAN admin creates LUN, presents it to server
- SAN admin tells Linux/System Admin it’s presented
- Linux/System Admin rescans the SCSI bus to discover the lun
- Linux/System Admin does all the multipath mumbojumbo to create the /dev/mapper crap
- Linux/System Admin fdisks partition (aligned to SAN I hope)
- DBA uses /etc/init.d/oracleasm to tag the new lun to ASM
- Done… ugh. wtf thats too many handoffs
NetApp created snapdrive as a way that a sysadmin or DBA can interface with the SAN, without having direct access to SSH or SystemManager or Web console into the SAN. Role based access control can be used to limit what the admin can do.
With some clever scripting, you can do ALL the steps above with just one script.
When I do new SMO installs, I typically break the database into the following groups: UNDO REDO CTRL1 CTRL2 ARCH DATA TEMP FRA. This is the most logical way if you want to later clone or replicate you data in the most efficient way.
I like to have a logical naming method to luns and volumes
/vol/OraTABLESPACENAMEVolVolIdentified/OraTABLESPACENAMEVolNumber_LunUniqId.lun
Right now, this script requires the volumes to be precreated, but it would be pretty easy to add a command to create the volume.
It is up to the person running this to provide good tablespace naming and a unique identified for the lun or else itll crap out. This was a very very quick script I coded on a customer site while I had 3 people staring over my shoulder.
Note: This script should be used for partitions OVER 8gb. Anything under can have an issue with fdisk depending on your linux distribution at the point it tries to automatically align the partition to the 1M block boundary. With how a SAN tries to mount small luns to a box, it “sort of” changes the presentation style to an older LBA like the box would see on smaller bootable disks. You can, however, just change the fdisk code or put it in an if statement to not align the filesystem on ( ${3} -lt 8 ). I had to do this for a customer, and it’s not that complex.
This is an Example usage and output. You see the makemount.sh command.
Script Name: makemount.sh
Input 1: Tablespace Name
Input 2: Volume Identified Number
Input 3: Lun Size
Input 4: Lun Identified Number
Input 5: Controller Name
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 |
[root@oraclebox bin]# <strong>./makemount.sh OCR1 1 5 D netappc1</strong> SID = 5 SIZE = 1 LUN netappc1:/vol/OraOCR1Vol1/OraOCR11_D.lun ... created mapping new lun(s) ... done discovering new lun(s) ... done LUN to device file mappings: - netappc1:/vol/OraOCR1Vol1/OraOCR11_D.lun => /dev/sdal, /dev/sdam Getting lun info Getting Multipath information Partitioning Disk Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel Building a new DOS disklabel. Changes will remain in memory only, until you decide to write them. After that, of course, the previous content won't be recoverable. Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite) Command (m for help): Command action e extended p primary partition (1-4) Partition number (1-4): First cylinder (1-652, default 1): Last cylinder or +size or +sizeM or +sizeK (1-652, default 652): Using default value 652 Command (m for help): Expert command (m for help): Partition number (1-4): New beginning of data (63-10474379, default 63): Expert command (m for help): The partition table has been altered! Calling ioctl() to re-read partition table. WARNING: Re-reading the partition table failed with error 22: Invalid argument. The kernel still uses the old table. The new table will be used at the next reboot. Syncing disks. Disk /dev/mapper/360a980006473444b48346b796e6c5749: 5368 MB, 5368709120 bytes 255 heads, 63 sectors/track, 652 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System /dev/mapper/360a980006473444b48346b796e6c5749p1 1 652 5237158 83 Linux <em>Partition 1 has different physical/logical beginnings (non-Linux?): phys=(0, 1, 1) logical=(0, 1, 2)</em> kpartx on disk add map 360a980006473444b48346b796e6c5749p1 (253:40): 0 10474316 linear /dev/mapper/360a980006473444b48346b796e6c5749 64 CREATEDISK in /etc/init.d /etc/init.d/oracleasm createdisk OCR1_D /dev/mapper/360a980006473444b48346b796e6c5749p1 Marking disk "OCR1D" as an ASM disk: [ OK ] Cleaning up |
And the official makemount.sh script:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 |
#!/bin/bash ################################################ # makemount.sh v0.2 # -- Jarett "JK-47" Kulm ################################################ # This script is a helper utility to create luns # kpartx, multipath, fdisk, and add to oracleasm # # This utilizes SnapDrive for creation of luns # # ** NOTE: It is up to you to make sure your # ** lun and diskgroup names are UNIQUE!!!! # ** Not doing so will cause fdisk to wipe # ** out an existing partition!!!!!! # ** # ** I am not responsible for any damage # ** to any system or database! # ** # ** This script is for example use only! # ################################################ # Script Name: makemount.sh # Input 1: Tablespace Name # Input 2: Volume Identified Number # Input 3: Lun Size # Input 4: Lun Identified Number # Input 5: Controller Name # Usage Syntax: # ./makemount.sh OCR1 1 5 D netappc1 ################################################ export COUNT=0 echo "Creating lun: ${5}:/vol/Ora${1}Vol${2}/Ora${1}${2}_${4}.lun of size ${3}g" snapdrive storage create -lun ${5}:/vol/Ora${1}Vol${2}/Ora${1}${2}_${4}.lun -lunsize ${3}g echo "Getting lun info" ls -l `sanlun lun show | grep ${1}${2}_${4} | tail -1 | awk '{print $3}' ` | awk -F'/dev/' '{print $2'} > /tmp/mylun.$$ echo " Getting Multipath information" multipath -v2 -ll | sed -e 's/^ //g' | grep -v "\[size" | grep -v "round-robin" | tr '\n' '\t' | sed -e 's/360/\n360/g' | grep `cat /tmp/mylun.$$` | awk '{print $1}' > /tmp/my360dm.$$ echo "Partitioning Disk" # sfdisk no longer being used, but it is a viable alternative.. sometimes. I didnt like the output of it. # echo "2048,," | sfdisk -uC /dev/mapper/`cat /tmp/my360dm.$$` fdisk /dev/mapper/`cat /tmp/my360dm.$$` << EOF n p 1 1 x b 1 64 w EOF fdisk -l /dev/mapper/`cat /tmp/my360dm.$$` echo "Multipath setup - kpartx on disk" kpartx -av /dev/mapper/`cat /tmp/my360dm.$$` dmsetup table | grep `cat /tmp/my360dm.$$` | grep linear | awk '{print $1}' | sed -e 's/\://g' > /tmp/mypartx.$$ echo "About to Oracle ASM - CREATEDISK in /etc/init.d" sleep 3 echo "/etc/init.d/oracleasm createdisk ${1}${4}" /dev/mapper/`cat /tmp/mypartx.$$` sleep 3 /etc/init.d/oracleasm createdisk ${1}${4} /dev/mapper/`cat /tmp/mypartx.$$` /etc/init.d/oracleasm listdisks | grep ${1}${4} echo "Cleaning up" rm /tmp/mylun.$$ rm /tmp/my360dm.$$ rm /tmp/mypartx.$$ |
[asa]0071752625[/asa]
[asa]0470345233[/asa]
[asa]0132757273[/asa]
