Monday, December 31, 2012

Oracle VM Server for SPARC - Cheat sheet / Quick steps to configure VM (LDom)

~~~~~~~~~~~~~~Installation~~~~~~~~~~~~~~~

Download latest binary from Oracle website & upgrade the system firmware.
http://www.oracle.com/technetwork/server-storage/vm/downloads/index.html 

Note: On Oracle Solaris 10 you need to manually install Ldom packages. In Solaris 11 its part of default installation. LDom is supported in Oracle's SPARC T-Series Servers only  
 
unzip LDom_3.0.zip
./Install/install-ldm

You can choose no to ldom configuration assistant if you want to manually configure. Below is just a quick reference of commands to configure ldom(Its recommend you go through the Oracle VM for SPARC (Ldom) administration guide for any details)

 

~~~~~~~~~~~Control Domain Initial Configuration~~~~~~~~~~

1. After installation ldmd service will be enabled by default
svcs svc:/ldoms/ldmd:default

To enable :  
svcadm enable svc:/ldoms/ldmd:default
 

2. Adding virtual disk service
ldm add-vds primary-vds0 primary

3. Adding console service
ldm add-vcc port-range=5000-5100 primary-vcc0 primary

4. Adding Virtual network switch

ldm add-vsw net-dev=e1000g0 primary-vsw0 primary

5.
ldm list-services primary
 

6. Starting delayed configuration, requires reboot
ldm start-reconf primary

7. Setting 2 cores of CPU & 8GB Memory for control domain
ldm set-vcpu -c 2 primary
ldm set-memory 8G primary
ldm add-config initial
ldm list-config


8. IO domain configuration. 
Latest LDom supports PCIE bus/Leaf level devices can be directly allocated to guest domains (Ref Admin Guide for any details)

a. List the devices/Bus which can be added to guest domains. 
ldm list-io
--Remove IO device you want to allocate to guest domain 
b. ldm remove-io <Bus/device> primary    
   ldm add-io <Bus/device> primary

Note: If you remove a Bus, internal disks and onboard network ports associated to that PCI controller/bus will be removed and once you add that bus to guest ldom it will appear in the guest ldom. If you are creating a IO domain no need of virtualising the disks and network (in step 2 & 4). 
   
shutdown -y -g0 -i6

~~~~~~~~~~~~~~~~Configuring Guest LDom~~~~~~~~~~~~~~~~~~

1. Create a logical domain.
ldm add-domain ldom1

2. Add CPUs to the guest domain.

ldm set-vcpu -c 4 ldom1

3. Add memory to the guest domain.
 

ldm set-memory 8G ldom1

4. Add IO device/bus directly to guest domain.

You can add io which is removed from primary in step 8 above. so that on-board disk & network which are associated with that bus or leaf level devices will be directly available on the guest LDom. You may not need to virtualise as is below step 5 & 6. 

ldm list-io
ldm add-io <bus/device> ldom1     

5. Add a virtual network device to the guest domain.

ldm add-vnet vnet1 primary-vsw0 ldom1

6, Add virtual disk to LDom (with any of the below three option) 

a. Adding physical disk

ldm add-vdsdev /dev/dsk/c0t0d0s2 vol1@primary-vds0
ldm add-vdisk vdisk0 vol1@primary-vds0 ldom1

.b Create (mkfile)and add as another disk (install_disk) on which to install the Solaris OS.
primary# mkfile 30g /LDomFS/Ldisk1 

primary# ldm add-vdsdev /LDomFS/Ldisk1 install_vol@primary-vds0
primary# ldm add-vdisk vdisk_install install_vol@primary-vds0 ldom1 

.c Create disk with zfs create -v and add as disk (install_disk) on which to install the Solaris OS.
(If you are using zfs file system you can also create disk as below and use zfs snapshot/clone feature if you want to replicate LDoms later) 
 primary# zfs create -V 30g rpool/Ldisk1 
 primary# ldm add-vdsdev /rpool/Ldisk1 install_vol@primary-vds0
 primary# ldm add-vdisk vdisk_install install_vol@primary-vds0 ldom1
 

7,
a. Adding OS ISO Image and Installing

ldm add-vdsdev /solaris/sol-10-u10-ga2-sparc.iso iso_vol@primary-vds0
ldm add-vdisk vdisk_iso iso_vol@primary-vds0 ldom1

b. Installing from Physical DVD drive.

svcadm disable volfs
ldm add-vdsdev /dev/dsk/c0t0d0s2 dvd_vol@primary-vds0
ldm add-vdisk vdisk_cd_media dvd_vol@primary-vds0 ldom1


8. set auto-boot variable false to go to 'ok' prompt

ldm set-variable auto-boot\?=false ldom1
or
ldm set-variable auto-boot\?=true ldom1

ldm set-var boot-device=vdisk ldom1 


9. Enable console service
svcadm enable vntsd
svcs -l vntsd

 
Finally:
ldm bind ldom1


ldm list-domain

ldm start ldom1 or ldm stop ldom1


10. Save the config
ldm add-config complete  


~~~~~~Installing OS~~~~~

ldm list

telnet localhost 5000
{0} ok show-disks
{0} ok devalias
{0} ok boot vdisk_iso


Proceed with regular installation. 

Ctr ]
>q -> to come out from telnet session 



Sunday, December 23, 2012

Handy tool to find out IOPS in Solaris.


-----------------------------------------------------------------------------------------------------------------------------
Keywords : Solaris, IOPS, Calculate IOPS, Collect IOPS
-----------------------------------------------------------------------------------------------------------------------------
When there are multiple volumes involved in disk IO operation, there is no simple way to calculate Total IOPS in Solaris command line options.

This simple script helps to calculate Total, Avg, Max IOPS & %Read/Write  in run time or offline from iostat output file quickly. You can simply put this perl script in /usr/bin directory with execute permission.

Usage:  [You must use " iostat -xncz" options with this script] 

1. To find IOPS at Run Time : $ iostat -xncz 5 |iopscal

Sample output:
# iostat -xncz 5 |iopscal   or   # iostat -xncz 5 10 |iopscal
        |--------------TOTAL IOPS--------------------| |------KB_Read------| |-----KB_Write------| |------Device Status------|
Count        r/s         w/s        io/s   io/s(K)      kr/s                kw/s                  AvgSrvTm %Wait %DiskBusy
-------------------------------------------------------------------------------------------------------------------------------------------------------
1       3.0 (  3%)   112.9 ( 97%)    116 (  0.12K)      48 (   0.05MB/s)    3686 (   3.6MB/s)      0.6ms    0.0%     5%
2       1.0 ( 50%)     1.0 ( 50%)      2 (  0.00K)      16 (   0.02MB/s)       0 (   0.0MB/s)      0.2ms    0.0%     0%
3       6.0 ( 67%)     3.0 ( 33%)      9 (  0.01K)      96 (   0.09MB/s)      32 (   0.0MB/s)      0.1ms    0.0%     0%
4       2.0 ( 50%)     2.0 ( 50%)      4 (  0.00K)      32 (   0.03MB/s)       8 (   0.0MB/s)      0.2ms    0.0%     0%
5     156.0 ( 99%)     1.0 (  1%)    157 (  0.16K)    2496 (   2.44MB/s)       0 (   0.0MB/s)      0.9ms    0.0%     1%
6       7.0 (  7%)    95.0 ( 93%)    102 (  0.10K)     112 (   0.11MB/s)    5133 (   5.0MB/s)      0.7ms    0.0%    13%
7       2.0 ( 22%)     7.0 ( 78%)      9 (  0.01K)      32 (   0.03MB/s)       4 (   0.0MB/s)      0.2ms    0.0%     0%
8       1.0 ( 50%)     1.0 ( 50%)      2 (  0.00K)      16 (   0.02MB/s)       0 (   0.0MB/s)      0.2ms    0.0%     0%
9      15.0 ( 50%)    15.0 ( 50%)     30 (  0.03K)     238 (   0.23MB/s)     669 (   0.7MB/s)      0.2ms    0.0%     0%
10      1.0 ( 17%)     5.0 ( 83%)      6 (  0.01K)      16 (   0.02MB/s)      20 (   0.0MB/s)      0.2ms    0.0%     0%
11      3.0 (  2%)   136.0 ( 98%)    139 (  0.14K)      48 (   0.05MB/s)    5709 (   5.6MB/s)      0.8ms    0.0%     8%
12      3.0 ( 43%)     4.0 ( 57%)      7 (  0.01K)      48 (   0.05MB/s)      32 (   0.0MB/s)      0.2ms    0.0%     0%
-------Reads/s---------Writes/s---------IOPS-----------KBRead/s--------------KBWrite/s-------------asvc_t--%Wait--%DiskBusy-
Avg    16.7 ( 34%)    31.9 ( 66%)     49 (  0.05K)     267 (   0.26MB/s)    1275 (   1.2MB/s)      0.4ms    0.0%     2%
-----------------------------------------------------------------------------------------------------------------------------------------------------------------
Max   156.0( 0.16K)  136.0( 0.14K)   157 (  0.16K)    2496 (   2.44MB/s)    5709 (   5.6MB/s)      0.9ms    0.0%    13%
------------------------------------------------------------------------------------------------------------------------------------------------------------------

2. Calculate IOPS from existing iostat output file (post process)


#iopscal <iostat output filename>
Sample :
$ iostat -xncz 5 10  > iostat_file.out
$iopscal   iostat_file.out

     |--------------TOTAL IOPS--------------------| |------KB_Read------| |-----KB_Write------| |------Device Status------|
Count        r/s         w/s        io/s   io/s(K)      kr/s                kw/s                  AvgSrvTm %Wait %DiskBusy
-----------------------------------------------------------------------------------------------------------------------------------------------------------------
1       3.0 (  3%)   112.9 ( 97%)    116 (  0.12K)      48 (   0.05MB/s)    3686 (   3.6MB/s)      0.6ms    0.0%     5%
2       1.0 ( 50%)     1.0 ( 50%)      2 (  0.00K)      16 (   0.02MB/s)       0 (   0.0MB/s)      0.2ms    0.0%     0%
3       6.0 ( 67%)     3.0 ( 33%)      9 (  0.01K)      96 (   0.09MB/s)      32 (   0.0MB/s)      0.1ms    0.0%     0%
4       2.0 ( 50%)     2.0 ( 50%)      4 (  0.00K)      32 (   0.03MB/s)       8 (   0.0MB/s)      0.2ms    0.0%     0%
5     156.0 ( 99%)     1.0 (  1%)    157 (  0.16K)    2496 (   2.44MB/s)       0 (   0.0MB/s)      0.9ms    0.0%     1%
6       7.0 (  7%)    95.0 ( 93%)    102 (  0.10K)     112 (   0.11MB/s)    5133 (   5.0MB/s)      0.7ms    0.0%    13%
7       2.0 ( 22%)     7.0 ( 78%)      9 (  0.01K)      32 (   0.03MB/s)       4 (   0.0MB/s)      0.2ms    0.0%     0%
8       1.0 ( 50%)     1.0 ( 50%)      2 (  0.00K)      16 (   0.02MB/s)       0 (   0.0MB/s)      0.2ms    0.0%     0%
9      15.0 ( 50%)    15.0 ( 50%)     30 (  0.03K)     238 (   0.23MB/s)     669 (   0.7MB/s)      0.2ms    0.0%     0%
-------Reads/s---------Writes/s---------IOPS-----------KBRead/s--------------KBWrite/s-------------asvc_t--%Wait--%DiskBusy-
Avg    16.7 ( 34%)    31.9 ( 66%)     49 (  0.05K)     267 (   0.26MB/s)    1275 (   1.2MB/s)      0.4ms    0.0%     2%
------------------------------------------------------------------------------------------------------------------------------------------------------------------
Max   156.0( 0.16K)  136.0( 0.14K)   157 (  0.16K)    2496 (   2.44MB/s)    5709 (   5.6MB/s)      0.9ms    0.0%    13%
------------------------------------------------------------------------------------------------------------------------------------------------------------------


----------------------------------------------------------------------------------------

iopscal [Dowload Here]

-----------------------------------------------------------------------------------------
Note: You might get this error, if there is no disk activity currently. 
"Input error: Disk count equals zero. Exiting"