This post is all about the purpose of I/O fencing and how i/o fencing works to avoid the split brain condition?.

Normally, I do spend  4 to 6 hours to post a technical article  with complete details, but for this one I have spent lot more time. And the reason is “my promise to you all to explain the complex technology in simple terms”.  And I believe, the I/O fencing is one of the complex and confusing topic that most people try to avoid it during their learning ( even I did the same), but unfortunately we cannot skip this part for so long due to the increased complexity of our UNIX networks. To make the concept simple to understand, I have made whole setup to simulate the real time failure.  Please Go Ahead …..

My Lab Configuration:

Gurkulvcs1 – Solaris 10 with VCS 5.0 + VxVM MP1

Gurkulvcs2 – Solaris 10 with VCS 5.0 + VxVM MP1

ISCSI SAN  SERVER – Sun ZFS 7000 Appliance

Storage Mulipath – DMP enabled  for all the VxVM Disks

Video to Explain Split Brain Condition ( Refresh Screen, if you don’t see the Video)

Full Story

Cluster heartbeat connections are important components of VCS cluster, to allow cluster to work as expected for fail over and failback operations. In this post i will be showing to some basic verification that helps unix admins to confirm the health status of the cluster heartbeats.

  Full Story

About the VCS configuration files

VCS configuration files include the following:

main.cf

The installer creates the VCS configuration file in the /etc/VRTSvcs/conf/config folder by default during the VCS configuration. The main.cf file contains the minimum information that defines the cluster and its nodes.

types.cf

The file types.cf, which is listed in the include statement in the main.cf file, defines the VCS bundled types for VCS resources. The file types.cf is also located in the folder /etc/VRTSvcs/conf/config. Full Story

Sample VCS configuration file for single Oracle instance Review the sample configuration with a resource of type Oracle that is configured as follows in main.cf file.

The shared disk groups and volumes in the cluster are configured as resources of type DiskGroup and Volume respectively. The volumes are mounted using the Mount agent. The virtual IP address for the service group is configured using the IP and NIC resource types. The Oracle server can be started after each of these resources is brought online.

Note: If your configuration does not use Veritas Volume Manager, use the DiskReservation resource type to configure shared storage instead of the DiskGroup and Volume resource types

VCS requires you to set up a private network between the systems that form a cluster. You can use either NICs or aggregated interfaces to set up private network.

You can use network switches instead of hubs. However, Sun systems assign the same MAC address to all interfaces by default. Thus, connecting two or more interfaces to a network switch can cause problems.

For example, consider the following case where:

• The IP address is configured on one interface and LLT on another
• Both interfaces are connected to a switch (assume separate VLANs)

The duplicate MAC address on the two switch ports can cause the switch to incorrectly redirect IP traffic to the LLT interface and vice versa. To avoid this issue, configure the system to assign unique MAC addresses by setting the eeprom(1M) parameter local-mac-address to true.

The following products make extensive use of the private cluster interconnects for distributed locking:

• Veritas Storage Foundation Cluster File System (CFS)
• Veritas Storage Foundation for Oracle RAC (SF Oracle RAC)

Symantec recommends network switches for the CFS and the SF Oracle RAC clusters due to their performance characteristics.

Full Story

VCS uses two components, LLT and GAB, to share data over private networks among systems. These components provide the performance and reliability that VCS requires.

LLT (Low Latency Transport) provides fast, kernel-to-kernel communications, and monitors network connections.

LLT configuration files are as follows:

/etc/llthosts—lists all the nodes in the cluster
/etc/llttab—describes the local system’s private network links to the other nodes in the cluster
GAB (Group Membership and Atomic Broadcast) provides the global message order that is required to maintain a synchronized state among the nodes. It monitors disk communications such as the VCS heartbeat utility. The /etc/gabtab file is the GAB configuration file.

LLT and GAB initialization configuration files include:

/etc/default/llt
/etc/default/gab Full Story