One of the Sample Issue: Customer frequently sees in /var/adm/messages

syslogd: going down on signal 15

Full Story

For example, the following route commands with -p option will add or delete the persistent static routes.

# route -p add 192.168.1.0 129.148.192.248 1
# route -p delete 192.168.1.0 129.148.192.248

Note: -p option works only on solaris 10 with Patch ID: 122539-06 or Solaris 10 KU patch 118833-36 or above installed

To setup static routing on Solaris 10 systems, just create your own /etc/rc*.d/S*script containing the “route add” invocations desired or create a transient SMF service that adds routes at boot time as shown in the following two options.

Option 1:

Create a new startup script in /etc/rc2.d directory say S99Networkroutes and then in that file put “route add net” lines. Those routes will get added when system reboots.

# cat /etc/rc2.d/S99Networkroutes

#!/sbin/sh #
case “$1″ in ‘start’)
# Put additional routes here:

# /usr/sbin/route add net 192.168.32.0 mygateway 1 -netmask 255.255.255.0
;;
‘stop’) exit 0 ;;
*) echo “Usage: $0 { start | stop }” exit 1 ;;
esac
Option 2:

- Create a script called /usr/bin/routeradd

# cat /usr/bin/routeradd

#!/bin/sh

# route add 192.168.1.0 129.148.192.248 1
– Create a .xml file

# cat /var/svc/manifest/network/routeadd.xml

 - Now import the xml data into the repository.

# svccfg -v import /var/svc/manifest/network/routeadd.xml
– Never try to disable it since this is not a service, it is a one shot script that will only run at boot time.

Here you go with the procedure to configure SAN ( EMC ) storage with native multipathing and veritas volume manager. You dont need to go with complete procedure in a single read, because it has different tasks involved and the procedure was well implemented on a production box without downtime and without disturbing existing setup.

These kind of tasks has to be performed with extreme care, and every step should be implemented after carefully observing the completion status of previous operations.

And from my experience, the most of the mistakes that happen during storage configuration is because of human errors instead of technical mistakes. I am explaining one of easy mistake here:

Example for a simple human mistake that could cause severe damage to the business:

During the veritas configuration, by mistake if we add a new disk to a different diskgroup other than the expected one. And after two/three steps we realized our mistake and started to correct the mistake (i.e removing new disks from the wrong diskgroup and add them back the correct one ) with a procedure that we dont have in our actual implementation plan.
In a normal situation it is simple and straight forward to fix this issue, but knowing the facts about business risk and deadly timelines to complete the task makes us feel nervous and could lead us to more serious errors while correcting early mistakes ( i.e. we may remove different disks from the wrong diskgroups). So be observed and focused while working on storage operations.

  Full Story

How to collect crash dump for solaris x-86 arch servers.

Sometimes your System crashed/paniced all of a sudden and you will see there will not be any crash dump files created for analysis. But Vendor always require system dump files to analyze the server to provide with the RCA. Here in I am presenting a simple way to generate crash dump files every time when your system got crashed. I am assuming that the dump devices (dumpadm) is already configured and crash directory (/var/crash/) is present. Full Story

System Administrators encounter frequent issues with onboard PCI slots after the maintenance operations like the OS upgrades, patch installations and firmware upgrades. And often it will leave us in a situation that we couldn’t  boot up the machine, because of the issues to recognize PCI slots after the maintenance. To troubleshoot the issue and to consider the operation like patch / firmware rollback, we should boot the machine without PCI slots.

For the models like V440 and V490, we have one nice command to disable the onboard devices safely, i,e, asr-disable. And the functionality of “asr-disable” command not limited to PCI slots alone, but we disable various hardware components of motherboard as part fault diagnosis.

In this post I will be explaining you the procedure to disable online PCI slots, which are normally used to host the HBA adapters for external SAN storage. Full Story

Creation of a Hardware Mirrored Volume in Solaris Servers:

1.) Verify which hard drive corresponds with which logical device name and physical device name, using the raidctl command:

# raidctl
Controller: 1
Disk: 0.0.0
Disk: 0.1.0
Disk: 0.2.0
Disk: 0.3.0
Disk: 0.4.0
Disk: 0.5.0
Disk: 0.6.0
Disk: 0.7.0 Full Story