Regular expressions are extremely useful in extracting information from text such as code, log files, spreadsheets, or even documents. As an unix administrator we do regularly use the regex to perform various problem diagnosis , automation or remediation tasks. And it is almost impossible to imagine the life of unixadmin without RegEx usage. This quick virtual lab session will give you the basic usage knowledge of regular expressions with grep, which is a  essential skill required for unix system administrators.

Below is the Learning Map, that makes you comfortable with Grep  and RegEx ( i.e. Regular Expressions) within a day:

  Full Story

First of all my apologies, I haven’t realized that my email posts changed from full posts to excerpt mode while changing some other feeder settings. Now, I have changed the posts to full content mode again, to let you read entire post from your inbox comfortably. If you don’t see complete post in your email, please drop me a message. 

If you haven’t subscribed for email posts  yet, you can do it now from the subscription page

Back to current post

As discussed in my earlier post , i.e. automation in system administration , automation become very important skill set in systems management. Most of the Organisations looking for the solution to rapidly configure thousands of servers.

Below are some of the system management tasks that organizations want to automate:

1. Tasks related to Software/Package Installation
2. Tasks related to the changes to be made to the configuration files
3. Tasks related to Scheduling jobs
4. Tasks related to Installing, Configuring and Managing various services

Traditionally System administrators trying to solve above challenges with the help of their awesome scripting knowledge and in-depth system administration experience.But there are some problems with these method, those are

1. Complexity involved in rollback the configuration changes
2. After certain period, we will lose the track of configuration changes that applied to a machine
3. And Most importantly, it requires a whole lot of efforts and people to keep up this scripts based automation system.

What are the alternative solutions ?

Puppet Tool ( from puppetlabs.com ) is one of the popular open-source platform, that allows Automation of System Management Tasks for thousands of servers. Puppet helps systems administrators to build a robust environment with the ability to rollback any configuration change to its previous state, without much complexity.

The biggest challenge in automating system configuration is understanding the complex relationships between servers, the services they give and the objects that comprises the services. Puppet addresses this challenge with the help of , easy to use , puppet language.

I started experimenting with puppet for past several days, as I was personally impressed with the way that puppet assists to the systems administrators  I am going to share my experiments with puppet  with you ( for my own benefit of learning through discussion) in a series of puppet tool posts. This is the first post of the series

Full Story

Here I am posting the script that mirrors primary root disk to selected mirror disk ( of same geometry and size ) . And script will automatically perform the mirror attach operations that required  before and after the server reboot.  The script  will perform following operations, just for your understanding purpose I have made it clear by making comments in each section so that the script will be self explanatory.

Note: You should have atleast one slice empty in primary harddisk which doesn’t have any filesystem and not mounted. This is for the purpose of storing Meta database replicas

### usage : SVM-root-mirror -d    ####
## Verify the Script arguments. If any error display the script usage
## Find out what is current root disk
## Find out what is mirror disk
## Backup the current partition table for both disks
## copy the Partition table from first disk to mirror disk
## find out the Disk slices from the primary disk, that needs to be mirrored
## Make entries to /etc/lvm/md.tab and Create  oneway mirror for root slice from first disk
## Create one way mirrors for remaining slices(from first disk) and modify /etc/vfstab.
## Create a Self destructive RUN level script to attach the second mirror components after the manual reboot Full Story

In our previous example we have discussed about a scenario , in which we have copied a script to each host and then we executed the script remotely using another calling script from the centralized host ( which is having publick key – password less authentication for each client ). But the problem in the previous scenario is we are making to connection to each host. And that is very expensive task in terms of network and time.

in real world, we will keep all the important scripts in a shared directory ( I mean NFS shared directory) and then we will call the client script using a remote calling script. In this way we can avoid the extra efforts require to copy the client script in each host every time when we modify the client script.

Coming to current scenario, I am showing you a demonstration of a task to remotely add a static route in each host mentioned in a hosts list file.

Full Story

It’s been long time that I have posted something on automation/scripting. And every time when I thought of writing a post for this section, I used to divert by one or another concept. But finally I got one quick post for you. And this post is mainly for the beginners who don’t know about “where the scripting knowledge fits in their job?”, and the purpose of this post is to explain a sample scenario where  scripting could be useful to complete the hours of tiresome work in few hours. 

Scenario :  Let us assume, you are supporting a client with 100 linux servers. And one fine day you have received a mail from your client, as below

” I am having a system  audit for my server environment, I want to know what are the hardware configurations I have in each system like CPU, Memory, Network, and filesystems. Can you please provide me this information by end of the day? “

And you know you have it provide the information to the client before the day. Now the question is how do you want  to collect this information.

Manual Solution:

 If you are not comfortable with any kind of scripting, then the answer is “manual login to each host, run the commands manually and collect the outputs to a mail “. And the time you spend for each system is minimum of 15 to 30 mins. And in overall you need more than a day to collect this information manually.

Scripting solution : If you know little bit of scripting ( note the word little bit, you don’t need to be a scripting master to complete your simple tasks) you can complete the same task within 90 mins. ( assuming you will take a hour to write a script for your requirement). Full Story

What is Automation?

Automation is nothing but performing system administration tasks without manual interventions, and complete the operations using specific tools/programs/scripts.

Example: Writing a script to backup the data from one server and run it using cron scheduler on every night.

Why the Automation is important for System Administrators?

My experience says ” if you ask a system admin to delete a single file from a set of 100 servers by login into each server one by one, he will be having 90% chances to make at least one human mistake during his task. And the mistake is not because of the person’s technical incompetence but it is because of natural human behavior called fatigue”.  The Sysadmins are more prone to this fatigue when they are involved in such manual repeated work during their routine job. Full Story