Announcement 1 : Introduction to Storage Administration at unixadminschool.com

In most of the startups and small scale organizations it is often possible that unix administrators will act as seasonal storage administrators to deal with Netapp and EMC devices. Considering the fact that storage articles might be useful for those folks, we have started posting about storage administration.  For past one week , you might have noticed several storage administration articles by Ravi Kiran.And I’m Thankful to Ravi for choosing unixadminschool.com as platform to share his knowledge.

About Ravi Kiran, in his own words:

My name is Ravi Paladugu, I am pursuing doctoral degree in the area of Data Science and currently working as a Storage and Hadoop consultant specializing in Apache Hadoop, EMC and NetApp products, USA.

As a storage consultant I am involved in day-to-day administration, leading projects, performing migrations and automation. I had good opportunity to work with industry leading cutting edge products like Isilon, DataOntap (7-mode and Cluster-mode), Clariion, Celerra, VNX, VMAX, Recoverpoint and multiple data protection technologies. My expertise partially covers Cisco Networking (FC and IP), Windows Administration, performance analysis and scipting. I have been certified as CCNA, MCSE, EMCSA, NCDA.

As a Hadoop consultant I have provided POC’s for couple of environments to deploy Hadoop traditionally and also on EMC Isilon. My experience with Big data includes design, deployment and administration of Hadoop and related components by providing end-to-end data protection and identifying performance bottlenecks in the infrastructure.

Full Story

Most of the technology geeks may have heard the recent buzz about Big Data; in recent times many of my colleagues and friends were asking several questions. So I thought I should write a blog post to better answer their questions.

What is Big Data?

Big Data is defined ‘n’ number of ways in the industry, so instead of trying to find the actual definition lets try to understand the concepts and idea behind it.

As the name says “big data” you may think it is all about size – but that is not just that. There is a lot more to deal with and enormous number of use cases in multiple domains. One of the ways to explain BigData is “V3”

Full Story

Hadoop is a platform that is well suited to deal with semi-structured & unstructured data, as well as when a data discovery process is needed. That isn’t to say that Hadoop can’t be used for structured dara that is readily available in a raw format; because it can.

Traditionally, data goes through a lot of rigor to make it into the warehouse. This data is cleaned up via various cleansing, enrichment, modeling, master data management and other services before it is ready for analysis; which is expensive process. Because of that expense, its clear that data that lands in warehouse is not just high value, but has a broad purpose; it is used to generate reports & dash-board where the accuracy is the key.

In contrast, Big Data repositories very rarely undergo the full quality control versions of data injected into a warehouse, Hadoop is built for the purpose of handling larger volumes of data, so prepping data and processing it should be cost prohibitive.

I say Hadoop as a system designed for processing mind-boggling amounts of data

Two main components of Hadoop:

1. Map – Reduce = Computation
2. HDFS = Storage Full Story

Regular expressions are extremely useful in extracting information from text such as code, log files, spreadsheets, or even documents. As an unix administrator we do regularly use the regex to perform various problem diagnosis , automation or remediation tasks. And it is almost impossible to imagine the life of unixadmin without RegEx usage. This quick virtual lab session will give you the basic usage knowledge of regular expressions with grep, which is a  essential skill required for unix system administrators.

Below is the Learning Map, that makes you comfortable with Grep  and RegEx ( i.e. Regular Expressions) within a day:

  Full Story

This is a following post for my previous SAN migration post  Solaris host level SAN migration (with SVM) from Clariion to VMAX.   As discussed earlier in my previous post,  many organisations started migrating their Server storage from Old legacy SAN devices ( e.g. EMC Clariion )  to new  powerful SAN storage ( e.g. EMC Symetrix VMAX) because of the low performance and maintenance costs involved with legacy storage.  

                           Depending on the budget allocated for the Storage Migration projects, some organisations  prefer the migration by “direct storage level data replication  using expensive migration tools”,  while the other companies ( who are with limited budget) prefer to do the migrations by host level data replication. In the later method,  the success rate of the migration project directly depends on the skill level and expertise of the  unix administrator who is implementing the migration project.  I believe this hands-on post will give some idea for the Solaris admins whoever responsible for the Storage migrations. Before going to this post you might want to refer this post for the pre-planning tasks. 

Full Story