Useful tips on sed and awk

I was recently developing a code where I came across various search techniques and pattern matching that is possible using ‘sed’ and ‘awk’. Am sharing some of those tips here :

  • You want to search and print the second category in a given file. Here is how a file looks like :















Command: sed -nr ‘/^Apple/H;//,/^done/G;s/\n(\n[^\n]*){2}$//p’  file_name





  • Printing the sum of a given column in a file :

1  1

2  4

3  9

Command: awk ‘{ sum += $2 } END { print sum }’  file_name

Output: 14

  • sed  has the N command which will read the next line into the pattern space.

Command:  sed -n ‘/pattern/{N;p}’ file_name

In order to print 2 lines simply put {N;N;p}

  • Print section of file based on line numbers (lines 2-10, inclusive)

Command: sed -n ‘2,10p’ file_name

  • Print the last line of a file (emulates “tail -1”) 

Command: sed -n ‘$p’ file_name

I came across this page to understand few one-liner commands.

PyCon India 2013

PyCon India 2013 – a premier conference in India on using and developing the Python programming language is conducted annually by the Python developer community and it attracts the best Python programmers from India and abroad. Conferences are the best place to meet old friends, make new one, discuss technical as well as non-technical things at same place. I made it to PyCon this year and met many old faces and made new friends too. It was a three day event starting from 30th August to 1st September 2013 with the first day focusing on workshops and last two days on conference with several talks lined up in two parallel tracks. There were BoF sessions conducted too.

Day Two – After Kiran’s keynote, I attended a session on building a super fast, scalable, distributed Python project using ZeroMQ by Srinivas. He explained about the message queues we can use to scale our code. Then I met Sanket, Co-Founder of CampusHash who conducts workshops-cum-hackathon; he discussed about packaging and distributing Python code. This talk changed the way I use to build modules and package them. I happen to meet a college pal of mine Rakesh at Plivo booth. He was demoing the company’s product, this called me to attend the talk on web telephony and how they use Python to build it. I also went to BlueJeans booth to see their demo on video collaboration in cloud. Discussed about the cloud vendors they use and learnt that they have a private cloud set up on OpenStack. I had some discussion on the networking piece, it went on for sometime, and learnt where I lagged in setting it up at our datacenter. We went for a lunch break and yes food was really tasty. After the break I attended a session on configuration management using Ansible. The talk was really interesting as I have been using Chef in my day to day work. Ansible is an IT orchestration engine written in Python and it makes deployment of applications, task-execution and systems easy. Post this session I met the speaker and discussed about functionality of Salt and Fabric, what differences does it make if I choose to use Ansible in the deployments.

Day Three – Keynote speaker for third day was Kenneth Reitz, sitting in the balcony I was silently listening to his talks. Until lunch I spent my time visiting various booths, seeing demo of their products. It was just awesome. I met Pavan at McAfee booth where I shared my exposure with anti-virus testing on Open Enterprise Server during my period of stay with Novell. Old memories cherished. I met few RedHat follks with whom I discussed my experiences with OpenStack and apparently one of them was a QA guy for OpenStack. May be next time when I give a shot to OpenStack I can keep in mind about his findings for the RDO project.
Python for testing, yes, a session by RedHat lady Anisha was the one which I couldn’t miss. Slides for the talks are here. Another session on penetration testing was cancelled due to unavailability of the speaker. The last talk that I attended was about using IPython Notebook by Konark Modi.
pyconI am seen busy arranging my laptop bag sitting next to Dhruv, a RedHat guy.Later in the evening we had a AGM of PSSI and various points related to spreading Python in India were discussed.

Journey from Linux Systems Admin to a DevOps Engineer

It was an open source event that delve me into *NIX and since then my real interest were in Systems. I was hired as a Linux Systems Administrator in a start-up named Atlantis Computing and I really enjoyed my work. It was in the year 2007 when the journey began.
System Administrators [SysAd] are the people who run millions of computer that makes everything in our daily life possible. Without them there will be no large working computers anywhere. Imagine a world where air-traffic controllers don’t have computers, where you need to talk to an operator anytime you want to make a phone call and where you have no e-mails. It’s hard to imagine. Right ?
The role of SysAd is quite diverse and encompasses variety of specialities ( same way as doctors graduate and specialize in surgery, pediatrics, and so on ). At the higher level SysAd is responsible for building and maintaining the computer systems for an organization. When it crashes, we fix it, when it under-performs, we tune it and when it grows old, we upgrade it.
The role of managing these and other computer systems comes under Operations group. An Operations group can include variety of people, from Desktop support team for a company, to people who design and build large scale computing infrastructures, to senior managements who decides company’s IT decisions.
I chose to be a SysAd and I enjoyed it greatly. I get to sit behind the scenes ensuring users get to information they need without interruption. It lead to a vast number of opportunities which can take whole life time to explore and enjoy. I actively participate in various local user group meetings and attend conferences to meet different kinds of people. I spend time in responding to e-mails in various mailing lists and feel happy when their problems are fixed. And yes we have our day too.

Well, after being in Operations field for more than four years I got an opportunity to attend DevOps Days 2011 at ThoughtWorks, Bangalore and I have already shared my experience with it in one of my previous posts. I was overwhelmed and thought I was not giving justice to myself after learning about the creative ideas and solutions. I started interacting with folks about a career in DevOps. A fellow ThoughtWorker named Ram guided and motivated me to a large extent.
So what are DevOps skills is another question in my mind ? And it is answered very nicely on Puppet blog. Today, I can best utilize my Operational skills while interacting with Developers. I now see that continuous product deployments have improved. Making realistic deployment environments available to the team so that production deployments can be exercised early and automated. I am now in the process of hardening my skills on Chef, Puppet because I like automating most of my tasks.
A couple of months back OpsCode hosted ChefConf 2013, a 4 day event focused on adopters and contributors to Chef. Our organization Relevance Lab was participating sponsor. While my colleague attended it, I had an opportunity to attend live webcast. It was a rich experience, ChefConf demonstrated the electricity of how DevOps is shaping the future of how businesses compete in market place.
I am exposed to many automation tools and cloud computing providers like Amazon, OpenStack. I can now apply thoughts on how Infrastructure can be programmable. I am breathing a DevOps culture.

Laptop lid close behavior on Fedora Core 18

I just installed Fedora Core 18 on my laptop and learnt that GNOME no longer handles power policies on closing the laptop lid. It is done by systemd now. You can read the reference here.

I installed gnome-tweak-tool to see if any setting there can affect the behavior. Gnome-Tweak-Tool has the following settings done on my laptop – Click on “Shell” ; ‘Laptop lid,when closed, will suspend even if there is an external monitor ‘ Set this to “OFF”. Apparently it doesn’t effect.

Non-working stuffs :
Use dconf-editor to establish a setting of /org/gnome/settings-daemon/plugins/power of lid-close-ac-action to one of blank or nothing.
gnome-tweak-tool (Advanced Settings) settings on the “Shell” tab is no longer available in GNOME 3.6

Solution :
The solution for this issue is : Edit (sudo) vi /etc/systemd/logind.conf and change the line #HandleLidSwitch=suspend with HandleLidSwitch=ignore
Then restart the service using the following command : (sudo) systemctl restart systemd-logind.service
Now close the laptop lid, it will not suspend.

This is helpful when you put your laptop on a dock, and connect with external keyboard, mouse and external monitor with laptop lid closed.

Solved wireless issue on Compaq 420 laptop

My laptop ( Compaq 420 ) is armed with CentOS 6. Freshly installed.
There was some issue with the wifi driver, because of which I remained connected to a wired network.
I sat and spent some time in troubleshooting it and landed up with solutions posted on CentOS forum.
Few details regarding the Operating System :
[root@ashutosh ~]# uname -arm
Linux ashutosh 2.6.32-279.22.1.el6.x86_64 #1 SMP Wed Feb 6 03:10:46 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux

[root@ashutosh ~]# lspci | grep controller
00:02.0 VGA compatible controller: Intel Corporation Mobile 4 Series Chipset Integrated Graphics Controller (rev 07)
00:02.1 Display controller: Intel Corporation Mobile 4 Series Chipset Integrated Graphics Controller (rev 07)
00:1a.0 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #4 (rev 03)
00:1a.1 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #5 (rev 03)
00:1a.2 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #6 (rev 03)
00:1a.7 USB controller: Intel Corporation 82801I (ICH9 Family) USB2 EHCI Controller #2 (rev 03)
00:1d.0 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #1 (rev 03)
00:1d.1 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #2 (rev 03)
00:1d.2 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #3 (rev 03)
00:1d.7 USB controller: Intel Corporation 82801I (ICH9 Family) USB2 EHCI Controller #1 (rev 03)
02:00.0 Network controller: Realtek Semiconductor Co., Ltd. RTL8191SEvA Wireless LAN Controller (rev 10)
85:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8101E/RTL8102E PCI Express Fast Ethernet controller (rev 02)

From the above output it’s clear that the laptop has a Realtek RTL8191SEvA Wireless device.
Luckily it’s supported by the kmod-8192se package from ELRepo. Please follow the instructions to setup ELRepo repository and install the package by :
yum install kmod-r8192se
and reboot.
Wireless works out of the box.

Exploring SuSE Studio

I started exploring with SuSE Studio at work when I found issues working with various hardware ; installing and re-installing virtual machines on them for testing software patches which were released at very short interval of time. I had issues with logging in to SuSE Studio website earlier which is now resolved as per this blog post. I am now creating appliances for various test suites and in a process to upload them for users. I will better write a page on my experiences once am done with building appliances.

%d bloggers like this: