Ubuntu Commands for viewing hardware details

View Hard disk space
bacharya@bacharya:~$df -h

Filesystem Size Used Avail Use% Mounted on
/dev/sda3 185G 59G 118G 34% /
varrun 1.7G 124K 1.7G 1% /var/run
varlock 1.7G 0 1.7G 0% /var/lock
udev 1.7G 44K 1.7G 1% /dev
devshm 1.7G 336K 1.7G 1% /dev/shm
lrm 1.7G 40M 1.6G 3% /lib/modules/2.6.24-28-generic/volatile
/dev/sda1 464M 47M 393M 11% /boot
/dev/sda4 43G 190M 40G 1% /others


View memory details (in MB)

bacharya@bacharya:~$free -m
total used free shared buffers cached
Mem: 3286 3109 177 0 340 992
-/+ buffers/cache: 1776 1510
Swap: 3906 0 3906

View Processes

bacharya@bacharya:~$top

View processor details
bacharya@bacharya:~$ cat /proc/cpuinfo


processor : 0
vendor_id : GenuineIntel
cpu family : 15
model : 4
model name : Intel(R) Pentium(R) 4 CPU 2.80GHz
stepping : 9
cpu MHz : 2800.531
cache size : 1024 KB
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 5
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx lm constant_tsc up pni monitor ds_cpl tm2 cid cx16 xtpr lahf_lm
bogomips : 5601.06

Apache Pig Tutorial

Apache Pig Tutorial


Pig is a platform for analyzing large data sets that consists of a high-level language for expressing data analysis programs, coupled with infrastructure for evaluating these programs. The salient property of Pig programs is that their structure is amenable to substantial parallelization, which in turns enables them to handle very large data sets. Defined by apache

Requirements for Pig

have your Hadoop system setup. Because Pig is not meant to run alone but in coordination with HDFS/MapReduce.

General WorkFlow of Pig is depicted in the workFlow image.
First step invloves loading data . Data can be logs of sql dumps or text files. The second phase involves manipulation of data like filtering, using foreach, distinct or any user defined functions. The third step involves grouping the data. And the final stage involves writing the data into the dfs or repeating the step if another dataset arrives.

Installation of Hadoop is required for Pig.Make sure we have our namenode and tasktracker set in hadoop configuration files. These configurations are necessary because Pig has to connect to the HDFS namenode and JobTracker node.

Running pig jobs in a cluster
Copy the sample dump to the DFS

bin/hadoop fs -copyFromLocal /home/hadoop/passwd .
In my case I am using a dump of Passwd from /etc/passwd

Edit the pig.properties to mention cluster = nameofJobTracker(Map/Reduce)

Edit /bin/Pig.sh file to include the following line

export JAVA_HOME =path to your java_home
export PIG_CLASSPATH=pathto hadoop/conf

Now run the pig command from the pig installation directory
/bin/pig
this should start the namenode and tasktracker of hadoop
If not then we have problems in the configurations. check configuration settings.

Now run your queries, and finally dump the output .

Success !

Disperity and dearth : Does it portrays my identity : A retrospective view of my understanding to my country! Search for Hope.

Being born in the one of the most impoverished nations in the world, having seen and experienced what hunger is, I have seen children's playing in war zone and conflict shattering dreams of many, resulting in a bruised socio-economical and political condition of the beautiful Himalayan Shangrilla. I supposed, the main cause of this was nothing but the poverty and the gap created by poverty between different classes of our society. My definition of a better scientific world when I was in Nepal was (not necessarily a world created with great scientific achievements and lavishness),a place where people would get two times a meal daily, a place where children would not be forced to carry weapons but rather get a chance to go to school and get proper education. Place where there would  be no discrimination amongst caste and creed, where peace and harmony are the languages known rather than war and anguish and where nobody has to give his life either out of any conflict or poverty.This would really be a great challenge for science filling the gap ,the difference between the west and the third world countries like Nepal. My thought is that the next step of science should be in the scientific and uniform distribution of the existing technological achievements throughout the world. As long as there is unevenness in the distribution of the assets of science there will be war zone, inequality, malnutrition and diseases. Up until now, we have had scientific achievements and its distribution in a localized way, leading to the socio-economical segregation between different parts of the world. So, the next step should be to bridge that gap created between technological society and the existing backward society.

Science is knowledge. As until now we have utilized this knowledge in a positive as well as a negative way. Today we are proud to have achieved what our forefathers used to see in their dreams, However, still our achievements are in a tarnished state and the finishing is still left to be done. Hereafter, science should focus on solidifying its achievements in order to avoid the further imbalance prevalent today amongst different sectors of world and society. Similar to the ecosystem if the balance between production and consumption is uneven it will result in the unnatural disturbance, we have to mobilize the achievements of science in order to avoid such disturbance like hunger, poverty, diseases and illiteracy.


Having got a chance to study in Japan, Luckily I got a chance to see what science and technology has had its effect in this foreign soil. At last for a moment I felt I have arrived to the place where I wanted my childhood to nurture, the difference I saw was imminent. A big building I could see reminded me of a common hut in my village made up of bamboo and soil, huge bridge across the river I compared with the wooden logs we used to cross the river, big roads with huge number of vehicles was of reminder of a pathway in my village where I watched sheep's march and I followed behind those. When I saw those crowds of people running after their destinations I compared myself when I used to run after the lost kites in the sky. My stay in Japan was really amazing, great food to eat, I could use laptops now and learned a lot of things about computers, society and cultures. My experience was similar to the dream I used to see when I was a child, At last I was in my dreamland. I was so much immersed in this world that I left all the agony and frustrations at my country.

At the same time, when I looked back the ruins of my motherland I believed that the role science has played in getting the human generation to 21st century has had its limitations and its role is still unfulfilled up until when country like mine will exist in this world.

My personal view regarding scientific change is that Change depends upon the heart of those who initiates it. There may arrive time when civilization may reach inter-terrestrial level and expand its horizon throughout this universe, As the foundation of which has already been created in the developed countries like the US.Poverty is like a disease, it should be stopped from spreading. Back in my country there are many places like Rolpa where we see people fighting for rotten grains unsuitable to eat, Although science has gone a long way still we see such happenings in many parts of the world, while the rest of the world had been developing My country I looked back was going to dark ages day by day. But, one day I heard that some initiation to bridge the gap between technology had been taken by some group of enthusiasts back in my country and they were working for was for the complete transformation of the villages in remote areas of Nepal.

The place called Myagdi,in the heights of the Himalayas it lied seven hours hard climb to the nearest roadway, These people never saw a telephone and even electricity was out of reach. School was miles away and students didn't have pencils or paper to write with. There arose an inspirational man. A Nepali citizen after he finished his studies in USA came to build a high school in this remote area.While he was teaching in that school, he had to travel for two days to reach the nearby city to check his emails. This condition, he informed to BBC and when BBC publicized his ideas, lots of volunteers form Europe and Americas came to help. When he installed four used donated computers in his school teaching students how to use it, it became a start. With the help of donated computers and wireless devices, he had setup relay stations around his village. Now all of sudden these peoples were connected to rest of the world with the latest technology. The impact was imminent. The people have started using computers that were once a waste for the westerners, Suddenly teleteaching has started, local people benefited form the medical facilities from the doctors of big cities, and villagers now e marketed their local products and household goods. People now use online libraries,online health clinic, and e shopping. Really the wireless boon of the science really had a tremendous impact upon the socio-economical aspects of the rural people. The same moment concentrated not only to Myagdi village, but seeing its success the wireless were similarly set up in the nearby villages and rural parts of Nepal, and the once isolated villages are now a global village connected by one of the most fantastic technologies of Science.

So, An example set up, the impact the wireless project had brought about a revolutionary effect in a then so called Dark Village. There are certainly more such rural parts as Myagdi in Nepal and in many parts of the world. Together and with the aid of some of the fantastic innovations of science we can make the world a better place to live.

The places where it was very difficult to reach and where it was almost impossible to access information regarding outside world. Science was making things possible and was easing peoples life in a remarkable way. How did that happened is really a wonderful story I take a great morale from. A man called Mahabir Pun a Nepali student in USA initiated a per dollar program from the abroad Nepalese and collected that money and went back to the inaccessible villages of Nepal and set up wireless devices which enabled the poor mountain peoples an not only an easy access to information and internet but also they could now easily communicate with other peoples of other mountainous regions easily. It seems a small progress to most of the people, But it turned out to be a miraculous aid for the people of that region. Just after 1 year of the installation of the wireless device suddenly the lifestyle of those people changed dramatically and their purchasing power rose at once. Now here we see how can a small change brought about by science can bring about a tremendous change in peoples life. Not only it was the technology which changed their status from normal people but also it enabled them to learn and acquire knowledge in a much similar way the westerners do.

This step was a bold step which filled the gap between the people of remote areas the rich peoples of the cities. A small step like that can really be a giant step for the whole mankind.

Unicode Encoding Problems with Java- Mysql-Tomcat web page and solution

There are many things to take care if we want to correctly display utf-8 characters from mysql properly. Also if we wish to do internationalization based upon database, the following steps serves as a good place to start with.

Thing to take care :

first : while entering utf-8 data into the table from shell be careful to set names as follows :
By setting name as utf8 we are telling mysql to use the given charset. We need to do it every time we log into the server. If not done mysql regards whatever entered data to the table in the default charset format, which will not make your program display characters properly.

set names utf8 collate utf8_general_ci

Also we can use this command mysql > default-character-set=utf8;

Check your mysql server encodings by this command:

show variables like 'character\_set\_%';
Make sure you have similar response.

Variable_name | Value |

+--------------------------+--------+
| character_set_client | utf8 |
| character_set_connection | utf8 |
| character_set_database | utf8 |
| character_set_filesystem | binary |
| character_set_results | utf8 |
| character_set_server | utf8 |
| character_set_system | utf8


If not edit /etc/my.cnf to make the encodings to utf8 as shown in the above table. Although its not necessary to change all the fields to utf8. I have the following configuration and its working perfectly fine. I am facing no problem of encodings provided i fulfill all other steps carefully.

Variable_name | Value |
+--------------------------+--------+
| character_set_client | latin1 |
| character_set_connection | latin1 |
| character_set_database | utf8 |
| character_set_filesystem | binary |
| character_set_results | latin1 |
| character_set_server | utf8 |
| character_set_system | utf8 |
+--------------------------+--------+


second : create the database with character set utf8, create table with charset utf8 also make the columns of table utf8 . we need to do this to make sure we get characters correctly.


Third : set the page charset and default encoding to utf8 if we are displaying table data in a web page.


<%@ page pageEncoding="UTF-8" %>



Adding these lines instructs the browser to use the utf8 encoding for the page.
By performing step three we will be able to see hard coded utf8 characters in the page properly but this might not work in case of database backed utf8 encoded data. For it to work we need to follow step first and second properly.


This also might not work in case the reponse is coming from server. So we have to explicitely tell the server to use utf8 as encoding for our pages, So we need to go to step fourth.

Fourth : We may also have to write a custom filter to encode the response sent by the server as utf-8. Define a filter and set the responseEncoding to utf-8


Write a ResponseFilter class implementing Filter. and in the doFilter() method set following to tell the server explicitely that you want utf-8 encoded response.


response.setCharacterEncoding("utf-8");
request.setCharacterEncoding("utf-8");
chain.doFilter(request,response);


This should display any utf8 characters properly in any web page or console.


Step Five : Well this is not necessary if we performed all the above steps above, at least not necessary for latest Mysql connectors. But for older versions of the driver we need to add the following to the connection url

jdbc:mysql://localhost/table-test?useUnicode=yes&characterEncoding=UTF-8


Note : - Have to set charset utf8 before entering any data into table from the mysql prompt.

Must Read article :
unicode-how-to-get-characters-right


If problems persists ! feel free to post your problem in this blog. I shall try to solve the issue. Well it worked for me in case of Japanese, Arabian, DevNagari characters and others.

Richfaces -Facelets -Spring JBoss configuration

JBOSS version is : jboss-4.2.0.GA

In order to run richfaces application with JBOSS-4.2.0.GA we need to have the following libraries in the web-inf /lib directory

Commons-beanutils
commons-digester
el-impl
el-ri
jsf-facelets
richfaces-api
richfaces-impl
richfaces-ui

Other libraries are by default included in JBOSS installation so we dont need to add other JSF libraries and commons libraries. If added could cause deployment problems.

eclipse java.lang.Error: Unresolved compilation problem solved

Recently I was working on a project when suddenly i had this error in my eclipse :

java.lang.Error : Unresolved compilation problem

Error bubble were shown at all the source files. I went through the web to solve this issue but nothing was working to me. Finally what I did was

Right clicked the project
set the compiler compliance level to 1.6
Initially it was set to 1.5
Now the problem was solved. I guess it was the issue with different versions of JRE used to build.