Overview on Apache Pig

What is pig?

• Implemented by Yahoo.
• Pig Hadoop echo system s/w from apache foundation used for analysing the data.
• Pig uses pig latin language.
• Data flow language.
• handle structured, semi-structured and un-structured
• Replacement of mapreduce(not 100%)
• Pig internally uses MapReduce.

Components/architecture of pig:

Pig Data Model

Atom
Tuple
Map
Bag
Relation

Pig Execution Modes(connection to the pig)

1) Local mode ---> Data will be used from Local file system. commands will run locally.

[cloudera@quickstart ~]$ pig -x local

2) MapReduce Mode(HDFS mode) ---> Data should be part of HDFS. commands will run in MapReduce(Hadoop)

[cloudera@quickstart ~]$ pig -x mapreduce   (or)    [cloudera@quickstart ~]$ pig


Execution Mechanisms( how many ways we can execute pig scripts)

1) Interactive mode (in grunt shell)
2) Batch mode (in unix/linux prompt)


Interactive mode (in grunt shell)

grunt> customers= LOAD '/home/cloudera/training/pigdata/customers.txt' USING PigStorage(',');
grubt> dump;

Batch mode (in unix/linux prompt)


1) Local mode

[cloudera@quickstart ~]$ cat pig_local.pig
customers= LOAD '/home/cloudera/training/pigdata/customers.txt' USING PigStorage(',') as (id:int,name:chararray,age:int,address:chararray,salary:int);
dump customers;

[cloudera@quickstart ~]$ pig -x local pig_local.pig


2) MapReducemode (HDFS Mode)

[cloudera@quickstart ~]$ cat pig_global.pig
customers= LOAD '/user/cloudera/customers.txt' USING PigStorage(',') as (id:int,name:chararray,age:int,address:chararray,salary:int);
dump customers;

[cloudera@quickstart ~]$ pig -x mapreduce pig_global.pig

[cloudera@quickstart ~]$ hdfs dfsadmin -safemode leave;           ---> Optional

Note:  script file extention is  .pig