Published December 18, 2017 by

Apache Sqoop Overview & Import Data From MySQL to HDFS



Overview on Sqoop

Sqoop is open source s/w from Apache used for transfer data between RDBMS(Oracle, SQL Server, MySQL) and HDFS.

MySQL Database
Connecting to MySQL Database in cloudera VM:

root user: root/cloudera
other user: cloudera/cloudera

[cloudera@quickstart ~]$ mysql -u root -p
Read More
    email this
Published December 18, 2017 by

Streaming data into Hadoop using Apache Flume


Flume:Flume a hadoop echo system s/w used for streaming the logs file from applications int o HDFS.

In this post let's discuuss about following topics.
  • Overview on Flume
  • Streaming log files data into HDFS
  • Streaming Twitter App logs into HDFS
Read More
    email this
Published December 16, 2017 by

Overview on Apache Pig



What is pig?

  • Implemented by Yahoo.
  • Pig Hadoop echo system s/w from apache foundation used for analysing the data.
  • Pig uses pig latin language.
  • Data flow language.
  • handle structured, semi-structured and un-structured
  • Replacement of mapreduce(not 100%)
  • Pig internally uses MapReduce.
Read More
    email this
Published December 16, 2017 by

Apache Hive UDF'S (User Defined Functions)


In this post, let's discuss about Hive UDF's.

  • Creating UDF
  • How to packaging UDF(creating jar file)
  • Add jar file in to hive
  • Test UDF


Steps to create and test UDF's

1) Implement the code for UDF in Java
2) Package java class into jar file copy in some location
3) Add jar file in to Hive CLI
4) Create temporary function in hive
5) Use hive UDF BY  using Query.

Prerequiste: Table should have some data.

Problem statement-1
Find the maximum marks obtained out of four subject by an student.

Package java class into jar file copy in some location.

SELECT CLASS IN ECLIPSE-->RIGHT-->EXPORT-->JAVA-->JAR--> BROWSE THE LOCATION-->PROFILE FILENAME WITH .JAR Extension.

Add jar file in to Hive CLI

hive> add jar /home/cloudera/training/HiveUDFS/getMaxMarks.jar;

Create temporary function in hive

hive> create temporary function getmaxmarks as 'udfs.GetMaxMarks';

Use hive UDF BY  using Query

hive> select getmaxmarks(10,20,30,40) from dummy;   // sanity test

There are 2 types of UDF'S

1) Regular UDF( UDF) ---> Applied on more number of rows in a table
2) User Defined aggregate function (UDAF) --> Group of result sets.

Problem statement-2: Find the mean of marks obtained in maths by all the students.

Package java class into jar file copy in some location

Right click onth package-->export-->java-->provide jar file name.

Add jar file in to Hive CLI

hive> add jar /home/cloudera/training/HiveUDFS/getMeanMarks.jar;

Create temporary function in hive

hive> create temporary function getmeanmarks as 'udaf.GetMeanMarks';

Use functions with queries

hive> select getmeanmarks(social)from t_student_record;

Read More
    email this
Published December 02, 2017 by

Popular Open Source Big Data Tools

Data has become a powerful tool in today’s society, where it translates into direct knowledge and tons of money. Companies are paying through the nose to get their hands on data, so that they can modify their strategies, based on the wants and needs of their customers. But, it doesn’t stop there! Big Data is also important for governments, which helps run countries – such as calculating the census.
Read More
    email this
Published November 25, 2017 by

JavaScriptExecutor in Selenium WebDriver




JavaScriptExecutor in Selenium WebDriver:
In general, we do click on an element using click() method in Selenium.

Sometimes web controls don’t react well against selenium commands and we may face issues with the above statement (click()). To overcome such kind of situation, we use JavaScriptExecutor interface.
Read More
    email this
Published November 25, 2017 by

Scrolling Web Pages using Selenium WebDriver

There are 3 different ways to Scroll Web Pages.

1.Scroll page by proving pixel number.
2. Scroll page till we find a web element on the page
3. Scroll page till bottom of the page


To scroll web page using Selenium, we can use JavaScriptExecutor interface that helps to execute JavaScript methods through Selenium Webdriver.


Syntax:
window.scrollBy(xnum, ynum)
Parameters:
  • xnum is a Number
  • Required. How many pixels to scroll by, along the x-axis (horizontal). Positive values will scroll to the right, while negative values will scroll to the left
  • ynum is a Number
  • Required. How many pixels to scroll by, along the y-axis (vertical). Positive values will scroll down, while negative values scroll up

Return Value: No return value



Examples:
Scroll page by proving pixel number
  js.executeScript("window.scrollBy(0,500)");

Scroll page till we find a web element on the page
  js.executeScript("arguments[0].scrollIntoView();",Element );

 Scroll page till bottom of the page
  js.executeScript("window.scrollTo(0, document.body.scrollHeight)");

Test Script

import org.openqa.selenium.By;
import org.openqa.selenium.JavascriptExecutor;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;

public class Scrolling {
public static void main(String[] args) {
System.setProperty("webdriver.chrome.driver", "C://Drivers/chromedriver_win32/chromedriver.exe");
WebDriver driver = new ChromeDriver();

driver.get("https://www.countries-ofthe-world.com/flags-of-the-world.html");
driver.manage().window().maximize(); // maximum browser window

JavascriptExecutor js = (JavascriptExecutor) driver;

// 1. scrolling by using pixel
js.executeScript("window.scrollBy(0,1000)", "");

// 2. scrolling page till we find element
WebElement flag = driver
.findElement(By.xpath("//*[@id='content']/div[2]/div[2]/table[1]/tbody/tr[86]/td[1]/img"));
js.executeScript("arguments[0].scrollIntoView();", flag);

// 3. scroll page till bottom
js.executeScript("window.scrollTo(0,document.body.scrollHeight)");
}
}
Read More
    email this
Published November 25, 2017 by

How To Run WebDriver Test Case using Microsoft Edge Browser


Download Microsoft WebDriver from here to launch Edge Browser. Download the proper version of the driver based on your OS build number. If the extension of the downloaded file is “.msi“, you need to install it to get the “.exe” driver.

Read More
    email this
Published November 23, 2017 by

How to Write Excel Files Using Apache POI In Selenium WebDriver

Let’s see how to Read excel files using Apache POI in Selenium WebDriver:
Read More
    email this
Published November 23, 2017 by

How To Resize Browser Window Using Selenium WebDriver


To resize browser window to particular dimensions, we use ‘Dimension’ class to resize the browser window.
Read More
    email this
Published November 23, 2017 by

How to Read Excel Files Using Apache POI In Selenium WebDriver

Let’s see how to Read excel files using Apache POI in Selenium WebDriver:
Read More
    email this
Published November 23, 2017 by

Handling Excel Files Using Apache POI In Selenium WebDriver


Selenium supports only Web browser automation. We need to get the help of third party API like Apache POI to handle (read and write) excel files using Selenium WebDriver.
Read More
    email this
Published November 23, 2017 by

How To Find Broken Links Using Selenium WebDriver




One of the key test case is to find broken links on a webpage. Due to existence of broken links, your website reputation gets damaged and there will be a negative impact on your business. It’s mandatory to find and fix all the broken links before release. If a link is not working, we face a message as 404 Page Not Found.
Read More
    email this
Published November 23, 2017 by

Double Click Action In Selenium WebDriver


In some scenarios, we may need to do double click action on a particular element to move further. In such cases, we use Actions class in Selenium WebDriver to work on Mouse and Keyboard Actions. Check out the below code for detailed explanation of Actions Class.

Read More
    email this
Published November 23, 2017 by

Right Click Action (Context Click) In Selenium



In some scenarios, we may need to do right click action / context click on an element to do some actions. We use Actions class in Selenium WebDriver to work on Mouse and Keyboard Actions. 

Read More
    email this
Published November 23, 2017 by

Drag And Drop In Selenium WebDriver


In some applications, we may face a situation to automate drag and drop an item from one location to another location.  Selenium has provided an “Actions” class to handle this kind of scenarios. 
Read More
    email this
Published November 23, 2017 by

Mouse Hover Actions Using Actions Class In Selenium


Sometimes, sub menu items render in DOM only when we mouse hover on main menu. In that case, we face difficulty to click on sub menu item. In order to perform mouse hover actions, we need to chain all of the actions that we want to achieve in one go. To do this we need to make the driver move to the parent element that has child elements and click on the child element.
Read More
    email this
Published November 23, 2017 by

How To Handle Javascript Alerts/PopUps In Selenium WebDriver


In this post, we see how to handle javascript alerts/popus. Alerts are basically popup boxes that take your focus away from the current browser and forces you to read the alert message. You need to do some action such as accept or dismiss the alert box to resume your task on the browser.
Read More
    email this
Published November 23, 2017 by

How To Upload Files Using AutoIT in Selenium


Selenium can not handle file downloading because browsers use native dialogs for downloading files. Sometime we need to download file from AUT(Application Under Test). There are several ways to automate download file in Selenium but here we see download file using AutoIT in Selenium WebDriver.





AutoIt Introduction:

AutoIt Tool is an open source tool. It is a freeware BASIC-like scripting language designed for automating the Windows GUI and general scripting. It uses a combination of simulated keystrokes, mouse movement and window/control manipulation in order to automate tasks in a way not possible or reliable with other languages (e.g. VBScript and SendKeys). AutoIt is also very small, self-contained and will run on all versions of Windows out-of-the-box with no annoying “runtimes” required!
Now the question is how we do download file using AutoIT Tool in Selenium WebDriver.
Steps to integrate autoit with selenium webdriver

1) Write AutoIT script for file uploading( AutoIT Editor)


ControlFocus() --> focus on the text box

ControlSetText() --> providing path of a file
ControlClick() --> clicking on open button

2) Compile AutoIT script and generate .exe file


Tools-->Compile-->Select x64--> Compile --> generated .exe file


3) use and integrate .exe file in selenium webdriver script


Ex: 


Runtime.getRuntime().exec("C://autoitfiles/fileupload.exe"+" "+"C:\\SeleniumPractice\\Fruites\\apple.jpg");
Uploading Single File

AutoIT script

ControlFocus("File Upload","","Edit1")
Sleep(3000)
ControlSetText("File Upload","","Edit1","C:\SeleniumPractice\Fruites\apple.jpg")
Sleep(3000)
ControlClick("File Upload","","Button1")
Sleep(3000)

Note: We need to compile AutoIT script and generate .exe file (SingleFileUpload.exe)


Selenium Test case


import java.io.IOException;

import org.openqa.selenium.By;
import org.openqa.selenium.JavascriptExecutor;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.firefox.FirefoxDriver;

public class UploadSingleFile {


public static void main(String[] args) throws IOException, InterruptedException {


System.setProperty("webdriver.gecko.driver","C://Drivers/geckodriver-v0.19.1-win64/geckodriver.exe");
WebDriver driver=new FirefoxDriver();

driver.get("http://demo.automationtesting.in/Register.html");

WebElement button=driver.findElement(By.xpath("//*[@id='imagesrc']"));

JavascriptExecutor executor = (JavascriptExecutor)driver;
executor.executeScript("arguments[0].click();", button);

Runtime.getRuntime().exec("C://SeleniumPractice/SingleFileUpload.exe"); // execute .exe file

//driver.quit();
}

}


Uploading Multiple Files

AutoIT script

Sleep(500)
ControlFocus("File Upload","","Edit1")
Sleep(500)
ControlSetText("File Upload","","Edit1",$CmdLine[1])
Sleep(500)
ControlClick("File Upload","","Button1")
Sleep(500)

Selenium Test case

import java.io.IOException;
import org.openqa.selenium.By;
import org.openqa.selenium.JavascriptExecutor;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.firefox.FirefoxDriver;

public class UploadMultipleFiles {

public static void main(String[] args) throws IOException, InterruptedException {
System.setProperty("webdriver.gecko.driver","C://Drivers/geckodriver-v0.19.1-win64/geckodriver.exe");
WebDriver driver=new FirefoxDriver();
driver.get("http://demo.automationtesting.in/Register.html");
WebElement button=driver.findElement(By.xpath("//*[@id='imagesrc']"));
JavascriptExecutor executor = (JavascriptExecutor)driver;
Thread.sleep(3000);
//round1- first file
executor.executeScript("arguments[0].click();", button);
Runtime.getRuntime().exec("C://SeleniumPractice/MultipleFilesUpload.exe"+" "+ "C:\\SeleniumPractice\\Fruites\\apple.jpg");
Thread.sleep(5000);
//round2-  second file
executor.executeScript("arguments[0].click();", button);
Runtime.getRuntime().exec("C://SeleniumPractice/MultipleFilesUpload.exe"+" "+ "C:\\SeleniumPractice\\Fruites\\Mangoes.jpg");
Thread.sleep(5000);
//round3-  third file file
executor.executeScript("arguments[0].click();", button);
Runtime.getRuntime().exec("C://SeleniumPractice/MultipleFilesUpload.exe"+" "+ "C:\\SeleniumPractice\\Fruites\\PineApple.jpg");
Thread.sleep(5000);
driver.quit();
}

}




Read More
    email this
Published November 22, 2017 by

TestNG Listeners & Extent Reports in Selenium




In this post, we see TestNG listeners. Listeners “listen” to the event defined in the selenium script and behave accordingly. The main purpose of using listeners is to create logs. There are many types of listeners such as WebDriver Listeners and TestNG Listeners.

Here in this post, we see TestNG Listeners. Using TestNG listeners we could generate logs and customize TestNG Reports.

Read More
    email this
Published November 22, 2017 by

How to use HashMap in Selenium WebDriver





In this article, I discussed how to create hash map in Java for data driven tests and read the data from the HasMap object. Please go through below example.




1) HashMap in Java

Example:

public class HashMapExample {

public static void main(String[] args) {

HashMap hm = new HashMap();

// Adding pairs to HashMap
hm.put(101, "John");
hm.put(102, "Scott");
hm.put(103, "David");

// Printing pairs from HashMap
System.out.println(hm);

// Remove a pair from HashMap by using key
hm.remove(102);

System.out.println(hm);

// Retrive a single pair from HashMap using key value
System.out.println(hm.get(103));

// Reading Keys and values from HashMap
for (Map.Entry m : hm.entrySet()) {

System.out.println((m.getKey() + "  " + m.getValue()));
}

}

}

Output:

{101=John, 102=Scott, 103=David}
{101=John, 103=David}
David
101  John
103  David



2) How to use HashMap in Selenium WebDriver Test case






Example:

import java.util.HashMap;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;

public class LoginTestHM {

// method return HashMap object with data pairs
static HashMap logindata() {
HashMap hm = new HashMap();
hm.put("x", "mercury@mercury");
hm.put("y", "mercury1@mercury1");
hm.put("z", "mercury2@mercury2");

return hm;
}

public static void main(String[] args) throws Exception {

System.setProperty("webdriver.chrome.driver", "C://Drivers/chromedriver_win32/chromedriver.exe");
WebDriver driver = new ChromeDriver();

driver.get("http://newtours.demoaut.com/");

// Login as X
String credentials = logindata().get("x"); // Retriving value 'x' from
// HashMap

// Login as y
// String credentials = logindata().get("y");

// Login as z
// String credentials = logindata().get("x");

String uarr[] = credentials.split("@"); // separting value of 'x' int o
// 2 parts using delimeter '@'

driver.findElement(By.name("userName")).sendKeys(uarr[0]); // Passing
// value 1
// i.e
// username
// from
// array
driver.findElement(By.name("password")).sendKeys(uarr[1]); // Passing
// value 2
// i.e
// password
// from
// array
driver.findElement(By.name("login")).click();

// Validation
if (driver.getTitle().equals("Find a Flight: Mercury Tours:")) {
System.out.println("Test Passed");

} else {
System.out.println("Test failed");

}

driver.findElement(By.linkText("Home")).click();
}

}


Read More
    email this
Published November 09, 2017 by

Functional API Testing - How to Do it Right

API Testing - A Short Introduction

 API, or application program interface, is a system of communication methods that gives developers and non-developers access to programs, procedures, functions and services. The most common protocol used in API is HTTP, together with REST architecture. Developers who program using REST make their code easy to understand. They and others know which language they will be using, how the functions work, which parameters can be used, etc.
Read More
    email this
Published November 08, 2017 by

What Is Big Data?

Big Data refers to large volume of data, that cannot be processed using traditional databases. When we have reasonable amounts of data we typically use traditional relational databases like Oracle, MySQL, SQL Server to store and work with the data. However when we have large volume of data then traditional databases will not be able to handle the data.
Read More
    email this
Published October 27, 2017 by

API Testing Interview Questions

If you're looking for API Testing Interview Questions, you are at right place. There are lot of opportunities from many reputed companies in the world. According to research API Testing has a market share of about 16.7%. So, You still have opportunity to move ahead in your career in API Testing. Mindmajix offers advanced API Testing Interview Questions that helps you in cracking your interview & acquire dream career.
Read More
    email this
Published August 16, 2017 by

ETL Design Process & Best Practices


Introduction

ETL stands for Extract Transform and Load. Typical an ETL tool is used to extract huge volumes of data from various sources and transform the data dependi­ng on business needs and load into a different destination. In the modern business world the data has been stored in multiple locations and in many incompatible formats. The business data might be stored in different formats such as Excel, plain text, comma separated, XML and in individual databases of various business systems used etc. Handling all this business information efficiently is a great challenge and the ETL tool plays an important role in solving this problem.
Read More
    email this
Published July 26, 2017 by

Career in Software Testing for Non-IT Graduates

I am from non – IT background. Can I make my career in Software Testing industry? What is growth path in this field? Can I expect stable career in this sector? Being a non IT graduate I am not pretty good in programming. I don’t have sound knowledge of technologies as well. Will it be a good decision to switch into IT field (especially software testing)?
Read More
    email this
Published July 17, 2017 by

BDD and ATDD – Exploring the Differences

It is a good idea to explore Behavior Driven Development and Acceptance Test Driven Development methods. This post will help you understand how are these two development strategies different from TDD and each other.
Read More
    email this
Published July 17, 2017 by

Top 12 Tools for Web Services Testing

Thanks to the rising emergence of applications for social networking, online shopping, banking and many more which have triggered the requirement of web services.
Read More
    email this
Published July 17, 2017 by

Who Earns more, a Software Tester or a Developer?

Everyone remembers the day they joined their first company. Fresh out from college, all freshers are filled with energy and are pumped up to make their presence felt in the Software world. Eventually, the training starts and everyone gets acquainted with the IT terminologies, different roles in a Software firm, and how different the actual world is outside the textual confinement.
Read More
    email this
Published July 17, 2017 by

Mind-set Difference between Tester & Developer

Some IT professionals think that a tester’s role is redundant and that a developer alone can test the code that was written by him. Such notion is completely wrong. But can you guess why? A developer can definitely test his own code and also the code was written by other developers. But one thing that a developer would lack is the tester’s strategy, approach, and his mindset.
Read More
    email this
Published July 17, 2017 by

Selenium Common Exceptions


When you work with Selenium you willencounter many exceptions. Solving of those exceptions would be sometimes very tricky but If you read the exception the answer to solving is in the meaning of Exception name. We will be going through some most commonly used Exception in Selenium.
  • NoSuchAttributeException find_element_by_* can’t find the element.
  • NoSuchElementException find_element_by_* can’t find the element.
  • InvalidSelectorException Thrown when the selector which is used to find an element does not return a WebElement. Currently, this only happens when the selector is an XPath expression is used which is either syntactically invalid (i.e. it is not an XPath expression) or the expression does not select WebElements.
  • ElementNotVisibleException Thrown to indicate that although an element is present on the DOM, it is not visible, and so is not able to be interacted with.
  • RemoteDriverServerException
  • NoSuchWindowException
  • ElementNotSelectableException
  • NoSuchFrameException
  • WebDriverException
  • UnexpectedTagNameException Thrown when a support class did not get an expected web element
  • UnableToSetCookieException Thrown when a driver fails to set a cookie.
  • MoveTargetOutOfBoundsException Indicates that the target provided to the actions move() method is invalid.
  • InvalidSwitchToTargetException The frame or window target to be switched doesn’t exist.
  • InvalidElementStateException
  • InvalidCookieDomainException Thrown when attempting to add a cookie under a different domain than the current URL
  • ImeNotAvailableException Indicates that IME support is not available. This exception is thrown for every IME-related method call if IME support is not available on the machine.
  • ImeActivationFailedException Indicates that activating an IME engine has failed.
  • ErrorInResponseException An error has occurred on the server side.
  • TimeoutException Thrown when a command does not complete in enough time.
  • StaleElementReferenceException Indicates that a reference to an element is now “stale” — the element no longer appears on the DOM of the page.
Read More
    email this