Monday, 29 April 2013

Java Web Service HelloWorld (Document Style)

Java Web Service HelloWorld (Document style) Example

I will show you how to use JAX-WS to create a SOAP-based web service (document style) endpoint, compare it with RPC style,
It need to do some extra efforts to work it.

There are some steps for creating web service HelloWorld (Document style) Example.
Step 1 : Create a Web Service Endpoint Interface

package com.test.ws;
import javax.jws.WebMethod;
import javax.jws.WebService;
import javax.jws.soap.SOAPBinding;
import javax.jws.soap.SOAPBinding.Style;
@WebService
@SOAPBinding(style = Style.DOCUMENT, use=Use.LITERAL) //optional
public interface HelloWorld{
@WebMethod String SayHello(String name);
}
Note : In JAX-WS development, convert from 'RPC style' to 'Document style' is very easy,
only change the @SOAPBinding style option.

Step 2 : Create a Web Service Endpoint Implementation
package com.test.ws;
import javax.jws.WebService;
@WebService(endpointInterface = "com.test.ws.HelloWorld")
public class HelloWorldImpl implements HelloWorld{
@Override
public String SayHello(String name) {
return "Hello JAX-WS " + name;
}

}

Step 3 : Create a Endpoint Publisher

package com.test.endpoint;
import javax.xml.ws.Endpoint;
import com.test.ws.HelloWorldImpl;
//Endpoint publisher
public class HelloPublisher{
public static void main(String[] args) {
Endpoint.publish("http://localhost:9999/ws/hello", new HelloWorldImpl());
}
}
when you run the publisher, you get the following error message
Wrapper class com.test.ws.jaxws.SayHello is not found.
Have you run APT to generate them?

Step 4 : Run wsgen command to generate the required artifact
Document style requires extra classes to run, you can use 'wsgen tool to generate all necessary Java artifacts
(mapping classes, wsdl or xsd schema). The 'wsgen' command is required to read a service endpoint
implementation class.

wsgen -keep -cp . com.test.ws.HelloWorldImpl

The above command will generates two classes, copy it to your com.test.ws.jaxws folder.

The *****SayHello.java**** File

package com.test.ws.jaxws;
import javax.xml.bind.annotation.XmlAccessType;
import javax.xml.bind.annotation.XmlAccessorType;
import javax.xml.bind.annotation.XmlElement;
import javax.xml.bind.annotation.XmlRootElement;
import javax.xml.bind.annotation.XmlType;
@XmlRootElement(name = "sayHello", namespace = "http://ws.test.com/")
@XmlAccessorType(XmlAccessType.FIELD)
@XmlType(name = "sayHello", namespace = "http://ws.test.com/")
public class SayHello {
@XmlElement(name = "arg0", namespace = "")
private String arg0;
/**
*
* @return
* returns String
*/
public String getArg0() {
return this.arg0;
}
/**
*
* @param arg0
* the value for the arg0 property
*/
public void setArg0(String arg0) {
this.arg0 = arg0;
}
}
The *****SayHelloResponse.java**** File

package com.test.ws.jaxws;
import javax.xml.bind.annotation.XmlAccessType;
import javax.xml.bind.annotation.XmlAccessorType;
import javax.xml.bind.annotation.XmlElement;
import javax.xml.bind.annotation.XmlRootElement;
import javax.xml.bind.annotation.XmlType;

@XmlRootElement(name = "sayHelloResponse", namespace = "http://ws.test.com/")
@XmlAccessorType(XmlAccessType.FIELD)
@XmlType(name = "sayHelloResponse", namespace = "http://ws.test.com/")
public class SayHelloResponse {
@XmlElement(name = "return", namespace = "")
private String _return;
/**
*
* @return
* returns String
*/
public String getReturn() {
return this._return;
}
/**
*
* @param _return
* the value for the _return property
*/
public void setReturn(String _return) {
this._return = _return;
}
}

It is done you can access URL as http://localhost:9999/ws/hello?wsdl
Step 5 :Web Service Client
Create a web service client to access your published service as

package com.test.client;

import java.net.URL;
import javax.xml.namespace.QName;
import javax.xml.ws.Service;
import com.test.ws.HelloWorld;

public class HelloClient{
public static void main(String[] args) throws Exception {
URL url = new URL("http://localhost:9999/ws/hello?wsdl");
QName qname = new QName("http://ws.test.com/", "HelloWorldImplService");
Service service = Service.create(url, qname);
HelloWorld hello = service.getPort(HelloWorld.class);
System.out.println(hello.sayHello("Wakil"));
}
}

Thursday, 25 April 2013

Java Web Service HelloWorld (RPC Style)

Java Web Service

JAX-WS (Java api for xml web service) : JAX-WS is set of api for creating web services in XML format i.e SOAP
(Simple Object Access Protocol). The JAX-WS provides many java annotations for simplify devlopment and deployment of
web service and client.
There are two main commands for creating and deployment webservice and clients ie wsimport and wsgen with many arguments
for details we are giving examples with which you can easily understand the use of wsimport/wsgen commands.
HelloWorld Example (RPC style) : This example show you how to create web service endpoint with RPC style using JAX-WS
web service client with the wsimport command.
JAX-WS comes with jdk1.6 and provides web service development easy.
what is web service endpoint ? i.e web service endpoint is service which published outside for user to access them, and
and web service client is the application who access the published web service.

Steps for creating web service and client
Step 1 : Create web service endpoint interface :
package com.test.ws;
import javax.jws.WebMethod;
import javax.jws.WebService;
import javax.jws.soap.SOAPBinding;
import javax.jws.soap.SOAPBinding.Style;
@WebService
@SOAPBinding(style = Style.RPC)
public interface HelloWorld{
@WebMethod String sayHello(String name);
}
Step 2 : Create web service endpoint Implementation :

package com.test.ws;
import javax.jws.WebService;
@WebService(endpointInterface = "com.test.ws.HelloWorld")
public class HelloWorldImpl implements HelloWorld{
@Override
public String sayHello(String name) {
return "Hello " + name;
}
}
Step 3 : Create web service endpoint publisher :

package com.test.endpoint;
import javax.xml.ws.Endpoint;
import com.test.ws.HelloWorldImpl;
//Endpoint publisher
public class TestPublisher{
public static void main(String[] args) {
Endpoint.publish("http://localhost:9999/ws/hello", new HelloWorldImpl());
}
}
Run the web service, your hello world service is deployed in url 'http://localhost:9999/ws/hello'
you can now test using browser as http://localhost:9999/ws/hello?wsdl, after running the wsdl a xml
output comes in the browser for describing the webservice.
Step 4 : Create web service Client:
package com.test.client;
import java.net.MalformedURLException;
import java.net.URL;
import javax.xml.namespace.QName;
import javax.xml.ws.Service;
import com.test.ws.HelloWorld;
public class HelloClient {
public static void main(String[] args) throws MalformedURLException {
URL url = new URL("http://localhost:9999/ws/hello?wsdl");
//1st argument service URI, refer to wsdl document above
//2nd argument is service name, refer to wsdl document above
QName qname = new QName("http://ws.test.com/", "HelloWorldImplService");
Service service = Service.create(url, qname);
HelloWorld hello = service.getPort(HelloWorld.class);
System.out.println(hello.sayHello("Wakil Ahamad"));
}
}

After Running the client the output would be :
Hello Wakil Ahamad
Java Web service client via wsimport tool
The wsimport command parsed the generated wsdl file and create the necessary client (stub) files to accessing
the publised web service.

Issue the following command as
wsimport -verbose -keep http://localhost:9999/ws/hello?wsdl

The above command generates the necessary client files which is provided by wsdl file. It will generate one Interface and one service implementation file

The generated files are :
********HelloWorld.java *******
********HelloWorldImplService.java *********
The Client :

import com.test.ws.HelloWorld;
import com.test.ws.HelloWorldImplService;
public class HelloClient{
public static void main(String[] args) {
HelloWorldImplService helloService = new HelloWorldImplService();
HelloWorld hello = helloService.getHelloWorldImplPort();
System.out.println(hello.getHelloWorldAsString("wakil Ahmad"));
}
}
The output :
Hello Wakil Ahmad


Monday, 22 April 2013

Java Interview Questions and Answers

Java Interview Questions and Answers





  • Question 1: What is immutable object? Can you write immutable class? what is advantage of Immutable Objects?
  • Answer :Immutable classes are those classes which objects can not be modified once created,
    Any modification in Immutable object result a new object. Yes we can write a Immutable
    class. for example String is Immutable class. Mostly Immutable classes are final in java.
    How to write Immutable Object ?
    1 : State of Immutable object can not be changed, any modification of object result new one.
    2 : Class and their members should be final.
    3 : Members should be private.
    4 : Only accessors(i.e get) methods should be implemented.
    For Example :
    public final class MyPerson {
    private final String name;
    private final String mobile;
    public MyPerson(String name, String mobile) {
    this.name = name;
    this.mobile = mobile;
    }
    public String getName(){
    return name;
    }
    public String getMobile(){
    return mobile;
    }
    }
    Advantage of Immutable Object
    There are several benifits of Immutable objects some are given here
    1 : Immutable objects are by default thread safe, can be shared without synchronization in concurrent environment.
    2 : Immutable objects simplifies development, because it is easier to share between multiple threads without doing external synchronization.
    3 : Immutable objects enhance the java application performance by reducing the synchronization in code.
    4 : You can cache the Immutable objects and reused them
    Disadvantage of Immutable Objects
    Immutable objects are creating garbage because immutable objects are does not cached, it just use and throw.

    Question 2 :Difference between creating String as new() and literal?
    Answer :When we create string with new() Operator, it is created in heap and not added into string pool
    we need to call String.intern() method which is used to put them into String pool explicitly.
    its only when you create String object as String literal e.g. String s = "Ahamad" Java automatically put that into String pool.

    Question 2 :Difference between creating String as new() and literal?
    Answer :When we create string with new() Operator, it is created in heap and not added into string pool
    we need to call String.intern() method which is used to put them into String pool explicitly.
    its only when you create String object as String literal e.g. String s = "Ahamad" Java automatically put that into String pool.

    Question 3 : Which two methods of Object class need to implement in a class for a key Object of HashMap ?
    Answer : In order to use any object as key in HashMap, it must be implements the equals() and hashcode() methods.

    Question 4 :What are the principle concepts of OOPS?
    Answer :The four principle concepts upon which object oriented design and programming based.
    These are:
    Abstraction
    Polymorphism
    Inheritance
    Encapsulation Question 5 : What is abstraction ?
    Answer : Abstraction is the process of selecting important data sets for an Object and leaving out the insignificant ones.
    Once you have modeled your object using Abstraction, the same set of data could be used in different applications.
    Abstraction is achieved by using interface and abstract class in Java. An interface or abstract class is something which is not
    concrete , something which is incomplete. In order to use interface or abstract class we need to extend and implement abstract
    method with concrete behavior. One example of Abstraction is creating interface to denote common behavior without specifying any
    details about how that behavior works e.g. You create an interface called Server which has start() and stop() method.
    This is called abstraction of Server because every server should have way to start and stop and details may differ.
    What is abstract class in Java
    An abstract class is something which is incomplete and you can not create instance of abstract class.
    If you want to use it you need to make it complete or concrete by extending it. A class is called concrete
    if it does not contain any abstract method and implements all abstract method inherited from abstract class or
    interface it has implemented or extended. By the way Java has concept of abstract classes, abstract method but a
    variable can not be abstract in Java. Popular example of abstract class in Java is ActionListener which has abstract
    method called actionPerformed(ActionEvent ae). This method is called when an ActionEvent is fired like when you click on JButton.
    Its common in java to attach ActionListener with JButton by implementing abstract method
    actionPerformed(ActionEvent ae) using Anonymous class, as shown in below Example :

    JButton oh = new JButton("oh");
    oh.addActionListener(new ActionListener(){
    public void actionPerformed(ActionEvent ae){
    //code to handle event
    }
    });
    An abstract method in Java doesn't have body , its just a declaration. In order to use abstract method you need to override that method in SubClass.
    so when do you use abstraction ? when You know something needs to be there but not sure how exactly it should look like.
    e.g. when I am creating a class called Vehicle, I know there should be methods like start() and Stop() but don't know start and stop mechanism of every
    vehicle since they could have different start and stop mechanism e.g some can be started by kick or some can be by pressing buttons .
    the same concept apply to interface in Java also , which we will discuss in some other post.
    so implementation of those start() and stop() methods should be left to there concrete implementation e.g. Scooter , MotorBike , Car etc.
    In Java Interface is an another way of providing abstraction, Interfaces are by default abstract and only contains
    public static, final constant or abstract methods. Its very common interview question is that where should we use abstract class and
    where should we use Java Interfaces in my view this is important
    to understand to design better Java application, you can go for java interface if you only know the name of methods
    your class should have e.g. for Server it should have start() and stop() method but we don't know how exactly these
    start and stop method will work. if you know some of the behavior while designing
    class and that would remain common across all subclasses add that into abstract class. Interface like Runnable
    are good example of abstraction in Java which is used to abstract task executed by multiple thread.

    The Summary of abstraction are :
    1) Use abstraction if you know something needs to be in class but implementation of that varies.
    2) You can not create instance of abstract class , its compiler error.
    3) a class automatically becomes abstract class when any of its method declared as abstract.
    4) abstract method doesn't have method body.
    5) variable can not be made abstract , its only behavior or methods which would be abstract.
    6) If a class extends an abstract class or interface it has to provide implementation
    to all its abstract method to be a concrete class. alternatively this class can also be abstract.

    Question 5 :

    JDBC & PreparedStatement , CallableStatement

    JDBC & PreparedStatement

    The PreparedStatement interface is extended Statement, with extra feature to send a pre-compiled SQL statement
    with parameters. For create, insert, update or delete statement, uses executeUpdate(sql);
    for select query, you uses executeQuery(sql)

    1: Example of PreparedStatement for update a record into a table



    import java.sql.DriverManager;
    import java.sql.Connection;
    import java.sql.PreparedStatement;
    import java.sql.SQLException;

    public class PreparedStatementUpdateDemo {
    private static final String DB_DRIVER_TEST = "oracle.jdbc.driver.OracleDriver";
    private static final String DB_CONNECTION_TEST = "jdbc:oracle:thin:@localhost:1521:mytest";
    private static final String DB_USER_TEST = "user";
    private static final String DB_PASSWORD_TEST = "password";
    public static void main(String[] args) {
    try {
    updateRecordToTable();
    } catch (SQLException e) {
    System.out.println(e.getMessage());
    }
    }
    private static void updateRecordToTable() throws SQLException {
    Connection dbConnection = null;
    PreparedStatement preparedStatement = null;
    String updateTableSQL = "UPDATE testuser SET USERNAME = ? "+ " WHERE USER_ID = ?";
    try {
    dbConnection = getDBConnection();
    preparedStatement = dbConnection.prepareStatement(updateTableSQL);
    preparedStatement.setString(1, "ahmad_new_value");
    preparedStatement.setInt(2, 1004);
    // execute update SQL stetement here
    preparedStatement.executeUpdate();
    System.out.println("Record is updated in testuser table");
    } catch (SQLException e) {
    System.out.println(e.getMessage());
    } finally {
    if (preparedStatement != null) {
    preparedStatement.close();
    }
    if (dbConnection != null) {
    dbConnection.close();
    }
    }
    }
    private static Connection getDBConnection() {
    Connection dbConnection = null;
    try {
    Class.forName(DB_DRIVER_TEST);
    } catch (ClassNotFoundException e) {
    System.out.println(e.getMessage());
    }
    try {
    dbConnection = DriverManager.getConnection(DB_CONNECTION_TEST, DB_USER_TEST,DB_PASSWORD_TEST);
    return dbConnection;
    } catch (SQLException e) {
    System.out.println(e.getMessage());
    }
    return dbConnection;
    }
    }
    After running the code you can view in testuser table one record is updated.
    The Result is :
    The username of user_id = 1004 is updated to a new value 'ahmad_new_value'.

    You can insert/delete records in testuser table similarly.

    2:Example of PreparedStatement to select a list of records from a table



    import java.sql.DriverManager;
    import java.sql.Connection;
    import java.sql.PreparedStatement;
    import java.sql.ResultSet;
    import java.sql.SQLException;

    public class PreparedStatementSelectDemo{

    private static final String DB_DRIVER_TEST = "oracle.jdbc.driver.OracleDriver";
    private static final String DB_CONNECTION_TEST = "jdbc:oracle:thin:@localhost:1521:mytest";
    private static final String DB_USER_TEST = "user";
    private static final String DB_PASSWORD_TEST = "password";

    public static void main(String[] args) {
    try {
    recordsFromTable();
    } catch (SQLException e) {
    System.out.println(e.getMessage());
    }
    }

    private static void recordsFromTable() throws SQLException {
    Connection dbConnection = null;
    PreparedStatement preparedStatement = null;
    String selectSQL = "SELECT USER_ID, USERNAME FROM testuser WHERE USER_ID = ?";
    try {
    dbConnection = getDBConnection();
    preparedStatement = dbConnection.prepareStatement(selectSQL);
    preparedStatement.setInt(1, 1004);
    // execute select SQL stetement here
    ResultSet rs = preparedStatement.executeQuery();
    while (rs.next()) {
    String userid = rs.getString("USER_ID");
    String username = rs.getString("USERNAME");
    System.out.println("userid : " + userid);
    System.out.println("username : " + username);
    }
    } catch (SQLException e) {
    System.out.println(e.getMessage());
    } finally {
    if (preparedStatement != null) {
    preparedStatement.close();
    }
    if (dbConnection != null) {
    dbConnection.close();
    }
    }
    }
    private static Connection getDBConnection() {
    Connection dbConnection = null;
    try {
    Class.forName(DB_DRIVER_TEST);
    } catch (ClassNotFoundException e) {
    System.out.println(e.getMessage());
    }
    try {
    dbConnection = DriverManager.getConnection(DB_CONNECTION_TEST, DB_USER_TEST,DB_PASSWORD_TEST);
    return dbConnection;
    } catch (SQLException e) {
    System.out.println(e.getMessage());
    }
    return dbConnection;
    }
    }

    Result after running the code
    List of the records with user_id = 1004 are retrieved from table testuser and displayed.

    3 : Example of Insert records in batch process, via JDBC PreparedStatement



    import java.sql.DriverManager;
    import java.sql.Connection;
    import java.sql.PreparedStatement;
    import java.sql.SQLException;
    public class BatchUpdateDemo {
    private static final String DB_DRIVER_TEST = "oracle.jdbc.driver.OracleDriver";
    private static final String DB_CONNECTION_TEST = "jdbc:oracle:thin:@localhost:1521:mytest";
    private static final String DB_USER_TEST = "user";
    private static final String DB_PASSWORD_TEST = "password";
    public static void main(String[] args) {
    try {
    batchInsertRecordsIntoTable();
    } catch (SQLException e) {
    System.out.println(e.getMessage());
    }
    }

    private static void batchInsertRecordsIntoTable() throws SQLException {
    Connection dbConnection = null;
    PreparedStatement preparedStatement = null;
    String insertTableSQL = "INSERT INTO testuser"+ "(USER_ID, USERNAME, CREATED_BY, CREATED_DATE) VALUES"+ "(?,?,?,?)";
    try {
    dbConnection = getDBConnection();
    preparedStatement = dbConnection.prepareStatement(insertTableSQL);
    dbConnection.setAutoCommit(false);
    preparedStatement.setInt(1, 101);
    preparedStatement.setString(2, "wakil101");
    preparedStatement.setString(3, "system");
    preparedStatement.setTimestamp(4, getCurrentTimeStamp());
    preparedStatement.addBatch();

    preparedStatement.setInt(1, 102);
    preparedStatement.setString(2, "wakil102");
    preparedStatement.setString(3, "system");
    preparedStatement.setTimestamp(4, getCurrentTimeStamp());
    preparedStatement.addBatch();

    preparedStatement.setInt(1, 103);
    preparedStatement.setString(2, "wakil103");
    preparedStatement.setString(3, "system");
    preparedStatement.setTimestamp(4, getCurrentTimeStamp());
    preparedStatement.addBatch();

    preparedStatement.executeBatch();

    dbConnection.commit();
    System.out.println("Record is inserted into testuser table");
    } catch (SQLException e) {
    System.out.println(e.getMessage());
    dbConnection.rollback();
    } finally {
    if (preparedStatement != null) {
    preparedStatement.close();
    }
    if (dbConnection != null) {
    dbConnection.close();
    }
    }
    }
    private static Connection getDBConnection() {
    Connection dbConnection = null;
    try {
    Class.forName(DB_DRIVER_TEST);

    } catch (ClassNotFoundException e) {
    System.out.println(e.getMessage());
    }
    try {
    dbConnection = DriverManager.getConnection(DB_CONNECTION_TEST, DB_USER_TEST,DB_PASSWORD_TEST);
    return dbConnection;
    } catch (SQLException e) {
    System.out.println(e.getMessage());
    }
    return dbConnection;
    }
    private static java.sql.Timestamp getCurrentTimeStamp() {
    java.util.Date today = new java.util.Date();
    return new java.sql.Timestamp(today.getTime());
    }
    }
    After running code the Result are :
    3 records are inserted into database via batch update process.


    JDBC and Stored Procedure


    1 : Example of Stored procedure IN parameter via JDBC CallableStatement.


    The code will show you how to pass IN parameter in Stored procedure and call it via jdbc CallableStatement
    First create a stored procedure in oracle database as below :


    CREATE OR REPLACE PROCEDURE insertTestuser(
    p_userid IN testuser.USER_ID%TYPE,
    p_username IN testuser.USERNAME%TYPE,
    p_createdby IN testuser.CREATED_BY%TYPE,
    p_date IN testuser.CREATED_DATE%TYPE)
    IS
    BEGIN
    INSERT INTO testuser ("USER_ID", "USERNAME", "CREATED_BY", "CREATED_DATE")
    VALUES (p_userid, p_username,p_createdby, p_date);
    COMMIT;
    END;
    /

    Second Calls Stored Procedure via CallableStatement
    import java.sql.CallableStatement;
    import java.sql.DriverManager;
    import java.sql.Connection;
    import java.sql.SQLException;
    public class CallableStatementINParameterDemo {
    private static final String DB_DRIVER_TEST = "oracle.jdbc.driver.OracleDriver";
    private static final String DB_CONNECTION_TEST = "jdbc:oracle:thin:@localhost:1521:mytest";
    private static final String DB_USER_TEST = "user";
    private static final String DB_PASSWORD_TEST = "password";
    public static void main(String[] args) {
    try {
    callStoredProcINParameter();
    } catch (SQLException e) {
    System.out.println(e.getMessage());
    }
    }

    private static void callStoredProcINParameter() throws SQLException {
    Connection dbConnection = null;
    CallableStatement callableStatement = null;
    String insertStoreProc = "{call insertTestuser(?,?,?,?)}";

    try {
    dbConnection = getDBConnection();
    callableStatement = dbConnection.prepareCall(insertStoreProc);

    callableStatement.setInt(1, 1000);
    callableStatement.setString(2, "wakil");
    callableStatement.setString(3, "system");
    callableStatement.setDate(4, getCurrentDate());
    // execute insertDBUSER store procedure here
    callableStatement.executeUpdate();
    System.out.println("Record is inserted into testuser table
    } catch (SQLException e) {
    System.out.println(e.getMessage());
    } finally {
    if (callableStatement != null) {
    callableStatement.close();
    }
    if (dbConnection != null) {
    dbConnection.close();
    }
    }
    }
    private static Connection getDBConnection() {
    Connection dbConnection = null;
    try {
    Class.forName(DB_DRIVER_TEST);
    } catch (ClassNotFoundException e) {
    System.out.println(e.getMessage());
    }
    try {
    dbConnection = DriverManager.getConnection(DB_CONNECTION_TEST, DB_USER_TEST,DB_PASSWORD_TEST);
    return dbConnection;
    } catch (SQLException e) {
    System.out.println(e.getMessage());
    }
    return dbConnection;
    }
    private static java.sql.Date getCurrentDate() {
    java.util.Date today = new java.util.Date();
    return new java.sql.Date(today.getTime());
    }
    }
    Result is : When above example is executed, a new record will be inserted into database via stored procedure.

    2 : Example of Stored procedure OUT parameter via JDBC CallableStatement.


    The code will show you how to pass OUT parameter in Stored procedure and call it via jdbc CallableStatement

    First create a Stored Procedure with IN and OUT parameters. Later, calls it via JDBC.

    CREATE OR REPLACE PROCEDURE getTestuserByid(
    p_userid IN DBUSER.USER_ID%TYPE,
    o_username OUT DBUSER.USERNAME%TYPE,
    o_createdby OUT DBUSER.CREATED_BY%TYPE,
    o_date OUT DBUSER.CREATED_DATE%TYPE)
    IS
    BEGIN
    SELECT USERNAME , CREATED_BY, CREATED_DATE
    INTO o_username, o_createdby, o_date
    FROM testuser WHERE USER_ID = p_userid;
    END;
    /

    Second Call it via JDBC CallableStatement

    import java.sql.CallableStatement;
    import java.sql.Date;
    import java.sql.DriverManager;
    import java.sql.Connection;
    import java.sql.SQLException;

    public class CallableStatementOUTParameterDemo{
    private static final String DB_DRIVER_TEST = "oracle.jdbc.driver.OracleDriver";
    private static final String DB_CONNECTION_TEST = "jdbc:oracle:thin:@localhost:1521:mytest";
    private static final String DB_USER_TEST = "user";
    private static final String DB_PASSWORD_TEST = "password";
    public static void main(String[] args) {
    try {
    callStoredProcOUTParameter();
    } catch (SQLException e) {
    System.out.println(e.getMessage());
    }
    }

    private static void callStoredProcOUTParameter() throws SQLException {
    Connection dbConnection = null;
    CallableStatement callableStatement = null;
    String getTestuserByid = "{call getTestuserByid(?,?,?,?)}";
    try {
    dbConnection = getDBConnection();
    callableStatement = dbConnection.prepareCall(getTestuserByid);
    callableStatement.setInt(1, 10);
    callableStatement.registerOutParameter(2, java.sql.Types.VARCHAR);
    callableStatement.registerOutParameter(3, java.sql.Types.VARCHAR);
    callableStatement.registerOutParameter(4, java.sql.Types.DATE);
    // execute getTestuserByid store procedure here
    callableStatement.executeUpdate();
    String userName = callableStatement.getString(2);
    String createdBy = callableStatement.getString(3);
    Date createdDate = callableStatement.getDate(4);
    System.out.println("UserName : " + userName);
    System.out.println("CreatedBy : " + createdBy);
    System.out.println("CreatedDate : " + createdDate);
    } catch (SQLException e) {
    System.out.println(e.getMessage());
    } finally {
    if (callableStatement != null) {
    callableStatement.close();
    }
    if (dbConnection != null) {
    dbConnection.close();
    }
    }
    }
    private static Connection getDBConnection() {
    Connection dbConnection = null;
    try {
    Class.forName(DB_DRIVER_TEST);
    } catch (ClassNotFoundException e) {
    System.out.println(e.getMessage());
    }
    try {
    dbConnection = DriverManager.getConnection(DB_CONNECTION_TEST, DB_USER_TEST,DB_PASSWORD_TEST);
    return dbConnection;
    } catch (SQLException e) {
    System.out.println(e.getMessage());
    }
    return dbConnection;
    }
    }


    JDBC CallableStatement CURSOR



    3 : Example of Stored procedure INOUT parameter via JDBC CallableStatement


    JDBC example to call above stored procedure, cast the returned CURSOR to ResultSet and
    loop through the records sequentially.

    First Create a StoredProcedure

    CREATE OR REPLACE PROCEDURE getTestuserCursor(
    p_username IN DBUSER.USERNAME%TYPE,
    c_dbuser OUT SYS_REFCURSOR)
    IS
    BEGIN

    OPEN c_dbuser FOR
    SELECT * FROM testuser WHERE USERNAME LIKE p_username || '%';

    END;
    /

    import java.sql.CallableStatement;
    import java.sql.Date;
    import java.sql.DriverManager;
    import java.sql.Connection;
    import java.sql.ResultSet;
    import java.sql.SQLException;
    import oracle.jdbc.OracleTypes;
    public class CallableStatementCURSORDemo {
    private static final String DB_DRIVER_TEST = "oracle.jdbc.driver.OracleDriver";
    private static final String DB_CONNECTION_TEST = "jdbc:oracle:thin:@localhost:1521:mytest";
    private static final String DB_USER_TEST = "user";
    private static final String DB_PASSWORD_TEST = "password";
    public static void main(String[] args) {
    try {
    callStoredProcCURSORParameter();
    } catch (SQLException e) {
    System.out.println(e.getMessage());
    }
    }

    private static void callStoredProcCURSORParameter() throws SQLException {
    Connection dbConnection = null;
    CallableStatement callableStatement = null;
    ResultSet rs = null;
    String getTestuserCursorSql = "{call getTestuserCursor(?,?)}";

    try {
    dbConnection = getDBConnection();
    callableStatement = dbConnection.prepareCall(getTestuserCursorSql);
    callableStatement.setString(1, "wakil"); callableStatement.registerOutParameter(2, OracleTypes.CURSOR);
    // execute getTestuserCursor store procedure here
    callableStatement.executeUpdate();
    // get cursor and cast it to ResultSet
    rs = (ResultSet) callableStatement.getObject(2);
    while (rs.next()) {
    String userid = rs.getString("USER_ID");
    String userName = rs.getString("USERNAME");
    String createdBy = rs.getString("CREATED_BY");
    String createdDate = rs.getString("CREATED_DATE");
    System.out.println("UserName : " + userid);
    System.out.println("UserName : " + userName);
    System.out.println("CreatedBy : " + createdBy);
    System.out.println("CreatedDate : " + createdDate);
    }
    } catch (SQLException e) {
    System.out.println(e.getMessage());
    } finally {
    if (rs != null) {
    rs.close();
    }
    if (callableStatement != null) {
    callableStatement.close();
    }
    if (dbConnection != null) {
    dbConnection.close();
    }
    }
    }
    private static Connection getDBConnection() {
    Connection dbConnection = null;
    try {
    Class.forName(DB_DRIVER_TEST);

    } catch (ClassNotFoundException e) {

    System.out.println(e.getMessage());
    } try {
    dbConnection = DriverManager.getConnection(DB_CONNECTION_TEST, DB_USER_TEST,DB_PASSWORD_TEST);
    return dbConnection;
    } catch (SQLException e) {
    System.out.println(e.getMessage());
    }
    return dbConnection;
    }
    }

    4 : JDBC Transaction CURSOR



    import java.sql.DriverManager;
    import java.sql.Connection;
    import java.sql.PreparedStatement;
    import java.sql.SQLException;
    public class TransactionDemo {
    private static final String DB_DRIVER_TEST = "oracle.jdbc.driver.OracleDriver";
    private static final String DB_CONNECTION_TEST = "jdbc:oracle:thin:@localhost:1521:mytest";
    private static final String DB_USER_TEST = "user";
    private static final String DB_PASSWORD_TEST = "password";

    public static void main(String[] args) throws SQLException {
    Connection dbConnection = null;
    PreparedStatement preparedStatementInsert = null;
    PreparedStatement preparedStatementUpdate = null;
    String insertTableSQL = "INSERT INTO testuser"
    + "(USER_ID, USERNAME, CREATED_BY, CREATED_DATE) VALUES"
    + "(?,?,?,?)";
    String updateTableSQL = "UPDATE testuser SET USERNAME =? "+ "WHERE USER_ID = ?";
    try {
    dbConnection = getDBConnection();
    dbConnection.setAutoCommit(false);
    preparedStatementInsert = dbConnection.prepareStatement(insertTableSQL);
    preparedStatementInsert.setInt(1, 999);
    preparedStatementInsert.setString(2, "wakil101");
    preparedStatementInsert.setString(3, "system");
    preparedStatementInsert.setTimestamp(4, getCurrentTimeStamp());
    preparedStatementInsert.executeUpdate();

    preparedStatementUpdate = dbConnection.prepareStatement(updateTableSQL);
    preparedStatementUpdate.setString(1, "new string");
    preparedStatementUpdate.setInt(2, 999);
    preparedStatementUpdate.executeUpdate();
    dbConnection.commit();
    } catch (SQLException e) {
    System.out.println(e.getMessage());
    dbConnection.rollback();
    } finally {
    if (preparedStatementInsert != null) {
    preparedStatementInsert.close();
    }
    if (preparedStatementUpdate != null) {
    preparedStatementUpdate.close();
    }
    if (dbConnection != null) {
    dbConnection.close();
    }
    }
    }
    private static Connection getDBConnection() {
    Connection dbConnection = null;
    try {
    Class.forName(DB_DRIVER_TEST);
    } catch (ClassNotFoundException e) {
    System.out.println(e.getMessage());
    }
    try {
    dbConnection = DriverManager.getConnection(DB_CONNECTION_TEST, DB_USER_TEST,DB_PASSWORD_TEST);
    return dbConnection;
    } catch (SQLException e) {
    System.out.println(e.getMessage());
    }
    return dbConnection;
    } private static java.sql.Timestamp getCurrentTimeStamp() {
    java.util.Date today = new java.util.Date();
    return new java.sql.Timestamp(today.getTime());
    }
    }

    JDBC & Statement

    JDBC and Statement

    Java database connectivity is API which enable Java application to interact with a database. In this post we provides many step by step examples on using JDBC Statement, PreparedStatement ,CallableStatement ,JDBC Transaction and how to integrate with other frameworks for example spring.

    1 : JDBC with Mysql database


    import java.sql.DriverManager;
    import java.sql.Connection;
    import java.sql.SQLException;
    public class MyJDBCExample{
    public static void main(String[] argv) {
    System.out.println("--MySQL JDBC Connection demo --");
    try {
    Class.forName("com.mysql.jdbc.Driver");
    } catch (ClassNotFoundException e) {
    System.out.println("Ops ! Driver is not found, please provide a appropriate driver");
    e.printStackTrace();
    return;
    } Connection connection = null;
    try {
    connection = DriverManager.getConnection("jdbc:mysql://localhost:3306/test","root", "password");
    } catch (SQLException e) {
    System.out.println("Connection Failed! Check output console");
    e.printStackTrace();
    return;
    }
    if (connection != null) {
    System.out.println("Your database is connected");
    } else {
    System.out.println("Database connection failed please check.");
    }
    }
    }
    Before you run the application you must have mysql connector in your java lib path.
    you can download the mysql driver from here: Mysql Driver
    To Run it
    suppose your MyJDBCExample file is located in c:\test with java driver
    Go to test folder and compile and run as
    Compile
    C:\test>javac -cp c:\test\mysql-connector-java-5.1.8-bin.jar;c:\test MyJDBCExample.java
    Run
    C:\test>java -cp c:\test\mysql-connector-java-5.1.8-bin.jar;c:\test MyJDBCExample
    The output is :
    --MySQL JDBC Connection demo --
    Your database is connected

    2 : JDBC with Oracle database


    First you need to download the Oracle driver, you can download from here :Oracle Driver
    For Example :
    import java.sql.DriverManager;
    import java.sql.Connection;
    import java.sql.SQLException;
    public class MyOracleJDBC {
    public static void main(String[] args) {
    System.out.println("--Oracle JDBC Connection Demo--");
    try {
    Class.forName("oracle.jdbc.driver.OracleDriver");
    } catch (ClassNotFoundException e) {
    System.out.println("Ops ! No jdbc driver found, please provide the appropriate driver");
    e.printStackTrace();
    return;
    } Connection connection = null;
    try {
    connection = DriverManager.getConnection("jdbc:oracle:thin:@localhost:1521:mytest", "username","password");
    } catch (SQLException e) {
    System.out.println("Connection Failed");
    e.printStackTrace();
    return;
    }
    if (connection != null) {
    System.out.println("Oracle Database connected");
    } else {
    System.out.println("Connection Failed");
    }
    }
    }
    To run it
    C:\test>javac MyOracleJDBC.java
    C:\test>java -cp c:\test\ojdbc6.jar;c:\test MyOracleJDBC
    Output
    --Oracle JDBC Connection Demo--
    Oracle Database connected

    3 : Jdbc and Statement,PreparedStatement,CallableStatement


    The Statement interface is used to execute a simple SQL statement with no parameters.
    The create, insert, update or delete statement uses executeUpdate(sql) and select query, uses executeQuery(sql).

    3.1 : Jdbc and Satement


    Here Statement is used to insert a record in a table, for example
    import java.sql.DriverManager;
    import java.sql.Connection;
    import java.sql.SQLException;
    import java.sql.Statement;
    import java.text.DateFormat;
    import java.text.SimpleDateFormat;
    public class JDBCStatementInsertDemo {
    private static final String DB_DRIVER_TEST = "oracle.jdbc.driver.OracleDriver";
    private static final String DB_CONNECTION_TEST = "jdbc:oracle:thin:@localhost:1521:mytest";
    private static final String DB_USER_TEST = "user";
    private static final String DB_PASSWORD_TEST = "password";
    private static final DateFormat dateFormat = new SimpleDateFormat("yyyy/MM/dd HH:mm:ss");
    public static void main(String[] args) {
    try {
    insertRecordIntoTable();
    } catch (SQLException e) {
    System.out.println(e.getMessage());
    }
    }
    private static void insertRecordIntoTable() throws SQLException {
    Connection dbConnection = null;
    Statement statement = null;
    String insertTableSQL = "INSERT INTO testuser" + "(USER_ID, USERNAME, CREATED_BY, CREATED_DATE) " + "VALUES" + "(1,'wakil','system', " + "to_date('" + getCurrentTimeStamp() + "', 'yyyy/mm/dd hh24:mi:ss'))";
    try {
    dbConnection = getDBConnection();
    statement = dbConnection.createStatement();
    // execute insert SQL stetement here
    statement.executeUpdate(insertTableSQL);
    System.out.println("Record is inserted into testuser table");
    } catch (SQLException e) {
    System.out.println(e.getMessage());
    } finally {
    if (statement != null) {
    statement.close();
    }
    if (dbConnection != null) {
    dbConnection.close();
    }
    }
    }
    private static Connection getDBConnection() {
    Connection dbConnection = null;
    try {
    Class.forName(DB_DRIVER_TEST);
    } catch (ClassNotFoundException e) {
    System.out.println(e.getMessage());
    }
    try {
    dbConnection = DriverManager.getConnection(DB_CONNECTION_TEST, DB_USER_TEST,DB_PASSWORD_TEST);
    return dbConnection;
    } catch (SQLException e) {
    System.out.println(e.getMessage());
    }
    return dbConnection;
    }
    private static String getCurrentTimeStamp() {
    java.util.Date today = new java.util.Date();
    return dateFormat.format(today.getTime());
    }
    }
    Similarly you can use the Statement for updating/deleting records via JDBC and Statement

    3.2 : Jdbc and Satement for select


    I will show you how to select the entire records from table via JDBC statement,
    and display all the records via a ResultSet object.
    import java.sql.DriverManager;
    import java.sql.Connection;
    import java.sql.ResultSet;
    import java.sql.SQLException;
    import java.sql.Statement;
    public class JDBCStatementSelectDemo {
    private static final String DB_DRIVER_TEST = "oracle.jdbc.driver.OracleDriver";
    private static final String DB_CONNECTION_TEST = "jdbc:oracle:thin:@localhost:1521:mytest";
    private static final String DB_USER_TEST = "user";
    private static final String DB_PASSWORD_TEST = "password";
    public static void main(String[] args) {
    try {
    selectRecordsFromTable();
    } catch (SQLException e) {
    System.out.println(e.getMessage());
    }
    }
    private static void selectRecordsFromTable() throws SQLException {
    Connection dbConnection = null;
    Statement statement = null;
    String selectTableSQL = "SELECT USER_ID, USERNAME from testuser";
    try {
    dbConnection = getDBConnection();
    statement = dbConnection.createStatement();
    // execute select SQL stetement here
    ResultSet rs = statement.executeQuery(selectTableSQL);
    while (rs.next()) {
    String userid = rs.getString("USER_ID");
    String username = rs.getString("USERNAME");
    System.out.println("userid : " + userid);
    System.out.println("username : " + username);
    }
    } catch (SQLException e) {
    System.out.println(e.getMessage());
    } finally {
    if (statement != null) {
    statement.close();
    }
    if (dbConnection != null) {
    dbConnection.close();
    }
    }
    }
    private static Connection getDBConnection() {
    Connection dbConnection = null;
    try {
    Class.forName(DB_DRIVER_TEST);
    } catch (ClassNotFoundException e) {
    System.out.println(e.getMessage());
    }
    try {
    dbConnection = DriverManager.getConnection(DB_CONNECTION_TEST, DB_USER_TEST,DB_PASSWORD_TEST);
    return dbConnection;
    } catch (SQLException e) {
    System.out.println(e.getMessage());
    }
    return dbConnection;
    }
    }
    Here The Result are :
    A list of the records are retrieved from table testuser table and displayed.

    3.3 : Jdbc and Satement for Batch



    JDBC and Statement
    My Example to insert records in batch process, via JDBC Statement.
    Note Batch Update is not limited to Insert statement, it can be apply for Update/Delete statement as well.
    import java.sql.DriverManager;
    import java.sql.Connection;
    import java.sql.SQLException;
    import java.sql.Statement;
    import java.text.DateFormat;
    import java.text.SimpleDateFormat;

    public class JDBCBatchUpdateDemo {
    private static final String DB_DRIVER_TEST = "oracle.jdbc.driver.OracleDriver";
    private static final String DB_CONNECTION_TEST = "jdbc:oracle:thin:@localhost:1521:mytest";
    private static final String DB_USER_TEST = "user";
    private static final String DB_PASSWORD_TEST = "password";
    private static final DateFormat dateFormat = new SimpleDateFormat("yyyy/MM/dd HH:mm:ss");
    public static void main(String[] args) {
    try {
    batchInsertRecordsIntoTable();
    } catch (SQLException e) {
    System.out.println(e.getMessage());
    }
    }
    private static void batchInsertRecordsIntoTable() throws SQLException {
    Connection dbConnection = null;
    Statement statement = null;
    String insertTableSQL1 = "INSERT INTO testuser" + "(USER_ID, USERNAME, CREATED_BY, CREATED_DATE) " + "VALUES" + "(101,'wakil','system', " + "to_date('" + getCurrentTimeStamp() + "', 'yyyy/mm/dd hh24:mi:ss'))";
    String insertTableSQL2 = "INSERT INTO testuser" + "(USER_ID, USERNAME, CREATED_BY, CREATED_DATE) " + "VALUES" + "(102,'wakil','system', " + "to_date('" + getCurrentTimeStamp() + "', 'yyyy/mm/dd hh24:mi:ss'))";
    String insertTableSQL3 = "INSERT INTO testuser" + "(USER_ID, USERNAME, CREATED_BY, CREATED_DATE) " + "VALUES" + "(103,'wakil','system', " + "to_date('" + getCurrentTimeStamp() + "', 'yyyy/mm/dd hh24:mi:ss'))";
    try {
    dbConnection = getDBConnection();
    statement = dbConnection.createStatement();
    dbConnection.setAutoCommit(false);
    statement.addBatch(insertTableSQL1);
    statement.addBatch(insertTableSQL2);
    statement.addBatch(insertTableSQL3);
    statement.executeBatch();
    dbConnection.commit();
    System.out.println("Records are inserted into testuser table");
    } catch (SQLException e) {
    System.out.println(e.getMessage());
    } finally {
    if (statement != null) {
    statement.close();
    }
    if (dbConnection != null) {
    dbConnection.close();
    }
    }
    }
    private static Connection getDBConnection() {
    Connection dbConnection = null;
    try {
    Class.forName(DB_DRIVER_TEST);
    } catch (ClassNotFoundException e) {
    System.out.println(e.getMessage());
    }
    try {
    dbConnection = DriverManager.getConnection(DB_CONNECTION_TEST, DB_USER_TEST,DB_PASSWORD_TEST);
    return dbConnection;

    } catch (SQLException e) {
    System.out.println(e.getMessage());
    }
    return dbConnection;
    }
    private static String getCurrentTimeStamp() {
    java.util.Date today = new java.util.Date();
    return dateFormat.format(today.getTime());
    }
    }
    Here the result are :
    3 records are inserted into database via batch update process.

    i.e. JDBC and Statement

    Thursday, 18 April 2013

    Installation and Configuration of HBase


     HBase is an open-source, distributed, versioned, column-oriented store modeled.
    
     A distributed HBase depends on a running ZooKeeper cluster. All participating nodes and clients need to be able to get to the running ZooKeeper cluster. HBase by default manages a ZooKeeper cluster for you, or you can manage it on your own and point HBase to it. In our case, we are using default ZooKeeper cluster, which is manage by Hbase Following
    are the capacities in which nodes may act in our cluster:
    1.Hbase Master:- The HbaseMaster is responsible for assigning regions to HbaseRegionserver, 
    monitors the health of each HbaseRegionserver.
    2.Zookeeper: - For any distributed application, ZooKeeper is a centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services.
    3.Hbase Regionserver:- The HbaseRegionserver is responsible for handling client read and write requests. It communicates with the Hbasemaster to get a list of regions to serve and to tell the master that it is alive.In our case, one machine in the cluster is designated as Hbase master and Zookeeper. The rest of machine in the cluster act as a Regionserver.
     Before installation of hbase you must have to installed the hadoop in cluster.
    Hbase run in three modes, Standalone, Pseudo-Distributed and Fully-Distributed.
    We only gives the Fully-Distributed mode.
    There are following steps for installing and configuring the hbase as
    Step 1 : Download the stable version for hbase from http://www.apache.org/dyn/closer.cgi/hbase/
    suppose it is donloaded into the /usr/local/ directory. Suppose you have downloaded as hbase-0.90.5.tar.gz
    Step 2 : untar the downloaded file
    # cd /usr/local/
    # tar xvfz hbase-0.90.5.tar.gz
    Step 3 :  copy the hbase-0.90.5 to all RegionServer machines
    Step 4 : Edit the /etc/hosts on master-hbase machine and add the following lines
      192.168.0.50  master-namenode     hbase-master            
     192.168.0.51     datanode-1     regionserver-1                  
     192.168.0.52     datanode-2     regionserver-2                   
     192.168.0.53     datanode-3     regionserver-3                    
     192.168.0.54     datanode-3     regionserver-4
    
    where hbase-master and master-namenode is configured on same machine and
    datanode-1,regionserver-1 is configured on machine 192.168.0.51 and so on.
    Step 5 : We have needed to configure password less login from hbase-master to all 
    regionservers.
    Execute the following commands on hbase-master machine
    # ssh-keygen -t rsa
    # ssh-copy-id -i ~/.ssh/id_rsa.pub  hadoop@regionserver-1
    # ssh-copy-id -i ~/.ssh/id_rsa.pub  hadoop@regionserver-2
    # ssh-copy-id -i ~/.ssh/id_rsa.pub  hadoop@regionserver-3
    # ssh-copy-id -i ~/.ssh/id_rsa.pub  hadoop@regionserver-4
    Step 6 : Open the file /usr/local/hbase-0.90.5/conf/hbase-env.sh and set the 
    JAVA_HOME as
    export JAVA_HOME=/user/java/java-1.6.x
    Step 7 :Open the file /usr/local/hbase-0.90.5/conf/hbase-site.xml and add the
    following lines in configuration element.
    <property>
        <name>hbase.rootdir</name>
        <value>hdfs://java-rnd-server:9000/hbase</value>
      </property>
      <property>
        <name>dfs.replication</name>
        <value>1</value>
      </property>
      <property>
      <name>hbase.cluster.distributed</name>
     <value>true</value>
      </property>
      <property>
        <name>hbase.master</name>
        <value>hbase-master:60000</value>
      </property>
      <property>
        <name>hbase.zookeeper.property.clientPort</name>
        <value>2222</value>
      </property>
      <property>
        <name>hbase.zookeeper.quorum</name>
        <value>hbase-master</value>
      </property>
      <property>
        <name>hbase.zookeeper.property.dataDir</name>
        <value>/usr/local/zookeeper/hbase</value>
      </property>
    Step 8 : Open the file /usr/local/hbase-0.90.5/conf/hbase-env.sh and uncomment the following line:
    export HBASE_MANAGES_ZK=true
    Step 9 : Open the file
    /usr/local/hbase-0.90.5/conf/regionservers
    and add all the regionserver machine names.
    regionserver-1
    regionserver-2
    regionserver-3
    regionserver-4
    Step 10 : Edit the file /etc/hosts on the regionserver-1..4 machines and add the following lines.
    192.168.41.50 master-namenode hbase-master
    Step 11 : We have needed to configure password less login from regionserver-1..4 to hbase-master machine.
    Execute the following commands from all regionservers to hbase-master as follows.
    # su - hadoop
    # ssh-keygen -t rsa
    # ssh-copy-id -i ~/.ssh/id_rsa.pub hadoop@hbase-master
    Step 12 : Repeat step 7,8 for all regionservers machines.
    Step 13 : go /usr/local/hbase-0.90.5/
    #bin/start-hbase.sh
    Step 14 : Starting the hbase shell
    #bin/hbase shell
    and you can access the web interface of hbase

    Install and Configure Hadoop on Linux


                   Configure and Install Hadoop on Linux
    
    The Apache Hadoop software library is a framework that allows for the distributed processing of large data sets across clusters of computers using a simple programming model. It is designed to scale up from single servers to thousands of machines, each offering local computation and storage. 
    
    Hadoop has mainly three parts
    1 : Hadoop Common :The common utilities that support the other Hadoop subprojects.  
    For example hbase,hive,cassendra, pig, zookeeper etc.
    2 : Hadoop Distributed File System(HDFS) :A distributed file system that provides high-throughput access to application data.
    3 : Hadoop MapReduce :A software framework for distributed processing of large data sets on compute clusters. 
    
    1. NameNode:-Manages the namespace, file system metadata, and access control. There is exactly one 
    NameNode in each cluster.
    2.SecondaryNameNode:-Downloads periodic checkpoints from the nameNode for fault-tolerance. There is exactly one SecondaryNameNode in each cluster.
    3.JobTracker: - Hands out tasks to the slave nodes. There is exactly one JobTracker in each cluster.
    4.DataNode: -Holds file system data. Each data node manages its own locally-attached storage (i.e., the node's hard disk) and stores a copy of some or all blocks in the file system. There are one or more DataNodes in each cluster.
    5.TaskTracker: - Slaves that carry out map and reduce tasks. There are one or more TaskTrackers in each cluster.
                            Installation

    Required Software :

    1 : java-1.6.x must be installed into your system and set the environment path as JAVA_HOME
    2 : ssh must be installed and sshd must be running to use the Hadoop scripts that manage remote Hadoop daemons.
    For windows operating system
    
    Cygwin- Required for shell support in windows
    
     --------------------------------------------------------------------------
    
    Download the hadoop stable version from this given link  http://hadoop.apache.org/common/releases.html 
    
    and click on download link, and download the stable version into your prefered
    directory, for example it is downloaded into here /usr/local/hadoop-0.20.2.tar.gz
    
    Go to the download directory and untar it with the following command
    
    # cd /usr/local/
    # tar xvfz hadoop-0.20.2.tar.gz
    # cd hadoop-0.20.2
    
    set the JAVA_HOME path in conf/hadoop-env.sh file as
    
    export JAVA_HOME=/usr/java/jdk1.6.0_18
    
    
    Now you are ready to start the hadoop cluster into one of three mode
    
     Local (Standalone) Mode
     Pseudo-Distributed Mode
     Fully-Distributed Mode
    
    Standalone Mode : By Default hadoop is configured to run in non-distributed mode,
    as a single java process. It is useful for debugging.
    
    For running in standalone you can easily test it as follows.
    
    # mkdir input
    # cp conf/*.xml input
    # bin/hadoop jar hadoop-examples-*.jar grep input output 'dfs[a-z.]+' 
    # cat output/*
    
    Pseudo-Distributed :Hadoop can also be run on a single-node in a pseudo-distributed mode where each Hadoop daemon runs in a separate Java process.
    
    If you would like to start the hadoop as a custom linux user for example hadoop
    
    you have to create group hadoop and user hadoop and give the permission for directories. 
    
    # goupadd hadoop
    # useradd -g hadoop hadoop
    # passwd hadoop
    
    # mkdir /usr/local/hadoop-data
    
    # chown -R hadoop:hadoop hadoop-data
    # chown -R hadoop:hadoop hadoop-0.20.2
    
    # su – hadoop
    
    # cd /usr/local
    
    
    Step 1 : edit the file conf/core-site.xml and add the following line within the
    configuration element.
         <property>
             <name>fs.default.name</name>
             <value>hdfs://localhost:9000</value>
         </property>
        <property>
         <name>hadoop.tmp.dir</name>
         <value>/usr/local/hadoop-data</value>
        </property>
    where hadoop-data is directory in /usr/local/, store all file system data
    Step 2 : edit the file conf/hdfs-site.xml and add the following line within the
    configuration element.
        <property>
             <name>dfs.replication</name>
             <value>1</value>
         </property>
    Step 3 : edit the file conf/mapred-site.xml and add the following line within the
    configuration element.
         <property>
             <name>mapred.job.tracker</name>
             <value>localhost:9001</value>
         </property>
    Step 4 : Setup passphraseless ssh If you cannot ssh to localhost without a passphrase, execute the following commands:
    # ssh-keygen -t dsa -P '' -f ~/.ssh/id_dsa 
    # cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
    or 
    sh-keygen -t dsa -P '' -f ~/.ssh/id_dsa 
    ssh-copy-id -i ~/.ssh/id_rsa.pub  slave-machines
    
    Step 5 : execute the following command
    # bin/hadoop namenode -format
    Step 6 : Start the hadoop daemons:
    # bin/start-all.sh
    after starting the above command, you can view the log from HADOOP_HOME/logs
    directory. The above .sh file started the NameNode and the JobTracker; by default they are available at:
    Fully-Distributed Mode :
    
    Suppose we want to set up hadoop in cluster mode we have to identified the machines for namenode and datanodes. For example
    
    suppose there are 5 machines in which one machine is namenode and other 4 
    are datanodes which have name and ip addresses are given here.
    
     MachineName                     IP Address
     -------------------------------------------------------
     master-namenode                 192.168.0.50
     datanode-1                      192.168.0.51
     datanode-2                      192.168.0.52   
     datanode-3                      192.168.0.53
     datanode-3                      192.168.0.54
    
     -------------------------------------------------------
    
    The following steps are given below for installing the hadoop in cluster environment.
    
    Step 1 : Install the pre-require softwares in all machines. Ie
    jdk-1.6.x,ssh etc.
    
    Step 2 : Download the hadoop from apache site. Suppose it is downloaded into
    /usr/local/, suppose the downloaded version is hadoop-0.20.2.tar.gz
    
    untar it as
    
    # tar xvfz hadoop-0.20.2.tar.gz
    
    Go to the directory 
    
    cd /usr/local/
    
    Step 2 : copy the hadoop-0.20.2 directory into /usr/local/ of all machines including master-namenode
    
    Step 3 : edit /etc/hosts and add the following ip's into this for all machines
    
    192.168.0.50 master-namenode
     192.168.0.51           datanode-1                      
     192.168.0.52           datanode-2                        
     192.168.0.53           datanode-3                      
     192.168.0.54           datanode-3                    
    
    
    Step 4 : create a user named hadoop in all machines and create a directory named
    
    hadoop-data  in /usr/local/ for data storage of hadoop i.e.
    
    execute the following command in all machines
    
    # groupadd hadoop
    # useradd -g hadoop hadoop
    # chown -R hadoop:hadoop /usr/local/hadoop-0.20.2
    # chown -R hadoop:hadoop /usr/local/hadoop-data
    
    Step 5 : Edit the config file – /usr/local/hadoop-0.20.2/conf/masters
    
    192.168.0.50 
    or 
    master-namenode
    
    Step 6 :Edit the config file – /usr/local/hadoop-0.20.2/conf/slaves
    
    datanode-1 
    datanode-2
    datanode-3
    datanode-4
    
    Note : every datanode server must be separate line.
    
    Step 7 : Edit the config file – /usr/local/hadoop-0.20.2/conf/core-site.xml and add the following attribute in configuration element.
    
    <property>
      <name>hadoop.tmp.dir</name>
      <value>/usr/local/hadoop-data</value>
    </property>
    <property>
      <name>fs.default.name</name>
      <value>hdfs:master-namenode:9000</value>
    </property>
    
    Step 8 : Edit the config file - /usr/local/hadoop-0.20.2/conf/hdfs-site.xml and add the following attribute in configuration element.
    
    <property>
    
        <name>dfs.replication</name>
    
        <value>4</value>
    
      </property>
    
      <property>
    
        <name>dfs.permissions</name>
    
        <value>true</value>
    
      </property>
    
    
    
     <property>
    
         <name>dfs.safemode.threshold.pct</name>
    
         <value>0</value>
    
     </property>
    
    
    Step 9 : Edit the config file – /usr/local/hadoop-0.20.2/conf/mapred-site.xml and 
    add the following lines in configuration element
    
      <property>
    
        <name>mapred.job.tracker</name>
    
        <value>master-namenode:9001</value>
    
      </property>
    
    
    repeat the steps from 7-9 in all datanode server too.
    
    Step 10 : make the passwordless ssh from master-namenode to datanode-1 ...4
    
    type the following command for making the passphraseless
    
     In master-namenode machine, type the following command
     # su – hadoop 
     
    # sh-keygen -t dsa -P '' -f ~/.ssh/id_dsa 
    # ssh-copy-id -i ~/.ssh/id_rsa.pub  hadoop@datanode-1
    # ssh-copy-id -i ~/.ssh/id_rsa.pub  hadoop@datanode-2
    # ssh-copy-id -i ~/.ssh/id_rsa.pub  hadoop@datanode-3
    # ssh-copy-id -i ~/.ssh/id_rsa.pub  hadoop@datanode-4
    Step 11 : check the passwordless ssh for all datanode servers from master-namenode to namenode-1..4
    
    ssh namenode-1
    ssh namenode-2
    ssh namenode-3
    ssh namenode-4
    if all will be connected without password, then it is fine.
    
    Step 12 : Format the namenode only on master-namenode machine
    # cd /usr/local/hadoop-0.20.2/
    # bin/hadoop namenode -format
    Step 13 : After that start the hadoop only on master-namenode machine
    it start namenode,jobtracker and datanode,tasktracker
    # bin/start-all.sh
    or
    # bin/start-dfs.sh    
    # bin/start-mapred.sh
    
    Step 13 : After starting the above command on master-namenode, you can check
    web interface for namenode as http://master-namenode:50070/
    web interface for jobtracher as http://master-namenode:50030/
    After starting you can work on it.
    For stopping all the datanode,tasktracker, and namenode and jobtracker, you can execute the following command on master
    # bin/stop-all.sh
    or 
    # bin/stop-dfs.sh    
    # bin/stop-mapred.sh
    

    
    
    
    
    
    
    
     
    
    
    
    
    
    

    Install and configure svn repository



    Install guide of SVN repository

    1 : Download the svn exe from the following url :

    http://subversion.tigris.org/files/documents/15/39559/svn-1.4.5-setup.exe

    2 : Run the installer and follow the installation instruction

    3 :Setup server as a Windows service using the following command on command line:

       sc create SvnServe binpath= "C:\Program Files\Subversion\bin\svnserve.exe --service --listen-port 3690 -r \"C:\SvnData\"" start= auto

    4 : Start service manually for the first time (or restart system):

        net start SvnServe
    5 : Create directory for your repositories:

       md  C:\SvnData

    6 : Create repository for your project:

      "C:\Program Files\Subversion\bin\svnadmin.exe" create C:\SvnData\MyProject
    7 :Edit repository configuration file: C:\SvnData\MyProject\conf\svnserve.conf and edit the file. You can set there the following parameters:
     
    anon-access - defines the access level of the anonymous users - the options are read|wite|none with default value read
    auth-access - defines the access level of the authorized users - the options are read|wite|none with default value write
    password-db - defines file name of the password database - the default is passwd file in the same directory as svnserve.conf
    authz-db - defines file name of the authorization database - the default is authz file in the same directory as svnserve.conf. This file let you create user groups and manage access to the projects in more detailed way. You will no need it with a simple access scenarios.
    realm - defines the database realm, which is by default unique to the project. Set this value to some common with other project if you want to authorize once for all the projects (if you do so the password-db parameter should be set to common user database file).
    Sample configuration file:
    [general]
    anon-access=none
    auth-access=write
    password-db=passwd
    realm=LocalSvnRealm

    SVN server and project repository are ready now. All you need is to use any client (I prefer TortoiseSVN available from Tigris.org
    but you can do it using command line tools of course) and initially checkout repository to your working directory - the repository
    URL is svn://localhost:3690/MyProject.


    8 : Edit password database file: C:\SvnData\MyProject\conf\passwd and optionally move the file to the target location if you changed default
    location in configuration file. The file structure is:
    [users]
    wakil = XXXXX
    ahamad = XXXXX

    9 : defines the groups user authorization in authz file

    [groups]
    # harry_and_sally = harry,sally
    admin=wakil, ahamad
    # [/foo/bar]
    # harry = rw
    # * =

    # [repository:/baz/fuz]
    # @harry_and_sally = rw
    # * = r
    [/]
    @admin = rw
    * = r


    10 : "C:\Program Files\Subversion\bin\svn.exe" checkout --username=wakil --password=wakil123 svn://localhost:3690/MyProject c:\WorkingCopy


    11 :

    There are a few more things:
    adding Subversion binaries directory to your PATH variable will help you working with SVN on everyday use
    it's good (but not mandatory) to create standard repository layout to keep your project data. So you should create the following directory structure in the project
    trunk - this directory will store working copy
    branches - this directory will store branch copies
    tags - this directory will store tag copies
    After you create those directories you can switch your working directory to svn://localhost:3690/MyProject/trunk and start development.
    md c:\WorkingCopy\trunk c:\WorkingCopy\branches c:\WorkingCopy\tags
    "C:\Program Files\Subversion\bin\svn.exe" add c:\WorkingCopy\trunk c:\WorkingCopy\branches c:\WorkingCopy\tags
    "C:\Program Files\Subversion\bin\svn.exe" commit --username=user1 --password=password1 --message="Directory structure created" c:\WorkingCopy
    "C:\Program Files\Subversion\bin\svn.exe" switch svn://localhost:3690/MyProject/trunk c:\WorkingCopy

    the default port for svnserve is 3690 so you can skip it in service creation command as well as in repository URL (it was included here for presentation purposes