Tuesday, November 8, 2016

Setup Hive to run on Ubuntu 15.04

This is tested on hadoop-2.7.3, and apache-hive-2.1.0-bin.

Improvement on Hive documentation : https://cwiki.apache.org/confluence/display/Hive/GettingStarted

Step 1

Make sure Java is installed

Installation instruction : http://suhothayan.blogspot.com/2010/02/how-to-set-javahome-in-ubuntu.html

Step 2

Make sure Hadoop is installed & running

Instruction : http://suhothayan.blogspot.com/2016/11/setting-up-hadoop-to-run-on-single-node_8.html

Step3 

Add Hive and Hadoop home directories and paths

Run

$ gedit ~/.bashrc

Add flowing at the end (replace {hadoop path} and {hive path} with proper directory locations)

export HADOOP_HOME={hadoop path}/hadoop-2.7.3

export HIVE_HOME={hive path}/apache-hive-2.1.0-bin
export PATH=$HIVE_HOME/bin:$PATH

Run

$ source ~/.bashrc

Step4

Create /tmp and hive.metastore.warehouse.dir and set executable permission create tables in Hive. (replace {user-name} with system username)

hadoop-2.7.3/bin/hadoop fs -mkdir /tmp
$ hadoop-2.7.3/bin/hadoop fs -mkdir /user
$ hadoop-2.7.3/bin/hadoop fs -mkdir /user/{user-name}
$ hadoop-2.7.3/bin/hadoop fs -mkdir /user/{user-name}/warehouse
$ hadoop-2.7.3/bin/hadoop fs -chmod 777 /tmp
$ hadoop-2.7.3/bin/hadoop fs -chmod 777 /user/{user-name}/warehouse

Step5

Create hive-site.xml 

$ gedit apache-hive-2.1.0-bin/conf/hive-site.xml

Add following (replace {user-name} with system username):

<configuration>
  <property>
    <name>hive.metastore.warehouse.dir</name>
    <value>/user/{user name}/warehouse</value>
  </property>
</configuration>


Copy hive-jdbc-2.1.0-standalone.jar to lib

cp apache-hive-2.1.0-bin/jdbc/hive-jdbc-2.1.0-standalone.jar apache-hive-2.1.0-bin/lib/

Step6

Initialise Hive with Derby, run:

$ ./apache-hive-2.1.0-bin/bin/schematool -dbType derby -initSchema

Step7

Run Hiveserver2:

$ ./apache-hive-2.1.0-bin/bin/hiveserver2

View hiveserver2 logs: 

tail -f /tmp/{user name}/hive.log

Step8

Run Beeline on another terminal:

$ ./apache-hive-2.1.0-bin/bin/beeline -u jdbc:hive2://localhost:10000

Step9

Enable fully local mode execution: 

hive> SET mapreduce.framework.name=local;

Step10

Create table :

hive> CREATE TABLE pokes (foo INT, bar STRING);

Brows table 

hive> SHOW TABLES;

17 comments:

  1. A very nice guide. I will definitely follow these tips. Thank you for sharing such detailed article. I am learning a lot from you.

    Base SAS Training in Chennai

    MSBI Training in Chennai

    ReplyDelete
  2. A nice article here, i think that people who have grown up with the idea of using computers are showing more responsibility towards writing posts that are thoughtful, do not have grammar mistakes and pertinent to the post..

    Dot Net Training in Chennai

    Software Testing Training in Chennai

    ReplyDelete


  3. I have seen a lot of blogs and Info. on other Blogs and Web sites But in this Hadoop Blog Information is useful very thanks for sharing it........

    ReplyDelete
  4. I'm happy about everything you bring it very interesting and helpful, thanks

    http://word-cookies-answers.com

    ReplyDelete
  5. Thanks for one’s Good posting! I enjoyed reading it, I want to encourage you to definitely continue your great writing, have a nice evening! keeping this website.
    Packers And Movers In Jalandhar

    ReplyDelete
  6. Setting-up HIVE on Ubuntu is some thing which is very interesting and helpful as per the security purpose also it will be very effective and subsequent, As I being a PMP Certified and I have done my PMP Course in Mumbai with effective to certain projects and handled many projects, Thank you for the information definitely I will look forward for more Updates

    ReplyDelete
  7. Does your blog have a contact page? I’m having problems locating it but, I’d like to shoot you an email. I’ve got some recommendations for your blog you might be interested in hearing.
    AWS Training in Chennai |Best Amazon Web Services Training in Chennai
    AWS Training in Rajaji Nagar | Amazon Web Services Training in Rajaji Nagar
    Best AWS Training Institute in BTM Layout Bangalore ,AWS Coursesin BTM

    ReplyDelete
  8. Thanks for such a great article here. I was searching for something like this for quite a long time and at last, I’ve found it on your blog. It was definitely interesting for me to read about their market situation nowadays.AngularJS Training in Chennai | Best AngularJS Training Institute in Chennai | AngularJS Training in Velachery |
    Angular4 Training Institute in Chennai

    ReplyDelete
  9. Wow!! Really a nice Article. Thank you so much for your efforts. Definitely, it will be helpful for others. I would like to follow your blog. Share more like this. Thanks Again.
    iot training in Chennai | Best iot Training Institute in Chennai

    ReplyDelete
  10. I really like and appreciate your post.Thanks Again. Keep writing.

    ReplyDelete