Spark - Hive

1 - About

since Spark 2.0, Spark SQL supports builtin Hive features such as:

To use these features, you do not need to have an existing Hive setup ???


3 - Management

3.1 - Configuration

Configuration of Hive is done by placing your hive-site.xml, core-site.xml and hdfs-site.xml files in conf/. You may run ./bin/spark-sql --help for a complete list of all available options.

3.2 - Server

Sparks comes with a Hive Server 2 that you can start. See running-the-thrift-jdbcodbc-server

3.3 - Metastore

If the setting value of hive.metastore.warehouse.dir is null, it will get the value of spark.sql.warehouse.dir (ie workingDir/spark-warehouse/).

4 - Documentation / Reference

db/spark/sql/hive.txt ยท Last modified: 2018/06/12 17:42 by gerardnico