Spark SQL hive table error

Question

I am trying to get records from a hive table but when trying to make the selection it says that the table does not exist. The error say Table or view not found: clientes lines 1 pos 14

import java.io.File; import java.io.Serializable; import java.util.ArrayList; import java.util.List; import org.apache.spark.api.java.function.MapFunction; import org.apache.spark.sql.Dataset; import org.apache.spark.sql.Encoders; import org.apache.spark.sql.Row; import org.apache.spark.sql.SparkSession; public class Main { public static void main(String[] args) { String warehouseLocation = new File("spark-warehouse").getAbsolutePath(); SparkSession spark = SparkSession .builder() .appName("Java Spark Hive Example") .config("spark.sql.warehouse.dir", warehouseLocation) .enableHiveSupport() .getOrCreate(); Dataset<Row> df = spark.sql("select * from clientes"); df.show(); }

Is clientes the name of the table? Could it be a typo? Just making sure. — kRazzy R
– kRazzy R, Commented May 9, 2018 at 20:56

Yayati Sule · Accepted Answer · 2018-05-10 03:57:49Z

What you are trying to do is run a SQL on a pre-existing View. The View is created with the API df.createTempView("name_of_view"). To access the Hive table in question you need to use the table() API as follows : spark.table(String tablename) and you get the required Data frame. You can check before-hand if the table exists in Hives' database with the command spark.catalog.tableExists(String dbname,String tablename). You can refer to these APIs in the JavaDoc for Spark 2 Spark 2 JavaDoc

Collectives™ on Stack Overflow

Spark SQL hive table error

1 Answer 1

Comments

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Related