Create a "temporary" table in spark-SQL, not spark-scala?

Question

I have loaded a table from an input file.

CREATE TABLE MyTable ( ID INT, VALUE FLOAT, RATE INT ... LOAD DATA LOCAL INPATH 'MYPATH' INTO TABLE MyTable;

Now I'd like to create a new based on this one

DerivedTable = SELECT ID, VALUE*RATE AS Total FROM MyTable WHERE VALUE IS NOT NULL;

Then I'm going to use this table as a source for other tables and for outputs.

What is a correct Sql (or Hive) way to create this "temporary" table? This should work in spark-sql?

PS: I know how to do that in spark-shell. But that is not what I'm looking for.

Alper t. Turker · Accepted Answer · 2017-12-07 00:42:20Z

6

You can:

CREATE TEMPORARY VIEW DerivedTable AS ( SELECT ID, VALUE*RATE AS Total FROM MyTable WHERE VALUE IS NOT NULL);

answered Dec 7, 2017 at 0:42

35.3k9 gold badges89 silver badges118 bronze badges

Sign up to request clarification or add additional context in comments.

1 Answer 1