0

My pip install pyspark worked, I get a message in my command prompt that SparkSession available as 'spark'.

However, when I do:

from pyspark import SparkContext 

it gives me a:

ModuleNotFoundError: No module named 'pyspark' 

What's the problem and how do I fix it?

3
  • 1
    you need to set SPARK_HOME and PYTHONPATH variables set Commented Jun 22, 2018 at 14:07
  • Thanks, and how do I do that? Commented Jun 22, 2018 at 14:08
  • are you on linux or windows ? do you know the path to the spark directory? Commented Jun 22, 2018 at 14:30

1 Answer 1

1

You should install findspark modeule by pip then try:

import findspark findspark.init('<your spark home directory for example /home/spark>') 

then you can import spark modules.

Sign up to request clarification or add additional context in comments.

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.