ModuleNotFoundError: No module named 'py4j'

The error ModuleNotFoundError: No module named ‘py4j‘ is as shown below and getting it while importing some modules.

/opt/spark/python/pyspark/context.py in <module>
     27 from tempfile import NamedTemporaryFile
     28 
---> 29 from py4j.protocol import Py4JError
     30 
     31 from pyspark import accumulators

ModuleNotFoundError: No module named 'py4j'

If you can run spark directly, you may need to modify the PYTHONPATH environment setting. Check the filename in the $SPARK HOME/python/lib/ directory. If you’re using Spark 2.4.3, the file is py4j-0.10.7-src.zip then run:

export PYTHONPATH=$SPARK_HOME/python:$SPARK_HOME/python/lib/py4j-0.10.7-src.zip:$PYTHONPATH

Hope it solves your issue.

ModuleNotFoundError: No module named ‘py4j’

Related Codes