WARNING: An illegal reflective access operation has occurred $ /opt/spark/bin/spark-shellĢ1/04/27 08:49:09 WARN Utils: Your hostname, ubuntu resolves to a loopback address: 127.0.1.1 using 10.10.10.2 instead (on interface eth0)Ģ1/04/27 08:49:09 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to another address Use the spark-shell command to access Spark Shell. You can also use the absolute path to run the script.

If you don’t have the script in your $PATH, you can first locate it. Starting .worker.Worker, logging to /opt/spark/logs/. The start-slave.sh command is used to start Spark Worker Process. The process will be listening on TCP port 8080.

Starting .master.Master, logging to /opt/spark/logs/. You can now start a standalone master server using the start-master.sh command. source ~/.bashrc Step 3: Start a standalone master server vim ~/.bashrcĮxport PATH=$PATH:$SPARK_HOME/bin:$SPARK_HOME/sbinĪctivate the changes. sudo mv spark-3.2.1-bin-hadoop3.2/ /opt/spark Set Spark environment Move the Spark folder created after extraction to the /opt/ directory. OpenJDK 64-Bit Server VM (build 11.0.14.1+1-Ubuntu-0ubuntu1.20.04, mixed mode, sharing)įor missing add-apt-repository command, check How to Install add-apt-repository on Debian / Ubuntu Step 2: Download Apache Sparkĭownload the latest release of Apache Spark from the downloads page. Verify Java version using the command: $ java -version Step 1: Install Java RuntimeĪpache Spark requires Java to run, let’s make sure we have Java installed on our Ubuntu system.įor default system Java: sudo apt install curl mlocate default-jdk -y Now use the steps shown next to install Spark on Ubuntu 22.04|20.04|18.04. Sudo apt update & sudo apt -y full-upgradeĬonsider a system reboot after upgrade is required.

How to install spark on windows 10