启动Apache Spark集群

杰曼库索

我已经在四节点集群上安装了Hadoop。我还在每个服务器上都安装了Apache Spark。我可以在没有密码的情况下将ssh从主机发送到每个从机。我也可以启动我的主节点。但是,当我尝试使用/opt/spark/sbin/start-all.sh运行Spark时,出现以下错误:

starting org.apache.spark.deploy.master.Master, logging to /opt/spark/logs/spark-hduser-org.apache.spark.deploy.master.Master-1-lebron.out
doublet: chown: changing ownership of ‘/opt/spark/logs’: Operation not permitted
doublet: starting org.apache.spark.deploy.worker.Worker, logging to /opt/spark/logs/spark-hduser-org.apache.spark.deploy.worker.Worker-1-doublet.out
doublet: /opt/spark/sbin/spark-daemon.sh: line 149: /opt/spark/logs/spark-hduser-org.apache.spark.deploy.worker.Worker-1-doublet.out: Permission denied
kyrie: chown: changing ownership of ‘/opt/spark/logs’: Operation not permitted
kyrie: starting org.apache.spark.deploy.worker.Worker, logging to /opt/spark/logs/spark-hduser-org.apache.spark.deploy.worker.Worker-1-kyrie.out
kyrie: /opt/spark/sbin/spark-daemon.sh: line 149: /opt/spark/logs/spark-hduser-org.apache.spark.deploy.worker.Worker-1-kyrie.out: Permission denied
lebron: starting org.apache.spark.deploy.worker.Worker, logging to /opt/spark/logs/spark-hduser-org.apache.spark.deploy.worker.Worker-1-lebron.out
jr: chown: changing ownership of ‘/opt/spark/logs’: Operation not permitted
jr: starting org.apache.spark.deploy.worker.Worker, logging to /opt/spark/logs/spark-hduser-org.apache.spark.deploy.worker.Worker-1-jr.out
jr: /opt/spark/sbin/spark-daemon.sh: line 149: /opt/spark/logs/spark-hduser-org.apache.spark.deploy.worker.Worker-1-jr.out: Permission denied
doublet: failed to launch org.apache.spark.deploy.worker.Worker:
doublet: tail: cannot open ‘/opt/spark/logs/spark-hduser-org.apache.spark.deploy.worker.Worker-1-doublet.out’ for reading: No such file or directory
doublet: full log in /opt/spark/logs/spark-hduser-org.apache.spark.deploy.worker.Worker-1-doublet.out
kyrie: failed to launch org.apache.spark.deploy.worker.Worker:
kyrie: tail: cannot open ‘/opt/spark/logs/spark-hduser-org.apache.spark.deploy.worker.Worker-1-kyrie.out’ for reading: No such file or directory
jr: failed to launch org.apache.spark.deploy.worker.Worker:
kyrie: full log in /opt/spark/logs/spark-hduser-org.apache.spark.deploy.worker.Worker-1-kyrie.out
jr: tail: cannot open ‘/opt/spark/logs/spark-hduser-org.apache.spark.deploy.worker.Worker-1-jr.out’ for reading: No such file or directory
jr: full log in /opt/spark/logs/spark-hduser-org.apache.spark.deploy.worker.Worker-1-jr.out

(我的计算机名为lebron(主计算机),kyrie,jr,doublet(工作人员))

请帮忙!!!

哈里·辛格|

看起来/ opt / spark文件夹没有写日志文件的权限,请授予完全权限:-sudo chmod -R 777 / opt / spark并检查/ opt / spark文件夹user:group是否对所有机器(主机和从机都),否则运行此命令sudo chown -R userName:groupName / opt / spark

本文收集自互联网,转载请注明来源。

如有侵权,请联系 [email protected] 删除。

编辑于
0

我来说两句

0 条评论
登录 后参与评论

相关文章

Apache zeppelin:Spark 集群配置

Google Cloud 上的 Apache Spark、集群或无集群

Hive的Apache Spark部署问题(集群模式)

启动集群时在EMR上配置Zeppelin的Spark解释器

Apache Spark / Cassandra集群上的分区过多(任务过多)

Apache Spark无法在集群上反序列化“ TopicPartition”

Apache Spark + Ignite 集群瘦客户端

独立集群模式下具有Apache Spark的Docker容器

Apache Spark是否在节点级别或集群级别缓存RDD?

Apache Spark:具有不同配置节点的集群

从Eclipse调试Apache Spark集群应用程序

如何在Apache Spark和Drill AWS集群中优化AWS集群实例类型?

启动时Apache Spark错误

Apache Spark:启动PySpark时出错

无法启动apache.spark.master

如何使用Spark org.apache.spark.ml.clustering预测kmeans集群。{KMeans,KMeansModel}

如果 SparkContext 未启动,则集群模式下的 Spark 会引发错误

在AWS EMR集群上启动spark命令行时拒绝权限

多节点Spark集群上的spark-shell无法在远程工作节点上启动执行程序

YARN如何在集群模式下了解Apache Spark中的数据位置

只有 2 台计算机的集群上的 Apache spark

将 Apache Spark 应用程序提交到容器化集群时遇到问题

Apache Spark在SQL表达式中启动

从多线程驱动程序启动Apache Spark SQL作业

Apache Toree Spark内核无法启动(java.lang.NoSuchMethodError)

无法使用Cygwin在Windows上启动Apache Spark

高山Linux docker上的Apache-Spark启动错误

Apache Spark:“无法启动org.apache.spark.deploy.worker.Worker”或Master

启动MPI集群失败