在Mac OS X上安装,配置和运行Hadoop 2.2.0

MRP

我已经安装了hadoop 2.2.0,并根据此教程在此处设置了所有内容(针对单个节点):Hadoop YARN Installation但是,我无法让hadoop运行。

我认为我的问题是我无法连接到本地主机,但是我不确定为什么。我花了大约10个小时以上的时间来安装,谷歌搜索和讨厌开源软件安装指南,所以现在我转到了一个从未失败过的地方。

由于一幅图片价值一千个单词,因此我给您进行设置...图片中包含许多 单词


基本配置文件/设置


我正在运行Mac OS X(Mavericks 10.9.5)

分享

无论价值多少,这是我的/etc/hosts文件:

主机

我的bash个人资料:

重击


Hadoop文件配置


对于设置core-site.xmlhdfs-site.xml

核心hdfs.xlm

注意:我已经在上方看到的位置创建了文件夹

我的设置yarn-site.xml

纱

我的hadoop-env.sh文件的设置

hadoop-env


边注


在我告诉当我运行的结果start-dfs.shstart-yarn.sh并检查,看看有什么与运行jps,记住,我有一个hadoop指向hadoop-2.2.0

指针


启动Hadoop


现在,这是我启动守护进程的结果:

启动

对于那些没有显微镜的人(在本文的预览中看起来很小),这是上面显示的代码块:

mrp:~ mrp$ start-dfs.sh
2014-11-08 13:06:05.695 java[17730:1003] Unable to load realm info from SCDynamicStore
14/11/08 13:06:05 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Starting namenodes on [localhost]
localhost: starting namenode, logging to /usr/local/hadoop-2.2.0/logs/hadoop-mrp-namenode-mrp.local.out
localhost: starting datanode, logging to /usr/local/hadoop-2.2.0/logs/hadoop-mrp-datanode-mrp.local.out
localhost: 2014-11-08 13:06:10.954 java[17867:1403] Unable to load realm info from SCDynamicStore
Starting secondary namenodes [0.0.0.0]
0.0.0.0: starting secondarynamenode, logging to /usr/local/hadoop-2.2.0/logs/hadoop-mrp-secondarynamenode-mrp.local.out
0.0.0.0: 2014-11-08 13:06:16.065 java[17953:1403] Unable to load realm info from SCDynamicStore
2014-11-08 13:06:20.982 java[17993:1003] Unable to load realm info from SCDynamicStore
14/11/08 13:06:20 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

mrp:~ mrp$ start-yarn.sh
starting yarn daemons
starting resourcemanager, logging to /usr/local/hadoop/logs/yarn-mrp-resourcemanager-mrp.local.out
2014-11-08 13:06:43.765 java[18053:20b] Unable to load realm info from SCDynamicStore
localhost: starting nodemanager, logging to /usr/local/hadoop-2.2.0/logs/yarn-mrp-nodemanager-mrp.local.out

Check to see what's running:

jps


Time Out


OK. So far, I think, so good. At least this looks good based on all the other tutorials and posts. I think.

Before I try to do anything fancy, I'll just want to see if it's working properly, and run a simple command like hadoop fs -ls.


Failure


When I run hadoop fs -ls, here's what I get:

失败

Again, in case you can't see that pic, it says:

2014-11-08 13:23:45.772 java[18326:1003] Unable to load realm info from SCDynamicStore
14/11/08 13:23:45 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
ls: Call From mrp.local/127.0.0.1 to localhost:8020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused

I've tried to run other commands, and I get the same basic error in the beginning of everything:

Call From mrp.local/127.0.0.1 to localhost:8020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused

Now, I've gone to that website mentioned, but honestly, everything in that link means nothing to me. I don't get what I should do.

I would very much appreciate any assistance with this. You'll make me the happiest hadooper, ever.

...this should go without saying, but obviously I'd be happy to edit/update with more info if needed. Thanks!

mrp

So I've got Hadoop up and running. I had two problems (I think).

  1. When starting up the NameNode and DataNode, I received the following error: Unable to load realm info from SCDynamicStore.

To fix this, I added the following two lines to my hadoop-env.sh file:

HADOOP_OPTS="${HADOOP_OPTS} -Djava.security.krb5.realm= -Djava.security.krb5.kdc=" HADOOP_OPTS="${HADOOP_OPTS} -Djava.security.krb5.conf=/dev/null"

我在本文的解决方案中找到了这两行,即OSX上的Hadoop“无法从SCDynamicStore加载领域信息”答案由马修·L·丹尼尔(Matthew L Daniel)发布。

  1. 我已经多次格式化了NameNode文件夹,这显然搞砸了吗?

我无法验证此操作是否正常,因为我的任何日志文件中都没有任何错误,但是,一旦遵循本文中的解决方法1(删除并重新创建NameNode / DataNode文件夹,然后重新格式化),就没有数据节点了。开始后,我能够加载DataNode并使一切正常运行。

本文收集自互联网,转载请注明来源。

如有侵权,请联系 [email protected] 删除。

编辑于
0

我来说两句

0 条评论
登录 后参与评论

相关文章

在Mac OS上运行的Hadoop

在Mac OS X 10.9.5上安装Psycopg2

如何在Mac OS上安装2个Anacondas(Python 2和3)

在Mac OS(Yosemite)上使用psycopg2时出现问题

无法在Mac OS X 10.7上的virtualenv中使用pip安装psycopg2

是否存在与Amazon EC2按需实例相当的按需Mac OS X云服务?

Mac OS Lion上Sublime text 2的Packages文件夹的完整路径是什么

导入M2Crypto Mac OS X时出错

如何在Mac OS上安装libxml2-dev libxslt-dev

在Mac OS X(boot2docker)上的Docker中启用远程API

在Mac OS X上的boot2docker,docker,django

2 ^ 0 * 2和(2 ^ 0)* 2之间的区别?

Mac OS X批量重命名文件(2种模式)

laravel连接时,mac os上的mysql docker在caching_sha2上“崩溃”

您无权访问此服务器上的/-Mac OS X Mojave上的Apache 2

无法在Mac OS X Mojave上重新启动Apache 2

无法在Mac OS上安装Psycopg2

-bash:sudo:找不到Mac OS X的命令(EC2 CLI可能吗?)

使用Mac OS X下载距离2处的所有图像

Mac OS X更新后的GRUB 2问题

如何在Mac OS X的Apache2中轻松创建SSL证书并对其进行配置?

如何为Mac OS X合并Seagate 2 TB外部硬盘

py2app pyqt缺少库,应用程序无法在Mac OS X上启动

如何从Mac OS X终端启动Sublime Text 2?

如何删除Mac OS X中已安装的Apache2?

如何在bash上设置Mac OS X Terminal和在zsh上设置iTerm2

fail2ban / ipfw / Mac OS X服务器不拒绝连接

phpinfo()在Mac OS-X Capitan apache2服务器上不起作用

使用 YUV 4:2:0 平面颜色模型在 Mac OS X 上制作 ffmpeg 屏幕截图