搜索
搜索

RuntimeException，ClassNotFoundException：类WordCount $ Map未找到。映射器类问题

disjoint_molecule 发表于 Java
68
disjoint_molecule：

我正在沙盒HortonWork HDP上运行WordCount。我已经在许多地方搜索了此错误，并应用了他们所说的内容，但我无法使其运行。

逐步介绍：1-首先，我编译Java程序

javac -cp .:$(hadoop classpath) WordCount.java

2-接下来将类捆绑为jar文件

jar cvf WordCount.jar *.class

3-让我们看一下jar文件的内容：

jar tf WordCount.jar

META-INF/
META-INF/MANIFEST.MF
WordCount.class
WordCount$Map.class
WordCount$Reduce.class

4-接下来，将输入文件设置为hdfs并运行作业。

hdfs dfs -put file.txt

hadoop jar WordCount.jar WordCount /user/root/file.txt /user/root/output

5-我收到以下错误：

WARNING: Use "yarn jar" to launch YARN applications.
16/08/12 04:02:49 INFO impl.TimelineClientImpl: Timeline service address: http://sandbox.hortonworks.com:8188/ws/v1/timeline/
16/08/12 04:02:49 INFO client.RMProxy: Connecting to ResourceManager at sandbox.hortonworks.com/10.0.2.15:8050
16/08/12 04:02:50 WARN mapreduce.JobResourceUploader: No job jar file set.  User classes may not be found. See Job or Job#setJar(String).
16/08/12 04:02:50 INFO input.FileInputFormat: Total input paths to process : 1
16/08/12 04:02:51 INFO mapreduce.JobSubmitter: number of splits:1
16/08/12 04:02:51 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1470929339157_0022
16/08/12 04:02:51 INFO mapred.YARNRunner: Job jar is not present. Not adding any jar to the list of resources.
16/08/12 04:02:51 INFO impl.YarnClientImpl: Submitted application application_1470929339157_0022
16/08/12 04:02:52 INFO mapreduce.Job: The url to track the job: http://sandbox.hortonworks.com:8088/proxy/application_1470929339157_0022/
16/08/12 04:02:52 INFO mapreduce.Job: Running job: job_1470929339157_0022
16/08/12 04:03:03 INFO mapreduce.Job: Job job_1470929339157_0022 running in uber mode : false
16/08/12 04:03:03 INFO mapreduce.Job:  map 0% reduce 0%
16/08/12 04:03:10 INFO mapreduce.Job:  map 100% reduce 0%
16/08/12 04:03:10 INFO mapreduce.Job: Task Id : attempt_1470929339157_0022_m_000000_0, Status : FAILED
Error: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class WordCount$Map not found
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2195)
    at org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass(JobContextImpl.java:186)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:745)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:415)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
Caused by: java.lang.ClassNotFoundException: Class WordCount$Map not found
    at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2101)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2193)
    ... 8 more

16/08/12 04:03:11 INFO mapreduce.Job:  map 0% reduce 0%
16/08/12 04:03:16 INFO mapreduce.Job: Task Id : attempt_1470929339157_0022_m_000000_1, Status : FAILED
Error: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class WordCount$Map not found
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2195)
    at org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass(JobContextImpl.java:186)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:745)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:415)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
Caused by: java.lang.ClassNotFoundException: Class WordCount$Map not found
    at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2101)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2193)
    ... 8 more

16/08/12 04:03:23 INFO mapreduce.Job: Task Id : attempt_1470929339157_0022_m_000000_2, Status : FAILED
Error: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class WordCount$Map not found
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2195)
    at org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass(JobContextImpl.java:186)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:745)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:415)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
Caused by: java.lang.ClassNotFoundException: Class WordCount$Map not found
    at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2101)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2193)
    ... 8 more

16/08/12 04:03:31 INFO mapreduce.Job:  map 100% reduce 100%
16/08/12 04:03:31 INFO mapreduce.Job: Job job_1470929339157_0022 failed with state FAILED due to: Task failed task_1470929339157_0022_m_000000
Job failed as tasks failed. failedMaps:1 failedReduces:0

16/08/12 04:03:31 INFO mapreduce.Job: Counters: 13
    Job Counters 
        Failed map tasks=4
        Killed reduce tasks=1
        Launched map tasks=4
        Other local map tasks=3
        Data-local map tasks=1
        Total time spent by all maps in occupied slots (ms)=20865
        Total time spent by all reduces in occupied slots (ms)=0
        Total time spent by all map tasks (ms)=20865
        Total time spent by all reduce tasks (ms)=0
        Total vcore-seconds taken by all map tasks=20865
        Total vcore-seconds taken by all reduce tasks=0
        Total megabyte-seconds taken by all map tasks=5216250
        Total megabyte-seconds taken by all reduce tasks=0

`

以下代码是我尝试运行的代码：

import java.io.IOException;
import java.util.*;

import org.apache.hadoop.conf.*;
import org.apache.hadoop.fs.*;
import org.apache.hadoop.conf.*;
import org.apache.hadoop.io.*;
import org.apache.hadoop.mapreduce.*;
import org.apache.hadoop.mapreduce.lib.input.*;
import org.apache.hadoop.mapreduce.lib.output.*;
import org.apache.hadoop.util.*;

public class WordCount extends Configured implements Tool {

  public static void main(String args[]) throws Exception {
    int res = ToolRunner.run(new WordCount(), args);
    System.exit(res);
  }

  public int run(String[] args) throws Exception {
    Path inputPath = new Path(args[0]);
    Path outputPath = new Path(args[1]);

    Configuration conf = getConf();
    Job job = new Job(conf, this.getClass().toString());
    job.setJarByClass(WordCount.class);

    FileInputFormat.setInputPaths(job, inputPath);
    FileOutputFormat.setOutputPath(job, outputPath);

    job.setJobName("WordCount");
    //job.setJarByClass(WordCount.class);

    job.setMapperClass(Map.class);
    job.setCombinerClass(Reduce.class);
    job.setReducerClass(Reduce.class);
    job.setMapOutputKeyClass(Text.class);
    job.setMapOutputValueClass(IntWritable.class);
    job.setOutputKeyClass(Text.class);
    job.setOutputValueClass(IntWritable.class);
    job.setInputFormatClass(TextInputFormat.class);
    job.setOutputFormatClass(TextOutputFormat.class);
    //FileInputFormat.setInputPaths(job, inputPath);
    //FileOutputFormat.setOutputPath(job, outputPath);

   return job.waitForCompletion(true) ? 0 : 1;
  }

  public static class Map extends Mapper<LongWritable, Text, Text, IntWritable> {
    private final static IntWritable one = new IntWritable(1);
    private Text word = new Text();

    @Override
    public void map(LongWritable key, Text value,
                    Mapper.Context context) throws IOException, InterruptedException {
      String line = value.toString();
      StringTokenizer tokenizer = new StringTokenizer(line);
      while (tokenizer.hasMoreTokens()) {
        word.set(tokenizer.nextToken());
        context.write(word, one);
      }
    }
  }

  public static class Reduce extends Reducer<Text, IntWritable, Text, IntWritable> {

    @Override
    public void reduce(Text key, Iterable<IntWritable> values, Context context) throws IOException, InterruptedException {
      int sum = 0;
      for (IntWritable value : values) {
        sum += value.get();
      }

      context.write(key, new IntWritable(sum));
    }
  }

}

在此先感谢您的帮助/提示和建议。

vgunnu：

尝试在您的课程中添加以下内容

job.setJar("WordCount.jar");

本文收集自互联网，转载请注明来源。

如有侵权，请联系 [email protected] 删除。

编辑于
0

我来说两句

0 条评论
登录 后参与评论

相关文章

运行地图精简程序时出现错误java.lang.RuntimeException：java.lang.ClassNotFoundException：wordcount_classes.WordCount $ Map

java.lang.RuntimeException：无法实例化活动ComponentInfo {…}：java.lang.ClassNotFoundException：未找到类

找不到hazelcast映射器类

更改生成的映射器类的位置

Jackson映射器跳过类名称（反序列化问题）

CSV 映射器类确实从列表中映射

使用基类的映射器映射子类

ArgumentError：关系需要类或映射器参数

实体的 Morphia 映射器（用于整个 Java 类/POJO）

在Scala中具有通用类的Jackson映射器

Eclipse代码生成器生成映射器类

如何从对象映射器类过滤数据

如何通过数据驱动映射器类

模型映射器将 Map<String,Object> 映射到扩展另一个不适用于列表字段的通用类的类

改善Wordcount中的身份映射器

休眠-未找到类

未找到 mPDF 类

未找到“JWT”类

致命错误：未捕获错误：未找到类“ Ds \ Map”

自动映射器将2个实体映射到单个类

如何使用自动映射器映射具有多个自定义属性类型的类

自动映射器：映射由基类对象组成的集合

包含IncludeBase的自动映射器映射对抽象泛型类失败

如何快速使用对象映射器将JSON字符串映射到模型类

自动映射器从内部属性映射到目标类

存在控制器类但未找到

未找到 Doctrine 注释阅读器类

sqlalchemy.exc.ArgumentError：映射器映射的类无法为映射表“用户”组装任何主键列

ArgumentError：映射器映射类用户-> 用户无法为映射表“用户”组装任何主键列

TOP 榜单

  1. 1

    蓝屏死机没有修复解决方案

  2. 2

    计算数据帧中每行的NA

  3. 3

    UITableView的项目向下滚动后更改颜色，然后快速备份

  4. 4

    Node.js中未捕获的异常错误，发生调用

  5. 5

    在 Python 2.7 中。如何从文件中读取特定文本并分配给变量

  6. 6

    Linux的官方Adobe Flash存储库是否已过时？

  7. 7

    验证REST API参数

  8. 8

    ggplot：对齐多个分面图-所有大小不同的分面

  9. 9

    Mac OS X更新后的GRUB 2问题

  10. 10

    通过 Git 在运行 Jenkins 作业时获取 ClassNotFoundException

  11. 11

    带有错误“ where”条件的查询如何返回结果？

  12. 12

    用日期数据透视表和日期顺序查询

  13. 13

    VB.net将2条特定行导出到DataGridView

  14. 14

    如何从视图一次更新多行（ASP.NET - Core）

  15. 15

    Java Eclipse中的错误13，如何解决？

  16. 16

    尝试反复更改屏幕上按钮的位置 - kotlin android studio

  17. 17

    离子动态工具栏背景色

  18. 18

    应用发明者仅从列表中选择一个随机项一次

  19. 19

    当我尝试下载 StanfordNLP en 模型时，出现错误

  20. 20

    python中的boto3文件上传

  21. 21

    在同一Pushwoosh应用程序上Pushwoosh多个捆绑ID

热门标签

归档