AWS EMR客户端错误:java.lang.NoSuchFieldError:SIGNING_REGION

NateH06

在已部署的服务器上运行AWS EMR客户端时,此错误突然出现问题。这不会在本地发生,并且可以正常运行。基本上,我有一个EMR客户端,可用于构建和执行以下步骤:

class EMRClient(emrClusterId:String) extends LazyLogging{

  val accessKey = ...// access key
  val secretKey = ...//secret key
  val credentials = new BasicAWSCredentials(accessKey, secretKey)
  val REGION = <my region>
  println(">>>>>>>>>>>>>>>>>>>>Initializing EMR client for clusterId " + emrClusterId + " . The region is " + REGION)
  val emr = AmazonElasticMapReduceClientBuilder
    .standard()
    .withCredentials(new AWSStaticCredentialsProvider(credentials))
    .withRegion(REGION)
    .build()

def executeHQLStep(s3ScriptPath:String, stepName:String, args:String = ""): AddJobFlowStepsResult= {
    val hqlScriptStep = buildHQLStep(hqlScriptPath, stepName, args)
    val stepSet =  new java.util.HashSet[StepConfig]()
    //stepSet.add(enableDebugging)
    stepSet.add(hqlScriptStep)
    executeJobFlowSteps(stepSet)
  }

  /**
    * Builds a StepConfig to be executed in a job flow for a given .hql file from S3
    * @param hqlScriptPath the location in S3 of the script file containing the script to run
    * @param args optional field for arguments for hive script.
    * @param stepName An identifier to give to EMR to name your Step
    * @return
    */
  private def buildHQLStep(hqlScriptPath:String, stepName:String, args:String= ""): StepConfig = {
    new StepConfig()
      .withName(stepName)
      .withActionOnFailure(ActionOnFailure.CANCEL_AND_WAIT)
      .withHadoopJarStep(stepFactory.newRunHiveScriptStep(hqlScriptPath, args))
  }

  private def executeJobFlowSteps(steps: java.util.Set[StepConfig]): AddJobFlowStepsResult = {
    emr.addJobFlowSteps(new AddJobFlowStepsRequest()
      .withJobFlowId(emrClusterId)
          .withSteps(steps)) // where the error is thrown
      }
}

但是,当在服务器上实例化此类println时,顶部的所有语句都不可见,executeJobFlowSteps将调用我的方法并引发此错误:

java.lang.NoSuchFieldError:com.amazonaws.services.elasticmapreduce.AmazonElasticMapReduceClient.executeAddJobFlowSteps(AmazonElasticMapReduceClient.java:439)处的SIGNING_REGION com.amazonaws.services.elasticmapreduce.AmazonElasticMapReduceClient.addJobFlowSteps(AmazonElasticMapReduceClient.addJobFlowSteps)在emrservices.EMRClient.executeHQLStep(EMRClient.scala:44)上的executeJobFlowSteps(EMRClient.scala:64)

这个项目由几个项目组成,我遇到的类似问题都说这与它们的AWS依赖关系有关,但是所有项目在build.sbt的库依赖项中都有: "com.amazonaws" % "aws-java-sdk" % "1.11.286"

对这个问题有什么想法吗?

迈克尔·M

看来您正在将1.11.286版本的aws-java-sdk与旧版本(1.11.149)的aws-java-sdk-core混合在一起。较新的客户端正在使用添加到核心模块的新字段,但是由于核心模块已过期,因此您会看到没有此类字段错误。您可以确保所有依赖项彼此同步吗?

本文收集自互联网,转载请注明来源。

如有侵权,请联系 [email protected] 删除。

编辑于
0

我来说两句

0 条评论
登录 后参与评论

相关文章

适用于ssm的aws Java sdk提供了java.lang.NoSuchFieldError:SIGNING_REGION

无法解决错误:java.lang.NoSuchFieldError:FAIL_ON_SYMBOL_HASH_OVERFLOW- elasticsearch Java客户端api

使用 AWS Amplify API 時的 Android java.lang.NoSuchFieldError

线程“主”中的异常java.lang.NoSuchFieldError:AWS SNS中的INSTANCE

AWS Lambda java.lang.NoClassDefFound 错误

5.3中的Elasticsearch客户端java.lang.ExceptionInInitializerError

NullnessChecker错误:java.lang.NoSuchFieldError:释放

Java中的AWS Lambda字节流客户端

java.lang.NoSuchFieldError:INSTANCE

java.lang.NoSuchFieldError:默认

java.lang.NoSuchFieldError:反思

AWS Client VPN客户端-客户端通信

AWS SES - SMTP 与 SES 客户端

Terraform AWS Cognito App客户端

AWS多个VPN客户端证书

如何从客户端启动AWS Codepipeline?

存根 AWS SQS 客户端

Laravel 5 Amazon AWS S3错误:客户端错误:403 RequestTimeTooSkewed

java.lang.NoSuchFieldError:IBM_JAVA

java.lang.NoSuchFieldError: INSTANCE - WebDriver

更改类后java.lang.NoSuchFieldError

ScalaFx MouseEvent:java.lang.NoSuchFieldError:返回

HttpClientBuilder-java.lang.NoSuchFieldError:INSTANCE

HBase客户端-java.lang.ClassNotFoundException:org.apache.hadoop.crypto.key.KeyProviderTokenIssuer

Resteasy客户端java.lang.IllegalStateException:响应已关闭

Scala udf在flink sql客户端中抱怨“ java.lang.ClassNotFoundException”

Couchbase DCP 客户端 0.10.0 因 java.lang.NoSuchMethodError 失败

Firebase Android 客户端:“在对象中不能应用于 java.lang.class”

为什么AWS Lambda调用客户端错误地返回ClientExecutionTimeoutException?