Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] [k8s_Deploy] The client submits tasks and the home directory of the cluster must be consistent?? #8371

Open
2 of 3 tasks
reformdai opened this issue Dec 24, 2024 · 0 comments
Labels

Comments

@reformdai
Copy link

Search before asking

  • I had searched in the issues and found no similar issues.

What happened

客户端提交任务到集群,报错 source 的connector 的jar 包 class not found
When the client submits a task to the cluster, an error message is displayed: 【com.hazelcast.nio.serialization.HazelcastSerializatcionException: java.lang.ClassNotFoundException: orgg.apache.seatunnel.connectors.seatunnel.feishu.source.Feishusource】

image

由于启动的 client 的 SEATUNNEL_HOME 目录是本地的,然后本地启动cluster,提交成功!
Since the SEATUNNEL_HOME directory of the started client is local, and then the cluster is started locally, the submission is successful!

最后发现原因是client提交的任务,SEATUNNEL_HOME 是需要与 cluster 中的保持一致。
Finally, I found that the reason was the task submitted by the client. SEATUNNEL_HOME needs to be consistent with that in the cluster.

除了使用 REST 提交,有没有任何可以解决的办法呢?
Is there any solution besides using REST submission?

SeaTunnel Version

2.3.8

SeaTunnel Config

env {
  job.mode = "BATCH"
  parallelism = 1
}

source {
  Feishu {
    url = "https://open.feishu.cn/open-apis"
    appId = "xxxxx"
    appSecret = "xxxxx"
    nodeToken = "xxxxx"
    tableId = "xxxxx"
    format = "json"
    pageing = {
      page_field = "page_token"
      batch_size = "200"
    }
}

transform {
}

sink {
  Console {
  }
}

Running Command

ClientJobExecutionEnvironment jobExecutionEnv = seaTunnelClient.createExecutionContext(filePath, jobConfig, seaTunnelConfig);
ClientJobProxy clientJobProxy = jobExecutionEnv.execute();

Error Exception

com.hazelcast.nio.serialization.HazelcastSerializatcionException: java.lang.ClassNotFoundException: orgg.apache.seatunnel.connectors.seatunnel.feishu.source.Feishusource

Zeta or Flink or Spark Version

Zeta

Java or Scala Version

Java

Screenshots

image

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@reformdai reformdai added the bug label Dec 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant