麦粉社区
>
帖子详情

ETL抽取数据报错

数据准备 发表于 7 天前
发表于 7 天前


2025-03-24 15:36:04.670 [1490] ERROR node.GenericNode.handleExecuteError:148 - Node execution failed.(id:89424d9e5b4514ad4d14da4a9cd0a632,name:JDBC_DATASOURCE)
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

The last packet successfully received from the server was 10,011 milliseconds ago.  The last packet sent successfully to the server was 10,010 milliseconds ago.
        at sun.reflect.GeneratedConstructorAccessor138.newInstance(Unknown Source) ~[?:?]
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_181]
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_181]
        at com.mysql.jdbc.Util.handleNewInstance(Util.java:425) ~[_JDBC_mysql-connector-java-5.1.48.jar:5.1.48]
        at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:990) ~[_JDBC_mysql-connector-java-5.1.48.jar:5.1.48]
        at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3517) ~[_JDBC_mysql-connector-java-5.1.48.jar:5.1.48]
        at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3417) ~[_JDBC_mysql-connector-java-5.1.48.jar:5.1.48]
        at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3860) ~[_JDBC_mysql-connector-java-5.1.48.jar:5.1.48]
        at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:864) ~[_JDBC_mysql-connector-java-5.1.48.jar:5.1.48]
        at com.mysql.jdbc.MysqlIO.proceedHandshakeWithPluggableAuthentication(MysqlIO.java:1707) ~[_JDBC_mysql-connector-java-5.1.48.jar:5.1.48]
        at com.mysql.jdbc.MysqlIO.doHandshake(MysqlIO.java:1217) ~[_JDBC_mysql-connector-java-5.1.48.jar:5.1.48]
        at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2189) ~[_JDBC_mysql-connector-java-5.1.48.jar:5.1.48]
        at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2220) ~[_JDBC_mysql-connector-java-5.1.48.jar:5.1.48]
        at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2015) ~[_JDBC_mysql-connector-java-5.1.48.jar:5.1.48]
        at com.mysql.jdbc.ConnectionImpl.(ConnectionImpl.java:768) ~[_JDBC_mysql-connector-java-5.1.48.jar:5.1.48]
        at com.mysql.jdbc.JDBC4Connection.(JDBC4Connection.java:47) ~[_JDBC_mysql-connector-java-5.1.48.jar:5.1.48]
        at sun.reflect.GeneratedConstructorAccessor103.newInstance(Unknown Source) ~[?:?]
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_181]
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_181]
        at com.mysql.jdbc.Util.handleNewInstance(Util.java:425) ~[_JDBC_mysql-connector-java-5.1.48.jar:5.1.48]
        at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:385) ~[_JDBC_mysql-connector-java-5.1.48.jar:5.1.48]
        at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:323) ~[_JDBC_mysql-connector-java-5.1.48.jar:5.1.48]
        at org.apache.spark.sql.execution.datasources.jdbc.connection.BasicConnectionProvider.getConnection(BasicConnectionProvider.scala:49) ~[spark-sql_2.12-3.1.3.jar:3.1.3]
        at org.apache.spark.sql.execution.datasources.jdbc.connection.ConnectionProvider$.create(ConnectionProvider.scala:77) ~[spark-sql_2.12-3.1.3.jar:3.1.3]
        at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$.$anonfun$createConnectionFactory$1(JdbcUtils.scala:62) ~[spark-sql_2.12-3.1.3.jar:3.1.3]
        at org.apache.spark.sql.execution.datasources.jdbc.JDBCRDD$.resolveTable(JDBCRDD.scala:56) ~[spark-sql_2.12-3.1.3.jar:3.1.3]
        at org.apache.spark.sql.execution.datasources.jdbc.JDBCRelation$.getSchema(JDBCRelation.scala:226) ~[spark-sql_2.12-3.1.3.jar:3.1.3]
        at org.apache.spark.sql.execution.datasources.jdbc.JdbcRelationProvider.createRelation(JdbcRelationProvider.scala:35) ~[spark-sql_2.12-3.1.3.jar:3.1.3]
        at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:355) ~[spark-sql_2.12-3.1.3.jar:3.1.3]
        at org.apache.spark.sql.DataFrameReader.loadV1Source(DataFrameReader.scala:325) ~[spark-sql_2.12-3.1.3.jar:3.1.3]
        at org.apache.spark.sql.DataFrameReader.$anonfun$load$3(DataFrameReader.scala:307) ~[spark-sql_2.12-3.1.3.jar:3.1.3]
        at scala.Option.getOrElse(Option.scala:189) ~[scala-library-2.12.10.jar:?]
        at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:307) ~[spark-sql_2.12-3.1.3.jar:3.1.3]
        at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:225) ~[spark-sql_2.12-3.1.3.jar:3.1.3]
        at org.apache.spark.sql.DataFrameReader.jdbc(DataFrameReader.scala:340) ~[spark-sql_2.12-3.1.3.jar:3.1.3]
        at smartbix.datamining.engine.datasource.DefaultDsExecutor.read(DefaultDsExecutor.java:48) ~[EngineCore-1.0.jar:?]
        at smartbix.datamining.engine.execute.node.datasource.JdbcDatasourceNode.execute(JdbcDatasourceNode.java:159) ~[EngineCommonNode-1.0.jar:?]
        at smartbix.datamining.engine.execute.node.GenericNode.start(GenericNode.java:117) ~[EngineCore-1.0.jar:?]
        at smartbix.datamining.engine.agent.execute.executor.DefaultNodeExecutor.execute(DefaultNodeExecutor.java:43) ~[EngineAgent-1.0.jar:?]
        at smartbix.datamining.engine.agent.execute.launcher.DefaultLauncher.run(DefaultLauncher.java:67) ~[EngineAgent-1.0.jar:?]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_181]
        at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_181]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_181]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_181]
        at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_181]
Caused by: java.io.EOFException: Can not read response from server. Expected to read 4 bytes, read 0 bytes before connection was unexpectedly lost.
        at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:2969) ~[_JDBC_mysql-connector-java-5.1.48.jar:5.1.48]
        at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3427) ~[_JDBC_mysql-connector-java-5.1.48.jar:5.1.48]
        ... 39 more
发表于 7 天前
各位大哥有没有知道的
回复

使用道具 举报

发表于 7 天前
测试连接是通过的,数据查询的方式也跑出来数据了,就是执行的时候报错
回复

使用道具 举报

是不是数据库不稳定?有时可以连上有时不行?
  •   zxx
    不太可能,因为我用datagrip是可以连接的,而且这个ETL是一直报错,就是什么时候执行都不行,改了timeout默认值了
    6 天前| 回复

回复

使用道具 举报

发表于 6 天前
好像是有条 id 为 89424d9e5b4514ad4d14da4a9cd0a632 的数据有问题
  •   zxx
    这个id应该是节点的id,数据的话我们没有那么长的
    6 天前| 回复

回复

使用道具 举报

发表于 6 天前
本帖最后由 王先生 于 2025-3-25 11:34 编辑

连不上数据库。


1) 数据源连接数超了;


2) 但你“测试连接”是可以的,数据源没问题,应该是网络问题。猜测 测试连接 与 ETL是两个服务,且不在同一服务器(或者有不同限制),后者与数据源的网络不通。

回复

使用道具 举报

发表于 6 天前

报错看着是mysql返回的报错,网上查可能性比较多,参考网上看看?


回复

使用道具 举报

高级模式
B Color Image Link Quote Code Smilies
您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

8回帖数 0关注人数 274浏览人数
最后回复于:6 天前
快速回复 返回顶部 返回列表