Spark ml报错：Error:scalac: missing or invalid dependency detected while loading class file 'SQLImplici

1、问题描述

采用spark2.3.0的 ml库进行推荐算法建模，可是，运行代码到以下推荐个数的时候

...
model.recommendForAllUsers(7).show(false)
...

报错以下图：

具体报错以下：

​
Information:Module "sparktest" was fully rebuilt due to project configuration/dependencies changes
Information:2019/5/20 8:53 - Build completed with 23 errors and 0 warnings in 4 s 71 ms
Error:scalac: missing or invalid dependency detected while loading class file 'SQLImplicits.class'.
Could not access type Encoder in package org.apache.spark.sql,
because it (or its dependencies) are missing. Check your build definition for
missing or conflicting dependencies. (Re-run with `-Ylog-classpath` to see the problematic classpath.)
A full rebuild may help if 'SQLImplicits.class' was compiled against an incompatible version of org.apache.spark.sql.
Error:scalac: missing or invalid dependency detected while loading class file 'LowPrioritySQLImplicits.class'.
Could not access type Encoder in package org.apache.spark.sql,
because it (or its dependencies) are missing. Check your build definition for
missing or conflicting dependencies. (Re-run with `-Ylog-classpath` to see the problematic classpath.)
A full rebuild may help if 'LowPrioritySQLImplicits.class' was compiled against an incompatible version of org.apache.spark.sql.
Error:scalac: missing or invalid dependency detected while loading class file 'package.class'.
Could not access type Row in package org.apache.spark.sql,
because it (or its dependencies) are missing. Check your build definition for
missing or conflicting dependencies. (Re-run with `-Ylog-classpath` to see the problematic classpath.)
A full rebuild may help if 'package.class' was compiled against an incompatible version of org.apache.spark.sql.
Error:scalac: missing or invalid dependency detected while loading class file 'Dataset.class'.
Could not access type Encoder in package org.apache.spark.sql,
because it (or its dependencies) are missing. Check your build definition for
missing or conflicting dependencies. (Re-run with `-Ylog-classpath` to see the problematic classpath.)
A full rebuild may help if 'Dataset.class' was compiled against an incompatible version of org.apache.spark.sql.
Error:scalac: missing or invalid dependency detected while loading class file 'SparkSession.class'.
Could not access type Encoder in package org.apache.spark.sql,
because it (or its dependencies) are missing. Check your build definition for
missing or conflicting dependencies. (Re-run with `-Ylog-classpath` to see the problematic classpath.)
A full rebuild may help if 'SparkSession.class' was compiled against an incompatible version of org.apache.spark.sql.

​

2、问题缘由

百思不得其解，由于pom依赖也已经引入，并且代码也不报错，纠结了一周

<dependency>
      <groupId>org.apache.spark</groupId>
      <artifactId>spark-mllib_2.11</artifactId>
      <version>2.3.0</version>
<!--      <scope>runtime</scope>-->
    </dependency>

网上诸多查找，而后发现，十分多是下载依赖的时候出现问题。查看依赖位置，发现依赖jar包后面有lastUpdated字样，证实是依赖下载失败。

 

3、解决办法

删除下载依赖，从新pom导入依赖。

而后从新运行程序，成功~