spark 开发环境

1. Eclipse开发环境搭建

1.1. 安装scala插件

安装eclipse-scala-plugin插件，下载地址http://scala-ide.org/download/prev-stable.html

解压缩之后把plugins和features复制到eclipse目录，重启eclipse之后便可。

Window -> Open Perspective -> Other…，打开Scala，说明安装成功。

1.2. 建立maven工程

打开File -> New -> Other…，选择Maven Project：

点击Next，输入项目存放路径：

点击Next，选择org.scala-tools.archetypes：

点击Next，输入artifact相关信息：

点击Finish便可。默认建立好的工程目录结构以下：

前面用的别人的截图

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

<modelVersion>4.0.0</modelVersion>

<groupId>com.dianru</groupId>

<artifactId>SparkSqlTest</artifactId>

<version>1.0-SNAPSHOT</version>

<packaging>jar</packaging>

<name>demon-parent</name>

<url>http://maven.apache.org</url>

<dependencies>

<dependency>

<groupId>org.apache.spark</groupId>

<artifactId>spark-sql_2.10</artifactId>

<version>1.6.0</version>

</dependency>

<dependency>

<groupId>org.apache.spark</groupId>

<artifactId>spark-core_2.10</artifactId>

<version>1.6.0</version>

</dependency>

<dependency>

<groupId>net.debasishg</groupId>

<artifactId>redisclient_2.10</artifactId>

<version>3.0</version>

</dependency>

</dependencies>

<build>

<plugins>

<plugin>

<artifactId>maven-assembly-plugin</artifactId>

<configuration>

<descriptorRefs>

<descriptorRef>jar-with-dependencies</descriptorRef>

</descriptorRefs>

</configuration>

<executions>

<!-- execution元素包含了插件执行须要的信息 -->

<execution>

<id>make-assembly</id>

<phase>package</phase>

<goals>

<goal>single</goal>

</goals>

</execution>

</executions>

</plugin>

<plugin>

<groupId>org.scala-tools</groupId>

<artifactId>maven-scala-plugin</artifactId>

<version>2.15.2</version>

<executions>

<execution>

<goals>

<goal>compile</goal>

<goal>testCompile</goal>

</goals>

</execution>

</executions>

</plugin>

</plugins>

</build>

</project>