![]() ![]() Hadoop fs -put /tmp/spark-1.6.0-bin-hadoop2.6/lib/spark-assembly-1.6.0-hadoop2.6.0.jar /tmp/spark/įor using the Spark Script operator, you need to have Python 2.6+ or Python 3.4+ (for PySpark scripts) and R 3.1+ (for SparkR scripts) installed on the cluster nodes. Installing Spark 1.6.0 for Hadoop 2.6 or later (you need to change the download link and the path for older Hadoop or newer Spark versions): hadoop fs -mkdir -p /tmp/spark Please take care that the package type should meet your cluster setup. You can do so by downloading it from the Apache Spark download page. If you want to use every Spark operator and your Hadoop cluster does not have 1.6 or above, then it needs to be installed on the cluster manually. See the table below for information for which Radoop Spark operators work with specific Spark versions. RapidMiner Radoop supports most Spark versions 1.6.0 and above. Below you can find detailed descriptions about the Spark requirements on the cluster. Java 8 on the cluster nodes (necessary for applying most RapidMiner models in-Hadoop and using Process Pushdown operators).a distributed data warehouse system (Hive or Impala). ![]()
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |