Ali Ghodsi
05a0df2b9e
Makes Spark SIMR ready.
2013-10-24 11:59:51 -07:00
Aaron Davidson
74737264c4
Spark shell exits if it cannot create SparkContext
...
Mainly, this occurs if you provide a messed up MASTER url (one that doesn't match one
of our regexes). Previously, we would default to Mesos, fail, and then start the shell
anyway, except that any Spark command would fail.
2013-10-17 18:51:19 -07:00
Patrick Wendell
aa9fb84994
Merging build changes in from 0.8
2013-10-05 22:07:00 -07:00
Patrick Wendell
e2ff59af72
Bug fix in master build
2013-09-26 13:06:51 -07:00
Patrick Wendell
6079721fa1
Update build version in master
2013-09-24 11:41:51 -07:00
Jey Kottalam
30a32c8335
Minor YARN build cleanups
2013-09-06 11:31:16 -07:00
Matei Zaharia
f586c8ef38
Updated LICENSE with third-party licenses
2013-09-02 16:43:06 -07:00
Matei Zaharia
0a8cc30921
Move some classes to more appropriate packages:
...
* RDD, *RDDFunctions -> org.apache.spark.rdd
* Utils, ClosureCleaner, SizeEstimator -> org.apache.spark.util
* JavaSerializer, KryoSerializer -> org.apache.spark.serializer
2013-09-01 14:13:16 -07:00
Matei Zaharia
5701eb92c7
Fix some URLs
2013-09-01 14:13:16 -07:00
Matei Zaharia
46eecd110a
Initial work to rename package to org.apache.spark
2013-09-01 14:13:13 -07:00
Mark Hamstra
ff6f1b0500
Synced sbt and maven builds
2013-08-21 13:50:24 -07:00
Jey Kottalam
23f4622aff
Remove redundant dependencies from POMs
2013-08-18 18:53:57 -07:00
Jey Kottalam
c1e547bb7f
Updates to repl and example POMs to match SBT build
2013-08-16 13:50:12 -07:00
Jey Kottalam
ad580b94d5
Maven build now also works with YARN
2013-08-16 13:50:12 -07:00
Jey Kottalam
11b42a84db
Maven build now works with CDH hadoop-2.0.0-mr1
2013-08-16 13:50:12 -07:00
Jey Kottalam
353fab2440
Initial changes to make Maven build agnostic of hadoop version
2013-08-16 13:50:12 -07:00
Shivaram Venkataraman
a1227708e9
Set SPARK_CLASSPATH for maven repl tests
2013-08-13 20:06:47 -07:00
Benjamin Hindman
7bdafa918a
Format cleanup.
2013-07-30 17:01:00 -07:00
Benjamin Hindman
f6f46455eb
Added property 'spark.executor.uri' for launching on Mesos without
...
requiring Spark to be installed. Using 'make_distribution.sh' a user
can put a Spark distribution at a URI supported by Mesos (e.g.,
'hdfs://...') and then set that when launching their job. Also added
SPARK_EXECUTOR_URI for the REPL.
2013-07-29 23:32:52 -07:00
Matei Zaharia
af3c9d5042
Add Apache license headers and LICENSE and NOTICE files
2013-07-16 17:21:33 -07:00
Matei Zaharia
2bd04c3513
Formatting
2013-06-25 18:37:14 -04:00
Matei Zaharia
f2263350ed
Added a local-cluster mode test to ReplSuite
2013-06-25 18:35:35 -04:00
Matei Zaharia
0e0f9d3069
Fix search path for REPL class loader to really find added JARs
2013-06-22 17:44:04 -07:00
Matei Zaharia
b5df1cd668
ADD_JARS environment variable for spark-shell
2013-06-22 17:14:44 -07:00
Matei Zaharia
7902baddc7
Update ASM to version 4.0
2013-06-19 13:34:30 +02:00
Jey Kottalam
207afe4088
Remove spark-repl's extraneous dependency on spark-streaming
2013-05-01 16:57:31 -07:00
Mridul Muralidharan
afee902443
Attempt to fix streaming test failures after yarn branch merge
2013-04-28 22:26:45 +05:30
Mridul Muralidharan
dd515ca3ee
Attempt at fixing merge conflict
2013-04-24 09:24:17 +05:30
Matei Zaharia
65caa8f711
Merge remote-tracking branch 'jey/bump-development-version-to-0.8.0'
...
Conflicts:
docs/_config.yml
project/SparkBuild.scala
2013-04-08 12:43:17 -04:00
Mridul Muralidharan
6798a09df8
Add support for building against hadoop2-yarn : adding new maven profile for it
2013-04-07 17:47:38 +05:30
Jey Kottalam
bc8ba222ff
Bump development version to 0.8.0
2013-03-28 15:42:01 -07:00
Mikhail Bautin
7fd2708eda
Add a log4j compile dependency to fix build in IntelliJ
...
Also rename parent project to spark-parent (otherwise it shows up as
"parent" in IntelliJ, which is very confusing).
2013-03-15 11:41:51 -07:00
Mark Hamstra
8b06b359da
bump version to 0.7.1-SNAPSHOT in the subproject poms to keep the maven build building.
2013-02-28 23:34:34 -08:00
Matei Zaharia
db9b90fdbd
Change version to 0.7.1-SNAPSHOT for development branch
2013-02-27 09:15:26 -08:00
Mikhail Bautin
fe3eceab57
Remove activation of profiles by default
...
See the discussion at https://github.com/mesos/spark/pull/355 for why
default profile activation is a problem.
2013-01-31 13:30:41 -08:00
Stephen Haberman
7dfb82a992
Replace old 'master' term with 'driver'.
2013-01-25 11:03:00 -06:00
Mikhail Bautin
325297e5c3
Add an Avro dependency to REPL to make it compile with Hadoop 2
2013-01-22 18:11:51 -08:00
Matei Zaharia
6e3754bf47
Add Maven build file for streaming, and fix some issues in SBT file
...
As part of this, changed our Scala 2.9.2 Kafka library to be available
as a local Maven repository, following the example in
(http://blog.dub.podval.org/2010/01/maven-in-project-repository.html )
2013-01-20 19:22:24 -08:00
Tathagata Das
cd1521cfdb
Merge branch 'master' into streaming
...
Conflicts:
core/src/main/scala/spark/rdd/CoGroupedRDD.scala
core/src/main/scala/spark/rdd/FilteredRDD.scala
docs/_layouts/global.html
docs/index.md
run
2013-01-15 12:08:51 -08:00
Shivaram Venkataraman
bbc56d85ed
Rename environment variable for hadoop profiles to hadoopVersion
2013-01-12 15:24:13 -08:00
Shivaram Venkataraman
9262522306
Activate hadoop2 profile in pom.xml with -Dhadoop=2
2013-01-10 22:07:34 -08:00
Shivaram Venkataraman
f7adb382ac
Activate hadoop1 if property hadoop is missing. hadoop2 can be activated now
...
by using -Dhadoop -Phadoop2.
2013-01-08 03:19:43 -08:00
Shivaram Venkataraman
4bbe07e5ec
Activate hadoop1 profile by default for maven builds
2013-01-07 17:46:22 -08:00
Tathagata Das
4719e6d8fe
Changed locations for unit test logs.
2013-01-07 16:06:07 -08:00
Thomas Dudziak
4af6cad37a
Fixed repl maven build to produce artifacts with the appropriate hadoop classifier and extracted repl fat-jar and debian packaging into a separate project to make Maven happy
2012-12-18 12:08:19 -08:00
Thomas Dudziak
c1d15ae3d5
Shaded repl jar for hadoop1 profile needs to include hadoop classes
2012-12-10 15:06:28 -08:00
Matei Zaharia
a9ea14d6e7
Merge pull request #318 from tomdz/master
...
Minor tweaks to the debian build
2012-12-10 10:59:41 -08:00
Matei Zaharia
ccff0a089a
Use the same output directories that SBT had in subprojects
...
This will make it easier to make the "run" script work with a Maven build
2012-12-10 10:58:56 -08:00
Thomas Dudziak
0e5b1f7981
Minor tweaks to the debian build
2012-12-10 10:30:30 -08:00
Thomas Dudziak
3b643e86bc
Updated versions in the pom.xml files to match current master
2012-11-27 17:50:42 -08:00