Reynold Xin
9db1e50344
Revert "Merge pull request #841 from rxin/json"
...
This reverts commit 1fb1b09928
, reversing
changes made to c69c48947d
.
2013-08-26 11:05:14 -07:00
Jey Kottalam
a9db1b7b6e
Upgrade SBT IDE project generators
2013-08-23 10:27:18 -07:00
Jey Kottalam
b7f9e6374a
Fix SBT generation of IDE project files
2013-08-23 10:26:37 -07:00
Jey Kottalam
281b6c5f28
Re-add removed dependency on 'commons-daemon'
...
Fixes SBT build under Hadoop 0.23.9 and 2.0.4
2013-08-22 15:45:45 -07:00
Matei Zaharia
ae8ba83ef2
Merge pull request #855 from jey/update-build-docs
...
Update build docs
2013-08-22 10:14:54 -07:00
Matei Zaharia
8a36fd09dd
Merge pull request #854 from markhamstra/pomUpdate
...
Synced sbt and maven builds to use the same dependencies, etc.
2013-08-22 10:13:35 -07:00
Jey Kottalam
f9cc1fbf27
Remove references to unsupported Hadoop versions
2013-08-21 17:14:36 -07:00
Mark Hamstra
ff6f1b0500
Synced sbt and maven builds
2013-08-21 13:50:24 -07:00
Reynold Xin
af602ba9d3
Downgraded default build hadoop version to 1.0.4.
2013-08-21 11:38:24 -07:00
Matei Zaharia
aa2b89d98d
Merge remote-tracking branch 'jey/hadoop-agnostic'
...
Conflicts:
core/src/main/scala/spark/PairRDDFunctions.scala
2013-08-20 10:14:15 -07:00
Jey Kottalam
6f6944c807
Update SBT build to use simpler fix for Hadoop 0.23.9
2013-08-19 12:33:13 -07:00
Jey Kottalam
67b593607c
Rename YARN build flag to SPARK_WITH_YARN
2013-08-16 14:00:05 -07:00
Jey Kottalam
b1d99744a8
Fix SBT build under Hadoop 0.23.x
2013-08-16 13:50:12 -07:00
Jey Kottalam
8add2d7a59
Fix repl/assembly when YARN enabled
2013-08-16 13:50:12 -07:00
Jey Kottalam
3f98eff63a
Allow make-distribution.sh to specify Hadoop version used
2013-08-16 13:50:09 -07:00
Reynold Xin
c961c19b7b
Use the JSON formatter from Scala library and removed dependency on lift-json.
...
It made the JSON creation slightly more complicated, but reduces one external dependency. The scala library also properly escape "/" (which lift-json doesn't).
2013-08-15 18:23:01 -07:00
Jey Kottalam
a0f0848463
Update default version of Hadoop to 1.2.1
2013-08-15 16:50:37 -07:00
Jey Kottalam
cb4ef19214
yarn support
2013-08-15 16:50:37 -07:00
Jey Kottalam
273b499b9a
yarn sbt
2013-08-15 16:50:37 -07:00
Jey Kottalam
69c3bbf688
dynamically detect hadoop version
2013-08-15 16:50:37 -07:00
Matei Zaharia
d9588183fa
Update to Mesos 0.12.1
2013-08-13 18:51:35 -07:00
jerryshao
320e87e7ab
Add MetricsServlet for Spark metrics system
2013-08-12 13:23:23 +08:00
Matei Zaharia
dce5e47435
Merge pull request #800 from dlyubimov/HBASE_VERSION
...
Pull HBASE_VERSION in the head of sbt build
2013-08-09 21:53:45 -07:00
Matei Zaharia
cd247ba5bb
Merge pull request #786 from shivaram/mllib-java
...
Java fixes, tests and examples for ALS, KMeans
2013-08-09 20:41:13 -07:00
Dmitriy Lyubimov
27f674f82b
fewer words
2013-08-09 13:54:41 -07:00
Dmitriy Lyubimov
ae95b57469
Pull HBASE_VERSION in the head of sbt build
2013-08-09 12:45:18 -07:00
Matei Zaharia
5a4003c1ac
Update to Chill 0.3.1
2013-08-08 13:30:27 -07:00
Shivaram Venkataraman
471fbadd0c
Java examples, tests for KMeans and ALS
...
- Changes ALS to accept RDD[Rating] instead of (Int, Int, Double) making it
easier to call from Java
- Renames class methods from `train` to `run` to enable static methods to be
called from Java.
- Add unit tests which check if both static / class methods can be called.
- Also add examples which port the main() function in ALS, KMeans to the
examples project.
Couple of minor changes to existing code:
- Add a toJavaRDD method in RDD to convert scala RDD to java RDD easily
- Workaround a bug where using double[] from Java leads to class cast exception in
KMeans init
2013-08-06 15:43:46 -07:00
Matei Zaharia
e466a55a6b
Revert Mesos version to 0.9 since the 0.12 artifact has target Java 7
2013-08-01 15:45:21 -07:00
Matei Zaharia
b2b86c2575
Merge pull request #753 from shivaram/glm-refactor
...
Build changes for ML lib
2013-07-31 15:51:39 -07:00
Matei Zaharia
14bf2fe039
Merge pull request #749 from benh/spark-executor-uri
...
Added property 'spark.executor.uri' for launching on Mesos.
2013-07-31 14:18:16 -07:00
Shivaram Venkataraman
15fd0d619d
Add mllib, bagel to repl dependencies
...
Also don't build an assembly jar for them
2013-07-30 18:31:11 -07:00
Reynold Xin
3b1ced83fb
Exclude older version of Snappy in streaming and examples.
2013-07-30 17:25:36 -07:00
Reynold Xin
368c58eac5
Merge branch 'lazy_file_open' of github.com:lyogavin/spark into compression
...
Conflicts:
project/SparkBuild.scala
2013-07-30 16:04:18 -07:00
Shivaram Venkataraman
48851d4dd9
Add bagel, mllib to SBT assembly.
...
Also add jblas dependency to mllib pom.xml
2013-07-30 14:03:15 -07:00
Benjamin Hindman
f6f46455eb
Added property 'spark.executor.uri' for launching on Mesos without
...
requiring Spark to be installed. Using 'make_distribution.sh' a user
can put a Spark distribution at a URI supported by Mesos (e.g.,
'hdfs://...') and then set that when launching their job. Also added
SPARK_EXECUTOR_URI for the REPL.
2013-07-29 23:32:52 -07:00
ryanlecompte
8e0939f5a9
refactor Kryo serializer support to use chill/chill-java
2013-07-24 20:43:57 -07:00
jerryshao
5730193e0c
Fix some typos
2013-07-24 14:57:47 +08:00
jerryshao
576528f0f9
Add dependency of Codahale's metrics library
2013-07-24 14:57:46 +08:00
Josh Rosen
c83680434b
Add JavaAPICompletenessChecker.
...
This is used to find methods in the Scala API that
need to be ported to the Java API. To use it:
./run spark.tools.JavaAPICompletenessChecker
Conflicts:
project/SparkBuild.scala
run
run2.cmd
2013-07-22 16:11:49 -07:00
Liang-Chi Hsieh
d1738d72ba
also exclude asm for hadoop2. hadoop1 looks like no need to do that too.
2013-07-20 00:37:24 +08:00
Liang-Chi Hsieh
3aad452653
fix a bug in build process that pulls in two versionf of ASM.
2013-07-19 02:29:46 +08:00
Matei Zaharia
cad48edb70
Merge pull request #708 from ScrapCodes/dependencies-upgrade
...
Dependency upgrade Akka 2.0.3 -> 2.0.5
2013-07-16 21:41:28 -07:00
Matei Zaharia
af3c9d5042
Add Apache license headers and LICENSE and NOTICE files
2013-07-16 17:21:33 -07:00
Prashant Sharma
2748e73eb9
Dependency upgrade Akka 2.0.3 -> 2.0.5
2013-07-16 16:08:46 +05:30
Matei Zaharia
668b0dc6a7
Merge branch 'master' of github.com:mesos/spark
2013-07-13 19:10:46 -07:00
Matei Zaharia
cd28d9c147
Merge remote-tracking branch 'origin/pr/662'
...
Conflicts:
bin/compute-classpath.sh
2013-07-13 19:10:00 -07:00
seanm
c4d5b01e44
changing com.google.code.findbugs maven coordinates
2013-07-13 14:56:23 -06:00
Matei Zaharia
3cc6818f13
Merge pull request #668 from shimingfei/guava-14.0.1
...
update guava version from 11.0.1 to 14.0.1
2013-07-06 19:51:20 -07:00
Matei Zaharia
1ffadb2d9e
Merge remote-tracking branch 'pwendell/ui-updates'
...
Conflicts:
core/src/main/scala/spark/scheduler/DAGScheduler.scala
core/src/main/scala/spark/util/AkkaUtils.scala
pom.xml
2013-07-06 15:51:41 -07:00