[SPARK-5789][SQL]Throw a better error message if JsonRDD.parseJson encounters unrecoverable parsing errors.

Author: Yin Huai <yhuai@databricks.com>

Closes #4582 from yhuai/jsonErrorMessage and squashes the following commits:

152dbd4 [Yin Huai] Update error message.
1466256 [Yin Huai] Throw a better error message when a JSON object in the input dataset span multiple records (lines for files or strings for an RDD of strings).
This commit is contained in:
Yin Huai 2015-02-13 13:51:06 -08:00 committed by Michael Armbrust
parent 2cbb3e433a
commit 2e0c084528

View file

@ -306,6 +306,10 @@ private[sql] object JsonRDD extends Logging {
val parsed = mapper.readValue(record, classOf[Object]) match {
case map: java.util.Map[_, _] => scalafy(map).asInstanceOf[Map[String, Any]] :: Nil
case list: java.util.List[_] => scalafy(list).asInstanceOf[Seq[Map[String, Any]]]
case _ =>
sys.error(
s"Failed to parse record $record. Please make sure that each line of the file " +
"(or each string in the RDD) is a valid JSON object or an array of JSON objects.")
}
parsed