spark-instrumented-optimizer/python
Davies Liu d48935400c [SPARK-16077] [PYSPARK] catch the exception from pickle.whichmodule()
## What changes were proposed in this pull request?

In the case that we don't know which module a object came from, will call pickle.whichmodule() to go throught all the loaded modules to find the object, which could fail because some modules, for example, six, see https://bitbucket.org/gutworth/six/issues/63/importing-six-breaks-pickling

We should ignore the exception here, use `__main__` as the module name (it means we can't find the module).

## How was this patch tested?

Manual tested. Can't have a unit test for this.

Author: Davies Liu <davies@databricks.com>

Closes #13788 from davies/whichmodule.
2016-06-24 14:35:34 -07:00
..
docs [SPARK-15162][SPARK-15164][PYSPARK][DOCS][ML] update some pydocs 2016-06-22 11:54:49 +02:00
lib [SPARK-15061][PYSPARK] Upgrade to Py4J 0.10.1 2016-05-13 08:59:18 +01:00
pyspark [SPARK-16077] [PYSPARK] catch the exception from pickle.whichmodule() 2016-06-24 14:35:34 -07:00
test_support [SPARK-14555] First cut of Python API for Structured Streaming 2016-04-20 10:32:01 -07:00
.gitignore [SPARK-3946] gitignore in /python includes wrong directory 2014-10-14 14:09:39 -07:00
pylintrc [SPARK-13596][BUILD] Move misc top-level build files into appropriate subdirs 2016-03-07 14:48:02 -08:00
run-tests [SPARK-8583] [SPARK-5482] [BUILD] Refactor python/run-tests to integrate with dev/run-tests module system 2015-06-27 20:24:34 -07:00
run-tests.py [SPARK-13579][BUILD] Stop building the main Spark assembly. 2016-04-04 16:52:22 -07:00