_pickle.PicklingError: Can't pickle

[sjsy]

Hi, when I ran the test unit on my system, I got the following errors:

_pickle.PicklingError: Can't pickle <function Streamer.__read_process at 0x000002012024BF28>: attribute lookup Streamer.__read_process on multitables failed

Please help me.

[ghcollin]

What is the full output? (stack trace and any HDF5 errors)

[QCmonk]

I have a similiar problem. I have attached the output of running the testing suite.
error_log.txt

[ghcollin]

Thanks for the error log. I think I may have found that the problem lies in multitables. It seems older versions of python don't support pickling static methods of classes.

If you are running a version older than 3.5.2, can you try finding where "multitables.py" is installed on your system and replacing it with the version in this link?

And because I found a regression in tftables when using python3 while debugging this, you may want to update "tftables.py" to the version in the link as well, if the unit tests still fail.

https://gist.github.com/ghcollin/7fbfaa78b540b4ab9320c78134f110f1

[QCmonk]

Thank you for the rapid debug. I have done as you suggested, updating both multitables and tftables using the suggested link. Unfortunately I am still failing the unit tests however (log file attached, I doubt the tensorflow warnings are important but it is true heresy to omit such things from an error log). The testing seems to hang which is perhaps the recursion issue? I'll try running in a >=3.5.2 environment and will update iff it works

error_log_2.txt

[ghcollin]

It seems you're running on Windows. In theory, multitables should run on Windows, but I've never set up a VM to try it out. Windows and Linux/OSX have different threading/processing models, so it's possible there is some deeper incompatibility here.

Could you try downloading and running the unit tests for multitables, just to see if it can isolate the problem? You can find them here: https://github.com/ghcollin/multitables/blob/master/multitables_test.py

Could you also find out what version of python you're running? Just in case I need to start debugging that particular version.

Thanks.

[QCmonk]

Ah of course. While I would ideally change my OS I am sadly constrained to Windows in this instance. I am running python 3.5.4 and the multitables test outputs the attached log file.

multitables_log.txt

The rval assertion on line 215 of multitables.py seems to be the root of the problem.

[ghcollin]

Sorry I haven't had a lot of time to look into this lately. It's a definite possibility that the way windows handles shared memory or condition variables is causing the corruption of the internal queues in multitables, as that assertion should never trigger if the queue is properly synchronised. I should hopefully have more time to look into this in a few weeks.