Project Information
- Repository: https://github.com/holgern/beem
- Project Name: Beem
- Publisher: @holger80
The blockchain class enables to access the Steem blockchain and read data from it. blockchain.blocks() can be used to stream a number of blocks from an API node. blockchain.blocks() has an option to use multi-threading to speed up fetching the blocks from API nodes considerably.
Expected behavior
blockchain.blocks(start=start, stop=stop, threading=True) should return all requested blocks.
Actual behavior
blockchain.blocks(start=start, stop=stop, threading=True) may raise a RuntimeError due to a race condition when a dictionary is changed while being iterated on. The streaming of the blocks stops at non-deterministic block numbers.
Traceback (most recent call last):
File "block_race_condition.py", line 9, in <module>
for block in b.blocks(start=10000, stop=20000, threading=True, thread_num=8):
File "/usr/local/lib/python3.6/site-packages/beem/blockchain.py", line 241, in blocks
results = [r.result() for r in as_completed(futures)]
File "/usr/local/lib/python3.6/site-packages/beem/blockchain.py", line 241, in <listcomp>
results = [r.result() for r in as_completed(futures)]
File "/usr/local/lib/python3.6/concurrent/futures/_base.py", line 425, in result
return self.__get_result()
File "/usr/local/lib/python3.6/concurrent/futures/_base.py", line 384, in __get_result
raise self._exception
File "/usr/local/lib/python3.6/concurrent/futures/thread.py", line 56, in run
result = self.fn(*self.args, **self.kwargs)
File "/usr/local/lib/python3.6/site-packages/beem/block.py", line 68, in __init__
steem_instance=steem_instance
File "/usr/local/lib/python3.6/site-packages/beem/blockchainobject.py", line 138, in __init__
self.cache()
File "/usr/local/lib/python3.6/site-packages/beem/blockchainobject.py", line 163, in cache
BlockchainObject._cache[self.get(self.id_item)] = self
File "/usr/local/lib/python3.6/site-packages/beem/blockchainobject.py", line 33, in __setitem__
self.clear_expired_items()
File "/usr/local/lib/python3.6/site-packages/beem/blockchainobject.py", line 48, in clear_expired_items
for key in self.keys():
RuntimeError: dictionary changed size during iteration
The corresponding code part is in https://github.com/holgern/beem/blob/bd8cf746f7a3287654c9084da6e641bb4838e7ca/beem/blockchainobject.py#L47
self.keys() changes while being iterated on.
How to reproduce
#!/usr/bin/python
from beem.blockchain import Blockchain
from beem import Steem
import sys
b = Blockchain()
for block in b.blocks(start=10000, stop=20000, threading=True, thread_num=8):
sys.stdout.write("%s (%s)\r" % (block['id'], block['timestamp']))
This code sometimes results in the RuntimeError above. I've been using the same structure in a long running loop and hit the RuntimeError by chance every now and then. With the example code above, the block number at which the error occurs changes and it may need dozens of runs until it actually fails. However, the problem can be articifially amplified by adding a time.sleep(1) into the corresponding for key in self.keys(): loop:
diff --git a/beem/blockchainobject.py b/beem/blockchainobject.py
index 03d22de..09d56ca 100644
--- a/beem/blockchainobject.py
+++ b/beem/blockchainobject.py
@@ -9,6 +9,7 @@ from beemgraphenebase.py23 import bytes_types, integer_types, string_types, text
from beem.instance import shared_steem_instance
from datetime import datetime, timedelta
import json
+import time
@python_2_unicode_compatible
@@ -46,6 +47,7 @@ class ObjectCache(dict):
keys = []
for key in self.keys():
keys.append(key)
+ time.sleep(1)
for key in keys:
value = dict.__getitem__(self, key)
if datetime.utcnow() >= value["expires"]:
With this change, the library spends significant amounts of time in the iteration loop on self.keys(). Technically it does not make a difference apart from being terribly slow now. With this in place, the above exception can be triggered with the given code sample within seconds.
The exception is not raised when multi-threading is disabled via threading=False. To my understanding, multiple threads are accessing the same ObjectCache() instance. Disabling the cache via the use_cache=False flag in the BlockchainObject constructor defaults for testing purposes did not work:
Traceback (most recent call last):
File "block_race_condition.py", line 10, in <module>
sys.stdout.write("%s (%s)\r" % (block['id'], block['timestamp']))
File "/usr/local/lib/python3.6/site-packages/beem/blockchainobject.py", line 173, in __getitem__
self.refresh()
File "/usr/local/lib/python3.6/site-packages/beem/block.py", line 76, in refresh
self.identifier = int(self.identifier)
TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'
Using a copy of self.keys() via for key in self.keys()[:]: will only shift the problem, because two instances may still try to access/delete the same cache entries independently.
Environment
# beempy --version
beempy, version 0.19.32
# python --version
Python 3.6.5
GitHub Account
https://github.com/crokkon
A Github issue has been created:
https://github.com/holgern/beem/issues/16