-
Notifications
You must be signed in to change notification settings - Fork 254
Open
Description
Hello,
Recently we have noticed some hangs occurring, with no error messages.
Environment:
billiard version: 4.2.0
Python 3.10.12.
Cases which don't cause hanging, and cause hanging:
In test_cases.py
import time
def no_hang(id):
raise Exception
def no_hang_1(id):
time.sleep(1)
raise Exception
def no_hang_2(id):
try:
time.sleep(1)
raise Exception
except:
raise
def no_hang_3(id):
try:
pass
finally:
raise Exception
def no_hang_4(id):
try:
try:
time.sleep(1)
finally:
raise Exception
except:
pass
def hang(id):
try:
time.sleep(1)
finally:
raise Exception
def hang_2(id):
try:
try:
time.sleep(1)
finally:
raise Exception
except:
raise
In call_test_cases.py
import billiard as mp
# uncomment this line to use multiprocessing.
# No hangs occur, unlike with billiard
#import multiprocessing as mp
import func_that_fails
if __name__ == "__main__":
for f in [
test_cases.no_hang,
test_cases.no_hang_1,
test_cases.no_hang_2,
test_cases.no_hang_3,
test_cases.no_hang_4,
test_cases.hang,
test_cases.hang_2]:
print(f"testing {f.__name__}")
try:
spawned_thread = mp.get_context("spawn")
with spawned_thread.Pool(2) as pool:
pool.map(f, [[1], [2], [3], [4]])
except:
print("correctly caught exception from inside pool.map")
print("")
When using the standard multiprocessing library, no hangs occur. In order to reproduce this hang make sure the length of the iterable passed to map
is greater than the number of worker processes spawned with Pool
Metadata
Metadata
Assignees
Labels
No labels