semaphore_tracker is not reused by child processes

[tomMoral]

BPO	36668
Nosy	@pitrou, @applio, @tomMoral
PRs	bpo-36668: FIX reuse semaphore tracker for child processes #5172

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2019-04-24.19:46:31.192>
created_at = <Date 2019-04-19.13:44:01.381>
labels = ['3.8', 'type-bug', 'library']
title = 'semaphore_tracker is not reused by child processes'
updated_at = <Date 2019-04-24.19:46:31.192>
user = 'https://github.com/tomMoral'

bugs.python.org fields:

activity = <Date 2019-04-24.19:46:31.192>
actor = 'pitrou'
assignee = 'none'
closed = True
closed_date = <Date 2019-04-24.19:46:31.192>
closer = 'pitrou'
components = ['Library (Lib)']
creation = <Date 2019-04-19.13:44:01.381>
creator = 'tomMoral'
dependencies = []
files = []
hgrepos = []
issue_num = 36668
keywords = ['patch']
message_count = 2.0
messages = ['340543', '340804']
nosy_count = 3.0
nosy_names = ['pitrou', 'davin', 'tomMoral']
pr_nums = ['5172']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue36668'
versions = ['Python 3.8']

[tomMoral]

The current implementation of the semaphore_tracker creates a new process for each children.

The easy fix would be to pass the _pid to the children but the current mechanism to check if the semaphore_tracker is alive relies on waitpid which cannot be used in child processes (the semaphore_tracker is only a sibling of these processes). The main issue is to have a reliable check that either:

The pipe is open. This is what is done here by sending a message. I don't know if there is a more efficient way to check it.
Check that a given pid is alive. As we cannot rely on waitpid, I don't see an efficient mechanism.

I propose to add a PROBE command in the semaphore tracker. When the pipe is closed, the send command will fail, meaning that the semaphore tracker is down.

[pitrou]

New changeset 004b93e by Antoine Pitrou (Thomas Moreau) in branch 'master':
bpo-36668: FIX reuse semaphore tracker for child processes (bpo-5172)
004b93e