我最终的目标是将stdout
子进程从几个子进程重定向到一些队列,并将它们打印到某个地方(也许在一个小的GUI中)。
第一步是将其子类Queue
化为行为类似于的对象stdout
。但这就是我被困住的地方。Queue
在Python v3.5中似乎无法将多处理子类化。
# This is a Queue that behaves like stdout
# Unfortunately, doesn't work in Python 3.5 :-(
class StdoutQueue(Queue):
def __init__(self,*args,**kwargs):
Queue.__init__(self,*args,**kwargs, ctx='')
def write(self,msg):
self.put(msg)
def flush(self):
sys.__stdout__.flush()
我在以下文章中找到了此代码段(当时可能还不存在Python 3.5):Python多处理将子进程的stdout重定向到Tkinter Text
在Python v3.5中,当子类化多处理Queue
类时,您会偶然发现奇怪的错误消息。我发现了两个描述该问题的错误报告:
https://bugs.python.org/issue21367
https://bugs.python.org/issue19895
我有两个问题:
编辑:
当您尝试对Queue
此处找到的类进行子类化时,存在一个已知问题:
from multiprocessing import Queue # <- known issue: you cannot subclass
# this Queue class, because it is
# not a genuine python class.
但是以下应该起作用:
from multiprocessing.queues import Queue # <- from this Queue class, you
# should be able to make a
# subclass. But Python 3.5
# refuses :-(
可悲的是,即使那样在Python v3.5中也不起作用。您收到以下错误:
C:\Users\..\myFolder > python myTest.py
Traceback (most recent call last):
File "myTest.py", line 49, in <module>
q = StdoutQueue()
File "myTest.py", line 22, in __init__
super(StdoutQueue,self).__init__(*args,**kwargs)
TypeError: __init__() missing 1 required keyword-only argument: 'ctx'
编辑:
感谢Darth Kotik解决问题!这是完整的代码,并使用他的解决方案进行了更新。现在可以了。
import sys
import time
import multiprocessing as mp
import multiprocessing.queues as mpq
from threading import Thread
from tkinter import *
'''-------------------------------------------------------------------'''
''' SUBCLASSING THE MULTIPROCESSING QUEUE '''
''' '''
''' ..and make it behave as a general stdout io '''
'''-------------------------------------------------------------------'''
# The StdoutQueue is a Queue that behaves like stdout.
# We will subclass the Queue class from the multiprocessing package
# and give it the typical stdout functions.
#
# (1) First issue
# Subclassing multiprocessing.Queue or multiprocessing.SimpleQueue
# will not work, because these classes are not genuine
# python classes.
# Therefore, you need to subclass multiprocessing.queues.Queue or
# multiprocessing.queues.SimpleQueue . This issue is known, and is not
# the reason for asking this question. But I mention it here, for
# completeness.
#
# (2) Second issue
# There is another problem that arises only in Python V5 (and beyond).
# When subclassing multiprocessing.queues.Queue, you have to provide
# a 'multiprocessing context'. Not doing that, leads to an obscure error
# message, which is in fact the main topic of this question. Darth Kotik
# solved it.
# His solution is visible in this code:
class StdoutQueue(mpq.Queue):
def __init__(self,*args,**kwargs):
ctx = mp.get_context()
super(StdoutQueue, self).__init__(*args, **kwargs, ctx=ctx)
def write(self,msg):
self.put(msg)
def flush(self):
sys.__stdout__.flush()
'''-------------------------------------------------------------------'''
''' TEST SETUP '''
'''-------------------------------------------------------------------'''
# This function takes the text widget and a queue as inputs.
# It functions by waiting on new data entering the queue, when it
# finds new data it will insert it into the text widget.
def text_catcher(text_widget,queue):
while True:
text_widget.insert(END, queue.get())
def test_child(q):
# This line only redirects stdout inside the current process
sys.stdout = q
# or sys.stdout = sys.__stdout__ if you want to print the child to the terminal
print('child running')
def test_parent(q):
# Again this only redirects inside the current (main) process
# commenting this like out will cause only the child to write to the widget
sys.stdout = q
print('parent running')
time.sleep(0.5)
mp.Process(target=test_child,args=(q,)).start()
if __name__ == '__main__':
gui_root = Tk()
gui_txt = Text(gui_root)
gui_txt.pack()
q = StdoutQueue()
gui_btn = Button(gui_root, text='Test', command=lambda:test_parent(q),)
gui_btn.pack()
# Instantiate and start the text monitor
monitor = Thread(target=text_catcher,args=(gui_txt,q))
monitor.daemon = True
monitor.start()
gui_root.mainloop()
>>> import multiprocessing
>>> type(multiprocessing.Queue)
<class 'method'>
AttributeError: module 'multiprocessing' has no attribute 'queues'
>>> import multiprocessing.queues
>>> type(multiprocessing.queues.Queue)
<class 'type'>
如您所见multiprocessing.Queue
,只是multiprocessing.queues.Queue
类的构造方法。如果您想让孩子上课,那就去做吧class MyQueue(multiprocessing.queues.Queue)
您可以在这里查看此方法的来源
编辑:好的。我现在知道了你的问题。如您在上面的链接中所见,multiprocessing.Queue
将ctx
参数传递给Queue。所以我设法自己通过__init__
方法来使它起作用。我不完全理解BaseContext
对象应该在哪里获得_name
属性,因此我手动传递了它。
def __init__(self,*args,**kwargs):
from multiprocessing.context import BaseContext
ctx = BaseContext()
ctx._name = "Name"
super(StdoutQueue,self).__init__(*args,**kwargs, ctx=ctx)
编辑2:原来文档在这里有一些关于上下文的信息。因此,您不必像我一样手动创建它
import multiprocessing
ctx = multiprocessing.get_context()
它将创建具有_name
设置的正确上下文(在特定情况下为“ fork”),您可以将其传递到队列中。
本文收集自互联网,转载请注明来源。
如有侵权,请联系 [email protected] 删除。
我来说两句