QueueLogger and Python JSON Logger

Using QueueLogger with Python JSON Logger

When logging from multiple processes (via multiprocessing module), using QueueHandler is one approach with Python 2.

QueueHandler however sets exc_info attribute of a LogRecord to None since it is not "pickleable" (more on this later). This becomes a problem when you use python-json-logger to format your logs as JSON since it relies on exc_info being set. The result is you don't get exc_info in your logs. This is no however no longer true since this PR was merged. Sample Code.

Solution #1: Modify Python JSON logger

The first solution is to look for exc_text which is set by the standard Formatter class as per my PR and set that as the value of exc_info which means at least we get the string representation of the traceback. This PR is now merged.

Solution #2: Subclass QueueHandler

The second solution is it to subclass logutils.queue.QueueHandler as follows and add pickling support for exc_info via tblib:

# Bring in support for serializing/deserializing tracebacks
# needed by QueueHandler
from tblib import pickling_support
pickling_support.install()


class QueueHandlerWithTraceback(logutils.queue.QueueHandler):
    """ QueueHandler with support for pickling/unpickling
        Tracebacks via tblib. We only override the prepare()
        method to *not* set `exc_info=None`
    """
    def __init__(self, *args, **kwargs):
        logutils.queue.QueueHandler.__init__(self, *args, **kwargs)

    def prepare(self, record):
        self.format(record)
        record.msg = record.message
        record.args = None
        return record

Instead of logutils.queue.QueueHandler, we will then use QueueHandlerWithTraceback instead above (Sample Code).

::...
免责声明:
当前网页内容, 由 大妈 ZoomQuiet 使用工具: ScrapBook :: Firefox Extension 人工从互联网中收集并分享;
内容版权归原作者所有;
本人对内容的有效性/合法性不承担任何强制性责任.
若有不妥, 欢迎评注提醒:

或是邮件反馈可也:
askdama[AT]googlegroups.com



自怼圈/年番新

DU21.4
关于 ~ DebugUself with DAMA ;-)


关注公众号, 持续获得相关各种嗯哼:
zoomquiet


粤ICP备18025058号-1
公安备案号: 44049002000656 ...::