QueueLogger and Python JSON Logger

Using QueueLogger with Python JSON Logger

When logging from multiple processes (via multiprocessing module), using QueueHandler is one approach with Python 2.

QueueHandler however sets exc_info attribute of a LogRecord to None since it is not "pickleable" (more on this later). This becomes a problem when you use python-json-logger to format your logs as JSON since it relies on exc_info being set. The result is you don't get exc_info in your logs. This is no however no longer true since this PR was merged. Sample Code.

Solution #1: Modify Python JSON logger

The first solution is to look for exc_text which is set by the standard Formatter class as per my PR and set that as the value of exc_info which means at least we get the string representation of the traceback. This PR is now merged.

Solution #2: Subclass QueueHandler

The second solution is it to subclass logutils.queue.QueueHandler as follows and add pickling support for exc_info via tblib:

# Bring in support for serializing/deserializing tracebacks
# needed by QueueHandler
from tblib import pickling_support
pickling_support.install()


class QueueHandlerWithTraceback(logutils.queue.QueueHandler):
    """ QueueHandler with support for pickling/unpickling
        Tracebacks via tblib. We only override the prepare()
        method to *not* set `exc_info=None`
    """
    def __init__(self, *args, **kwargs):
        logutils.queue.QueueHandler.__init__(self, *args, **kwargs)

    def prepare(self, record):
        self.format(record)
        record.msg = record.message
        record.args = None
        return record

Instead of logutils.queue.QueueHandler, we will then use QueueHandlerWithTraceback instead above (Sample Code).

::...
免责声明:
当前网页内容, 由 大妈 ZoomQuiet 使用工具: ScrapBook :: Firefox Extension 从互联网中抓取并分享;
内容版权归原作者所有;
本人对内容的有效性/合法性不承担任何强制性责任.
若有不妥, 欢迎评注提醒:

蟒营®编程思维提高班 Python版/第11期 正在报名

精品小班/ 每期<42人


扫描报名: 101camp12py

蟒营®式 原创课程

伴你重享学习乐趣

官网: py.101.camp

Reactivate Joy by Self-teching with You


任何问题可先进入知识星球(免费)咨询:
FAQ

关注公众号, 持续获得相关各种咨询:
mainium


追问

任何问题, 随时邮件提问可也:
askdama@googlegroups.com


...::