How to use rich logger without enforced wrapping?

[ssbarnea]

While replacing the standard python logger with a rich one I discovered that longer lines get wrapped and that normal lines do get extra padding at the end, both being undesired actions, especially as same logging result can be sent to files or other apps for further processing.

I tried adding width=0 to Console instance used for logger but that basically broke it and produced no output. None seems to be reserved to auto-detection. I also tried putting a very big value but that produced very long spam lines, even when called with console=False

LOGGING_CONSOLE = Console( file=sys.stderr, force_terminal=False, width=300) 

How can I make the rich logging not add extra padding and avoid any wrapping?

[willmcgugan]

If you are using RichHandler, then you can't change that behaviour. It's designed to format it for the terminal and wrapping is necessary for neat columns.

If you want to also log to a file, you can configure a FileHandler (or any other logging handler(s)) in conjunction with the Rich handler.

[ssbarnea]

As "final" console width cannot be always be known at the time logging happens (CI is best example, where you want to produce nice output, but you never known the width, as this will endup being rendered in a browser).

This means that we either need a new handler that does not wrap or another option to disable wrapping in current one. Considering the "0" value is not currently used, it could make sense to use it to disable wrapping. Still, as did not look on how is implemented yet, I will likely wait for you to say if is doable or not.

I do love the richhandler, it would a pity not to be able to make use of it.

[willmcgugan]

It's not possible to disable hard wrapping in the Rich logging handler because it would break the columns. If this is a requirement for you I would suggest implementing your own handler based on rich/logging.py.

[ssbarnea]

I was already able to created a custom FluidLogRender class that does not make use of Table(Columns) and had partial success with it.

Still, I am still facing some issues related to some wrapping coming from other places. For example I found that Text() has an implicit no_wrap=None (same as False) which is called from various places. I tried to hack Rich to temporary make no_wrap=True but it was not enough.

Maybe you could give me a hint on how to render the text without any wrapping to occur.

[Toreno96]

Thank you very much @ssbarnea!

While the rich as a whole is amazing library, I wasn't really fond of the RichHandler formatting (I simply wanted some coloring, that's all), and I gave up after trying for definitely too long to hack the original implementation.

You did amazing work at providing the solution 💪

[ssbarnea]

Thanks @Toreno96 for the kind word. To be honest I still hope I will be able to persuade @willmcgugan to get this fluidlogger into rich itself. The reason I want this is because packaging another library just for these few lines of code is not a joy.

It seems that these 85 lines did not need any maintenance in more than an year and I would be more than happy to be the one assigned to address issues if reported, just to avoid having to ask other projects to add an extra library. It is hard for me not to remember nightmares from npm registry where people created packages for even a 3 line function.

[Toreno96]

@willmcgugan Could you share what are the arguments against integrating this into rich? 🙏

[willmcgugan]

I'm not apposed to a fluid logger in principle. But frankly its not for me to say why it shouldn't go in, you will need to make the case why it should.

As soon as I add that, I will get issue after issue for requests for flags and options, because everyones needs differ.

@ssbarnea 's implementation is unnecessarily convoluted. You don't need to derive from RichHandler or create a log render object, when you can create a logging Handler of your own and print a Text instance . In a few lines of code you can have a custom handler that does exactly what you need. Start from the Python docs -- don't hack RichHandler, and it will be much easier.

Ultimately, its an uphill battle to expect me to maintain a FluidLogHandler, when rolling your own is trivial.

[davidparsson]

This is how i managed to print log lines that are colored by another formatter:

import logging from rich.console import Console from rich.text import Text class FluidRichHandler(logging.Handler): def __init__(self, console: Console) -> None: super().__init__() self._console = console def emit(self, record: logging.LogRecord) -> None: log_line = self.format(record) self._console.print(Text.from_ansi(log_line))

[anvaari]

Maybe simple workaround is this use custome handler like this. You can modify format to format other things to

import logging from rich.console import Console rprint = Console(soft_wrap=True).print class MyRichLogHandler(logging.Handler): LEVEL_MAPPING = { logging.DEBUG: "[blue]DEBUG[/blue]", logging.INFO: "[green]INFO[/green]", logging.WARNING: "[yellow]WARNING[/yellow]", logging.ERROR: "[red]ERROR[/red]", logging.CRITICAL: "[bold red]CRITICAL[/bold red]", } def emit(self, record): msg = self.format(record) rprint(msg) def format(self, record): levelname = self.LEVEL_MAPPING.get(record.levelno, str(record.levelno)) file_name_line = ( f"[link file://{record.filename}#{record.lineno}]" f"{record.filename}:{record.lineno}" f"[/link file://{record.filename}#{record.lineno}]") record.levelname = levelname record.filename = file_name_line return super().format(record)

You can modify format method to format other things too.

in this example if you set base_log_format = %(asctime)s - %(levelname)s in %(filename)s: %(message)s", the `filename wil be link to that number of error in the file which long belong to.

[demizer]

For those that want pretty output in the terminal AND Gitlab CI.

Can probably be cut down further but it works.

Set CI=true to limit table column width.

Edit: for python 3.12.

Warning: I haven't fully tested this with all the renderables, only with what my project is currently using (tables).

import logging import os from collections.abc import Callable from collections.abc import Iterable from datetime import datetime from logging import Handler from logging import LogRecord from types import ModuleType from typing import Any from typing import ClassVar from rich.console import Console as RichConsole from rich.console import ConsoleRenderable from rich.console import JustifyMethod from rich.console import NewLine from rich.highlighter import Highlighter from rich.highlighter import ReprHighlighter from rich.panel import Panel from rich.scope import render_scope from rich.segment import Segment from rich.style import Style from rich.styled import Styled from rich.text import Text from rich.traceback import Traceback FormatTimeCallable = Callable[[datetime], Text] """An example for monkey patching Rich to allow fluid output in terminal and CI and also keep pretty highlighting.  File links are removed since those break in Gitlab.  Gitlab issue that will never get fixed: https://gitlab.com/gitlab-org/gitlab-runner/-/issues/29365 """ class Console(RichConsole): """A modified version of Rich Console to remove file links in output for Gitlab.   From https://github.com/Textualize/rich/blob/master/rich/console.py#L593   """ def log( self, *objects: Any, sep: str = " ", end: str = "\n", style: str | Style | None = None, justify: JustifyMethod | None = None, emoji: bool | None = None, markup: bool | None = None, highlight: bool | None = None, log_locals: bool = False, _stack_offset: int = 1, ) -> None: """Log rich content to the terminal.   Args:  objects: Objects to log to the terminal.  sep: String to write between print data. Defaults to " ".  end: String to write at end of print data. Defaults to "\\\\n".  style: A style to apply to output. Defaults to None.  justify: One of "left", "right", "center", or "full". Defaults to ``None``.  emoji: Enable emoji code, or ``None`` to use console default. Defaults to None.  markup: Enable markup, or ``None`` to use console default. Defaults to None.  highlight: Enable automatic highlighting, or ``None`` to use console default. Defaults to None.  log_locals: Boolean to enable logging of locals where ``log()``  was called. Defaults to False.  _stack_offset: Offset of caller from end of call stack. Defaults to 1.  """ if not objects: objects = (NewLine(),) render_hooks = self._render_hooks[:] with self: renderables: list[ConsoleRenderable] = self._collect_renderables( objects, sep, end, justify=justify, emoji=emoji, markup=markup, highlight=highlight, ) if style is not None: renderables = [Styled(renderable, style) for renderable in renderables] filename, line_no, locals = self._caller_frame_info(_stack_offset) # link_path = None if filename.startswith("<") else os.path.abspath(filename) path = filename.rpartition(os.sep)[-1] if log_locals: locals_map = {key: value for key, value in locals.items() if not key.startswith("__")} renderables.append(render_scope(locals_map, title="[i]locals")) renderables = [ self._log_render( self, renderables, log_time=self.get_datetime(), path=path, line_no=line_no, # all because of this # link_path=link_path, ) ] for hook in render_hooks: renderables = hook.process_renderables(renderables) new_segments: list[Segment] = [] extend = new_segments.extend render = self.render render_options = self.options for renderable in renderables: extend(render(renderable, render_options)) buffer_extend = self._buffer.extend for line in Segment.split_and_crop_lines(new_segments, self.width, pad=False): buffer_extend(line) class FluidRichHandler(Handler): LEVEL_MAPPING = { logging.DEBUG: "[blue]DBUG[/blue]", logging.INFO: "[green]INFO[/green]", logging.WARNING: "[yellow]WARN[/yellow]", logging.ERROR: "[red]ERRR[/red]", logging.CRITICAL: "[bold red]CRIT[/bold red]", } HIGHLIGHTER_CLASS: ClassVar[type[Highlighter]] = ReprHighlighter KEYWORDS: ClassVar[list[str] | None] = [ "GET", "POST", "HEAD", "PUT", "DELETE", "OPTIONS", "TRACE", "PATCH", ] def __init__( self, console: Console, level: int | str = logging.NOTSET, *, enable_link_path: bool = True, highlighter: Highlighter | None = None, markup: bool = False, rich_tracebacks: bool = False, tracebacks_width: int | None = None, tracebacks_code_width: int = 88, tracebacks_extra_lines: int = 3, tracebacks_theme: str | None = None, tracebacks_word_wrap: bool = True, tracebacks_show_locals: bool = False, tracebacks_suppress: Iterable[str | ModuleType] = (), tracebacks_max_frames: int = 100, locals_max_length: int = 10, locals_max_string: int = 80, columns: int = -1, keywords: list[str] | None = None, ) -> None: super().__init__(level=level) self.console = console self.highlighter = highlighter or self.HIGHLIGHTER_CLASS() self.markup = markup self.enable_link_path = enable_link_path self.markup = markup self.rich_tracebacks = rich_tracebacks self.tracebacks_width = tracebacks_width self.tracebacks_extra_lines = tracebacks_extra_lines self.tracebacks_theme = tracebacks_theme self.tracebacks_word_wrap = tracebacks_word_wrap self.tracebacks_show_locals = tracebacks_show_locals self.tracebacks_suppress = tracebacks_suppress self.tracebacks_max_frames = tracebacks_max_frames self.tracebacks_code_width = tracebacks_code_width self.locals_max_length = locals_max_length self.locals_max_string = locals_max_string self.keywords = keywords if columns > 0: self.columns(columns) def columns(self, size: int) -> None: self.columns_size = size # rich will default to 80 columns if there isn't a tty # https://github.com/Textualize/rich/blob/master/docs/source/console.rst#terminal-detection os.environ["COLUMNS"] = str(size) def emit(self, record): message = self.format(record) traceback = None if self.rich_tracebacks and record.exc_info and record.exc_info != (None, None, None): exc_type, exc_value, exc_traceback = record.exc_info assert exc_type is not None assert exc_value is not None traceback = Traceback.from_exception( exc_type, exc_value, exc_traceback, width=self.tracebacks_width, code_width=self.tracebacks_code_width, extra_lines=self.tracebacks_extra_lines, theme=self.tracebacks_theme, word_wrap=self.tracebacks_word_wrap, show_locals=self.tracebacks_show_locals, locals_max_length=self.locals_max_length, locals_max_string=self.locals_max_string, suppress=self.tracebacks_suppress, max_frames=self.tracebacks_max_frames, ) message = record.getMessage() message_renderable = self.render_message( record, [message] if not traceback else [message, traceback], ) try: self.console.print(message_renderable, soft_wrap=True) except Exception: self.handleError(record) def render_message(self, record: LogRecord, message: list[str | Traceback]) -> "ConsoleRenderable": """Render message text in to Text.   Args:  record (LogRecord): logging Record.  message (str): String containing log message.   Returns:  ConsoleRenderable: Renderable to display log message.  """ message_text = Text() for elem in message: if isinstance(elem, str): use_markup = getattr(record, "markup", self.markup) message_text = Text.from_markup(elem) if use_markup else Text(record.msg) else: message_text += elem highlighter = getattr(record, "highlighter", self.highlighter) if highlighter: message_text = highlighter(message_text) if self.keywords is None: self.keywords = self.KEYWORDS if self.keywords: message_text.highlight_words(self.keywords, "logging.keyword") return message_text def format(self, record): record.levelname = self.LEVEL_MAPPING[getattr(logging, str(record.levelname))] return super().format(record) def banner(text: str) -> None: """Print a banner""" console.log(Panel(text), _stack_offset=2) if __name__ == "__main__": console = Console( soft_wrap=True, color_system="256", force_terminal=True, ) format = "%(asctime)s %(levelname)s \\[%(filename)s:%(lineno)s] :: %(message)s" sink = FluidRichHandler( console=console, level=logging.DEBUG, markup=True, rich_tracebacks=True, enable_link_path=False, ) if os.getenv("CI") == "true": sink.columns(150) logging.basicConfig(level=logging.INFO, format=format, datefmt="[%X]", handlers=[sink]) banner("link to https://alexandrite.app/") logging.info("A really long line with a long link that should be clickable (alexandrite.app): http://www.reallylong.link/rll/4QMpSmNQFPAzm6xssWDkgcED_nBklJunV2HwJKvBSjLOOIq3Pn/RvmOrS6lZmzNonqJ0X4Y_P4iBuLoMX3et/F9lebVVSnufNSssnWAiZUL1Q7aj9mnII9aQWQ2mhuPidyTYaOunxFGgKO8/NEzPfVjNDbNsZ61xhx0tQGfEerdwFtRagcaM0vI0XDLD2VwrRw/OOGxnPFH9HL_c5oe3w15WfFfALrqAP58/hI6Pm82wTzucsF08ak5Eh5A/6kt3q3FXPetygBQqxkVHqBt52dK0fYz4AShrsn4Rfj9Qc6KKN/HzJsUOV7nU6CSlo5BALkgUt5Uh0YhC_G5OQzGyNWC/qSZbTfL9mulULs8FM0s__JbLb949SP9nevrdMrJeznIPhPw9IHvbuJJtPNuaAcIVklk9r2mrf/8WoE3H8l_UKh175r_hvQaP4mOG2u91trDDlWiC5HeO2gemnpqhLX3GDYQuWUO8imD9hY3GtiqUMQ65STCluNolWkpJEO/6vXisSkC7RpoBqTSGBc3a9AeuK/F/NvcxZdZPLdkpcMnihcZjgFE31oed5Le25YxBd5EFEm_aXTXsjIJuUMuTZgcpiumYDnDosI9xxAVYeKt5z0AGeVr2tl3HH27Oi451qAgbMOgszdMT8Bpe0smUGXiKsPPKtwOh8RNqjX7XeiCDwlS_YN0MikEkqoB42hK87rOfQHm7ubPbVVIk8uv9d6sz66e/7lSsNg7ySfPMyhFAqSDvLdT5V2DWD7MAPyBJpqmcPyZPc3J9882fYlREgelmuKQRTGMzc36tV4WtOCnlH/Gq86Dc9Gq2ogZp2gzR9gNqiK1HaSYMiVkNdIpPYRGoPeUMAhJzaCfGFpGqvD6rl3bkOFv1q4DaAIynDJkUpgcrevR2Ds77rl9GeHbyHDRn4cqXI5PpVZKh5lVSiZ_PViXKQLW_e_w2UwCVxu5bSHiM28c/dRoZfU3VHBgYCp2PPgIYuMfS2reGhdn8ppHgSpXppn15iGw/_zVaLWJ2F0XbzoOBA4pRI5wmRxvIbNHEj3prswk4Ya_uXe3urInAaBK5oy5XmuO_hqr/3uyqb8v9ZmvnT_h_fxL3er_IVfsEb0Nx1ramqKZmkACbbnQ3nqibFmzs9EflzPvEudNPyim8QbIGDJ9ZflPSCFjOuqp_Na7SJKbKRMVOP1xsV_K_xMC6me6g1s0A79TXZPfNKJ1jH5VbmlwgnIoRGoFtLp10ecLuCfeYR2/yhsqlXz3kM7xNv09LyPOA5EmV9_64wlDjRCgxhZOZNvS4P5QP4vbnSU8PC6JKS/TVgywW/UZyyfPoUsi9ePeDXzRrvfKFpZnrPFySLz6VNiYjnEV09Zv9CXMIJaNbWGY98oQcDXwx3kHmDiYRaTF4SPgfyeJDl5hjt8LOQWZYMz_x07KW/5DZo1Dexf/3k4wNRxTytEsWlLVDJ20eN/lRDCckix_6CToi8oQA7jZP9j5elkjkPXDwm231_SSjgRZe52vWI1vErr8KgpZ5GKJLSVyUkPOjVK07jj/7A0mDGUdI_x7lyeUjIwTlZvtEf0YWpsZJpOoZ7Kb2kKApAVRV2YdZspMRmjP6suWnThSgQaLO9m5dFH/7i2oaLRBAK8158k0JhPMMUweI8UNq/8UwzHqBYw6IZ1eRKgQMyFsTaLi44SYeZiGIMDBPksKNJAuv_qNqklD9wNIYpouYQxz3_cHPzEdzyUHc7ub82oBw9joACFrnLR1IJZaG3QsULwsNqAcU7Vpz/ZPyaYgwim9H3lAGnjLI8uSzFoaNUt49g_fSaHX/ozqAPjyeX8g1LFXPJHwOXPQSwRHBLh4Av_qaeL7Xd/ateWXnp_WtrXzPW0Bf8UhMi9y9Y75cTM2AzT2Qfa91NXka/vg1Uf2a2BWCGcADTgHUl8yDYrUvcpgXFWHpr2ZEm49SwM24H91GYYqVbTUMcvoPvMq") banner("end https://alexandrite.app/")

[amgcc]

I started using rich for the colors but the wrapping logic is a major negative. Enough so that I'll switch to another tool as soon as I can find one that doesnt butcher my console output.

[willmcgugan]

This is not an airport. No need to announce your departure.

[amgcc]

Well for what its worth, using soft_wrap=True with Console seems to have fixed my issue of extra line feeds (effectively double spaced output). It seems the others on this thread are dealing with a different problem.

[pbower]

As a user coming to this library for better logging I was pretty shocked to see this enforced as the defaults, without the ability to access standard lines. As a general principle, one expects libraries to offer enhancements rather than regressions of functionality.

[liiight]

As a user coming to this library for better logging I was pretty shocked to see this enforced as the defaults, without the ability to access standard lines. As a general principle, one expects libraries to offer enhancements rather than regressions of functionality.

As a general principle, users of freely distributed software are expected to be polite and civil and not act like entitled assholes, but it is what is is 🤷‍♂️

[liiight]

I solved this issue by subclassing the LogRender class and injecting it to the RichHandler like so:

class CustomLogRender(LogRender): def __call__( self, console: Console, renderables: Iterable[ConsoleRenderable], log_time: datetime | None = None, time_format: str | Callable[[datetime], Text] | None = None, level: TextType = "", path: str | None = None, line_no: int | None = None, link_path: str | None = None, ): output = Text(no_wrap=True) if self.show_time: log_time = log_time or console.get_datetime() time_format = time_format or self.time_format log_time_display = time_format(log_time) if callable(time_format) else Text(log_time.strftime(time_format), style="log.time") if log_time_display == self._last_time and self.omit_repeated_times: output.append(" " * len(log_time_display), style="log.time") output.pad_right(1) else: output.append(log_time_display) output.pad_right(1) self._last_time = log_time_display if self.show_level: level.rstrip() output.append(level) output.pad_right(1) if self.show_path and path: path_text = Text(style="log.path") path_text.append(path, style=f"link file://{link_path}" if link_path else "") if line_no: path_text.append(":") path_text.append( f"{line_no}", style=f"link file://{link_path}#{line_no}" if link_path else "", ) output.append(path_text) output.pad_right(1) for renderable in Renderables(renderables): if isinstance(renderable, Text): renderable.stylize("log.message") output.append(renderable) output.pad_right(1) return output

(This is very rough and custom tailored to my needs though)
Then I set it like so:

 r_handler = RichHandler( rich_tracebacks=True, markup=True, console=LOGGER_CONSOLE, enable_link_path=self.should_enable_link_path, show_time=self.should_show_time, ) r_handler._log_render = CustomLogRender(omit_repeated_times=True, show_time=self.should_show_time, show_level=True, show_path=True)

@willmcgugan would you consider a PR that will enable subclassing the currently private LogRender (from rich._log_render import LogRender and allow setting the currently private _log_render method?

I feel like this constructor class is a very natural entry point to customize the logger output without adding maintenance overhead to rich, by making this section a bit more flexible via a small refactor.

edit:

on additional thought, maybe the ideal approach would be to make LogRender actually be a logging.Formatter?

[huxiang]

Thanks for the implement of CustomLogRender, I can finally show pretty logging content in my CI console!

I just found a small problem with it: level_width parameter of the CustomLogRender is not working. Here's my fix:

... if self.show_level: level.rstrip() output.append(level) output.pad_right(self.level_width - len(level)) # Adjust pad size according to "level" length

[huxiang]

Another problem is logging.exception(...) content cannot be handled. A TypeError: Only str or Text can be appended to Text exception will be threw for it. Here's my fix:

 if False in [isinstance(_, Text) for _ in Renderables(renderables)]: for renderable in Renderables(renderables): segments = console.render(renderable) for segment in segments: output.append(segment.text, style=segment.style) else: for renderable in Renderables(renderables): if isinstance(renderable, Text): renderable.stylize("log.message") output.append(renderable) output.pad_right(1)

[trothwell]

This solution works for me. 🤷

 class CustomConsole(rich.console.Console): def log( self, *args, **kwargs, ) -> None: # Backup existing before override bak_split_and_crop_lines = rich.console.Segment.split_and_crop_lines bak_width = self.width try: # Override for custom behavior rich.console.Segment.split_and_crop_lines = functools.partial( rich.console.Segment.split_and_crop_lines, pad=False, ) self.width = sys.maxsize result = super().log(*args, **kwargs) return result finally: # Restore to original rich.console.Segment.split_and_crop_lines = bak_split_and_crop_lines self.width = bak_width