API Reference¶

Global Objects¶

twiggy.log¶: the magic log object

twiggy.internal_log¶: InternalLogger for reporting errors within Twiggy itself

twiggy.devel_log¶: InternalLogger for use by developers writing extensions to Twiggy

twiggy.emitters¶: the global emitters dictionary, tied to the log

twiggy.add_emitters(*tuples)¶: Add multiple emitters. tuples should be (name_of_emitter, min_level, filter, output). The last three are passed to Emitter.

twiggy.quick_setup(min_level=<LogLevel DEBUG>, file=None, msg_buffer=0)¶

Quickly set up emitters.

Parameters:	min_level (LogLevel) – lowest message level to cause output file (string) – filename to log to, or `sys.stdout`, or `sys.stderr`. `None` means standard error. msg_buffer (int) – number of messages to buffer, see `outputs.AsyncOutput.msg_buffer`

Features¶

Optional additions of logging functionality

procinfo¶

Logging feature to add information about process, etc.

twiggy.features.procinfo.procinfo(self)¶

Adds the following fields:

Hostname:	current hostname
Pid:	current process id
Thread:	current thread name

socket¶

Logging feature to add information about a socket

twiggy.features.socket.socket(self, s)¶

Adds the following fields:

ip_addr: numeric IP address

port: port number

host: peer hostname, as returned by getnameinfo()

service: the human readable name of the service on port

Parameters:	s (socket) – the socket to extract information from

twiggy.features.socket.socket_minimal(self, s)¶: Like socket, but only log ip_addr and port

Filters¶

twiggy.filters.filter(msg : Message) → bool¶: A filter is any function that takes a Message and returns True if it should be emitted.

twiggy.filters.msg_filter(x) → filter¶

create a filter intelligently

You may pass:

None, True: the filter will always return True

False: the filter will always return False

string: compiled into a regex

regex: match() against the message text

callable: returned as is

list: apply msg_filter to each element, and all() the results

Return type:	`filter` function

twiggy.filters.names(*names) → filter¶

create a filter, which gives True if the messsage’s name equals any of those provided

names will be stored as an attribute on the filter.

Parameters:	names (strings) – names to match
Return type:	`filter` function

twiggy.filters.glob_names(*names) → filter¶

create a filter, which gives True if the messsage’s name globs those provided.

names will be stored as an attribute on the filter.

This is probably quite a bit slower than names().

Parameters:	names (strings) – glob patterns.
Return type:	`filter` function

class twiggy.filters.Emitter¶

Hold and manage an Output and associated filter()

min_level¶: only emit if greater than this LogLevel

filter¶: arbitrary filter() on message contents. Assigning to this attribute is intelligent.

_output¶: Output to emit messages to. Do not modify.

Formats¶

Formats are single-argument callables that take a Message and return an object appropriate for the Output they are assigned to.

class twiggy.formats.LineFormat(separator=':', traceback_prefix='\nTRACE', conversion=line_conversion)¶

separator¶: string to separate line parts. Defaults to :.

traceback_prefix¶

string to prepend to traceback lines. Defaults to \nTRACE.

Set to '\\n' (double backslash n) to roll up tracebacks to a single line.

conversion¶: ConversionTable used to format fields. Defaults to line_conversion

format_text(msg)¶: format the text part of a message

format_fields(msg)¶: format the fields of a message

format_traceback(msg)¶: format the traceback part of a message

twiggy.formats.line_conversion¶

a default line-oriented ConversionTable. Produces a nice-looking string from fields.

Fields are separated by a colon (:). Resultant string includes:

time: in iso8601 format (required)

level: message level (required)

name: logger name

Remaining fields are sorted alphabetically and formatted as key=value

twiggy.formats.line_format¶

a default LineFormat for output to a file. Sample output.

Fields are formatted using line_conversion and separated from the message text by a colon (:). Traceback lines are prefixed by TRACE.

twiggy.formats.shell_conversion¶: a default line-oriented ConversionTable for use in the shell. Returns the same string as line_conversion but drops the time field.

twiggy.formats.shell_format¶: a default LineFormat for use in the shell. Same as line_format but uses shell_conversion for fields.

Levels¶

Levels include (increasing severity): DEBUG, INFO, NOTICE, WARNING, ERROR, CRITICAL, DISABLED

class twiggy.levels.LogLevel(name, value)¶

A log level. Users should not create new instances.

Levels are opaque; they may be compared to each other, but nothing else.

twiggy.levels.name2level(name)¶: return a LogLevel from a case-insensitve string

Library¶

twiggy.lib.iso8601time(gmtime=None)¶

convert time to ISO 8601 format - it sucks less!

Parameters:	gmtime (time.struct_time) – time tuple. If None, use `time.gmtime()` (UTC)

XXX timezone is not supported

twiggy.lib.thread_name()¶: return the name of the current thread

Converter¶

class twiggy.lib.converter.Converter(key, convert_value, convert_item, required=False)¶

Holder for ConversionTable items

Variables:	key – the key to apply the conversion to convert_value (function) – one-argument function to convert the value convert_item (function) – two-argument function converting the key & converted value required (bool) – is the item required to present. Items are optional by default.

twiggy.lib.converter.same_value(v)¶: return the value unchanged

twiggy.lib.converter.same_item(k, v)¶: return the item unchanged

twiggy.lib.converter.drop(k, v)¶: return None, indicating the item should be dropped

New in version 0.5.0: Add same_value, same_item, drop.

class twiggy.lib.converter.ConversionTable(seq)¶

Convert data dictionaries using Converters

For each item in the dictionary to be converted:

Find one or more corresponding converters c by matching key.
Build a list of converted items by calling c.convertItem(item_key, c.convertValue(item_value)). The list will have items in the same order as converters were supplied.
Dict items for which no converter was found are sorted by key and passed to generic_value / generic_item. These items are appended to the list from step 2.
If any required items are missing, ValueError is raised.
The resulting list of converted items is passed to aggregate. The value it returns is the result of the conversion.

Users may override generic_value/generic_item/aggregate by subclassing or assigning a new function on a ConversionTable instance.

Really, it’s pretty intuitive.

__init__(seq=None)¶

Parameters:	seq – a sequence of Converters

You may also pass 3-or-4 item arg tuples or kwarg dicts (which will be used to create Converters)

convert(d)¶

do the conversion

Parameters:	d (dict) – the data to convert. Keys should be strings.

generic_value(value)¶: convert values for which no specific Converter is supplied

generic_item(key, value)¶: convert items for which no specific Converter is supplied

aggregate(converteds)¶: aggregate list of converted items. The return value of convert

copy()¶: make an independent copy of this ConversionTable

get(key)¶: return the first converter for key

get_all(key)¶: return a list of all converters for key

add(*args, **kwargs)¶: Append a Converter. args & kwargs will be passed through to its constructor

delete(key)¶: delete the all of the converters for key

Logger¶

Loggers should not be created directly by users; use the global log instead.

class twiggy.logger.BaseLogger(fields=None, options=None, min_level=None)¶

Base class for loggers

_fields¶: dictionary of bound fields for structured logging. By default, contains a single field time with value time.gmtime(). This function will be called for each message emitted, populating the field with the current time.struct_time.

_options¶: dictionary of bound options.

min_level¶: minimum LogLevel for which to emit. For optimization purposes only.

fields(**kwargs) → bound Logger¶: bind fields for structured logging. kwargs are interpreted as names/values of fields.

fields_dict(d) → bound Logger¶

bind fields for structured logging. Use this instead of fields if you have keys which are not valid Python identifiers.

Parameters:	d (dict) – dictionary of fields. Keys should be strings.

options(**kwargs) → bound Logger¶: bind options for message creation.

trace(trace='error') → bound Logger¶: convenience method to enable traceback logging

name(name) → bound Logger¶: convenvience method to bind name field

struct(**kwargs) → bound Logger¶: convenience method for structured logging. Calls fields() and emits at info

struct_dict(d) → bound Logger¶

convenience method for structured logging. Use instead of struct if you have keys which are not valid Python identifiers.

Parameters:	d (dict) – dictionary of fields. Keys should be strings.

The following methods cause messages to be emitted. format_spec is a template string into which args and kwargs will be substitued.

debug(format_spec='', *args, **kwargs)¶: Emit at DEBUG level

info(format_spec='', *args, **kwargs)¶: Emit at INFO level

notice(format_spec='', *args, **kwargs)¶: Emit at NOTICE level

warning(format_spec='', *args, **kwargs)¶: Emit at WARNING level

error(format_spec='', *args, **kwargs)¶: Emit at ERROR level

critical(format_spec='', *args, **kwargs)¶: Emit at CRITICAL level

class twiggy.logger.Logger(fields=None, options=None, min_level=None)¶

Logger for end-users. The type of the magic log

filter¶

Filter on format_spec. For optimization purposes only. Should have the following signature:

func(format_spec : string) → bool: Should the message be emitted.

classmethod addFeature(func, name=None)¶

add a feature to the class

Parameters:	func – the function to add name (string) – the name to add it under. If None, use the function’s name.

classmethod disableFeature(name)¶

disable a feature.

A method will still exist by this name, but it won’t do anything.

Parameters:	name (string) – the name of the feature to disable.

classmethod delFeature(name)¶

delete a feature entirely

Parameters:	name (string) – the name of the feature to remove

class twiggy.logger.InternalLogger(output, fields=None, options=None, min_level=None)¶

Special-purpose logger for internal uses. Sends messages directly to output, bypassing emitters.

Variables:	output (Output) – an output to write to

twiggy.logger.emit(level)¶

a decorator that emits at level after calling the method. The method should return a Logger instance.

For convenience, decorators for the various levels are available as emit.debug, emit.info, etc..

Message¶

class twiggy.message.Message(level, format_spec, fields, options, args, kwargs)¶

A logging message. Users never create these directly.

Changed in version 0.4.1: Pass args/kwargs as list/dict instead of via */** expansion.

The constructor takes a dict of options to control message creation. In addition to suppress_newlines, the following options are recognized:

trace: control traceback inclusion. Either a traceback tuple, or one of the strings always, error, in which case a traceback will be extracted from the current stack frame.

style: the style of template used for format_spec. One of braces, percent, dollar. The aliases {}, % and $ are also supported.

Any callables passed in fields, args or kwargs will be called and the returned value used instead. See dynamic messages.

All attributes are read-only.

fields¶: dictionary of structured logging fields. Keys are string, values are arbitrary. A level item is required.

suppress_newlines¶: should newlines be escaped in output. Boolean.

traceback¶: a stringified traceback, or None.

text¶: the human-readable message. Constructed by substituting args/kwargs into format_spec. String.

__init__(level, format_spec, fields, options, args, kwargs)¶

Parameters:

level (LogLevel) – the level of the message
format_spec (string) – the human-readable message template. Should match the style in options.
fields (dict) – dictionary of fields for structured logging
args (tuple) – substitution arguments for format_spec.
kwargs (dict) – substitution keyword arguments for format_spec.
options (dict) – a dictionary of options to control message creation.

Outputs¶

class twiggy.outputs.Output(format=None, close_atexit=True)¶

_format¶: a callable taking a Message and formatting it for output. None means return the message unchanged.

use_locks¶: Class variable, indicating that locks should be used when running in a synchronous, multithreaded environment. Threadsafe subclasses may disable locking for higher throughput. Defaults to True.

__init__(format=None, close_atexit=True)¶

Parameters:	format (format) – the format to use. If None, return the message unchanged. close_atexit (bool) – should `close()` be registered with `atexit`. If False, the user is responsible for closing the output.

New in version 0.4.1: Add the close_atexit parameter.

close()¶: Finalize the output.

The following methods should be implemented by subclasses.

_open()¶: Acquire any resources needed for writing (files, sockets, etc.)

_close()¶: Release any resources acquired in _open

_write(x)¶

Do the work of writing

Parameters:	x – an implementation-dependent object to be written.

class twiggy.outputs.AsyncOutput(msg_buffer=0)¶

An Output with support for asynchronous logging.

Inheriting from this class transparently adds support for asynchronous logging using the multiprocessing module. This is off by default, as it can cause log messages to be dropped.

Parameters:	msg_buffer (int) – number of messages to buffer in memory when using asynchronous logging. `0` turns asynchronous output off, a negative integer means an unlimited buffer, a positive integer is the size of the buffer.

class twiggy.outputs.FileOutput(name, format, mode='a', buffering=1, msg_buffer=0, close_atexit=True)¶

Output messages to a file

name, mode, buffering are passed to open()

class twiggy.outputs.StreamOutput(format, stream=sys.stderr)¶

Output to an externally-managed stream.

The stream will be written to, but otherwise left alone (i.e., it will not be closed).

class twiggy.outputs.NullOutput(format=None, close_atexit=True)¶: An output that just discards its messages

class twiggy.outputs.ListOutput(format=None, close_atexit=True)¶

an output that stuffs messages in a list

Useful for unittesting.

Variables:	messages (list) – messages that have been emitted

Changed in version 0.4.1: Replace DequeOutput with more useful ListOutput.