Copyright © 2005-2006 Forest Bond
Table of Contents
sclapp is a Python module providing functionality intended
for use by command line applications. It provides some functions and classes
that are intended to reduce the boiler-plate required when writing small
command-line applications in Python. Currently, this includes:
logging module).
-h/--help and
-v/--version.
While some of this functionality can be used through a bits-and-pieces approach, much of it is best integrated into your program's main() function with a particular order of calls. This is achieved through the use of a wrapper function that contains and calls the existing main() function. An existing program can usually be modified very easily to take advantage of this.
sclapp was written primarily because I found myself including most of the same code in every command line application I was writing. Putting this code in a separate module makes my life easier, and makes my programs more consistent. Others may find these benefits to their liking as well, which is why I've made it available as a separate module.
Using sclapp should be fairly straight-forward. Here's the simplest scenario:
import sclapp
def main(argv):
do_something()
main = sclapp.mainWrapper(main)
if __name__ == '__main__':
sys.exit(main())
The immediate benefits of making use of sclapp's main() wrapper are:
Let's look at a slightly more complicated example. Suppose your program needs
to perform some cleanup action before terminating. Python's default signal
handling configuration results in the receipt of SIGPIPE to cause a traceback
due to an IOError. Even if the
IOError is handled properly using a
try...finally block, a traceback is still likely to occur (unless the
IOError is explicitly caught, something most
programmers do not implement in practice).
Even worse, if stderr is being redirected to stdout (not an entirely uncommon
situation), and output causes SIGPIPE (and, consequently, an
IOError), any output generated during the
shutdown sequence will re-trigger the error. In some cases, your program can
spiral out of control in a chain reaction of exceptions that prevent your
cleanup code from ever running!
Consider the following example:
import sclapp, signal
def main(argv):
try:
sclapp.setErrorOutputLevel(sclapp.INFO)
sclapp.printDebug('calling do_something()...')
do_something()
finally:
sclapp.printWarning('cleaning up...')
cleanup()
main = sclapp.mainWrapper(
main,
exit_signals = ( signal.SIGPIPE, )
)
if __name__ == '__main__':
sys.exit(main())
If this program receives a SIGPIPE, the following events will occur:
SIGPIPE will now cause an
ExitSignalError. An
IOError may also be raised, however sclapp's
protected output handles that situation by disabling the file object that
caused the exception, thus preventing further exceptions for the same reason.
ExitSignalError will be handled properly by
the try...finally block, and your cleanup code will execute as expected.
ExitSignalError and return 0 to
trigger normal program termination.
This example also illustrates usage of sclapp's prioritized error output functions. The first message printed will not typically be visiable, as it has priority "DEBUG," which is lower than the specified output level, "INFO." For more information on the prioritized error output functions and constants, see the documentation specific to that functionality below.
Signal handling is something that can be tedious to implement properly due to the asynchronous nature of signals. The typical approach is to set global flags when signals are received, and poll those flags periodically. This is tedious for programmers, and frequently results in poor responsiveness.
sclapp's signal handling strategy is to map signals to exceptions, which are also asynchronous by nature. Exceptions, however, are trivially dealt with in Python, which has language constructs to make them easy to work with. Python's default signal configuration leads to less-than-satisfactory results in most situations, and even the most trivial command-line applications should improve upon this to function as expected. For instance, no command-line application should print a traceback when SIGPIPE is received.
sclapp's default signal handling configuration is intended to be sane, without providing unnecessary behavior in simple applications. Signals that are commonly expected to result in normal program termination are set to the default operating system behavior (SIG_DFL). Currently, this includes any of the following signals supported by the underlying system:
Additionally, signals that are usually explicitly dealt with (one way or another) are ignored by default:
The consequences of these defaults are that simple programs (that do not need to perform cleanup actions prior to exit) will behave roughly as expected. Programs that do require cleanup should not use the default signal handling configuration. This is a considerable improvement over Python's default behavior, which results in unnecessary tracebacks in normal use of command-line applications.
sclapp categorizes signals into one of four groups of signals:
Exit and notify signals trigger
ExitSignalErrors and
SignalErrors, respectively. Both
ignore and default signals are handled
by the operating system without any notification to the caller. This is
typical behavior for UNIX-like systems. Callers set their signal-handling
configurations by mapping signals to these categories, and handling exceptions
appropriately.
It is important to note that, apart from generating different types of
exceptions, sclapp handles exit signals and notify signals slightly differently.
At times, a program may receive a signal while a previously caught signal is
being handled. Also, due to the way Python handles output, certain conditions
result in both receipt of SIGPIPE as well as one or more
IOErrors being raised. In either of these
scenarios, it is possible for more than one signal to have been caught before
sclapp has an opportunity to raise an appropriate
exception. If this is the case, sclapp will always attempt
to give the exit priority, and the exception resulting
from the caught notify signal will be dropped in favor
of the ExitSignalError.
It should be understood that exit and
notify signals are to be considered fundamentally
different in a semantic sense. sclapp makes the
assumption that signals categorized as exit signals are
truly intended to cause program exit. Consequently, while it is possible to
catch ExitSignalErrors and handle them
explicitly, doing so is highly discouraged, as it would not be consistent with
the approach that sclapp takes. In summary,
ExitSignalErrors should only be handled using
try...finally blocks, and not explicitly caught with try...except statements.
Finally, do note that, since default signals do not trigger exceptions to be raised, callers will have no opportunity to perform cleanup tasks if such signals are caught. Thus, those signals that are normally considered default signals should be re-mapped as exit signals by programs requiring cleanup.
Situations may arise when stdout and/or stderr become unusable, and writing
to them raises an IOError. sclapp's output
protection catches this exception, and handles it appropriately. Namely, if
sclapp is handling SIGPIPE, the error is dropped, and, if
necessary, the exception resulting from SIGPIPE is forcibly
raised, if it was interrupted by the IOError.
If sclapp is not handling SIGPIPE or the program is
running on a non-POSIX system, the IOError is
converted to a CriticalError, which is caught in
the main function (after any cleanup has occured), and reported to the user.
Consequently, these situations should never result in a traceback.
sclapp achieves this output protection by replacing sys.stdout and sys.stderr with alternative, limited, file-like objects. Be aware of this if you are also attempting to modify these objects (it is likely this will not work very well).
Be advised that sclapp will not treat all
IOErrors as described above, only
those that are either likely to occur as a result of permanent
problems with the respective file object, or those that are not likely to
have been caused by a bug in the calling program.
(Currently, sclapp only deals with EPIPE errors, and passes
all others along to the caller).
It is common for command-line applications to have variable levels of verbosity
with regard to program output. The logging from Python's
standard library is ideal for this sort of task, however, the module is quite
flexible, and configuring it for typical usage by simple command-line programs
can be tedious. For this reason, sclapp provides access to
the functionality offered by the logging module
pre-configured to be appropriate for the kind of prioritized error output
simple command-line applications sometimes require.
sclapp's prioritized error output functionality is accessed
via several functions, each of which prints a message of a given priority to
stderr. These functions are listed below:
printDebug: prints a message with priority
DEBUG.
printInfo: prints a message with priority
INFO.
printWarning: prints a message with priority
WARNING.
printError: prints a message with priority
ERROR.
printCritical: prints a message with priority
CRITICAL.
ALL, DEBUG, INFO,
WARNING, ERROR,
CRITICAL are constants defined by sclapp
(actually, they are defined in the logging module, but
sclapp makes them available for import as well). Just as
with the logging module, these constants correspond with
integer values the indicate a priority level; these values are 10, 20, 30, 40,
and 50, respectively.
The signficance of the integer priority levels is simple. When your program is run, it will set the desired level of error output, which is itself an integer (although it is normally set using one of the pre-defined constants listed above). Any messages printed with an associated priority that is greater than or equal to this error output level will appear on stderr; messages whose priorities are less than the error output level do not actually appear on the terminal.
The error output level can be set using the sclapp function
setErrorOutputLevel(). This is typically done as soon as
possible in the program's main function. If you are using
sclapp's built-in main() wrapper,
mainWrapper(), the desired output level can be specified
using a keyword argument, error_output_level. For a more
thorough explanation of the concepts surrounding prioritized output, see the
documentation for the logging module.
sclapp provides some "bonus" convenience functionality if your program needs it.
sclapp will daemonize your program if given the appropriate argument to the main() wrapper.
sclapp will, by default, handle uncaught exceptions (except for CriticalErrors, ExitSignalErrors, and some IOErrors, as described above) by telling the user that a bug has been encountered, and asking him to file a bug report. The exact message can be changed (see the documentation for the main() wrapper).
Note: this section is automatically generated from source code.
CriticalError (sclapp.exceptions.Error)indicates a fatal error (can and should be raised by sclapp caller)
ExitSignalError (sclapp.exceptions.CriticalError)indicates that an exit signal has been caught (raised by sclapp)
SignalError (sclapp.exceptions.Error)indicates that a notify signal has been caught (raised by sclapp)
UsageError (sclapp.exceptions.CriticalError)indicates a usage error (can and should be raised by sclapp caller)
disableSignalHandling()
disableSignalHandling() -> None
Disables sclapp signal handling.
enableSignalHandling(exit_signals = None, |
notify_signals = None, |
default_signals = None, |
ignore_signals = None, |
announce_signals = True |
enableSignalHandling() -> None
Enables sclapp signal handling.
getCaughtSignals()
getCaughtSignals() -> list
Returns a list containing all signal numbers that have been caught by sclapp signal handlers.
getDefaultSignals()
getDefaultSignals() -> list
Returns a list containing all signal numbers that are currently being handled by sclapp as default signals (SIG_DFL).
getExitSignals()
getExitSignals() -> list
Returns a list containing all signal numbers that are currently being handled by sclapp as exit signals.
getIgnoreSignals()
getIgnoreSignals() -> list
Returns a list containing all signal numbers that are currently being handled by sclapp as ignore signals (SIG_IGN).
getNotifySignals()
getNotifySignals() -> list
Returns a list containing all signal numbers that are currently being handled by sclapp as notify signals.
mainWrapper(realmain, |
name = None, |
author = 'the author', |
version = None, |
doc = None, |
handle_signals = True, |
exit_signals = None, |
notify_signals = None, |
default_signals = None, |
ignore_signals = None, |
protect_output = True, |
daemonize = False, |
bug_message = '${traceback}\nSomething bad happened, and is most likely a bug.\nPlease file a bug report to ${author}.\nInclude the error message(s) printed here.', |
version_message = '${name} version ${version}', |
ignore_unrecognized_options = True, |
error_output_level = None |
Some of the optional parameters mentioned above cause sclapp to parse argv and respond appropriately:
* If version_message is not None, sclapp will respond to the -v command line switch by printing the version_message to stdout. * If doc is not None, sclapp will respond to the -h command line switch by printing doc to stdout.
Both bug_message and version_message are parsed for substitution strings prior to printing. Specifically, sclapp uses the Template class of the standard library's string module to make the following substitutions:
${name} is replaced by the name parameter ${author} is replaced by the author parameter ${version} is replaced by the version parameter ${doc} is replaced by the doc parameters
The doc parameter is parsed for all other substitutions before it is itself substituted. Thus, callers should feel free to add ${name}, ${author}, and ${version} to the doc parameter, and they will be replaced appropriately before printing.
Be default, sclapp installs signal handlers that are easily configurable using module functions. Setting handle_signals to False will disable this behavior.
sclapp can also be used to write simple daemons. To cause the program to daemonize before launching the wrapped main() function, set daemonize to True.
bug_message, if not None, will be printed in the event that an unhandled exception is caught by the main wrapper. The default message simply informs the user that a likely bug has been encountered, and asks them to file a bug report to the author.
If ignore_unrecognized_options is False, sclapp will complain about options it isn't expecting. If your program does not explicitly handle command line options, you probably want this. Otherwise, you definately don't want this, as any of your options will cause a UsageError to be reported before you have a chance to handle them.
If sclapp finds that it can't do anything useful with -h/--help or -v/--version (if version, doc are None, or contain substitutions sclapp can't fulfill), sclapp does nothing with command line options.
The new main() function will return the wrapped function's return value, unless an exception is raised (including exceptions resulting from an exit signal being received), in which case the return value is an integer intended to act as the program's termination status.
printCritical(message)
printCritical(message) -> None
Prints a message to stderr with priority CRITICAL.
printDebug(message)
printDebug(message) -> None
Prints a message to stderr with priority DEBUG.
printError(message)
printError(message) -> None
Prints a message to stderr with priority ERROR.
printInfo(message)
printInfo(message) -> None
Prints a message to stderr with priority INFO.
printWarning(message)
printWarning(message) -> None
Prints a message to stderr with priority WARNING.
protectOutput()
setErrorOutputLevel(level)
setErrorOutputLevel(level) -> None
level: int minimum priority of error messages that are displayed
Sets the minimum priority required for an error output message to actually be seen. sclapp defines the following constants for convenience:
ALL = 1 SCLAPP_DEBUG = 5 DEBUG = 10 INFO = 20 WARNING = 30 ERROR = 40 CRITICAL = 50
unprotectOutput()
ALL = 1
ALL_SIGNALS = (6, 14, 7, 17, 17, 18, 8, 1, 4, 2, 29, 6, 9, 13, 29, 27, 30, 3, 64, 34, 11, 19, 31, 15, 5, 20, 21, 22, 23, 10, 12, 26, 28, 24, 25)
CRITICAL = 50
DEBUG = 10
ERROR = 40
INFO = 20
SCLAPP_DEBUG = 5
STD_DEFAULT_SIGNALS = (1, 2, 3, 4, 6, 8, 13, 15, 7, 27, 31, 5, 24, 25)
STD_EXIT_SIGNALS = ()
STD_IGNORE_SIGNALS = (10, 12, 14)
STD_NOTIFY_SIGNALS = ()
WARNING = 30
BackgroundCommand (sclapp.processes._BackgroundProcess)Runs an external command in a forked process. The status of the forked process can be monitored by the caller.
__init__(self, |
command, |
args, |
stdin = None, |
stdout = None, |
stderr = None |
Overrides sclapp.processes._BackgroundProcess.__init__
command will be run with arguments args. If stdin, stdout, or stderr are specified, the standard I/O file descriptors for the sub-process will be redirected. stdin, stdout, and stderr should be specified as for redirectFds(), or left unspecified if no I/O redirection is desired.
cont(self)Inherited from sclapp.processes._BackgroundProcess
Re-starts a stopped (paused) process (usually by sending SIGCONT).
getExitSignal(self)Inherited from sclapp.processes._BackgroundProcess
Returns the signal that caused the process to terminate.
getExitStatus(self)Inherited from sclapp.processes._BackgroundProcess
Returns the exit status of the process.
getPid(self)Inherited from sclapp.processes._BackgroundProcess
Returns the PID (process ID) of the process.
isRunning(self)Inherited from sclapp.processes._BackgroundProcess
Returns True if process is running, False otherwise.
isStopped(self)Inherited from sclapp.processes._BackgroundProcess
Returns True if process is stopped, False otherwise.
kill(self, signum = 2)Inherited from sclapp.processes._BackgroundProcess
Kills the process, or sends an arbitrary signal (specified by the signum argument) to the process. Returns False if the process doesn't appear to be running in the first place, True otherwise.
reap(self)Inherited from sclapp.processes._BackgroundProcess
Tries to reap the process. Returns True if successful, False otherwise.
run(self)Overrides sclapp.processes._BackgroundProcess.run
Runs the command.
stop(self)Inherited from sclapp.processes._BackgroundProcess
Stops (pauses) the process (usually by sending SIGSTOP).
wait(self, block = True)Inherited from sclapp.processes._BackgroundProcess
Blocks until the process has terminated. If the block argument is False, process will simply be reaped (if it has terminated), and the function will return. Returns True if process is reaped by this call, False otherwise.
BackgroundFunction (sclapp.processes._BackgroundProcess)Runs a function in a forked background process. The status of the forked process can be monitored by the caller during function execution.
__init__(self, |
function, |
args, |
kwargs, |
stdin = None, |
stdout = None, |
stderr = None |
Overrides sclapp.processes._BackgroundProcess.__init__
cont(self)Inherited from sclapp.processes._BackgroundProcess
Re-starts a stopped (paused) process (usually by sending SIGCONT).
getExitSignal(self)Inherited from sclapp.processes._BackgroundProcess
Returns the signal that caused the process to terminate.
getExitStatus(self)Inherited from sclapp.processes._BackgroundProcess
Returns the exit status of the process.
getPid(self)Inherited from sclapp.processes._BackgroundProcess
Returns the PID (process ID) of the process.
isRunning(self)Inherited from sclapp.processes._BackgroundProcess
Returns True if process is running, False otherwise.
isStopped(self)Inherited from sclapp.processes._BackgroundProcess
Returns True if process is stopped, False otherwise.
kill(self, signum = 2)Inherited from sclapp.processes._BackgroundProcess
Kills the process, or sends an arbitrary signal (specified by the signum argument) to the process. Returns False if the process doesn't appear to be running in the first place, True otherwise.
reap(self)Inherited from sclapp.processes._BackgroundProcess
Tries to reap the process. Returns True if successful, False otherwise.
run(self)Overrides sclapp.processes._BackgroundProcess.run
stop(self)Inherited from sclapp.processes._BackgroundProcess
Stops (pauses) the process (usually by sending SIGSTOP).
wait(self, block = True)Inherited from sclapp.processes._BackgroundProcess
Blocks until the process has terminated. If the block argument is False, process will simply be reaped (if it has terminated), and the function will return. Returns True if process is reaped by this call, False otherwise.