• Preamble

eepgwde

• Overview

This packages some generic utilities and decorators, for which it uses functools. TimeOps and others.

It also provides a configuration framework, in the pitono.weaves.config It provides a Config0.

• Packaging

This has been migrated from a setup.py and setup.cfg configuration to the new pypa recommended namespace. See notes from pitono.buildr.

In application code, one must then use the namespace to get to the package

from pitono import weaves

But also the sub-package can be accessed like this

from pitono.weaves import TimeOps

• Modules

** pitono.weaves

Has a number of classes that when instantiated are singletons. They are accessed like this:

helper0 = TimeOps()

and then invoke any methods on helper0. Because it is a singleton, it retains its state. So you can use it as a cache for results to pass to different sub-systems of your application.

The framework of singletons is designed to be easily sub-classed.

from pitono.weaves import DataFrames

from pitono.weaves import TimeOps from pitono.weaves import Utility

from pitono.weaves import POSetOps

from pitono.weaves import MCast from pitono.weaves import Enqueue from pitono.weaves import StoppableThread

DataFrames is a class with methods for working with Pandas dataframes. It does a lot of the simple transformations.

It uses some methods from TimeOps and Utility. TimeOps has day of year and week and so on. Utility has the very general methods to extend Python Programming: sub-sets of dictionaries, making hash codes, temporary file and a lot of others.

POSetOps is a class for working with Partially-Ordered Sets - or preferences.

MCast Enqueue and StoppableThread are examples. They are designed to be sub-classed. They demonstrate MultiCast messaging, and thread management.

• Configuration Framework - Config0

This design was inspired by the Java Spring Boot system.

** Overview

The pitono.weaves.config module provides a configuration singleton Config0(). This has been designed to support INI, JSON, TOML format files.

You obtain one like this:

from pitono.weaves.config import Config0, JsonConfigHandler from PathLib import Path

controller = Config0() handler = controller.handler(Path("config.json"), JsonConfigHandler)

The controller is a Singleton. The handler is not. This will create a handler onto the file at the path $PWD/config.json.

If you want to provide your own class MyJsonConfigHandler(JsonConfigHandler)

handler = controller.handler(Path("config.json"), MyJsonConfigHandler)

should also work. In providing the implementation for MyJsonConfigHandler, you can override the get method.

There is an example in pitono.weaves.config2 with the NetrcConfigHandler. It has a get() method that supports get("url"). JsonConfigHandler and NetrcConfigHandler are both sub-classes of ConfigHandler. There are other ConfigHandler sub-classes: TomlConfigHandler and IniConfigHandler.

*** Configuration Management like Spring for Java and Kotlin

The design is taken from the Spring system available to Java and Kotlin developers.

You can build handlers for all the sub-systems you use in an application. And you would do this early in the start-up phase of your system.

** Using the Config0() instance

Very often when constructing a sub-system, you want to override its constructor and add your own values drawn from configuration files.

To aid you with that Config0() supports a handler0() method. It returns the last handler built.

It has another method get_configuror() which interrogates the keywords passed to it. The keyword is "config_file", it can polymorphically be an instance of ConfigHandler (Netrc, Json and so on) or it can be a Path, or it will use the last handler via handler0().

*** Use the Config0() as an application cache

Because the Config0() instance is a singleton, you can attach handlers or other application information to it. You can use the kwargs argument to handle.

Its super-class is Helper0 and this class has methods to accrue default keywords within it.

*** Config0() can try a number of ConfigHandler classes

Internally, the Config0() uses a factory to try a sequence of classes. Which classes and the order is specified in Defaults1 or a sub-class. The default Defaults1 tries all the ConfigHandler classes that have been defined in the pitono.weaves.config module.

** Defaults0 Defaults1 and others

The Config0() single can be assigned a new set of defaults. To do that, you need to make use of the Config0().defaults() method.

Defaults1 has Defaults0 for a super-class. And a Defaults1 instance is used by Config0() to the object that creates the ConfigHandler instances. To use your own Defaults1 sub-class, Defaults3 for example, define it somewhere and use it like this.

defaults0 = Defaults3() defaults1 = Config0().defaults(defaults0=defaults0)

handler = controller.handler()

Here you don't need to specify a class or a file as before, you expect the handler to build its default handler. The Defaults3 instance can specify the order in which the ConfigHandler handlers are tried.

The Defaults3 class can provide defaults for config_dir, name and others. These can then be used to generate a config_file. This uses a private method called _mkname(), see the constructor for JsonConfigHandler for example.

To reset Config0() to use its own defaults Defaults1 use

defaults1 = Config0().defaults() handler = controller.handler()

This will then use the Defaults1 configuration and provide a handler using the defaults in that class.

** Practical Matters

*** Mamba and Debian

I now use Mamba to manage my full Python installation. I try to use my packages on Debian without the Mamba environment.

Large packages, pandas and jupyter as well as scipy, are only available using the mamba environment.

Because of that, my components that use scipy and pandas are in a separate sub-module. pitono.weaves.datumoj.

Many python3 modules can be installed into Debian. Because of that, the PYTHONPATH has been modified to use it /usr/lib/python3/dist-packages.

*** Logging

If you use the Config0() singleton, it will be one of the first objects you will use. And it will start logging.

If you have your own logger, you should pass it as the keyword "logger"

If you don't there is an environment variable PITONO_DEBUG that should be set to a number 0 to 50 - the logging module values. 0 and 10 give all log messages. After that, info, warning, error and critical.

It logs to a file by default and the name is "test.log". It also applies a transformation to the file's path so that if an environment variable TOP is defined, this is used as the path of the file. This logger is created by pitono.buildr.Pdb0Helper0 and the transformation is performed by Pdb0Helper0Defaults0.

The Config0() is a Helper0 singleton. This means that you can only use the constructor once. After that, you can partially reload it with the defaults() method.

The logging is set to a default of WARN, 30, which is not at all verbose. You can change this with the environment variable.

The tests/ classes all have their own logging settings.