Feature: template in configuration file, 2nd variation

[kparasch]

Implements a Template category in accelerator that allows generation of devices from templates. Uses a "template resolver" to generate from templates.

The template class uses the standard python string ".format" function to replace text and generate the template.
The template class stores an expanded template dictionary, which is registered when loading a file with JSONLoader.load and YAMLLoader.load. When the template resolver is found (e.g. ${template:SHI,SHI,01}), it replaces the parameters walking through the config dictionary and using str.replace.

example config with template:

class: pyaml.accelerator.Accelerator
machine: sr
facility: PETRAIII
energy: 6e9
controls:
 - class: tango.pyaml.controlsystem.TangoControlSystem
   tango_host: ebs-simu-3:10000
   name: live
   catalog:
     class: tango.pyaml.tango_catalog.TangoCatalog
     disconnected: True
devices:
- ${template:SHI,SHI,01}
- ${template:SHI,SHI,02}
- ${template:SHI,SHI,03}
templates:
  - name: SHI
    parameters:
      - sh_name
      - ii
    config:
      class: pyaml.magnet.hcorrector.HCorrector
      name: "{sh_name}-{ii}"
      model:
        class: pyaml.magnet.linear_model.LinearMagnetModel
        unit: rad
        hardware_unit: str
        calibration_factor: 1.0
        powerconverter: tango/MAGNET/{sh_name}-{ii}/B1L

The implementation adds:

• TemplateManager to store definitions, validate argument counts, and generate independent configuration structures through recursive string substitution.
• A ${template:...} resolver that generates the configuration and passes it through normal expansion, supporting nested templates, file references, and environment variables.
• Registration during YAML/JSON loading, before recursive expansion, so definitions are available to calls within the file.
• A registry owned by each ConfigurationManager, shared across its file fragments through LoadContext. Standalone loads create a fresh registry unless one is supplied explicitly.

The LoadContext (which is passed to the resolvers) is extended to hold:

• The expand function used by the ConfigLoader, so that the resolver can expand files found inside the template.
• A reference to the TemplateManager.

Will close issue #443

[kparasch]

@JeanLucPons It worked with numbers

[gubaidulinvadim]

@kparasch It would be good to add a section to the tutorials in the documentation on how to use it.

[TeresiaOlsson]

@kparasch It would be good to add a section to the tutorials in the documentation on how to use it.

A how-to guide is likely also needed.

[gubaidulinvadim]

I didn't look too much into templates, but for me, one of the features should be to generate a full config from a template. I personally find that reading a plain config, without template syntax, is easier. But generating a config is easier with a template :) All I'd like is a utility function that spits out a full config (or even Accelerator.save(yamlfile) ) could be a desired feature.

[TeresiaOlsson]

I didn't look too much into templates, but for me, one of the features should be to generate a full config from a template. I personally find that reading a plain config, without template syntax, is easier. But generating a config is easier with a template :) All I'd like is a utility function that spits out a full config (or even Accelerator.save(yamlfile) ) could be a desired feature.

I think that utility tool is the one @simoneliuzzo has started on in: https://github.com/python-accelerator-middle-layer/configuration-generator.

Personally I think implementing Accelerator.save would be much easier if we implemented a configuration registry inside the Accelerator which has the responsibility for storing the current state of the configuration. At the moment the responsibility for storing the configuration data is on the same objects that are responsible for performing the business logic. I don't think that is an especially robust architecture and that we should learn from MML where the separation between data storage and business logic exists. Having a configuration registry would also make it easier to implement dynamic changes to the configuration. But so far no one seemed to like the idea except maybe @gupichon so I have not worked on it further. But I had a prototype of such a configuration registry in my first PR for the schema registry. The idea was the following: load configuration -> validation using schema registry -> create the accelerator which internally will create an instance of a configuration registry where the current configuration is stored. And then the factory would be changed to extract the data from the configuration registry instead of working directly on the data as loaded from the configuration file.