feat: Enable tedge-config to support a generic mapper/bridge configuration

[jarhodes314]

Proposed changes

Allow tedge-config to store a generic configuration which can have a specialised sub configuration. For instance, we currently have configurations like:

c8y.url = "..."
c8y.root_cert_path = "..."
c8y.enable.log_upload = true
az.url = "..."
az.root_cert_path = "..."

With a generalised mapper, it would be useful to de-duplicate these configurations, so we can have something like:

mapper.url = "..."
mapper.root_cert_path = "..."

where profiles are used to support the different possible clouds. The changes this PR aims to make would add a new configuration mapper.type, which could be set to c8y or az (or another value), and then specialised sub-fields can be added, for instance:

mapper.c8y.enable.log_upload = true

Note about profiles I'm anticipating the mapper configuration to be profiled, however I'm omitting this detail in the example for simplicity. This means profiled mappers would be used like they are today, except that profiles of the mapper group would be used to support multiple cloud types (simultaneously) as well as multiple clouds of the same type.

This mapper.c8y group of configurations would only exist if mapper.type == "c8y", and we can have other specialised configurations for different clouds (or a mapper.type == "custom" where no specialised configuration is applied).

Currently this consists of some significant changes to the define_tedge_config macro to support this work. At the moment, there is one very small piece of code that isn't generated by the macro, and I haven't added example usage to tedge_config.rs, I've only created a separate example file.

Usage wise, these changes have two major changes. Firstly, there is a new macro for defining sub configurations:

define_sub_config! {
    C8y {
        enable: {
            log_upload: bool,
        }
    }
}

This is used to define the specialised configuration. It works very much like define_tedge_config, except it takes a name (in this case C8y) which is used to namespace the various generated structs and enums (TEdgeConfigDto => C8yDto, ReadableKey => C8yReadableKey).

The other change added to the macro usage is the sub_fields attribute:

define_tedge_config! {
    mapper: {
        #[tedge_config(sub_fields = [C8y(C8y), Az(Az), Custom])]
        type: MapperType,
    }
}

This defines the mapper.type field with three possible valid values: c8y, az and custom. If the type is set to c8y, the mapper group will have the C8y specialised sub configurations available, e.g. mapper.c8y.enable.log_upload. Similarly for az, the mapper will have the Az sub configurations. custom doesn't have a specialised configuration, so the mapper group will only have the standard fields, and no "sub-config" will be applied.

Types of changes

• Bugfix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Improvement (general improvements like code refactoring that doesn't explicitly fix a bug or add any new functionality)
• Documentation Update (if none of the other choices apply)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)

Paste Link to the issue

Checklist

• I have read the CONTRIBUTING doc
• I have signed the CLA (in all commits with git commit -s. You can activate automatic signing by running just prepare-dev once)
• I ran just format as mentioned in CODING_GUIDELINES
• I used just check as mentioned in CODING_GUIDELINES
• I have added tests that prove my fix is effective or that my feature works
• I have added necessary documentation (if appropriate)

Further comments

[didier-wenzek]

I don't fully get what's the plan here.

So a mapper can be given a specific type, but what if want to run 2 mappers on my device.

• Is this setting about a mapper or the mapper?
• How this works in combination with profiles?

[jarhodes314]

I've now attempted to clarify this in the PR description. It's about a mapper (#[tedge_config(multi)] is what dictates that, though now we've definitely called the feature in question "profiles", we probably ought to change the attribute to say something more like #[tedge_config(multi_profile)] instead.

[didier-wenzek]

If my understanding is correct we will then be able to configure:

A main c8y mapper:

$ tedge config set mapper.type c8y --profile default
$ tedge config set mapper.c8y.url "my.c8y.com" --profile default 

as well as a second c8y mapper:

$ tedge config set mapper.type c8y --profile fallback
$ tedge config set mapper.c8y.url "fallback.c8y.com" --profile fallback 

[jarhodes314]

If my understanding is correct

Yes, that is broadly correct, however instead of

$ tedge config set mapper.c8y.url "my.c8y.com" --profile default 

it would likely be:

$ tedge config set mapper.url "my.c8y.com" --profile default 

as the url is a common configuration to all mappers. The mapper.c8y. would be restricted just to the Cumulocity-specific configurations

[albinsuresh]

Even though mapper.url would be the new "recommended" config, the old c8y.url setting would still be maintained for backward compatibility, right?

[github-actions]

Robot Results

✅ Passed	❌ Failed	⏭️ Skipped	Total	Pass %	⏱️ Duration
713	0	3	713	100	2h5m14.937971999s

[codecov]

Codecov Report

❌ Patch coverage is 88.57715% with 57 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
crates/common/tedge_config_macros/impl/src/lib.rs	74.60%	24 Missing and 8 partials ⚠️
...common/tedge_config_macros/impl/src/input/parse.rs	80.70%	3 Missing and 8 partials ⚠️
...mon/tedge_config_macros/impl/src/input/validate.rs	91.39%	6 Missing and 2 partials ⚠️
...ates/common/tedge_config_macros/impl/src/reader.rs	96.89%	3 Missing and 2 partials ⚠️
crates/common/tedge_config_macros/macro/src/lib.rs	83.33%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[albinsuresh]

Note about profiles I'm anticipating the mapper configuration to be profiled, however I'm omitting this detail in the example for simplicity. This means profiled mappers would be used like they are today, except that profiles of the mapper group would be used to support multiple cloud types (simultaneously) as well as multiple clouds of the same type.

With this eventual goal in mind, would setting the default c8y.url config look like this?

tedge config set mapper.url "my.c8y.com" --profile c8y

Does it mean that we'll always have to specify the --profile c8y argument for the common settings like url and root_cert_path that don't belong to the C8y::sub_config section?

That UX inconsistency would be a bit weird, where you're trying to configure a single cloud but some are configured with a profile and others aren't.

[albinsuresh]

Even though mapper.url would be the new "recommended" config, the old c8y.url setting would still be maintained for backward compatibility, right?

[jarhodes314]

Note about profiles I'm anticipating the mapper configuration to be profiled, however I'm omitting this detail in the example for simplicity. This means profiled mappers would be used like they are today, except that profiles of the mapper group would be used to support multiple cloud types (simultaneously) as well as multiple clouds of the same type.

With this eventual goal in mind, would setting the default c8y.url config look like this?

tedge config set mapper.url "my.c8y.com" --profile c8y

Does it mean that we'll always have to specify the --profile c8y argument for the common settings like url and root_cert_path that don't belong to the C8y::sub_config section?

That UX inconsistency would be a bit weird, where you're trying to configure a single cloud but some are configured with a profile and others aren't.

Yes, you would need to specify the profile name, though currently you could just skip using profiles entirely if you only need to connect to a single cloud. That said, I'm inclined to suggest that for this configuration, profiles should be required (and therefore the c8y profile for mapper could be accessed using mapper.c8y.url instead of faffing around with a --profile argument). However, if you did name the profile c8y, you would then have some confusion between profile names and the c8y config, such as mapper.c8y.c8y.enable.log_upload.