If you set the SVG cache to be on in appsettings.config, and then wish to disable it on a specific tag instance then it is not possible.
Ideally InlineSvgTagHelper.Cache (https://github.com/umbraco-community/Our-Umbraco-TagHelpers/blob/main/Our.Umbraco.TagHelpers/InlineSvgTagHelper.cs#L72) would be nullable, and it would respect a true/false value before considering the appsettings.config value.