-
Notifications
You must be signed in to change notification settings - Fork 50
dcnm_vrf_v2: Pydantic integration (replaces #400) #550
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: develop
Are you sure you want to change the base?
Conversation
No functional changes in this commit. - diff_for_attach_deploy Update code comments for better clarity.
No functional changes in this commit. - format_diff_orig Remove method after verifying integration tests pass with refactored methods.
1. get_diff_delete Refactor into the following two methods: - _get_diff_delete_with_config Detach, undeploy, and delete the VRFs specified in self.config - _get_diff_delete_without_config Detach, undeploy, and delete all VRFs. We are retaining the original method, renamed to get_diff_delete_orig, until we’ve verified that integration tests pass.
1. get_diff_replaced - Simplify logic We are retaining the original method, renamed to get_diff_replaced_orig, until we’ve verified that integration tests pass.
Remove the following original methods after verifying that integration tests pass. - get_diff_delete_orig - get_diff_replace_orig
- update_vrf_attach_vrf_lite_extensions - Simplify logic This is part 1 of what is likely a multi-part refactor. We are retaining the original method, renamed to update_vrf_attach_vrf_lite_extensions_orig, until we verify that integration tests still pass.
1. update_vrf_attach_vrf_lite_extensions_orig Remove method after verifying integration tests pass.
NOTE: We are retaining the original methods (renamed to *_orig) until we’ve verified that integration tests pass. This commit infuses Pyydatic deeper into the dcnm_vrf’s methods. The methods below are all within class NdfcVrfV12. 1. update_lan_attach_list - call get_vrf_lite_objects_model instead of get_vrf_list_objects - operate on the list[ExtensionPrototypeValue] (rather than original list[dict]) - pass lite (now a list of ExtensionPrototypeValue) to update_vrf_attach_vrf_lite_extensions 2. update_vrf_attach_vrf_lite_extensions - Modify to work with list of ExtensionPrototypeValue - call get_extension_values_from_lite_objects with list of ExtensionPrototypeValue - get_extension_values_from_lite_objects now returns a list of VrfLiteConnProtoItem - Use this list of VrfLiteConnProtoItem to update nbr_dict 3. get_extension_values_from_lite_objects This method now operates on Pydantic models exclusively. 4. log_list_of_models It is anticipated that we will be working with lists of models extensively. This utility method logs a list of pydantic models by calling json.dumps(model.model_dump()) on each model in the list. It is currently called from all of the above methods.
Remove original methods after verifying intergration tests pass. Removed the following: - get_extension_values_from_lite_objects_orig - update_vrf_attach_vrf_lite_extensions_orig - update_lan_attach_list_orig Updated docstrings for the replacement methods - get_extension_values_from_lite_objects - update_vrf_attach_vrf_lite_extensions - update_lan_attach_list
1. _update_vrf_lite_extension - call self.get_vrf_lite_objects_model instead of self.get_vrf_lite_objects - leverage vrf_lite model (ControllerResponseVrfsSwitchesV12) to update attach - Rather than hardcode AUTO_VRF_LITE_FLAG to “false”, try to assign the value from the switch (vrf_lite_conn_model.auto_vrf_lite_flag). Assign “false” only if the switch does not provide a value. - Add debug logs.
1. get_vrf_lite_objects_model - Return a list of VrfsSwitchesDataItem models - Renamed to get_list_of_vrfs_switches_data_item_model 2. get_vrf_lite_objects - Remove unused method 3. Multiple methods - Update calls to renamed get_vrf_lite_objects_model to call updated name get_list_of_vrfs_switches_data_item_model - Update accesses to the model given the new returned value is a list of models. 4. Multiple methods - Update debug log to call log_list_of_models when logging the list of VrfsSwitchesDataItem models
No functional changes in this commit. - compare_properties Rename to property_values_match
1. get_vrf_objects - Rename to get_controller_vrf_object_models - Return a list of VrfObjectV12 rather than full controller response 2. Update all methods that directly call this method (listed below) - get_have - get_diff_query 3. Update all methods, called from the above two methods, that process the resulting list of models (listed below) - populate_have_create - get_diff_query_for_vrfs_in_want - get_diff_query_for_all_controller_vrfs 4. Rename var vrf_objects_model to vrf_object_models to better reflect that this is a list of vrf object models. - Rename this var across all methods above. 5. Call log_list_of_models on this list of models for debug logging in the methods below - get_diff_query - get_have
No functional changes in this commit. 1. get_have - Update log message with correct var name - Add missing call to self.log.debug() 2. get_diff_query - Update log message to match message in 1 above.
- _extension_values_match want_ext is set twice in a row. Remove redundant occurance.
1. get_vrf_lan_attach_list - Rename to get_controller_vrf_attachment_models - Return a list of VrfsAttachmentsDataItem 2. Update all calling methods - Use the new name and return value - Log return value using log_list_of_models - Rename var containing the return value to vrf_attachment_models
No functional changes in this commit. Rename vars that contain models to: - *_model - for single model - *_models - for lists of models
No functional changes in this commit. 1. get_diff_replace - Rename vars for readability
No functional changes in this commit. 1. get_diff_replace - Rename vars for readability
No functional changes in this commit.
No functional changes in this commit.
Fix errors below. ERROR: Found 4 pep8 issue(s) which need to be resolved: ERROR: plugins/module_utils/vrf/dcnm_vrf_v12.py:1905:32: E251: unexpected spaces around keyword / parameter equals ERROR: plugins/module_utils/vrf/dcnm_vrf_v12.py:1905:34: E251: unexpected spaces around keyword / parameter equals ERROR: plugins/module_utils/vrf/dcnm_vrf_v12.py:1906:32: E251: unexpected spaces around keyword / parameter equals ERROR: plugins/module_utils/vrf/dcnm_vrf_v12.py:1906:34: E251: unexpected spaces around keyword / parameter equals
1. plugins/module_utils/network/dcnm/dcnm.py No changes were made to this file. However, the function dcnm_get_url is not a generic function and so should be moved under module_utils/vrf (IMHO). We’ve left the original method, since it it still used by: - dcnm_vrf_v11.py - test_dcnm_vrf_v11.py We have rewritten and renamed dcnm_get_url and now use the rewritten version in the following files: - plugins/module_utils/vrf/dcnm_vrf_v12.py - tests/unit/modules/dcnm/test_dcnm_vrf_12.py The rewritten version is renamed to get_endpoint_with_long_query_string and is located here: - plugins/module_utils/vrf/vrf_utils.py 2. While debugging this new function, it was observed that the following unit test was passing, but it was passing for the wrong reason: - test_dcnm_vrf_12_merged_redeploy The code tested by the above unit test calls dcnm_get_url, but there was no side_effect defined for it e.g.: self.run_dcnm_get_url.side_effect = [self.mock_vrf_attach_object_pending] Because of this, the controller response was a string containing something along the lines of “<Mock id=(memory_address)>” rather than an actual mocked response. After adding the above side_effect, the test is now passing for (hopefully) the right reason, and the controller response is as expected. 3. plugins/module_utils/vrf/dcnm_vrf_v12.py - Remove import for dcnm_get_url - Add import for get_endpoint_with_long_query_string - get_have - modify to use get_endpoint_with_long_query_string - Validate the controller response with a Pydantic model We don’t currently pass the model to populate_have_deploy and populate_have_attach. We’ll do that in a separate commit.
No functional changes in this commit.
1. plugins/module_utils/vrf/dcnm_vrf_v12.py
- populate_have_attach
Now that the last commit fixed the unit test Mock issue, we can use json.dumps() to log have_attach, since it will never be a string.
2. Use json.dumps(indent=4, sort_keys=True) wherever possible.
- send_to_controller
- Use json.dumps() only if payload is not None.
- update_lan_attach_list
- Usee json.dumps() only if vrf_attach.get('vrf_lite') is not None
3. Standardize message when logging list of models.
This commit contains experimental changes. Original code is retained so that we can easily revert. 1. get_have - Call populate_have_attach_model rather than populate_have_attach 2. populate_have_attach_model - Same functionality as populate_have_attach, but using models. - Uses new model HaveLanAttachItem, which models new_attach (populate_have_attach). - Calls _update_vrf_lite_extension_model 3. _update_vrf_lite_extension_model - Same functionality as _update_vrf_lite_extension, but uses the new HaveLanAttachItem model. 4. format_diff_attach - The new model (HaveLanAttachItem) has key vlanId instead of vlan. For now, we set vlan_id from either vlan or vlanId, whichever is not None. We’ll fix this later. 4. format_diff_create - Same temporary hack as format_diff_attach 5. tests/unit/modules/dcnm/test_dcnm_vrf_12.py Update unit tests to expect an integer vlan_id rather than a string, since the model uses integers. 6. populate_have_attach - Update the docstring with more detail about what is mutated - sort new_attach alphabetically 7. plugins/module_utils/vrf/controller_response_vrfs_attachments_v12.py - Add the following optional fields - freeform_config - extension_values - instance_values
Fixed the following errors: ERROR: plugins/module_utils/vrf/dcnm_vrf_v12.py:1402:22: f-string-without-interpolation: Using an f-string that does not have any interpolated variables ERROR: plugins/module_utils/vrf/dcnm_vrf_v12.py:1419:22: f-string-without-interpolation: Using an f-string that does not have any interpolated variables ERROR: plugins/module_utils/vrf/dcnm_vrf_v12.py:1425:22: f-string-without-interpolation: Using an f-string that does not have any interpolated variables ERROR: plugins/module_utils/vrf/dcnm_vrf_v12.py:1558:8: logging-fstring-interpolation: Use lazy % formatting in logging functions
Fix the. following: ERROR: plugins/module_utils/vrf/dcnm_vrf_v12.py:1262:161: E501: line too long (548 > 160 characters)
Add have_attach_post_mutate_v12.py
1. plugins/module_utils/vrf/dcnm_vrf_v12.py - populate_have_attach_model - Add type hints for all lists of models - Rename dict new_attach to new_attach_dict - Remove unsed method_name var - _update_vrf_lite_extension_model - Fix type hint for return value
commit c619a1f Author: Sivakami Sivaraman <sivakasi@cisco.com> Date: Sat Nov 15 20:22:37 2025 +0530 Resolving sanity errors in base collection without ignore (#539) * dcnm_policy and dcnm_vrf changes in integ tests * Changeset to resolve sanity errors in base collection without ignore * Revert "dcnm_policy and dcnm_vrf changes in integ tests" This reverts commit a2c4322. * Changes to ignore GPL validation * Addressed import issues with netcommon in environment. Changes not required * Resolving few lint errors in addition * Removing a temporary import added commit dd9d426 Author: mwiebe <mwiebe@cisco.com> Date: Fri Nov 14 12:15:23 2025 -0500 Revert "Allow DHCP server to be configured without a VRF (#533)" This reverts commit 2a5de47. commit 2a5de47 Author: Miguel Corona <miguel_corona@hotmail.com> Date: Thu Nov 13 13:31:44 2025 -0600 Allow DHCP server to be configured without a VRF (#533) * Fix for allowing DHCP srvr config without VRF * Fix issue to allow DHCP server config without VRF in Networks * Fix issue with networks that don't have DHCP servers from the begining * Fix merge issue with develop --------- Co-authored-by: miguecor <miguel_corona@hotmail.com.com>
No functional changes in this commit. This commit adds import guards for all VRF classes that require pydantic.
Remove unused import
Use from __future__ import annotations to enable using pipe operator rather than typing.Union in Python 3.9 This avoids Ansble sanity test errors like below when running ansible-test santiy against Python 3.9 ERROR: plugins/module_utils/vrf/dcnm_vrf_v12.py:84:0: traceback: TypeError: unsupported operand type(s) for |: 'types.GenericAlias' and 'type' (at plugins/module_utils/vrf/model_controller_response_vrfs_v12.py:185:0)
A new ignore-2.19.txt was added recently. Adding dcnm_vrf_v2.py to this file to pass sanity.
1. Leverage central pydantic import guards 2. Leverage from __future__ import annotations to remove Python 3.9 Union[] in favor of pipe operator
1. Convert all import guards related to Pydantic to leverage the centralized guards in module_utils/common/third_party/pydantic.py 2. Add from __future__ import annotations so that we no longer have to use legacy typing.Union for Python 3.9 3. Replace all Union[foo, bar] with foo | bar
While _ is a widely-accepted variable name to signify that it is unused, Ansible is behind the curve here. So, change the name to _value instead.
Actually not strictly needed in dcnm_vrf_v2.py, but doesn’t hurt to add it. Needed in vrf_controller_to_playbook_v11.py
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR introduces Pydantic-based validation for the dcnm_vrf module through a new v2 implementation (dcnm_vrf_v2.py). The changes bring the v2 module to feature-parity with the existing dcnm_vrf.py module while refactoring to use Pydantic models for validation and separating DCNM v11 and NDFC v12 code into distinct classes.
Key changes include:
- Integration of Pydantic model-based validation replacing custom in-code validations
- Separation of DCNM v11 and NDFC v12 implementations into separate classes
- Comprehensive test coverage for both v11 and v12 versions
- Updated test fixtures and new test files for model validation
Reviewed changes
Copilot reviewed 86 out of 99 changed files in this pull request and generated 17 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/unit/modules/dcnm/test_dcnm_vrf_v2_12.py | New comprehensive unit tests for NDFC v12 implementation (1278 lines) |
| tests/unit/modules/dcnm/test_dcnm_vrf_v2_11.py | New comprehensive unit tests for DCNM v11 implementation (1249 lines) |
| tests/unit/modules/dcnm/test_dcnm_vrf.py | Renamed test methods from test_dcnm_vrf_* to test_dcnm_vrf_v1_* for clarity |
| tests/unit/modules/dcnm/fixtures/dcnm_vrf_12.json | New test fixtures for v12 testing (1637 lines) |
| tests/unit/modules/dcnm/fixtures/dcnm_vrf_11.json | New test fixtures for v11 testing (2011 lines) |
| tests/unit/modules/dcnm/fixtures/dcnm_vrf.json | Minor formatting fix (added missing newline) |
| tests/unit/module_utils/vrf/test_model_payload_vrfs_deployments.py | New unit tests for PayloadVrfsDeployments model |
| tests/unit/module_utils/vrf/fixtures/model_payload_vrfs_attachments.json | New test fixtures for VRF attachments payload model |
| tests/unit/module_utils/common/models/test_ipv6_host.py | New unit tests for IPv6HostModel validation |
| tests/unit/module_utils/vrf/init.py | New empty init file for vrf test module |
| tests/unit/module_utils/common/models/init.py | New empty init file for models test module |
| tests/unit/init.py | New empty init file for unit tests |
| tests/sanity/ignore-2.19.txt | Added ignore rule for dcnm_vrf_v2.py license validation |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
plugins/module_utils/vrf/model_controller_response_vrfs_switches_v12.py
Outdated
Show resolved
Hide resolved
plugins/module_utils/vrf/model_controller_response_vrfs_switches_v12.py
Outdated
Show resolved
Hide resolved
plugins/module_utils/vrf/model_controller_response_vrfs_switches_v12.py
Outdated
Show resolved
Hide resolved
plugins/module_utils/vrf/model_controller_response_vrfs_switches_v12.py
Outdated
Show resolved
Hide resolved
plugins/module_utils/vrf/model_controller_response_vrfs_switches_v12.py
Outdated
Show resolved
Hide resolved
1. Remove commented code We accidtemtly left these comments in place while testing our solution to 3rd-party library imports.
Will remove commented code if this succeeds.
1. Remove unneeded 3rd-party import checks (these are handled in support libraries). 2. Add comment as to why we simpy pass when logging setup fails.
Replaced unsed vars with __. These are using merely to test for various conditions.
1. Add comment to explain why we are passing without raising an exception. 2. Change prefixlen to __ per conventions around unused vars.
Remove unreachable code.
Summary
This PR is based on #400 and integrates all PRs related to dcnm_vrf.py that have been merged into the develop branch back into dcnm_vrf_v2.py. Hence, dcnm_vrf_v2.py is now at feature-parity with dcnm_vrf.py in the develop branch.
Below is the content of #400 describing the objectives for dcnm_vrf_v2.py.
dcnm_vrfintegration tests for readabilityThis PR also introduces the following:
Notes to reviewers
Details
Added files
plugins/module_utils/common/enums/ansible.pyEnumerations related to Ansible.
plugins/module_utils/common/enums/bgp.pyplugins/module_utils/common/enums/request.pyEnumerations related to HTTP requests
plugins/module_utils/vrf/vrf_controller_to_playbook_v12.pyVrfControllerToPlaybookV12ModelSerialize NDFC version 12 controller payload fields to fields used in a dcnm_vrf playbook.
plugins/module_utils/vrf/vrf_controller_to_playbook.pyVrfControllerToPlaybookModelSerialize payload fields (common to NDFC versions 11 and 12) to fields
used in a dcnm_vrf playbook.
plugins/module_utils/vrf/vrf_playbook_model.pyValidation model for dcnm_vrf playbooks
plugins/module_utils/common/models/ipv4_cidr_host.pyModel to validate a CIDR-format IPv4 host address.
Changed files
All integration tests
"{{ var }}"was used in asserts, change tovarplugins/modules/dcnm_vrf.py