Skip to content

Commit 07c6a28

Browse files
breaking changes run in ci (Azure#16170)
* adding some work, very wip * running from tox now * refactored breaking changes checkers into class * major refactoring of checkers - no longer dividing client, now looking at base class vars, lists->dicts in json reports * another major refactor, added more checkers, moved to scripts dir * a few fixes, better names * changed function report, added new checks * update to ivars so i'm not passing so many parameters into check funcs * adding first tests, fixing bugs * adding check for ignore breaking changes, fixes a few bugs * added more tests, moved tracker to its own file, fixed ignore, started some docs * added checker for removed kwargs * fix table in readme * fixing naming * add more docs * fix readme formatting * rename model to generic class * bug fixes, redo tests * cleanup * try run in ci * intentionally create a breaking change in form * try removing extra steps * make sure passes with no breaking changes in ci * remove from breaking changes allowlist and make sure doesn't run * add form and text to allowlist * fix tox.ini config * revert options
1 parent a0e9f65 commit 07c6a28

File tree

13 files changed

+6493
-0
lines changed

13 files changed

+6493
-0
lines changed

eng/pipelines/templates/steps/analyze.yml

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -88,3 +88,10 @@ steps:
8888
BuildTargetingString: ${{ parameters.BuildTargetingString }}
8989
TestMarkArgument: ${{ parameters.TestMarkArgument }}
9090
AdditionalTestArgs: ${{parameters.AdditionalTestArgs}}
91+
92+
- template: ../steps/run_breaking_changes.yml
93+
parameters:
94+
ServiceDirectory: ${{ parameters.ServiceDirectory }}
95+
BuildTargetingString: ${{ parameters.BuildTargetingString }}
96+
TestMarkArgument: ${{ parameters.TestMarkArgument }}
97+
AdditionalTestArgs: ${{parameters.AdditionalTestArgs}}
Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
parameters:
2+
BuildTargetingString: 'azure-*'
3+
ServiceDirectory: ''
4+
TestMarkArgument: ''
5+
EnvVars: {}
6+
7+
steps:
8+
- task: PythonScript@0
9+
displayName: 'Run Breaking Changes'
10+
inputs:
11+
scriptPath: 'scripts/devops_tasks/setup_execute_tests.py'
12+
arguments: >-
13+
"${{ parameters.BuildTargetingString }}"
14+
--mark_arg="${{ parameters.TestMarkArgument }}"
15+
--service="${{ parameters.ServiceDirectory }}"
16+
--toxenv="breaking"
17+
--disablecov
18+
env: ${{ parameters.EnvVars }}
19+
condition: and(succeededOrFailed(), ne(variables['Skip.BreakingChanges'],'true'))

eng/tox/tox.ini

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -266,3 +266,16 @@ deps =
266266
commands =
267267
{envbindir}/python -m pip freeze
268268
{envbindir}/python {toxinidir}/../../../scripts/devops_tasks/test_run_samples.py -t {toxinidir}
269+
270+
271+
[testenv:breaking]
272+
skipsdist = false
273+
skip_install = false
274+
usedevelop = true
275+
changedir = {toxinidir}
276+
deps =
277+
{[base]deps}
278+
jsondiff==1.2.0
279+
-e {toxinidir}/../../scripts/breaking_changes_checker
280+
commands =
281+
{envbindir}/python {toxinidir}/../../../scripts/breaking_changes_checker/detect_breaking_changes.py -t {toxinidir}
Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
# Breaking Changes Detector Tool
2+
3+
The breaking changes tool compares the last stable/GA version of the library (if it exists) against the current code
4+
(in master) and reports any breaking changes found.
5+
6+
## How to opt-in to running the tool in CI
7+
8+
Add your package name to the `RUN_BREAKING_CHANGES_PACKAGES` found [here](https://github.com/Azure/azure-sdk-for-python/tree/master/scripts/breaking_changes_checker/breaking_changes_allowlist.py).
9+
10+
## Run locally with tox
11+
12+
**1) Install tox and tox-monorepo:**
13+
14+
`pip install tox tox-monorepo`
15+
16+
**2) Run the `breaking` environment.**
17+
18+
Here we run the breaking changes tool against azure-storage-blob, for example:
19+
20+
`C:\azure-sdk-for-python\sdk\storage\azure-storage-blob>tox -c ../../../eng/tox/tox.ini -e breaking`
21+
22+
23+
## Ignore a reported breaking change
24+
25+
A breaking change reported by this tool may be an acceptable change. If it is an **approved** breaking change (signed off by architecture board)
26+
or a false positive, then you should add the breaking change to the ignore list.
27+
28+
The ignore list is found [here](https://github.com/Azure/azure-sdk-for-python/tree/master/scripts/breaking_changes_checker/breaking_changes_allowlist.py).
29+
30+
To add an ignore, you will need the identifier of the breaking change. This includes the breaking change type,
31+
module name, and optionally class and/or function name, in this order, e.g.
32+
33+
`(breaking-change-type, module-name, class-name, function-name)`
34+
35+
Add this signature as a list of tuples under your package name in the [IGNORE_BREAKING_CHANGES](https://github.com/Azure/azure-sdk-for-python/tree/master/scripts/breaking_changes_checker/breaking_changes_allowlist.py) dictionary.
36+
Note that the names used should be those from the _stable_ package. If I renamed my function from `begin_training` to
37+
`begin_train_model`, I would put `begin_training` as my function name.
38+
39+
See ignore signature skeletons for each type of breaking change [below](#types-of-breaking-changes-detected)
40+
41+
**Examples:**
42+
43+
```python
44+
IGNORE_BREAKING_CHANGES = {
45+
"azure-ai-formrecognizer": [
46+
("RemovedOrRenamedClientMethod", "azure.ai.formrecognizer.aio", "FormTrainingClient", "begin_training"),
47+
("RemovedOrRenamedClass", "azure.ai.formrecognizer", "FormElement"),
48+
],
49+
"azure-storage-queue": [
50+
("RemovedOrRenamedModule", "azure.storage.queue.aio"),
51+
("RemovedOrRenamedModuleLevelFunction", "azure.storage.queue", "generate_queue_sas")
52+
]
53+
}
54+
```
55+
56+
57+
## Types of Breaking Changes Detected
58+
59+
> Note that this does not cover every kind of breaking change possible.
60+
61+
| Breaking Change Type | Explained (changes are relative to the stable/GA library version) | Ignore signature IF an approved breaking change or false positive |
62+
|------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------|
63+
| RemovedOrRenamedModule | An entire module was removed or renamed in the current version. E.g. `aio` module was removed. | ("RemovedOrRenamedModule", "module-name")
64+
| RemovedOrRenamedClient | A client was removed or renamed in the current version. | ("RemovedOrRenamedClient", "module-name", "client-name")
65+
| RemovedOrRenamedClientMethod | A client method was removed or renamed in the current version. | ("RemovedOrRenamedClientMethod", "module-name", "client-name", "function-name")
66+
| RemovedOrRenamedClass | A model or publicly exposed class was removed or renamed in the current version. | ("RemovedOrRenamedClass", "module-name", "class-name")
67+
| RemovedOrRenamedClassMethod | A model or publicly exposed class' method was removed or renamed in the current version. | ("RemovedOrRenamedClassMethod", "module-name", "class-name", "function-name")
68+
| RemovedOrRenamedInstanceAttribute | An instance attribute was removed or renamed in the current version. | ("RemovedOrRenamedInstanceAttribute", "module-name", "class-name")
69+
| RemovedOrRenamedEnumValue | An enum value was removed or renamed in the current version | ("RemovedOrRenamedEnumValue", "module-name", "class-name")
70+
| RemovedOrRenamedModuleLevelFunction | A module level function was removed or renamed in the current version. | ("RemovedOrRenamedModuleLevelFunction", "module-name", "function-name")
71+
| RemovedOrRenamedPositionalParam | A positional parameter on a function was removed or renamed. | ("RemovedOrRenamedPositionalParam", "module-name", "class-name", "function-name")
72+
| AddedPositionalParam | `def my_function(param1) --> def my_function(param1, param2)` | ("AddedPositionalParam", "module-name", "class-name", "function-name")
73+
| RemovedParameterDefaultValue | `def my_function(param=None) --> def my_function(param)` | ("RemovedParameterDefaultValue", "module-name", "class-name", "function-name")
74+
| ChangedParameterDefaultValue | `def my_function(param="yellow") --> def my_function(param="blue")` | ("ChangedParameterDefaultValue", "module-name", "class-name", "function-name")
75+
| ChangedParameterOrdering | `def my_function(a, b, c=None) --> def my_function(b, c=None, a=None)` | ("ChangedParameterOrdering", "module-name", "class-name", "function-name")
76+
| RemovedFunctionKwargs | A function was changed to no longer accept keyword arguments. `def my_func(param, **kwargs) --> def my_func(param)` | ("RemovedFunctionKwargs", "module-name", "class-name", "function-name")
77+
| ChangedParameterKind | `def my_function(a, b, c) --> def my_function(a, b, *, c)` | ("ChangedParameterKind", "module-name", "class-name", "function-name")
78+
| ChangedFunctionKind | `async def my_function(param) -> def my_function(param)` | ("ChangedFunctionKind", "module-name", "class-name", "function-name")
79+

scripts/breaking_changes_checker/__init__.py

Whitespace-only changes.
Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
#!/usr/bin/env python
2+
3+
# --------------------------------------------------------------------------------------------
4+
# Copyright (c) Microsoft Corporation. All rights reserved.
5+
# Licensed under the MIT License. See License.txt in the project root for license information.
6+
# --------------------------------------------------------------------------------------------
7+
8+
9+
RUN_BREAKING_CHANGES_PACKAGES = [
10+
"azure-ai-formrecognizer",
11+
"azure-ai-textanalytics"
12+
]
13+
14+
15+
# See Readme for ignore format
16+
17+
IGNORE_BREAKING_CHANGES = {}

0 commit comments

Comments
 (0)