Version 0.22

[dpo]

@amontoison @tmigot

I'm opening this discussion to better understand the situation and so we can discuss a path forward.

The good news is that, currently, nothing is broken if no package upgrades to 0.22.

Things I need to understand:

• Is the new API for jac_lin_coord the only offending change?
• Looking at Version 0.22.0 #519, the breakage worklow shows 100% red. So why was the PR merged?
• Even in Implement default jac_lin* API to patch 0.22 #522 it's all red. Is something broken with the breakage workflow? That would be the first priority.

Observations open for comments:

• Even if we removed v0.22 from the registry, because the offending commits are already in main, any future release will contain them. So I don't see how removing the release solves anything.
• As far as I can tell, only a handful of packages are affected directly by the changes in v0.22 (https://github.com/search?q=org%3AJuliaSmoothOptimizers+jac_lin+language%3AJulia&type=code&l=Julia). I count 5 packages there.
• The changes in v0.22 that were not in the previous release don't break anything in the remaining packages that depend on NLPModels directly. So upgrading those should be trivial (though I realize it takes some time).

[amontoison]

If we remove the release 0.22, even if we got breaking changes with some previous commits, we can do modifications to revert them or make them non-breaking.
Between two releases we can what we want without consequences and it is what will happen if we yank the release 0.22.

[amontoison]

@tmigot @dpo,

I will try to explain clearly why the situation is very problematic on my side, and why the (core) issues we are seeing are not caused by the code itself. I polished my answer 😉

Is the new jac_lin_coord API the only offending change?

Yes, this is the only breaking change at the code level.

Why is the breakage workflow all red, and why was the PR merged?

Nothing is actually "broken" in the workflow.
The failures come from version constraints, not code breakage:

• All downstream packages currently cap NLPModels to 0.21.
• Because of that, CI refuses to install any release with NLPModels ≥ 0.22, meaning it cannot run tests at all.
• As a result, the breakage workflow returns 100% red simply because it cannot instantiate the environment.

The package manager is blocking installation of 0.22, which prevents us from testing or releasing new features safely.

Why this is a serious problem for the ecosystem

The core problem is the version bump, not the code.

A minor release like 0.22 means that:

• any package that adopts new features must require NLPModels ≥ 0.22,
• while packages that haven't migrated still require ≤ 0.21.

This causes an ecosystem-wide divergence:

• solvers cannot test or release new versions,
• modeling packages cannot install 0.22 in CI,
• the breakage workflow becomes useless,
• new features in NLPModels cannot be introduced without blocking everything downstream.

This effectively freezes development.

Concrete impact on my work

To be totally transparent, this situation has a very significant impact on my work, and is extremely frustrating.

I have several non-breaking improvements planned for NLPModels and the solvers (Uno, KNITRO, MadNLP, ExaModels, etc.), for which I am paid to deliver.

A few examples:

• Adding boolean attributes to NLPModelsMeta to indicate support for the linear API,
• Specifying availability of operator routines (hprod, jprod, jtprod) important for UnoSolver.jl and NLPModelsJuMP.jl,
• Adding future APIs such as batch evaluations.

All of these would have been trivial non-breaking additions in 0.21.x, deployable in minutes and usable immediately across the ecosystem.

But now, because 0.22 is a breaking release:

• I cannot add these features until the entire ecosystem moves to 0.22,
• This requires updating dozens of packages,
• Most of which you can't update because in other organization (Exanauts, MadNLP, OptimalControl) or standalone packages (GALAHAD.jl, UnoSolver.jl).

Although I did not request this release, the current state requires me to coordinate a massive migration before I can continue development.
This is extremely disruptive and blocks concrete deliverables I am supposed to finish before leaving ANL.

Why #522 does not solve the issue

As I explained in #522 comment:

Even if #522 makes the API non-breaking, the version bump remains.
All dependents still cap at 0.21, meaning CI will still refuse to install 0.22.

Therefore, updating all Project.toml and new releases of (almost) all dependent packages is required before any new features can be safely added.

About "removing the release doesn’t help"

Actually, removing (yanking) the release does help if we also revert or adjust the commits in the branch main:

• We can revert or rewrite the changes to make them non-breaking,
• We can continue releasing non-breaking updates in the 0.21.x series,
• We avoid freezing the entire ecosystem,
• Development of new features can proceed normally.

Right now, every future version of NLPModels must inherit from a breaking release, forcing a massive migration effort before anything else can be done.

Documentation: When is yanking a release appropriate?

Final request

Given the constraints above, and the impact on both my work and the ecosystem, I sincerely ask that we yank NLPModels 0.22 from the registry so we can either:

• Revert the breaking commits, or
• Redesign the changes to be non-breaking and release them safely in 0.21.x.

This is the only way to avoid freezing the ecosystem and to allow continued development of modeling tools and solvers.

I hope this explanation clarifies the situation.

[amontoison]

cc @frapac @sshin23

[tmigot]

My plan was to upgrade all the JSO packages to NLPModels 0.22. Most of the important changes had already been prepared by someone else, and I had already scheduled time to coordinate the update across the organization myself — which usually takes about two days.

Outside the organization, I’m of course happy to help, but I’m not responsible for coordinating updates there.

Changing a function signature is indeed rare, but in this case it’s a relatively straightforward modification. The only time-consuming part is bumping the versions, and we had good momentum, with several people available to work on it. This did not assume that you would have to handle it, Alexis.

In my opinion, yanking the release and re-adding code to maintain the deprecated signature only postpones the issue. I had the opportunity to update things quickly, which is why the release was made.

For urgent code extending the extension, you can always add code directly in the package where you use it and later add it to NLPModels also.

Despite all this, I’m completely fine with going with the majority decision from here.

[amontoison]

Outside the organization, I’m of course happy to help, but I’m not responsible for coordinating updates there.

Nobody ask for a breaking release for removing a x in the signature of 3 functions of the linear API.
When we have a package like NLPModels.jl used by more 30 other registered packages, we can't do that without thinking of consequences.

In my opinion, yanking the release and re-adding code to maintain the deprecated signature only postpones the issue. I had the opportunity to update things quickly, which is why the release was made.

Do you think that having a x in the linear API was a big issue? Guys, come on we took decision a long time ago and it was fine. Nobody complained about that and after all it leads to the same signature in jac_coord, jac_lin_coord and _jac_nnln_coord`.
It was really not critical...
The cost of maintenance and the risk of divergence that we have now make non-sense for this modification.

We vould have also waited other breaking changes to do a breaking release. It was what I did with Krylov.jl v0.10, I waited 3 years before doing it.

For urgent code extending the extension, you can always add code directly in the package where you use it and later add it to NLPModels also.

No I can't because it should be inside the AbstractNLPModel...
How do you add a boolean in your model to specify that you have a linear api or an operator available from a solver package like UnoSolver.jl?
You can't.

Despite all this, I’m completely fine with going with the majority decision from here.

I sincerely think than yanking the release is the best solution.