Feedback after migrating from diffx

[ghostbuster91]

The purpose of this issue is to gather feedback from migrating projects to difflicious.

ghostbuster91/pgn-parser#2 - .assertNoDiff has to be called on the differ instance. For more complex types it is cumbersome. It would be better to write it in a way where the type can be inferred from the passed arguments.

[kostaskougios]

Yes me too. I just gave a try to the library but gave up when I realized what I have to do something like Diff[Seq[Player]].assert... . I think something like Diff.assertNoDiff(a1,a2) that infers the type would be a lot simpler.

[2m]

I also recently migrated from diffx and I am using this assertion method variation from the munit support:

https://github.com/2m/rallyeye/blob/391227a93eb4b608416b6745595280d2c9ac04aa/modules/backend/src/test/scala/difflicious.scala#L24-L26

[nafg]

Would be good to have an equivalent of Diff.approximate

[nafg]

Why do only primitive differs have contramap?

[jatcwang]

@nafg Thanks for the feedback. It can be done - just need to capture the name of the underlying type for better error reporting. Created #117

[baldram]

Hey @jatcwang and @ghostbuster91 !

Our migration from Dffx to Difflicious turned out to be a failure.

Preparing for the migration to Scala 3, we first switched from Dffx to Difflicious, and that was seamless. Everything continued to work, no code changes were needed (since we have our own mini assert wrapper, so the swap was internal and in target applications practically nothing had to be changed). The only modification was changing .ignore to .ignoreAt and something with .modify, which is rarely used—some test had to be rewritten. Other than that, across dozens of products, it was a mechanical, very low-effort process.
So far, so good—almost too good. The calm before the storm.

Then the moment came. We switched to Scala 3, and that’s where our adventure ended. It was Scala 3.7.1. When the compiler told me Try increasing -Xmax-inlines above 300, I knew it was pointless to continue. The lack of semi-auto makes the library unusable in an industrial setting.

Knowing how things work, I tried workarounds and in each test I created a single top-level differ, one for all test scenarios, to limit resource consumption. Then the whole hierarchy would be handled automatically. Writing everything fully manually is pointless, because if a case class has numerous nested types at multiple levels—a complex hierarchy—that means 20-30 or more manually written differs for each type for a single model, which is unmaintainable. Every model modification requires changes. And the compilation times are a nightmare. We still needed "auto" for these nested types, so it was slow.

For medium-sized models, I managed to gradually work around the problems by creating manual differs. But for the large ones, there was no chance (even manual types, trying different things, it was not compiling at all). I saw enough stack overflow and other interesting logs to lose all hope. And we’re not doing anything extraordinary—no recursive structures, nothing like that. Just integration with other services, event model implementation, or responses from some API—typical stuff. Jsoniter Scala generates codecs hierarchically for these models without a hitch, supporting even the most complex structures without any impact on compilation time. This is nothing beyond typical use cases in somewhat larger applications—exactly what we do.
Generally, generic derivation should be so fast you don't notice. It's not an issue with Scala 3 or something. However, using semi-auto is a good way to dramatically reduce the amount of work that's required. And we miss this feature in Difflicious.

As long as there’s no support for semi-auto, this is not a library for industrial use.
I’m looking for an alternative. I’ll check if Diffx still works after upgrading the Magnolia version and whether things are better there, since it has semi-auto. I’m also considering soundness/chiaroscuro (from the creator of Magnolia), as well as adopting ZIO Schema, which doesn’t have a presentation layer but does have a diff algorithm implemented, which is at least a starting point.

This is my feedback.
Thank you, cheers,
Marcin

[nafg]

I'm guessing it wouldn't be to hard to add semi-auto, whether as a contribution or in your own codebase?

[baldram]

I'm guessing it wouldn't be to hard to add semi-auto, whether as a contribution or in your own codebase?

If I have some guidance, as I'm not macro beast. Maybe someone has an idea and the motivation to take care of this; it would be a highly desirable feature.

In the meantime, I tested Diffx—a bit of an exhumation ;) I forced the latest version of Magnolia. I checked on Scala 3.3.x and 3.7.x, and unfortunately in both cases, the library only works if I manually create a Diff for each individual element. There is no semi-auto anymore, only full manual remains. Hierarchical derivation is broken. This means that for a simple, slightly more complex case, it’s about 30 manually written differs. It’s unmaintainable for users.

[jatcwang]

@baldram Thanks for the feedback and I can sympathize. We're currently using magnolia for both Scala 2 & 3 version of this library. While I'm not sure writing Difflicious's derivation code by hand (i.e. not use magnolia) for Scala 3 will help fix the Stack Overflow issues or reduce compile-time overhead, I think it's worth trying so we have a bit more flexibility when it comes to derviation behaviour.

Our definition of semi-auto is different: Yours is based on jsoniter-scala's derivation behaviour where there's "No need to derive intermediate codecs for inner nested classes like Device if you are not going to parse/serialize them from/to JSON in isolation". This is different from the "semi-auto" derivation behaviour of other libraries (e.g. Circe) and I'm not sure magnolia supports it either.
I think the behaviour you're looking for makes sense for a test library. Created #121 to look into it

[baldram]

@jatcwang Thank you for reading my comments and creating an issue based on them.

I managed to make some improvements to Difflicious, which we currently have in our codebase. I’ll need to gather those changes and propose some PRs. However, for now, we’re replacing Difflicious with a workaround (we need to unblock the Scala 3 migration). I was able to implement type-checked paths for the .ignoreAt feature, identical to what Difflicious offers. That was super easy via QuickLens, here is how. I’ll also be extending the built-in assertEquals in MUnit, which provides highlighted diffs for case classes (by the way, it’s implemented in a surprisingly simple way—objects are rendered as pretty-printed strings and compared). I’m still testing how reliable this workaround will be, but it seems promising so far.

At this point, I’ll need to write an MUnit-to-ScalaTest adapter, since we’re using ScalaTest. In the future, we can always replace this with something else—maybe even Difflicious again. As we don't use libraries directly in our products for such things, but via adapters/facades. Thanks to that we can easily swap underlying library anytime without doing any changes in dozens of products.