Skip to content

Conversation

@iturcino
Copy link

@iturcino iturcino commented Jun 4, 2020

Fixes #25

@OneCricketeer
Copy link
Owner

Thank you!

Overall makes sense. Few comments.

Otherwise, I'm not sure I understand why it was ultimately needed... MM2 should work on its own since this SMT doesn't package anything but Avro and the Registry client, which the HTTP client hasn't changed that much, and it's shaded here too

… ZK lib in provided scope. Added property for jersey.bean.validation version
@iturcino
Copy link
Author

iturcino commented Jun 7, 2020

Updated PR according to comments here

Copy link
Owner

@OneCricketeer OneCricketeer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please bump the pom version as well?

@iturcino
Copy link
Author

iturcino commented Jun 9, 2020

Version bumped to v.0.2.1

@kokachev
Copy link

kokachev commented Jan 20, 2021

Given that this PR has been approved, can @OneCricketeer please merge it and build new 0.2.1 version?

@jipipayo
Copy link

are you going to merge that?

@OneCricketeer
Copy link
Owner

Hi all, I no longer use Kafka for my role, nor this SMT, so am not actively looking at maintaining it.

That being said, Confluent has documentation about migrating schemas, and I am curious if anyone has tried those methods and has feedback on how they differ from this repo?

@msillence
Copy link

Hi all, I no longer use Kafka for my role, nor this SMT, so am not actively looking at maintaining it.

That being said, Confluent has documentation about migrating schemas, and I am curious if anyone has tried those methods and has feedback on how they differ from this repo?

The approach listed here involves setting the schema registry into a special mode where you can specify the id of the schema when you load it. This does work and you can use mirror maker without this SMT and then some other program to simply push the existing schema and id's in

Limitations

  • only works if the target schema registry does not contain any of the id's in your source schema registry
  • other schema registries don't support this import mode such as apicurio though you presumably could use their backup and restore process
  • still requires something to clone the schema registry - may even need a smt if your using mm with an evolving schema

I like your approach and think it's odd that it's not part of mirror maker

Limitations of this approach

  • the magic byte isn't always a guarentee that a binary topic is avro such as the internal ksql or __consumer_offsets

my quick fix is to add an ignore list: iturcino#2

I'm tempted to think it would be better for my use case if it simply ignored the not found or bad magic number

@vl-kp
Copy link

vl-kp commented Apr 8, 2024

anyone can review and merge it?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Mirrormaker 2 updates

6 participants