fix: Restore fails with bufio.Scanner: token too long when importing large dump files

[n2em]

bug fix for #469

[deitch]

I guess #469 was not totally fixed?

What is the root cause? Looking at the code here, it looks to me like you just recreated what scanner does? So why does this work?

[n2em]

The core issue lead to this bug is the const value of defaultMaxAllowedPacket, it just hard code to 4194304, if the line of SQL statements exceed this limitation, the bufio.Scanner will throw exception: token too long. In our case, the dump file size is about 7GB, and the max records in single table is about 50M, it just broken when restore with version 1.3.0, with this commit , the restore works fine.

[n2em]

We can simply reproduce the issue by restore the database by below SQL.

create database test_restore;
use test_restore;
create table test_large(
	id int not null primary key,
    content mediumtext null
);
insert into test_large values(1,  REPEAT('A', 4194304)); 

Once restore the backup, the restore got error.

 $ ./mysql-backup restore db_backup_2025-11-17T02:27:02Z.tgz
INFO[0000] beginning restore                             run=e911423c-32d8-4196-b305-a1917a4bb172
Error: error restoring: failed to restore database: failed to restore database: bufio.Scanner: token too long
FATA[0035] error restoring: failed to restore database: failed to restore database: bufio.Scanner: token too long

[deitch]

So something about bufio.Scanner hits a limit? That reproduction is good, I am going to dig a little, see if I can figure out what it is about bufio.Scanner.

[deitch]

Thanks for the reproduction use case; it is very helpful.

[deitch]

Right. Scanner has a max size, which you can increase via s.Buffer(). Reader, on the other hand, does not appear to have such a limit, which is strange. Either way, I can see why what you did here makes sense.

[deitch]

I think it needs a rebase.

[n2em]

rebase has done.

[deitch]

Let CI do its magic.

[deitch]

linting error

[n2em]

I should remove the unused const field. I will commit a update.