[wip][core] improve SST File format

[steFaiz]

Purpose

The linked issue: #6734

This PR is about to bring SST File VERSION 1

NOTE: current SST File do not have a VERSION, so this version is not compatible with current SST File Format (At least it's hard to do so)

Core improvements

1. Introduce leveled data index: user can specify a MaxIndexBlockSize, if the index block memory exceeds this threshold, the index block will be spilled to SST File as a B-Tree like structure. The reader will only load the root index on opening.
2. Introduce a new FileInfo block, containing some stats and users are free to add new k-v pairs.
3. Modify the footer structure, including:
   1. add some stats such as uncompressed data size, uncompressed index size, row count and more
   2. compression type is moved from BlockTrailer. This follows Hbase's design, so that we do not have to create a compressionFactory as well as a decompressor for each block
   3. Add a VERSION number.

Tests

Please see

• org.apache.paimon.sst.IndexTest for index test
• org.apache.paimon.sst.SstFileTest for file test
• org.apache.paimon.lookup.sort.SortLookupStoreFactoryTest for lookup store test

API and Format

This pr do not change any public api

Documentation

todo

[JingsongLi]

Do not change this, MAGIC_NUMBER is the version.

[steFaiz]

Ok, Thanks! I will modify this soon. I was thinking about version is easier to understand if we want to evolve the file format. For example, if (version == 1) is clearer than if (magic_number == 187762739)

[JingsongLi]

SST is already a file format, If there is an incompatibility, we should modify magic_number

[steFaiz]

Converting to draft for compatibility issues. We could move forward this work in the future!