Add suppport for Thai#117

Open

jadechip wants to merge 30 commits intomyshell-ai:mainfrom

jadechip:add-thai

Commits on Apr 29, 2024

Add suppport for Thai

Jim Eric Skogman
authored and
Jim Eric Skogman
committed
Add Thai Wiktionary Corpus

Jim Eric Skogman
authored and
Jim Eric Skogman
committed
Fix issues with g2p function

Jim Eric Skogman
authored and
Jim Eric Skogman
committed
Add better test cases

Jim Eric Skogman
authored and
Jim Eric Skogman
committed
Remove debug line

Jim Eric Skogman
authored and
Jim Eric Skogman
committed
More fixes

Jim Eric Skogman
authored and
Jim Eric Skogman
committed
Fix normalize and dictionary

Jim Eric Skogman
authored and
Jim Eric Skogman
committed
Update expected format of test_word_tokenize test case

Jim Eric Skogman
authored and
Jim Eric Skogman
committed

Commits on Apr 30, 2024

Fix format of test_thai_text_to_phonemes test case

Jim Eric Skogman
authored and
Jim Eric Skogman
committed
Add test to compare with Korean

Jim Eric Skogman
authored and
Jim Eric Skogman
committed
Clean up import statements

Jim Eric Skogman
authored and
Jim Eric Skogman
committed

Commits on May 8, 2024

Add missing code for training Thai language and Thai Grapheme to Phoneme corpus file

Jim Eric Skogman
authored and
Jim Eric Skogman
committed

Commits on May 9, 2024

Initialize the TextEncoder with fixed size

Jim Eric Skogman
authored and
Jim Eric Skogman
committed
Add get_resized_embeddings to SynthesizerTrn class

Jim Eric Skogman
authored and
Jim Eric Skogman
committed
Use self.enc_p.emb

Jim Eric Skogman
authored and
Jim Eric Skogman
committed
Remove th_symbols from sorted list

Jim Eric Skogman
authored and
Jim Eric Skogman
committed

Commits on May 10, 2024

Fix unsupported characters in g2p function

Jim Eric Skogman
authored and
Jim Eric Skogman
committed

Commits on May 16, 2024

Fix tones list from g2p function being initliazed to zeroes and adjust test case

Jim Eric Skogman
authored and
Jim Eric Skogman
committed
Assign multiple phones based on the number of phones

Jim Eric Skogman
authored and
Jim Eric Skogman
committed
Fix regression

Jim Eric Skogman
authored and
Jim Eric Skogman
committed

Commits on May 18, 2024

Fix tones mapping

Jim Eric Skogman
authored and
Jim Eric Skogman
committed
Add new test case

Jim Eric Skogman
authored and
Jim Eric Skogman
committed

Commits on May 19, 2024

Change format of word2ph to inlclude count

Jim Eric Skogman
authored and
Jim Eric Skogman
committed

Commits on May 20, 2024

Keep special characters in word2ph to be consistent with other languages

Jim Eric Skogman
authored and
Jim Eric Skogman
committed
Update test case for Thai bert to match format without special cases

Jim Eric Skogman
authored and
Jim Eric Skogman
committed

Commits on May 28, 2024

Fix dictionary lookup and use bert tokenizer in g2p

Jim Eric Skogman
authored and
Jim Eric Skogman
committed
Fix tone issues and add warning for mismatch due to _ underscore character

Jim Eric Skogman
authored and
Jim Eric Skogman
committed
Squash 2 nasty bugs: 1.) Assign tone value to each phoneme excluding the tone marker only if there is one and Keep only non-empty groups for cases with a trailing dot

Jim Eric Skogman
authored and
Jim Eric Skogman
committed

Commits on Jun 5, 2024

Add gradient clipping, process underscores to align with bert features, add additional symbols from error during training and update tones

Jim Eric Skogman
authored and
Jim Eric Skogman
committed

Commits on Aug 7, 2024

Refactor Thai g2p lookup and add new tokenizer

Jim Eric Skogman
authored and
Jim Eric Skogman
committed