Official PyTorch implementation of singable similarity for cross-lingual lyric retrieval. This repository contains inference code and pretrained models for finding lyric lines that can be sung over the same melody, across languages. (Training code and experiments soon!)
Keywords: singable similarity, lyric translation, cover song identification, lyric plagiarism, cross-lingual retrieval
Lyric similarity is commonly modeled through various NLP models, implicitly treating lyrics as ordinary text. However, lyrics are written to be sung, and their similarity is fundamentally constrained by singability: whether different sentences can be performed over the same melody. We reconceptualize lyric similarity as singable similarity and propose a framework that learns lyric representations grounded in this principle. Our approach leverages lyric translation pairs that are optimized to fit the same melody, providing natural supervision for learning sentence-level similarity under singability constraints. We introduce a singability encoder model that jointly captures semantic content and syllable structure. Evaluated on line-level translated lyric retrieval and lyric-based cover song identification (CSI), our singability-aware representations consistently perform well on lyric tasks. These results highlight singability as a crucial dimension for lyric-based music information retrieval.
This repository can be installed simply by cloning the GitHub repository and installing the required dependencies.
git clone https://github.com/Mippia/singable-lyric-similarity.git
cd singable-lyric-similarityTo set up their environment, please run:
pip install -r requirements.txt
sudo apt-get install espeak-ng # required for IPA phonemizationTo get started, download the pre-trained checkpoints:
bash scripts/download_checkpoints.shThis fetches checkpoints/rhyme_encoder.pt (6 MB) from the repository's release. The
meaning component is the pretrained multilingual-e5-large-instruct backbone, which
Hugging Face downloads on first run — nothing is trained on top of it.
Retrieve the lyric line that can be sung over the same melody as the query. singability
scores meaning, rhyme and syllable count as one value; semantic scores meaning alone.
python inference.py --query "낡은 일기장이 내 손에 잡혀" --gallery examples/en_lines.txt --topk 5Query: "낡은 일기장이 내 손에 잡혀" (11 syllables)
Gallery: 368 lines · singability
rank score #syl lyric
1 3.880 11 Bursting with dreams that I once had in my heart <- ground truth
2 3.656 10 All my stories hidden deep inside me
3 3.491 7 Flipping through my old diary
4 2.845 11 We used to walk together down this winding road
5 2.760 11 For your happiness, I have lost all of me
The ground truth is the official English lyric written for this melodic slot, and it has the same 11 syllables as the query. Scoring meaning alone puts it third and promotes a line that says what the query says — an old diary in hand — in seven syllables that will not fit:
python inference.py --query "낡은 일기장이 내 손에 잡혀" --gallery examples/en_lines.txt --topk 5 --score semantic rank score #syl lyric
1 4.490 7 Flipping through my old diary
2 2.717 10 All my stories hidden deep inside me
3 2.525 11 Bursting with dreams that I once had in my heart <- ground truth
4 2.032 6 Empty-handed, now I'm lost
5 1.902 12 You don't realize deep inside that I'm broken
--gallery takes a .txt file with one lyric line per row, or a directory of such files.
Private research code. Not for redistribution.