Cross-Cultural Music Similarity

Leveraging human annotations, signal processing methods, and foundation models to analyze audio signals and explore similarity between music pieces from different cultures.

This repository accompanies the paper "Cross-Cultural Music Similarity: Bridging Human Perception, Signal Processing, and Foundation Models" (Papaioannou, Benetos and Potamianos, TISMIR). It contains: the full human annotation dataset, the track selections used from each of the nine source datasets, and the complete implementation and experimental pipeline needed to reproduce the paper's results. See Citation below.

Due to third-party licensing, the audio itself is not redistributed here; see Datasets for how to obtain each source collection.

Requirements

• Python 3.10 or later
• To set up the environment and install the necessary packages, run:

python -m venv ccmsimvenv
source ccmsimvenv/bin/activate    # On Windows: ccmsimvenv\Scripts\activate
pip install -r requirements.txt

Data

• data/annotations/; the published human annotation dataset (see Annotation dataset below)
• data/combined_similarities.csv, data/combined_similarities_normalized.csv; combined per-pair similarity scores across all methods (signal processing features, foundation models, human ratings), produced by step 5
• data/datasets/; local audio/embeddings/mel-spectrograms once you've run step 0 against your own copies of the source datasets (gitignored)

report/ in this repository contains the figures, tables, and their source CSVs that are shown in the manuscript or supplementary material. Re-running the pipeline will regenerate the complete set locally.

Annotation dataset

data/annotations/ contains the full human similarity-judgment dataset described in the paper: 125 participants, 1130 unique audio pairs, three similarity dimensions (overall, cultural, recommendation-level).

• responses/; one anonymized JSON per participant (<uuid>.json), each with their demographic answers and the 10 pairwise ratings they gave. Free-text fields collected during the survey (e.g. an "other" comment on musical training) have been removed to avoid re-identification risk.
• audio_annotations_mapping.csv; maps each survey_audio_id to its source dataset and audio_file_path, plus the survey_start_second/survey_end_second of the 20-second clip that was rated. This is the "track selection" referenced in the paper, combine it with the source datasets (see below) to reconstruct the exact audio clips used.
• demographics_answers.json; the fixed set of categorical options offered for each demographic question.
• human_similarity.csv, human_similarity_normalized.csv; the responses aggregated per audio pair, produced from responses/ by step 1.

Licensed under CC BY 4.0, see License.

Datasets

The study draws 52 representative 20-second clips per dataset (468 total) from nine musical collections. None of the audio is redistributed here; obtain each dataset from its original source:

Dataset	Tradition	Access
MagnaTagATune	Western	https://mirg.city.ac.uk/codeapps/the-magnatagatune-dataset
FMA-medium	Western	https://github.com/mdeff/fma (open access, MIT License)
CorpusCOFLA	Flamenco	https://zenodo.org/records/1324163 (upon request; see Kroher et al., 2016)
Arab-Andalusian	Arab-Andalusian	https://dunya.compmusic.upf.edu/ (CompMusic corpus, registration required)
Lyra	Greek traditional/folk	https://github.com/pxaris/lyra-dataset (via YouTube IDs)
Turkish-makam	Turkish	https://dunya.compmusic.upf.edu/ (CompMusic corpus, registration required)
Hindustani	North Indian classical	https://dunya.compmusic.upf.edu/ (CompMusic corpus, registration required)
Carnatic	South Indian classical	https://dunya.compmusic.upf.edu/ (CompMusic corpus, registration required)
Jingju	Chinese (Beijing Opera)	https://dunya.compmusic.upf.edu/ (CompMusic corpus, registration required)

Full dataset descriptions, cultural context, and musicological metadata are in the paper's Supplementary Material, Section 1.

Pipeline

0. Data pre-processing (1-off operation)

Only needed if you're starting from your own raw survey export and source audio rather than the published data/annotations/ dataset. Requires the CCMSIM_GLOBAL_DATA_DIR and CCMSIM_SURVEY_RESPONSES_DIR environment variables to point at your local copies (see config.py):

python preprocessing/prepare_data.py

which includes the following:

1. Copy the annotations from the responses keeping the uuid from filename, remove answers.terms_agreement, playersTime, datetime from JSON and replace "/" with "_" in clips; put the files under RESPONSES_DIR.

2. Create human_similarity.csv under ANNOTATIONS_DIR with the columns audio_pair, overall_music_similarity, cultural_similarity, recommendation_level, overall_music_similarity_annotations, cultural_similarity_annotations, recommendation_level_annotations, num_annotators.

3. Create annotations_mapping.csv with the columns survey_audio_id, dataset, audio_file_path, survey_start_second, survey_end_second utilizing SURVEY_METADATA_FILE. Also, create a demographics.csv with all the given answers per category.

4. By parsing the annotations_mapping.csv, copy the audio files from GLOBAL_DATA_DIR to data/datasets keeping the same parent folder structure and convert them to Mono with 16kHz sample rate and audio bitrate 32 kbps. Create also the 20 secs segments utilizing the survey_start_second and survey_end_second columns.

5. Create the mel-spectrograms of the whole pieces and of the 20secs segments for all datasets and place them under the respective directories.

1. Parse and normalize annotations to create the similarity CSV files

python preprocessing/parse_annotations.py

2. Extract Signal Processing MIR features and calculate datasets similarities

python sp_features/run.py

To force recompute all features, run:

python sp_features/run.py --recompute

Compute/analyze only specific features:

python sp_features/run.py --features melody

Compute/analyze only specific features and force recompute:

python sp_features/run.py --features melody --recompute

3. Compute FMs embeddings

python preprocessing/compute_embeddings.py --device "cuda:0"

4. Compute Foundation models similarities

All models:

python scripts/compute_models_similarities.py

Specific models:

python scripts/compute_models_similarities.py --models mert_95 qwen

5. Create combined similarity files

python scripts/create_combined_similarity_files.py

6. Compute Alignment scores between SP-, FM- representations and Human ratings

python scripts/evaluate_alignment_with_humans.py

7. Run feature analysis to map human perception and FMs to SP-features (by learning weights through linear regression)

python scripts/run_feature_analysis.py

8. Analyze human annotations

python scripts/analyze_user_study.py

9. Run Ensemble regressor

python scripts/run_ensemble_regressor.py

10. Run Permutation tests

• demonstrating stable perceptual factors exist despite single-rater annotations

python scripts/run_permutation_test.py

License

Code is licensed under the MIT License. The annotation dataset (data/annotations/) is licensed under CC BY 4.0.

Citation

If you use this code or dataset, please cite:

@article{papaioannou2026ccmsim,
  title   = {Cross-Cultural Music Similarity: Bridging Human Perception, Signal Processing, and Foundation Models},
  author  = {Papaioannou, Charilaos and Benetos, Emmanouil and Potamianos, Alexandros},
  journal = {Transactions of the International Society for Music Information Retrieval},
  volume  = {9},
  number  = {1},
  year    = {2026}
}

See CITATION.cff.