fonn_demo.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "# Folk $n$-gram aNalysis (FONN)\n",
    "\n",
    "*Fonn* (pronounced \"fun\") is an Irish (*Gaeilge*) word for \"tune\".\n",
    "\n",
    "In this strand of research we have created three Polifonia components:\n",
    "\n",
    "1. Folk $n$-gram aNalysis (FONN)\n",
    "2. Ceol Rince na hÉireann (CRE) MIDI corpus\n",
    "3. Root note detection\n",
    "\n",
    "This Demo notebook will demonstrate how we can process a corpus such as the CRE corpus using FONN.\n",
    "\n",
    "### Prerequisites\n",
    "\n",
    "* In `<basepath>/MIDI` we should have a corpus of folk tunes in MIDI format. By default `basepath` is `./corpus/`. If the corpus is elsewhere, change `basepath` below. We will be writing outputs to subdirectories of `basepath`. The corpus should include a `roots.csv` file containing a root note (integer between 0 and 11) for each MIDI file.\n",
    "\n",
    "* Install the following libraries:\n",
    "\n",
    "    `pip install fastDamerauLevenshtein music21 numpy pandas tqdm`\n",
    "    \n",
    "    or just:\n",
    "    \n",
    "    `pip install -r requirements.txt`\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "pd.options.mode.chained_assignment = None\n",
    "\n",
    "basepath = \"corpus\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### MIDI-to-feature sequence conversion\n",
    "\n",
    "By running `setup_corpus.py` script we can read monophonic MIDI files and convert them to feature sequence representation.\n",
    "\n",
    "NOTE: Feature sequence representation represents each piece of music in the corpus as a sequence of note events. Each note event has _primary features_, such as pitch, duration, and onset, which can be extracted directly from the MIDI file. From these features, further _secondary features_ such as interval, key-invariant pitch and pitch class can be calculated.\n",
    "\n",
    "Running the script will create subfolders under `<basepath>/feat_seq_corpus` and populate with `.csv` feature sequence data files:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\r\n",
      "Setting up lookup table for root assignment:\r\n",
      "  note names  midi num  root num\r\n",
      "0          C        60         0\r\n",
      "1   C# or D-        61         1\r\n",
      "2          D        62         2\r\n",
      "3   D# or E-        63         3\r\n",
      "4          E        64         4 \r\n",
      "\r\n",
      "\r\n",
      "\r\n",
      "Setting up Music21 root detection lookup table:\r\n",
      "  note name  pitch class\r\n",
      "0         C          0.0\r\n",
      "1        C#          1.0\r\n",
      "2        D-          1.0\r\n",
      "3         D          2.0\r\n",
      "4        D#          3.0 \r\n",
      "\r\n",
      "\r\n",
      "Reading corpus MIDI files to Music21 streams: 100%|█| 1225/1225 [00:20<00:00, 60\r\n",
      "Calculating feature sequences from music21 scores: 100%|█| 1224/1224 [00:05<00:0\r\n",
      "Calculating pitch class sequences: 100%|██| 1224/1224 [00:00<00:00, 1232.46it/s]\r\n",
      "Calculating interval sequences: 100%|█████| 1224/1224 [00:01<00:00, 1131.71it/s]\r\n",
      "\r\n",
      "Reading roots data from: ./corpus/roots.csv\r\n",
      "\r\n",
      "                             root\r\n",
      "title                            \r\n",
      "Tureengarbh Jig, The            2\r\n",
      "Young And Stylish               9\r\n",
      "Fun at the Fair                 2\r\n",
      "Buckley the Fiddler  (reel)     7\r\n",
      "Tommy Coen's Reel               7\r\n",
      "<class 'pandas.core.frame.DataFrame'>\r\n",
      "Index: 1224 entries, Tureengarbh Jig, The to Willy Reidy's #2  (polka)\r\n",
      "Data columns (total 1 columns):\r\n",
      " #   Column  Non-Null Count  Dtype\r\n",
      "---  ------  --------------  -----\r\n",
      " 0   root    1224 non-null   int64\r\n",
      "dtypes: int64(1)\r\n",
      "memory usage: 19.1+ KB\r\n",
      "None\r\n",
      "\r\n",
      "Converting root values from pitch classes to MIDI note numbers:\r\n",
      "\r\n",
      "                             root  midi_root\r\n",
      "title                                       \r\n",
      "Tureengarbh Jig, The            2         62\r\n",
      "Young And Stylish               9         69\r\n",
      "Fun at the Fair                 2         62\r\n",
      "Buckley the Fiddler  (reel)     7         67\r\n",
      "Tommy Coen's Reel               7         67\r\n",
      "<class 'pandas.core.frame.DataFrame'>\r\n",
      "Index: 1224 entries, Tureengarbh Jig, The to Willy Reidy's #2  (polka)\r\n",
      "Data columns (total 2 columns):\r\n",
      " #   Column     Non-Null Count  Dtype\r\n",
      "---  ------     --------------  -----\r\n",
      " 0   root       1224 non-null   int64\r\n",
      " 1   midi_root  1224 non-null   int64\r\n",
      "dtypes: int64(2)\r\n",
      "memory usage: 28.7+ KB\r\n",
      "None\r\n",
      "\r\n",
      "\r\n",
      "Mapping root values to all tunes in corpus from Corpus.roots_lookup: 100%|█| 122\r\n",
      "Calculating relative pitch sequences: 100%|█| 1224/1224 [00:01<00:00, 828.11it/s\r\n",
      "Calculating relative pitch class sequences: 100%|█| 1224/1224 [00:00<00:00, 1817\r\n",
      "Extracting accent-level feature sequence data: 100%|█| 1224/1224 [00:00<00:00, 2\r\n",
      "Calculating Parsons code sequences: 100%|██| 1224/1224 [00:03<00:00, 399.82it/s]\r\n",
      "Calculating cumulative Parsons code sequences: 100%|█| 1224/1224 [00:01<00:00, 1\r\n",
      "Calculating duration-weighted feature sequences: 100%|█| 1224/1224 [00:03<00:00,\r\n",
      "Calculating accent-level duration-weighted feature sequences: 100%|█| 1224/1224 \r\n",
      "Saving feature sequence data to csv: 100%|█| 1224/1224 [00:04<00:00, 275.64it/s]\r\n"
     ]
    }
   ],
   "source": [
    "!python 'setup_corpus.py'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "Sample output for `Lord McDonald's (reel).csv`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "   midi_note  onset  duration  velocity  pitch_class  interval  root  \\\n0         67      0         1       105            7         0     7   \n1         62      1         1       105            2        -5     7   \n2         67      2         1        80            7         5     7   \n3         71      3         1        80           11         4     7   \n4         67      4         1        80            7        -4     7   \n\n   relative_pitch  relative_pitch_class  parsons_code  parsons_cumsum  \n0               0                     0             0               0  \n1              -5                     7            -1              -1  \n2               0                     0             1               0  \n3               4                     4             1               1  \n4               0                     0            -1               0  ",
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>midi_note</th>\n      <th>onset</th>\n      <th>duration</th>\n      <th>velocity</th>\n      <th>pitch_class</th>\n      <th>interval</th>\n      <th>root</th>\n      <th>relative_pitch</th>\n      <th>relative_pitch_class</th>\n      <th>parsons_code</th>\n      <th>parsons_cumsum</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>67</td>\n      <td>0</td>\n      <td>1</td>\n      <td>105</td>\n      <td>7</td>\n      <td>0</td>\n      <td>7</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>62</td>\n      <td>1</td>\n      <td>1</td>\n      <td>105</td>\n      <td>2</td>\n      <td>-5</td>\n      <td>7</td>\n      <td>-5</td>\n      <td>7</td>\n      <td>-1</td>\n      <td>-1</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>67</td>\n      <td>2</td>\n      <td>1</td>\n      <td>80</td>\n      <td>7</td>\n      <td>5</td>\n      <td>7</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>71</td>\n      <td>3</td>\n      <td>1</td>\n      <td>80</td>\n      <td>11</td>\n      <td>4</td>\n      <td>7</td>\n      <td>4</td>\n      <td>4</td>\n      <td>1</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>67</td>\n      <td>4</td>\n      <td>1</td>\n      <td>80</td>\n      <td>7</td>\n      <td>-4</td>\n      <td>7</td>\n      <td>0</td>\n      <td>0</td>\n      <td>-1</td>\n      <td>0</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sample = pd.read_csv(basepath + \"/feat_seq_corpus/feat_seq/Lord McDonald's (reel).csv\", index_col=0)\n",
    "sample.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "Here, we see the primary feature sequences generated by Music21 (i.e.: sequences derived directly from the MIDI file).\n",
    "\n",
    "- *midi_note* -- MIDI note number, chromatic integer scale\n",
    "- *onset* -- note onset, eighth notes\n",
    "- *duration* -- note duration, eighth notes\n",
    "- *velocity* -- MIDI velocity\n",
    "\n",
    "And the secondary feature sequences (derived from the primary sequences):\n",
    "\n",
    "- *pitch_class* -- key-invariant chromatic pitch class\n",
    "- *interval* -- chromatic interval\n",
    "- *root* -- (scalar) chromatic pitch class representing root / tonal centre of tune.\n",
    "- *relative_pitch* -- key-invariant chromatic pitch, relative to root\n",
    "- *relative_pitch_class* -- key-invariant chromatic pitch class, relative to root\n",
    "- *parsons_code* -- simple melodic contour: up = 1 ; down = -1; repeat = 0\n",
    "- *parsons_cumsum* -- cumulative Parsons code\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### Pattern extraction with $n$-grams\n",
    "\n",
    "Next, using $n$-grams, we extract all *relative_pitch_class* patterns of 3-7 notes in length which occur at least once in the corpus, and count their occurrences in each tune.\n",
    "\n",
    "Results are saved to `<basepath>/pattern_corpus` subdirectory as two sparse pandas Dataframes in .pkl format: one containing pattern frequency counts, the other pattern TF-IDF values.\n",
    "\n",
    "Default pattern extraction parameters such as target feature sequence, length of pattern(s), and level (accent- or note-) can be accessed and edited via pattern_extraction.main() in `<basepath>/pattern_extraction.py` file.\n",
    "\n",
    "NOTE: The work below targets accent-level feature sequence data, which is obtained by filtering the note-level feature sequences\n",
    "and retaining only notes which occur on rhythmically-accented beats."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\r\n",
      "Initial n-gram corpus data:\r\n",
      "       ngram  freq  doc_freq      idf\r\n",
      "0  (0, 7, 0)   930       319  5.46672\r\n",
      "1  (0, 0, 0)  1104       318  5.46985\r\n",
      "2  (0, 0, 7)   775       295  5.54462\r\n",
      "3  (2, 0, 0)   736       285  5.57897\r\n",
      "4  (7, 0, 7)   742       281  5.59306\r\n",
      "<class 'pandas.core.frame.DataFrame'>\r\n",
      "RangeIndex: 75183 entries, 0 to 75182\r\n",
      "Data columns (total 4 columns):\r\n",
      " #   Column    Non-Null Count  Dtype  \r\n",
      "---  ------    --------------  -----  \r\n",
      " 0   ngram     75183 non-null  object \r\n",
      " 1   freq      75183 non-null  int64  \r\n",
      " 2   doc_freq  75183 non-null  int64  \r\n",
      " 3   idf       75183 non-null  float64\r\n",
      "dtypes: float64(1), int64(2), object(1)\r\n",
      "memory usage: 2.3+ MB\r\n",
      "None\r\n",
      "\b\b\r\n",
      "Extracting n-gram patterns...: 100%|████████| 1224/1224 [00:23<00:00, 51.56it/s]\r\n",
      "\b\b\r\n",
      "Populated n-gram corpus dataframe:\r\n",
      "       ngram  freq  ...  Sheahan's Fancy  Sligo Jig, The\r\n",
      "0  (0, 7, 0)   930  ...                2               0\r\n",
      "1  (0, 0, 0)  1104  ...                0               0\r\n",
      "2  (0, 0, 7)   775  ...                0               0\r\n",
      "3  (2, 0, 0)   736  ...                0               2\r\n",
      "4  (7, 0, 7)   742  ...                4               4\r\n",
      "\r\n",
      "[5 rows x 1228 columns]\r\n",
      "<class 'pandas.core.frame.DataFrame'>\r\n",
      "RangeIndex: 75183 entries, 0 to 75182\r\n",
      "Columns: 1228 entries, ngram to Sligo Jig, The\r\n",
      "dtypes: Sparse[int64, 0](1224), float64(1), int64(2), object(1)\r\n",
      "memory usage: 4.2+ MB\r\n",
      "None\r\n",
      "\b\b\r\n",
      "Calculating TF-IDF values...: 100%|█████████| 1224/1224 [00:17<00:00, 71.35it/s]\r\n",
      "Populating TF-IDF corpus...: 100%|██████████| 1224/1224 [00:16<00:00, 75.85it/s]\r\n",
      "\b\b\r\n",
      "Populated TF-IDF corpus dataframe:\r\n",
      "                        ngram  freq  ...  Sheahan's Fancy  Sligo Jig, The\r\n",
      "37591      (7, 9, 5, 2, 7, 6)     2  ...                0               0\r\n",
      "50110     (2, 0, 10, 8, 0, 0)     1  ...                0               0\r\n",
      "50126  (2, 0, 11, 0, 2, 4, 0)     1  ...                0               0\r\n",
      "50125     (2, 0, 11, 0, 2, 4)     1  ...                0               0\r\n",
      "50124        (2, 0, 11, 0, 2)     1  ...                0               0\r\n",
      "\r\n",
      "[5 rows x 1228 columns]\r\n",
      "<class 'pandas.core.frame.DataFrame'>\r\n",
      "Int64Index: 75183 entries, 37591 to 0\r\n",
      "Columns: 1228 entries, ngram to Sligo Jig, The\r\n",
      "dtypes: Sparse[int64, 0](1224), float64(1), int64(2), object(1)\r\n",
      "memory usage: 4.8+ MB\r\n",
      "None\r\n"
     ]
    }
   ],
   "source": [
    "!python 'pattern_extraction.py'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "Now we have TF-IDF and frequency values for all $n$-gram pattern instances in the corpus.\n",
    "This data is stored in sparse pandas Dataframes, which are written to .pkl respectively at `<basepath>/pattern_corpus/freq.pkl` and `<basepath>/pattern_corpus/tfidf.pkl`.\n",
    "\n",
    "See output of cell above for summary info and head for both tables.\n",
    "\n",
    "The above table contains all pitch class patterns of 3-7 items in length which occur at least one time in the corpus.\n",
    "<br><br>These patterns are held in the *ngrams* column.\n",
    "\n",
    "To the right of *ngrams*, are simple corpus-level statistics (frequency, document frequency, IDF), followed by a column for every tune in the corpus.\n",
    "In `<basepath>/pattern_corpus/freq.pkl` these columns hold frequency values for each pattern;\n",
    "in `<basepath>/pattern_corpus/tfidf.pkl` they hold TF-IDF values."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "Finally, we will run the work-in-progress script for calculating similarity between tunes (based on similarity between frequent local $n$-gram patterns). This uses the Damerau-Levenshtein local alignment algorithm."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Locating candidate tune in pattern corpus...: 100%|██████████| 1226/1226 [00:00<00:00, 1064861.61it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "'idx' mode selected -- extracting pattern(s) as ranked by TF-IDF according to their indices...\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Calculating pattern similarity...: 100%|██████████| 75183/75183 [00:00<00:00, 424588.17it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Frequent n-gram pattern(s) extracted from Lord McDonald's (reel):\n",
      "(2, 7, 2, 7, 4, 7)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "34 Similar patterns detected:\n",
      "                   ngram  (2, 7, 2, 7, 4, 7)\n",
      "0     (2, 4, 2, 7, 4, 7)                 1.0\n",
      "1  (0, 2, 7, 2, 7, 4, 7)                 1.0\n",
      "2     (7, 7, 2, 7, 4, 7)                 1.0\n",
      "3    (2, 11, 2, 7, 4, 7)                 1.0\n",
      "4     (2, 7, 2, 7, 9, 7)                 1.0\n",
      "\n",
      "Searching corpus for similar tunes...\n",
      "\n",
      "Similarity results for Lord McDonald's (reel):\n",
      "                             title  count\n",
      "0           Lord McDonald's (reel)      8\n",
      "1            Tim Mulloney's (reel)      5\n",
      "2  Maid Of Mount Kisco (reel), The      3\n",
      "3                    Ballykeal Jig      3\n",
      "4        Biddy the Darling (slide)      2\n"
     ]
    }
   ],
   "source": [
    "from similarity_search import PatternSimilarity\n",
    "\n",
    "# read TF-IDF pattern corpus:\n",
    "basepath = \"./corpus\"\n",
    "f_in = basepath + \"/pattern_corpus/tfidf.pkl\"\n",
    "pattern_search = PatternSimilarity(f_in)\n",
    "# set up out path for results:\n",
    "res_path = basepath + \"/results\"\n",
    "pattern_search.results_path = res_path\n",
    "# set up search candidate tune and extract search term pattern:\n",
    "# First arg: 'title' parameter: tune title per original MIDI file.\n",
    "# Second arg: 'n' parameter: number of items in search term pattern(s) to be extracted.\n",
    "# In this notebook 'n' can be any int value from 3-7.\n",
    "# third arg: 'mode' parameter: can be 'max' (extracts patterns for mx TF-IDF value in candidate tune)\n",
    "# or 'idx': extract pattern(s) at specific indices as ranked by TF-IDF.\n",
    "# Note: IF using 'idx' mode, an additional 'indices' arg must be passed, pointing to a list of indices per example below\n",
    "pattern_search.extract_candidate_patterns(\"Lord McDonald's (reel)\", n=6, mode='idx', indices=[0])\n",
    "# run pattern similarity search:\n",
    "# 'edit_dist_threshold' arg sets the number of differences above which a pattern is considered dissimilar.\n",
    "pattern_search.find_similar_patterns(edit_dist_threshold=1)\n",
    "# run local pattern-based tune similarity search:\n",
    "pattern_search.find_similar_tunes()\n",
    "# display and save results table:\n",
    "pattern_search.compile_results_table()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "The above code cell performs the following operations:\n",
    "- Read the $n$-gram pattern corpus ranked by TF-IDF at `<basepath>/pattern_corpus/tfidf.pkl`.\n",
    "- Extract the first-indexed 6-gram pattern from the tune *Lord McDonald's (reel)*, as ranked by tf-idf.\n",
    "- Find all patterns in the corpus within a Damerau-Levenshtein edit distance of 1 from the search term pattern.\n",
    "- Filter the pattern corpus, retaining only tunes in which similar local patterns occur.\n",
    "- Count the number of similar patterns per retained tune and print results table.\n",
    "- Save csv table of pattern counts per tune to `<basepath>/pattern_corpus/results` directory.\n",
    "\n",
    "As can be seen above, the top result is the candidate tune itself, which contains 8 similar patterns to our search term.\n",
    "Next is the tune *Tim Mulloney's (Reel)*, which contains 5 similar patterns, and so on down the table.\n",
    "\n",
    "Change arg values in calls above, including target tune title, n value, edit distance threshold, to explore the similarity search tool.\n",
    "Tune titles must be formatted per the MIDI tune filenames in `<basepath>/MIDI/`.\n",
    "For  information on the args please see comments in code cell above.\n",
    "\n",
    "This approach to measuring similarity gives musically plausible results in informal testing but has not yet been\n",
    "formally evaluated. The methodology is currently undergoing testing on the corpus of 40,000+ Irish tunes held on\n",
    " [thesession.org](https://thesession.org). We are also currently compiling a test subset of ground-truth anotated tunes from *The Session* to allow quantitative testing and tuning of the similarity search tool."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}