salASR: Performing Legilimency using SALience on ASR models

This repository contains code for the project on extending common explainability approaches used in Deep Learning literature to the task of Automatic Speech Recognition. This project was done for the course CS 6966 Local Explanations for Deep Learning Models taught by Professor Ana Marasovic.

Contributors

• Jacob Johnson
• Gurunath Parasaram
• Rishanth Rajendhran

Dataset

We use the librispeech_asr dataset for this project.

Data Format

{
    'chapter_id': int,
    'file': str,
    'audio': {
        'path': str,
        'array': array(float32),
        'sampling_rate': int
    },
    'id': str,
    'speaker_id': int,
    'text': str
}
        

Getting started

• Clone the github repo:
  

  git clone https://github.com/jacobkj314/salASR
          

• Install conda if you don't have it already

  wget https://repo.anaconda.com/miniconda/Miniconda3-py37_4.12.0-Linux-x86_64.sh
  bash Miniconda3-py37_4.12.0-Linux-x86_64.sh
          

• Once you have installed conda, use it to create and set up the conda environment

  conda create -n salASR python=3.8
  conda activate salASR
  pip install -r requirements.txt
          

• If you wish to finetune whisper, create and set up another conda environment

  conda create -n finetuneWhisperEnv python=3.9
  conda activate finetuneWhisperEnv
  pip install -r requirementsFinetuneWhisper.txt