Prog6-NaiveBayesianDoc.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Program 6 - Naive Bayesian (Doc)\n",
    "\n",
    "We use Multinomial Naive Bayes classifier of the `scikit-learn` library."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.feature_extraction.text import CountVectorizer\n",
    "from sklearn.naive_bayes import MultinomialNB\n",
    "from sklearn import metrics"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Load the data set from local file with labels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total instances in the data set:\n",
      " 18\n"
     ]
    }
   ],
   "source": [
    "msg = pd.read_csv('prog6_dataset.csv', names = ['message', 'label'])\n",
    "print(\"Total instances in the data set:\\n\", msg.shape[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`x` contains messages (feature)\n",
    "\n",
    "`y` contains label number (target)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(0                      I love this sandwich\n",
       " 1                  This is an amazing place\n",
       " 2        I feel very good about these beers\n",
       " 3                      This is my best work\n",
       " 4                      What an awesome view\n",
       " 5             I do not like this restaurant\n",
       " 6                  I am tired of this stuff\n",
       " 7                    I can't deal with this\n",
       " 8                      He is my sworn enemy\n",
       " 9                       My boss is horrible\n",
       " 10                 This is an awesome place\n",
       " 11    I do not like the taste of this juice\n",
       " 12                          I love to dance\n",
       " 13        I am sick and tired of this place\n",
       " 14                     What a great holiday\n",
       " 15           That is a bad locality to stay\n",
       " 16           We will have good fun tomorrow\n",
       " 17         I went to my enemy's house today\n",
       " Name: message, dtype: object, 0     1\n",
       " 1     1\n",
       " 2     1\n",
       " 3     1\n",
       " 4     1\n",
       " 5     0\n",
       " 6     0\n",
       " 7     0\n",
       " 8     0\n",
       " 9     0\n",
       " 10    1\n",
       " 11    0\n",
       " 12    1\n",
       " 13    0\n",
       " 14    1\n",
       " 15    0\n",
       " 16    1\n",
       " 17    0\n",
       " Name: labelnum, dtype: int64)"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "msg['labelnum'] = msg.label.map({\n",
    "    'pos': 1,\n",
    "    'neg': 0\n",
    "})\n",
    "x = msg.message\n",
    "y = msg.labelnum\n",
    "x, y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First 5 msgs with labels are printed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "I love this sandwich , pos\n",
      "This is an amazing place , pos\n",
      "I feel very good about these beers , pos\n",
      "This is my best work , pos\n",
      "What an awesome view , pos\n"
     ]
    }
   ],
   "source": [
    "x5, y5 = x[0:5], msg.label[0:5]\n",
    "for x1, y1 in zip(x5, y5):\n",
    "    print(x1, ',', y1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Split the data set into training and testing data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total training instances:  13\n",
      "Total testing instances:  5\n"
     ]
    }
   ],
   "source": [
    "xtrain, xtest, ytrain, ytest = train_test_split(x, y)\n",
    "print('Total training instances: ', xtrain.shape[0])\n",
    "print('Total testing instances: ', xtest.shape[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`CountVectorizer` is used for feature extraction\n",
    "\n",
    "The output of the count vectorizer is a sparse matrix"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total features extracted using CountVectorizer:  43\n",
      "Features for first 5 training instances are listed below\n",
      "\n",
      "   am  an  and  awesome  bad  best  boss  can  dance  deal  ...   to  today  \\\n",
      "0   1   0    1        0    0     0     0    0      0     0  ...    0      0   \n",
      "1   0   0    0        0    0     0     0    0      0     0  ...    1      1   \n",
      "2   0   0    0        0    1     0     0    0      0     0  ...    1      0   \n",
      "3   1   0    0        0    0     0     0    0      0     0  ...    0      0   \n",
      "4   0   0    0        0    0     0     1    0      0     0  ...    0      0   \n",
      "\n",
      "   tomorrow  view  we  went  what  will  with  work  \n",
      "0         0     0   0     0     0     0     0     0  \n",
      "1         0     0   0     1     0     0     0     0  \n",
      "2         0     0   0     0     0     0     0     0  \n",
      "3         0     0   0     0     0     0     0     0  \n",
      "4         0     0   0     0     0     0     0     0  \n",
      "\n",
      "[5 rows x 43 columns]\n"
     ]
    }
   ],
   "source": [
    "count_vec = CountVectorizer()\n",
    "xtrain_dtm = count_vec.fit_transform(xtrain) # sparse matrix\n",
    "xtest_dtm = count_vec.transform(xtest)\n",
    "print(\"Total features extracted using CountVectorizer: \", xtrain_dtm.shape[1])\n",
    "\n",
    "print(\"Features for first 5 training instances are listed below\\n\")\n",
    "df = pd.DataFrame(xtrain_dtm.toarray(), columns = count_vec.get_feature_names())\n",
    "print(df[:5])\n",
    "# print(xtrain_dtm) # same as above, but sparse matrix representation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training Naive Bayes Classifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [],
   "source": [
    "clf = MultinomialNB().fit(xtrain_dtm, ytrain)\n",
    "predicted = clf.predict(xtest_dtm)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Classification results of testing samples are:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "I do not like the taste of this juice -> neg\n",
      "I feel very good about these beers -> pos\n",
      "This is an amazing place -> neg\n",
      "I do not like this restaurant -> neg\n",
      "This is an awesome place -> pos\n"
     ]
    }
   ],
   "source": [
    "for doc, p in zip(xtest, predicted):\n",
    "    pred = 'pos' if p == 1 else 'neg'\n",
    "    print('%s -> %s' % (doc, pred))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Metrics"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy:  0.8\n",
      "Recall:  0.6666666666666666\n",
      "Precision:  1.0\n"
     ]
    }
   ],
   "source": [
    "print(\"Accuracy: \", metrics.accuracy_score(ytest, predicted))\n",
    "print(\"Recall: \", metrics.recall_score(ytest, predicted))\n",
    "print(\"Precision: \", metrics.precision_score(ytest, predicted))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Confusion Matrix"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[2 0]\n",
      " [1 2]]\n"
     ]
    }
   ],
   "source": [
    "print(metrics.confusion_matrix(ytest, predicted))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}