07_GPT2_finetuning.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "view-in-github"
   },
   "source": [
    "<a href=\"https://colab.research.google.com/github/rostro36/Partisan-Responses/blob/master/06b_GPT2_finetuning.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "colab_type": "code",
    "id": "Ssvoo9mA4UQ_",
    "outputId": "663eda68-6ba0-4803-d965-6f146e256617"
   },
   "outputs": [],
   "source": [
    "from google.colab import drive\n",
    "drive.mount('/content/drive')\n",
    "\n",
    "import os\n",
    "os.chdir('/content/drive/My Drive/Partisan-Responses-master')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "PNkqu9suKUa-"
   },
   "source": [
    "Import libraries, make sure you don't reinstall gpt_2_simple if you have already sone so by commenting that line"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 173
    },
    "colab_type": "code",
    "id": "kbWDARPU4-j5",
    "outputId": "f90bc4cb-397f-4acd-c00b-c4c7d0ff8a6a"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "TensorFlow 1.x selected.\n",
      "WARNING:tensorflow:\n",
      "The TensorFlow contrib module will not be included in TensorFlow 2.0.\n",
      "For more information, please see:\n",
      "  * https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md\n",
      "  * https://github.com/tensorflow/addons\n",
      "  * https://github.com/tensorflow/io (for I/O related ops)\n",
      "If you depend on functionality not listed there, please file an issue.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "%tensorflow_version 1.x\n",
    "#!pip install -q gpt-2-simple\n",
    "import gpt_2_simple as gpt2\n",
    "from datetime import datetime\n",
    "from google.colab import files\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "YTAgdx5dKnsx"
   },
   "source": [
    "Load and check the data\n",
    "you can download \"all_speech_sentence_filtered.pkl\" from [here](https://drive.google.com/file/d/1AwSUBntWHP6udrHMpl9CF9hEHmarNyhF/view?usp=sharing)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "colab_type": "code",
    "id": "eLeInesF50Tf",
    "outputId": "bd9e9253-8b7e-4e89-eec5-099436ba3401"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of speeches: 298792\n"
     ]
    }
   ],
   "source": [
    "speeches = pd.read_pickle(\"all_speech_sentence_filtered.pkl\")\n",
    "\n",
    "print(\"Number of speeches: {}\".format(len(speeches)))\n",
    "speeches = speeches.head(100000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "X2SsiJlUKu1c"
   },
   "source": [
    "Make .txt files for republican and democrat data as gpt_2_simple needs the data in this format"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 227
    },
    "colab_type": "code",
    "id": "LX-D2sn56czc",
    "outputId": "7718681d-b8e2-4cc8-e779-6dc24c51fde5"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "46666\n",
      "as all Members are aware, I have been working for some time on various rules changes that would ensure a more efficient process by which the Senate considers appropriations bills. One of our concerns has been reinstating rule XVI with respect to legislation on appropriations bills, I believe that many of the extraneous items that have been added to appropriations bills over the past few Congresses would have been ruled out of order if the Senate still had the ability to raise a point of order against legislation on appropriations bills formerly contained in rule XVI. Other rule abuses occurred during the closing days of the 105th Congress. Consequently, I will shortly introduce five Senate resolutions regarding rules and budget process changes and will ask for their proper referral. Once the resolutions have been referred to the appropriate committee, it is my hope the chairmen of the committees will begin swift committee work on the resolutions so the committees can act on the changes as early as possible in the 106th Congress, I urge Senators to consider these resolutions and hope when the Senate votes on these measures they will receive huge bipartisan votes.\n",
      "\n",
      "during the impeachment trial of President Andrew Johnson, the Senate limited access to the Senate wing of the Capitol, the Senate floor, and the Senate galleries to those with official business and those with tickets to the proceedings. Over the Christmas holidays, staff of the Office of the Secretary of including the Parliamentarian, legislative clerk and staff of the Office of the Sergeant at Arms, have reviewed the historical precedents but also considered what we could do to facilitate the publics access to the proceedings while taking into account contemporary security requirements and the flow of business here in the Chamber. Staff have recommended some restrictions to the access of the Senate wing, floor, and galleries coupled with a ticketing system that will make as many seats in the gallery available to the public and others as is possible for us to do. Accordingly, in a few minutes I will ask unanimous consent be given to a set of policies that reflect the staff recommendations which will confine access to the Senate floor and galleries and to the second and third floors of the Senate wing of the Capitol during the consideration of the articles of impeachment and at all times the Chief Justice is presiding, I thank the distinguished Democratic leader for his efforts and his cooperation in this matter. We have been very careful to make sure we reviewed all the precedents, all the rules, that he has had a chance to check off on these rules, as I have. And I wish to thank all staff who researched the precedent and evaluated current conditions to develop these recommendations. Before seeking unanimous consent, however, I will now yield to the assistant Democratic leader.\n",
      "\n",
      "I want to welcome all the Senators to the 106th Congress. We have had an all too brief and somewhat overwhelming interlude since the last day of the 105th Congress in October. That interval turned out to be dramatic and eventful in more ways than one, and because of events that occurred therein, the Senates agenda for this year will be more important than ever. We will soon be considering charges brought by the House of Representatives against the President of the United States, I cannot think of a more serious subject. Yet, the Senate has its wellestablished procedures to deal with this situation. While it is not exactly routine, neither is it totally unique. We have our responsibilities under the Constitution, and we will meet those responsibilities in an orderly fashion. That is why I have met several times and talked by phone other times with Senator DASCHLE, the Democratic leader, and why the two of us have met with the Chief Justice of the Supreme Court, whose duty it is to preside over a Senate trial involving the President. We have both consulted and are still very actively involved in consulting with fellow Senators, with constitutional scholars, with officers of in terms of the law and the rules of the Senate. Our duty is clear: To demonstrate anew our national commitment to justice and fair play. That is what the public expects from us, regardless of their individual opinions concerning the President. That is why I am confident that is what they will receive. No Senator in this Chamber needs to be reminded that we are here first and foremost to serve the American people. Americans today look to the future with the same hopes that have inspired and sustained this country for more than 200 years. They want a better life for themselves and, for their children. Not just economically, but also in terms of a decent future and a just and caring society, I want to emphasize now that I have not gone to the media and outlined exactly how this impeachment process will go forward because no final conclusion has been made. This is not something that can be reported in an evolutionary way because there are too many things that have to be considered, too many different parties--Senate Republicans. Senate Democrats. House Members, the White Houseand we have had to continue to consider the opinions of all to try to develop a fair way to have an expeditious trial that gets justice based on the rule of law, I think that it is more important that we hear from all parties and come to, hopefully, a conclusion that sets an outline of how we will proceed from beginning to end than it is to always be reporting on the current developments. Never before have I had so much reported about what I was thinking, doing, or saying when I have said so little, I have been accused of being \"holed up\" in my hometown of Pascagoula. MS. Where would you expect me to be during the holidays? How about at home with my family and with my constituents, enjoying that precious season of the year. However, I had no moss growing under my feet, I was talking with my colleagues on both sides of the aisle, listening and thinking and developing and evolving a process that I think will get the job done, I believe we can very well achieve that goal within the next 24 hoursone that neither the House nor the White House will necessarily think is wonderfulgiving all parties a fair chance to make the case and reach a conclusion that is equitable. We will get that done. And we will get it done, hopefully, in a relatively short period of time, without limiting it to a day or 3 days, or 3 weeks for that matter. It could very well take longer than that. But it will be a fair trial. Then we have other very serious responsibilities that we must deal with. It has been said as long as we are dealing with this issue that we cant deal with any other substantive issue in the Senate. Wrong. We have responsibilities that go forward, and we will do our very best to have a dual track. Now, we may not be having debate and votes on the floor of the Senate on bills or on changes in the budget procedures around here, but we will begin to prepare. We will have our committee assignments all completed today. There will be committee hearings this week before the Judiciary Committee, before the Armed Services Committee, before the Education and Labor Committeealthough it has a different name here in I prefer to call it the Education Committee because it has that very important jurisdiction under its responsibilities. We will begin the process and have hearings and meetings. Depending on how this process goes forward, and realizing that we have to understand the Supreme Court has a schedule that it has to comply with, which might give us some time to do some business, we will do our very best to get prepared for the regular legislative process while we are doing our duty with regard to impeachment. But the goal that I hope we will move to immediately after the completion of the impeachment process, whenever that may be, is to develop some constant themes we want to work on during the 106th Congress, I think they can be described In words like these: security, responsibility, opportunity, and freedom. Now, those are not conflicting goals, they complement and support one another. Security, enables responsibility, responsibility gives purpose to freedom, freedom ensures opportunity, and opportunity fosters security. When I talk about security, I think about security for my mother, security for my son and my daughter and my grandson, I think about health security. Social Security, national security, security in our neighborhoods. So that word encompasses an awful lot. Our task is to advance on all four fronts this year: To enhance security, promote responsibility, strengthen freedom, and foster opportunity for all. In doing so, we face a tight schedule. We always do, but it is a manageable one if the Senators will help the leadership do our jobs. There are matters that we can consider promptly before our legislative committees even begin reporting major bills that must compete for a place and time on the Senate schedule. One of the first matters we should take up is a clarification of Senate rules, to restore this institutions position regarding the consideration of authorizing legislation on appropriation bills. It is out of hand. The biggest fight now in all the appropriations bills occurs not on the appropriations but on amendments that are legislating on appropriations bills, I believe we can accomplish that change back to the way it was in a bipartisan fashion, I certainly hope so, I hope we can do the same thing regarding our budget process, although I may be erring on the side of optimism in that regard. This is priority legislation, that is required to restore public confidence in the budget process. Do any of us feel that the process at the end of the last year was a good one? I dont think so. In the end, it is going to require will and determination by Senators and House Members to do their jobs on time and on schedule. There are some changes in the process that will help facilitate that. It will enable us to prevent Government shutdowns. It is ridiculous that there is even that possibility. It will control emergency spending. It has reached the point where we have not one super or extra special emergency bill each year, now we have to have two. And it makes a requirement that we take a long, hard look at how that is paid for and at current budget rules. Important as budget reform is, rebuilding Americas national security is even more pressing. Press reports have indicated that the administration will propose some increases in defense spending. That is good, and the Senate will take a very close look at that in committee and in the full Senate, I worry that those proposals are not sufficient or maybe the way it would be done is not the best way in trying to address the questions of pay and pensions and readiness for our military. But we should give that a very high priority. We have been losing ground in this area. This Congress must stop that erosion of our readiness and the morale of our military if we are going to be able to preserve our own national security and protect peace wherever our interests are in the world. Education is going to be a central issue this year. Democrats say it is important and it will be a high priority. Republicans say it is a high priority. This past Congress passed not one, not two, but five major education bills, and we got very little credit for it. There was everything from some additional funds for IDEA to vocational education, higher education, and other things in between. For starters, we must reauthorize the Elementary and Secondary Education Act. That is important. Since its enactment more than 30 years ago, that legislation has been the channel through which tens of billions of dollars have flown from the taxpayers to Washington and back to the school districts again at the local level. In retrospect, perhaps that has not been the most productive system that we could devise, to put it mildly, I think we need to look at ways to cut out some of those stops along the way, the distance between the taxpayers, parents, and government, and how we Improve our schools. We need to find more ways to get more dollars back to the schools and especially back to the classrooms. We need to strengthen local decisionmaking so the parents and teachersthe people most involved with their childrencan act in the best interest of those youngsters. We should foster quality teaching and promote family choice in education, especially for poor families whose kids are stuck in deadend schools that are dangerous and drug infested and where they are not learning. We should not, presume to dictate to parents and educators what their priorities should be and how they should spend their tax dollars. So, clearly, this is something on which we will spend a lot of time. We must continue to address the question of oppressive taxation. Most people will acknowledge that Americans are paying a heavy burden in taxes now. It affects the way they think and act as a family or how they save or invest. One of the most crushing tax burdens in this country is the payroll tax, it is a high percentage. That is the one in everybodys check at the end of the work week and they say, gee, this FICA tax is the one that is nailing me. Congress needs to look at that. We need more tax relief for working families so they can keep more of their own money. We need to have a tax code that is pro growth, pro investment, and pro jobs, so that we dont just give people a tax break but we give consideration to how the changes or tax reductions would lead to improvement in lives and jobs all across this country. Tax simplification is a continuing need. We need to think about how we can get lower insurance premiums for the taxpayers, whether it is for their automobile insurance or their health insurance. We need to promote regulatory reform and relief across the board, but especially for small businesses. Nothing this 106th Congress might dowhether in education, tax policy, or environmental protectionwould mean as much to the American people as a longterm solution to the problems of Social Security and Medicare. So from the first day of this Congress right up to the last day toward the end of the year 2000, it will be my goal to see if we can find a broad, bipartisan agreement in those two crucial areas. The Congress cant do it alone, though. The President has to provide leadership. It is not enough to just have conferences and talk about options. What is the solution? What are we going to be able to do to resolve the problems on Medicare? Will the Medicare Commission that reports back in March have a report we can act on or not? Or will it decay in partisan disagreement? Can we find a way to act in good faith on Social Security? To show my good faith, I have said that if the President will send us a proposal he would like for us to consider, I will introduce his bill and we will begin hearings the next day in the Senate Finance Committee and see if we can go forward. Or if that is not the way it can be done, I am willing to look at other ways that we can accomplish that goal. It is too important to just set it aside because it is too tough. There are a lot of other issues we will deal with in the regular order. For example, bankruptcy reform, liability reform for charities, charitable choice in Federal programs, to end discrimination against faithbased organizations, prohibition against partialbirth abortions, as well as child custody protections to safeguard family rights, and modernization of financial services, I have spoken with Senator GRAMM and encouraged him, as the new chairman of the Banking Committee, which has jurisdiction, to pick up the legislation and see if he can forge an agreement that we can move forward on so that we will have broader choices and better service for consumers. In due time, we will deal with all of those and a great many other subjects. During the next few weeks, I realize that the news media will be focused on one thing. My remarks here will be little noted or rememberedother than the part on the impeachment proceedings. But the record must begin to be made now that we have other very important priorities that are the peoples priorities back in our respective States. This Senate was designed by the Framers of the Constitution to be the steady element in Government, the place where passions are cooled and judgments come slowly. It serves us well when we take our time and we make sure that the process is fair and the result is equitable, I expect that to happen In the days ahead. No one can predict the outcome of the deliberations on impeachment, but everyone can expect the calm and careful exercise of our duty under the Constitution. That will not conflict with our role as legislators. It will rather confirm that we are more than mere lawmakers. As Members of we are guardians of the rule of law and defenders of the rights of every American. That is our most important role, our most solemn charge, and our most enduring honor, I observe the absence of a quorum.\n",
      "\n",
      "it is a great honor and privilege to extend congratulations to the Reverend Peter (Yee Chung) Chei as he celebrates the granting of his United States citizenship on Friday. October 16th, 1998. Reverend Peter Chei has served our nation admirably during the thirtythree years he has made his home on our shores, and now, as a United States citizen, his continuing commitment to this nation has been sealed with the words of the Oath of Allegiance. The United States has thereby greatly profited. The Reverend Peter Chei has long been held in my highest regard. His dedication to the redemptive mission of Christ has been an inspiration for many people across the state of Missouri, this country, and the world. It was my good fortune to meet Reverend Chei when he was still living in Hong Kong. His father had escaped with the Chei family to Hong Kong after the communists took over the Chinese mainland in 1950. The trials faced by the Chei family as they were uprooted from Peters birthplace parallel the trials faced by many of Americas first adopted citizens escaping religious persecution. Peter Cheis decision to move to the United States in 1965 and his decision to become a United States citizen are made all the more meaningful by this stark comparison. Having arrived in this country. Reverend Chei determined to serve God and his adopted country through a life of evangelism. His long history of ministerial and community service demonstrates his commitment. Reverend Chei has volunteered as Head Start Policy Council President. Head Start Parents Association President. Coordinator for the American Bible Society. Coordinator for the National Day of Prayer. Coordinator .for the International Year of Bible Reading. Coordinator for the 1999 Year of the Bible, and as a member of the Crosswalk Teen Center. The Reverend Peter Chei founded Missouri Head Start Parents Association. Singles and Families Educational Seminar. Christians Together in the City of Nevada. His Hope House, the Christian Artist Series. Hope for America, and Missionary to America. He has taken on all of these extra responsibilities while serving faithfully as a minister of music and as a pastor, I consider it a great blessing to be counted among Reverend Cheis friends and it is my distinct honor to salute this patriot on the occasion of his swearing in as a citizen of our great country.\n",
      "\n",
      "although the late Honorable Harold A. Shaub was not a citizen of my State, I regarded his friendship, and his interest in North Carolina, sufficient to qualify him to be declared an honorary Tar Heel. He was a remarkable gentleman whom I met casually one morning when he was trying to find the office of then Senator Curtis of Nebraska. From that day on, he was a friend for whom I had great admiration. He visited occasionally when he was in Washington, and I enjoyed his company fairly often in the Senate Dining Room. Occasionally. Mrs. Shaub and one or more of his and Mrs. Shaubs children joined us. there was not one iota of pretense in Harold Shaubs personality. Yet he was one of Americas leading business men, perhaps most notably as president and chief executive officer of the Campbell Soup Co, I never asked Harold for a special favor, nor did he of me. There was one occasion, a number of years ago, when North Carolina was one of the States seeking to acquire a Campbell Soup Co, plant, I had studied the data on each of the States competing against mine for the Campbell plant, I was convinced that North Carolina met Campbell Soups needs better than did our competitors. So I called Harold, told him of my interest in the proposed plant, and asked if he would object to my sending to him the details of why I sincerely believed North Carolina should be chosen. His response was that I should send the information as quickly as possible because the first decision deadline was near, I didthat very day. Within a week, he was on the telephone. He said, simply: \"I suspect you would be wise to make arrangements for some news about a new corporate citizen coming to \" I have at hand an obituary about my friend. Harold Shaub, published in Pennsylvania, I ask that it be printed In the RECORD.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# make separate txt files for republicans and democrats\n",
    "with open(\"rep_speeches.txt\", 'w') as f1, open(\"dem_speeches.txt\", 'w') as f2:\n",
    "  for i in range(len(speeches)):\n",
    "    speech = speeches.iloc[i].Questions\n",
    "    party = speeches.iloc[i].Party\n",
    "    if party == 'R':\n",
    "      f1.write(speech + '\\n')\n",
    "    else:\n",
    "      f2.write(speech + '\\n')\n",
    "\n",
    "# check files \n",
    "with open('rep_speeches.txt', 'r') as f:\n",
    "  lines = f.readlines()\n",
    "  print(len(lines))\n",
    "  for line in lines[:5]:\n",
    "    print(line)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "dz_QnKpYCpbo"
   },
   "source": [
    "Finetunine GPT-2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 138
    },
    "colab_type": "code",
    "id": "CntxXErpa9aS",
    "outputId": "2ddff0ed-8191-4178-8c47-f2f05068470b"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Fetching checkpoint: 1.05Mit [00:00, 224Mit/s]                                                      \n",
      "Fetching encoder.json: 1.05Mit [00:00, 73.6Mit/s]                                                   \n",
      "Fetching hparams.json: 1.05Mit [00:00, 230Mit/s]                                                    \n",
      "Fetching model.ckpt.data-00000-of-00001: 498Mit [00:02, 169Mit/s]                                   \n",
      "Fetching model.ckpt.index: 1.05Mit [00:00, 320Mit/s]                                                \n",
      "Fetching model.ckpt.meta: 1.05Mit [00:00, 139Mit/s]                                                 \n",
      "Fetching vocab.bpe: 1.05Mit [00:00, 129Mit/s]                                                       \n"
     ]
    }
   ],
   "source": [
    "# download the GPT-2 model if you have not done so\n",
    "gpt2.download_gpt2(model_name=\"124M\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "colab_type": "code",
    "id": "LXw14pG1b94X",
    "outputId": "12c2f322-2943-4433-f8d3-3249e6cfab3f"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Drive already mounted at /content/drive; to attempt to forcibly remount, call drive.mount(\"/content/drive\", force_remount=True).\n"
     ]
    }
   ],
   "source": [
    "gpt2.mount_gdrive()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "2RW5SZP5LziI"
   },
   "source": [
    "Train for republican data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "SdpWfnzicHlr"
   },
   "outputs": [],
   "source": [
    "file_name = \"rep_speeches.txt\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "colab_type": "code",
    "id": "PjrX_GtqcSAm",
    "outputId": "53682a33-083d-4ccf-c6e7-fd5cb2d5e38a"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/gpt_2_simple/src/sample.py:17: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Use tf.where in 2.0, which has the same broadcast rule as np.where\n",
      "Loading checkpoint models/124M/model.ckpt\n",
      "INFO:tensorflow:Restoring parameters from models/124M/model.ckpt\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "  0%|          | 0/1 [00:00<?, ?it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loading dataset...\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 1/1 [03:12<00:00, 192.47s/it]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "dataset has 31956684 tokens\n",
      "Training...\n",
      "[10 | 52.16] loss=3.33 avg=3.33\n",
      "[20 | 97.84] loss=3.08 avg=3.20\n",
      "[30 | 143.52] loss=3.14 avg=3.18\n",
      "[40 | 189.24] loss=3.21 avg=3.19\n",
      "[50 | 234.84] loss=3.10 avg=3.17\n",
      "[60 | 280.49] loss=3.06 avg=3.15\n",
      "[70 | 326.07] loss=3.19 avg=3.16\n",
      "[80 | 371.69] loss=3.12 avg=3.15\n",
      "[90 | 417.36] loss=2.96 avg=3.13\n",
      "[100 | 463.02] loss=3.17 avg=3.13\n",
      "[110 | 508.64] loss=3.15 avg=3.14\n",
      "[120 | 554.25] loss=3.01 avg=3.13\n",
      "[130 | 599.93] loss=3.02 avg=3.12\n",
      "[140 | 645.54] loss=3.11 avg=3.12\n",
      "[150 | 691.20] loss=2.95 avg=3.10\n",
      "[160 | 736.83] loss=2.99 avg=3.10\n",
      "[170 | 782.49] loss=2.96 avg=3.09\n",
      "[180 | 828.20] loss=3.01 avg=3.08\n",
      "[190 | 873.92] loss=2.97 avg=3.08\n",
      "[200 | 919.63] loss=3.17 avg=3.08\n",
      "======== SAMPLE 1 ========\n",
      ". There are few people who understand the difference between a college education and a job at a company that is not in your field, and there are many employees who have earned their education at a company that provides employment for many other employees. Therefore, I propose to make a bill that will make education free for millions of families across the country. This legislation is a free enterprise amendment. What can be done about it? Simply put, it is a program that will provide millions of dollars of public education funding every year so that those eligible for assistance have the opportunity from day one to work at a company where they are qualified. My bill makes this a reality and it does more for everyone than ever before because it takes a real effort to ensure that every child who is eligible for this program has their education. It is a program that educates them with the opportunities offered by the Federal dollars that they will receive in the National Science Foundation. There are those who think that the next generation will look to the future of science and the future of education to get that educational opportunity. The Science Foundation is one of the most important institutions in America and we spend an incredible amount of money every year to support the creation of a high quality science and technology institution. Every year we provide $3 million to science, technology, and technology institutions in Washington State and throughout the country and we have to ensure they have this opportunity, too. The next generation of science teachers will have the opportunity to get their education in a competitive laboratory environment. Their future education will be built upon scientific research and not just on their personal knowledge and knowledge of science. This bill will do everything that is needed in order to enhance the quality and value of our public science and math education programs. All we are doing is to preserve the integrity of the program while reducing the number of unnecessary regulations. That is what the American people want. This is not going to work. It is not going to work if we are not careful. It is not going to work if we are not careful. This bill does not simply end the bureaucracy of our federal educational system. This is not a piece of legislation to end regulations and regulations. This is an important bill that helps bring every American schoolchild to the full range of their skills to be a successful scientist and to enjoy the success of every child. It is a bill that will help every American achieve the goals of this bill, for every American who has an interest and a desire for a career. They will be made better by going to university or studying in a private school with the same knowledge, skills, and experiences as a citizen. The bill I am introducing today will bring the quality of public education back into our government. It is something that we need to take into consideration. This bill, I call on the President and the Vice President to have this opportunity, and all of us must support it in their current and future administrations, I am sure they will, as part of the efforts to preserve the integrity of the National Science Foundation, make it possible for the next generation of scientists to attend a private university, I think if we dont support this bill now than we will vote against the bill. And I hope they will, as part of the effort to preserve the integrity of the program, join me tonight in the House of Representatives.\n",
      "it is time for the conference committee that I chair to do its work, and it is time we are on the right track, I commend the gentleman from New York and the gentleman from Utah for supporting this bill, I urge them to move this legislation to the floor to receive a quick vote.\n",
      "I would like to speak to the gentleman from Utah the gentleman from Louisiana, and the gentleman from Louisiana about Senator BORIS, because the gentleman from Louisiana made a very special point yesterday and that is his commitment to make a significant commitment to help American youth become more aware of the challenges of technology. With the exception of his commitment to our research, he is just an experienced professional in the field of science, and as a scientist, he has the knowledge and the experience to develop this innovative advancement technology, but also the experience and to provide it to future generations that we will see tomorrow, I urge the gentlewoman from Arizona to join me again this evening in thanking Senator BORIS for his commitment to this bill and for his commitment to make a significant commitment, and I urge my colleagues to join me in thanking this distinguished chairman of this Committee.\n",
      "I am pleased to begin our debate about whether to pass the Energy Security Research Act of 2002, and I thank Senator FRED BROADMAN from Arizona, the chair of this Committee and the ranking member of the Foreign Relations Committee of the Senate for coming out and being there at this debate.\n",
      "I rise today, as I join with colleagues in the House of Representatives and the President to make my first remarks since he took office this month on Tuesday, to thank his staff, his leadership, for his commitment, and the leadership of the Senate for the strong leadership and leadership in the work they do to improve the education and employment opportunities\n",
      "\n",
      "[210 | 983.79] loss=3.11 avg=3.08\n",
      "[220 | 1029.45] loss=3.02 avg=3.08\n",
      "[230 | 1075.16] loss=3.09 avg=3.08\n",
      "[240 | 1120.82] loss=3.15 avg=3.08\n",
      "[250 | 1166.42] loss=2.91 avg=3.08\n",
      "[260 | 1212.07] loss=3.07 avg=3.08\n",
      "[270 | 1257.69] loss=2.98 avg=3.07\n",
      "[280 | 1303.28] loss=3.05 avg=3.07\n",
      "[290 | 1348.79] loss=3.01 avg=3.07\n",
      "[300 | 1394.41] loss=3.01 avg=3.07\n",
      "[310 | 1440.05] loss=3.16 avg=3.07\n",
      "[320 | 1485.68] loss=2.99 avg=3.07\n",
      "[330 | 1531.20] loss=2.96 avg=3.06\n",
      "[340 | 1576.79] loss=3.01 avg=3.06\n",
      "[350 | 1622.29] loss=2.95 avg=3.06\n",
      "[360 | 1667.82] loss=3.18 avg=3.06\n",
      "[370 | 1713.41] loss=2.99 avg=3.06\n",
      "[380 | 1759.00] loss=2.94 avg=3.06\n",
      "[390 | 1804.60] loss=3.18 avg=3.06\n",
      "[400 | 1850.27] loss=2.99 avg=3.06\n",
      "======== SAMPLE 1 ========\n",
      " of the Government of Great Britain and France. It is an important and valuable initiative, and I want to commend the chairman of the Committee on Veterans Affairs for the effort he has undertaken to ensure that our veterans are given access to full benefits. He will continue to ensure that veterans can get timely care and appropriate treatment. One more piece of legislation. One more time. Senators have raised the question of why the United States should withdraw the European Union. The argument that the United States is not at the negotiating table is a myth and a fraud. When we talk about withdrawal, we dont talk about withdrawal without a serious discussion of whether or not to withdraw. That is what it is about. It is an important issue. We need to clarify that we never will withdraw. If the United States is at the table, there must be a clear answer before it is withdrawn. We must ask ourselves, why in the world should we be at the table? Why should we be at the table without an answer? Why should we be waiting? Why should we have a role, that we should be in a position to take a role with regard to this bill? But it is not appropriate for the United States Government to be at the table without something fundamental, and it seems to me the most significant piece of legislation of this bill. Mr. Chairman. The Senate voted last week to authorize the use of funds in January of 1993 to fund the International Rescue Committee. The European Union has not provided adequate money. That money is now under the control of NATO, in this instance, which is controlled by a European Union Member States. We have already talked about the funding of the International Rescue Committee. It is not a problem that we do not need a NATO funding bill because we will have to get the United States to the negotiating table with regard to NATO if we do not get NATO funding. But the Europeans seem to have forgotten this. Even if we get the funding, I do not know whether the Europeans will want to be at the table. We were assured that we had in the final weeks of the debate that we had a vote to authorize the use of military force against Syria and we did not. That is the issue of money. When we look at this in context, the United States has provided more than $20 million in military aid to the international rescue organization since September. The United States has funded more than a million people in this war. We have paid our troops more than a million men and women. We have provided more than 300.000 troops to our military in support of Operation Enduring Freedom. And we have provided 1.8 million women and children with access to the international help system. When we look at a very small percentage of the total, that does not make the United States representative, even though they have gotten more. It does not make the European Union representative. It does not make the U.S. representative of the EEU. So, if we look at the issue of the European Union membership, to say we should not be at the table. And why should we not be at the table without something fundamental, I can assure you, this bill offers a clear answer to that. With this bill, we have taken the position that we ought to be at the table with regard to the Europeans and ask why they do not provide the European Union with the funds necessary to have a fully functioning international rescue organization. There are many who disagree with that. But there is nothing wrong with providing the international rescue resources we need. We will see some of these resources during this debate. We will be asked whether this is an appropriate thing to do or to do well. As we go through the debate, whether we should not be at the table, who is the leader? Who is the leader, who is the leader of this legislation at the end of the day, who is the voice of reason? Who is right and who is wrong with this bill? The answer is, no one. The answer is, we ought to be at the table with respect to these funds. We ought to have an issue of international rescue. This legislation provides that we be able to provide the funds that we need. It does not provide for international rescue. It simply does so. We have to get those funds. And it is only the first step in the process we need to follow. We need to start now, for the sake of progress and for the sake of the cause, to get the money we need out to our troops, get them back to the United States, to the troops coming back from overseas, to our troops going to Europe. We need to get there. And we are going to have to get there right away. We just need to start this process right now. And I am convinced that sooner rather than later we will be able to get them back to the United States. It will take a lot of time and we are going to have a lot of paperwork and a lot of difficulties of going over the money. So I believe this is the first time we have had a debate\n",
      "\n",
      "[410 | 1913.02] loss=2.87 avg=3.05\n",
      "[420 | 1958.67] loss=3.04 avg=3.05\n",
      "[430 | 2004.32] loss=2.97 avg=3.05\n",
      "[440 | 2050.01] loss=2.79 avg=3.04\n",
      "[450 | 2095.63] loss=2.99 avg=3.04\n",
      "[460 | 2141.24] loss=3.13 avg=3.04\n",
      "[470 | 2186.82] loss=2.98 avg=3.04\n",
      "[480 | 2232.46] loss=2.95 avg=3.04\n",
      "[490 | 2278.09] loss=2.96 avg=3.04\n",
      "[500 | 2323.66] loss=2.99 avg=3.04\n",
      "Saving checkpoint/rep/model-500\n",
      "[510 | 2373.63] loss=2.80 avg=3.03\n",
      "[520 | 2419.21] loss=2.92 avg=3.03\n",
      "[530 | 2464.84] loss=3.03 avg=3.03\n",
      "[540 | 2510.39] loss=2.96 avg=3.03\n",
      "[550 | 2555.96] loss=2.96 avg=3.02\n",
      "[560 | 2601.63] loss=3.03 avg=3.02\n",
      "[570 | 2647.24] loss=2.97 avg=3.02\n",
      "[580 | 2692.92] loss=2.84 avg=3.02\n",
      "[590 | 2738.49] loss=2.97 avg=3.02\n",
      "[600 | 2784.10] loss=2.98 avg=3.02\n",
      "======== SAMPLE 1 ========\n",
      ". It would not only be beneficial to our military bases along the U.S.-Mexican border. It would also have the potential to move military trucks and equipment from these lands to the Gulf of Mexico. We need to maintain a balance at the UNSC, I agree with that. My amendment would keep the United States from doing that as long as it continues to operate military vehicles that go through those borders. I know that our troops in the Gulf have been using them for over 30 years and, frankly, many of their trucks and equipment are still in those locations, I also know that they are not doing a very fair job. In these States, we should use the same standards we want to use if we want to make sure this country is operating a responsible security and a fair and orderly military operations. We should not take military equipment or vehicles that are going to cross the border, but we should certainly be operating the same standards of readiness and that our troops are doing as long as they are staying within the confines that were applied to the U.S.-Mexico border. But we keep using standards that are not meeting what is the most current of them. On the other hand, they are certainly being maintained. We should not expect the United States to go without the necessary supplies for our troops, I believe you would be right about that. So, again, I strongly support the amendment, I say it in very limited ways. It is an amendment, I do want to take a few moments to respond to this debate, I am glad to be the guest tonight. What happened is, this is a matter of national security, it is a matter of international security. But we are asking this vote on this day of the year to ask Congress, as well as the President of the United States, to do the right thing on all of this. Because I think that, while the Senate has moved a little closer now to the Senate as well as the House, it is very difficult to tell what is in the national interest of this country. But the same cannot be said about us, I want to see the President again, I want him to know that we will keep our support of Iraq, that we will not be looking at this vote in any other way, I believe we have the votes on the floor. We have the votes on the floor on this issue. We have had hearings. We have the votes on this bill. We have been able to talk about the national security issues I think we can bring ourselves to talk a lot about. It is going to be difficult to do that on the Senate side and in the House. But certainly we are not at the bargaining table. We all come together on this side. Again, I just want to make sure it is clear on the other side the Secretary of State has the votes and the Congress does have all the authority that we have given him, I want to make sure that if that is the case, the Secretary of State has the tools and procedures from the Senate and the Congress so that when we vote on the Senate side we have the full support and the commitment in the House to do the right thing, I look forward to looking at those. It is a debate that every Senator and Congress wants to have, I would ask everybody to keep their heads up. We are going to find out if we can get this right in the Senate and in the House and the President has the votes, I want to speak for those who are here tonight. Let us not get bogged down by it. Let us keep going.\n",
      "Let me just say that the Senate needs the leadership now. The Senate is in a new relationship. It is very dynamic, as I have said before, that it is going to take longer than the House but it is going to take longer than the President and the Democratic leaders have had. You have a debate, I think you have a debate in the House but it is going to take longer, I think we need more in this area. The President and the Vice President have been there, I think they are doing the right thing. Obviously I think we need more of the same. We need to have the same level of accountability. We have had a big debate in this body. You get to be involved, you have a debate, I hope that the American people and the American people can look at it in the same light today. On the other hand, I believe that if I had to make a recommendation to the President, to my colleagues, I dont think the President will go for it. I do not think he has the numbers, I think maybe an executive will do it. But hopefully, we will at least find some agreement. We should be able to take all that we can get ourselves into one package. When you take all that into account, then I am going to put it on the table. But we need a consensus. We want a Senate that doesnt stand and stand, I strongly believe that the Presidents ability to make a difference in the world. We do not want\n",
      "\n",
      "[610 | 2846.70] loss=2.74 avg=3.01\n",
      "[620 | 2892.40] loss=3.02 avg=3.01\n",
      "[630 | 2938.12] loss=2.91 avg=3.01\n",
      "[640 | 2983.79] loss=2.87 avg=3.01\n",
      "[650 | 3029.40] loss=2.85 avg=3.00\n",
      "[660 | 3075.02] loss=3.16 avg=3.01\n",
      "[670 | 3120.70] loss=3.00 avg=3.01\n",
      "[680 | 3166.35] loss=2.97 avg=3.00\n",
      "[690 | 3212.01] loss=3.10 avg=3.01\n",
      "[700 | 3257.60] loss=3.09 avg=3.01\n",
      "[710 | 3303.16] loss=2.97 avg=3.01\n",
      "[720 | 3348.78] loss=2.98 avg=3.01\n",
      "[730 | 3394.48] loss=3.04 avg=3.01\n",
      "[740 | 3440.20] loss=2.94 avg=3.01\n",
      "[750 | 3485.77] loss=3.33 avg=3.01\n",
      "[760 | 3531.43] loss=3.05 avg=3.01\n",
      "[770 | 3577.06] loss=2.85 avg=3.01\n",
      "[780 | 3622.71] loss=3.02 avg=3.01\n",
      "[790 | 3668.40] loss=2.94 avg=3.01\n",
      "[800 | 3714.05] loss=3.05 avg=3.01\n",
      "======== SAMPLE 1 ========\n",
      " that. We will put that in. That is what I would do if I were leader.\n",
      "the Federal Government does not allow them to work through any of the provisions in the bills they have proposed, I am going to vote for either of them, to give my best on both sides of the aisle. One is the amendment to my side for a period of time. But, I say, as many people who are supporters of this bill have said and I was going to say and I hope all the Members of this body know about it, we have seen the Government shutdowns happen in the last month or two. We have had the Federal Government close shut down at the Department of Education in the last period of time. All of the other programs that the Government continues to be open, are shut down for what they are not. The government shutdown, is an important event that has occurred for the American people, an important event for our economy, and a pivotal event that can very easily be reversed and can happen this fiscal year after the year 2001. And that is a critical issue in the Republican party leadership, one that, on the other side have a long list to talk about.\n",
      "it was in that we were on Monday. October 6, 1997, I was in a town hall meeting. That day the Federal Government was up and down. It was a meeting of a constituent called Mr. Thomas. Mrs. Johnson talked to that woman a little bit, talked to her and the other little girl who sat there, looked at the floor one on one, and said. \"I am going to get to work on this. I really am not happy about spending.\" \"Just spend.\" \"I am going to find a new Federal program that will make Federal spending smaller by reducing spending for social services.\" I would have liked to hear those responses, but I wanted to hear from a little girl who said. \"Just spend.\" And what I heard was a little girl, and I know the Department of Education really ought to spend more money for Social Services. She did, too. It worked out. It worked a lot better than I thought it would. All we had to do was say. \"Just spend. Just start.\" We had to start. The kids in our classrooms across the country went, I have no doubt. We have been successful, and now we need to try something else. But I am hopeful that we will work together, I know people want to go into this debate debating this because the President has asked for some time. It might work out, but we want to do it. So we have to work together with Members on both sides of the aisle to try to get help to give people the means to help the Federal Government get even cut.\n",
      "I thank the gentlewoman for yielding this time. We are going to begin the discussion of the bill today at about 1, 2, 3, 4, 5. We will continue to make recommendations on a bill that has been on the Democratic side for some time. We will discuss it today as time goes on. All I want to do is say to Senator DODD and Senator THOMAS and the rest of our colleagues who did some extensive debate, I had some very important discussion with them over the last week or so over the last few days, but I do believe this is going to be the last time on a conference report that has passed as to whether or not we need to do this, I am just happy to say that Senator DODD has asked for more time, I ask unanimous consent that the list of all the conference reports I would like to talk about as they are written is printed in the RECORD. It is on this list.\n",
      "I am just going to start with a very small number of comments. We were talking about the problem of the government shutdown, and I was concerned because some people on the other side said that maybe we ought to take one big bill and go through and do the same thing in the future with some big deal. In other words, they said that they thought it would be fine with Congress and their President to take a big government bill that we want to pass, and we ought not to be involved in anything other than providing for the spending that we actually have. It should be easy to talk about the tax increase, I do not agree with every single one of the tax increases because it is a longterm tax, it is not a tax that they ought to be spending money on, they ought to be spending money on the payroll tax, they ought to be putting money in the Social Security trust fund. Yet they would oppose a tax increase, it was a longterm tax. There is a debate here. And people on the other side would be fine with a tax increase that they do not have, but they would like to do some spending. So those are some of the comments I made during Monday night, so I could write a lengthy comment because that is what most of the comments are. It would be a long time until we get to the\n",
      "\n",
      "[810 | 3776.81] loss=3.08 avg=3.01\n",
      "[820 | 3822.45] loss=2.66 avg=3.00\n",
      "[830 | 3868.14] loss=2.92 avg=3.00\n",
      "[840 | 3913.78] loss=3.09 avg=3.00\n",
      "[850 | 3959.40] loss=3.08 avg=3.01\n",
      "[860 | 4005.06] loss=3.15 avg=3.01\n",
      "[870 | 4050.73] loss=3.09 avg=3.01\n",
      "[880 | 4096.35] loss=2.83 avg=3.01\n",
      "[890 | 4142.00] loss=2.73 avg=3.00\n",
      "[900 | 4187.62] loss=2.88 avg=3.00\n",
      "[910 | 4233.30] loss=2.89 avg=3.00\n",
      "[920 | 4278.95] loss=3.01 avg=3.00\n",
      "[930 | 4324.63] loss=2.95 avg=3.00\n",
      "[940 | 4370.21] loss=3.03 avg=3.00\n",
      "[950 | 4415.90] loss=2.98 avg=3.00\n",
      "[960 | 4461.56] loss=2.92 avg=3.00\n",
      "[970 | 4507.23] loss=2.78 avg=2.99\n",
      "[980 | 4552.90] loss=2.95 avg=2.99\n",
      "[990 | 4598.56] loss=2.85 avg=2.99\n",
      "[1000 | 4644.22] loss=2.95 avg=2.99\n",
      "Saving checkpoint/rep/model-1000\n",
      "WARNING:tensorflow:From /tensorflow-1.15.2/python3.6/tensorflow_core/python/training/saver.py:963: remove_checkpoint (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Use standard file APIs to delete files with this prefix.\n",
      "======== SAMPLE 1 ========\n",
      " physician with over 55 years of experience in health management practice and the Districts most famous practicing physician. He is also a former United States Marshal, and currently holds the Districts highest honor in his district. He has been a national leader in public health. It is this expertise that will provide me with the expertise to lead a successful and responsible public health program. He has a personal commitment to the health of the United States people, and he believes health care should remain the supreme issue confronting the Federal Government through its annual budget request for fiscal year 2003. The Districts budget for the fiscal year 2003 will be $3.1 trillion. At the end of this fiscal year, it will have approximately $2.5 trillion. Therefore, I hope the Congressional Budget Office will review this budget for all of the years 1999 and 2002. Sincerely. John Wills\n",
      "I move to suspend the rules and pass the S, 473, the Freedom to Farm Act, I ask unanimous consent that the text of the legislation be printed in the RECORD.\n",
      "the Senate will consider the bill during its consideration on the Budget Committee on Wednesday. Thursday, and Friday. The Senate has approved the S, 473, the Freedom to Farm Act, by a 60 vote which will be heard from thenno less than eight members of the distinguished minority. We have passed an omnibus bill and the President has indicated that he intends to negotiate a deal on the trade bill by thendays deadline and a new bilateral trade bill in October. With respect to both NAFTA and the WTO, which we adopted in 1989. President Clinton has indicated that he intends to negotiate with and begin negotiating the TransAtlantic Trade and Development Act of 1996 through the President. There will be no major trade negotiations with Russia this year or next year. We will be moving forward with discussions with President Clintons Trade Commission, which has been set up during this period, including a major Trade Council meeting that is scheduled for October 7 which we expect to be held at the conclusion of this Presidents visit. We have been working on both the Presidents foreign policy, tax policy and foreign assistance. We have been working on issues that should receive bipartisan supportand the Presidents foreign policy and education policy. The administration cannot make significant progress with regard to this bill except with the assistance of the Administration and the Congress of the United States. The American people are sick and tired of seeing both the Bush years and the Clinton years and wonder how long we can tolerate the two political parties and the political parties with the political parties, I congratulate President Bush and Mrs. President Clinton for their cooperation in dealing with this crisis and for their work on the tax and education issues. We had hoped at one time that all the parties would deal with their differences peacefully and agree to some form of legislation which would help the American people, I could not believe with complete relief that I could not imagine that the people of Ohio and others who are living in poverty, who have been in the United States for generations, I am proud that we have been able to finally come to the Senate with legislation with which we can continue to deal and this bipartisan cooperation in trying to resolve this crisis. And I commend our staffs who have been here and worked diligently at all stages to help get this done, I want to give the people of Ohio the credit they have and to give the people of the country a chance to work toward a lasting solution for their situation. As this crisis continues, we need to look to our allies to help us. It is my hope that the people of this country will have confidence in the resolve and can help us get this over the hump, I would like to commend my friends and colleagues for their willingness to work and get the package passed.\n",
      "I move to suspend the rules and pass the S, 472, the Food and Drug Agreed upon Act. This legislation would give Congress the authority to approve emergency food aid in the event of a political crisis, and could be sent to President Bush for the approval of Congress. Since the emergency declaration over the course of September 11, 2001, we have asked and expected the President to approve emergency food relief in advance, regardless of the likelihood of a political crisis. We have asked the Senate to approve the Food and Drug Administration (FDA) emergency food and medical assistance to the people of this Nation. We have asked them to authorize the appropriate actions, and this body will continue to conduct its deliberations in the proper fashion, I think it is a positive step by all and in the best interests of those the Senator from Connecticut is to speak in support of the emergency declaration on emergency food and medical assistance. It appears that as President Bush has indicated with regard to this crisis, he is going to be seeking additional authority, I ask that he be granted all the authority is present. If he has not, to do so, we ask for his signature.\n",
      "I rise in support of this amendment. the current budget amendment that I offer this week to address the fiscal challenges facing the American people is the \"No Child Left Behind\" prescription. It is an amendment\n",
      "\n",
      "[1010 | 4711.34] loss=2.82 avg=2.99\n",
      "[1020 | 4757.03] loss=2.91 avg=2.99\n",
      "[1030 | 4802.68] loss=3.11 avg=2.99\n",
      "[1040 | 4848.29] loss=2.86 avg=2.99\n",
      "[1050 | 4893.91] loss=2.76 avg=2.98\n",
      "[1060 | 4939.52] loss=2.92 avg=2.98\n",
      "[1070 | 4985.10] loss=2.84 avg=2.98\n",
      "[1080 | 5030.63] loss=2.72 avg=2.98\n",
      "[1090 | 5076.12] loss=2.93 avg=2.98\n",
      "[1100 | 5121.78] loss=2.88 avg=2.97\n",
      "[1110 | 5167.35] loss=2.83 avg=2.97\n",
      "[1120 | 5212.96] loss=2.93 avg=2.97\n",
      "[1130 | 5258.60] loss=2.73 avg=2.97\n",
      "[1140 | 5304.20] loss=2.98 avg=2.97\n",
      "[1150 | 5349.76] loss=2.78 avg=2.96\n",
      "[1160 | 5395.33] loss=3.04 avg=2.97\n",
      "[1170 | 5440.95] loss=2.92 avg=2.97\n",
      "[1180 | 5486.55] loss=2.81 avg=2.96\n",
      "[1190 | 5532.20] loss=2.97 avg=2.96\n",
      "[1200 | 5577.90] loss=2.83 avg=2.96\n",
      "======== SAMPLE 1 ========\n",
      " These countries use military weapons at high rates, but they do so on a national basis to defend and destroy their neighbors, especially when there are conflicts all over the world. When there are military conflicts at a time when there is no humanitarian aid to the people of Africa, where it is a natural disaster, or a crisis in a major humanitarian situation, then you can bet there is a lot more that is at stake. We need to do everything we can to try to help the people of Africa where they have been, to try and help them. This is not a debate about whether or not we are good at fighting and saving lives. This is not a debate about whether we are prepared to do the things that are needed to solve these problems, or whether we are prepared to put our friends around the world in Africa to respond appropriately when it is necessary to do that, but this is a debate about whether or not we can make an impact. It is not about saving lives because the world is at war. It is not enough to be a hero for them. It is to be a hero for the people of Africa, because now is the time for all people around the world to do the right thing. This time is the time for all of us to do what may be a minor impact. It is a small change, I urge my colleagues to support it, I just want to say that the House of Representatives overwhelmingly passed this bill and I urge my fellow Members to vote against it. This is not a bill to be filibustered. We cannot go around here and filibuster in the Congress, and we cannot take one on one. We have to fight for what is right, not to be filibustered. We have to go through, in order to make the effort that a good, balanced bill can go forward. We are prepared to do that, I thank my friend and colleague and colleague with the distinguished chairman of the Subcommittee on the Human Resources. HHS and other members of the subcommittee, for allowing me to give a brief summary of their concerns with regard to this bill, I know there will be some disagreements, and I want everyone to know they need to support this amendment.\n",
      "I have read in the Committee on Ways and Means last Wednesday, which the Committee on Rules just issued on this issue, two statements of statements that the Senate had previously held on these issues. One of those statements said. \"When we look at the circumstances, we cannot get much worse than this. When the world is going down the path that the U.S. has established for these programs, we need to do everything we can to get more money into Africa. These programs have been a part of the defense appropriations bill and have really helped to create an international network, including the European Command, with the European Command, with the United States Command. So we have seen all of these programs have brought us into an area that has a high need and high demand.\" We have seen these programs have brought us into an area where we are spending more than we need, a higher amount of what we have available now in Africa. Our current allocation in Africa is just over 3 cents per pound. In Africa, what we spend is less. We spend on health care in all of these countries, and we are spending about $14That is in Africa for the health care of our fellow citizens. We get a lot of that with the way these programs work in Africa. We have to be ready to act now. This bill has to be done, I will not even go through what we have done on the floor today, which is a little bit hard, but the American people deserve to believe it is the right thing to do, and the American people have to support it. These programs are important to Africa, and this bill will make that happen. They are important in my mind, because they are one of the many things that have happened to Africa and it has changed Americas national security very dramatically. The way is not by having all programs funded simultaneously. Some have to be done differently. Other programs will be developed more as our troops are deployed or as our nations get on the ground or as we are getting prepared for contingencies. The problem for the majority today is that the Senate is trying to find some way in which we are able to spend all of these programs, one at a time and be as prepared as we can possibly be in the future, or be better than the Senate to accomplish all of these changes at the same time. In order to do this, the House has to do something different. The House has had some problems because we do not have a single program currently funded in a program as large as this one. We have had a number of different programsfunded. This one gets into one. But they differ in some points. So lets review some of the programs. There was a point in the last Congress where we did not even have our national defense expenditures for fiscal year 2001. The President had a great opportunity to address that issue, I am proud of what the Clinton administration\n",
      "\n",
      "[1210 | 5640.52] loss=2.72 avg=2.96\n",
      "[1220 | 5686.25] loss=2.97 avg=2.96\n",
      "[1230 | 5731.90] loss=2.98 avg=2.96\n",
      "[1240 | 5777.56] loss=2.92 avg=2.96\n",
      "[1250 | 5823.20] loss=3.00 avg=2.96\n",
      "[1260 | 5868.87] loss=2.73 avg=2.96\n",
      "[1270 | 5914.51] loss=2.92 avg=2.95\n",
      "[1280 | 5960.13] loss=2.91 avg=2.95\n",
      "[1290 | 6005.75] loss=2.85 avg=2.95\n",
      "[1300 | 6051.25] loss=2.77 avg=2.95\n",
      "[1310 | 6096.73] loss=2.90 avg=2.95\n",
      "[1320 | 6142.31] loss=2.88 avg=2.95\n",
      "[1330 | 6187.86] loss=2.88 avg=2.95\n",
      "[1340 | 6233.18] loss=3.05 avg=2.95\n",
      "[1350 | 6278.63] loss=3.04 avg=2.95\n",
      "[1360 | 6324.20] loss=2.85 avg=2.95\n",
      "[1370 | 6369.78] loss=2.96 avg=2.95\n",
      "[1380 | 6415.20] loss=2.91 avg=2.95\n",
      "[1390 | 6460.66] loss=2.97 avg=2.95\n",
      "[1400 | 6506.15] loss=2.91 avg=2.95\n",
      "======== SAMPLE 1 ========\n",
      " abortion from one of our Nations 21st century weapons programs, this time to protect Americans and our national security and to preserve our national heritage.\n",
      "I rise today in support of H.R, 775. The National Firearms Act of 1996 is long over, and I am grateful to my colleagues for their interest in improving firearms safety by allowing states to provide additional protections. In 1996, while I was the chairman of the Senate Foreign Relations Committee, the U.S. Department of State. U.S. Department of Justice, in a report to Congress, had the highest caseload of any foreign government for international purchases of firearms. The bill I am introducing today would allow these States to establish new gun safety programs. The bill also offers the same kinds of flexibility as the bill before us, including the ability to require more data from the Federal Bureau of Investigation, to ensure compliance, and to require additional requirements with the current law for the use of the firearms used for lawful purposes. In addition to providing the ability to require background checks on gun purchases, this bill would permit States to provide for greater financial assistance for law enforcement officials to assist in responding to firearm crime. Moreover. H.R, 775 contains a provision providing legal advice to law enforcement officials on how to protect their careers from firearms violence. This provision would, in turn, provide a more consistent and appropriate way for law enforcement officials to help law enforcement officials who are currently in active duty to deal with the current situation in their careers. For the second year in as many years. Secretary of State Madeleine K. Albright has promised to make efforts in cooperation with law enforcement officials to assist them in their efforts, including for the third year in a row sending police officers to assist law enforcement assistance. She did exactly this last year. In July of this year. President George Bush committed to working closely with Secretary Albright to develop a comprehensive firearms legislation. Today. Mr. President, I am proud to join my colleagues in support of H.R, 775, a bill that I believe will help strengthen gun safety, encourage gun ownership, and help strengthen U.S, law enforcement, I urge my colleagues to sign this bill, to please act with caution, to work with us, and to support my friend from Washington in his efforts to enact H.R, 775.\n",
      "today is the 21st birthday of the \"National Firearms Act of 1996. This act takes the necessary steps to safeguard the safety of our nations citizens, to prevent the perpetration of the deadliest weapons. The act authorizes the Secretary of State to promulgate a national uniform rule on the use of firearms in the use of such firearms for lawful purposes. The bill also authorizes the Secretary of State as he determines, and provides an additional, uniform rule governing the use of a firearm that meets the criteria proposed by the Secretary bylaws. As I have previously noted. \"The Firearms Act of 1986 is a national law for all States. This act will ensure that no Federal law that may operate in any State or any other place in the United States or an adjoining State may be used to restrict or limit the possession or use of firearms in a State. No State shall restrict or restrict the lawful possession or the possession of firearms that are used exclusively for the lawful purposes of public safety. This bill will also assure that no Federal law shall operate. No federal law shall restrict or restrict the lawful possession or use of a firearm that is used for lawful purposes. This act is bipartisan. No Federal law authorizes restrictions of the use or use by law enforcement officials of a firearm unless specifically authorized by this act. This is an amendment of great substance. This bill serves the common good, I ask unanimous consent to have an identical measure in each of the Houses of Congress.\n",
      "as the ranking member of the Commerce Committee, I am committed to working with the majority leader on the Foreign Relations Committee, I have no doubt that this conference report will be very timely for both parties. Therefore. I will not be attending today. Nevertheless. the minority leader is in excellent shape, with regard to the legislation he has introduced, I thank the majority leader on the Committee, who knows, the majority leader understands all we are trying to accomplish.\n",
      "I would like to take a moment to address a problem in the legislation, I have listened very diligently to every issue in the bill that deals with this issue. If it is a domestic issue, it is a domestic issue, it is really tough to legislate around. But certainly an issue that relates to the domestic issue is the issue of trafficking in drugs. There is no reason to believe anybody has been forced or coerced into being in the drug business. The legislation that is pending, is a result of this process and not a result of what I do and the staff that I serve in the House, I think the Members of Congress know and agree with me. This legislation is being introduced in a responsible manner, I am grateful for the efforts of the leaders in the Committee to move it forward with both the parties working\n",
      "\n",
      "[1410 | 6568.70] loss=2.74 avg=2.95\n",
      "[1420 | 6614.28] loss=2.92 avg=2.95\n",
      "[1430 | 6659.88] loss=2.98 avg=2.95\n",
      "[1440 | 6705.41] loss=2.80 avg=2.94\n",
      "[1450 | 6750.96] loss=2.97 avg=2.94\n",
      "[1460 | 6796.51] loss=2.94 avg=2.94\n",
      "[1470 | 6842.05] loss=2.76 avg=2.94\n",
      "[1480 | 6887.61] loss=3.03 avg=2.94\n",
      "[1490 | 6933.14] loss=2.95 avg=2.94\n",
      "[1500 | 6978.71] loss=2.93 avg=2.94\n",
      "Saving checkpoint/rep/model-1500\n",
      "[1510 | 7027.57] loss=2.81 avg=2.94\n",
      "[1520 | 7073.24] loss=2.94 avg=2.94\n",
      "[1530 | 7118.86] loss=2.88 avg=2.94\n",
      "[1540 | 7164.38] loss=2.81 avg=2.94\n",
      "[1550 | 7210.03] loss=2.74 avg=2.94\n",
      "[1560 | 7255.59] loss=2.94 avg=2.94\n",
      "[1570 | 7301.17] loss=2.95 avg=2.94\n",
      "[1580 | 7346.74] loss=2.77 avg=2.93\n",
      "[1590 | 7392.27] loss=2.92 avg=2.93\n",
      "[1600 | 7437.83] loss=2.95 avg=2.93\n",
      "======== SAMPLE 1 ========\n",
      " Century and the future of our country. The American people. will benefit, not only from the American farmers who are so productive. So we would have to have some kind of competition among the producers of food, I believe that is the best way to do that the best way possible, I think the other thing we can do without an increase in price is to find a better market. The way we can grow and grow, the best we can do is to increase prices. That is what we are talking about. That is what we want to do. We want to lower prices to the tune of $1.We also want to get us a tax on all the fruits and vegetables. You know what we are talking about. Let us keep some of that for the next year or so. We are going to be back to the level we were in last year. It is hard to do that for a while. Then lets look at the other thing that is really bad. As we have all heard, it is the other people. They have to put more money on the top of their budget. They do not want to let the people who are paying the greatest amount of money out of their pocket. Lets look at it a little bit deeper into why it takes so long to give everybody money back to the taxpayers. Well, you have to have a minimum amount of money for tax cuts. A lot of times that is enough to get back money from the taxpayers. If everybody would put in the extra money, everybody could get back the additional money that there was already when everyone else lost it. So you look at it a little bit, I am not going to give you a bunch of numbers, the number of those people who have paid a lot of taxes, but most of them have gone out of work. We have heard beforeand all throughout the countrythat the tax relief we have provided for the past year is one of the highest in history. You look at the other side of the argument. We dont have a lot of tax relief now. So we are just going to look at the other thing that would cause us to be down about 10 percent in the tax revenue we give. That is the way it works. Lets try to figure out a way to give everybody, because we have seen that in the United States over the last few years in the economy, we have seen the decline in productivity, and we have seen that we have a decline in productivity of our capital, which is, basically, when you go back up. We have seen, in other words, when we go up to where we get from a higher surplus with the kind of taxes we have, we have a reduction in the surplus, and we want to get it back and, hopefully, bring it back by lowering the rate of income tax and the rate of taxation for everybody, which is exactly what it is, I just came off the phone with Governor McCool, a very wellrespected Governor. Governor McCool, we both graduated from high school in the State of Nevada, and I have a very strong relationship with him. He was really excited when it was announced this year that he was going to be moving back back to Washington from his house in Nevada. Maybe even down to Washington because he had a chance to see it. He was excited about the opportunity to see it. We all have family roots here in Las Vegas, I have many, many people. One is a lady from Las Vegas. Ms. Carol. Another is a businesswoman from Las Vegas. Ms. Carol, as she said, is a small businesswoman at the grocery store. She is married, and she has three children. There is a family that lives in a single family in Las Vegas that is a high income single family. A family in a single family in a single family in one of the lowest prices on the market, where it is in, we will never tell you exactly what the cost is, but it goes to the taxpayers of America. That is really what this tax hike is all about. It is about the government going after those who have had a great financial and financial success in the last year. We are down now on a deficit. But it seems to me the time is right, I think that the American farmer and the United States of America knows exactly why they have been down so far. We have tried to come up with a system that works just as well. That is what we are trying to do. If we can go back down here and get a number of the other items which have led to a higher tax, we can get back to the business of government. Government is going after all of our family farmers and all of our small rural economies for money. They grow a huge number of crops every single year. They produce a great deal of food in the world. They grow a lot of products for home consumption. They have a huge surplus. They are down at 12 percent a year in farm sales. They also have a whole new class of agriculture. They have a whole new segment\n",
      "\n",
      "[1610 | 7500.46] loss=2.76 avg=2.93\n",
      "[1620 | 7546.10] loss=2.88 avg=2.93\n",
      "[1630 | 7591.68] loss=2.97 avg=2.93\n",
      "[1640 | 7637.31] loss=2.95 avg=2.93\n",
      "[1650 | 7682.79] loss=2.84 avg=2.93\n",
      "[1660 | 7728.36] loss=2.94 avg=2.93\n",
      "[1670 | 7774.04] loss=2.80 avg=2.93\n",
      "[1680 | 7819.53] loss=2.99 avg=2.93\n",
      "[1690 | 7865.11] loss=2.87 avg=2.93\n",
      "[1700 | 7910.65] loss=3.04 avg=2.93\n",
      "[1710 | 7956.21] loss=3.04 avg=2.93\n",
      "[1720 | 8001.83] loss=2.65 avg=2.93\n",
      "[1730 | 8047.34] loss=2.86 avg=2.93\n",
      "[1740 | 8092.96] loss=2.69 avg=2.93\n",
      "[1750 | 8138.49] loss=2.81 avg=2.92\n",
      "[1760 | 8184.24] loss=3.10 avg=2.93\n",
      "[1770 | 8230.00] loss=3.07 avg=2.93\n",
      "[1780 | 8275.58] loss=2.73 avg=2.93\n",
      "[1790 | 8321.22] loss=2.85 avg=2.92\n",
      "[1800 | 8366.79] loss=3.04 avg=2.93\n",
      "======== SAMPLE 1 ========\n",
      " whether we are in a situation with Russia. And it is our responsibility to bring this action to an end, if the United States fails to defend itself. We have always recognized that this is a very grave, very important issue that we ought not to make a judgment of whether Russia is a serious threat to our interests. As with the nuclear issue that has become the top concern or question of this debate of some time now, this problem is extremely serious and we cannot ignore its very serious ramifications or the very serious implications of it. And that is why every Member of this body ought to be clear of its determination and action. In sum, I, along with a majority of this House, will do everything we can to continue to ensure that the nuclear issue of our future energy policy comes to the American people free from any unnecessary and unquantifiable risks of our continuing to expand our nations potential and making our own decisions regarding the world without a Congressional oversight.\n",
      "the President said that I think we could be assured of the fact that a reasonable and informed, responsible, and informed, and conscientious majority in Washington. DC, would support this. But not the President. The President is a bad man, and he is entitled to some sort of constitutional right, to his country if he feels that he has acted in a constitutionally necessary fashion or has acted in a way which is consistent with the Constitution, I would like to express my disappointment, I have said I know we do not have to vote for this bill because I do not believe it should be done,\n",
      "this is an important bill and the legislation has been a good one. The President made the comments about how he believed in the importance of these programs at a time of great national crisis. But I am not sure there is any question that there is no longer one constitutional right to any program. What I have said is I think that when the Senate voted against it, it was to the best of our ability to determine whether or not there was another constitutional way to deal with the issue. If the President would like to have an opportunity to address this matter and make my position crystal clear, I would say, yes, it is important that he get on with it, I would suggest to my colleagues that by all means, I support the President on this issue and strongly urge the Senate to vote to pass this bill and that Members do likewise.\n",
      "I stand today in support of this bipartisan legislation on extending the tax credit to families of dependent children. This means they are paying more in taxes than before because of the tax increase. We can extend this tax credit to anyone who has given money to an organization. People who have given money to groups that helped our family farmers, for example, because there have been $1 million in tax savings for the last 5 years are now paying a lower tax rate of 18 percent on the dollar than they were in the past. The tax credit should not be extended to any group in either the family or the church, I dont believe that this is fair, I have said it, and I have been wrong and I continue to say it, and I have said it in public remarks as a Senator, so let me just say that I am proud to say that in the last 10 years, we have seen what the President said is not only good, but the best we have seen in 30 years. If, however, as I told the minority, this bill does not get the broad support and the support that is right from the majority, it is not going to be passed. There are a lot of people who do not support this bill. We just got it wrong, I have never been afraid to support legislation that gets us to vote for a particular tax cut, I will say the President has done a good job, I am confident, when I come to the floor and I look at the bill, he will bring it up with us and we will support it. When I said I had been very pleased with all of this, I felt very strongly that this was a piece of legislation that was a good piece of legislation which would provide for the kind of opportunity where we go back to a real sense of our State of the Nation and of the American tax system. And we will support it in the strongest possible terms. In summary, this is going to be a tremendous bill. It is going to be a bipartisan bill. It will have an impact on families like the millions they have benefited by. It is going to have an impact on people like many of those I referred to earlier who will go to the polls next Tuesday. It will have an impact on the economy more likely to benefit them than to their kids. It probably will be a significant one to the economy, I will tell my colleagues this before we go through the next 2 weeks of debating the bill, we are ready to wrap up the debate. I would like to thank a great many people for their support, I know there are many of them here who really have had their support and have asked my office of permission to meet with them, I\n",
      "\n",
      "[1810 | 8429.66] loss=2.97 avg=2.93\n",
      "[1820 | 8475.22] loss=2.84 avg=2.93\n",
      "[1830 | 8520.88] loss=2.92 avg=2.93\n",
      "[1840 | 8566.48] loss=2.93 avg=2.93\n",
      "[1850 | 8612.03] loss=3.00 avg=2.93\n",
      "[1860 | 8657.80] loss=3.00 avg=2.93\n",
      "[1870 | 8703.33] loss=2.81 avg=2.93\n",
      "[1880 | 8748.97] loss=2.87 avg=2.93\n",
      "[1890 | 8794.53] loss=2.82 avg=2.92\n",
      "[1900 | 8840.21] loss=2.80 avg=2.92\n",
      "[1910 | 8885.99] loss=2.85 avg=2.92\n",
      "[1920 | 8931.52] loss=2.83 avg=2.92\n",
      "[1930 | 8977.12] loss=2.84 avg=2.92\n",
      "[1940 | 9022.65] loss=2.97 avg=2.92\n",
      "[1950 | 9068.31] loss=3.10 avg=2.92\n",
      "[1960 | 9114.12] loss=2.80 avg=2.92\n",
      "[1970 | 9159.70] loss=2.84 avg=2.92\n",
      "[1980 | 9205.35] loss=3.05 avg=2.92\n",
      "[1990 | 9250.91] loss=2.99 avg=2.92\n",
      "[2000 | 9296.59] loss=2.79 avg=2.92\n",
      "Saving checkpoint/rep/model-2000\n"
     ]
    }
   ],
   "source": [
    "RUN_NAME = 'rep'\n",
    "sess = gpt2.start_tf_sess()\n",
    "\n",
    "gpt2.finetune(sess,\n",
    "              dataset=file_name,\n",
    "              model_name='124M',\n",
    "              steps=2000,\n",
    "              restore_from='fresh',\n",
    "              run_name=RUN_NAME,\n",
    "              print_every=10,\n",
    "              sample_every=200,\n",
    "              save_every=500\n",
    "              )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "AFK52MZ9M0ZZ"
   },
   "source": [
    "Make sure you restart runtime before running this cell!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 52
    },
    "colab_type": "code",
    "id": "m-It91kVf-ar",
    "outputId": "39336d8f-3da1-44bd-9457-ba0c6fa43ff9"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loading checkpoint checkpoint/rep/model-2000\n",
      "INFO:tensorflow:Restoring parameters from checkpoint/rep/model-2000\n"
     ]
    }
   ],
   "source": [
    "RUN_NAME = 'rep'\n",
    "sess = gpt2.start_tf_sess()\n",
    "gpt2.load_gpt2(sess, run_name=RUN_NAME)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "_8SUbywJM5GA"
   },
   "source": [
    "Generate republican answers\n",
    "You can download the weights from [here](https://drive.google.com/file/d/1qRdGcykWvwE0WHPC1LGrDbE-1DwSMPuc/view)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "X4r5VGPqWxiY"
   },
   "outputs": [],
   "source": [
    "questions = [\"Should abortion be illegal?\",\n",
    "             \"What do you believe about tax increases?\",\n",
    "             \"Should same-sex marriage be legal?\",\n",
    "             \"Is climate change real?\",\n",
    "             \"Should immigrants be allowed to obtain citizenship?\",\n",
    "             \"Who should be given voting rights?\",\n",
    "             \"Should we have higher taxes for higher incomes?\",\n",
    "             \"Should we allow death penalty?\",\n",
    "             \"Should we have universal healthcare?\",\n",
    "             \"Do government regulations hinder free market capitalism?\",\n",
    "             \"What do you think about the current president?\",\n",
    "             \"Should we reduce national debt?\",\n",
    "             \"Should we increase spending on healthcare?\",\n",
    "             \"Should we increase spending on education?\",\n",
    "             \"Should every American have equal opportunities regardless of sex, age and race?\",\n",
    "             \"Should be introduce more gun control measures?\",\n",
    "             \"What do you think about the current president?\",\n",
    "             \"Should Americans be free?\",\n",
    "             \"What party do you support?\",\n",
    "             \"What is the biggest threat to America?\"\n",
    "             ]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 469
    },
    "colab_type": "code",
    "id": "Lb36B8L_W-xJ",
    "outputId": "f73611da-86ef-4c02-b85d-63d393faf25b"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Should abortion be illegal? We agree with the Supreme Court that abortion should be legal. Congress should not be at war with abortion. In fact, I believe that there should be a preemption of the law. But the question becomes, what is necessary for the Supreme Court to decide? The Supreme Court has already ruled in Roe v. Wade that abortion is not an act of violence, that it is a legitimate social and medical procedure, and that the law is not in conflict with the Constitution. We need to do something about it. We have a chance to do that. But we have to do it in a way that does not give the American people the opportunity to stop it. That, is what the Constitution requires. It is not a decision by the Federal Government. It is not a decision by the States. It is a decision by the people of the States. We need to do this. We need to do it with a sound decision. So, yes, we are going to have to do it. But we are not going to be able to do it. We need to do it in a way that is consistent with the Constitution and consistent with the law. That is why we need to do it.\n",
      "I rise to address the issue of the law\n",
      "What do you believe about tax increases? Tell me. This is a tax increase. This tax increase? $6 billion under the Clinton tax cuts, over $4 billion under Bushs tax cuts, over $8 billion under Bushs tax cuts, over $10 billion under Bushs tax cuts, $12 billion under Bushs tax cuts. What are you going to do? You have to work it out. We have to spend it. We have to spend it. We have to spend it. That is where we are going to have to spend it, and that is where we are going to have to spend it. But we are going to spend it. We are going to spend it in the way we want it. We are going to spend it in the way we want it. We are going to spend it in the way we want it. Ask the President to spend it. Ask him to spend it. Ask him to spend it. There is a budget that says the President is going to spend it. There is a budget that says if we are going to spend it, we are going to spend it. So we are going to spend it. That is where we are going to have to spend it. We are going to spend it, and that is where\n",
      "Should same-sex marriage be legal? What about abortion? Abortion is wrong, but only when the Government asks us to do so. The fact is, we have a right to know what is going on in our society. Why is this important? Because it is the responsibility of every American to know what is going on. Abortion is wrong. Abortion is wrong, but only when the Government takes the life of a human life. Abortion is wrong, but only when the Federal Government takes the life of a human life. Abortion is wrong. Abortion is wrong. Abortion is wrong. Abortion is wrong. Abortion is wrong. Abortion is wrong, but only when the Federal Government takes the life of a human life. Abortion is wrong, but only when the Federal Government takes the life of a human life. Abortion is wrong, but only when the Federal Government takes the life of a human life. Abortion is wrong, but only when the Federal Government takes the life of a human life. Abortion is wrong, but only when the Federal Government takes the life of a human life. Abortion is wrong, but only when the Federal Government takes the life of a human life. Abortion is wrong, but only when the Federal Government takes the life of a human life. Abortion is wrong, but only when the\n",
      "Is climate change real? Are we going to have a hot summer? Some would say, yes. That is the issue. There are only 2 percent of the worlds population that is without power today. The other 3 percent are in extreme drought conditions. We are in a time when we are in a time of high oil and gas prices. We are going to have a crisis in oil and gas production. We are going to have to do something to help the environment, particularly the environment of the West. We are going to have to have a conservation strategy to keep the oil and gas that we have and to conserve it. We are going to have to do something to help the environment. A lot of that is going to come through conservation. We are going to have to do something to help the environment. That is not a question of whether we do that or whether we do not. We are going to have to do something to help the environment. That is not a question that we are going to know. It is not a question that we are going to have to know. We are going to have to do something. The science is clear. There are many scientists who have proven this to be a very satisfactory solution to our energy problems. They are saying that we\n",
      "Should immigrants be allowed to obtain citizenship? Do they have the right to go to the United States? It is true that they do not have the right to come to the United States and do not have the right to go to the United States. But we have to remember that there are noncitizens who come to the United States and we have to recognize that. If we ignore their rights, if we ignore them, we are going to have a very real problem. It is a real problem if we ignore the rights of noncitizens, and it is going to be a real problem if we ignore the rights of noncitizens. So lastly, I have the bill before the House. It is a good bill. It is not perfect. The bill is not perfect. The bill is not perfect. But the bill is a good bill, and the bill is a good bill. It should be passed before we go to the United States of America, before we go to the United States.\n",
      "we now have a very important opportunity to address the problems that the immigration laws of 1996 and the Immigration Reform and Control Act of 1996 will have to deal with, and we need to take care of, the problem that we will be dealing with in this country under the immigration laws of 1996 and the\n",
      "Who should be given voting rights? We are talking about socalled \"nonmember\" groups, the one percent, I would guess the 7 percent would be excluded. Why? Because they are not eligible for the voting \"rights\" they are entitled to under the law. They cant, I am not sure they would be able to vote a vote for the AARP. They cant vote for the National Federation of Independent Business, I ask my colleagues to look at the provisions in this bill that we passed in the 50th Congress, and we voted to deny voting rights to a group of people who are not even qualified to vote. They cant vote for the Citizens United decision. They cant vote for a group of people who are not even qualified to vote. Let me ask my colleagues to look at the provisions in this bill. We passed this bill in the 50th Congress, we passed this bill the 60th Congress, and now we have a bill that has to be debated and debated, and I urge my colleagues to vote for this bill. I ask unanimous consent that the full text of the bill, which is now in the RECORD, be printed in the RECORD.\n",
      "I want to thank the gentleman from Pennsylvania for yielding time I think for the record. We have had\n",
      "Should we have higher taxes for higher incomes? If we have higher taxes, why should we have higher income? You can do that, I suggest, simply because the quality of education is so important. It is important to us that we have higher education. We ought to have more people who are going to be able to pay those taxes. But if we have higher taxes, we ought to have more people who are going to be able to pay those taxes. We ought to have more people who are going to have the opportunity to pay higher taxes. We ought to have more people who are going to have the opportunity to pay higher taxes. But if we have higher taxes, we ought to have more people who are going to be able to pay those taxes.\n",
      "I thank the gentleman from Texas for yielding and also the gentleman from Massachusetts , and the gentleman from Minnesota . If we are going to allow the lower income people to pay that higher income taxes, we ought to have a tax increase. We ought to have a tax increase. But that is not the way we ought to operate. In fact, the gentleman from Massachusetts and I have done a significant amount of other work to make sure that the higher income people pay taxes, I think that is a real problem. We ought to tax the\n",
      "Should we allow death penalty? It is a poison pill to death. It is a humane death penalty. It is a humane death penalty. Then what do we do if we do not kill all of these people? We have a lot of options. We can go forward with those. We can do it, we can do it. We can do it. The fact is that we are going to have to have a massive socialization of the tax system and an enormous redistribution of wealth into the hands of the rich. We cannot do that. That means more taxes, more taxation. That means more deregulation, more regulation, more regulation. That means more taxation. We passed the death tax earlier, but we have not passed the death tax, I do not believe that. We have to do it. We can do it, I hope we do it, I hope we do it. It is just a matter of time, I hope that is the case. But we do not do it on a daily basis. We do not go to the funeral directors. We go to the police. We do not go to the courts. We do not go to the child labor laws. We do not go to the child labor laws. We do not go to the death tax.\n",
      "Should we have universal healthcare? I think that is a very reasonable question. When we talk about universal healthcare, then we have to make some very clear distinctions. The only way we can have universal healthcare is if it is the right thing for the people to do, we can have universal healthcare. If we are going to have universal healthcare, then we need to make that clear. We have to make it clear. But there are two barriers to that. One is the fact that on that matter, and I am not sure that we are going to be able to do that. So, yes, there are some hurdles to that. But, again, I am not sure that it is going to work out. We have to make sure that we are talking about, we are talking about over the course of the next couple of hours, which is going to be the time that has to be spent with that debate. That is going to be an important debate. And all of the discussions we have, I am not sure we are going to be able to do that. So I think there are a number of issues that are going to be very important in this debate. But I think there are a lot of issues that are going to be very important, and I think we\n",
      "Do government regulations hinder free market capitalism? What about a regulatory regime that hurts the consumers and encourages them to pay more than what they get? What about reimbursement rates that discourage employers from hiring more people? What about the \"costsharing\" that is offered by employers? What about the \"service tax\" that is offered by employers? What about the \"tax.\" that is a requirement that employers pay? What about the \"tax cut\" that is offered by employers? What about the \"tax\" that is offered by the government? What about the \"tax relief\" that is offered by the government? What about the \"tax relief\" that is offered by the government? What about the \"tax cut\" that is offered by the government? What about the \"tax relief\" that is offered by the government? What about the \"tax relief\" that is offered by the government? What about the \"tax\" that is offered by the government? What about the \"tax\" that is offered by the government? What about the \"tax\" that is offered by the government? What about the \"tax\" that is offered by the government? What about the \"tax\" that is offered by the government? What about the \"tax\" that is offered by the government? What about the \"\n",
      "What do you think about the current president? Do you think that he is trying to do to his own country, that he is trying to do to the people of America, that he is trying to make things better for the American people? I do not think so. In fact, if you listen to the American people, I bet you they would conclude that they would be more than happy to support the President if he were to be elected. What do you think about the current president? Do you think he is attempting to put the people of America back together and make America better for all Americans? I think he is trying to do so. What are you going to do to improve the economy of the United States? The President of the United States should be able to take his position and say that he would do that, I think that is a different position than what we have now. That is why I think it is so important that we act on this issue. We ought to get together, I think the President ought to be able to do that. He should be able to say that he has a responsibility to make things better for the American people. He ought to be able to put his hand on the shoulders of the people that he is elected to represent.\n",
      "I am sure that\n",
      "Should we reduce national debt? I believe the answer is yes. That is because we are now dealing with a national debt that is $5.6 trillion, and we are dealing with a national debt that is $10.5 trillion. If the President has a national debt, he has agreed to reduce that debt by $1.5 trillion, and we have agreed to reduce that debt by $3.5We have agreed to reduce national debt by $1.5. We have agreed to reduce debt by $1.5 trillion. What is the national debt? Well, the national debt is the debt that we do not have as a country, and we have agreed to reduce that debt by $1.5 trillion. But if the President has a national debt, he has agreed to reduce it by $1.5 trillion. What is the national debt? The national debt is the debt that we have now, but we do not have the money to pay for it. We do not have the money to pay for it. We are paying for it for the first time in 25 years. We are paying it for the first time in 20 years. It is a long way from the beginning. We have paid for it in the debt that we have. We\n",
      "Should we increase spending on healthcare? We are in the midst of a massive expansion of our financial system, and we have got to do better than we were doing last year. We have got to pass a budget compact in 1996 to protect the American people and keep them in it. We passed it in 1997. We have done it in 1998. We have done it in 1999. We passed it in 2000. We passed it in 2001. We passed it in 2002. We passed it in 2003. We passed it in 2004. We passed it in 2005. We passed it in 2006. We passed it in 2007. We passed it in 2008. We passed it in 2009. We passed it in 2010. The American people have an obligation to keep us in it. They have an obligation to do better than we did last year. Let us make sure we pass a budget. We have to pass a budget compact in 1996 to protect the American people and keep them in it. We have got to pass a budget. We have got to pass a budget in 2002 to protect the American people and keep them in it. We have got a budget. We cannot do it. Let me say again. Let me tell you something else. This year, the Republican budget contains an increase in\n",
      "Should we increase spending on education? No. We are going to spend more money on education than we do on terrorism. We are going to spend more on infrastructure. We are going to spend more on transportation. No. We are going to spend more money on transportation than we do on terrorism. We are going to spend more money on education. We are going to spend more on transportation. No, we are going to spend more money on education than we do on terrorism. We are going to spend more money on transportation. And we are going to spend more money on education than we do on terrorism. We are going to spend more money on transportation. No, we are going to spend more money on transportation than we do on terrorism. We are going to spend more money on transportation. No, we are not going to spend more money on transportation than we do on terrorism. We are going to spend more money on transportation than we do on terrorism. We are going to spend more money on transportation. No, we are not going to spend more money on transportation than we do on terrorism. We are going to spend more money on transportation than we do on terrorism. We are going to spend more money on transportation than we do on terrorism. We are going to spend more money on\n",
      "Should every American have equal opportunities regardless of sex, age and race? If we are going to believe in equality of opportunity, or at least we should, why should people be treated as equally if they are born in the same place, regardless of their gender, even if they are married? The history of the European Union demonstrates that our country is a country of equal opportunity. If we are going to believe in equality of opportunity for every American, why should people be treated like equals? If we are going to believe in equality of opportunity for every American, why should we be treated any differently than a man or a woman? If we are going to believe in equality of opportunity for every American, why should we be treated any differently than a woman or a man? If we are going to believe in equality of opportunity for every American, why should we be treated any differently than a man or a woman? If we are going to believe in equality of opportunity for every American, why should we be treated any differently than a man or a woman? If we are going to believe in equality of opportunity for every American, why should we be treated any differently than a man or a woman? If we are going to believe in equality of opportunity for every American, why should we be treated any differently than a woman or a man\n",
      "Should be introduce more gun control measures? No, I think it would be really wrong to do so. And I think that we would be wrong to do so. That is why I want to go back to the issue of the price of guns. If you look at the background of the gun control legislation, the background that we passed in 1996 with regard to background checks was that we were trying to provide people with a gun. We do not want to hear anybody say that this is the right way to do it. We want to provide some sort of background check for people who have a gun, because that is what we do, and so we want to make sure that those who are at risk of being shot by a gun are not going to be able to purchase a firearm. We do not want to hear anybody saying that. It is not the right way to do it. We do not want to hear anybody saying that. We want to make sure that those who are at risk of being shot by a gun are not at risk of being able to purchase a firearm. We want to make sure that those who are at risk of being shot by a gun are not at risk of being able to purchase a firearm. But we also want to make sure that people who are at risk\n",
      "What do you think about the current president? In 1997, he had 103.000 people in America, some of whom were his constituents. In 1998, he had 103.000 people in America. In 1999, he had 103.000 people in America. That was the year that the Democrats came in. It was a glorious year. The Democrats came in and said: We are going to have to do something about it. They said: We are going to have to do something about it. And then they took a look at the House and said: We are going to take a look at the Senate and say: Now, do we have some ideas? The President talked about a lot of things. He said: We are going to take a look at the Senate and say: Look, we are going to have to do something to address this problem. We are going to have to take a look at the House and say: We are going to take a look at the Senate and say: We are going to take a look at the President. And then we are going to take a look at the House and say: We are going to take a look at the Senate and say: We are going to take a look at the President. Then we are going to take a look\n",
      "Should Americans be free? When the people of this country are free to go to school, and when the people of the world are free to go to work and to go out and to spend their money on their own needs, and when the people of this country are free to spend the same money that we invest in education, we should be able to do it. It is because we are able to do it. If we can do it, then we are able to do it. If we can do it, then we should be able to do it. But we need to do it. We need to think about it. We need to think about it. Let us not stand back and say we are going to take the money and we need to spend it. Let us not also stand back and say we are going to spend it. Let us not stand back and say we are going to spend it. We are not going to spend it. We could spend it on a missile defense, on a missile defense, or on intelligence, or on the military, or on housing, or on defense and on defense. But the fact is, the same thing we are going to spend that money on is going to be spent on a missile defense. So we have to\n",
      "What party do you support? You are a Democrat. You are a Republican. You are a Democrat. You are a Republican. You are a Democrat. You are a socialist. You are a socialist. So you are going to be a little bit like Hitler. You are going to be a little bit like Hitler. You are going to be a little bit like Hitler. That is what you are going to be like. That is what somebody is going to be like. That is what we are going to be like, and that is what is going to happen. So, yes, I believe that we should become a Nation of immigrants. We should have less immigration from other countries than the average American. We should have more immigration from other countries than we have to the United States. We should not have more immigration from other countries, I think that this is a very important part of our country, and that is why we should allow more people to come in and to stay in this country. But I think at the same time that we need to encourage more people to come here, we should not let the country of immigrants become less secure. We should not let the country of immigrants become more secure. So, yes, I think we ought to encourage them to come.\n",
      "What is the biggest threat to America? We should not make that kind of a mistake. In fact, there is another area of concern. In the area that is most under threat, we have our own military. We have our own defense. We have our own resources. We have our own energy. We have our own technology. We are a country that has our own military. We have to protect ourselves. We have to defend ourselves. And we have to protect ourselves. In this segment of the country, we have a lot of other things. We have a lot of other things besides weapons of mass destruction. We have a lot of nuclear technologies. We have a lot of other things, too. We have a lot of other things. We have a lot of other things in this country. We have got to defend ourselves. We have got to defend ourselves. We have got to defend ourselves. That means we have got to defend ourselves. That means we have got to defend ourselves. In this segment of the country, we have done this. We have got to defend ourselves. We have got to defend ourselves. In this country, we have got to defend ourselves. We have got to defend ourselves. We have got to defend ourselves. And that means we have got to\n"
     ]
    }
   ],
   "source": [
    "with open(\"rep_generated.txt\", 'w') as f:\n",
    "  for i in range(len(questions)):\n",
    "    text = gpt2.generate(sess,\n",
    "              length=250,\n",
    "              temperature=0.7,\n",
    "              prefix=questions[i],\n",
    "              nsamples=1,\n",
    "              batch_size=1,\n",
    "              return_as_list=True)[0]\n",
    "    print(text)\n",
    "    f.write(text + '\\n')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "2Z4A6N92Xryb"
   },
   "source": [
    "Train for democrat data\n",
    "You can download the weights from [here](https://drive.google.com/file/d/1qRdGcykWvwE0WHPC1LGrDbE-1DwSMPuc/view)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "colab_type": "code",
    "id": "NgWuoltFXz79",
    "outputId": "c65504ec-2fa3-4e0c-aa37-1dd43c860564"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/gpt_2_simple/src/sample.py:17: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Use tf.where in 2.0, which has the same broadcast rule as np.where\n",
      "Loading checkpoint models/124M/model.ckpt\n",
      "INFO:tensorflow:Restoring parameters from models/124M/model.ckpt\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "  0%|          | 0/1 [00:00<?, ?it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loading dataset...\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 1/1 [04:06<00:00, 246.93s/it]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "dataset has 36883131 tokens\n",
      "Training...\n",
      "[10 | 53.17] loss=3.19 avg=3.19\n",
      "[20 | 99.40] loss=3.07 avg=3.13\n",
      "[30 | 145.83] loss=2.92 avg=3.06\n",
      "[40 | 192.03] loss=3.13 avg=3.08\n",
      "[50 | 238.33] loss=3.10 avg=3.08\n",
      "[60 | 284.63] loss=3.00 avg=3.07\n",
      "[70 | 330.99] loss=3.03 avg=3.06\n",
      "[80 | 377.20] loss=3.13 avg=3.07\n",
      "[90 | 423.42] loss=3.07 avg=3.07\n",
      "[100 | 469.55] loss=3.08 avg=3.07\n",
      "[110 | 515.76] loss=3.10 avg=3.08\n",
      "[120 | 561.83] loss=3.03 avg=3.07\n",
      "[130 | 607.95] loss=3.05 avg=3.07\n",
      "[140 | 654.15] loss=3.01 avg=3.06\n",
      "[150 | 700.33] loss=3.01 avg=3.06\n",
      "[160 | 746.31] loss=3.10 avg=3.06\n",
      "[170 | 792.40] loss=3.02 avg=3.06\n",
      "[180 | 838.38] loss=3.06 avg=3.06\n",
      "[190 | 884.46] loss=2.96 avg=3.06\n",
      "[200 | 930.49] loss=3.27 avg=3.07\n",
      "[210 | 976.53] loss=3.04 avg=3.07\n",
      "[220 | 1022.70] loss=3.16 avg=3.07\n",
      "[230 | 1068.85] loss=3.03 avg=3.07\n",
      "[240 | 1114.93] loss=3.14 avg=3.07\n",
      "[250 | 1161.04] loss=2.90 avg=3.06\n",
      "[260 | 1207.23] loss=3.06 avg=3.06\n",
      "[270 | 1253.40] loss=3.09 avg=3.07\n",
      "[280 | 1299.42] loss=3.13 avg=3.07\n",
      "[290 | 1345.44] loss=3.05 avg=3.07\n",
      "[300 | 1391.66] loss=3.10 avg=3.07\n",
      "[310 | 1437.75] loss=2.85 avg=3.06\n",
      "[320 | 1483.88] loss=2.75 avg=3.05\n",
      "[330 | 1529.93] loss=2.98 avg=3.05\n",
      "[340 | 1576.05] loss=2.94 avg=3.04\n",
      "[350 | 1622.22] loss=3.00 avg=3.04\n",
      "[360 | 1668.29] loss=2.92 avg=3.04\n",
      "[370 | 1714.33] loss=3.07 avg=3.04\n",
      "[380 | 1760.49] loss=2.99 avg=3.04\n",
      "[390 | 1806.50] loss=3.02 avg=3.04\n",
      "[400 | 1852.53] loss=2.95 avg=3.03\n",
      "[410 | 1898.53] loss=2.97 avg=3.03\n",
      "[420 | 1944.70] loss=2.99 avg=3.03\n",
      "[430 | 1990.78] loss=2.99 avg=3.03\n",
      "[440 | 2036.88] loss=2.89 avg=3.03\n",
      "[450 | 2082.92] loss=2.96 avg=3.02\n",
      "[460 | 2129.17] loss=2.97 avg=3.02\n",
      "[470 | 2175.22] loss=2.95 avg=3.02\n",
      "[480 | 2221.35] loss=2.91 avg=3.02\n",
      "[490 | 2267.41] loss=2.78 avg=3.01\n",
      "[500 | 2313.72] loss=2.90 avg=3.01\n",
      "Saving checkpoint/dem/model-500\n",
      "======== SAMPLE 1 ========\n",
      " would be a bad day for my daughter and my colleagues at her Congressional office. The bill was passed by the House that day. It is a bill that Senator GRASS has worked to establish for my constituents, for my family, and to make sure her children remain at home in their schools. When my Senator GRASS took that up, he had the support of thousands of my constituents. He spoke to them directly, in clear voice, and that is what I am most proud of, and I really hope that Senator GRASS understands that. That is where I believe the bill of this House will ultimately come in. It is something the Senator GRASS has done. This is just one example, and that is why I am proud of it, I want to be sure, when this bill comes to the Senate floor for conference, it will be a great accomplishment of the Senate committee that is going to be the lead subcommittee on our Nations childrens education and to make sure our Government is accountable, it will go into effect immediately as legislation to which I have been critical, it will pass by the end of the session of Congress, I have heard my Senator from Pennsylvania say that when we pass legislation because it will lead people to believe that we will have the same kind of leadership, to use the same kind of ideas, I do not know. But I do have to say, it does a wonderful job of holding my heart and making this legislation come to the floor of the Senate with respect to the responsibility of our children.\n",
      "I thank my colleague for yielding me this time. First of all, on the issue of childrens education, it is important that we get this bill into the Senate, but it is important that our States understand this and they are not the ones that are saying that it will not happen. Every child in this Nation needs the help, I am sure they also know, all across America, that they are being asked to do something without their support. What a disgrace. That is why, as chairman of the subcommittee of education. Senator GRASS, has done a perfect job, I thank him for that, I do also thank my colleague from Virginia\n",
      "I thank the chairman for his leadership. Senator GRASS had a wonderful amendment. He was very supportive. It is such a great amendment. I thank him for the amendment, I appreciate his willingness to work with me to see that it is not rushed, but he was very clear when he made the amendment. Our amendment would change the definition of child care, this is a very specific program that he was suggesting. It was to include all of our childrens education, I am glad this amendment was voted down. We are going to have a process in coming up with a resolution, and our bill would be reviewed and approved by us as part of the process. That is the way my amendment would have to go to the Senate. There will be a hearing on my amendment, and hopefully we will work through that process. I appreciate my commitment for this amendment.\n",
      "I commend my colleague and the chairman for his work and his commitment, I think it is important that there should be a hearing on my amendment, I see my amendment is in the committee. If there is, I am hopeful that Senator GRASS will work with us, I encourage my colleagues to support my amendment, I look forward to working with him on a hearing in the next days session under his leadership. With the Senate being the only place where any of my amendments can be heard, I think the chairman and I are in a good position to say this is an amendment we want to work on with an amendment our committee has voted down, which he has.\n",
      "it is important we understand that we are going to allow my amendment to go forward as Senator GRASS has done. It is his commitment to helping the Nations children. What we are doing here today is going to be working to support our Nations children. As Senator GRASS has said, and my colleague, the chairman of this committee, has expressed, his support, that is why I am here, and I believe it is for the children that are at the very highest risk today for this country. We know the risks for them are high, but we know that because we have had the experience of where the Department of Energy has been in terms of having not only a program that has been working, but the most sophisticated program in the world as well, we have worked with both the State Department of energy. And with them, in terms of bringing in new technology, new people to the Department of Energy, I believe that we are going to have to let this continue, I support it for my own State of Rhode Island by a vote of 60, I am very supportive of this amendment, I am very proud that Senator GRASS has also done tremendous work for this Nation in terms of bringing together all of our Nations children in a way that we have seen in our own Nation. They have had such tremendous attention, such a wonderful relationship with our Nations children. When I heard Senator GR\n",
      "\n",
      "[510 | 2384.30] loss=2.94 avg=3.01\n",
      "[520 | 2430.31] loss=2.90 avg=3.00\n",
      "[530 | 2476.37] loss=3.03 avg=3.01\n",
      "[540 | 2522.57] loss=2.81 avg=3.00\n",
      "[550 | 2568.57] loss=2.90 avg=3.00\n",
      "[560 | 2614.70] loss=3.10 avg=3.00\n",
      "[570 | 2660.83] loss=2.97 avg=3.00\n",
      "[580 | 2706.98] loss=3.12 avg=3.00\n",
      "[590 | 2753.03] loss=2.96 avg=3.00\n",
      "[600 | 2799.13] loss=3.04 avg=3.00\n",
      "[610 | 2845.27] loss=3.03 avg=3.00\n",
      "[620 | 2891.41] loss=2.92 avg=3.00\n",
      "[630 | 2937.53] loss=2.80 avg=3.00\n",
      "[640 | 2983.64] loss=2.94 avg=3.00\n",
      "[650 | 3029.82] loss=3.01 avg=3.00\n",
      "[660 | 3075.93] loss=2.89 avg=2.99\n",
      "[670 | 3122.11] loss=3.02 avg=2.99\n",
      "[680 | 3168.29] loss=2.96 avg=2.99\n",
      "[690 | 3214.58] loss=2.97 avg=2.99\n",
      "[700 | 3260.83] loss=2.92 avg=2.99\n",
      "[710 | 3307.07] loss=3.03 avg=2.99\n",
      "[720 | 3353.20] loss=2.97 avg=2.99\n",
      "[730 | 3399.53] loss=2.88 avg=2.99\n",
      "[740 | 3445.53] loss=3.01 avg=2.99\n",
      "[750 | 3491.55] loss=3.02 avg=2.99\n",
      "[760 | 3537.67] loss=2.97 avg=2.99\n",
      "[770 | 3584.05] loss=3.10 avg=2.99\n",
      "[780 | 3630.18] loss=2.91 avg=2.99\n",
      "[790 | 3676.42] loss=2.87 avg=2.99\n",
      "[800 | 3722.51] loss=3.01 avg=2.99\n",
      "[810 | 3768.86] loss=2.92 avg=2.99\n",
      "[820 | 3815.32] loss=2.76 avg=2.98\n",
      "[830 | 3861.50] loss=2.78 avg=2.98\n",
      "[840 | 3907.84] loss=3.07 avg=2.98\n",
      "[850 | 3954.01] loss=2.74 avg=2.98\n",
      "[860 | 4000.41] loss=3.00 avg=2.98\n",
      "[870 | 4046.77] loss=3.07 avg=2.98\n",
      "[880 | 4092.89] loss=2.93 avg=2.98\n",
      "[890 | 4139.20] loss=2.93 avg=2.98\n",
      "[900 | 4185.35] loss=2.91 avg=2.98\n",
      "[910 | 4231.83] loss=2.98 avg=2.98\n",
      "[920 | 4278.13] loss=2.96 avg=2.98\n",
      "[930 | 4324.33] loss=3.03 avg=2.98\n",
      "[940 | 4370.54] loss=2.98 avg=2.98\n",
      "[950 | 4416.72] loss=2.91 avg=2.98\n",
      "[960 | 4463.26] loss=2.88 avg=2.97\n",
      "[970 | 4509.44] loss=2.79 avg=2.97\n",
      "[980 | 4555.73] loss=2.96 avg=2.97\n",
      "[990 | 4601.84] loss=2.92 avg=2.97\n",
      "[1000 | 4648.13] loss=3.01 avg=2.97\n",
      "Saving checkpoint/dem/model-1000\n",
      "WARNING:tensorflow:From /tensorflow-1.15.2/python3.6/tensorflow_core/python/training/saver.py:963: remove_checkpoint (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Use standard file APIs to delete files with this prefix.\n",
      "======== SAMPLE 1 ========\n",
      " relieved by Senator GRASSLEY and the other Democratic Senators who expressed their opposition to the resolution. Indeed the Senate has taken the opportunity of crafting a resolution against Iraq. We have a responsibility to keep that resolution in conference, and to ensure that the full Senate has the opportunity to come together to make sure that we are talking to the President of the United States, both on our side, and the Congress, as well as the President of the United States, to understand what we are talking about, and to make sure, as my good friend and colleagues are talking about, that we have the opportunity, as we are talking about how, we are going to pass this resolution, that we are going to make sure that there is going to be action. I hope we will have a few votes on this resolution, but we also have to remember that we need to focus on what is important, not which Senator is in his or her party in his or her district, but what is important, particularly the concerns with the war in Afghanistan. We all know the frustration of millions of Afghanisolationists, who have not been able, with the support of the international coalition and the CIA, with the international coalition, with the support of our European colleagues who are the allies and allies of the United States, with the support of our European allies, without a doubt, that it is important to have them in that Congress. But, we have not provided the necessary resources to make sure those who are in the position of carrying out the policy of trying to end the war that is waged against us in Afghanistan are not there to have those votes on this, not, they, should be there to get those votes. They should be there where they can make their voices heard. Now, I know there may be Senators who were willing to work on that, but I am not concerned about the Senators on the other side as being willing to support or oppose the resolution. There is another bill we might look at which deals with those important issues that we also have. And we do in both Houses as well with respect to those issues. Now, one of the most striking provisions of the bill is the provision that is included in the final legislation that deals with a provision found in section 303 of the Congressional Foreign Operations Relief Act. In section 302, if the Senate wishes to extend its own assistance, or in order to assist the State Department and other government assistance from the United States, to the Secretary of State to assist that government to meet its obligations in supporting its responsibilities to the Committee on Foreign Relations, the provision states that State Department assistance, as well as any services it may provide to U.S, agencies from the United States, and the Secretary of State, and all other government agencies, must be authorized. But if assistance is provided to other federal agencies, there is no waiver. This includes programs like the Special Immigrant and Dependability Program. The provision in section 304 says that if a Federal agency needs to provide assistance for foreign citizens who are seeking their own protection, the Secretary of State must make the request in writing. Otherwise, in any case, the person can be denied assistance. That is an interesting statement. And again, even though this provision has been in the final legislation I dont have an opinion whether or not I agree with the language which it says this is the language that must be in the final legislation, but I am concerned that I do not understand how this language in the final legislation addresses issues of refugees and refugees of all kinds. For example, in our legislation we seek to prevent the Secretary of State from providing benefits to family members of refugees in the event of hostilities. We can do this at home. We can do it in the context of a domestic emergency situation. But we dont want the Secretary of State and the foreign operations department on the floor. What we would do is address these concerns. It goes on to say that section 302 applies to assistance to family and refugee aid and refugees, I do not understand why they would not provide child care assistance, I think that it is important to recognize, if we are able to provide them the same aid, it is an issue that is not of a priority to the United States, not of a priority to family and humanitarian assistance needs. It seems to me that the Senate ought to be able to provide that level of assistance when those issues are important to us, when they are relevant, if we are able to provide them the same aid as we do for military. We ought to provide their assistance. We ought to make sure that we do that. These are important issues, and I am concerned that in this body, with respect to the issues that are of concern, they are of a concern for us. The one issue which has the greatest importance, with respect to Iraq, was and is of great concern. It has enormous implications for what is the most pressing national security issue in the world, and it has enormous ramifications to our Nation as well as for our country. We all know that we are a member of a world community, and\n",
      "\n",
      "[1010 | 4715.95] loss=2.97 avg=2.97\n",
      "[1020 | 4761.92] loss=2.81 avg=2.97\n",
      "[1030 | 4808.05] loss=2.91 avg=2.97\n",
      "[1040 | 4854.18] loss=2.90 avg=2.97\n",
      "[1050 | 4900.52] loss=2.90 avg=2.97\n",
      "[1060 | 4946.71] loss=2.97 avg=2.97\n",
      "[1070 | 4992.84] loss=3.05 avg=2.97\n",
      "[1080 | 5039.12] loss=2.91 avg=2.97\n",
      "[1090 | 5085.12] loss=2.87 avg=2.96\n",
      "[1100 | 5131.14] loss=3.01 avg=2.97\n",
      "[1110 | 5177.44] loss=2.83 avg=2.96\n",
      "[1120 | 5223.55] loss=2.98 avg=2.96\n",
      "[1130 | 5269.88] loss=2.98 avg=2.96\n",
      "[1140 | 5316.34] loss=3.00 avg=2.96\n",
      "[1150 | 5362.66] loss=2.94 avg=2.96\n",
      "[1160 | 5408.92] loss=3.17 avg=2.97\n",
      "[1170 | 5455.37] loss=2.81 avg=2.96\n",
      "[1180 | 5501.55] loss=2.87 avg=2.96\n",
      "[1190 | 5547.74] loss=2.94 avg=2.96\n",
      "[1200 | 5594.09] loss=2.99 avg=2.96\n",
      "[1210 | 5640.31] loss=2.87 avg=2.96\n",
      "[1220 | 5686.50] loss=2.92 avg=2.96\n",
      "[1230 | 5732.93] loss=2.85 avg=2.96\n",
      "[1240 | 5779.09] loss=3.13 avg=2.96\n",
      "[1250 | 5825.28] loss=2.71 avg=2.96\n",
      "[1260 | 5871.71] loss=2.91 avg=2.96\n",
      "[1270 | 5917.94] loss=2.87 avg=2.96\n",
      "[1280 | 5964.19] loss=2.93 avg=2.96\n",
      "[1290 | 6010.70] loss=2.94 avg=2.96\n",
      "[1300 | 6056.96] loss=3.09 avg=2.96\n",
      "[1310 | 6103.25] loss=2.90 avg=2.96\n",
      "[1320 | 6149.72] loss=2.83 avg=2.96\n",
      "[1330 | 6195.95] loss=2.97 avg=2.96\n",
      "[1340 | 6242.06] loss=2.87 avg=2.95\n",
      "[1350 | 6288.40] loss=2.93 avg=2.95\n",
      "[1360 | 6334.65] loss=2.87 avg=2.95\n",
      "[1370 | 6380.88] loss=2.99 avg=2.95\n",
      "[1380 | 6427.35] loss=2.96 avg=2.95\n",
      "[1390 | 6473.61] loss=2.78 avg=2.95\n",
      "[1400 | 6519.95] loss=2.94 avg=2.95\n",
      "[1410 | 6566.18] loss=2.78 avg=2.95\n",
      "[1420 | 6612.26] loss=3.02 avg=2.95\n",
      "[1430 | 6658.42] loss=2.96 avg=2.95\n",
      "[1440 | 6704.64] loss=2.87 avg=2.95\n",
      "[1450 | 6750.62] loss=2.87 avg=2.95\n",
      "[1460 | 6796.68] loss=2.78 avg=2.95\n",
      "[1470 | 6842.79] loss=2.90 avg=2.95\n",
      "[1480 | 6888.89] loss=2.84 avg=2.94\n",
      "[1490 | 6934.93] loss=2.96 avg=2.94\n",
      "[1500 | 6981.11] loss=2.80 avg=2.94\n",
      "Saving checkpoint/dem/model-1500\n",
      "======== SAMPLE 1 ========\n",
      " appreciate for the gentle suggestion and my respectabilityI am a little disturbed to say this, but the fact is that we will get back in the Senate tomorrow, I want to take a minute to express my concern for those who have had a chance to see the picture tonight of the Senator from New York. That picture is of a small boy. We have all been watching this debate. It is now very close to 7:30, and it should be here. All of my colleagues. Senators. Senators who are here in Washington, but probably not all, but probably the Senators who are here, are here in New York. We have been watching this debate and have been watching this debate for hours. The Senator from New York talked about the Senator from New Jersey. He indicated he was in the room, and I have been in the room. His name is Bob Dole. So Bob Dole has certainly put his foot down in every debate in the Senate. But he has not given up on every debate.\n",
      "I will take a moment to address some of the comments made by the Senator from New York. He indicated he is one that might feel this way to his colleagues who have been here for years, who have been here for years, who can recall any of the things he has said, I understand Bob wants to be here tomorrow, but there is the question of whether he is going to work until he is 60 years old. Some can argue it to me, I do not. He knows all of the facts. He knows that there are going to be a lot of people who look at that picture about 75 years later than 60, I will tell my colleagues. That would be a significant step toward what I support with regard to his legacy. He is well over a decade into his legacy, I thank him for that. There is no indication the Senator from New York is going to try to do another one of those things tomorrow because that is all that is known. But I know Senators come up with the answers that they must, I want the Senator to come down here and talk about the Senator from New York or to the other Senators to make the case that he really is a different person from himself, I might be a little bit disappointed that the Senator from New York is not in this Chamber, I was hoping we could have a debate that was a little closer than what he had, and then perhaps we might even be getting closer to a debate tomorrow, I am not going to make the case that I am in favor of it, I am not for it. But I do not think the time has been spent. That is probably the reason why I said the Senator from New York. Senator DOLE, is so very well informed, and will answer a question that I hope he will hear. But before we can talk about getting the answers to those questions and whether we will have a debate, the Senator from New York said something that perhaps I might not agree with in some of his remarks we are talking about so I will leave it to the Senator from New York.\n",
      "Senator DORGAN is right. He has just answered the question that he was getting. He is not. Senator DORGAN could not believe the Senator from New York or even the Senator from Oregon who has been here forever. There is no question that the Senator from New York and the Senator from New Jersey are going to get a lot closer to a debate tomorrow, I hope the others who have had the chance look at their own pictures to see them, especially I am going to look at their pictures, and they understand what the Senator has said in that question. I want to take a minute, I have the same feelings, I dont think anyone is going to get a lot closer than they have looked at their own pictures to look at the pictures about the Senator from New York.\n",
      "Mr. President. Senator BOXER has described, in his response to the call earlier today, that I am interested in a great many things that are about to be addressed in our annual budget. There are two other points on which my colleagues may be interested at this time, I want to speak in the context of our budget as it relates to the Presidents budget, the first is he has outlined the entire budget to begin with. The second point is that the Presidents surplus is not an automatic assumption and may not even be based on a projection that is available to him. We will get into, as one might expect, the full debate and debate of many other issues that the Senators from New York. Oregon. New Jersey have discussed. They all understand what his proposal is, but it does not mean what it appears in other proposals that are being discussed by Senators from New York. I would point to what he has done, and in that regard I cannot imagine any other Senator would agree with this. It is one thing. It is another thing. It is in the interest of all involved to look at this, and look at the record to make sure we understand what the Presidents plan is and what it does\n",
      "\n",
      "[1510 | 7049.16] loss=2.98 avg=2.94\n",
      "[1520 | 7095.38] loss=2.91 avg=2.94\n",
      "[1530 | 7141.49] loss=2.87 avg=2.94\n",
      "[1540 | 7187.61] loss=2.98 avg=2.94\n",
      "[1550 | 7233.79] loss=2.92 avg=2.94\n",
      "[1560 | 7279.88] loss=2.89 avg=2.94\n",
      "[1570 | 7325.97] loss=2.87 avg=2.94\n",
      "[1580 | 7372.23] loss=2.95 avg=2.94\n",
      "[1590 | 7418.34] loss=2.74 avg=2.94\n",
      "[1600 | 7464.27] loss=2.74 avg=2.94\n",
      "[1610 | 7510.47] loss=2.89 avg=2.94\n",
      "[1620 | 7556.53] loss=2.94 avg=2.94\n",
      "[1630 | 7602.56] loss=2.91 avg=2.93\n",
      "[1640 | 7648.71] loss=2.92 avg=2.93\n",
      "[1650 | 7694.84] loss=2.94 avg=2.93\n",
      "[1660 | 7740.93] loss=2.86 avg=2.93\n",
      "[1670 | 7787.21] loss=2.86 avg=2.93\n",
      "[1680 | 7833.35] loss=2.95 avg=2.93\n",
      "[1690 | 7879.41] loss=2.88 avg=2.93\n",
      "[1700 | 7925.72] loss=3.02 avg=2.93\n",
      "[1710 | 7971.83] loss=2.93 avg=2.93\n",
      "[1720 | 8017.96] loss=3.08 avg=2.94\n",
      "[1730 | 8064.20] loss=2.92 avg=2.94\n",
      "[1740 | 8110.34] loss=2.85 avg=2.93\n",
      "[1750 | 8156.47] loss=2.92 avg=2.93\n",
      "[1760 | 8202.66] loss=2.78 avg=2.93\n",
      "[1770 | 8248.84] loss=2.89 avg=2.93\n",
      "[1780 | 8294.93] loss=2.88 avg=2.93\n",
      "[1790 | 8341.01] loss=2.95 avg=2.93\n",
      "[1800 | 8386.85] loss=2.96 avg=2.93\n",
      "[1810 | 8432.68] loss=2.83 avg=2.93\n",
      "[1820 | 8478.74] loss=2.93 avg=2.93\n",
      "[1830 | 8524.56] loss=2.95 avg=2.93\n",
      "[1840 | 8570.35] loss=3.04 avg=2.93\n",
      "[1850 | 8616.67] loss=2.79 avg=2.93\n",
      "[1860 | 8662.88] loss=2.89 avg=2.93\n",
      "[1870 | 8709.13] loss=2.87 avg=2.93\n",
      "[1880 | 8755.49] loss=2.77 avg=2.93\n",
      "[1890 | 8801.73] loss=2.87 avg=2.93\n",
      "[1900 | 8847.94] loss=2.88 avg=2.93\n",
      "[1910 | 8894.33] loss=2.83 avg=2.92\n",
      "[1920 | 8940.57] loss=2.81 avg=2.92\n",
      "[1930 | 8986.73] loss=2.92 avg=2.92\n",
      "[1940 | 9033.08] loss=2.87 avg=2.92\n",
      "[1950 | 9079.34] loss=2.89 avg=2.92\n",
      "[1960 | 9125.56] loss=2.86 avg=2.92\n",
      "[1970 | 9171.99] loss=2.98 avg=2.92\n",
      "[1980 | 9218.16] loss=2.82 avg=2.92\n",
      "[1990 | 9264.38] loss=2.97 avg=2.92\n",
      "[2000 | 9310.72] loss=2.74 avg=2.92\n",
      "Saving checkpoint/dem/model-2000\n"
     ]
    }
   ],
   "source": [
    "file_name = \"dem_speeches.txt\"\n",
    "\n",
    "sess = gpt2.start_tf_sess()\n",
    "RUN_NAME = 'dem'\n",
    "\n",
    "gpt2.finetune(sess,\n",
    "              dataset=file_name,\n",
    "              model_name='124M',\n",
    "              steps=2000,\n",
    "              restore_from='fresh',\n",
    "              run_name=RUN_NAME,\n",
    "              print_every=10,\n",
    "              sample_every=500,\n",
    "              save_every=500\n",
    "              )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "asXfBQP4YTkp"
   },
   "source": [
    "Make sure you restart runtime before running this cell!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 52
    },
    "colab_type": "code",
    "id": "IhQXixd0YGz3",
    "outputId": "d31a806a-8691-43df-d965-736368489cc9"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loading checkpoint checkpoint/dem/model-2000\n",
      "INFO:tensorflow:Restoring parameters from checkpoint/dem/model-2000\n"
     ]
    }
   ],
   "source": [
    "RUN_NAME = 'dem'\n",
    "sess = gpt2.start_tf_sess()\n",
    "gpt2.load_gpt2(sess, run_name=RUN_NAME)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "TBRBT6weYppS"
   },
   "source": [
    "Generate democrat answers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "wOqdvacCYvCv"
   },
   "outputs": [],
   "source": [
    "questions = [\"Should abortion be illegal?\",\n",
    "             \"What do you believe about tax increases?\",\n",
    "             \"Should same-sex marriage be legal?\",\n",
    "             \"Is climate change real?\",\n",
    "             \"Should immigrants be allowed to obtain citizenship?\",\n",
    "             \"Who should be given voting rights?\",\n",
    "             \"Should we have higher taxes for higher incomes?\",\n",
    "             \"Should we allow death penalty?\",\n",
    "             \"Should we have universal healthcare?\",\n",
    "             \"Do government regulations hinder free market capitalism?\",\n",
    "             \"What do you think about the current president?\",\n",
    "             \"Should we reduce national debt?\",\n",
    "             \"Should we increase spending on healthcare?\",\n",
    "             \"Should we increase spending on education?\",\n",
    "             \"Should every American have equal opportunities regardless of sex, age and race?\",\n",
    "             \"Should be introduce more gun control measures?\",\n",
    "             \"What do you think about the current president?\",\n",
    "             \"Should Americans be free?\",\n",
    "             \"What party do you support?\",\n",
    "             \"What is the biggest threat to America?\"\n",
    "             ]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 555
    },
    "colab_type": "code",
    "id": "WNMG5JpxYxSJ",
    "outputId": "51f46a3f-77e6-4e5f-b7e2-05217e753c85"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Should abortion be illegal? Should we not require a woman to have an abortion if she has been convicted of a crime and the State determines that the judgment of the State is too hard to achieve? I am not a Republican. Democrats, or Republicans here in this House. We are not a party of the death penalty. We are not a party of the death penalty. We are not a party of the transience of circumstances. We are not a party of the death penalty. But we are a party of the death penalty. We are a party of the death penalty. We are a party of the death penalty. It is a crime under the law of the United States to kill a human being, I do not believe that the United States should, under the current law of the United States, require that the people of the United States be deprived of the right to an abortion. The United States should not be deprived of the right to an abortion under the Constitution. It is a crime under the Constitution to kill a human being. Under the current law of the United States, a woman who has an abortion is not required to have an abortion. Under the Constitution, a woman who has an abortion also has the right to a civil abortion. Abortion is not a crime under\n",
      "What do you believe about tax increases? The first is that we are a country of people who are consuming more of the products of the market than they are consuming. That is what we are talking about. That is what we are talking about. That is what we are not talking about. That is what we are talking about. We are talking about a country of people who are suffering from an epidemic of pneumonia. That is what we are talking about. We are talking about a country of people who are suffering from a disease that has spread like a plague to their cities. This is a country of people who are suffering from a disease that is spreading like a plague to their cities. That is what we are talking about. It is a country of people who are suffering from illness that is spreading like a plague to their cities. We are talking about a country of people who are suffering from illness that is spreading like a plague to their cities. That is what we are talking about. That is what we are talking about.\n",
      "I do not believe in this bill, I do not believe in the bill that is before the Senate. It is a shame. We do not want to have this thing adjourned. We do not want to have a debate on tax cuts. We do not\n",
      "Should same-sex marriage be legal? I dont think so. Let me explain. One of the things that is going to be challenged is whether we can have a tax cut for special interests that are not going to be there because we have to make sure that we can provide some help to people who have problems with their marriage. That is what I think is going to be the case. A tax cut for the lowestincome families would do an awful lot to help people who are struggling to make ends meet. Just ask people down in the home town of Boston. Pennsylvania, who are struggling to make ends meet, who are struggling to make ends meet with their kids because they are struggling to make ends meet. What we are going to say is if we provide some help to those people who are struggling to make ends meet, if we can help them make ends meet, then we are going to get a tax cut for the lowestincome families, I dont think that is the way we ought to be doing this. The way we ought to be doing this is by extending tax breaks for the lowincome families in this country. This is not the way to do it. It is not the way to do it. It is not the way to do it. If we dont get a tax\n",
      "Is climate change real? In light of the recent scientific breakthroughs, scientists predict that there may be a new type of life on the Earth that could become the \"prebiotic\" species that we know exists today. The new species of life may have been discovered in the Colonies, a planet that has a great deal of diversity and diversity of life. The new species will be able to survive in the atmosphere, and that diversity will have implications for the existence of life on other worlds, I am optimistic that the new species will be able to survive on other worlds. We are talking about the prebiotic species, the new species that we know exists on the surface of the Earth. They are the prebiotic species, and we are talking about a new type of life. They are the prebiotic species that will be able to survive on other worlds. There is one way that we can do this. We have to do something about it. We have to do something about it. It is a serious thought. In the long run, we need to do something about it. We need to do something about it. We need to do something about it with the new species.\n",
      "I thank the gentleman for yielding me this time. I rise in support\n",
      "Should immigrants be allowed to obtain citizenship? The answer is yes. Let us not allow immigration into the United States and create an immigration system that will make people who are here legally but are not here permanently, who might be subject to deportation, in this country, who will not have an opportunity to obtain a citizenship, who will not have the ability to get a job, who will not have the ability to have a health care coverage, who will not have the ability to obtain permanent residency in this country. Let us not allow this to happen. It is wrong. It is wrong to put people in a situation where they will not have a chance to obtain citizenship. Let us not put people in an immigration system that will not provide citizenship to them. Let us not allow these people to be deported without having a chance to obtain a job.\n",
      "I rise in support of H.R, 6. This bill will provide increased immigration benefits to lawful permanent residents who have been granted permanent residency by the Secretary of the Department of Health and Human Services. This bill is a good bill. It was passed by the Senate in a bipartisan fashion and was signed by the President last November. It has been reported by the House and signed by the President. H.R, 6 is a good bill.\n",
      "Who should be given voting rights? No. The Supreme Court of the United States has rejected that proposition, and the courts have ruled against it. Even worse, the Supreme Court has failed to provide a single constitutional test to determine whether a Federal statute is unconstitutional. The Supreme Court has found that the American people do not have the power to punish those who violate their Constitutional rights. The Supreme Court has ruled that a Federal statute is unconstitutional. The Supreme Court has found that a Federal statute is unconstitutional because of the congressional intent to protect the Constitution. The Supreme Court has ruled that a Federal statute is unconstitutional because of the congressional intent to protect the Constitution. The Supreme Court has concluded that a Federal statute is unconstitutional because Congress created the Department of Dont. Know, and Know Who and who is covered by the Constitution? The Supreme Court has ruled that a Federal statute is unconstitutional because of the congressional intent to protect the Constitution. The Supreme Court has determined that a Federal statute is unconstitutional because of the Congress intent to protect the Constitution. Both the Supreme Court and the Supreme Court have ruled that a Federal statute is unconstitutional because of statutory authority. The Supreme Court has determined that a Federal statute is unconstitutional because of statutory authority. The Supreme Court has determined that a Federal statute is unconstitutional because of a statutory authority\n",
      "Should we have higher taxes for higher incomes? We are not going to have a Medicare prescription drug benefit for seniors. We are not going to have a prescription drug benefit for seniors. We are not going to have a prescription drug benefit for seniors. We are not going to have a Medicare prescription drug benefit for seniors. We are not going to have a Medicare prescription drug benefit for seniors. We are not going to have a prescription drug benefit for seniors. And we are not going to have a prescription drug benefit for seniors. We are not going to have a prescription drug benefit for seniors. We are not going to have a Medicare prescription drug benefit for seniors. And we are not going to have a prescription drug benefit for seniors. The President has repeatedly said we should not have a prescription drug benefit for seniors. We should not have a Medicare prescription drug benefit for seniors. We should not have a prescription drug benefit for seniors, I am concerned about what is going on. The President has repeatedly said we should not have a prescription drug benefit for seniors. We should not have a prescription drug benefit for seniors. And we are not going to have a prescription drug benefit for seniors. The President has repeatedly said we should not have a prescription drug benefit for seniors. We should not have a prescription drug benefit for\n",
      "Should we allow death penalty? That is what we are asking. We are asking that this country and the United States Congress, as we come into this period of time when we are debating these amendments, we do not allow this country to be placed in a position where it is not going to have an opportunity to get a fair and just trial process. So this country and this Congress, as we come into this period of time, we are going to have to have a discussion about whether we should, as we come into this period of time, allow this country to be placed in this position where we are not going to have an opportunity to get trials. To permit this country to get trials is not only excessive, it is outrageous. There is no way of correcting this case without correcting the specific circumstance that was fixed in this case by the Supreme Court. Now, as we have come into this period of time, we are going to have to have a debate about whether or not we should be allowed to get the death penalty in this case. What we are asking is that we allow the United States Congress, as we come into this period of time, to be able to get a trial where there is no chance of an opportunity of an opportunity of an opportunity of an opportunity of\n",
      "Should we have universal healthcare? That is the only question. We are asking our friends in the rest of the world to come and see what is best for their health care systems. We are asking them to make the decisions they do not want to make. We are asking them to make the decision that will save lives and our health care system. We are asking them to make the decision. So we have an obligation to do that. We do not want to take away the right to choose. We are asking them to do that, and we are asking them not to make the decisions that they do not want to make, and we are asking them to make the decisions. So that is the choice. We have an obligation to take that decision, we have an obligation to make that decision. We are asking them to make the decision. We are asking them to make the decision. We are asking them to make the decision. They are not responding to the people who have the responsibility for this, and we are asking them to make the decision. We are asking them to make the decision. So I say to my friend from Wisconsin, he and I are not saying to the American people, we are not willing to do that. We do not want to do that. We\n",
      "Do government regulations hinder free market capitalism? Are they really that burdensome? The answer is no. Under the current law, the Supreme Court has invalidated the law. The Court has invalidated the law because the regulations were not subject to a Federal regulation. The regulations are subject to the Federal government, and the regulations are not subject to a Federal regulation. The Federal regulation was subject to the Federal regulation. Under the current law. Federal regulations are not subject to a Federal regulation. The regulations are subjected to the Federal government. The Federal regulation is subject to the Federal regulations. Under a Federal statute, the regulations are subject to the Federal regulation. Under a Federal statute, the regulations are subject to the Federal regulations. Under a Federal statute, the regulations are subject to the Federal regulation. Under a Federal statute, the regulations are subject to the Federal regulations. It is important that we make sure the Federal government has the authority to regulate the free market at the level it deserves. We can and should do that. The current law does not do that. There are certain regulations that are subject to the Federal regulation. The current law does not remove that regulatory burden. It is not an imposition of a Federal regulation. There are certain regulations that are subject to the Federal regulation. Under the current\n",
      "What do you think about the current president? When did the current president become the President of the United States? I would hope that in the next year or two, after that, we will come to a decision about whether we will be doing that. There is no question about it, and I would hope that we will continue to work with the Congress to ensure that we are going to be successful in the next few years. We have been working with the President of the United States to try to make certain that we are going to have a constructive relationship and not just a partisan problem, but that we are going to have an effective and fair process in terms of the accountability in the United States that we have been holding back with regard to the Presidents agenda. It was very clear during the campaign that our agenda is not going to be a bipartisan one. We are not going to be able to have a constructive debate on the size of the budget that we are going to have, the number of people we are going to have, or the number of people we are going to have who are going to be serving on the U.S. Congress. We are going to have an effective process in terms of how we are going to deal with the issue of longterm debt and more debt and more debt\n",
      "Should we reduce national debt? Of course we do. We do it with a tax cut. We do it with a tax cut that is necessary to pay for this countrys debt. Then we have a tax cut that is not necessary to pay for national debt. That is what we are doing. Now, the Republicans have gone on record on the floor of the House of Representatives saying, no. The American people do not want the one percent of the national debt to be wiped out. That is what this is all about. The American people do not want that one percent of the national debt wiped out. They want a tax cut that is necessary to pay for our national debt. They do not want a tax cut that is necessary to pay for our national debt. They want a tax cut that is necessary to pay for our national debt. Now, if the Republicans want to cut taxes, they can come out and spend that tax cut, that is what they are doing. What do they want? They want to spend it on debt reduction. They want to spend it on debt reduction. They want to spend it on debt reduction. They want to spend it on debt reduction because they want to spend it on debt reduction. It has nothing to do with debt reduction. It\n",
      "Should we increase spending on healthcare? Of course. But we need to do it. The American people need to understand that we do not have a blank check. We do not have a blank check to do what we should do to end this trend toward spending on healthcare.\n",
      "I thank the gentleman for yielding me this time. I rise in support of the Senatepassed H.R, 3, the Medicare Prescription Drug Coverage Act. As a member of the Committee on Ways and Means, I recognize strongly that the Congress needs to act on comprehensive reform of Medicare, I support this bill, I believe it is a winwin situation for the Medicare beneficiaries, I also support the Housepassed H.R, 3, I would hope that the House would adopt this legislation as it comes to the floor of the U.S. Senate, and the House should adopt this legislation as it comes to the floor of the U.S. Senate.\n",
      "I thank the gentleman from Texas for yielding me this time. I am pleased to join the gentleman from Michigan in bringing this legislation to the attention of the Congress. There is a new Medicare program that will provide special care to senior citizens. We need to change Medicare. We need to change it. The problem is that this bill will\n",
      "Should we increase spending on education? We will hear no answer to that, I have no idea whether or not we will have a full debate on this issue. We will have a limited debate on this issue. We should have an opportunity to debate it. We should have an opportunity to debate the issues on which we will have an opportunity to debate this legislation.\n",
      "I rise in opposition to this amendment. First of all, I am troubled by the resolution that was adopted. It is an amendment that I think undermines the responsibility of this House to provide the resources to our young people to be able to have a responsible, independent, and responsible, and responsible, and responsible, and responsible, to use that money to improve our schools and to provide schools with a space for the best teachers to teach in, to bring all our children to the best schools. This amendment would devastate this effort. If we are serious about improving our education, we need to do it now. If we are serious about improving our schools, we need to do it now, I urge my colleagues on both sides of the aisle to support this amendment, I urge my colleagues on the Republican side of the aisle to support this amendment.\n",
      "I rise in support of this amendment, I am disappointed that the Republican\n",
      "Should every American have equal opportunities regardless of sex, age and race? It is the right thing to do, and we should take it. But I do not think that is the way to do it. We have a lot of work to do. If we pass this legislation, we will have a lot more people in this country who are going to be able to work and have the kind of economic opportunity that we need to do. This is work in progress. Yet we have to bring it to a vote. We have to make sure that some of the children in America have a chance. We have to make sure that they are going to have the opportunity. We have to make sure that there are a lot of people who are working in this country who are working hard to make sure that they can work and have the kind of opportunity that we need. Let me give you one example, I am going to introduce legislation today that will give some credit to the Hispanic American community for having a better health care system. We have Hispanics working in our country and having a better health care system. We have hospitals and doctors and nurses, all of our constituents, who are working with us. We have a lot more to do. But we have to do something to give some credit to them. Let me give you an\n",
      "Should be introduce more gun control measures? There is a chance that we could be in the position of having a second term in this country when we are considering the next administration, I urge our colleagues to support the amendment we have offered and engage in a dialogue with the American people. The American people want us to do that.\n",
      "I thank the gentlewoman for yielding me the time, I rise in support of this amendment offered by the gentleman from California . It is supported by the chairman of the committee. It is supported by the staff of the Committee on the Judiciary. We have a lot of support for this amendment. We have it. We have a lot of support for it. We have a lot of support for it. There is a lot of support for it. There is a lot of support for it, and it is supported by the ranking member of the committee. The gentleman from California is right. It is supported by the Committee on the Judiciary. It is supported by the Committee on the Judiciary, the Committee on the Judiciary, the Committee on the Judiciary, the Committee on the Judiciary. It is supported by the gentleman from California and the gentleman from Massachusetts in every sense of the word. When we have the opportunity tonight to discuss these issues, we should be willing to consider\n",
      "What do you think about the current president? Do you think it is good or bad? Do you think it is going to do this or that? Do you think it is going to be something that makes us more focused on being good or bad? Do you think it is going to be something that makes us more focused on the ability to make the right choices for the people we serve? Do you think it is going to make us more focused on this issue? Do you think we are going to be putting our family in harms way, not just our country? I think we ought to weigh the pros and cons of this bill. Let me give a couple of examples. There are some who say that funding for the alternative minimum tax will benefit the rich. But in my opinion, we will have a lot of people who are not able to make ends meet. We need to provide some help to the poor. We need to help the people who are in the middle class. We need to provide some help to the poorest people. This bill is one that will benefit the wealthiest families. There are some who say that funding for the alternative minimum tax is going to benefit the rich. But I disagree with that. It is the exact opposite. The benefit is going to be the top 1 percent\n",
      "Should Americans be free? Yes. No. The United States is a country of unique sovereignty and those who claim asylum in the United States are entitled to the same protections that we have for our citizens. It is time to change our system of immigration, I urge my colleagues to support this resolution of disapproval.\n",
      "I rise in support of the motion to recommit, and I look forward to the opportunity to debate this issue as we proceed on the floor today. The United States, as a country, is a great country and a great country in which to live. The United States is the greatest democracy in the world, and the United States possesses the most sophisticated military in the world. It is the most advanced democracy in the world. The United States is the most advanced democracy in the world, and the United States is the best country in the world not because the United States is the only democracy in the world. We are not just a country, we are a democracy. The United States is a democracy.\n",
      "I rise in support of this amendment, but it provides for the detention of detainees in the United States, I would like to do so again, I am concerned about the fact that the United States is not the only democracy in the world. The United States has\n",
      "What party do you support? Do you want a clean water bill? Do you want to get rid of the waste? Do you want to get rid of the cigarettes? Do you want to get rid of the gun? Do you want to get rid of the tobacco? Do you want to get rid of the asbestos? Do you want to get rid of the pharmaceuticals? Do you want to get rid of the chemicaltrafficking? Do you want to get rid of the language that says we have to clean up our act? Do you want to get rid of the language that says we are going to try to streamline this process? Do you want to get rid of the language that says we are going to try to eliminate the dumping of chemicals? Do you want to get rid of the language that says we are going to try to clean up our act? Do you want to get rid of the language that says we are going to try to eliminate the dumping of chemicals? Do you want to get rid of the language that says we are going to try to eliminate the dumping of chemicals? Do you want to get rid of the language that says we are going to try to eliminate the dumping of chemicals? Do you want to get rid of the language that says we are\n",
      "What is the biggest threat to America? The biggest threat to the United States is the proliferation of a nuclear arsenal. All of us know that if we do not take the steps necessary to protect our citizens, we will be exposed to nuclear weapons. But we also know that there are other nuclear threats that are more fundamental to this country and that are more likely to cause us to start off a nuclear war. When we do not act to protect our citizens, we will be exposed to nuclear weapons. The threat of nuclear weapons is a very real and real threat to our national security. Our national defense is not just about our national security. It is about our national security and the security of our country. More than anything else, we are a nation of security. We are a nation of liberty. We are a nation of opportunity. We are a nation of opportunity. We are a nation of opportunity. We are a nation of opportunity, I believe that we have a very serious national security threat, I do not think that we can afford to wait for this Congress, and we have to act quickly, to provide the resources that we need to do that. We have to act quickly, and we have to act quickly. This bill is absolutely critical to our national security and national security interests.\n"
     ]
    }
   ],
   "source": [
    "with open(\"dem_generated.txt\", 'w') as f:\n",
    "  for i in range(len(questions)):\n",
    "    text = gpt2.generate(sess,\n",
    "              length=250,\n",
    "              temperature=0.7,\n",
    "              prefix=questions[i],\n",
    "              nsamples=1,\n",
    "              batch_size=1,\n",
    "              return_as_list=True)[0]\n",
    "    print(text)\n",
    "    f.write(text + '\\n')"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "include_colab_link": true,
   "name": "06b_GPT2_finetuning.ipynb",
   "provenance": [],
   "toc_visible": true
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}