{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<small><i>This notebook is a reviewed English version (2020) of \"*03.1b-Ley-de-Luhn.ipynb*\" on the collection [nlp_pydata2018 on GitHub](https://github.com/sorice/nlp_pydata2018/).</i></small>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Advanced Preprocessing: Luhn Eg.\n",
    "\n",
    "This section is an advanced section on NLP and its basic processes. To \n",
    "reach the goal of this notebook you must know how to filter the stopwords \n",
    "which are words without semantic content and the most frequent ones, also \n",
    "how to steamm word suffixes. In any of those cases, statistics are essential \n",
    "for the algorithm or for the learning mechanism to be used, and in every \n",
    "step we can see how the number of words is reduced trying to find the most \n",
    "important ones.\n",
    "\n",
    "Let's now look at an example of NLP very close related to the field of \n",
    "_Information Retrieval_. It is all about **Luhn's Law** [2], or the problem \n",
    "of finding the boundaries of the most important words in a document, usually \n",
    "for long texts like books."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Roadmap Summary\n",
    "\n",
    "###  Datasets\n",
    "\n",
    "For all the examples, two books will be used, one in English and the \n",
    "other in Spanish. One if the translation of the other, and both are under\n",
    "Creative Common license. The first is _Free Culture_ and the second is its\n",
    "Spanish translation _Cultura Libre_.\n",
    "\n",
    "### Transforming Dataset Texts\n",
    "\n",
    "Generally, almost all the materials that we have are PDFs and to operate \n",
    "with texts in python it is best to use simple text formats like .txt. Then,\n",
    "how to transform PDF into TXT?\n",
    "\n",
    "Our recommendation is to use pypdf2 that appears in the GNU / Linux \n",
    "repositories and it is a pure python library \n",
    "(see [Text Extraction with pdfMiner](https://github.com/sorice/nlp_pydata2018/blob/master/02.1-Text-Extraction-with-PDFMiner.ipynb)). \n",
    "If you know any script suitable for this task using the ghostscript \n",
    "library, we recommend it over pdftotext. However it is quite difficult \n",
    "to find such a custom script for our needs.\n",
    "\n",
    "The _text-preproc_ library since v0.2 includes a helper function that \n",
    "wraps the pypdf2 library to extract the text.\n",
    "\n",
    "\n",
    "```python\n",
    "from preprocess.utils.io import pdf\n",
    "PDF = pdf('/path/to/pdf/file.pdf')\n",
    "text = PDF.extractText() \n",
    "with open('/path/for/new/file.txt','w') as doc:\n",
    "    doc.write(text)\n",
    "```\n",
    "\n",
    "If you are using preprocess v0.1 use _pdftotext_ included in the poppler-utils package.\n",
    "\n",
    "```bash\n",
    "~$ pdftotext file.pdf\n",
    "```\n",
    "\n",
    "**Result:** file.txt\n",
    "\n",
    "Keep in mind that text extraction scripts generate .txt files with many \n",
    "problems: mainly rare characters. Perhaps it will be more useful for \n",
    "the reader to study more specialized applications and libraries for \n",
    "this type of word processing problem such as: Apache-Tika, among others. \n",
    "However, in the following sections _Transforming text_ some tips can be\n",
    "found out about how to solve these problems without using large libraries to \n",
    "obtain good result in basic NLP subprocess."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Transforming Text\n",
    "\n",
    "The next codes transform text to allow the application of statistical analysis to build Luhn model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import preprocess"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from preprocess.data import load_freeculture\n",
    "from preprocess.demo import preProcessFlow\n",
    "text = load_freeculture()\n",
    "text = preProcessFlow(text).lower()\n",
    "text = preprocess.del_digits(text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total words:  120303\n"
     ]
    }
   ],
   "source": [
    "from nltk.tokenize import RegexpTokenizer\n",
    "tokenizer = RegexpTokenizer(\"\\s+\", gaps=True)\n",
    "tokens = tokenizer.tokenize(text)\n",
    "print (\"Total words: \", len(tokens))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "En ambos casos se devuelve el total de palabras(o tokens) divididos por el caracter\n",
    "*espacio*. Sin embargo las palabras en un texto se repiten. ¿Cómo saber las palabras\n",
    "únicas?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Unique words: 8233\n"
     ]
    }
   ],
   "source": [
    "tokens_unique=set([])\n",
    "tokens_unique = set(tokens)\n",
    "print (\"Unique words:\", len(tokens_unique))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As it can be seen in the book of Bird et al.[[1](#Bird2009)] the simplest \n",
    "question we can make when we see these numbers are: Which is the average\n",
    "of words by page? Which is the must used or frequent word? What words\n",
    "are used only one time? Lets see some ways to calculate that, to get those\n",
    "goals will be needed some extra functions.\n",
    "\n",
    "<a id='sect2.5'></a>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Start a dict to keep the frequence of every word\n",
    "dict = {}\n",
    "for word in tokens_unique:\n",
    "    dict[word]=0\n",
    "#Dict with word = frequence of word.\n",
    "for token in tokens:\n",
    "    dict[token]+=1\n",
    "#A list of tuples with more info can result in a better resource. List([(Frequence1, word1),(freq2,word2),...])\n",
    "tupla = []\n",
    "for word in dict:\n",
    "    tupla.append([dict[word],word])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This tuple can be ordered like this: because the frequence is in the first\n",
    "position, the list tupla can be sortered using *tupla[i][0]*."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The 10 must frequent words are:\n",
      ". : 9178\n",
      "the : 6909\n",
      "of : 3479\n",
      "to : 3180\n",
      "a : 2408\n",
      "is : 2198\n",
      "that : 2052\n",
      "and : 1922\n",
      "in : 1718\n",
      "it : 1276\n"
     ]
    }
   ],
   "source": [
    "tupla=sorted(tupla)\n",
    "print (\"The 10 must frequent words are:\")\n",
    "for i in range(1,11):\n",
    "    print (tupla[-i][1],\":\",tupla[-i][0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As can be seen the words are non useful words, or stopwords or the puntuaction sign."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'some', 'their', 'again', 'itself', 'where', 'pages', 'should', 'while', 'so', 'into', 'me', 'any', 'same', 'with', 'did', 'herself', 'which', 'why', 'ours', 'just', 'were', 'through', 'each', 'being', 'by', 'our', 'its', 'than', 'we', 'it', 'after', 'between', 'would', 'off', 'could', 'don', 'an', 'both', 'over', 'to', 'under', 'more', 'not', 'once', 'having', 'what', 'before', 'one', 'do', 'yourself', 'out', 'this', 'about', 'my', 'his', 'yourselves', 'does', 'because', 'against', 'myself', 'they', 'up', 'here', '_r', 'for', 'nor', 'most', 'jm_qxd', 'yours', 'doing', 'if', 'those', 'all', 'pass', 'when', 'that', 'who', 'of', 'are', 'above', 'them', 'ourselves', 'during', 'be', 'on', 'down', 'themselves', 'only', 'and', 'can', 'these', 'theirs', 'such', 'how', 'but', 'in', 'have', 'your', 'very', 'no', 'hers', 'she', 'whom', 'further', 'i', 's', 'is', 'below', 't', 'the', 'until', 'am', 'few', 'now', 'her', 'will', 'too', 'a', 'from', 'has', 'himself', 'he', 'own', 'you', 'been', 'other', 'him', 'had', 'at', 'page', 'was', 'there', 'pm', '❚❘', 'as', 'then', 'or', 'th'}\n"
     ]
    }
   ],
   "source": [
    "import time\n",
    "from nltk.corpus import stopwords\n",
    "timei = time.time()\n",
    "english_stops = set(stopwords.words('en'))\n",
    "#This updates are agregated after some experimental observation, because\n",
    "#normalization process required more analysis time \n",
    "english_stops.update(['would','pm','pages','page','th', '_r', 'could','pass','jm_qxd','❚❘','one'])\n",
    "print(english_stops)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopword filtering time:  2.2512850761413574\n",
      "Uunique words without stopw: 8071\n",
      "The 10 must frequent words after stopwords filtering are:\n",
      "copyright : 720\n",
      "law : 615\n",
      "free : 441\n",
      "culture : 374\n",
      "property : 356\n",
      "content : 281\n",
      "right : 246\n",
      "work : 242\n",
      "use : 239\n",
      "internet : 238\n",
      "Total words:  120303\n",
      "Total text words without stopwords: 55482\n",
      "Deleted words after stopwords filtering: 64821\n"
     ]
    }
   ],
   "source": [
    "tokens_afterstops=[]\n",
    "for k in range(len(tokens)-1):\n",
    "    if tokens[k] not in english_stops and len(tokens[k])>1:\n",
    "        tokens_afterstops.append(tokens[k])\n",
    "timef = time.time()-timei\n",
    "print (\"Stopword filtering time: \",timef)\n",
    "\n",
    "tokens_unique1 = set(tokens_afterstops)\n",
    "dict1 = {} #dict con keys = set de tokens after stops\n",
    "for word in tokens_unique1:\n",
    "    dict1[word]=0\n",
    "\n",
    "tupla1 = [] #Creating list of tuples (frequence,word) without stopwords\n",
    "for token in tokens_afterstops:\n",
    "    dict1[token]+=1\n",
    "for word in dict1:\n",
    "    tupla1.append([dict1[word],word])\n",
    "tupla1=sorted(tupla1)\n",
    "\n",
    "print (\"Uunique words without stopw:\", len(tokens_unique1))\n",
    "print (\"The 10 must frequent words after stopwords filtering are:\")\n",
    "for i in range(1,11):\n",
    "    print (tupla1[-i][1],\":\",tupla1[-i][0])\n",
    "print (\"Total words: \", len(tokens))\n",
    "print (\"Total text words without stopwords:\", len(tokens_afterstops))\n",
    "print (\"Deleted words after stopwords filtering:\", \n",
    "len(tokens)-len(tokens_afterstops))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Luhn Model in Linguistics\n",
    "\n",
    "Luhn argues that long texts have a logaritmic function behavior or a long\n",
    "tal function. Where the stopwords are at the beginning and the less \n",
    "frequent words are in the long tail. Lets see a graph of the book Free \n",
    "Culture. Using the variable _tupla_  the [2.5](#sect2.5) section code \n",
    "a _numpy_ array is built and the media($\\mu$) and the variance($\\sigma$) \n",
    "are extrated to be used in future calculations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 147,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Populating the interactive namespace from numpy and matplotlib\n"
     ]
    }
   ],
   "source": [
    "%pylab inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "14.612291995627354\n",
      "153.12755040987275\n",
      "(8233,)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEICAYAAACzliQjAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAAAm3UlEQVR4nO3deXxddZ3/8dcnN8nN2iRN0i3pSkuxbC0WRARlcWTVogOIMj+rw4jO6Kg/FcVxZnAZ13Hlhw8dBlDcAAUVlCIii7tACy0UCrR2oXvSJU2TNPvn98f5ptymSZNmufcm5/18PPLIueece87nnpu8z/d+z7nnmLsjIiLxkJPpAkREJH0U+iIiMaLQFxGJEYW+iEiMKPRFRGJEoS8iEiMK/XHEzDaa2evTuL5/M7ObR3B5TWY2Jwx/z8z+awSX/R0z+4+RWl46mNmjZvZPma5jtIz0eyyDo9DPQukO735qeNTMWs1sv5k1mtkKM7vOzJI987j75919wFAabHi5e4m7rx+B2t9pZn/stez3uvtnh7vsPtb1KTPrCDusJjNbY2Z/P9LrGUQdG83sQEodTWY2LQ3rPc3MlplZg5ntMbPHzexdQ1jO2Wa2ZTRqlEMp9OVI3u/upcBU4CPAlcAyM7ORXImZ5Y7k8jLgzrDDKgE+BPzQzCZnoI439tQRfralThzp7WxmrwYeBn4HzAUqgX8GLhzJ9QyylrH+N5Q2Cv0xpPfH4X5aRwvN7Gkz22dmd5pZQeq8ZvYRM6szs+2DbZG5e7O7Pwq8CXg1cHFY5qfM7IdhuMDMfmhmu0Or7wkzm2xmnwPOAm4Mrc8bw/xuZu8zs7XA2pRxc1NWXWVmD4ZPG78zs5lhvllh3oP/6D2fJszsFcB3gFeH9TX0s+3ebWbrQuv03tRWcVj2e81sbXgt3xrsjs7dHwD2A8cMcl1nhG21L/w+o6/lmtnU8L5eO5g6er2W3tv5EjNbGV7bn83spJT5p5nZ3WZWb2YbzOwDR1j8fwO3ufuX3H2XR1a4+xVhWYd94urjPcbMioH7gWmpn1IG+nsPn24+bmZPA81mlmtmp4fX1GBmq8zs7KPZXnGg0B9/rgAuAGYDJwHvTJk2BSgDaoCrgW+ZWcVgF+zuLwHLiUK8t6Vh2dOJWnzvBQ64+yeBPxB9aihx9/enPOdS4FXAgn5WeRXwWaAKWAn8aBA1rgnr/ktYX3nveczsXOALRNtqKrAJuKPXbJcApxJtwyuA8wdat0UuBvKB5wZal5lNBO4DbiDaZl8D7jOzyl7LnU3Umr7R3f97oDr6cClhO5vZIuBW4D1hnf8D3GtmSTPLAX4JrCL6GzkP+JCZHfbazayIqAFw1xDqOYS7NxN9OtjW36eUI3gbUSOkHJhMtD3/C5gIfBS428yqh1vjeKLQH39ucPdt7r6H6B94Ycq0DuAz7t7h7suAJmD+US5/G9E/VG8dRCEy1927QouvcYBlfcHd97j7gX6m3+fuv3f3NuCTRK336UdZb1+uAm519yfDsj8Rlj0rZZ4vuntD2NE9wqHbsbcrwieKJuBe4PPu3jCIdV0MrHX3H7h7p7vfDjwPvDFl2QvC+q9395sGeF2/CC3cBjP7Rcr41O18DfA/7v5YeJ9uA9qA04l2ctXu/hl3bw/HV/6XqFuvtwqi/Ng+QE2j7QZ33xxe2z8Ay9x9mbt3u/uDRI2UizJbYnZR6I8/O1KGW4CSlMe73b3zCNMHowbY08f4HwAPAHeY2TYz+7KZ5Q2wrM2Dne7uTWG9I3FwchpRizt12buJXluPI23H3n7i7uXuXkzUrfMOM3vPINZ1yLRgU686rgK2MrgW9aWhjnJ3vzRlfOp2ngl8JGXn0ED06WxamDat17R/I2pB97YX6Cb69JJJvV/b5b3qP5PM15hVFPpjSzNQlPJ4SjpXHlrZryTqrjlE+PTwaXdfAJxB1D3yjp7J/SxyoEu8HmzVm1kJ0SeMbUTbAfrfFgMtdxtRQPQsu5joU8rWAZ43IHffSNQ/3dNaP9K6DpkWzOhVx6eAXcCPzSwx1LJShjcDn0vZOZS7e1H4lLEZ2NBrWqm7H9ZSdvcW4C/Akc5UOuTv1cyO9Pfa13s2mL/33q/tB73qL3b3Lx5hvbGj0M9eeRYdHO35ySXq177IzCaGf6APpaMQMysys9cB9wCPA8v6mOccMzsxBFMjUXdPd5i8E5gzhFVfZGZnmlk+Ud/+X8NH+XqiYPwHM0uY2T+ScuA0rK82PK8vtwPvMrOFFp2C+nngsRDYw2JmtUTHVJ4dxLqWAcea2dvDQci3EnXn/CplkR3A5UAx8P3Q7z4c/wu818xeFY5BFJvZxWZWSvTe7g8HRwvDtj3BzE7tZ1kfA95pZtf2HIcws5PNrOf4yCrg+PDaC4h2YP3ZCVSaWVnKuJUc3d/7D4E3mtn5ofaCcPC3doDnxYpCP3stAw6k/HyKqAtlFbAR+A1w5yjXcKOZ7Sf6h/wGcDdwgbt39zHvFKIuiEZgDdGBxx+Ead8ELjOzvWZ2w1Gs/8fA9UTdOq8k6rPt8W7gWqKukuOBP6dMe5godHeY2a7eC3X33wL/EV7PdqIdRl/91oP11p6zToAngD8Bnx5oXe6+m+gT0UfC6/gYcIm7H1Kzu7cDbyHqZrl1OMHv7suJtt2NRF006wgH+929K9SzENhA9AnjZqID9H0t68/AueFnvZntAW4iNArc/UXgM8Bvic4c+mNfywnzPk+0g1wfumamcZR/7+6+GVhC1CVVT9Tyvxbl3CFMN1EREYkP7QFFRGJEoS8iEiMKfRGRGFHoi4jESFZfpKiqqspnzZqV6TJERMaUFStW7HL3Pi8/kdWhP2vWLJYvX57pMkRExhQz6/1N74PUvSMiEiMKfRGRGFHoi4jEiEJfRCRGFPoiIjGi0BcRiRGFvohIjIzL0N/WcICv/eYFNuxqHnhmEZEYGZehv6e5nRseXse6uqZMlyIiklXGZegX5Ud3lWtu6xxgThGReBmXoV+SjK4u0aTQFxE5xLgM/eIQ+mrpi4gcalyGflF+AjOFvohIb+My9M2M4vxcmtq6Ml2KiEhWGZehD1CcTKilLyLSyzgO/Vya2hX6IiKpxm3olyRz1dIXEell3IZ+cb5CX0Skt/Eb+kkdyBUR6W3chn6JDuSKiBxm3IZ+kfr0RUQOM25DvySZq8swiIj0Mm5Dvzg/l7bObjq7ujNdiohI1hi3oV9SoIuuiYj0Nm5Df/KEJADb97VmuBIRkewxYOib2a1mVmdmq1PGTTSzB81sbfhdEcabmd1gZuvM7GkzOyXlOUvD/GvNbOnovJyXTa8oAmDznpbRXpWIyJgxmJb+94ALeo27DnjI3ecBD4XHABcC88LPNcC3IdpJANcDrwJOA67v2VGMlukTQ+jvPTCaqxERGVMGDH13/z2wp9foJcBtYfg24NKU8d/3yF+BcjObCpwPPOjue9x9L/Agh+9IRlRFUR7F+Qm19EVEUgy1T3+yu28PwzuAyWG4BticMt+WMK6/8Ycxs2vMbLmZLa+vrx9iedHlladPLGLLXoW+iEiPYR/IdXcHfARq6VneTe6+2N0XV1dXD2tZtRVFbN6j7h0RkR5DDf2doduG8LsujN8KTE+ZrzaM62/8qJpSlqS+qW20VyMiMmYMNfTvBXrOwFkK3JMy/h3hLJ7TgX2hG+gB4A1mVhEO4L4hjBtVlcVJ9ra06wtaIiJB7kAzmNntwNlAlZltIToL54vAT8zsamATcEWYfRlwEbAOaAHeBeDue8zss8ATYb7PuHvvg8MjrqokH3fY09LOpNKC0V6diEjWGzD03f1t/Uw6r495HXhfP8u5Fbj1qKobpsqS6Atau5sU+iIiMI6/kQtQWZwPRKEvIiLjPPSrSkNLv1kHc0VEYLyHfnEU+rvU0hcRAcZ56E8ozCU3x9it0zZFRIBxHvpmRmVJPrsU+iIiwDgPfYjO1deBXBGRyLgP/arSJLuaFfoiIhCH0C/OV5++iEgw7kO/siRf3TsiIkEMQj/JgY4uWtp1r1wRkfEf+uFbubv2q7UvIjLuQ78qXH9nl76VKyISn9BXv76ISAxCv7IkdO/oDB4RkfEf+iUF0dWjm9t0IFdEZNyHfjI3eoltnbp7lojIuA/9/IRCX0Skx7gPfTMjmZtDW2dXpksREcm4cR/6EHXxtHWopS8iEo/Qz0uoe0dEhLiEvrp3RESAWIW+WvoiIjEJ/YT69EVEiEvo56l7R0QE4hL66t4REQFiE/o6e0dEBGIT+jm0dah7R0QkHqGfl6BdLX0RkeGFvpn9XzN71sxWm9ntZlZgZrPN7DEzW2dmd5pZfpg3GR6vC9NnjcgrGAT16YuIRIYc+mZWA3wAWOzuJwAJ4ErgS8DX3X0usBe4OjzlamBvGP/1MF9aFOjsHRERYPjdO7lAoZnlAkXAduBc4K4w/Tbg0jC8JDwmTD/PzGyY6x8UnacvIhIZcui7+1bgK8BLRGG/D1gBNLh7zx1LtgA1YbgG2Bye2xnmr+y9XDO7xsyWm9ny+vr6oZZ3CHXviIhEhtO9U0HUep8NTAOKgQuGW5C73+Tui919cXV19XAXB0Qt/faubrq7fUSWJyIyVg2ne+f1wAZ3r3f3DuBnwGuA8tDdA1ALbA3DW4HpAGF6GbB7GOsftGSebqQiIgLDC/2XgNPNrCj0zZ8HPAc8AlwW5lkK3BOG7w2PCdMfdve0NL1fvmWiDuaKSLwNp0//MaIDsk8Cz4Rl3QR8HPiwma0j6rO/JTzlFqAyjP8wcN0w6j4qydwEoJa+iEjuwLP0z92vB67vNXo9cFof87YClw9nfUN1sKWvM3hEJOZi8o1cde+IiEBcQj9077SqpS8iMReL0K8syQegvqk1w5WIiGRWLEJ/WlkhANsaFPoiEm+xCP3q0iS5Ocb2fQcyXYqISEbFIvQTOcbkCQVq6YtI7MUi9AGmlRewrUEtfRGJtxiFfiHb96mlLyLxFrPQP0CXLromIjEWm9CfXVVMR5ezeU9LpksREcmY2IT+vEklAKyta8pwJSIimROb0J97MPT3Z7gSEZHMiU3olxbkMWVCAet2qqUvIvEVm9CHqF9/4+7mTJchIpIxsQr9ssI8mto6B55RRGScilXolxTk0tSq0BeR+IpX6Cdz2a+WvojEWKxCv7Qgl+a2TtJ0a14RkawTq9AvTubS7XCgQ3fQEpF4ilXolySjWwKrX19E4ipWoV9aEIW++vVFJK5iFfpq6YtI3MUy9JvV0heRmIpV6Bcn1b0jIvEWq9Dv6dNX946IxFWsQv9gn75a+iISU/EK/QKFvojEW6xCP5mbIC9h7Ff3jojE1LBC38zKzewuM3vezNaY2avNbKKZPWhma8PvijCvmdkNZrbOzJ42s1NG5iUcnckTCtjWcCATqxYRybjhtvS/Cfza3Y8DTgbWANcBD7n7POCh8BjgQmBe+LkG+PYw1z0k8yaV6JaJIhJbQw59MysDXgvcAuDu7e7eACwBbguz3QZcGoaXAN/3yF+BcjObOtT1D9XcSSX8rb6Jrm5ddE1E4mc4Lf3ZQD3wXTN7ysxuNrNiYLK7bw/z7AAmh+EaYHPK87eEcYcws2vMbLmZLa+vrx9GeX2bN6mU9s5utuxtGfFli4hku+GEfi5wCvBtd18ENPNyVw4AHl3D+Kia1O5+k7svdvfF1dXVwyivb3Mnhxuk6165IhJDwwn9LcAWd38sPL6LaCews6fbJvyuC9O3AtNTnl8bxqXV3Ekh9NWvLyIxNOTQd/cdwGYzmx9GnQc8B9wLLA3jlgL3hOF7gXeEs3hOB/aldAOlzYSCPKZMKGCdQl9EYih3mM//V+BHZpYPrAfeRbQj+YmZXQ1sAq4I8y4DLgLWAS1h3oyYO6mEdXX7M7V6EZGMGVbou/tKYHEfk87rY14H3jec9Y2UuZNK+Mnyzbg7ZpbpckRE0iZW38jtcUJNGS3tXXz4J6syXYqISFrFMvTfvKiGq141g58/tZX19erbF5H4iGXoJ3KMD5w3DzP4xcptmS5HRCRtYhn6EF2DZ9H0cv68blemSxERSZvYhj7ASbXlPLe9kW5dkkFEYiLWob9g2gRa2rvYsLs506WIiKRFrEP/+GkTAFi9dV+GKxERSY9Yh/68SaVMKMjlN8/tzHQpIiJpEevQz8/N4YrF0/n16h3sbGzNdDkiIqMu1qEP8PZXzaCr27nv6bRfBkhEJO1iH/pzqks4bkopy55R6IvI+Bf70Ad408JpLN+0l0//8tlMlyIiMqqGe5XNceHdZ81hx75WvvunjRw/rYzLXlmb6ZJEREaFWvpAXiKH/7xkAafPmcjH7lrFik17M12SiMioUOgHuYkc/t/bTqHbYdXmhkyXIyIyKhT6KSqL80nkGLub2zJdiojIqFDop8jJMSYW57O7qT3TpYiIjAqFfi+Vxfnsblboi8j4pNDvpbIkn91N6t4RkfFJod9LZXFSLX0RGbcU+r1UluSzR336IjJOKfR7qSzOZ39bJ60dXZkuRURkxCn0e6ksSQJQ16h+fREZfxT6vbxyZgW5OcYX7l+Du26jKCLji0K/l2Mnl3Lt+fO5f/UOfvjYS5kuR0RkRCn0+/Dus+bw2mOr+eyvnuO5bY2ZLkdEZMQo9PuQk2N87YqTKS/M4/23P8kzW3QPXREZH4Yd+maWMLOnzOxX4fFsM3vMzNaZ2Z1mlh/GJ8PjdWH6rOGuezRVlST5xpUL2bynhSXf+qNuni4i48JItPQ/CKxJefwl4OvuPhfYC1wdxl8N7A3jvx7my2pnHFPFo9eeQ0kyl2/89sVMlyMiMmzDCn0zqwUuBm4Ojw04F7grzHIbcGkYXhIeE6afF+bPajXlhVxycnRnLRGRsW64Lf1vAB8DusPjSqDB3TvD4y1ATRiuATYDhOn7wvxZr7aikIaWDprbOgeeWUQkiw059M3sEqDO3VeMYD2Y2TVmttzMltfX14/kooesprwQgK0NBzJciYjI8Aynpf8a4E1mthG4g6hb55tAuZn13Hu3FtgahrcC0wHC9DJgd++FuvtN7r7Y3RdXV1cPo7yRU1sRQn+vQl9ExrYhh767f8Lda919FnAl8LC7XwU8AlwWZlsK3BOG7w2PCdMf9jHyldea8iIAtqilLyJj3Gicp/9x4MNmto6oz/6WMP4WoDKM/zBw3Sise1RMKk2SlzD+uLZel2YQkTEtd+BZBubujwKPhuH1wGl9zNMKXD4S60u3nBxj7qRSHnh2J1/9zYt89Pz5mS5JRGRI9I3cQbrj3afz5kU13PjIOs74wkNc+9NVtHXq8ssiMraMSEs/DsqK8vjCW05k/pRSntiwh5+u2EJrZzc3XLmQMfB1AxERQC39o1KQl+C9rzuGW955KteeP59frtrGbX/emOmyREQGTaE/RP/8umN4/Ssm8alfPseF3/wDv3sxO75TICJyJAr9IcrJMb555SL+/eJX0N7ZxT9+7wkeeaEu02WJiByRQn8YipO5/NNZc7jn/Wcyq7KIz/7qOTq7ugd+oohIhij0R0BJMpePXXAc6+ubufvJLZkuR0SkXzp7Z4S8YcFkFk4v5+N3P8OyZ3aweGYFi2ZUcOa8qkyXJiJykEJ/hJgZn3/zidz8x/X8ad2ugwd2b3z7Ii45aVqGqxMRiSj0R9CCaRP42hULcXda2rt4601/4fp7nuWMY6qYWJyf6fJERNSnPxrMjOJkLl+9fCGNrR18+dfPZ7okERFAoT+q5k8p5bJXTueeldt0AxYRyQoK/VH25kU1HOjo4v7VOzJdioiIQn+0LZ5ZwXFTSvni/c+zt7k90+WISMwp9EdZTo7xlctPZldTGz9/auvATxARGUUK/TQ4oaaM46aU8mt18YhIhin00+SCE6bwxKY97G5qy3QpIhJjCv00OXv+JNzhz3877F7wIiJpo9BPkxNryigtyOWXq7bR3a377IpIZij00ySRY5x73CR+89xOln73cfa1dGS6JBGJIYV+Gn3+zSfyiQuP40/rdnHmlx/muW2NmS5JRGJGoZ9Gxclc3vO6Y/jVv55FYV6CD97xFK0durm6iKSPQj8DFkybwFcuP5m1dU184Pan2LS7OdMliUhMKPQz5LXHVnPt+fN59IV6zvnKo3zuvudYtblBd94SkVFl7tl7JsnixYt9+fLlmS5jVO1sbOUrD7zAT1dEd9wqyk9w9ZmzedtpM5hWXpjh6kRkLDKzFe6+uM9pCv3ssGNfK09s3MOvV+/gvme2YwZXnjqDzyw5nryEPpCJyOAdKfR1E5UsMaWsgDeePI03njyND+7cz+2Pv8R3/7SR9fVNvPd1x3D2/GrMLNNlisgYpyZkFjp2cinXv/F4vnr5yTy7rZF3fe8JvnC/bsQiIsM35NA3s+lm9oiZPWdmz5rZB8P4iWb2oJmtDb8rwngzsxvMbJ2ZPW1mp4zUixiv/v6Vtaz4j9dz1atmcNPv13PPyq1kc3eciGS/4bT0O4GPuPsC4HTgfWa2ALgOeMjd5wEPhccAFwLzws81wLeHse7YSOYm+I9LFjC7qpgP3rGS99+uc/tFZOiGHPruvt3dnwzD+4E1QA2wBLgtzHYbcGkYXgJ83yN/BcrNbOpQ1x8nBXkJfv4vZ/Ch18/jvqe3c+aXHuaa7y/nhR37M12aiIwxI9Knb2azgEXAY8Bkd98eJu0AJofhGmBzytO2hHG9l3WNmS03s+X19fUjUd64UF6Uz4defyzffdepLJ45kSc27uGyb/+ZxzfsyXRpIjKGDDv0zawEuBv4kLsfcjEZjzqgj6oT2t1vcvfF7r64urp6uOWNO+fMn8R3/s8rWfbBs6iekOQfbn6MD9+5UlfuFJFBGVbom1keUeD/yN1/Fkbv7Om2Cb/rwvitwPSUp9eGcTIEU8sK+el7Xs25x03iZ09t5aHn6wZ+kojE3nDO3jHgFmCNu38tZdK9wNIwvBS4J2X8O8JZPKcD+1K6gWQIKkuS3Pj2RUyfWMh7f7iCJd/6Eys3N2S6LBHJYkP+Rq6ZnQn8AXgG6LlgzL8R9ev/BJgBbAKucPc9YSdxI3AB0AK8y92P+HXbOH0jdzg27mrm7ie38LMnt7KzsZWZlUW85ZRa3nfO3EyXJiIZoMswxETd/lZu+eMGfv/iLtZsb6SqJMlbTqnhuCmlzKws4sSacvJz9X08kfFOoR8znV3d/M/v17Ni015+92I9XSkHecuL8nj/OXP5p7PmZLBCERlNuvZOzOQmcg527TS3dbKzsZUXd+5nzfb9PPJCHV/69fOce9wk5lSXZLhSEUk3tfRjZmdjK+d+5VE6upyFM8r5+1NqeOupMzJdloiMoCO19NXBGzOTJxRw3wfO4p2vmUXjgQ4+fvcz/PsvnuGhNTt1AxeRGFBLP8Y6urr56E9Xcf/qHbR3dlNdmuQti2r4l3PmUlaYl+nyRGSIdCBXjqi1o4vfvVjPz57cwgPP7sQMZlUWU12S5JSZFVSXJplTVUwix5hZWcTMyuJMlywiR6ADuXJEBXkJzj9+CucfP4VVmxt49IV6Xty5n60NB7j5D+vp7HWJh8UzK3jTwmlc9aqZJHJ0YxeRsUQtfTmi9s5uGls72LS7hW53nty0l7uf3MKLO5sAmFNdzIyJRSw9YxaFeQmK8hOcWFOmu3yJZJC6d2REuTsPPLuDlZv3sb6+ib+u301ja+fB6bMqi5g7qZRjJhXzntcew8Ti/AxWKxI/6t6REWVmXHDCVC44IbodQkNLO89tjy6wumXPAZat3s6WvS08+kIdD62p498vfgULpk5g0oSCTJYtIqilL6PoL3/bzft+/CR7mtsBmFpWQGFeAoBEjnHhCVNYesYsKkuSmSxTZNxR945kzN7mdlZubuBv9U08u63x4EHhhpZ2/rB2F2Ywf3IpF584lanlhcysLGLxzAodExAZBnXvSMZUFOdzznGTOOe4SYdNW1e3n2XP7OCP63bx1QdfPGTarMoiqkqS1FYUkpvI4eTaMv5uwRSqSvLJTeg7hSJDpZa+ZIW6xlZaO7r564bdbNrdzIZdzexsbKN+fxutHV3U7W8DoCg/wQk1ZRTmJagqSXLKzHJKC/KYUJBLbUURtRWFFIQuJJG4Uktfsl7PQd4ZlUV9Tn96SwNPvdTA+vomVm9rpKGlndVb93H3k1sOmS8/kcMxk0qYPCHJqbMmclJtGVUlSY6dXKrvFIig0Jcx4qTack6qLT9knLuzZe8B2jq7qdvfSl1jG2u2N7K2rokte1v47wdeODhvTXkhx0+bwITCPM6eX82MiUWUF758KmkiYUwrK9CxBBn31L0j41ZDSztrtu9n+74D/PyprdTvb2Nbw4FDvlOQ6vhpE1g0o5zjp5XxiqkTAKgszmfyhAJyDB1LkDFDZ++IBC3tnWzY1cym3S0caO86OH5vSzu/fHo7G+qb+twpJHKMGROLSO0hqq0o4s2Lalg4vZzC/JePIyRzcygv0hfSJHMU+iKD5O6s2rKPveG7BVsbDrDvQAf7WzvZvLclZUZYvmkPOxvb+lxOeVEeCTNmVBZRWZxkZmURPfuL0oI8aioKyUsYs6uKyUvkMK2skLIiXdlURoYO5IoMkpmxcHr5oObt6nae39HIs1sbD7koXWNrB1v3HqCz29mwq4n1u5r4y992HZze0tHFQG2t6RMLKUnmMbWsgFfPqST1UIOZcdyUUhbNKKcoX//CcnT0FyMyRIkc4/hpZRw/reyonre/tYO9zR20dnaxvr6Zbveou6kj6m7q7nY27G6mraObVVsaePj5un7XX1WSj/HyHqHnstgFeYcff5hSVkBF6HaqLEkyecKh34SeObGYomTUTTVlQgHFScXDeKR3VSTNSgvyKC2IunKOnVx6xHm7up2W9kOPMbR3dvPM1n0s37iX+v2Hdi91dHWzYXczzb2e4w5PvtRAS3sn7hx2uezezCCvnwPXUyYUUFGcT3F+gpmVRRB2OhMKcqmpKKT3+U95iRxmVRWTGw6IFORFz8sJH1/yEjnk5+ogeboo9EWyWCLHDu4gUp09fxJnzz/8W86DVdfYyr4DHQcfd3Y7G3c1097VjTuHfPJI5Tib97TQ3NbF3pZ2frvm5U8h+1o6aB/CLTdzc4yp5QUkLDrGUVWSDDuPQ3cfebnGnKrigzuLHmbGrMqiQ76Ul8zLobokqVNw+6DQF4mhSRMKDrvqac9pqkPV1tnF/j7OfGpu6+SlPS8fBG880MnWhpcf72nuYGdjK+2d3azf1cyqLfsOXqRvOIryEyTMSCSinUVfp9wmzJhdXUx+H9NqKwopLThyRJYk86itKGSgfUsyN/p0M9gvCObm2KjtsBT6IjIikrkJkiWHXwKjqiR5VLfYdPc+D3Q3tXfy0u6Ww8b3HDDv7Hr5SftbO9my9wAABzq62LirGefwhR7o6OL+Z7bTu7er273PHVi6lCZzufK06Xzy4gUjvmyFvohkFTPrs+U8oSCPE2r6Pmg+2DOujsbupjbaOo/cXbWrqe2w4yp9iXZCh++w+rN9XytTywoHPf/RUOiLiPRhMPd5mFY+OsE8mnTIXEQkRtIe+mZ2gZm9YGbrzOy6dK9fRCTO0hr6ZpYAvgVcCCwA3mZmI3+kQkRE+pTulv5pwDp3X+/u7cAdwJI01yAiElvpDv0aYHPK4y1h3EFmdo2ZLTez5fX19WktTkRkvMu6A7nufpO7L3b3xdXV1ZkuR0RkXEl36G8Fpqc8rg3jREQkDdId+k8A88xstpnlA1cC96a5BhGR2Er7TVTM7CLgG0ACuNXdP3eEeeuBTUdYXBWw6wjTM0m1DY1qGxrVNnTZXN9Qa5vp7n32j2f1nbMGYmbL+7s7TKaptqFRbUOj2oYum+sbjdqy7kCuiIiMHoW+iEiMjPXQvynTBRyBahsa1TY0qm3osrm+Ea9tTPfpi4jI0RnrLX0RETkKCn0RkRgZk6GfbZdnNrONZvaMma00s+Vh3EQze9DM1obfFWms51YzqzOz1Snj+qzHIjeEbfm0mZ2Sgdo+ZWZbw/ZbGb7L0TPtE6G2F8zs/FGsa7qZPWJmz5nZs2b2wTA+W7Zbf/Vlw7YrMLPHzWxVqO3TYfxsM3ss1HBn+EImZpYMj9eF6bMyUNv3zGxDynZbGMan9X0N60yY2VNm9qvweHS3W3Q/yrHzQ/Slrr8Bc4B8YBWwIMM1bQSqeo37MnBdGL4O+FIa63ktcAqweqB6gIuA+wEDTgcey0BtnwI+2se8C8L7mwRmh/c9MUp1TQVOCcOlwIth/dmy3fqrLxu2nQElYTgPeCxsk58AV4bx3wH+OQz/C/CdMHwlcOcobrf+avsecFkf86f1fQ3r/DDwY+BX4fGobrex2NIfK5dnXgLcFoZvAy5N14rd/ffAnkHWswT4vkf+CpSb2dQ019afJcAd7t7m7huAdUTv/2jUtd3dnwzD+4E1RFeAzZbt1l99/UnntnN3bwoP88KPA+cCd4Xxvbddzza9CzjPrK+74o5qbf1J6/tqZrXAxcDN4bExytttLIb+gJdnzgAHfmNmK8zsmjBusrtvD8M7gMmZKe2g/urJlu35/vBx+taUrrCM1BY+Ni8iahVm3XbrVR9kwbYLXRQrgTrgQaJPFg3u3tnH+g/WFqbvAyrTVZu792y3z4Xt9nUz67khbrrf128AHwN67sBeyShvt7EY+tnoTHc/heiOYO8zs9emTvTo81jWnBubbfUA3waOARYC24GvZqoQMysB7gY+5O6NqdOyYbv1UV9WbDt373L3hURXzj0NOC4TdfSld21mdgLwCaIaTwUmAh9Pd11mdglQ5+4r0rnesRj6WXd5ZnffGn7XAT8n+qPf2fOxMPyuy1yFcIR6Mr493X1n+MfsBv6Xl7sh0lqbmeURBeqP3P1nYXTWbLe+6suWbdfD3RuAR4BXE3WN5Pax/oO1hellwO401nZB6C5zd28DvktmtttrgDeZ2UaibupzgW8yytttLIZ+Vl2e2cyKzay0Zxh4A7A61LQ0zLYUuCczFR7UXz33Au8IZy2cDuxL6c5Ii159pm8m2n49tV0ZzlqYDcwDHh+lGgy4BVjj7l9LmZQV262/+rJk21WbWXkYLgT+juiYwyPAZWG23tuuZ5teBjwcPkWlq7bnU3bkRtRnnrrd0vK+uvsn3L3W3WcR5djD7n4Vo73dRvIodLp+iI6wv0jUb/jJDNcyh+gsiVXAsz31EPW1PQSsBX4LTExjTbcTfdTvIOoTvLq/eojOUvhW2JbPAIszUNsPwrqfDn/YU1Pm/2So7QXgwlGs60yirpungZXh56Is2m791ZcN2+4k4KlQw2rgP1P+Nx4nOoj8UyAZxheEx+vC9DkZqO3hsN1WAz/k5TN80vq+ptR5Ni+fvTOq202XYRARiZGx2L0jIiJDpNAXEYkRhb6ISIwo9EVEYkShLyISIwp9EZEYUeiLiMTI/wfQb6XkXb8j3QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "xarray = []\n",
    "t=numpy.linspace(0,len(tupla),num=len(tupla))\n",
    "for i in range(len(tupla)):\n",
    "    xarray.append(tupla[i][0])\n",
    "xarray.reverse()\n",
    "\n",
    "x = numpy.array(xarray,dtype=numpy.int16)\n",
    "mux = x.mean()\n",
    "sigmax = x.std()\n",
    "print (mux) \n",
    "print (sigmax)\n",
    "print (x.shape)\n",
    "\n",
    "plt.plot(t[10:400],x[10:400])\n",
    "plt.title(\"Luhn Distribution Book Free Culture\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Generate a normal distribution based on Zeta.\n",
    "\n",
    "This step is personal proposal because of the dark in the original\n",
    "Luhn article [[2](#Luhn1958)] with respect to the boundaries for cutting. \n",
    "Some papers consider that in the obtention of these limits there is\n",
    "certain empirism. Calcs are made based on a nule hypotesis.\n",
    "[Keith web book, chapter2](http://www.dcs.gla.ac.uk/Keith/Chapter.2/Ch.2.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 140,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "113 427\n",
      "270.686232489232 78.46649164766025\n"
     ]
    }
   ],
   "source": [
    "sigma = x_s.std()\n",
    "mu = x_s.mean()+3*sigma\n",
    "\n",
    "pdf = (1/(sigma * numpy.sqrt(2 * numpy.pi))\n",
    "* numpy.exp( - (t - mu)**2 / (2 * sigma**2) ))\n",
    "\n",
    "perc_05 = int(mu-2*sigma)\n",
    "perc_95 = int(mu+2*sigma)\n",
    "print(perc_05, perc_95)\n",
    "print (mu, sigma)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 145,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(2054,)\n",
      "22.27677449217962\n",
      "5.143847691885105\n"
     ]
    }
   ],
   "source": [
    "x_s = x[(x > 5) & (x < 1000)]\n",
    "print(x_s.shape)\n",
    "print(x[113]/(x.max()*pdf[113]))\n",
    "print(x[427]/(x.max()*pdf[427]))\n",
    "factor = x[113]/(x.max()*pdf[113]) - x[427]/(x.max()*pdf[427])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 142,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "138"
      ]
     },
     "execution_count": 142,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x[113]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 146,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAAAtQElEQVR4nO3de3wU9b3/8dcnm3vCnRAuCQS5RxSRlFsAuclFFGwVBa+tWo/t8ViP7c9LL7Sltaf2eEptpba0Wq31gpcqUZGgAipBkaAoIATCNQECAcIlJCG37++P2YUQErJJdnd2Zz/PxyOP3Z2ZnfmMg++Z/c7Md8QYg1JKKeeKsLsApZRS/qVBr5RSDqdBr5RSDqdBr5RSDqdBr5RSDhdpdwH1de7c2aSlpdldRvPl5VmvAwbYW4cDleWVARA/IN7mSpQKXuvXrz9sjElqaFzQBX1aWhq5ubl2l9F848dbr6tW2VmFI30x/gsAhq4aanMlSgUvEdnT2DhtulFKKYfToFdKKYfToFdKKYfToFdKKYfToFdKKYfToFdKKYfToFdKKYdzTNAXFsK8ebBtm92VKKVUcHFM0BcVwa9+dfYGVaWUUhbHBH1MjPVaUWFvHUopFWwcE/Sxsdbr6dP21qGUUsHGMUHvOaLXoFdKqXNp0CullMNp0CullMM5Jug9bfR6MlYppc7lmKCPjrZe9YheKaXO5Zigj4iAqCgNeqWUqs8xQQ9WO70GvVJKnctRQR8bq230SilVn1dBLyLTRCRPRPJF5OEGxo8Tkc9FpFpErq837nYR2e7+u91XhTdEj+iVUup8TQa9iLiAhcB0IB2YKyLp9SbbC3wbeLHedzsCPwdGAMOBn4tIh9aX3TANeqWUOp83R/TDgXxjzE5jTCXwMjCr7gTGmN3GmK+A2nrfnQq8Z4w5aowpAd4Dpvmg7gZp0Cul1Pm8CfoeQEGdz4XuYd5ozXebTdvolVLqfEFxMlZE7haRXBHJLS4ubvF89IheKaXO503Q7wNS63xOcQ/zhlffNcYsMsZkGGMykpKSvJz1+TTolVLqfN4E/Tqgn4j0FpFoYA6Q5eX8s4EpItLBfRJ2inuYX2jQK6XU+ZoMemNMNXAvVkBvAV4xxmwWkfkiMhNARL4hIoXAbOCvIrLZ/d2jwK+wdhbrgPnuYX6hbfRKKXW+SG8mMsYsBZbWGzavzvt1WM0yDX33GeCZVtToNT2iV0qp8wXFyVhfiYyE6mq7q1BKqeDiqKB3uaC2/pX8SikV5hwV9BERUFNjdxVKKRVcHBX0LpcGvVJK1eeooI+I0KYbpZSqz1FBr0f0Sil1PscFvR7RK6XUuRwV9HoyVimlzueooNemG6WUOp/jgl6bbpRS6lyOCnptulFKqfM5Kui16UYppc7nuKDXphullDqXo4Jem26UUup8jgp6zxG9MXZXopRSwcNRQR/hXhsNeqWUOstRQe9yWa/afKOUUmd59YSpUOEJej0h27jKmkqy87PZUbKDizpcxLS+04h2RdtdllLKjxwV9J6mGz2ib9gnBZ9wyxu3sLNk55lhvdr14oVvvUBmz0wbK1NK+ZM23YSJVbtXMfn5yQBkzcmi+P8V8/bct4lyRTHpn5N4f+f7NleolPIXRwa9Nt2ca8+xPXxz8TdJa5/GmjvWcM2Aa+gc35kZ/Wfw6Z2f0r9Tf2a/OptdJbvsLlUp5QeOCnptujlfranl1jdupaa2hqw5WSQnJp8zvlN8J96c8ybGGG7+983UGt1LKuU0jgp6bbo537+++hcf7/2YBVMX0KdjnwanuajDRTwx7Qk+KfyEZzc8G9gClVJ+58ig16YbS3lVOQ+9/xAjeozgO0O/c8FpbxtyG5mpmTzywSOUVZUFqEKlVCA4Kui16eZcf//87xSVFvHY5MeIkAtvahHht5N/y6FTh/jb+r8FqEKlVCA4Kui16easyppKfrfmd4zpOYZxvcZ59Z0xPccwPm08v1vzOyqqK/xcoVIqUDToHeqljS9ReKKQH4/5MSLi9fd+Ovan7D+5n5c2vuTH6pRSgeSooI+NtV4r9GCUv6z/CwM6DWBa32nN+t7E3hNJT0rnqdyn/FSZUirQvAp6EZkmInkiki8iDzcwPkZEFrvHrxWRNPfwKBF5TkQ2isgWEXnEx/WfIyHBei0t9edSgt+XRV/yaeGn3JNxT7OO5sFqq79n2D2s27+O9fvX+6lCpVQgNRn0IuICFgLTgXRgroik15vsTqDEGNMXWAA85h4+G4gxxlwCDAP+w7MT8IfEROv11Cl/LSE0LFq/iBhXDLcNua1F379tyG3ER8Xz1/V/9XFlSik7eHNEPxzIN8bsNMZUAi8Ds+pNMwt4zv3+NWCSWIeSBkgQkUggDqgETvik8gZ4jujDOegrayp5adNLXJd+HR3jOrZoHu1i2/GtQd/i1a9f5XT1aR9XqJQKNG+CvgdQUOdzoXtYg9MYY6qB40AnrNA/BRwA9gKPG2OO1l+AiNwtIrkikltcXNzslfDQoIflO5ZTUlHCTYNvatV8bhp8E8cqjrEsf5mPKlNK2cXfJ2OHAzVAd6A38EMRuaj+RMaYRcaYDGNMRlJSUosXpk038NKml+gY15Er+1zZqvlMvmgyneM789ImvfpGqVDnTdDvA1LrfE5xD2twGnczTTvgCHATsMwYU2WMOQTkABmtLbox4X4ytqyqjCVbl3D9oOtb3cd8lCuKG9JvICsvi5OnT/qoQqWUHbwJ+nVAPxHpLSLRwBwgq940WcDt7vfXAyuMMQaruWYigIgkACOBrb4ovCHh3nTz9ra3OVV1ijmD5/hkfnMvmUt5dTlvbXvLJ/NTStmjyaB3t7nfC2QDW4BXjDGbRWS+iMx0T/Y00ElE8oEHAM8lmAuBRBHZjLXD+Icx5itfr4RHXByIhG/QL8lbQuf4zl7fCduU0amjSU5IZkneEp/MTyllD6+eMGWMWQosrTdsXp33FViXUtb/XmlDw/1FxDqqD8egr6qpYun2pVw78FpcES6fzDNCIrim/zUs3ryYyppKfeSgUiHKUXfGQvgG/eq9qzlWcYyZ/Wc2PXEzzBo4i5OVJ1m1e5VP56uUChxHBn04nox9a9tbRLuiW321TX2Tek8iPiqeJVu1+UapUOW4oI+Ph7Iw607dGENWXhaTek8iMTrRp/OOi4pjap+pZG3Lwjq/rpQKNY4L+qgoqK62u4rA2nJ4CztKdjBzgG+bbTyu7n81hScK2Vy82S/zV0r5lyODvqrK7ioCa+l26zz5jH4z/DL/KX2mAJCdn+2X+Sul/MtxQR8ZGX5H9O/tfI+BnQeS2i616YlbIKVtCulJ6WTv0KBXKhQ5LujD7Yi+orqCj/Z8xJUX+fYkbH1T+0zloz0f6fNklQpBjgv6cDuiz9mbQ0V1xZnmFX+Z2mcqp2tO89Gej/y6HKWU7zku6MPtiP69ne8RGRHJFb2u8OtyxvUaR2xkrLbTKxWCHBf04XZE/97O9xiVMoo2MW38upy4qDjG9Rqn7fRKhSDHBX04HdEfLjvMFwe+8Hv7vMfk3pPZcngLRaVFAVmeUso3HBf04XRE/8HODzAYn98N25jxaeMB+HD3hwFZnlLKNxwX9OF0RL9i1wraxrQlo7vfuvg/x9BuQ2kb05aVu1cGZHlKKd9wXNCH0xH96oLVjE4dTWSEV52QtlpkRCRje47VDs6UCjGOC/pwOaI/Wn6Ur4u/ZkzqmIAud3zaePKO5LH/5P6ALlcp1XKOC/pwOaL/pOATADJ7ZgZ0uRPSJgDaTq9UKHFc0IfLEf3qvauJjIhkeI/hAV3uZV0vo11MO22+USqEOC7ow+WIPqcgh8u7XU58VHxAl+uKcDGu1zg9IatUCHFc0MfGQnm53VX4V2VNJev2ryMzNbDNNh7j08az/eh29p3YZ8vylVLN47igb9vWaro5fdruSvzn8wOfU1FdYWvQA9p8o1SIcFzQt3H3BHDihL11+NPqvauBwJ+I9RiSPIS2MW3P1KGUCm6OC/q2ba3XkyftrcOfcgpy6NOhD10Tu9qyfFeEi1Epo8gpyLFl+Uqp5nFs0Dv1iN4YQ87eHNuO5j0yUzPZdGgTxyqO2VqHUqppjgt6pzfdbD+6neKyYtva5z3G9ByDwZy5nl8pFbwcF/ROb7rJ2Ws1l9gd9MN7DCcyIlLb6ZUKAY4Neqce0ecU5NAhtgODkgbZWkdCdAJDuw5ldYEGvVLBznFB7/Smm5yCHEanjiZC7N90Y3qO4bN9n1FZU2l3KUqpC7A/LXzMyU03h8sOs/XwVtubbTwyUzOpqK7g8wOf212KUuoCvAp6EZkmInkiki8iDzcwPkZEFrvHrxWRtDrjLhWRT0Rks4hsFJFYH9Z/noQEiIuD3Fx/LsUeawrWANaRdDDwXPmj7fRKBbcmg15EXMBCYDqQDswVkfR6k90JlBhj+gILgMfc340E/gXcY4y5GBgP+LXLMRG46ipnBn3O3hyiIqIC9qCRpnRN7Erfjn31enqlgpw3R/TDgXxjzE5jTCXwMjCr3jSzgOfc718DJomIAFOAr4wxXwIYY44YY2p8U3rj+vSBwkKorfX3kgIrpyCHYd2HERcVZ3cpZ2SmZpKzNwdjjN2lKKUa4U3Q9wAK6nwudA9rcBpjTDVwHOgE9AeMiGSLyOci8mBDCxCRu0UkV0Ryi4uLm7sO5xfcw+rrpqSk1bMKGhXVFbZ2ZNaYMT3HUFxWzPaj2+0uRSnVCH+fjI0ExgA3u1+/KSKT6k9kjFlkjMkwxmQkJSW1eqEdOlivx461elZBY/3+9VTWVAZN+7yHZ8ej7fRKBS9vgn4fkFrnc4p7WIPTuNvl2wFHsI7+PzLGHDbGlAFLgctbW3RT2re3Xp0U9J528NGpo22u5FwDOw+kU1wnDXqlgpg3Qb8O6CcivUUkGpgDZNWbJgu43f3+emCFsRpts4FLRCTevQO4AvjaN6U3zhP0Tmq6Wb13Nf069qNLQhe7SzmHiDA6dbSekFUqiDUZ9O4293uxQnsL8IoxZrOIzBeRme7JngY6iUg+8ADwsPu7JcDvsXYWG4DPjTHv+Hwt6vEE/ZEj/l5SYBhjWFOwJuiabTwyUzPZdmQbxadaf35FKeV7kd5MZIxZitXsUnfYvDrvK4DZjXz3X1iXWAZMWpr1mp8fyKX6T96RPI6UHwm6E7Eenuvp1xSsYdbA+hdkKaXs5rg7Y8HqBiElBbZssbsS3zjTkZnNXRM3Zli3YURFRJ25oUspFVwcGfQAgwY5J+hXF6ymU1wnBnQaYHcpDYqLimNY92HaTq9UkHJ00G/d6oybpjwPGrHuQQtOmamZ5O7P5XS1gx/Wq1SIcmzQp6dDWRns3Wt3Ja1z6NQhth/dHrTt8x6ZqZmcrjnN+gPr7S5FKVWPY4N+xAjr9cMP7a2jtTzt3sEe9J7r+7WdXqng49igv/RSSE6G996zu5LWWb13NTGumKDpyKwxyYnJ9OnQR9vplQpCjg36iAjo1w/277e7ktbJKcgho3sGMZExdpfSpMye2sGZUsHIsUEPEBNjdW4Wqsqrylm/f33QN9t4ZKZmUlxWTP5Rh9zAoJRDaNAHsdz9uVTVVgXt9fP1aTu9UsFJgz6IeToKC7aOzBqTnpRO+9j22k6vVJDRoA9iOQU5DOw8kM7xne0uxSsREsGolFEa9EoFGQ36IFVrallTsCZk2uc9MlMz+br4a46WH7W7FKWUm+ODvqLC7ipaZkvxFkoqSoK2x8rGeJqZPi381OZKlFIejg/6UD2i9zR/hNoR/fAew3GJ60xHbEop+2nQB6mcghyS4pPo27Gv3aU0S0J0AkO7DdV2eqWCiKODPjbWCvpQ7NgsFDoya0xmaiaf7fuMqpoqu0tRSuHwoO/XD4yBzZvtrqR5ikqL2FGygzGpodU+7zE6dTTl1eVsKNpgdylKKRwe9GPHWq8ffWRvHc0V7A8aaYrnvII23ygVHBwd9Glp1pOmQi7oC3KIjYzl8m6X211Ki/Ro24Ne7Xpp0CsVJBwd9CIwbhysWmU14YSKnIIchvcYTrQr2u5SWkw7OFMqeDg66AEmToRDh+DNN+2uxDtlVWV8fuDzkLussr7RKaM5UHqAPcf32F2KUmHP8UF/663gcsGaEOln67N9n1FdWx3yQe85v6DX0ytlP8cHfXS0dfXNrl12V+IdT0dmo1JH2VxJ61zS5RISoxO1nV6pIOD4oAdITLSeHxsKVu9dzeAug+kY19HuUlrFFeFiZMpIDXqlgkBYBH1CQmgEfU1tDWsK1jC251i7S/GJzNRMNh7cyInTJ+wuRamwFhZBHx8fGkH/5cEvOVl5MuQ6MmtMZmomBqMdnCllMw36IOJpn3fKEf2IlBFESISekFXKZl4FvYhME5E8EckXkYcbGB8jIovd49eKSFq98T1FpFREfuSjupslVIL+470f06tdL1Lbpdpdik+0jWnLJV0u0XZ6pWzWZNCLiAtYCEwH0oG5IpJeb7I7gRJjTF9gAfBYvfG/B95tfbktk5AABw8G901TxhhW713tmGYbj8zUTNbuW0t1bbXdpSgVtrw5oh8O5BtjdhpjKoGXgVn1ppkFPOd+/xowSdzdLorItcAuwLauxfr3t47oV6+2q4Km7SjZQVFpkWOabTwye2ZSWlnKxoMb7S5FqbDlTdD3AArqfC50D2twGmNMNXAc6CQiicBDwC9bX2rL3X47RETAzTdDebmdlTTO0z7vxCN60A7OlLKTv0/G/gJYYIwpvdBEInK3iOSKSG5xcbHPi+jYEX76UygogPfe8/nsfeLjPR/TMa4jg5IG2V2KT/Vs15Pubbpr0CtlI2+Cfh9Q9+xgintYg9OISCTQDjgCjAB+JyK7gfuBH4vIvfUXYIxZZIzJMMZkJCUlNXcdvPLjH0NcHHzwgV9m32of7/2YzNRMIsRZF0KJCJmpmawpCJE+KJRyIG9SZR3QT0R6i0g0MAfIqjdNFnC7+/31wApjGWuMSTPGpAF/AH5jjHnSN6U3T0wMZGbC4sVQUmJHBY07WHqQ7Ue3O6593iMzNZO9x/dSeKLQ7lKUCktNBr27zf1eIBvYArxijNksIvNFZKZ7sqex2uTzgQeA8y7BDAY/+YnVk+UTT9hdybmc2j7voR2cKWUvr9oJjDFLjTH9jTF9jDGPuofNM8Zkud9XGGNmG2P6GmOGG2N2NjCPXxhjHvdt+c0zfjyMHBl87fSr964mLjKOYd2H2V2KXwxJHkJ8VPyZHZpSKrCc1SDshcsus54hG0zX1K/as4oRKSNC+kEjFxLlimJ06mhW7VlldylKhaWwC/pLLoHjx2HrVrsrsRwtP8qXRV8yIW2C3aX41YS0CWw6tIniU76/qkopdWFhF/TDh1uvl1wC27fbWwvAh7s/xGDCIugBVu1eZW8hSoWhsAv6YcPgtdegpgZeftnuamDl7pXERcYxImWE3aX4VUb3DBKiEli5e6XdpSgVdsIu6AGuuw4GDYK1a+2uxAr6MT3HOLZ93iPKFcXYXmM16JWyQVgGPVhNOLm59tZw6NQhNh3a5PhmG48JaRPYengrB04esLsUpcJK2AZ9v35Wj5Z29n3jaa+e0Dt8gh60nV6pQAvboE91d+pQUHDh6fxp5a6VJEYnMqybM6+fr29ot6G0jWmrzTdKBVjYBv0gd99hX35pXw0rd69kbM+xRLmi7CsigCIjIhnXa5wGvVIBFrZBf9llVtfFX31lz/L3n9xP3pG8sGmf95iQNoH8o/na741SARS2QR8VBV26QFGRPcsPt/Z5D8+ObcWuFTZXolT4CNugB+ja1b6gX75jOR1iOzC061B7CrDJkK5DSIpPYvmO5XaXolTY0KC3IeiNMSzfsZwr+1yJK8IV+AJsFCERXNnnSpbvWE6tqbW7HKXCQlgHfbducMCGS7o3HtrIgdIDTO0zNfALDwJT+0yluKyYDUUb7C5FqbAQ1kHftSvs2xf4B5Fk52cDMKXPlMAuOEh41tvz30Ep5V9hHfQzZlivTz0V2OVm78jm4qSLSWmbEtgFB4muiV0ZkjyE7B0a9EoFQlgHfWYmTJoETz4ZuP7pT1We4uO9HzOt77TALDBITe0zlZyCHE6ePml3KUo5XlgHPcD111vt9DvPeyaWf3y450MqayrDtn3eY2rfqVTXVuvNU0oFQNgH/Rj3Y1rfeScwy8vOzyYuMo6xvZz5IHBvZaZmEh8Vr+30SgVA2Af9xRfDuHHws58Fpt+b7B3ZjE8bT2xkrP8XFsRiImOYkDaBZTuWYYLpuY5KOVDYB72I1UZfVWV1XVzsxyfdbT+ynbwjeUzvO91/CwkhV/e/mp0lO9lyeIvdpSjlaGEf9GA9VvCDD6yQv/12/y1nSd4SAGYOmOm/hYSQq/tfDUBWXpbNlSjlbBr0bqNGwa9+Be++678eLbPyshiSPIRe7Xv5ZwEhJqVtCsO6DdOgV8rPNOjr+O53IS4OfvhD38/7cNlhcgpymDVglu9nHsJmDZjFp4WfcrD0oN2lKOVYGvR1dO4MDzwAK1bAsWO+nfc7296h1tRqs009MwfMxGB4Z3uALntSKgxp0NczZox189Tixb6d75K8JfRo04PLu13u2xmHuEuTL6Vnu57afKOUH2nQ1zNpknXJ5cMP++55shXVFWTvyGbmgJmIiG9m6hAiwsz+M1m+YzllVWV2l6OUI2nQ1xMVBQ89ZDXd/Pd/+2aey/KXUVZVxrUDr/XNDB3mm4O+SXl1OUu3L7W7FKUcyaugF5FpIpInIvki8nAD42NEZLF7/FoRSXMPv1JE1ovIRvfrRB/X7xe33GJ1jfDii1Ba2vr5Ld68mM7xnZnYOyRWP+Cu6HUFyQnJLN7s4/YypRTgRdCLiAtYCEwH0oG5IpJeb7I7gRJjTF9gAfCYe/hh4BpjzCXA7cDzvircn0Tg3nvh5En4859bN69TlafIysviukHXERkR6ZsCHcYV4eL69Ot5Z9s7lFb6YM+qlDqHN0f0w4F8Y8xOY0wl8DJQ/xrBWcBz7vevAZNERIwxXxhj9ruHbwbiRCTGF4X72xVXQFoafPpp6+bzzvZ3KKsq48aLb/RJXU5148U3Ul5dzlt5b9ldilKO403Q9wDq9gJT6B7W4DTGmGrgONCp3jTXAZ8bY07XX4CI3C0iuSKSW+zPPgiaafRoeOMN+Oijls9j8ebFdE3syrhe43xXmANl9sykR5se2nyjlB8E5GSsiFyM1ZzzHw2NN8YsMsZkGGMykpKSAlGSV558Etq3h2nTrCdRNdeJ0ydYun0ps9Nnh92zYZsrQiKYnT6bd/Pf5XjFcbvLUcpRvAn6fUBqnc8p7mENTiMikUA74Ij7cwrwBnCbMWZHawsOpA4drDb68nK45hooLGze95dsXUJFdYU223jpxsE3UllTyetbXre7FKUcxZugXwf0E5HeIhINzAHq392ShXWyFeB6YIUxxohIe+Ad4GFjTI6Pag6oOXPgvvvgiy+gVy/4+9+9/+4/NvyDPh36MDp1tP8KdJARPUYwoNMA/rHhH3aXopSjNBn07jb3e4FsYAvwijFms4jMFxHP/fxPA51EJB94APBcgnkv0BeYJyIb3H9dfL4WfiQCTzwBmzZBcjI8+6x339tZspOVu1fyncu+ozdJeUlEuGPoHazeu5ptR7bZXY5SjuFVG70xZqkxpr8xpo8x5lH3sHnGmCz3+wpjzGxjTF9jzHBjzE738F8bYxKMMZfV+Tvkv9Xxn4svtq6tz82Fzz5revpnNzyLINw25Db/F+cgt156Ky5x8Y8v9KheKV/RO2Ob4YEHrHb7W2658MPEa2preHbDs0zpM4XUdqmNT6jO061NN67qdxXPffkc1bXVdpejlCNo0DdDWhr84hewfTt873uNh/37O9+n4EQBdwy9I5DlOcYdQ+/gQOkBluUvs7sUpRxBg76Zvvtd6+TsX/8K8+Y13PHZk+uepEtCF+17voVm9JtBckIyf17XytuSlVKABn2zRUTA44/D5Mnw619bfdjPnQulNXEA5B/N551t73DPsHuIiQyJm4CDTpQriu9lfI9389/Vk7JK+YAGfQtERcHy5fDOOzBlCrz8Mozd8EeKKjuy8LOFuCJc/EdGg/eGKS/dk3EP0a5o/rT2T3aXolTI06BvIRG46iqri4Rf/hI2lPaj2/p/8pe1zzA7fTbd23S3u8SQlpyYzJzBc3j2y2c56TppdzlKhTQNeh/42c/gL/3+j+iMhVRwgs+fuo+KCrurCn0/GPEDSitLeaP7G3aXolRI06D3ARG4LeVNOl4xj95mInkfjOQXv7C7qtB3ebfLmXzRZJ5PfZ7yCB897kupMKRB7yNPdyuiKKaSv932U9LT4bHHrB3Az39ud2Whbd64eRyJOaJH9Uq1gga9D1TWVPJYzwIyj7VlYu/xrF0LCxfCjBkwfz58+9tw9KjdVYamsb3GklGSwbM9n6WiWtvDlGoJDXofeGrdUxTGnmbe7l6ICImJ8P3vw+uvw223wXPPQadOcPPNsH693dWGnrt238XhmMM8te4pu0tRKiRp0LdSSXkJ8z+az+Sj7bmypMM542JirJB//33rCp3XX4eMDKsLhWXLoLLSpqJDzDeOfYORR0cy/6P5HCk7Ync5SoUcDfpWevTjRykpL+Hx/D4IDfdSOWmSdc19fj7ceCO8+ipMn271hvnYY1BVFeCiQ4wgPJD/ACdOn2D+h/PtLkepkKNB3wrbjmzjT5/9ie9c9h2GnEpscvqUFOvmqqIi60h/4EB4+GHo3t36fKGO0sJd31N9+e7l3+XPuX9mS/EWu8tRKqRo0LdQranlrqy7iI+K59FJjzbrux06WG33n3wCS5ZAba11wjYlBcaOhZyQfESL/82fMJ820W246627qKmtsbscpUKGBn0LLVq/iI/3fszvp/yeroldWzyfmTOhoMBqwsnIsPq6HzMGhgyB+++3HmW4davv6g5lXRK68Idpf2BNwRoWrltodzlKhYxIuwsIRTtLdvLgew8y+aLJfPuyb7d6fvHx8OCD1vt9+6xeMXNzrSdbAbRpA9/6FnTtChMnwoQJEBlpXacfbm699FZe2vQSj3zwCFf1u4q+HfvaXZJSQU+P6JuporqC2a/OxhXh4m/X/M3njwns0QOefhq+/BJOnLCO8C+/HLKzraP+qVMhOtp6fu0tt8CCBdaJ3nA5oSsiLLp6ETGuGGa/OpvyKr1jVqmm6BF9M/0w+4d8fuBzlsxZQlr7NL8uq00b+MY3YNUq6/OhQ7B0KeTlWTuAf/8bXnjBGhcfD/37w+zZ1lU+nTtbOwMRcLn8WmbApbZL5flvPs/VL13ND5b9gEXXLLK7JKWCmgZ9Mzz52ZP8OffP/GjUj5g5YGbTX/CxLl2sk7YeVVVw4IAV+B9+CIcPw09+Yv15REXB4MGQnm5dznnppTBggLUDGDQI2rYN+Gr4xIz+M3hkzCP8z+r/YXCXwdw34j67S1IqaGnQe+nfW/7Nfe/ex7UDr+W3k39rdzmAFeI9e1onbe+/37o8c+1aOHLEOuovK4M9e+Crr6xfBfv2nT+P/v3hoougWzdo3976PHCgtTPo1i2w69Ncv5rwK7Ye3sr9y+4nOSGZGwffaHdJSgUlDXovvLHlDea+PpdRqaN48Vsv4ooIzrYQERg50no/Y8b540+ehI0b4fhx6/2aNbBtm7VT2LQJCgvPnX7gQBgxwjonMGAAxMZC377WL4vBg60djZ1cES5evO5Fpjw/hVvfuJXIiEiuS7/O3qKUCkIa9E14bsNz3Jl1J8N7DOftuW8TFxVnd0kt1qYNjB599vMNN5w7vrQU9u61Hn6+fj289RasWGH9QigrO39+/ftb3TwkJ1v3ALRpA/36nT0vMHCgtXOAs+cMPFwu35w7iI2MJWtuFjNenMENr93AoqsXcefld7Z+xko5iAZ9I6pqqnjo/YdY8OkCJvWexJtz3iQxuum7X0NZYqLVlp+eDrNmWT1vAlRXQ0mJtSPYts3aGWzcaP0CqKqyrvPPy7M+e3t3b3S0tVNo29baYdS9eKnuDgOg9khb2kZW4/rq7PkFkbO/KNrHtmf5Lcu5/tXrueutu9h4aCP/e+X/EuWy+SeHUkFCg74BeYfzuDPrTnIKcviv4f/F41MeJ9oVbXdZtomMhKQk669378anKys7e+RfXGztEMC689dzzsBjxw6r6+b8fFi58uxwY6xzCbW1defcx3oZcu7yeveGhATrfbduCSR3X8Kgjg/yxNonWJL7GTcnPkOyayA9e1q1e0REwMUXW79G6oqKCs97E5TzadDXcfL0SRZ8uoDffPwb4qLieOFbL3DTJTfZXVbIiI+3/sBqqhk06Oy46dO9n0/dHUZ5Obw9I59awPX9vmf69S8pgZ07rfeVldavim3bojHmD0S0G83uq+7h0dNDYPXDsOZHUNmmyeV27WrVXZeI9Ysj8QI/5hISzv7SqC8lxWraakp0tLXziWjkzpbISOddJqsCR4Meq6vhZ754hsdyHqO4rJjZ6bP54/Q/tqprA9VydXcYACM7Wg8HH3qPd9+vqLiBXcVX8OOPf8CbkfNpO3khs7s/yPTk79ImsgNHjsCuXed+p7zc+tVRU68LnWPHrLuUG+P5BVL/e74WF2f9gmloZ5KWZj3vwFvx8db5k8Z2KvVFRVnTR3v5ozYpyboarDmio/XXlD+FbdCfrj7Nil0reOXrV1i8aTHl1eVM6j2J30z6DcN7DLe7PNUKsbEwKDWZN256mc/2PcC8lfN4esdDvLD/59xw8Q3MyZjDA7MnEBsZ65PllZc3fLL69Gnrl0Z1ddPzKCyEgwcbHmeMtRM6der8cWVlsGVL/aauppflTU2B1LVr83ZWddU/0d8S3bpZvcj6Ut++0K5d877TubP169DXvAp6EZkGPAG4gL8bY35bb3wM8E9gGHAEuNEYs9s97hHgTqAGuM8Yk+2z6pvheMVxvjr4FWsK1rCmcA0rdq2gtLKUxOhEbr30Vu7JuIeh3YbaUZryo+E9hrPslmVsKNrAovWL+NdX/+KfX/6ThKgEJvaeyOjU0YxKGcVlXS+jXWwz/690i4uz/hri6/DwhYqKhncajSkqavgejIbU1lo7t/Jm9EzR2K8pb9TUWMvbs6f53/WorIT9+1v+fV+68UarK3NfE9PEZRIi4gK2AVcChcA6YK4x5us603wfuNQYc4+IzAG+aYy5UUTSgZeA4UB34H2gvzGm0U2akZFhci/0W7kRJ06fYP3+9Rw8dZCi0iIOlh6k6FQRu0p2sfXwVg6eOnu41L9Tf8b3Gs+1A69lYu+JxETGXGDOXho/3nr19FegfOaL8V8AMHRV63fEFdUVrNq9iiVbl7Bi9wq2Hdl2ZlxyQjIDOg+gT4c+dE3sSnJCMl0Tu5KUkESb6DYkRCeQGJ1IYnQiCVEJRLuifd7XkbLHyZO+7S/q1Clr59XcZ0x47l5vCRFZb4zJaGicN0f0w4F8Y8xO98xeBmYBX9eZZhbwC/f714Anxfo/YBbwsjHmNLBLRPLd8/ukJStyIVuKtzDxnxPPfI6MiCQ5IZme7XpyVb+rGNh5IOlJ6YxMGUnn+M4XmJNystjIWKb1nca0vtMAOFx2mLWFa9l0aBPbjmwj70ge2TuyOXTqENW1TbdvREZENvrnEteZHUHdp4+1dFjdnUpjTzNToW167XT+79L/8/l8vQn6HkBBnc+FwIjGpjHGVIvIcaCTe/in9b7bo/4CRORu4G6Ans09i+OWnpTOB7d9cOZIrENcByJEO+dUF9Y5vjMz+s9gRv9zbyWuNbWUlJdQVFpEcVkxpypPUVpZSmllKaeqTnHy9Ekqayqprq1u/M9YO4q6v5oNpkXDPJ/rT6ecJbVdql/mGxQnY40xi4BFYDXdtGQebWLaMLH3xKYnVMoLERJBp/hOdIpv4RlCpYKIN4e8+4C6u5kU97AGpxGRSKAd1klZb76rlFLKj7wJ+nVAPxHpLSLRwBwgq940WcDt7vfXAyuM9fsyC5gjIjEi0hvoB3zmm9KVUkp5o8mmG3eb+71ANtbllc8YYzaLyHwg1xiTBTwNPO8+2XoUa2eAe7pXsE7cVgP/eaErbpRSSvmeV230xpilwNJ6w+bVeV8BzG7ku48Cj7aiRqWUUq2gl6UopZTDadArpZTDadArpZTDadArpZTDNdnXTaCJSDHQii6K6Awc9lE5wUbXLXQ5ef103YJDL2NMUkMjgi7oW0tEchvr2CfU6bqFLievn65b8NOmG6WUcjgNeqWUcjgnBv0iuwvwI1230OXk9dN1C3KOa6NXSil1Lice0SullKpDg14ppRzOMUEvItNEJE9E8kXkYbvraS4RSRWRlSLytYhsFpEfuId3FJH3RGS7+7WDe7iIyB/d6/uViFxu7xo0TURcIvKFiLzt/txbRNa612Gxuxts3N1aL3YPXysiabYW7gURaS8ir4nIVhHZIiKjnLLtROS/3f8mN4nISyISG8rbTkSeEZFDIrKpzrBmbysRud09/XYRub2hZQULRwS9+wHmC4HpQDow1/1g8lBSDfzQGJMOjAT+070ODwMfGGP6AR+4P4O1rv3cf3cDTwW+5Gb7AbClzufHgAXGmL5ACXCne/idQIl7+AL3dMHuCWCZMWYgMARrPUN+24lID+A+IMMYMxirq/I5hPa2exaYVm9Ys7aViHQEfo71WNXhwM89O4egZIwJ+T9gFJBd5/MjwCN219XKdVoCXAnkAd3cw7oBee73fwXm1pn+zHTB+If1dLEPgInA24Bg3XEYWX8bYj37YJT7faR7OrF7HS6wbu2AXfVrdMK24+zzoDu6t8XbwNRQ33ZAGrCppdsKmAv8tc7wc6YLtj9HHNHT8APMz3sIeahw/9wdCqwFko0xB9yjioBk9/tQW+c/AA8Cte7PnYBjxrifoH1u/ec8bB7wPGw+WPUGioF/uJum/i4iCThg2xlj9gGPA3uBA1jbYj3O2XYezd1WIbMNwSFNN04iIonA68D9xpgTdccZ69Ah5K6HFZGrgUPGmPV21+InkcDlwFPGmKHAKc7+9AdCett1AGZh7cy6Awmc3+zhKKG6rS7EKUHviIeQi0gUVsi/YIz5t3vwQRHp5h7fDTjkHh5K65wJzBSR3cDLWM03TwDt3Q+Th3Prb+xh88GqECg0xqx1f34NK/idsO0mA7uMMcXGmCrg31jb0ynbzqO52yqUtqFjgt6bB5gHNRERrGfvbjHG/L7OqLoPXr8dq+3eM/w291UBI4HjdX56BhVjzCPGmBRjTBrWtllhjLkZWIn1MHk4f90aeth8UDLGFAEFIjLAPWgS1nOSQ37bYTXZjBSRePe/Uc+6OWLb1dHcbZUNTBGRDu5fPVPcw4KT3ScJfPUHXAVsA3YAP7G7nhbUPwbr5+JXwAb331VY7ZsfANuB94GO7ukF60qjHcBGrKsibF8PL9ZzPPC2+/1FwGdAPvAqEOMeHuv+nO8ef5HddXuxXpcBue7t9ybQwSnbDvglsBXYBDwPxITytgNewjrfUIX1a+zOlmwr4A73euYD37F7vS70p10gKKWUwzml6UYppVQjNOiVUsrhNOiVUsrhNOiVUsrhNOiVUsrhNOiVUsrhNOiVUsrh/j9DtgUsWKsDaQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "t = np.arange(0,1500,1)\n",
    "# lower cut-off.\n",
    "plt.axvline(perc_05,label='05 perc',c='r')\n",
    "# cut-off.\n",
    "plt.axvline(perc_95,label='95t perc',c='m')\n",
    "p=x[10:int(mu)].mean()\n",
    "#plt.twiny()\n",
    "#Ploting Zipf distribution\n",
    "plt.plot(t[:1100],x_s[:1100]/(x.max()), c='b')\n",
    "#Ploting density function\n",
    "plt.plot(t[:1100],pdf[:1100]*factor, c='g')\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Must Important Words"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 137,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "terms : 67\n",
      "user : 24\n",
      "All important words: [[24, 'user'], [24, 'write'], [25, 'ability'], [25, 'ask'], [25, 'based'], [25, 'data'], [25, 'debate'], [25, 'democracy'], [25, 'develop'], [25, 'easily'], [25, 'images'], [25, 'libraries'], [25, 'meaning'], [25, 'napster'], [25, 'person'], [25, 'progress'], [25, 'publishing'], [25, 'regulate'], [25, 'sales'], [25, 'school'], [25, 'stories'], [25, 'text'], [25, 'whose'], [26, 'began'], [26, 'david'], [26, 'effective'], [26, 'effectively'], [26, 'generally'], [26, 'gets'], [26, 'increase'], [26, 'knowledge'], [26, 'limit'], [26, 'movie'], [26, 'office'], [26, 'really'], [26, 'rule'], [26, 'songs'], [26, 'sony'], [26, 'stations'], [26, 'strategy'], [26, 'web'], [27, 'already'], [27, 'becomes'], [27, 'certain'], [27, 'consider'], [27, 'extend'], [27, 'extension'], [27, 'given'], [27, 'interests'], [27, 'kahle'], [27, 'matter'], [27, 'produce'], [27, 'publisher'], [27, 'side'], [27, 'special'], [27, 'station'], [27, 'william'], [27, 'wrote'], [28, 'artist'], [28, 'copying'], [28, 'decision'], [28, 'defend'], [28, 'far'], [28, 'issue'], [28, 'laws'], [28, 'limits'], [28, 'lost'], [28, 'past'], [28, 'song'], [28, 'true'], [29, 'armstrong'], [29, 'commerce'], [29, 'courts'], [29, 'extended'], [29, 'imagine'], [29, 'little'], [29, 'monopoly'], [29, 'notes'], [29, 'practice'], [29, 'published'], [29, 'records'], [29, 'september'], [30, 'allowed'], [30, 'asked'], [30, 'burden'], [30, 'concentration'], [30, 'five'], [30, 'gives'], [30, 'microsoft'], [30, 'patent'], [30, 'privacy'], [30, 'rather'], [30, 'sold'], [31, 'buy'], [31, 'call'], [31, 'called'], [31, 'cd'], [31, 'drug'], [31, 'gave'], [31, 'last'], [31, 'response'], [31, 'students'], [31, 'thing'], [31, 'users'], [31, 'words'], [32, 'benefit'], [32, 'brothers'], [32, 'come'], [32, 'future'], [32, 'keep'], [32, 'mp'], [32, 'review'], [32, 'using'], [33, 'commons'], [33, 'done'], [33, 'extraordinary'], [33, 'fm'], [33, 'go'], [33, 'information'], [33, 'kids'], [33, 'process'], [33, 'says'], [33, 'sell'], [33, 'single'], [33, 'sound'], [34, 'americans'], [34, 'association'], [34, 'best'], [34, 'changed'], [34, 'clause'], [34, 'cost'], [34, 'created'], [34, 'described'], [34, 'granted'], [34, 'material'], [34, 'old'], [34, 'open'], [34, 'reason'], [34, 'regulated'], [34, 'requirement'], [34, 'scope'], [34, 'source'], [34, 'three'], [35, 'enough'], [35, 'great'], [35, 'mean'], [35, 'original'], [35, 'paid'], [35, 'since'], [35, 'u_s_'], [36, 'bill'], [36, 'built'], [36, 'cases'], [36, 'derivative'], [36, 'find'], [36, 'hard'], [36, 'ideas'], [36, 'infringement'], [36, 'range'], [36, 'registration'], [36, 'therefore'], [36, 'video'], [36, 'wanted'], [36, 'ways'], [37, 'bit'], [37, 'day'], [37, 'drugs'], [37, 'four'], [37, 'going'], [37, 'innovation'], [37, 'longer'], [37, 'second'], [37, 'statute'], [37, 'taking'], [37, 'things'], [37, 'valenti'], [38, 'aim'], [38, 'always'], [38, 'assure'], [38, 'back'], [38, 'company'], [38, 'ever'], [38, 'jesse'], [38, 'let'], [38, 'noncommercial'], [38, 'quite'], [38, 'university'], [38, '❙❚❘'], [39, 'argued'], [39, 'business'], [39, 'create'], [39, 'end'], [39, 'library'], [39, 'put'], [39, 'rules'], [39, 'ten'], [40, 'architecture'], [40, 'archive'], [40, 'competition'], [40, 'context'], [40, 'formalities'], [40, 'high'], [40, 'house'], [40, 'powerful'], [41, 'anything'], [41, 'better'], [41, 'type'], [42, 'cable'], [42, 'give'], [42, 'problem'], [42, 'produced'], [42, 'society'], [42, 'want'], [43, 'america'], [43, 'either'], [43, 'however'], [43, 'lawyer'], [43, 'making'], [43, 'networks'], [43, 'thought'], [44, 'doubt'], [44, 'makes'], [44, 'patents'], [44, 'press'], [44, 'someone'], [44, 'spread'], [44, 'told'], [45, 'constitution'], [45, 'money'], [46, 'limited'], [47, 'existing'], [47, 'less'], [47, 'publishers'], [47, 'said'], [48, 'author'], [48, 'enable'], [48, 'long'], [48, 'protected'], [48, 'real'], [49, 'around'], [49, 'become'], [49, 'believe'], [49, 'claim'], [49, 'form'], [49, 'value'], [49, 'year'], [50, 'computer'], [50, 'harm'], [50, 'justice'], [50, 'possible'], [50, 'still'], [51, 'authors'], [51, 'exclusive'], [51, 'record'], [52, 'build'], [52, 'copies'], [52, 'indeed'], [52, 'john'], [52, 'license'], [52, 'network'], [52, 'number'], [52, 'obvious'], [52, 'simple'], [53, 'course'], [53, 'general'], [53, 'nothing'], [53, 'supreme'], [54, 'clear'], [54, 'kind'], [54, 'means'], [54, 'read'], [54, 'view'], [55, 'increasingly'], [55, 'protect'], [55, 'television'], [55, 'though'], [56, 'another'], [56, 'creators'], [56, 'films'], [56, 'know'], [56, 'policy'], [56, 'times'], [56, 'understand'], [56, 'uses'], [57, 'file'], [57, 'instead'], [57, 'lawyers'], [57, 'million'], [58, 'books'], [59, 'else'], [60, 'costs'], [60, 'eldred'], [60, 'freedom'], [60, 'particular'], [60, 'protection'], [60, 'simply'], [62, 'code'], [62, 'must'], [62, 'riaa'], [63, 'balance'], [63, 'never'], [63, 'say'], [64, 'effect'], [64, 'question'], [64, 'regulation'], [65, 'companies'], [65, 'pay'], [65, 'york'], [66, 'artists'], [66, 'changes'], [66, 'copyrighted'], [66, 'need'], [66, 'story'], [67, 'set'], [67, 'terms']]\n"
     ]
    }
   ],
   "source": [
    "print (tupla1[-perc_05][1],\":\",tupla1[-perc_05][0])\n",
    "print (tupla1[-perc_95][1],\":\",tupla1[-perc_95][0])\n",
    "print ('All important words:', tupla1[len(tupla1)-perc_95:len(tupla1)-perc_05+1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This method is widely used in the generation of automatic summaries. \n",
    "It can be used to give weight to the words within the document, and \n",
    "extract the most weighted structures: sentences, paragraphs, etc. \n",
    "As can be seen in the example of the book \"Free Culture\", this method \n",
    "leaves 300 significant words from 120303 words in the original text."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## References\n",
    "\n",
    "[1] *[Bird2009]* Steven Bird, Ewan Klain & Edward Loper,. \n",
    "Book **Natural Language Processing with Python**. 2009. \n",
    "p. 10 **ISBN**: 978-0-596-51649-9\n",
    "\n",
    "[2] *[Luhn1958]* H.P. Luhn. Paper **The Automatic Creation of Literature Abstract**. \n",
    "*IBM Journal*, 1958."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Alphabetic Index\n",
    "\n",
    "**Token**: in linguistics is an individual occurrence of a linguistic unit, generally referred as the smallest unit of processing: words, phonemes, n-grams, etc."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}