HeartPole.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Writing a custom OpenAI Gym environment\n",
    "\n",
    "*Prescriptum:* this is a tutorial on writing a custom OpenAI Gym environment that dedicates an unhealthy amount of text to selling you on the idea that you need a custom OpenAI Gym environment.\n",
    "If you don't need convincing, [click here](#start). We assume decent knowledge of Python and next to no knowledge of Reinforcement Learning.\n",
    "\n",
    "Reinforcement Learning arises in contexts where an agent (a robot or a software system):\n",
    "- observes some data about its environment\n",
    "- takes action in this environment\n",
    "- receives rewards that indicate how  successful its actions have been\n",
    "\n",
    "Say you have [200 million subscribers who want to receive movie recommendations](https://medium.com/inside-machine-learning/recommendation-systems-using-reinforcement-learning-de6379eecfde) and if the recommendations are good, they reward you with thumbs up, watch time and renewing their subscription[<sup>2</sup>](#fn2). Or you need [a program that decides which advertisement to show to the user so that they click](https://arxiv.org/abs/1909.03602). Or you have a spare [walking robot that controls its joints to avoid the negative reward for falling over](https://arxiv.org/pdf/1812.11103.pdf). In such an environment you can forgo programming/developing a strategy for your system and instead use on of the [industry standard reinforcement learning algorithms](http://www.incompleteideas.net/book/the-book-2nd.html) to learn a good strategy by trial and error.\n",
    "\n",
    "These algorithms are very accessible - there are software packages available like [keras-rl](https://github.com/keras-rl/keras-rl) and, of course, [cibi](https://github.com/vadim0x60/cibi)[<sup>1</sup>](#fn1) - all that's left is to plug your problem into their code and watch the neural network slooooowly become an expert in solving your problem. These packages, however, aren't written for your marketing software. They are written for this:\n",
    "\n",
    "<table>\n",
    "    <tr>\n",
    "        <td><img src=\"HeartPole-stuff/MountainCar.gif\"></td>\n",
    "        <td><img src=\"HeartPole-stuff/FetchPickAndPlace.gif\"></td>\n",
    "    </tr>\n",
    "    <tr>\n",
    "        <td><a href=\"https://gym.openai.com/envs/MountainCarContinuous-v0/\">MountainCarContinuous-v0</a></td>\n",
    "        <td><a href=\"https://gym.openai.com/envs/FetchPickAndPlace-v0/\">FetchPickAndPlace-v0</a></td>\n",
    "    </tr>\n",
    "</table>\n",
    "\n",
    "The images above are visualizations of environments from [OpenAI Gym](https://github.com/openai/gym) - a python library used as defacto standard for describing reinforcement learning tasks. In addition to [an array of environments to play with](https://gym.openai.com/envs/), OpenAI Gym provides us with tools to streamline development of new environments, promising us a future [so bright you'll have to wear shades](https://www.youtube.com/watch?v=8qrriKcwvlY) where there's no need to solve problems. Just define the problem as an OpenAI environment - and *keras-rl* will do the solving."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<span id=\"fn1\"><sup>1</sup>The author of this tutorial is the author of [cibi](https://github.com/vadim0x60/cibi), but will pinky swear that this recommendation is unbiased if necessary</span><br>\n",
    "<span id=\"fn2\"><sup>2</sup>If you do, not sure why you are taking business advice from a free tutorial, but you're welcome</span>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## OpenAI Gym\n",
    "<a id='start'></a>\n",
    "\n",
    "\n",
    "But first, let us examine what a Gym environment looks like:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: gym in /home/vadim0x60/.pyenv/versions/3.7.4/lib/python3.7/site-packages (0.17.2)\n",
      "Requirement already satisfied: pyglet<=1.5.0,>=1.4.0 in /home/vadim0x60/.pyenv/versions/3.7.4/lib/python3.7/site-packages (from gym) (1.5.0)\n",
      "Requirement already satisfied: scipy in /home/vadim0x60/.pyenv/versions/3.7.4/lib/python3.7/site-packages (from gym) (1.3.1)\n",
      "Requirement already satisfied: numpy>=1.10.4 in /home/vadim0x60/.pyenv/versions/3.7.4/lib/python3.7/site-packages (from gym) (1.19.0)\n",
      "Requirement already satisfied: cloudpickle<1.4.0,>=1.2.0 in /home/vadim0x60/.pyenv/versions/3.7.4/lib/python3.7/site-packages (from gym) (1.2.2)\n",
      "Requirement already satisfied: future in /home/vadim0x60/.pyenv/versions/3.7.4/lib/python3.7/site-packages (from pyglet<=1.5.0,>=1.4.0->gym) (0.18.0)\n",
      "\u001b[33mYou are using pip version 19.0.3, however version 20.2.3 is available.\n",
      "You should consider upgrading via the 'pip install --upgrade pip' command.\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "!pip install gym"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import gym\n",
    "\n",
    "mountaincar = gym.make('MountainCarContinuous-v0') "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Gym environments follow the framework known as **Partially Observable Markov Decision Process** (source for [science historians](https://lup.lub.lu.se/search/publication/8867084), source [for dummies](http://cs.brown.edu/research/ai/pomdp/tutorial/)[<sup>3</sup>](#fn3)):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "location -0.5212944403172561, speed 0.0\n",
      "location -0.5214880751653233, speed -0.00019363484806713432\n",
      "location -0.5204104041127724, speed 0.001077671052550878\n",
      "location -0.518938315020942, speed 0.0014720890918303526\n",
      "location -0.5168326558420309, speed 0.002105659178911124\n",
      "location -0.5150076376709338, speed 0.0018250181710969947\n"
     ]
    }
   ],
   "source": [
    "observation = mountaincar.reset()\n",
    "location, speed = observation\n",
    "print(f'location {location}, speed {speed}')\n",
    "\n",
    "for step in range(5):\n",
    "    observation, reward, done, _ = mountaincar.step(mountaincar.action_space.sample())\n",
    "    location, speed = observation\n",
    "    print(f'location {location}, speed {speed}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The code above:\n",
    "1. Initializes the MountainCar environment with `mountaincar.reset()`\n",
    "2. Outputs the observed information (location and speed of the car)\n",
    "3. Picks a random action form the space of actions possible in this environment - *action space* - with `mountaincar.action_space.sample()`\n",
    "4. Sends this decision to the environment with `mountaincar.step()` \n",
    "5. Outputs the newly observed information\n",
    "6. Repeats steps 3-5 5 times"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Every environment has to follow this pattern. The environments only differ in their\n",
    "1. Action space\n",
    "2. Observation space\n",
    "3. Implementation of `env.step()` function\n",
    "\n",
    "One can explore the environment action and observation spaces as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Box(1,)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mountaincar.action_space"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([-1.], dtype=float32)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mountaincar.action_space.low"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1.], dtype=float32)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mountaincar.action_space.high"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([-0.3828633], dtype=float32)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mountaincar.action_space.sample()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Box(2,)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mountaincar.observation_space"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([-0.98118836, -0.06498322], dtype=float32)"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mountaincar.observation_space.sample()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Car location can range from -1.2000000476837158 to 0.6000000238418579\n",
      "Car speed can range from -0.07000000029802322 to 0.07000000029802322\n"
     ]
    }
   ],
   "source": [
    "location_low, speed_low = mountaincar.observation_space.low\n",
    "location_high, speed_high = mountaincar.observation_space.high\n",
    "\n",
    "print(f'Car location can range from {location_low} to {location_high}')\n",
    "print(f'Car speed can range from {speed_low} to {speed_high}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice how this space is just a pair (a numpy array of shape `(2,)` to be precise) of numbers with no indication what they mean. You can find out that the first number is car location and the second number is car speed is taken from the environment's [source code](https://gym.openai.com/envs/MountainCarContinuous-v0) or from the original [PhD thesis describing it](https://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-209.pdf). But the environment abstracts this away as an unimportant detail. That's because for a reinforcement learning algorithm it is an unimportant detail - it will learn how to interpret these numbers on its own oevr the course of training"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In total, there are 4 types of action/observation spaces ([the docs mention just the first two](https://gym.openai.com/docs/#spaces), but [the code never lies](https://github.com/openai/gym/tree/master/gym/spaces)):\n",
    "1. `Discrete(n)` - the action/observation is one of integer numbers $0,1,\\dots,n-1$  \n",
    "2. `Box(lower_bounds, upper_bounds)` where `lower_bounds` and `upper_bounds` are tensors of the same `shape`. The action/observation is a tensor (a numpy array) of shape `shape` and its components are real numbers in the `[lower_bound;upper_bound]` interval\n",
    "3. `MultiBinary(n)` - the action/observation is an array of length `n` where each component is 0 or 1\n",
    "4. `MultiDiscrete(nvec)` - the action/observation is an array of length `len(nvec)` and its k-th component is one of integer numbers $0,1,\\dots,\\text{nvec}_k-1$  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<span id=\"fn3\"><sup>3</sup>The author of this tutorial tends to identify with the latter category</span>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Meet HeartPole\n",
    "\n",
    "**Disclaimer** the healthcare scenario presented here is not in any way based on anyone's real life experience"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The go to environment for first experiments in Reinforcement Learning is [CartPole](https://gym.openai.com/envs/CartPole-v0/):"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"HeartPole-stuff/CartPole.gif\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The agent observes the position of the black cart and its velocity, as well as the angle of the pole and its angular velocity. The agent has to push the cart right or left so that the Pole doesn't fall.\n",
    "\n",
    "But what if instead of balancing a wooden stick on a cart you were balancing your health and your creative productivity?\n",
    "\n",
    "Consider the following facts (or, rather, 7 facts and 1 fiction):"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Fact 0**. Your productivity and health depends on factors like \n",
    "- being sleepy v.s. alert\n",
    "- blood pressure\n",
    "- whether you are drunk\n",
    "- how long ago has you last slept"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "state = {\n",
    "    'alertness': 0,\n",
    "    'hypertension': 0,\n",
    "    'intoxication': 0,\n",
    "    'time_since_slept': 10,\n",
    "    'time_elapsed': 0,\n",
    "    'work_done': 0\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Fact 1.** After a good night's sleep you wake up quite alert (but with some random variance)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "def wakeup(state):\n",
    "    state['alertness'] = np.random.uniform(0.7, 1.3)\n",
    "    state['time_since_slept'] = 0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'alertness': 1.139388569045376,\n",
       " 'hypertension': 0,\n",
       " 'intoxication': 0,\n",
       " 'time_since_slept': 0,\n",
       " 'time_elapsed': 0,\n",
       " 'work_done': 0}"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "wakeup(state)\n",
    "state"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Fact 2.**  Caffeine increases your alertness level and blood pressure"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "def drink_coffee(state):\n",
    "    state['alertness'] += np.random.uniform(0, 1)\n",
    "    state['hypertension'] += np.random.uniform(0, 0.3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'alertness': 1.9749997381422668,\n",
       " 'hypertension': 0.2799835932936571,\n",
       " 'intoxication': 0,\n",
       " 'time_since_slept': 0,\n",
       " 'time_elapsed': 0,\n",
       " 'work_done': 0}"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "drink_coffee(state)\n",
    "state"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Fact 3.** Alcohol decreases your alertness level, increases intoxication and blood pressure"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "def drink_beer(state):\n",
    "    state['alertness'] -= np.random.uniform(0, 0.5)\n",
    "    state['hypertension'] += np.random.uniform(0, 0.3)\n",
    "    # source https://attorneydwi.com/b-a-c-per-drink/\n",
    "    state['intoxication'] += np.random.uniform(0.01, 0.03)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'alertness': 1.735892851530049,\n",
       " 'hypertension': 0.36651870820839466,\n",
       " 'intoxication': 0.011394355564560548,\n",
       " 'time_since_slept': 0,\n",
       " 'time_elapsed': 0,\n",
       " 'work_done': 0}"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "drink_beer(state)\n",
    "state"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Fact 4.**  Hypertension is a risk factor for heart attacks"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "def sigmoid(x):\n",
    "    return 1 / (1 + np.exp(-x))\n",
    "\n",
    "def heart_attack_risk(hypertension, heart_attack_proclivity=0.5):\n",
    "    return heart_attack_proclivity * sigmoid(hypertension - 6)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x7fb76cbcb5d0>]"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xd4HPW1xvHvUbVcZLnIVe4F424jYyCNHhMSm1ACJISOU3DgAikkEIrJhQQIuUngcimhQ0wNGGJiamiB4IKbXIXc5Co3Sbasfu4fu1YUx5bXtkazWr2f59GjndnZ3XdtaY9mfjPnZ+6OiIgIQFLYAUREJH6oKIiISB0VBRERqaOiICIidVQURESkjoqCiIjUUVEQEZE6KgoiIlJHRUFEROqkhB3gYHXu3Nn79u0bdgwRkWZlzpw5W9w9+0DbNbui0LdvX2bPnh12DBGRZsXMVseynQ4fiYhInUCLgplNMLNlZpZvZtfv4/6LzazIzOZFvy4PMo+IiDQssMNHZpYM3AecAhQCs8xsursv3mvTZ919SlA5REQkdkHuKRwN5Lt7gbtXAtOASQG+noiIHKYgi0JPYG295cLour2dZWYLzOwFM+sVYB4RETmAsAeaXwX6uvtI4E3g8X1tZGaTzWy2mc0uKipq0oAiIi1JkEVhHVD/L/+c6Lo67r7V3Suiiw8DR+3ridz9QXfPdffc7OwDnmYrIiKHKMjrFGYBg8ysH5FicB7w7fobmFl3d98QXZwILAkwj4hIs1FdU8uG4nIKt+9m3Y7dFG4v46QhXRmR0z7Q1w2sKLh7tZlNAWYCycAj7p5nZlOB2e4+HbjKzCYC1cA24OKg8oiIxBt3Z0NxOfmbd7Jyyy5WbtlFwZZdrNm6i8Ltu6mu9bptzaBz2/TAi4K5+4G3iiO5ubmuK5pFpLkpKa9i8foSlm4oYcmGUpZuKuXzzTvZWVFdt03b9BT6dm5Nn05t6N2xNb07tqZXh9bkdMige1Yr0lOSD/n1zWyOu+ceaLtm1+ZCRCTelVfVkLe+mM/W7GDe2h0sWlfMqq1ldfd3aJ3KkG6ZnDW2JwO7tmNgdlsGdGlDdtt0zCzE5CoKIiKHrXh3FbNWbmPWqm38c+U28tYXU1UTOQrTMyuDET3bc/ZROQzr2Z6h3TPp0i78D//9UVEQETlIVTW1zF61nQ9WFPHR51tZWLiDWofUZGNUThaXfbE/Y3tnMbp3Fl3atQo77kFRURARicG2XZW8tXgT7yzdzIf5W9hZUU1ykjGmVxZTThzEsf07MaZ3Fq1SD/24fzxQURAR2Y/NpeXMWLCB1xdtZNaqbdQ6dG/fim+M6sEJR2Rz3MDOtE1PrI/RxHo3IiKHqbS8ihkLN/DKvPV8UrCVWofBXdty5QkD+eqwbgzrkRm34wGNQUVBRFo8d+eTgm08P3stMxZtoLyqln6d2zDlxEFMHNWdgV3ahR2xyagoiEiLVby7ipfmFvLUJ6v5vGgX7VqlcObYHM45KofRvbISeo9gf1QURKTFWbO1jEc+Wsmzs9ayu6qG0b2yuPucUXx9ZPdmP1B8uFQURKTFWLSumPv//jmvL9pAcpLxjVE9uOS4foG3jmhOVBREJOHNW7uDP769greXbqZdqxS+95UBXHxcX7pmNq9rCJqCioKIJKzlm0q582/LeGvJJrJap/LjUwdz4XF9yWyVGna0uKWiICIJZ2NxOXe/sYyX5hbSJi2F604ZzCVf7Jdw1xQEQf9CIpIwyqtqePiDAu5793Nq3Lnsi/344fED6dAmLexozYaKgogkhLcWb+KWV/Mo3L6bCcO6ccPpR9KrY+uwYzU7Kgoi0qxtKinnlul5vL5oI4O6tOWZy8dz3MDOYcdqtlQURKRZcnemzVrL7X9dQmVNLT/56hFc8aX+pKUEOfV84lNREJFmZ2NxOT97cQHvLS/iuAGduP2bI+jbuU3YsRKCioKINCuvzl/PDX9ZSGVNLVMnDeOC8X1ISmp57SiCoqIgIs1CWWU1t05fzLOz1zKmdxb3fGs0/bR30OhUFEQk7i3bWMqVz8zl86KdXHnCAP7r5MGkJmvsIAgqCiIS16bPX8/PXlhAm/QUnrx0PF8cpDOLgqSiICJxqbqmlt/8bSkPfbCS3D4d+N/vjKWLehUFTkVBROJOcVkVP3xmDh/lb+WiY/tww+lDdappE1FREJG4snrrLi55bBZrt5Vx19kjOSe3V9iRWhQVBRGJG7NWbWPyE7Nx4KnLxjO+f6ewI7U4KgoiEhdm5m3kR3/+jJysDB65eJwuRguJioKIhG7ap2v4xV8WMjIni0cvHqeupiFSURCRUN33bj53zVzGVwZnc/8FY2mdpo+lMOlfX0RC4e7c8+Zy/vhOPpNG9+Duc0bpgrQ4oKIgIk3O3fn160t54P0CzhvXi//+5giS1b8oLgRals1sgpktM7N8M7u+ge3OMjM3s9wg84hI+Nydqa8t5oH3C/juMX24XQUhrgRWFMwsGbgPOA0YCpxvZkP3sV074Grgn0FlEZH44O7c8fpSHv1oFZd+oR9TJw1Th9M4E+SewtFAvrsXuHslMA2YtI/tbgN+A5QHmEVE4sDv3lzOg+8XcNGxffjl14/ETAUh3gRZFHoCa+stF0bX1TGzsUAvd/9rgDlEJA7c924+f3gnn/PG9eLmbwxTQYhToQ31m1kScA9wXQzbTjaz2WY2u6ioKPhwItKonvpkNXfNXMYZo3vw398coUNGcSzIorAOqN+0JCe6bo92wHDg72a2CjgGmL6vwWZ3f9Ddc909Nzs7O8DIItLYXl+4gV++sogTh3ThrnNGaVA5zgVZFGYBg8ysn5mlAecB0/fc6e7F7t7Z3fu6e1/gE2Ciu88OMJOINKGPP9/K1dPmMaZXFvd9e6yuQ2gGAvsfcvdqYAowE1gCPOfueWY21cwmBvW6IhIflm0sZfITs+nTqTWPXDyOjLTksCNJDAK9eM3dZwAz9lp30362PT7ILCLSdDaXlnPpY7NonZ7M45ceTVZr9TJqLnRFs4g0qt2VNVz++Gy2l1Xy3PeOpUdWRtiR5CDoAJ+INJraWue/nv2MReuK+cN5Yxjes33YkeQgqSiISKO5583lzMzbxI2nD+XkoV3DjiOHQEVBRBrFawvWc++7+Zx/dC8u+ULfsOPIIVJREJHDlre+mB8/P5/cPh24deJwXa3cjKkoiMhh2bqzgslPzKFD6zTuv+Ao0lL0sdKc6ewjETlkNbXO1dPmUbSzghe/fxzZ7dLDjiSHSSVdRA7Z/7y1nA/zt/CrScMZkaMzjRKBioKIHJK3l2zij+/kc25uL741rteBHyDNgoqCiBy0tdvKuObZeQzrkcmtk4aFHUcakYqCiByUyupapjwzFwfu/85RtEpVT6NEooFmETkod81cyvzCYv7vgrH07tQ67DjSyLSnICIxe2fpJh76YCUXHtuHCcO7hx1HAqCiICIx2VC8m+uem8/Q7pn84mtHhh1HAqKiICIHVFvrXPvsfCqqa7n322M0jpDANKYgIgf00AcFfFywlTvPGkn/7LZhx5EAaU9BRBq0aF0xd7+xjNOGd+Oc3Jyw40jADlgUzGzqXsvJZvZ0cJFEJF7srqzhqmmf0alNOnecOUKN7lqAWPYUepnZzwHMLB14CVgRaCoRiQu3z1hCQdEufvutUZpSs4WIpShcCoyIFoZXgXfd/ZZAU4lI6D5YUcSTn6zm0i/04wsDO4cdR5rIfgeazWxsvcXfAw8AHwHvm9lYd58bdDgRCUfx7ip+8vwCBnZpy08nHBF2HGlCDZ199Nu9lrcDQ6PrHTgxqFAiEq5bp+dRtLOCBy9UG4uWZr9Fwd1PaMogIhIfZuZt5KXP1nHVSYMYmZMVdhxpYrGcfXS1mWVaxMNmNtfMTm2KcCLStLbvquSGvyxkWI9MfnTiwLDjSAhiGmh29xLgVKAT8F3g14GmEpFQ3PJqHsW7q7j7nFGkJusyppYolv/1PScmfw14wt3z6q0TkQTxRt5GXpm3niknDOLI7plhx5GQxFIU5pjZG0SKwkwzawfUBhtLRJrSjrJKbnh5EUO7Z/LDEwaEHUdCFEvvo8uA0UCBu5eZWSfgkmBjiUhTmvraYrbvquSxS8bpsFEL19B1CkPcfSmRggDQX5e4iySe95YX8dLcdfzoxIEM69E+7DgSsob2FK4FJvOf1yuArlMQSQi7Kqr5xUsLGZDdhik620ho+DqFyWaWBNzo7h81YSYRaSJ3v7GMdTt28/z3jyU9RRepyQEGmt29Fri3ibKISBOau2Y7j/1jFd89pg/j+nYMO47EiVhGlN42s7PsEAYUzGyCmS0zs3wzu34f93/fzBaa2Twz+9DMhh7sa4jIwauqqeXnLy6kW2Yr9TaSfxNLUfge8DxQYWYlZlZqZiUHepCZJQP3AacR6Zl0/j4+9J9x9xHuPhq4E7jn4OKLyKF48P0Clm0qZeqk4bRrlRp2HIkjBzwl1d3bHeJzHw3ku3sBgJlNAyYBi+s9d/3i0obIALaIBGjVll38/u0VnDa8G6cM7Rp2HIkzQc7R3BNYW2+5EBi/90ZmdiWRM53S2M8ZTWY2mciZUPTu3bvRg4q0FO7ODS8vJD05iVsmDgs7jsSh0K9Scff73H0A8DPgxv1s86C757p7bnZ2dtMGFEkgf/lsHR/lb+WnE46ga2arsONIHAqyKKwDetVbzomu259pwBkB5hFp0XaUVfKrvy5hTO8svjO+T9hxJE7F0jr7sn2si6VL6ixgkJn1M7M04Dxg+l7PM6je4ulo7meRwPz69aUU767i9m+OIClJ3Qlk32IZUzjLzMrd/WkAM7sPOOB+p7tXm9kUYCaQDDzi7nlmNhWY7e7TgSlmdjJQRWRmt4sO9Y2IyP7NXrWNabPWMvnL/dUBVRoUU1EApptZLTAB2OHu/7H3sC/uPgOYsde6m+rdvvogsorIIaiqqeWGvyyiR/tWXH3SoAM/QFq0hhri1b/E8XLgZeAj4FYz6+ju24IOJyKH708frmTZplIeujCXNulBnnAoiaChn5A5RK4bsHrfT49+OdA/8HQiclgKt5fx+7dWcPKRXXVNgsSkoYZ4/ZoyiIg0vqmvRq4VvWWiOshIbGI5++hKM8uqt9zBzH4YbCwROVxvL9nEG4s3cdVJg8jp0DrsONJMxHKdwhXuvmPPgrtvB64ILpKIHK7dlTXcPD2PgV3actkXtdMvsYulKCTX75AabXSXFlwkETlc//v3fAq37+a2ScNJSwm9cYE0I7GcivA34FkzeyC6/L3oOhGJQwVFO3ngvQLOGN2DYwd0CjuONDOxFIWfESkEP4guvwk8HFgiETlk7s7N0/NIT0niF6cfGXYcaYZiaZ1dC9wf/RKRODZj4UY+WLGFW74xlC7t1PBODt4Bi0K0P9EdRCbKqfspc3ddpyASR3ZWVHPba4sZ2j2TC45Rwzs5NLGMQD1KZC+hGjgBeAJ4KshQInLwfv/WcjaWlHPbGcNJSdbgshyaWH5yMtz9bcDcfbW730LkqmYRiRPLN5Xy6EerODe3F0f16RB2HGnGYhlorjCzJGBFtOvpOqBtsLFEJFbuzo0vL6JtqxR+dtqQsONIMxfLnsLVQGvgKuAo4ALgwiBDiUjsXpm3nk9XbuOnXx1Cxza6hEgOTyxFoa+773T3Qne/xN3PAjRRskgcKCmv4ld/XcKonPacO67XgR8gcgCxFIWfx7hORJrY795cztZdFdx2xnCSNZuaNIKG5lM4Dfga0NPM/lDvrkwiZyKJSIgWry/hiY9X8+2jezMyJ+vADxCJQUMDzeuB2cBEInMr7FEKXBNkKBFpWG2t88tXFtE+I5WffPWIsONIAmloPoX5wHwz6+ruj9e/z8yuBn4fdDgR2bcX5xYyZ/V27jxrJFmtNbgsjSeWMYXz9rHu4kbOISIxKi6r4tevL2Vs7yzOPion7DiSYBoaUzgf+DbQz8ym17urHaD5mUVCctcbS9leVskTlx1NkgaXpZE1NKbwD2AD0Bn4bb31pcCCIEOJyL4tLCzm6X+u4aJj+zKsR/uw40gCamhMYTWwGji26eKIyP7U1Do3vryQTm3SufbUwWHHkQQVyxzNx5jZLDPbaWaVZlZjZiVNEU5E/uXPn65hfmExv/z6kWS2Sg07jiSoWAaa7wXOB1YAGcDlwH1BhhKRf7dlZwV3/m0px/bvxMRRPcKOIwkspv667p4PJLt7jbs/CkwINpaI1HfHjKXsrqrhtjOGUW/KdJFGF0uX1DIzSwPmmdmdRAaf1axdpIn8s2ArL84t5IfHD2Bgl3Zhx5EEF8uH+3ej200BdgG9gLOCDCUiEZXVtdz48iJ6ZmUw5cSBYceRFiCWOZpXR2+WA7cGG0dE6nv4wwJWbN7JIxfn0jotlh17kcOjw0AicWrttjL+8PYKJgzrxolDuoYdR1oIFQWROOTu3PTKIpLNuHni0LDjSAsSy3UK58Sybj+PnWBmy8ws38yu38f915rZYjNbYGZvm1mf2GKLJLa/LdrIu8uKuOaUwXRvnxF2HGlBAptkx8ySiVzPcBowFDjfzPb+k+czINfdRwIvAHfGkEckoZWUV3Hz9DyGds/k4uP6hh1HWpggJ9k5Gsh394Lo800DJgGL92zg7u/W2/4TIvM/i7Rod89cRtHOCh66MJeUZB3hlaYV5CQ7PYG19ZYLgfENbH8Z8HoMzyuSsD5bs50nP1nNRcf2ZVQvzaYmTa/BSXbMbBHw1b0n2WlsZnYBkAt8ZT/3TwYmA/Tu3TvIKCKhqaqp5ecvLaRru1Zcp4Z3EpIG903dvQboFb2i+WCtI3Kh2x450XX/xsxOBm4AJrp7xX5yPOjuue6em52dfQhRROLfnz5cydKNpdwycRjt1PBOQhLL1TArgY+iE+3s2rPS3e85wONmAYPMrB+RYnAekUl76pjZGOABYIK7bz6Y4CKJZOWWXfzuzeV8dVhXJgzvFnYcacFiKQqfR7+SiMy6FhN3rzazKcBMIBl4xN3zzGwqMNvdpwN3AW2B56NNvta4+8SDfA8izZq78/OXFpCWksTUScPDjiMtXCxtLg65tYW7zwBm7LXupnq3Tz7U5xZJFM/NXssnBdu448wRdM1sFXYcaeEOWBTMLBv4KTAMqPuJdfcTA8wl0iJsLi3nv/+6hPH9OnJubq8DP0AkYLGcBP00sBToR6Qh3ioi4wUichjcnV++vIjy6lruOHMESUmaJ0HCF0tR6OTufwKq3P09d78U0F6CyGH668INzMzbxLWnDKZ/dtuw44gAsQ00V0W/bzCz04lc1NYxuEgiiW/rzgpufiWPUTntufyL/cKOI1InlqLwKzNrD1wH/JFIm4tYrmgWkf245dXFlJRXcefZx6iVhcSVWM4+ei16sxg4Idg4IolvZt5GXp2/nmtPGcwR3TS9psSXWFpnD462tV4UXR5pZjcGH00k8WzdWcENf1nI0O6Z/OD4AWHHEfkPsey3PkSkVXYVgLsvIHJ1sogcBHfnxpcXUby7invOHUWqDhtJHIrlp7K1u3+617pYWmeLSD3T56/n9UUbueaUwQzplhl2HJF9iqUobDGzAYADmNnZwIZAU4kkmE0l5dz0Sh5jemcx+Uv9w44jsl+xnH10JfAgMMTM1hFpkKfJcERi5O789IUFVFTX8NtzRulsI4lrsZx9VACcbGZtgCR3Lw0+lkjieOLj1by3vIjbJg3TRWoS92LpfZQOnAX0BVKi3Uxx96mBJhNJACs2lXL7jCUcf0Q2FxzTJ+w4IgcUy+GjV4hcozAH2OckOCLynyqra7l62jzapKdw59kj2fMHlUg8i6Uo5Lj7hMCTiCSY376xjMUbSnjowly6tFNLbGkeYhnx+oeZjQg8iUgCeW95EQ+8X8C3x/fmlKFdw44jErP97imY2UIip6GmAJeYWQGRw0cGuLuPbJqIIs3L5tJyrntuHkd0bcdNXx8adhyRg9LQ4aOvN1kKkQRRW+tc++x8dlZU88wVx9AqNTnsSCIHZb9Fwd1XN2UQkUTwf+9/zof5W7jjzBEM7qpmd9L86CoakUbyScFW7p65jNNHdue8cZpaU5onFQWRRrC5pJwpz3xG385t+M1ZOv1Umq9YTkkVkQZU19Qy5c+fsauimqcvH0/bdP1aSfOln16Rw3TXG8v4dOU27vnWKE2aI82eDh+JHIbXFqzngfcK+M743pw5NifsOCKHTUVB5BAtXl/CT55fQG6fDtz8jWFhxxFpFCoKIodg+65KJj85m8yMFP73grGkpehXSRKDxhREDlJVTS1XPjOXzSUVPPf9Y9XXSBKKioLIQXB3bnplEf/4fCt3nzOK0b2ywo4k0qi0zytyEB7+YCV//nQtPzx+AGcfpYFlSTwqCiIxeiNvI7e/voSvjejGj089Iuw4IoFQURCJwby1O7h62jxG9mzPb88ZTVKSrliWxBRoUTCzCWa2zMzyzez6fdz/ZTOba2bVZnZ2kFlEDlVB0U4ufWwWndul8dBFuWSkqfOpJK7AioKZJQP3AacBQ4HzzWzv5vJrgIuBZ4LKIXI4NpeWc+Ejn2LAE5eO15lGkvCCPPvoaCDf3QsAzGwaMAlYvGcDd18Vva82wBwih6R4dxUXPzKLrTsrmTb5GPp1bhN2JJHABXn4qCewtt5yYXSdSNzbVVHNpY/NYsXmUu6/YCyjdOqptBDNYqDZzCab2Wwzm11UVBR2HElw5VU1XPHEbD5bs50/nDeG44/oEnYkkSYTZFFYB9SfaSQnuu6gufuD7p7r7rnZ2dmNEk5kXyqra7ny6bl1F6edNqJ72JFEmlSQRWEWMMjM+plZGnAeMD3A1xM5LBXVNfzgqTm8vXQzvzpjuLqeSosUWFFw92pgCjATWAI85+55ZjbVzCYCmNk4MysEzgEeMLO8oPKINKS8qobvP/mvgnDBMX3CjiQSikB7H7n7DGDGXutuqnd7FpHDSiKh2V1Zw/eemsP7y4u448wRnH9077AjiYRGDfGkRSveXcXlj89i9urt3HnWSL41rteBHySSwFQUpMXaXFrORY/MIn9zKfeeP5bTR2pQWURFQVqk1Vt3ceEjn7K5pII/XTSOLw/WWW0ioKIgLdCc1du44ok51Lrz9BXjGdu7Q9iRROKGioK0KK/OX891z8+nZ1YGj1w8Tq0rRPaioiAtQm2t84d3VvA/b61gXN8OPPjdXDq0SQs7lkjcUVGQhFdaXsU1z87nrSWbOHNsT27/5ghapar9tci+qChIQsvfXMr3npzDqq1l3PyNoVx8XF/MNEGOyP6oKEjCenFOITe+vIjWack8ddl4jh3QKexIInFPRUESTlllNTe9kscLcwoZ368jfzh/DF0zNTmOSCxUFCShzF2zneuem8+qrbu46sSBXHXSIFKSm0WHeJG4oKIgCaGyupY/vrOC+97Np3v7DJ65/BgdLhI5BCoK0uzNW7uD619cwNKNpZx9VA43fWMoma1Sw44l0iypKEiztauimnveXM6jH60ku106D12YyylDu4YdS6RZU1GQZsfdeW3BBm6fsYQNxeVccExvfjphiPYORBqBioI0K4vXl3Drq3n8c+U2hnbP5N5vj+GoPh3DjiWSMFQUpFlYt2M397yxnJc+K6R9Riq/OmM45x/dm+QkXYgm0phUFCSubdlZwQPvfc7jH68G4Iov9eeHxw8gq7X6FokEQUVB4tKWnRU8+H4BT368morqGr45JodrTx1Mz6yMsKOJJDQVBYkrq7bs4qEPCnhhTiFVNbWcMbonU04cSP/stmFHE2kRVBQkdO7OJwXbePwfq5i5eCOpSUmcObYnk7/cX8VApImpKEhoSsqreGXeep78eBXLN+0kq3UqP/jKAC4+ri9d1KtIJBQqCtKkamudT1Zu5fnZhcxYuIGK6lqG98zkzrNHMnFUD81zIBIyFQUJnLuTt76E6fPXM33eejaWlNMuPYWzj8rh3HG9GNGzveY4EIkTKgoSiNpaZ17hDmYu2sjrizayZlsZKUnGVwZn8/OvDeHUod3ISNNegUi8UVGQRrOjrJKP8rfyztLNvLd8M1t2VpKabBw3oDM/OH4AE4Z107zIInFORUEOWVllNXNWb+eTgq18uGILC9YV4w7tM1L5yuBsThzShROGdKF9hnoSiTQXKgoSs00l5cxdvZ25a7Yze/V2FhYWU13rJCcZY3plcdWJg/jSoM6M7pWliW1EmikVBfkP7s7GknKWbCghb10JC9YVs7CwmI0l5QCkpSQxomd7Jn+5P0f360hu3460TdePkkgi0G9yC+buFJVW8HnRLvKLdrJiUynLNpaybFMpO8qq6rbrn92GY/p3ZEROFmN7ZzG0RybpKRokFklEKgoJrrK6lg3FuyncvpvC7WWs3lrG6m1lrN66i1VbythZUV23bdv0FAZ1bcuEYd04sntm9Ksd7TRPgUiLoaLQTNXUOjvKKinaWUFRaQWbSyrYVFrO5pIKNhaXs6GknA07dlO0swL3fz0uJcnI6ZBB705tOKp3B/pnt6V/dhsGZLele/tWul5ApIULtCiY2QTg90Ay8LC7/3qv+9OBJ4CjgK3Aue6+KshM8aayupbS8ip2VlRTWl5N8e4qSnZXURz92rG7ih1lVewoq2Tbrn99bS+rpNb/8/natUqhS7t0emRlcMQR2XRvn0FOhwxyOrQmp0MG3du30iCwiOxXYEXBzJKB+4BTgEJglplNd/fF9Ta7DNju7gPN7DzgN8C5QWWKRU2tU1VTS2VNLVXVe747lTU1VFTXUhn9qqj7qqG8qpbyqhrKqyLb7K6M3C6rqmF3ZQ1lldWUVdawqyLyfWdFNbsqqtlVUUNlTW2DeVKSjKzWqXRonUaHNmn0z27DuH4d6dwmjY5t0uiS2Yrsdul0bptOl3bptNGAr4gchiA/QY4G8t29AMDMpgGTgPpFYRJwS/T2C8C9Zmbuvo+/gQ/Pc7PW8sD7n0c/9J3q2lqqa5zqWqe6ppaqaDFojFdOMshITSYjLYWMtCRap6aQkZZM2/QUOreNfHC3SU+mTXoK7dJTaNcqlbbpKbRrlUJmRiqZrVLJzEghq3UabdKSdUhHRJpMkEWhJ7C23nIhMH59TzFKAAAHLklEQVR/27h7tZkVA52ALfU3MrPJwGSA3r17H1KYrNapDOmWSUqykZxkpCYlkZJspCQZKclJpCYnkZb8r9upyRZZl5JEWnISqdHv6SmRdekpSbRKTSYtJYlWKcm0Sk0iPTWZjNRkUpNNH+Qi0iw1i2MN7v4g8CBAbm7uIf0tf+qwbpw6rFuj5hIRSTRBjjiuA3rVW86JrtvnNmaWArQnMuAsIiIhCLIozAIGmVk/M0sDzgOm77XNdOCi6O2zgXeCGE8QEZHYBHb4KDpGMAWYSeSU1EfcPc/MpgKz3X068CfgSTPLB7YRKRwiIhKSQMcU3H0GMGOvdTfVu10OnBNkBhERiZ2uYhIRkToqCiIiUkdFQURE6qgoiIhIHWtuZ4CaWRGw+hAf3pm9rpZuAfSeWwa955bhcN5zH3fPPtBGza4oHA4zm+3uuWHnaEp6zy2D3nPL0BTvWYePRESkjoqCiIjUaWlF4cGwA4RA77ll0HtuGQJ/zy1qTEFERBrW0vYURESkAS2mKJjZBDNbZmb5ZnZ92HmCZma9zOxdM1tsZnlmdnXYmZqCmSWb2Wdm9lrYWZqCmWWZ2QtmttTMlpjZsWFnCpqZXRP9mV5kZn82s1ZhZ2psZvaImW02s0X11nU0szfNbEX0e4cgXrtFFIV680WfBgwFzjezoeGmClw1cJ27DwWOAa5sAe8Z4GpgSdghmtDvgb+5+xBgFAn+3s2sJ3AVkOvuw4l0YE7E7sqPARP2Wnc98La7DwLeji43uhZRFKg3X7S7VwJ75otOWO6+wd3nRm+XEvmw6BluqmCZWQ5wOvBw2Fmagpm1B75MpAU97l7p7jvCTdUkUoCM6MRcrYH1IedpdO7+PpHpBOqbBDwevf04cEYQr91SisK+5otO6A/I+sysLzAG+Ge4SQL3P8BPgdqwgzSRfkAR8Gj0kNnDZtYm7FBBcvd1wN3AGmADUOzub4Sbqsl0dfcN0dsbga5BvEhLKQotlpm1BV4E/svdS8LOExQz+zqw2d3nhJ2lCaUAY4H73X0MsIuADinEi+hx9ElECmIPoI2ZXRBuqqYXnaEykFNHW0pRiGW+6IRjZqlECsLT7v5S2HkC9gVgopmtInJ48EQzeyrcSIErBArdfc8e4AtEikQiOxlY6e5F7l4FvAQcF3KmprLJzLoDRL9vDuJFWkpRiGW+6IRiZkbkWPMSd78n7DxBc/efu3uOu/cl8v/7jrsn9F+Q7r4RWGtmR0RXnQQsDjFSU1gDHGNmraM/4yeR4IPr9dSf0/4i4JUgXiTQ6Tjjxf7miw45VtC+AHwXWGhm86LrfhGdIlUSx4+Ap6N/7BQAl4ScJ1Du/k8zewGYS+QMu89IwCubzezPwPFAZzMrBG4Gfg08Z2aXEekU/a1AXltXNIuIyB4t5fCRiIjEQEVBRETqqCiIiEgdFQUREamjoiAiInVUFCShmFnf+p0lm+D1RpvZ1wJ43qlmdnJjP6/IgbSI6xREghBtyDYayAUa9foPd7+pMZ9PJFbaU5BElGxmD0V77r9hZsPMbO6eO81s0J5lM1tlZnea2UIz+9TMBkbXZ5vZi2Y2K/r1hej6W8zsSTP7CHgSmAqca2bzzOxcM2sT7YX/abRJ3aTo4y42s5fM7G/Rfvh3Rtcnm9lj0bkBFprZNdH1j5nZ2dHbJ0Wfa2H0udPrZb/VzOZG7xvSZP/CkrBUFCQRDQLuc/dhwA4iHWKLzWx09P5LgEfrbV/s7iOAe4l0WoXIPAW/c/dxwFn8ezvuocDJ7n4+cBPwrLuPdvdngRuItNg4GjgBuKte59LRwLnACCKFpFd0XU93Hx7NUD8X0QlkHgPOjd6fAvyg3iZb3H0scD/w44P9hxLZm4qCJKKV7r6ntcccoC+RD/VLohMunQs8U2/7P9f7vmfmspOBe6MtQqYDmdGOswDT3X33fl77VOD66OP+DrQCekfve9vdi929nEiPoj5EWlP0N7M/mtkEYO9OtkdE38/y6PLjROZQ2GNPo8M971PksGhMQRJRRb3bNUAGkW6xNwPvAHPcfWu9bXwft5OAY6If4HUiPdjY1cBrG3CWuy/b63Hj95Erxd23m9ko4KvA94n0s7m0wXf37/Y8Zw36fZZGoD0FaRGiH+4ziRxmeXSvu8+t9/3j6O03iDSbAyJnGe3nqUuBdvWWZwI/inbwxMzGNJTLzDoDSe7+InAj/9n6ehnQd89YB5Emh+819Jwih0NFQVqSp4nMyrb3TF0dzGwBkfmdr4muuwrINbMFZraYyF/x+/IuMHTPQDNwG5AKLDCzvOhyQ3oCf48ebnoK+Hn9O6PF7BLgeTNbGM3/fwd+qyKHRl1SpcUwsx8D7d39l/XWrSIyCfyW0IKJxBEdg5QWwcz+AgwATgw7i0g8056CiIjU0ZiCiIjUUVEQEZE6KgoiIlJHRUFEROqoKIiISB0VBRERqfP/ZPFfUr4DnqUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "xspace = np.linspace(0, 10, 100)\n",
    "plt.xlabel('hypertension')\n",
    "plt.ylabel('heart attack risk')\n",
    "plt.plot(xspace, heart_attack_risk(xspace))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def heart_attack_occured(state, heart_attack_proclivity=0.5):\n",
    "    return np.random.uniform(0, 1) < heart_attack_risk(state['hypertension'], heart_attack_proclivity)\n",
    "heart_attack_occured(state)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Fact 5.** Hypertension and intoxication decay exponentially over time, with a half-life of around 12 hours (24 half-hours)\n",
    "\n",
    "**Fact 6.** Alertness decays if you haven't slept for a long time"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "def alertness_decay(time_since_slept):\n",
    "    return sigmoid((time_since_slept - 40) / 10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x7fb76a8a0d10>]"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3XecXFX5x/HPsz2btimbuqmkkV6WBFCagATEUJWqgEgsgFgRRUGaPxVsCIJ0gxSRGiESBKI0Jb0XsukJKbupW7L9+f0xN+u6pkySvbm7M9/36zWvnVvmznP37s4z55x7zjF3R0REBCAl6gBERKTpUFIQEZE6SgoiIlJHSUFEROooKYiISB0lBRERqaOkICIidZQURESkjpKCiIjUSYs6gIPVsWNH7927d9RhiIg0K7NmzSpy99wD7dfskkLv3r2ZOXNm1GGIiDQrZrYmnv1UfSQiInWUFEREpI6SgoiI1FFSEBGROqElBTN7zMy2mNnCfWw3M7vXzArMbL6ZjQ4rFhERiU+YJYUngPH72X4m0D94TAQeCDEWERGJQ2hJwd3fAbbtZ5dzgEke828gx8y6hhWPiIgcWJT9FLoD6+otrw/WbYwmHBGRxlFT61RU11BRVUtFdS2V1bVU1tTUPa+q8eBnLZU1sZ+xh1Nd41TXxp7X1P10qmtqOfXozozokRNq7M2i85qZTSRWxUTPnj0jjkZEEkl5VQ3F5dXsKq+iuLya4vIqSsqrKa6opqS8mtKKakorayirrKa0IvZzd1UNZZU1lFfVsLuyht1VseflVbWUV9VQXeuhxNqpTVZCJ4UNQI96y3nBuv/h7g8BDwHk5+eH89sWkWavttbZVlZJYXEFRSUVbC2ppKikgm2llWwvq4z9LK1ix+5KdpRVsXN3FRXVtQc8bkZaCtkZqWSnp5KdmUZ2Riot0lNp3zKD7HapZKUHj7RUMtNTyExLISs9lcy0FDLTUslIS4k9UmPbMtJSSEuxuvXpqbFHWorFfqYa6SkppKZa3boUAzML/XcYZVKYDFxnZs8C44Cd7q6qIxHZq9pap7CkgvXby1i/fTcbd5bz8Y7dbNpZzuZd5WzeVUFhSQU1e/mWnpZitGuZQfvsDHKy0+nTsSU5LTJom51O2xbptMlKo02LdFpnpdE6K51WmWm0ykyjdVYaLTPTSE9Nnrv3Q0sKZvYMcDLQ0czWA7cC6QDu/iAwBTgLKADKgKvCikVEmofaWufjnbtZWVjKysISVm8tY83WUtZsK2P9tt1U1vz3t/rWWWl0bZtF5zZZDOjcmk5tMunUOovc1pl0bJVJx1YZdGiZSZsWaUfkW3YiCC0puPslB9juwLVhvb+ING2FxRUs2biLJRt3sWxzMcs3l1CwpYTdVTV1+7TMSKVXh5YM6tKa04/uTF77bPLatSAvpwVdc1rQKrNZNIs2K/qNikjotpdWMnfdDuat38GC9TuZv2EnhcUVdds7t8lkQOfWXDK2J/06taJvbkv65rYkt1WmvuEfYUoKItKo3J2128r4cOU2pq/exuw121lZVAqAGfTLbcUJ/ToyuFsbBndrw9Fd2tCuZUbEUcseSgoicti2FJfzQcFW3l1exAcriti4sxyAdtnpjOnVngvz8xjVox3D8tqqyqeJ09URkYNWW+vM37CTt5ds5u1lW1i4YRcAOdnpHH9UB77etwPj+nagf6dWqv5pZpQURCQuNbXOh6u2MnXhJqYu2symXeWkGIzu2Y7vnTGQE/vnMqRbG1JSlASaMyUFEdknd2f++p28Mvdj/jr/YwqLK8hKT+GkAbncOGQgpwzspPaABKOkICL/o6ikghdnr+fPM9axorCUjNQUThmUy4QR3TllUC7ZGfroSFS6siICxEoF01dtY9K/1jB10Saqa53RPXP4v/OHcdbQrrTNTo86RDkClBREklx5VQ0vz9nAEx+sZummYtq2SOfK43tz8dge9OvUOurw5AhTUhBJUjt3V/Gnf6/hiQ9WU1hcweCubfjFBcP57IhutMhIjTo8iYiSgkiS2VlWxaPvr+Lx91ZRXFHNiQNy+c1FfTn+qA66fVSUFESSRUlFNY+8u5JH340lg/FDunD9qf0Y0q1t1KFJE6KkIJLgqmpqeXb6Wn771nKKSioZP6QLN5zWn6O7tok6NGmClBREEtg/lm3h9r8uZmVRKWP7tOfhLw5iVM92UYclTZiSgkgCWru1jNtfXcybSzbTp2NLHvliPqce3UltBnJASgoiCaS6ppZH3lvFr//+EakpxvfHD+JLn+xNZpruJpL4KCmIJIgF63fy/Rfms3jjLs4Y0pnbJgylS9usqMOSZkZJQaSZq6qp5b63C7hvWgEdWmbw4OWjGT+0a9RhSTOlpCDSjBVsKeHbz81l/vqdnD+qO7dOGELbFhqOQg6dkoJIM+Tu/HnGOm6dvIjsjFQeuGw0Zw5T6UAOn5KCSDNTUlHNzS8t4JW5H/PJfh351UUj6NRabQfSOJQURJqRjzYX89UnZ7F6aynfOX0AXz+lH6ma1EYakZKCSDPx+sKNfOe5ebTISOPpa47l2L4dog5JEpCSgkgTV1vr/ObNj7j37QJG9sjhwcvH6FZTCY2SgkgTVl5Vw7efm8uUBZv4fH4ed5w7VB3RJFRKCiJNVFFJBddMmsncdTv40WeO5upP9tEwFRI6JQWRJmhlYQlXPD6dwuIKHrhsDOOHdok6JEkSSgoiTczCDTu54rHpADw78ThG9siJOCJJJkoKIk3Iv1Zs5ZpJM2nbIp0nrx5L39xWUYckSUZJQaSJeGvJZr721Gx6tc/myavH6Q4jiYSSgkgT8MaiTVz79GyO7tqGP141lnYtM6IOSZJUSpgHN7PxZrbMzArM7Ka9bO9pZtPMbI6ZzTezs8KMR6Qpen3hJr7+1GwGd2vLk1ePU0KQSIWWFMwsFbgfOBMYDFxiZoMb7PYj4Dl3HwVcDPw+rHhEmqLXF27k2qdnMzyvLU9ePVYjnErkwiwpjAUK3H2lu1cCzwLnNNjHgT2zh7cFPg4xHpEmZdqyLVz/zBxG5LVl0tXjaJOlhCDRC7NNoTuwrt7yemBcg31+ArxhZtcDLYHTQoxHpMn414qtfPXJWQzo3JrHrxpLq0w170nTEGqbQhwuAZ5w9zzgLOBJM/ufmMxsopnNNLOZhYWFRzxIkcY0d90OvvzHGfQM7jJSlZE0JWEmhQ1Aj3rLecG6+q4GngNw938BWUDHhgdy94fcPd/d83Nzc0MKVyR8KwpLuOrx6XRolcmfvjyO9mpUliYmzKQwA+hvZn3MLINYQ/LkBvusBU4FMLOjiSUFFQUkIW3ZVc4Vj00nxYxJXxpL5zbqhyBNT2hJwd2rgeuAqcASYncZLTKz281sQrDbd4BrzGwe8Axwpbt7WDGJRKW4vIorHp/BttJKHr/qGHp3bBl1SCJ7FWrrlrtPAaY0WHdLveeLgU+EGYNI1Kpqavn6U7NZvrmYR688huF5GstImi7d8iASInfn1smLeHd5Eb+4cDgnDVCbmDRtUd99JJLQHn1vFU9/uJavnnQUn8/vceAXiERMSUEkJG8u3sxdU5YwfkgXbjxjYNThiMRFSUEkBMs3F3PDs3MY2q0tv75oJCkpmjFNmgclBZFGtnN3FROfnEWLjFQe+uIYWmRoTmVpPtTQLNKIamqdG56dw7ptZTwz8Vi6tm0RdUgiB0VJQaQR/ervy/jHskLuPHcox/RuH3U4IgdN1UcijeTNxZu5f9oKLj6mB5eN6xl1OCKHRElBpBGs21bGt5+by5BubfjJhCGYqWFZmiclBZHDVF5Vw9eemoUDD1w2hqx0NSxL86U2BZHDdPuri1m4YRcPfzGfnh2yow5H5LCopCByGF6bv5GnP1zLxBP7cvrgzlGHI3LYlBREDtG6bWXc9OJ8RvbI4XvqsSwJQklB5BBU1dRy/TNzwOF3l4wiPVX/SpIY1KYgcgh++cZHzF23g/svHU2P9mpHkMShrzciB+mDFUX84Z0VXDK2B58Z3jXqcEQa1QGTgpnNMrNrzazdkQhIpCnbWVbFd56bR58OLfnx2YOjDkek0cVTUrgI6AbMMLNnzewMU88cSULuzs0vL6CwuILfXDyS7AzVvkriOWBScPcCd78ZGAA8DTwGrDGz28xMg7tI0nh57gZenb+Rb57WX1NqSsKKq03BzIYDvwTuBl4APgfsAt4OLzSRpmPDjt3c8vIi8nu142sn94s6HJHQHLD8a2azgB3Ao8BN7l4RbPrQzD4RZnAiTUFtrfP95+dT486vPj+SVE2YIwksnkrRz7n7yr1tcPfzGzkekSbnqQ/X8F5BEXedN1TDWEjCO2BScPeVZvYZYAiQVW/97WEGJtIUrC4q5adTlnJC/45cOlbDYUvii+eW1AeJ3YF0PWDE2hN6hRyXSORqap3v/mUeaanGLy4cruGwJSnE09B8vLt/Edju7rcBxxG7E0kkoT3xwWpmrtnOrZ8domk1JWnEkxR2Bz/LzKwbUAWoG6cktDVbS7l76lJOGZjLBaO7Rx2OyBETT0Pzq2aWQ+x21NmAA4+EGpVIhGprnRufn096Sgo/PX+Yqo0kqcTT0HxH8PQFM3sVyHL3neGGJRKdp6av5cNV2/j5BcNUbSRJJ56G5muDkgJBH4UUM/t66JGJRGDDjt38bMoSTujfkc/n94g6HJEjLp42hWvcfceeBXffDlwTXkgi0XB3fvTSAhz46XmqNpLkFE9SSK0/AJ6ZpQIZ4YUkEo3J8z5m2rJCvvvpgZojQZJWPA3NrwN/NrM/BMtfCdaJJIxtpZXc9tfFjOiRwxXH9446HJHIxFNS+D4wDfha8HgLuDGeg5vZeDNbZmYFZnbTPvb5vJktNrNFZvZ0vIGLNKY7X1vMrt1V/PyCYRrbSJJaPHcf1ZrZE8Db7r4s3gMH1Uz3A6cD64nNxzDZ3RfX26c/8APgE+6+3cw6HewJiByud5cX8uLsDVx3Sj8GdWkTdTgikYrn7qMJwFyCKiMzG2lmk+M49ligwN1Xunsl8CxwToN9rgHuDxqvcfctBxO8yOEqr6rh5pcW0rdjS677lIbEFomn+uhWYh/wOwDcfS7QJ47XdQfW1VteH6yrbwAwwMzeN7N/m9n4vR3IzCaa2Uwzm1lYWBjHW4vE5963lrN2Wxl3njeUrPTUqMMRiVw8SaFqL53VvJHePw3oD5wMXAI8vKdPxH+9mftD7p7v7vm5ubmN9NaS7JZtKuahd1Zy4Zg8jj+qY9ThiDQJ8SSFRWZ2KbFbU/ub2e+AD+J43Qagfu+fvGBdfeuBye5e5e6rgI+IJQmRUNXWOj98aQGts9L44VlHRx2OSJMRT1K4nthcChXAM8Sm4fxmHK+bAfQ3sz5mlgFcDDRsi3iZWCkBM+tIrDpprxP6iDSmZ2esY9aa7dz8mcG0b6luNyJ7xHP3URlwc/CIm7tXm9l1wFQgFXjM3ReZ2e3ATHefHGz7tJktBmqA77n71oM9CZGDUVRSwc/+toRj+7bXCKgiDewzKZjZX9lP24G7TzjQwd19CjClwbpb6j134NvBQ+SI+OlrS9hdVcOd52ooC5GG9ldSuCf4eT7QBfhTsHwJsDnMoETC8sGKIl6cE+uT0K9Tq6jDEWly9pkU3P2fAGb2S3fPr7fpr2Y2M/TIRBpZRXUNP3p5IT3bZ6tPgsg+xNPQ3NLM+u5ZMLM+QMvwQhIJx8PvrGRlYSm3nTNEfRJE9iGeAfG+BfzDzFYCBvQCJoYalUgjW7etjN+9XcCZQ7twykCNpiKyL/HcffR6MEbRoGDV0mCyHZFm47a/LiI1xfjx2YOjDkWkSYunpLBnxrV5IcciEoq/L97Mm0u28MOzBtEtR9NriuxPPG0KIs1WWWU1P5m8iAGdW3HVJ+IZskskucVVUhBpru57u4ANO3bz3FeOIz1V34FEDiSeobM/YWYtg+eXm9mvzKxX+KGJHJ6CLSU8/O5Kzh/dnbF92kcdjkizEM9XpweAMjMbAXwHWAFMCjUqkcPk7tw6eSFZ6an84EwNeCcSr3iSQnUwHMU5wH3ufj/QOtywRA7Pq/M38n7BVr53xkByW2dGHY5IsxFPm0Kxmf0AuBw40cxSgPRwwxI5dCUV1dz52mKGdGvDZeNU0ylyMOIpKVxEbNjsq919E7F5Ee4ONSqRw/DbNz9i864K7jh3KKkpGvBO5GDEVVIAfuvuNWY2gFgntmfCDUvk0CzbVMxj76/m4mN6MLpnu6jDEWl24ikpvANkmll34A3gC8ATYQYlcijcnR+/spDWWWncOH7QgV8gIv8jnqRgwUQ75wO/d/fPAUPDDUvk4L08dwPTV23jxjMGaTY1kUMUV1Iws+OAy4DXDuJ1IkfMrvIq7nptKSN65HDxMT0O/AIR2at42hS+CfwAeCmYTrMvMC3csEQOzq/e+IitpRU8dmU+KWpcFjlk8YyS+k/gn2aWHSyvBL4RdmAi8Vr08U4m/Ws1l47tyfC8nKjDEWnW4hnm4jgzWwwsDZZHmNnvQ49MJA61tc4trywiJzuD750xMOpwRJq9eNoGfgOcAWwFcPd5wIlhBiUSr+dnr2fWmu3cdOYgcrLVuCxyuOJqMHb3dQ1W1YQQi8hB2VFWyc/+tpQxvdpx4ei8qMMRSQjxJIV1ZnY84GaWbmbfBZaEHJfIAd09dRk7yiq545yhalwWaSTxJIWvAtcC3YENwMhgWSQyc9ft4Onpa7ni+N4M7tYm6nBEEkY8dx8VEeujINIk1NQ6P3p5AbmtMvn26QOiDkckoRwwKZhZLnAN0Lv+/u7+pfDCEtm3P/17DQs37OJ3l4yidZYG7BVpTPF0XnsFeBd4EzUwS8S2FJdzz9RlnNC/I2cP7xp1OCIJJ56kkO3u3w89EpE43PXaEiqqa7ltwhDM1Lgs0tjiaWh+1czOCj0SkQN4b3kRr8z9mK+e1Je+ua2iDkckIcWTFG4glhh2m9kuMys2s11hByZSX3lVDT9+ZSG9O2Tz9VP6RR2OSMKK5+4jzccskXvwnytYVVTKk1ePJSs9NepwRBJWPGMfvRXPun28dryZLTOzAjO7aT/7XWBmbmb58RxXksuqolJ+P20FE0Z044T+uVGHI5LQ9llSMLMsIBvoaGbtgD2tem2IdWTbLzNLBe4HTgfWAzPMbLK7L26wX2tiVVQfHtIZSEJzd3788kIy01P40dlHRx2OSMLbX0nhK8AsYnMyz6r3eAW4L45jjwUK3H2lu1cCzwLn7GW/O4CfA+UHEbckiZfmbOC9giJuHD+ITq2zog5HJOHtMym4+2+BfsCd7t7X3fsEjxHuHk9S6A7UH0hvPQ1KGGY2Gujh7q8h0sC20krufG0JY3q147KxPaMORyQp7LdNwd1riM3N3OjMLAX4FfCdOPadaGYzzWxmYWFhGOFIE3TXa0vYtbuKn543TAPeiRwh8dyS+lbQEHyw/5UbgPqT5eYF6/ZoDQwF/mFmq4Fjgcl7a2x294fcPd/d83Nz1dCYDD4oKOKF2ev5ykl9GdhFN8CJHCnxJIWvAH8BKg+yn8IMoL+Z9TGzDOBiYPKeje6+0907untvd+8N/BuY4O4zD/40JJGUV9Vw88uxPgnXf6p/1OGIJJXQ+im4e7WZXQdMBVKBx9x9kZndDsx098n7P4Ikq9+8uZxVRaU8/eVx6pMgcoTFM0qqERs6u4+732FmPYCu7j79QK919ynAlAbrbtnHvifHFbEktIUbdvLwuyu5KL8Hx/frGHU4Ikknnuqj3wPHAZcGyyXE+h+INKrqmlq+/8J82rfM4IdnqU+CSBTiGSV1nLuPNrM5AO6+PWgjEGlUD7+7ikUf7+KBy0bTNlvzJIhEIZ6SQlXQO9mhbtKd2lCjkqSzorCE37z5EWcM6cyZwzRPgkhU4kkK9wIvAZ3M7C7gPeCnoUYlSaWm1rnx+flkpadyx7lDow5HJKnFc/fRU2Y2CziV2PhH57r7ktAjk6Tx+PurmLVmO7++aISGshCJ2P4GxGtfb3EL8Ez9be6+LczAJDmsKirl7qnLOO3oTpw78oDjLIpIyPZXUphFrB2hfk/mPcsO9A0xLkkCsWqjeWSmpXDXecM0vaZIE7DPpODufY5kIJJ8HntvFTNWb+eez42gcxtVG4k0BfHckkown0J/oO4/193fCSsoSXwfbS7m7jeWcfrgzlwwWtVGIk1FPD2av0xsEpw8YC6xgev+BXwq3NAkUVXV1PLt5+bSKjON/ztf1UYiTUk8t6TeABwDrHH3U4BRwI5Qo5KEdt/bBSzcsIufnjeUjq0yow5HROqJJymUu3s5gJlluvtSYGC4YUmimrtuB/dNK+C8Ud0ZP1Sd1ESamnjaFNabWQ7wMvB3M9sOrAk3LElEpRXV3PDsHLq0yeInE4ZEHY6I7EU8ndfOC57+xMymAW2B10ONShLSbX9dxLptZTw78TjattDYRiJNUVx3H+3h7v8MKxBJbH9bsJHnZq7n2lOOYmyf9gd+gYhEIp42BZHDsnHnbm56cQHD89ryzdMGRB2OiOyHkoKEqrqmlm88M4fqmlp+e/Eo0lP1JyfSlB1U9ZHIwfrNm8uZsXo7v714JH06tow6HBE5AH1tk9C8t7yI+/9RwOfz8zhHg92JNAtKChKKLcXlfPPPc+mX20q3n4o0I6o+kkZXXVPLdU/PoaSiiqe+PI7sDP2ZiTQX+m+VRnf31GVMX7WNX180goFdWkcdjogcBFUfSaN6feFG/vDOSi4/tifnjcqLOhwROUhKCtJoVhSW8N2/zGdEXlt+fPbgqMMRkUOgpCCNYld5FddMmklGWgq/v3wMmWmpUYckIodASUEOW02tc8Mzc1i7tYzfXzaa7jktog5JRA6RkoIctnveWMa0ZYXcOmEIx/btEHU4InIYlBTksLw8ZwMP/GMFl4ztyeXjekYdjogcJiUFOWTTV23jxufnM7ZPe26bMETTaookACUFOSSrikqZ+ORM8tq14KEvjCEjTX9KIolA/8ly0LaXVvKlJ2aQYsbjVx1DTnZG1CGJSCNRj2Y5KLsra7j6jzPYsGM3T395HL06aORTkUQSaknBzMab2TIzKzCzm/ay/dtmttjM5pvZW2bWK8x45PBU1dRy7dOzmbNuB/dePJL83ppBTSTRhJYUzCwVuB84ExgMXGJmDbu5zgHy3X048Dzwi7DikcPj7tz0wgLeXrqFO88dyvihXaMOSURCEGZJYSxQ4O4r3b0SeBY4p/4O7j7N3cuCxX8DGiynCXJ37nptCS/MXs+3ThvAZeNUoBNJVGEmhe7AunrL64N1+3I18Le9bTCziWY208xmFhYWNmKIEo9fvvERj7y3iiuP7803Tu0XdTgiEqImcfeRmV0O5AN37227uz/k7vnunp+bm3tkg0ty9729nPumFXDxMT245ezB6osgkuDCvPtoA9Cj3nJesO6/mNlpwM3ASe5eEWI8cpAeemcF97zxEeeN6s5d5w0jJUUJQSTRhVlSmAH0N7M+ZpYBXAxMrr+DmY0C/gBMcPctIcYiB+m+t5fz0ylL+czwrtx94XBSlRBEkkJoJQV3rzaz64CpQCrwmLsvMrPbgZnuPplYdVEr4C9BtcRad58QVkxyYO7Or99czr1vLee8Ud25+8LhpKU2iVpGETkCQu285u5TgCkN1t1S7/lpYb6/HBx35//+tpSH3lnJ58bk8bMLVEIQSTbq0SwAVNfUctOLC3h+1nq+cGwvbpswRG0IIklISUHYXVnDdU/P5q2lW/jmaf254dT+ustIJEkpKSS5opIKJk6ayZx1O7jj3KF84Vh1TBNJZkoKSWz55mKuemIGhcUV/P7S0Zw5TENXiCQ7JYUk9c5HhVz71GyyMlJ57ivHMaJHTtQhiUgToKSQZNydP7yzkl+8vpQBnVvz6JXH0D2nRdRhiUgToaSQREoqqrnx+XlMWbCJzwzvyi8uGE7LTP0JiMh/6BMhSSzdtIvrnp7DysISfnjWIK45oa/uMBKR/6GkkODcnaenr+X2vy6mdVY6f7p6HMf36xh1WCLSRCkpJLBtpZXc/NIC/rZwEyf078ivPj+S3NaZUYclIk2YkkKCenPxZm56cQE7d1dy05mDmHhCX/VQFpEDUlJIMDvKKrnztSU8P2s9R3dtw6QvjWVwtzZRhyUizYSSQoJwd16Z+zF3vLqYHburuO6Ufnzj1P5kpGmEUxGJn5JCAlhRWMJPJi/i3eVFjOiRw5PnDVPpQEQOiZJCM7arvIp731zOEx+spkV6Kj/57GC+cFxvDXctIodMSaEZqqyu5dkZa7n3reVsLa3kovwefOfTA3VnkYgcNiWFZqSm1nl1/sf88o2PWLutjHF92vP4lYMZltc26tBEJEEoKTQD1TW1vDp/I/dNK6BgSwlHd23DE1cdw0kDctUrWUQalZJCE1ZeVcNLczbw0DsrWVVUysDOrfndJaP4zLCu6nMgIqFQUmiCCosrePrDtUz612q2llYytHsbHrx8NJ8e3EXJQERCpaTQRLg7s9ZsZ9K/1vC3hRupqnE+NagTXz6hD8f17aBqIhE5IpQUIraluJyXZm/gL7PWU7ClhNZZaVx+bC8uG9eLfp1aRR2eiCQZJYUIFJdX8caizUye9zHvFRRRU+uM6dWOn50/jAkju5GdocsiItHQp88RsqOskjeXbGHqok3886NCKqtr6Z7Tgokn9uXCMXkclatSgYhET0khJO7O8i0lTFu6hWnLtjBj9XZqap2ubbO4dGxPPjuiG6N75qitQESaFCWFRrR5VzkfrCjiveVb+WBFERt3lgMwqEtrvnJiX84Y0oXheW2VCESkyVJSOES1tc6KwhJmr93OjNXbmb5qG2u3lQGQk53O8Ud14Bv9czl5YC5d27aIOFoRkfgoKcShttZZt72MhRt2sWDDThZu2Mm8dTsorqgGoF12Osf0bs8Xj+vFsX07MLhrG/UnEJFmSUmhHnensLiC5VtKWL65mI+2lLB04y6WbSqmtLIGgPRUY0Dn1nx2ZDdG9chhVM929O3YUklARBJC0iWFmlpn065y1m8rY23wWLO1jFVFpawqKqUk+PYP0CYrjUFd23DhmDwGdW3DkG5tGNilNZlpqREGAlGcAAAJN0lEQVSegYhIeJImKTw7fS33TStg085yqmu9bn2KQfd2LejTsRVjerWjT8eW9O/Uin6dW5HbKlONwiKSVEJNCmY2HvgtkAo84u4/a7A9E5gEjAG2Ahe5++owYunYKpMxvdrRPacFee2y6d6uBb3ax36mp2rKShERCDEpmFkqcD9wOrAemGFmk919cb3drga2u3s/M7sY+DlwURjxnDa4M6cN7hzGoUVEEkaYX5HHAgXuvtLdK4FngXMa7HMO8Mfg+fPAqab6GhGRyISZFLoD6+otrw/W7XUfd68GdgIdQoxJRET2o1lUppvZRDObaWYzCwsLow5HRCRhhZkUNgA96i3nBev2uo+ZpQFtiTU4/xd3f8jd8909Pzc3N6RwRUQkzKQwA+hvZn3MLAO4GJjcYJ/JwBXB8wuBt93dERGRSIR295G7V5vZdcBUYrekPubui8zsdmCmu08GHgWeNLMCYBuxxCEiIhEJtZ+Cu08BpjRYd0u95+XA58KMQURE4tcsGppFROTIsOZWhW9mhcCaQ3x5R6CoEcNpDnTOyUHnnBwO55x7ufsB79RpdknhcJjZTHfPjzqOI0nnnBx0zsnhSJyzqo9ERKSOkoKIiNRJtqTwUNQBREDnnBx0zskh9HNOqjYFERHZv2QrKYiIyH4kTVIws/FmtszMCszspqjjCYOZ9TCzaWa22MwWmdkNwfr2ZvZ3M1se/GwXdayNycxSzWyOmb0aLPcxsw+Da/3nYJiVhGFmOWb2vJktNbMlZnZcElzjbwV/0wvN7Bkzy0q062xmj5nZFjNbWG/dXq+rxdwbnPt8MxvdWHEkRVKoN+HPmcBg4BIzGxxtVKGoBr7j7oOBY4Frg/O8CXjL3fsDbwXLieQGYEm95Z8Dv3b3fsB2YpM5JZLfAq+7+yBgBLFzT9hrbGbdgW8A+e4+lNiwOXsm5Uqk6/wEML7Bun1d1zOB/sFjIvBAYwWRFEmB+Cb8afbcfaO7zw6eFxP7sOjOf09m9Efg3GgibHxmlgd8BngkWDbgU8QmbYLEO9+2wInExg3D3SvdfQcJfI0DaUCLYDTlbGAjCXad3f0dYmPA1bev63oOMMlj/g3kmFnXxogjWZJCPBP+JBQz6w2MAj4EOrv7xmDTJiCR5iX9DXAjUBssdwB2BJM2QeJd6z5AIfB4UGX2iJm1JIGvsbtvAO4B1hJLBjuBWST2dd5jX9c1tM+0ZEkKScXMWgEvAN909131twVDkyfELWdmdjawxd1nRR3LEZQGjAYecPdRQCkNqooS6RoDBPXo5xBLiN2AlvxvNUvCO1LXNVmSQjwT/iQEM0snlhCecvcXg9Wb9xQtg59booqvkX0CmGBmq4lVCX6KWH17TlDNAIl3rdcD6939w2D5eWJJIlGvMcBpwCp3L3T3KuBFYtc+ka/zHvu6rqF9piVLUohnwp9mL6hPfxRY4u6/qrep/mRGVwCvHOnYwuDuP3D3PHfvTeyavu3ulwHTiE3aBAl0vgDuvglYZ2YDg1WnAotJ0GscWAsca2bZwd/4nnNO2Otcz76u62Tgi8FdSMcCO+tVMx2WpOm8ZmZnEat/3jPhz10Rh9TozOyTwLvAAv5Tx/5DYu0KzwE9iY0w+3l3b9ig1ayZ2cnAd939bDPrS6zk0B6YA1zu7hVRxteYzGwksYb1DGAlcBWxL3gJe43N7DbgImJ32M0BvkysDj1hrrOZPQOcTGwk1M3ArcDL7OW6BsnxPmLVaGXAVe4+s1HiSJakICIiB5Ys1UciIhIHJQUREamjpCAiInWUFEREpI6SgoiI1FFSkEiZWe/6o0LG+ZqfmNl3g+eDzGxuMOTDUQd7rLCZ2VfN7ItH+D1P3jNi7CG8NsfMvt7YMUnzoaQgzd25wPPBkA81YbxBvV6zB83dH3T3SY0ZT8hyACWFJKakIE1Bqpk9HIyX/4aZtQAws2vMbIaZzTOzF8wsu/6Lgg6J3wS+ZmbTDnCskWb272Ds+ZfqjUv/DzPLD553DIbMwMyuNLPJZvY28JaZdTWzd4JSyUIzO6HhSZjZzyw2l8V8M7snWFe/VPMPM/u5mU03s4/2HMNi80HcExx3vpldH6wfY2b/NLNZZjZ1b6NgmtnngtfNM7N39rK9pcXG6Z8elKbOqXd+rwQxLTezW4OX/Aw4KjjPuw/iGkqCUFKQpqA/cL+7DwF2ABcE619092Pcfc+cAf81Xr67TwEeJDam/ikHONYk4PvuPpxYj+9bObDRwIXufhJwKTDV3UcSm8Ngbv0dzawDcB4wJHiPO/dxzDR3H0ssme2JYSLQGxgZvPapYAyr3wXvPwZ4DNhbL/xbgDOC39GEvWy/mdjwH2OBU4C7g1FVITak/AXAcOBzQXK8CVjh7iPd/Xv7+d1IgjrkYrFII1rl7ns+ZGcR+4AEGGpmdxKr0mgFTD2UY1lsDoIcd/9nsP6PwF/iONbf6w0VMQN4LPiwfrnee+yxEygHHg3q8/dVp79nkML653ka8OCeYaCDYQyGAkOBv8dGNCCV2LDRDb0PPGFmz9U7dn2fJjZo4HeD5SxiQybsOb+tAGb2IvBJYsMqSBJTSUGagvrj1dTwny8rTwDXufsw4DZiH2iHeqx9qeY//wcNj1+650kwAcqJxEaifKJh43HwgT6W2KilZwOvHyC+A8VmwKLgG/tIdx/m7p9uuJO7fxX4EbERM2cFJZaGx7mg3nF6uvueWeoajnGjMW9ESUGatNbAxuDb+WWHehB33wlsr9cO8AVgT6lhNTAmeH4h+2BmvYDN7v4wscHoRjfY3gpoG1RpfYtYFVO8/g58ZU+Dtpm1B5YBuWZ2XLAu3cyG7CWuo9z9Q3e/hdjkOz0a7DIVuD4YQA0zG1Vv2+kWmwO4BbEG+/eBYmK/d0lSSgrSlP2Y2Aiv7wNLD/NYVxCrT58PjARuD9bfQ6yheg6x0Sn35WRgXrDfRcTmbaivNfBqcPz3gG8fRGyPEBseer6ZzQMuDaaNvRD4ebBuLnD8Xl57t5ktCG7F/QCY12D7HUB6cOxFwfIe04nNvTEfeMHdZwbVSe8HjddqaE5CGiVVJAmZ2ZVAvrtfF3Us0rSopCAiInVUUhARkToqKYiISB0lBRERqaOkICIidZQURESkjpKCiIjUUVIQEZE6/w+Nx67V3NRV8wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "xspace = np.linspace(0, 100, 100)\n",
    "plt.xlabel('halfhours since slept')\n",
    "plt.ylabel('alertness decay')\n",
    "plt.plot(xspace, alertness_decay(xspace))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.48141722191172337"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "decay_rate = 0.97\n",
    "half_life = decay_rate ** 24\n",
    "half_life"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "def half_hour_passed(state):\n",
    "    state['alertness'] -= alertness_decay(state['time_since_slept'])\n",
    "    state['hypertension'] = decay_rate * state['hypertension']\n",
    "    state['intoxication'] = decay_rate * state['intoxication']\n",
    "    state['time_since_slept'] += 1\n",
    "    state['time_elapsed'] += 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'alertness': 1.7179066415679574,\n",
       " 'hypertension': 0.3555231469621428,\n",
       " 'intoxication': 0.011052524897623732,\n",
       " 'time_since_slept': 1,\n",
       " 'time_elapsed': 1,\n",
       " 'work_done': 0}"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "half_hour_passed(state)\n",
    "state"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Fact 7**. In order to work productively, you have to be alert, but not so alert that crippling anxiety hits"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "def crippling_anxiety(alertness):\n",
    "    return sigmoid(alertness - 3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x7fb76a80fa10>]"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3XecVOXZ//HPtY2FZekdliYI0sGlGPKzhSRiItjFaKLGRJM8RlM0MXkSNZpiNNHoozExCYIaRWLFgBILhlhpIh1ZkLbS68L2mev3xwzrSihDOXt2Zr7v12tfM+fMmTPfYdm55pz7Pvdt7o6IiAhARtgBRESk/lBREBGRGioKIiJSQ0VBRERqqCiIiEgNFQUREamhoiAiIjVUFEREpIaKgoiI1MgKO8CRatWqlXft2jXsGCIiSWXu3Llb3b314bZLuqLQtWtX5syZE3YMEZGkYmZrEtlOp49ERKSGioKIiNRQURARkRoqCiIiUiOwomBm481ss5ktOsjjZmb3m1mRmS0wsyFBZRERkcQEeaQwATjrEI+PBnrGf64BHgowi4iIJCCwouDuM4Hth9hkLPCox7wLNDOz9kHlERGRwwvzOoWOwLpay+vj6zaEE0dE0l0k6lRFolRGolRVR2PLUac6EqU66lRHnOpofH3EiURr/bgTiUaJRPnk1p1orcfdvWa9xx+LOkTd8fjtJ8ux+15r+XMntWVgQbNA/w2S4uI1M7uG2CkmOnfuHHIaEakr7k5ZVYQ95dXsqahmb0WEPRXVlFZWU1oZoawyQmllNWVVUcqqIpRVVlNeFaW8KkJ5dZSKWrcV1VEqq6NUVEfiH/pe8+FfGYlSFYkSrcdT1ptB26a5KV0UioGCWsud4uv+i7s/DDwMUFhYWI9/bSJyIFWRKDv2VrJtb2XN7c7SSnaWVrGjtIpdZbGf3eVV7C6roqS8mt3lVeytqD6iD+rc7AwaZmfSMDuTBtmZNMjKqLnNz82iQVYGOVkZ5GTGb7MyyMnMJDvLyM6ILWdnZpCdaWRnZpCZYWRnGlkZGWR96tbIzIgtZ2YYWZlGhn2yPsNit5kZ1NzPMCMjw8g0I2PfeouttwzINMMMjE8eN+K3BmYW2O+ntjCLwhTgOjObBAwHdrm7Th2JJJFo1Nmyp4LinWVs2lXOhl3lbNpdzuaSCjaXlLN5dwVb91Swo7TqoPtolJNJs4bZNGmYTdOG2RS0aER+bhZNcrPJz82icYMsGudmkZeTRV6DLPIaZNIoJ4u8nEwa5sTuN8qJffDX1QdnKgusKJjZk8DpQCszWw/cCmQDuPufgGnA2UARUApcFVQWETl6eyqqWb11L2u2lbJ6217WbS9l3Y5S1m0vY8OuMqoin/4qn5OZQev8BrRp0oDurfMY3r0FLfMa0KpxDi0bN6B5oxxa5OXQPC9WBBpkZYb0zuRAAisK7n7pYR534H+Cen0ROTK7SqtYvqmE5ZtKWLGphJVb9lC0eQ+bdld8artWjXPo1LwRAwuacXb/9nRslkuHZg1p1zSX9k0b0rxRtr6xJ7GkaGgWkeNrS0kFC9bvZGHxLhZ/vJvFxbv4eFd5zeONG2RxQpvGjOzRihNaN6Zbqzy6tsyjS8tG5DXQx0Yq029XJMVVR6Is3VDCnDXbmbNmB/PX7qR4ZxkQ69HSvVUehV1bcFL7JvRul0+vdvm0b5qrb/tpSkVBJMVEo86SDbt5e+VW3lm5jdmrd7CnohqADk1zGdylOVeN7MrAgmb0ad9E3/zlU/S/QSQF7Cyt5N8fbuGN5Vv4z4otbN1TCUD31nmMHdSB4d1bUtilOR2aNQw5qdR3KgoiSerjnWW8tGgjryzZyOzVO4hEnRZ5OZzasxWnntiakT1a0bZJbtgxJcmoKIgkkc0l5fzzgw1MXbiBuWt2ANCrbT7fOq07o05qy8BOzcjIUFuAHD0VBZF6rrwqwitLNvHsvPXMXLGVSNQ5qX0TbvpiL87u355urfLCjigpREVBpJ4q2ryHSbPW8sy89eworaJ901yuPbU75w/pSI82+WHHkxSloiBSj0Sjzr8/3ML4tz7iPyu2kpVhfKFvWy4d1pmRJ7TSqSEJnIqCSD1QUR3huXnFPPyfVazaspe2TRpw4xdO5OKhBbTJV2Ox1B0VBZEQlVVGeGLWWv4ycxUbd5fTv2NT7hs3iNH92pOTpSnUpe6pKIiEoKI6wpPvreWBGSvZuqeCEd1bcPdFA/hsj1a6klhCpaIgUoeiUee594u555UPKd5ZxvBuLXjo8iEM7doi7GgigIqCSJ15b9U27pi6hEXFuxnQqSl3XtBfRwZS76goiARs465y7pi6hKkLNtC+aS5/uGQQYwZ2UE8iqZdUFEQCUh2JMuHt1dz7yodUR53vjzqRa07tTsMcTSoj9ZeKgkgAlm7YzU1Pf8Ci4t2c3qs1t4/pR+eWjcKOJXJYKgoix1FVJMofZ6zkgRkraNowmz9eNoTR/dqp3UCShoqCyHGycssebpj0PouKdzN2UAduO6cvzfNywo4lckRUFESOkbvz1Ox1/OLFJeRmZ/Cny0/mrH7two4lclRUFESOQUl5FT9+ZgHTFm5kZI+W3HPxIM1hIElNRUHkKC3fWMK3Hp/L2u2l/Pis3lx7and1M5Wkp6IgchRemF/Mzc8spHFuFk98YzjDu7cMO5LIcaGiIHIEolHnrunL+dO/VzKsWwse+MpgjWIqKUVFQSRBeyqq+d6k+by6dBOXDe/MbWP6kp2pkUwltagoiCRg465yrnxkFis27+EXY/rytVO66NoDSUkqCiKHsWJTCVeMn8Xu8moeuXIop57YOuxIIoFRURA5hNmrt/ONiXPIycrgqWtH0LdD07AjiQRKRUHkIGYs28y3Hp9Lx+YNmXjVMApaaOwiSX0qCiIH8NLCDVw/6X16tcvn0a8Pp4WGq5A0oaIgsp/n3l/Pjf9YwMBOTXnkqmE0bZgddiSROqOiIFLLM3PXc+PTHzCiW0v+ekUheQ30JyLpRf/jReJemF/MTU9/wGdOaMnfrhhKbrYmw5H0E+iVN2Z2lpktN7MiM7v5AI93NrMZZva+mS0ws7ODzCNyMFMXbOAHkz9gaNcW/PVrKgiSvgIrCmaWCTwIjAb6AJeaWZ/9NvsZMNndBwPjgD8GlUfkYF5ftokbJr3PkM7NGH/lUE2XKWktyCOFYUCRu69y90pgEjB2v20caBK/3xT4OMA8Iv9l9urtfPvxeZzUvgnjrxyqNgRJe0H+BXQE1tVaXg8M32+b24B/mdl3gTxgVIB5RD5l6YbdfH3CbDo2a8iEq4aSn6teRiJhj+Z1KTDB3TsBZwOPmdl/ZTKza8xsjpnN2bJlS52HlNSzbnspXxs/i8YNsnjsG8Np2bhB2JFE6oUgi0IxUFBruVN8XW1XA5MB3P0dIBdotf+O3P1hdy9098LWrTXujBybXaVVXPnILCqrozz69WF0bNYw7Egi9UaQRWE20NPMuplZDrGG5Cn7bbMW+ByAmZ1ErCjoUEACU1Ed4drH57B2eyl//urJ9GybH3YkkXolsKLg7tXAdcB0YCmxXkaLzex2MxsT3+yHwDfN7APgSeBKd/egMkl6c3dufmYh767azt0XDmSEZksT+S+BdrVw92nAtP3W3VLr/hJgZJAZRPZ54PUinnu/mB9+/kTOHdwx7Dgi9VLYDc0ideLlRRv5/Ssfcu6gDlx3Zo+w44jUWyoKkvKWbdzNDybPZ2BBM+68YIBmTBM5BBUFSWnb91byjYlzaNwgi4e/erKGrxA5DF2+KSkrEnWuf/J9NpdUMPnaU2jbJDfsSCL1no4UJGXd+8qHvFm0lV+O7ceggmZhxxFJCioKkpJeWbKJB2YUMW5oARcPLTj8E0QEUFGQFLRm215+MHk+/Ts25bYxfcOOI5JUVBQkpVRUR/jO3+eRYcYfLxuihmWRI6SGZkkpv5m2jMUf7+avXyukoEWjsOOIJB0dKUjKeHnRRia8vZqrP9uNUX3ahh1HJCmpKEhKWLe9lB89/QEDOjXlx2f1DjuOSNI6bFEws3MONMeBSH1RHYnyvafm4w4PXDqEnCz9dxU5Won89VwCrDCzu8xMX8Gk3nlwxkrmrtnBL8/rR+eWakcQORaHLQrufjkwGFgJTDCzd+IzoWkgegndvLU7uP/1FZw7qANjB2nkU5FjldBxtrvvBp4GJgHtgfOAefG5lUVCsaeimu9Nmk+7Jrncfm6/sOOIpIRE2hTGmtlzwBtANjDM3UcDA4lNkiMSittfXMz6HaX8YdwgmuRmhx1HJCUkcp3C+cC97j6z9kp3LzWzq4OJJXJory7ZxOQ56/nO6ScwtGuLsOOIpIxETh9t3L8gmNlvAdz9tUBSiRzC9r2V3PzsQnq3y+eGUT3DjiOSUhIpCp8/wLrRxzuISCLcnZ8/v4hdZZXce8kgGmRpGAuR4+mgp4/M7NvAd4ATzGxBrYfygbeCDiZyIC8u2MDUhRu46Yu9OKl9k7DjiKScQ7UpPAG8BPwGuLnW+hJ33x5oKpED2FJSwS0vLGJQQTOuPbV72HFEUtJBTx+5+y53X+3ulwIFwJnuvgbIMLNudZZQJO7WKYsorYjwu4sGkJWpq5ZFgpBIl9RbgR8DP4mvygEeDzKUyP6mLdzAtIUbuWFUT3q00XWTIkFJ5OvWecAYYC+Au39MrF1BpE5s31vJLS8son/HpjptJBKwRIpCpbs74ABmlhdsJJFPu/3Fxewqq+KuC3XaSCRoifyFTTazPwPNzOybwKvAX4KNJRIzY/lmnp//Md8+vYd6G4nUgcNe0ezuvzOzzwO7gV7ALe7+SuDJJO3trajmZ88tokebxvzPGSeEHUckLSQ0HWe8CKgQSJ36/b8+pHhnGf/41im6SE2kjhzq4rU33f2zZlZCvD1h30OAu7uO5SUwH6zbyYS3P+LyEZ01tpFIHTpoUXD3z8Zv1dNI6lRVJMrNzy6kdX4DfqSpNUXqVCLXKYw6wLorgokjAo+89RFLN+zmF2P6aUhskTqWSO+jW8zsITPLM7O2ZvYicE7QwSQ9rd9Ryr2vrGDUSW34Yt+2YccRSTuJFIXTiE3FOR94E3jC3S9MZOdmdpaZLTezIjO7+SDbXGxmS8xssZk9kXBySTnuzm1TFgNw25i+mFnIiUTSTyK9j5oDw4gVhk5AFzOz+AVtB2VmmcCDxIbeXg/MNrMp7r6k1jY9iQ2fMdLdd5hZm6N8H5ICpi/exKtLN/PTs3vTqXmjsOOIpKVEjhTeBV5297OAoUAHEhs6exhQ5O6r3L2S2PzOY/fb5pvAg+6+A8DdNyecXFLK3opqfvHiYnq3y+eqkRpvUSQsiRwpjHL3tQDuXgZcb2anJvC8jsC6WsvrgeH7bXMigJm9BWQCt7n7ywnsW1LMfa+tYMOuch74yhCyNZSFSGgSuaJ5rZl1BLoksv1RvH5P4HRip6Zmmll/d99ZeyMzuwa4BqBz587HOYKEbfnGEsa/+RHjhhZwcpfmYccRSWuH/ZCPz8d8CbAEiMRXOzDzoE+KKSY2D8M+neLralsPvOfuVcBHZvYhsSIxu/ZG7v4w8DBAYWHhIdsyJLnsm16zcW6WrkkQqQcS+eZ/LtDL3SuOcN+zgZ7xCXmKgXHAV/bb5nngUuARM2tF7HTSqiN8HUliz84rZtbq7dx5fn9a5OWEHUck7SVy8nYVcMRXELl7NXAdMB1YCkx298VmdruZjYlvNh3YZmZLgBnATe6+7UhfS5LTrrIqfj1tKYM7N+PiwoLDP0FEApfIkUIpMN/MXgNqjhbc/frDPdHdpwHT9lt3S637Dvwg/iNp5p5/LWdHaSUTvz6MjAxdkyBSHyRSFKbEf0SOm8Uf7+Kxd9dw2fAu9OvYNOw4IhKXSO+jiXURRNKHu3PrC4tp1iiHG7/QK+w4IlJLIr2PegK/AfoAufvWu7smy5Wj8uy8Yuas2cFdFwygaSMNeCdSnyTS0PwI8BBQDZwBPAo8HmQoSV27y6v4zUtLGVTQjAtP7hR2HBHZTyJFoaG7vwaYu69x99uALwUbS1LVH15Zwba9ldwxtp8al0XqoUQamivMLANYYWbXEbvmoHGwsSQVfbiphInvrGbc0M7076TGZZH6KJEjhRuARsD1wMnA5YAm2ZEjsm9Y7MYNsrjpi2pcFqmvEul9tG/IiT3AVcHGkVT10qKNvL1yG7eP7asrl0XqMQ1HKYErq4zwq6lL6d0un68M04CGIvXZ8R71VOS/PPRGEcU7y3jqmhFkaVhskXpNf6ESqHXbS/nTzFWMGdiB4d1bhh1HRA4jkYvX7j/A6l3AHHd/4fhHklTyy6lLyDTjJ2drWGyRZJDIkUIuMAhYEf8ZQGxuhKvN7A8BZpMk958VW5i+eBPXndmD9k0bhh1HRBKQSJvCAGCku0cAzOwh4D/AZ4GFAWaTJFYViXLblMV0admIqz+rOZdFkkUiRwrN+fTFanlAi3iRONKJdyRNTHx7NSu37OXnX+pDbnZm2HFEJEGJHCncRWw+hTcAA04Ffm1mecCrAWaTJLWlpIL7Xl3B6b1a87mT2oQdR0SOQCIXr/3NzKYBw+KrfuruH8fv3xRYMklad09fRnl1hJ9/uQ9mGt9IJJkk2iU1A9gC7AB6mNmpwUWSZDZ/3U4mz1nP10d244TWGiJLJNkk0iX1t8AlwGIgGl/twMwAc0kSikZj4xu1zm/AdWf2CDuOiByFRNoUzgV6ubsaleWQnpm3nvnrdvK7iwaSn6vJc0SSUSKnj1YB+guXQ9pdXsVvX17O4M7NOH9wx7DjiMhRSuRIoZRY76PXqNUF1d2vDyyVJJ37X13Btr0VPHLlUE2eI5LEEikKU+I/IgdUtLmECW+vZtzQAk2eI5LkEumSOrEugkhyik2es4RGOZnc+AVNniOS7A5aFMxssrtfbGYLifU2+hR3HxBoMkkK0xdv4s2irdx2Th9aNm4QdhwROUaHOlK4IX775boIIsmnrDLCHf9cQq+2+Vw+okvYcUTkODhoUXD3DfHbNXUXR5LJQ/9eSfHOMiZp8hyRlHGo00clfPq0kcWXDXB3bxJwNqnH1m4r5U//XsmYgR0YoclzRFLGoY4U8usyiCSX2/+5hKwM46dnnxR2FBE5jhKao9nMhhCbP8GBN939/UBTSb02Y9lmXl26iZtH96Zd09yw44jIcXTYE8FmdgswEWgJtAImmNnPgg4m9VN5VYTbXlxM91Z5fH2kJs8RSTWJHClcBgx093IAM7sTmA/8MshgUj89PHMVa7aV8vjVw8nJUuOySKpJ5K/6Y2LzNO/TACgOJo7UZ+u2l/LgjCK+NKA9n+3ZKuw4IhKARIrCLmCxmU0ws0eARcBOM7vfzO4/1BPN7CwzW25mRWZ28yG2u8DM3MwKjyy+1KVfvLiEzAzjZ19S47JIqkrk9NFz8Z993khkx2aWCTwIfB5YD8w2synuvmS/7fKJXSj3XiL7lXC8vmwTry7dxE9G96Z904ZhxxGRgCQ09pGZ5QC9ifU+Wu7ulQnsexhQ5O6rAMxsEjAWWLLfdncAv0VTe9ZbZZURbp2ymB5tGnOVGpdFUloivY/OBlYC9wMPAEVmNjqBfXcE1tVaXh9fV3vfQ4ACd5+acGKpcw/OKGLd9jLuGNtPjcsiKS6R00f3AGe4exGAmZ0ATAVeOpYXNrOM+L6vTGDba4BrADp37nwsLytHqGjzHv48cyXnD+7IKSfoymWRVJfI176SfQUhbhVQksDzioGCWsud+HSvpXygH/CGma0GRgBTDtTY7O4Pu3uhuxe2bt06gZeW48Hd+fnzi2iYnclP1bgskhYSOVKYY2bTgMnE2hQuItZofD6Auz97kOfNBnqaWTdixWAc8JV9D7r7LmIXwwFgZm8AN7r7nKN4HxKAF+Z/zDurtvHLc/vRSsNii6SFRIpCLrAJOC2+vAVoCJxDrEgcsCi4e7WZXQdMBzKB8e6+2MxuB+a4u2Zzq8d2llZyxz+XMLCgGZcO0yk7kXSRSO+jq4525+4+DZi237pbDrLt6Uf7OnL8/WbaMnaWVfHYef3J1JzLImnjUENn/8jd7zKz/+PAM69dH2gyCc17q7bx1Jx1XHtqd/p00AjpIunkUEcKS+O3OsefRiqqI/z0uYV0at6QG0b1DDuOiNSxQ82n8GL8quT+7n5jHWaSEP3pjVWs3LKXR64aSqOchEZWF5EUcsguqe4eAUbWURYJ2YpNJTwwYwVfHtCeM3q1CTuOiIQgka+C881sCvAPYO++lYfoiipJKBJ1fvzMAvIaZHHbmL5hxxGRkCTaJXUbcGatdQftiirJ6bF3VjNv7U7uuXigrkkQSWOBdkmV5LB+Ryl3TV/OqSe25rzBHQ//BBFJWYkMiDfRzJrVWm5uZuODjSV1xd35ybMLAfj1ef0w0zUJIukskbGPBrj7zn0L7r4DGBxcJKlLT81ex39WbOXHZ/WmU/NGYccRkZAlUhQyzKz5vgUza0FibRFSzxXvLOOXU5cyonsLvjqiS9hxRKQeSOTD/ffAO2b2j/jyRcCvgoskdWHfaaOoO3ddMJAMDWUhIiTW0Pyomc3hk95H5+8/paYkn6dmr2Pmh1u4fWxfOrfUaSMRiUnoNFC8CKgQpIh120trThtdPlynjUTkE5pbMc1Eos4PJ38AwO8u0mkjEfk0FYU089f/rGLW6u3cek4f9TYSkf+iopBGlm7Yze//9SFf7NuWC0/uFHYcEamHVBTSRHlVhO8/NZ8mDbP59Xn9dZGaiByQrjdIE3e+tIxlG0sYf2UhLTW2kYgchI4U0sDryzYx4e3VXPmZrpzZu23YcUSkHlNRSHGbd5dz4z8WcFL7Jtw8unfYcUSknlNRSGHRqPODyR9QWlnN/106iNzszLAjiUg9p6KQwh6cUcSbRVu59Zy+9GiTH3YcEUkCKgop6u2VW7n31Q8ZO6gD44YWhB1HRJKEikIK2lxSzvVPzqdbqzx1PxWRI6IuqSmmOhLlhifns6eiir9/Yzh5DfQrFpHE6RMjxdw9fTnvrNrG3RcOoFc7tSOIyJHR6aMU8s8FH/Pnmau4fERnLipUO4KIHDkVhRSxbONubvrHAk7u0pxbvtw37DgikqRUFFLAztJKrn1sLvm5WTx02RBysvRrFZGjozaFJFcVifLtx+exYWc5T14znDZNcsOOJCJJTEUhibk7t7ywmHdWbeOeiwdycpcWYUcSkSSn8wxJbPxbq3ly1lq+c/oJnD9E8yOIyLFTUUhSryzZxK+mLuGLfdty4xd6hR1HRFJEoEXBzM4ys+VmVmRmNx/g8R+Y2RIzW2Bmr5mZZpFPwLy1O/juk/Po37Ep914ySPMsi8hxE1hRMLNM4EFgNNAHuNTM+uy32ftAobsPAJ4G7goqT6pYtWUPV0+YTdsmufztyqE0ylGzkIgcP0EeKQwDitx9lbtXApOAsbU3cPcZ7l4aX3wX0InxQ9hcUs4Vj8wiw4yJVw2jlWZQE5HjLMii0BFYV2t5fXzdwVwNvHSgB8zsGjObY2ZztmzZchwjJo+dpZV87W+z2Lankr9dOZSurfLCjiQiKaheNDSb2eVAIXD3gR5394fdvdDdC1u3bl234eqBPRXVXPHIbFZt2ctfvlbIoIJmYUcSkRQV5AnpYqD2ADyd4us+xcxGAf8LnObuFQHmSUrlVRG+OXEOi4p38dBlQxjZo1XYkUQkhQV5pDAb6Glm3cwsBxgHTKm9gZkNBv4MjHH3zQFmSUrlVRG++egc3v1oG7+/aCBf6Nsu7EgikuICKwruXg1cB0wHlgKT3X2xmd1uZmPim90NNAb+YWbzzWzKQXaXdvYVhDeLtnLXBQM4d/ChmmNERI6PQPszuvs0YNp+626pdX9UkK+frMoqYwXhrZVbufvCgVx4sjpliUjdUCf3emZ3eRXfmDCH2Wu2qyCISJ1TUahHtu6p4Irxs1i+sYT7xg1mzMAOYUcSkTSjolBPFO8s46t/fY+Pd5XxlysKOaNXm7AjiUgaUlGoBxYV7+LrE2ZTVhnhsauHM7SrhsAWkXDUi4vX0tmMZZu5+M/vkJ2ZwdPf/owKgoiESkcKIXF3Hn1nDb94cTF9OjRh/BVDNWuaiIRORSEEFdURfv78IibPWc+ok9pw37jB5DXQr0JEwqdPojq2eXc51z4+l/fX7uS7Z/bg+6NO1HwIIlJvqCjUobeKtnLDpPmUVlbz0GVDGN2/fdiRREQ+RUWhDkSizv2vreD+11fQo3VjnvjmcE5smx92LBGR/6KiELD1O0r54eQPeO+j7VwwpBN3nNtXs6WJSL2lT6eAuDvPzivmtimLceB3F2nIChGp/1QUArBpdzm3vLCI6Ys3MaxrC35/8UAKWjQKO5aIyGGpKBxH7s6k2ev49bSlVFRHuXl0b775/7qTqd5FIpIkVBSOk+UbS7jlhUW899F2hndrwZ0XDKCb5lEWkSSjonCMdpVV8YdXP+TRd9aQn5vFb87vzyWFBbr2QESSkorCUaqsjvLEe2u4//UidpRW8pVhnbnxC71onpcTdjQRkaOmonCEolFn2qIN3D19OWu2lXJK95b875dOol/HpmFHExE5ZioKCYpGnemLN3LfaytYtrGEXm3zeeSqoZx+YmvMdKpIRFKDisJhVEWiTF2wgT/9eyXLNpbQvXUe940bxJcHdFCvIhFJOSoKB1FSXsXkOesZ/+ZHFO8so0ebxvzhkkGcM1DFQERSl4rCflZsKuHRd9bw7Lz17K2MMKxrC24f25czerVRjyIRSXkqCsDeimqmLtjAU3PWMXfNDnKyMjhnQAe+dkoXBhY0CzueiEidSduiUB2J8vbKbTw/v5jpizaytzJC99Z5/GR0by48uRMtGzcIO6KISJ1Lq6JQFYky66PtvLRoAy8v2sTWPRXk52bxpQHtuaiwgMIuzdWTSETSWtoUhUmz1nLny8vYWVpFw+xMzujdmjEDO3J6r9bkZmeGHU9EpF5Im6LQrmkuZ/Rqw1n92nHaiSoEIiIHkjZF4fRebTi9V5uwY4iI1GsZYQcQEZH6Q0VBRERqqCiIiEgNFQUREakRaFEws7PMbLmZFZnZzQd4vIGZPRV//D0z6xpkHhERObRn+w6RAAAGMUlEQVTAioKZZQIPAqOBPsClZtZnv82uBna4ew/gXuC3QeUREZHDC/JIYRhQ5O6r3L0SmASM3W+bscDE+P2ngc+ZLikWEQlNkEWhI7Cu1vL6+LoDbuPu1cAuoGWAmURE5BCS4uI1M7sGuCa+uMfMlh/lrloBW49PqqSh95we9J7Tw7G85y6JbBRkUSgGCmotd4qvO9A2680sC2gKbNt/R+7+MPDwsQYysznuXnis+0kmes/pQe85PdTFew7y9NFsoKeZdTOzHGAcMGW/baYAV8TvXwi87u4eYCYRETmEwI4U3L3azK4DpgOZwHh3X2xmtwNz3H0K8DfgMTMrArYTKxwiIhKSQNsU3H0aMG2/dbfUul8OXBRkhv0c8ymoJKT3nB70ntND4O/ZdLZGRET20TAXIiJSI22KwuGG3Eg1ZlZgZjPMbImZLTazG8LOVBfMLNPM3jezf4adpS6YWTMze9rMlpnZUjM7JexMQTOz78f/Ty8ysyfNLDfsTMebmY03s81mtqjWuhZm9oqZrYjfNg/itdOiKCQ45EaqqQZ+6O59gBHA/6TBewa4AVgadog6dB/wsrv3BgaS4u/dzDoC1wOF7t6PWCeWVOygMgE4a791NwOvuXtP4LX48nGXFkWBxIbcSCnuvsHd58XvlxD7sNj/ivKUYmadgC8Bfw07S10ws6bAqcR68eHule6+M9xUdSILaBi/tqkR8HHIeY47d59JrEdmbbWHBZoInBvEa6dLUUhkyI2UFR99djDwXrhJAvcH4EdANOwgdaQbsAV4JH7K7K9mlhd2qCC5ezHwO2AtsAHY5e7/CjdVnWnr7hvi9zcCbYN4kXQpCmnLzBoDzwDfc/fdYecJipl9Gdjs7nPDzlKHsoAhwEPuPhjYS0CnFOqL+Hn0scQKYgcgz8wuDzdV3Ytf5BtI19F0KQqJDLmRcswsm1hB+Lu7Pxt2noCNBMaY2WpipwfPNLPHw40UuPXAenffdwT4NLEikcpGAR+5+xZ3rwKeBT4Tcqa6ssnM2gPEbzcH8SLpUhQSGXIjpcSHIP8bsNTd7wk7T9Dc/Sfu3snduxL7/b7u7in9DdLdNwLrzKxXfNXngCUhRqoLa4ERZtYo/n/8c6R443ottYcFugJ4IYgXSYpRUo/VwYbcCDlW0EYCXwUWmtn8+Lqfxq8yl9TxXeDv8S87q4CrQs4TKHd/z8yeBuYR62H3Pil4ZbOZPQmcDrQys/XArcCdwGQzuxpYA1wcyGvrimYREdknXU4fiYhIAlQURESkhoqCiIjUUFEQEZEaKgoiIlJDRUEkzsxWm1mrI3zO98ysUVCZROqaioLIUYqPvvs9YoOyiaQEFQVJS2b2vJnNjY/Lf80BHr/czGaZ2Xwz+3O8AGBme8zs92b2AfC/xMbfmWFmM2o9/isz+8DM3jWztvH1rc3sGTObHf8ZGV9/Wvw15scHtcs3s/ZmNjO+bpGZ/b86+4eRtKeL1yQtmVkLd99uZg2JDYNyGjAXKARaA3cB57t7lZn9EXjX3R81MwcucffJ8f2sJja2/9b4sgNj3P1FM7sL2O3uvzSzJ4A/uvubZtYZmO7uJ5nZi8Cd7v5WfPDCcmJzQuS6+6/ixahRfPhzkcClxTAXIgdwvZmdF79fAPSs9djngJOB2bHhdWjIJ4OPRYgNMngwlcC+Wd/mAp+P3x8F9InvD6BJvAi8BdxjZn8HnnX39WY2GxgfH9DweXefj0gdUVGQtGNmpxP7kD7F3UvN7A2g9pSOBkx0958c4Onl7h45xO6r/JPD7wif/I1lACPcvXy/7e80s6nA2cBbZvZFd59pZqcSmzBogpnd4+6PHsl7FDlaalOQdNQU2BEvCL2JTVda22vAhWbWBmrmxu1ykH2VAPkJvOa/iA1eR3yfg+K3J7j7Qnf/LbHTWL3jr7XJ3f9CbBa5VB8OW+oRFQVJRy8DWWa2lNjIk+/WftDdlwA/A/5lZguAV4D2B9nXw8DL+xqaD+F6oNDMFpjZEuBb8fXfizcmLwCqgJeIjY75gZm9D1xCbB5mkTqhhmYREamhIwUREamhoiAiIjVUFEREpIaKgoiI1FBREBGRGioKIiJSQ0VBRERqqCiIiEiN/w/PzmWiy/xC5gAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "xspace = np.linspace(0, 10, 100)\n",
    "plt.xlabel('alertness')\n",
    "plt.ylabel('crippling anxiety')\n",
    "plt.plot(xspace, crippling_anxiety(xspace))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Fiction 8**. Relationship between intoxication level and productivity is modelled by the [Ballmer function](https://xkcd.com/323/):"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![Ballmer function](https://imgs.xkcd.com/comics/ballmer_peak.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "def gaussian(x, mu, sig):\n",
    "    return np.exp(-np.power(x - mu, 2.) / (2 * np.power(sig, 2.)))\n",
    "\n",
    "def ballmer_function(intoxication):\n",
    "    return sigmoid((0.05 - intoxication) * 50) + 2 * gaussian(intoxication, 0.135, 0.005)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x7fb76a777410>]"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAAEKCAYAAAAB0GKPAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xd4HOW1+PHvWa16syzJvYMxGDA2CNvE9FCcAqQQsIHQcQqkkf4jN+SSe29CctMhAeM4QEIPJU4uJTRTAjaWjStgcIyNZWwsVxWr6/z+mFmxllVG0s7OaHU+z7OPZqee8a519JZ5X1FVjDHGmO5Egg7AGGNM/2AJwxhjjCeWMIwxxnhiCcMYY4wnljCMMcZ4YgnDGGOMJ5YwjDHGeGIJwxhjjCeWMIwxxngSDTqARCopKdFx48YFHYYxxvQby5cv36mqpV72TamEMW7cOMrLy4MOwxhj+g0R2ex1X6uSMsYY44klDGOMMZ5YwjDGGOOJJQxjjDGeWMIwxhjjiSUMY4wxnviWMERktIg8LyJviMg6EflaB/uIiPxWRDaIyGoROTZu22Ui8o77usyvOI0xxnjjZwmjGfimqk4GZgLXisjkdvt8DJjovuYBfwAQkcHAjcAMYDpwo4gU+RirMUmlqjz6egUrt+wNOhRjPPMtYajqNlVd4S5XA28CI9vtdh5wtzqWAINEZDhwNvC0qu5W1T3A08Bsv2I1JtmeX7+DbzywiovvWEJTS2vQ4RjjSVLaMERkHDANWNpu00hgS9z7CnddZ+uNSQmL11cCUNvYQvmmPQFHY4w3vicMEckDHga+rqpVPpx/noiUi0h5ZWVlok9vjC9ef28vhw3NA+CNbQn/b2GML3xNGCKSjpMs7lHVRzrYZSswOu79KHddZ+sPoqrzVbVMVctKSz2Nn2VMoOqbWnhzWxUfPWIoxbkZrN9uCcP0D372khLgj8CbqvrLTnZbBFzq9paaCexT1W3AU8BZIlLkNnaf5a4zpt/bureO5lblsKF5TBqWz/rt1UGHZIwnfo5WOwv4PLBGRFa66/4fMAZAVW8DHgc+DmwA9gNXuNt2i8iPgWXucTep6m4fYzUmaSr21AEwqiiHQ0rz+NvKDgvPxoSObwlDVV8GpJt9FLi2k20LgYU+hGZMoCr27AdgVFE2wwdlUVXfTG1DM7mZKTXbgElB9qS3MUm2ZXcd6WnC0PwshhdmAbBtX33AURnTPUsYxiTZ1r11jBiUTSQiDC/MBmDbvrqAozKme5YwjEmyndUNDMnPBGBEW8KwEoYJP0sYxiTZrtoGinOdhDG00Pm53RKG6QcsYRiTZLtqGinOywAgM5pGflaU3bWNAUdlTPcsYRiTRC2tyu79jRTnZbatG5ybwS5LGKYfsIRhTBLt2d+IKpS4JQxwEsYeSximH7CEYUwS7apxEkOsDQNgcE6GVUmZfsEShjFJtKumAaCtDQOcEoYlDNMfWMIwJol2uomhfZXU7v2NOAMfGBNeljCMSaK2EkbugY3ejc2t1Da2BBWWMZ5YwjAmiXbWNJAWEQqz09vWFeU6pQ1r+DZhZwnDmCTaXdtEUU46kciH43IOzslwt1nCMOFmCcOYJKqqb6IgrnQBMDjPEobpHyxhGJNEVXVNFGS1SxhuCcMe3jNh59sA/CKyEPgksENVj+pg+7eBi+PiOAIodSdP2gRUAy1As6qW+RWnMclUVd98QPsFQJGbMPbVNQURkjGe+VnCuBOY3dlGVf25qk5V1anA94EX2s2qd5q73ZKFSRnVdU0UZB34d1qe+77KEoYJOd8Shqq+CHidVnUucJ9fsRgTFlX1TQeVMNIiQn5mlKp6Sxgm3AJvwxCRHJySyMNxqxX4p4gsF5F5wURmTGKpKlV1zQc1egMUZKdTVdccQFTGeBeGSYTPAf7VrjrqRFXdKiJDgKdF5C23xHIQN6HMAxgzZoz/0RrTS/VNrTS2tB7U6A2Qn2UlDBN+gZcwgDm0q45S1a3uzx3Ao8D0zg5W1fmqWqaqZaWlpb4GakxfxBJCQfbBf6cVZKdTbQnDhFygCUNECoFTgL/FrcsVkfzYMnAWsDaYCI1JnFijdkcljIIsq5Iy4ednt9r7gFOBEhGpAG4E0gFU9TZ3t08D/1TV2rhDhwKPikgsvntV9Um/4jQmWT4sYXTUhhHlre1WwjDh5lvCUNW5Hva5E6f7bfy6jcAx/kRlTHBiJYj23WqddenWrdaEXhjaMIwZEGIljPbdasFJItUNzbS22hDnJrwsYRiTJG1tGJ10q1WFmkZrxzDhZQnDmCSpqneSQX4nVVJgT3ubcLOEYUyS7KtrIis9QmY07aBtsa621lPKhJklDGOSpKORamPaShj2LIYJMUsYxiRJR3NhxMTWW5WUCTNLGMYkSVXdwUObx+RlOlVSNQ1WJWXCyxKGMUlSVd/UYYM3QK6bMGotYZgQs4RhTJLU1De3lSTaiyWSmoaWZIZkTI9YwjAmSWoaOk8YmdEIaRGhpsHaMEx4WcIwJklqG5rbqp7aExFyM9KotRKGCTFLGMYkQWurUtvY0mnCAMjPSqe63towTHhZwjAmCfY3OSWHvMyDH9qLyc1Ms0ZvE2qWMIxJglgi6KqEkZcZpdbGkjIhZgnDmCSIPV/RWaM3OMnEqqRMmFnCMCYJauq7Txh5mVGrkjKh5lvCEJGFIrJDRDqcXlVEThWRfSKy0n39MG7bbBFZLyIbROR7fsVoTLJ4rpKyhGFCzM8Sxp3A7G72eUlVp7qvmwBEJA24FfgYMBmYKyKTfYzTGN95rpKyhGFCzLeEoaovArt7ceh0YIOqblTVRuB+4LyEBmdMksUas7vuVuuUMFRt1j0TTkG3YZwgIqtE5AkROdJdNxLYErdPhbvOmH4rNuRHbpfdaqO0KtQ12cN7JpyCTBgrgLGqegzwO+Cx3pxEROaJSLmIlFdWViY0QGMSpdZjlRTYiLUmvAJLGKpapao17vLjQLqIlABbgdFxu45y13V2nvmqWqaqZaWlpb7GbExv1TY0ExHITu+8hJHfNmKtlTBMOAWWMERkmIiIuzzdjWUXsAyYKCLjRSQDmAMsCipOYxKhxh1Hyv3Kd6ithGHPYpiQ6rx83Ecich9wKlAiIhXAjUA6gKreBpwPfElEmoE6YI46rX3NInId8BSQBixU1XV+xWlMMnQ1tHmMTaJkws63hKGqc7vZfgtwSyfbHgce9yMuY4JQ29j5SLUxljBM2HX6DRaRvwOd9u9T1XN9iciYFFTT0PVItfBhDyp7eM+EVVff4P9NWhTGpLjahuYuR6oFyMuyEoYJt04Thqq+kMxAjElltQ3NFOfmdLmPVUmZsOuqSmoNXVdJTfElImNSUFfTs8Zkp6cREauSMuHV1Tf4k0mLwpgU19X0rDEiYkOcm1DrqkpqczIDMSaV1Ta0tLVRdMVGrDVh1lWV1MuqeqKIVHNg1ZQAqqoFvkdnTApoaG6hsaW12yopsFn3TLh1VcI40f2Zn7xwjEk9saE+cjO67iUFNuueCbduhwYRkas6WPdTf8IxJvV4mTwpxqqkTJh5GUvqsyJyceyNiNwKDPEvJGNSi5fJk2LyMqPWrdaElpehQT4LLBKRVpwZ9Paq6pX+hmVM6uhJCSM3M2qj1ZrQ6qrRe3Dc26tx5qv4F/CfIjJYVXszm54xA05Nj6qk0qyEYUKrq2/wcpzeURL38xPuS4EJvkdnTAqIlRi8VEnlZn44TWtXQ6EbE4SuekmNT2YgxqSqttn2PDyHkZsZpblVaWhuJauLyZaMCYKXXlKfE5F8d/kHIvKIiEzzPzRjUkN1LGFkeGv0BhsexISTl15S/6Gq1SJyInAG8EfgNn/DMiZ1fNjo7e05DOcYa/g24eMlYcS+uZ8A5qvq/wEZ3R0kIgtFZIeIrO1k+8UislpE1ojIKyJyTNy2Te76lSJS7uVGjAmr2oZmMqMRomnd/3eLDYFuDd8mjLwkjK0icjtwIfC4iGR6PO5OnG64nXkXOEVVjwZ+DMxvt/00VZ2qqmUermVMaHkZqTamrYRhw4OYEPLyi/8CnPm1z1bVvcBg4NvdHaSqLwKddr1V1VdUdY/7dgkwykMsxvQ7Xkaqjcm1OTFMiHX7LVbV/cAjce+3AdsSHMdVwBPxlwX+KSIK3K6q7UsfbURkHjAPYMyYMQkOy5i+8zI9a4w1epsw8/Yt9pGInIaTME6MW32iqm4VkSHA0yLylltiOYibTOYDlJWVdTrhkzFB8TI9a0yuJQwTYl6qpHwjIlOABcB5qrortl5Vt7o/dwCPAtODidCYvqtt9N6GEet6W2O9pEwIBZYwRGQMTlXX51X17bj1uXHPfeQCZwEd9rQypj+oqe9JG4ZTErEShgmjbr/FHUygBLAPKAe+qaobOznuPuBUoEREKoAbgXQAVb0N+CFQDPzeHQKh2e0RNRR41F0XBe5V1Sd7fGfGhERPeklF0yJkRiOWMEwoefkW/xqoAO7FGU9qDnAIsAJYiJMUDqKqc7s6qapejTOoYfv1G4FjDj7CmP6pJ72kwB1PyrrVmhDyUiV1rqrerqrVqlrlNjKfraoPAEU+x2dMv9baqtQ2eu8lBU61lD3pbcLIS8LYLyIXiEjEfV0A1LvbrFeSMV3Y3xQbqdb7QIK5GTaJkgknLwnjYuDzwA7gA3f5EhHJBq7zMTZj+r2eTJ4UY9O0mrDy8uDeRuCcTja/nNhwjEktPZmeNSY3M8re/Y1+hWRMr3npJVUKXAOMi9/fpmk1pnu1vUgYeZlRKvbs9yskY3rNy7f4b8BLwDN8OHKtMcaDmvqeV0lZo7cJKy/f4hxV/a7vkRiTgnpbJWVtGCaMvDR6/0NEPu57JMakoNjzFD1u9G505vU2Jky8JIyv4SSNOhGpEpFqEanyOzBjUkFsTCgvs+3F5GZGaVWoa7JqKRMuXnpJ5ScjEGNSUW8avePnxMjxMA+4McnS6bdRRA5X1bdE5NiOtqvqCv/CMiY11DY0ExHITvdewshrG4CwBezPNRMiXf35cj3OxES/6GCbAqf7EpExKaSmoZncjCjuYJqe5GbYnBgmnDpNGKo6z/15WvLCMSa11DY0k5fVs2qlPJum1YSUlwf30oBPcPCDe7/0LyxjUkNND0eqBZt1z4SXl2/y33EGG1wDtPobjjGppSfzecfkWgnDhJSXb/IoVZ3Sm5OLyELgk8AOVT2qg+0C/Ab4OLAfuDzWmC4ilwE/cHf9L1W9qzcxGBOknsznHZPXVsKwbrUmXLw8h/GEiJzVy/PfCczuYvvHgInuax7wBwARGYwzQ98MnPm8bxQRm3vD9Du1bqN3T9g0rSasvCSMJThTpvb4wT1VfRHY3cUu5wF3q2MJMEhEhgNnA0+r6m5V3QM8TdeJx5hQ6sn0rDGxBGNVUiZsvHyTfwmcAKzRxI9VMBLYEve+wl3X2Xpj+pWeTs8KEIkIORlpVsIwoeOlhLEFWOtDskgIEZknIuUiUl5ZWRl0OMYcoLYXjd5g83qbcPLyTd4ILBaRJ4CG2MoEdavdCoyOez/KXbcVOLXd+sUdncCdY3w+QFlZWSiTmhmYGppbaGxp7XGjNzgN3zXW6G1CxksJ413gWSADZ6CC2CsRFgGXimMmsE9VtwFPAWeJSJHb2H2Wu86YfiPWy6mnbRgQmxPDShgmXLwMPvifvT25iNyHU1IoEZEKnJ5P6e55bwMex+lSuwGnW+0V7rbdIvJjYJl7qptUtavGc2NCpzfzecfkZkSt0duEjpcnvcuAG4CxHPikd7fPZqjq3G62K3BtJ9sWAgu7u4YxYdWbyZNi8jKjbK+qT3RIxvSJl2/yPcC3sSe9jemRPpUwbNY9E0JevsmVqrrI90gC9LMn32Li0DxOP3wohdnpQYdjUkRNHxOGNXqbsPHyTb5RRBbgNHzH95J6xLeokqiusYVHX9/Ktn31RCPCCYcUM/uoYZw5eShD8rOCDs/0Y31p9M6zRm8TQl6+yVcAh+M0VseqpBRIiYSRnZHGv757Oqsq9vLkuu08tXY7Nzy6lv94bC0nTSzls8eN4qzJQ8nqwQQ4xkB8lVTPvzu5mVHqmlpoaVXSIt7n0jDGT14SxvGqOsn3SAIUiQjTxhQxbUwR35t9OG9/UMPfV73PIysq+Op9r5OfFeWcY0Zw0fQxHDWyMOhwTT/R10ZvgNrGZgqyrJrUhIOXb/IrIjJZVd/wPZoQEBEmDctn0rBJXH/mYby6cRcPL6/g0RVbuXfpe0wfN5jLZ43jrMlDiaZ5eYzFDFR9bcMAp5RiCcOEhZdv8kxgpYi8i9OGITg9Yns15Hl/EokIsw4tYdahJdx4bhMPlW/hrlc38eV7VjCiMIvLZ43j4hlje/ULwaS+2oZmMqMR0nvxh4VNomTCyMtvOhslFijMTufqkyZwxazxPPfWDha+/C7/8/hb/GHxv7n6pAlcesJY8u0vQROnNyPVxsSGE7GeUiZMvDzpvRlARIYAA77bUFpEOHPyUM6cPJTlm/fwu+fe4edPrWf+ixuZd/IErjpxvDWQG6B3I9XGxIY4txKGCZNuy8oicq6IvIMzptQLwCbgCZ/j6heOG1vEnVdMZ9F1sygbW8TPn1rPaf+7mIeXV9DaauMgDnS9mZ41xqZpNWHkpXL1xzjtGG+r6njgoziTKhnXlFGD+OPlx3P/vJmU5mfyzYdW8cnfvcySjbuCDs0EqDfTs8bkWRuGCSEvCaNJVXcBERGJqOrzQJnPcfVLMycU89iXZ/GbOVPZV9fEnPlL+OaDq9hV09D9wSblVDc09bpdyxq9TRh5SRh7RSQPeBG4R0R+A9T6G1b/FYkI500dyTPXn8KXTz2Ev63cykd/+QIPLHvPqqkGmOr6ZvKzetvoHauSskZvEx5eEsZ5OEOPfwN4Evg3cI6fQaWC7Iw0vjP7cB7/2kkcNiSf7z68hosXLKViz/6gQzNJ0peEkZUeISJWwjDh0mXCEJE04B+q2qqqzap6l6r+1q2iMh4cNjSfB74wk5985mhWV+xl9q9f4qHyLYR0xluTIKpKdX3vq6RExB2A0BKGCY8uE4aqtgCtImLjYfSBiDB3+hie/PrJTB5ewLf/upp5f17OTmvbSFkNza00tWivSxgA+ZYwTMh4qZKqAdaIyB9F5Lexl5eTi8hsEVkvIhtE5HsdbP+ViKx0X2+LyN64bS1x21JiePXRg3O4b95Mbvj4EbzwdiWf+O1LLLWeVCmpqr4JoE8Pc+ZnpVPtnseYMPDy588j9GJkWrc661bgTKACWCYii+LHpFLVb8Tt/xVgWtwp6lR1ak+vG3ZpEeGakycw69ASrr13BXPvWMI3z5rEl045hIiNSpoyquudkkFBX0oYWdG28xgTBl6e9L6rl+eeDmxQ1Y0AInI/TgN6Z4MYzsWZ83tAmDyigEXXzeL7j6zh50+tZ9mm3fzqgqkU5WYEHZpJgNgv+r5USRVkp7Oj2qZpNeHh5UnvNSKyut3rJbc6qbiLQ0cCW+LeV7jrOrrGWGA88Fzc6iwRKReRJSLyqS7im+fuV15ZWdnd7YRKflY6v5s7jf/61FG8smEX5976Mm9/UB10WCYBqhNSJWUlDBMuXtowngD+D7jYff0dKAe2A3cmKI45wF/dRvaYsapaBlwE/FpEDunoQFWdr6plqlpWWlqaoHCSR0S4ZOZYHvjCTBqaWvn0rf/imTc+CDos00eJKGFYwjBh4yVhnKGq31fVNe7rBuAUVb0ZGNfFcVuB0XHvR7nrOjIHuC9+hapudX9uBBZzYPtGypk2pohF153IIUPyuObP5fx+8QbretuPVdX1vYRRkJVOVV2TfQ9MaHhJGGkiMj32RkSOB2ID5HT1588yYKKIjBeRDJykcFBvJxE5HCgCXo1bVyQime5yCTCLzts+Usawwiwe/MIJnDNlBD97cj3XP7iKxubW7g80oZOYEkY6za1KfZN9B0w4ePk2Xw0sdIcHEaAKuEpEcoGfdHaQqjaLyHXAUzgJZqGqrhORm4ByVY0ljznA/Xrgn1FHALeLSCtOUvvpQJnxLys9jd/MmcrEIXn84um3qaxu4A+XHGtzbfQz1fVNiEBeRt+qpGLnys6wIfNN8Lz0kloGHB17eE9V98VtfrCbYx8HHm+37oft3v+og+NeAY7uLrZUJSJ85aMTGVaYxfcfWcMFty/hriuOZ0jBgJ+OpN+oqm8mLyPap67SBdnp7rma7LM3oeCll1ShiPwSeBZ4VkR+YU9+J8fnykaz4LIyNu+q5dO/f4UNO2qCDsl41JdxpGJix1dZw7cJCS9tGAuBauAC91UF/MnPoMyHTp00hAfmnUBDcwvn3/YKr7+3J+iQjAfV9U1tJYTeKmirkrKEYcLBS8I4RFVvVNWN7us/gQl+B2Y+dPSoQh750iwKstK5ZMFSm5ipH0hECaPAbbeK9bgyJmheEkadiJwYeyMis4A6/0IyHRlTnMNDXzyB4YOyuWzhayxevyPokEwX+jJ5UkzseCthmLDwkjC+CNwqIptEZBNwC/AFX6MyHRpakMUD82ZySGke19xdzpNrtwcdkulEItswbABCExbdzYcRASap6jHAFGCKqk5T1dVJic4cpDgvk/uumcmRIwq59t4V/G1lZ89CmiAlImHkZKSRFpG2kW+NCVp382G0At9xl6tUtSopUZkuFeak85erZ3D8uCK+/sBKHl5eEXRIJk5fJ0+KEREbHsSEipcqqWdE5FsiMlpEBsdevkdmupSXGeXOK6bzkUOK+dZfV/Ho65Y0wmJ/YwtNLcqgPvaSAhtPyoSLlzLzhe7Pa+PWKdZTKnBZ6WksuPR4rrxzGd98cBUREc6b2uGAwCaJ9rq9mgblJCBhZKZbLykTGl6e9B6fjEBM72RnpPHHy8u44k/L+MYDK4mIcM4xI4IOa0DbU9sIwKCcvs9tUpBtJQwTHl6e9M4SketF5BEReVhEvi4iNk5BiORkRPnTFcdTNnYwX39gJf+3elvQIQ1o+2IljARUSRVkpVujtwkNL20YdwNHAr/D6VJ7JPBnP4MyPRdLGtNGD+Kr97/OE2ssaQRl7/5YlVTfSxiDctLbzmdM0LwkjKNU9SpVfd59XYOTNEzI5GZGufPK6RwzqpCv3Pc6T62z5zSCsLcuViXV9xJGUU4Ge/Y39vk8xiSCl4SxQkRmxt6IyAycGfdMCOVlRrnryukcNbKQ6+5dwXNv2ex9yRYrERQmoEqqMCedhuZW6hpbut/ZGJ95SRjHAa/EPen9KnB8bK5vX6MzvZKflc5dV05n0rB8vviXFbz4dv+a67y/27u/kaz0CFnpfZ/Dosit1oqVWowJkpeEMRsYD5zivsa76z4JnNPVgSIyW0TWi8gGEfleB9svF5FKEVnpvq6O23aZiLzjvi7ryU0Z56/bv1w1gwkluVxzdzmvbNgZdEgDxt79TW2/6Psq1nC+p9baMUzwuk0Yqrq5q1dnx4lIGnAr8DFgMjBXRCZ3sOsDqjrVfS1wjx0M3AjMAKYDN4pIUS/ub0AblJPBPVfPYMzgHK66q5zX3t0ddEgDwt66poRUR8GHDed7rR3DhICXEkZvTQc2uEOiNwL3A+d5PPZs4GlV3a2qe4CncUo1poeK8zK555oZDB+UxRV/eo3lm20+Db/t29+UkAZvgKJc5zx77eE9EwJ+JoyRwJa49xXuuvY+KyKrReSvIjK6h8caD4bkZ3HfNTMpzc/k8oWvsbpib9AhpbS9dY0Myk5UlZRzHuspZcLAz4Thxd+Bcao6BacUcVdPTyAi80SkXETKKyutcbczQwuyuPeamRTmOJMwrXt/X/cHmV7Zm8ASRuw89iyGCQM/E8ZWYHTc+1HuujaquktVG9y3C3B6ZHk6Nu4c81W1TFXLSktLExJ4qhoxKJv7rplJXmaUSxYsZf326qBDSjmqyt79TRQmKGFkpaeRlR6xNgwTCn4mjGXARBEZLyIZwBxgUfwOIjI87u25wJvu8lPAWSJS5DZ2n+WuM300enAO914zk4xohIsXLGHDjpqgQ0opdU0tNLa0JqyXFMQe3rMShgmebwlDVZuB63B+0b8JPKiq60TkJhE5193tqyKyTkRWAV8FLneP3Q38GCfpLANucteZBBhXksu918wEhIvuWMK7O2uDDilltA0LkqBeUuD0lLIqKRMGvrZhqOrjqnqYqh6iqv/trvuhqi5yl7+vqkeq6jGqepqqvhV37EJVPdR9/cnPOAeiQ0rzuPeaGTS3KhfdsYT3du0POqSUEGucTlS3WnCSj1VJmTAIutHbBOiwofn85aoZ1DW1MPeOJVTssaTRV7vdoc2L8zITds6i3HTrVmtCwRLGADd5RAF/uWoGVfVNXHTHUrbvqw86pH5tZ43Th6MkL3FtGIXZGVbCMKFgCcNw1MhC7r5yOrtrG7nojiXsqLKk0Vu7anwoYbhDnKtqws5pTG9YwjAATBtTxJ1XHM/2qnouWrC07S9l0zOVNQ1kpEUoyPIy+7E3g3MzaG5Vqups5j0TLEsYpk3ZuMEsvPx4Kvbs55IFS9umGjXe7apppDgvAxFJ2DlL3NJKpSVxEzBLGOYAMycUs+DS49m4s5ZL/riUfdads0d21TS0/YJPlNj5rNRngmYJwxzkxIklzP/8cbzzQQ2XLlxqc0r3wE63hJFIJfkZ7rktYZhgWcIwHTp10hB+f/GxrHu/iiv+tIyaBqs/92JXTQPFuT6VMKotYZhgWcIwnTpj8lBuuWgaK7fs5UpLGt1SVXbWNLaVCBKlKCeDiDilF2OCZAnDdGn2UcP59YVTWf7eHi66Y0nbg2nmYNUNzTS2tFKS4BJGWkQYnJtpVVImcJYwTLfOOWYEt19yHOu3V3PB7a+ybV9d0CGF0o4q5xd6aX5iE0bsnJYwTNAsYRhPzpg8lLuvnM4H++o5/w+vsrHSRrltL/aU/PDCrISfuyQvg0qrkjIBs4RhPJsxoZj75s2kvqmFz932Kmu32iRM8WIlr+GF2Qk/d2lepjV6m8BZwjA9ctTIQh764glkRiPMmb+EF962WQ5jYiWMIQU+VEkVZFJZ3WDDg5hAWcIwPTahNI+Hv/wRRg/O4co7l3Hv0veCDikUtlXVU5ybQVZ6WsJ6RFTfAAASA0lEQVTPPbwgi8aWVnZZpwMTIEsYpleGF2bz0BdP4KSJJfy/R9fwkyfepLV1YP/1u21vHcN8aL8AGD4o272GDQxpguNrwhCR2SKyXkQ2iMj3Oth+vYi8ISKrReRZERkbt61FRFa6r0XtjzXBy8uMsuDSMi6ZOYbbX9jItfeuoK6xJeiwArNtX70vDd4AI9x2kfeth5oJkG8JQ0TSgFuBjwGTgbkiMrndbq8DZao6Bfgr8LO4bXWqOtV9nYsJpWhahB+fdxQ/+MQRPLluO+ff9gpbdg/MiZi2V9X7WMJwzrttryUMExw/SxjTgQ2qulFVG4H7gfPid1DV51U19ttlCTDKx3iMT0SEq0+awIJLy3hv937OueVlXnpnYDWG729sZu/+Jl96SAEU52aQEY2wzSa4MgHyM2GMBLbEva9w13XmKuCJuPdZIlIuIktE5FOdHSQi89z9yisrB9YvqbD56BFD+ft1JzI0P4vLFr7G7xdvGDC9eja7c6KPLc7x5fwiwvDCLN63hGECFIpGbxG5BCgDfh63eqyqlgEXAb8WkUM6OlZV56tqmaqWlZaWJiFa05VxJbk8eu1H+MSUEfzsyfXM+/PyATGvxuZdtQCMK8717RrDCrKsSsoEys+EsRUYHfd+lLvuACJyBnADcK6qtj2ZpKpb3Z8bgcXANB9jNQmUkxHlt3Om8h+fnMzi9Tv42G9e4pV/7ww6LF9tcksYY3wqYQCMLMqmYo8lDBMcPxPGMmCiiIwXkQxgDnBAbycRmQbcjpMsdsStLxKRTHe5BJgFvOFjrCbBRISrThzPo1+eRU5GGhcvWMrPnnyLppbWoEPzxeZdtRTnZlCQle7bNcYV57K9qn5A90QzwfItYahqM3Ad8BTwJvCgqq4TkZtEJNbr6edAHvBQu+6zRwDlIrIKeB74qapawuiHjhpZyD++eiIXlo3m94v/zad//y/eeL8q6LASbtPO/b61X8SMK3GquzbvrvX1OsZ0JnEz1XdAVR8HHm+37odxy2d0ctwrwNF+xmaSJycjyk8/O4VTJw3hB4+t5dxbXuZLpx7CdacfSmY08U9FB2HzrlpmTij29RoT3ITxbmUthw8r8PVaxnQkFI3eZmCYfdQwnrn+ZM6dOoLfPbeBT/z2Zco37Q46rD6rqm/i/X31HDIkz9frxEoY7+6yEoYJhiUMk1SDcjL45QVTufOK46lrbOH8217l6/e/3jZwX3/01rZqACYP9/ev/rzMKCV5mWzaaQnDBMMShgnEqZOG8M9vnMx1px3K42u3c/ovFnPr8xuob+p/DbpvvO8M8z55hP/VRBNKctlYaQnDBMMShglMbmaUb509iWe+cQonHlrCz59az0d/8QIPLttCcz/qTfXmtmqKczMY4sNMe+1NGpbPW9urB/xAjyYYljBM4MYU5zD/0jLuuXoGJXkZfOfh1Zz1qxdZtOr9fvGLcd22fRwxvAAR8f1aR44ooKahmfcG6HhdJliWMExozDq0hMeuncX8zx9HelqEr973Omf+yilxNDSHs6qqur6JN7dVM23MoKRc78gRhQCsS8GuySb8LGGYUBERzjpyGE987SR+O3camdE0vvPwak66+Xn+sPjf7NvfFHSIByjftIeWVuUEn7vUxhw2LI9oRFj7vk2Pa5LP1+cwjOmtSEQ495gRnDNlOC9v2MntL2zk5iff4tfPvM0np4zgohljOHbMoKRUA3Xl1Y27yEiLcOzYoqRcLzOaxqRh+ax8b29SrmdMPEsYJtREhJMmlnLSxFLWvb+Pe5a+x99e38rDKyo4fFg+F5SN5pPHDGdIvj/zUHTnxbcrOXbsIF+mZe3MzAnF/HnJZuqbWpJ6XWOsSsr0G0eOKOR/Pn00S284g5985mjS0yLc9I83mPk/z3LRHUu4/7X3klpltWFHNW9tr2b2kcOSdk2AWYcW09jcyvLNe5J6XWOshGH6nbzMKHOnj2Hu9DG880E1f1/1PotWvc/3HlnDDx5by/HjBvPRI4Zw+uFDmFDq39PXDyzbQlpE+PjRw327Rkemjy8mGhFeemcnsw4tSeq1zcAmqTTBTVlZmZaXlwcdhgmAqrJ2axWPr93Gc2/uYP0HztPX40tyOXliCTMnFDN9/GCK8xLzrMSe2kZO/vnznHJYKbdcdGxCztkTlyxYyubdtbz47dMCb8cx/ZuILHfnHuqWlTBMShARjh5VyNGjCvnu7MPZsns/z6/fwbNv7uDB8gruenUzAIcNzWPmhGKOG1vE0SMLGVecSyTS81+4Nz/5FrUNzXzl9ImJvhVPPnPsSK5/cBXlm/dw/LjBgcRgBh5LGCYljR6cw6UnjOPSE8bR2NzKmq37WLJxF0s27uKh8grudhNIflaUKaMKmTJqEEeOKODQIXmML8ntdBTdllblN8++w/3LtvDFUw5h0rD8ZN5Wm7OPHEZe5joWvvyuJQyTNJYwTMrLiEY4bmwRx40t4trTDqW5pZW3P6hhdcVeVlXsY3XFXu54cSPN7lPlaRFh7OAcJpTmMawwk+LcTKIRYWdNAy+8XcmmXfs5/7hRfOuswwK7p9zMKFeeOJ7fPvsOK7fsZero5Dw4aAY2X9swRGQ28BsgDVigqj9ttz0TuBs4DtgFXKiqm9xt3weuAlqAr6rqU91dz9owTG/VN7WwsbKWd3ZUs2FHDRt21PDvyhoqqxvY4/a8yk5P49ixg7h4xlg+dtSwwNsOquqbOPOXL5CVnsbfrp3FoJyMQOMx/VNP2jB8Sxgikga8DZwJVOBM2To3fuY8EfkyMEVVvygic4BPq+qFIjIZuA+YDowAngEOU9Uux4ewhGH80NTSiqpTUgmb8k27mXvHEkYPzuHmz06hbGxR4InM9C9hafSeDmxQ1Y1uUPcD53Hg3NznAT9yl/8K3CLOt/084H5VbQDeFZEN7vle9TFeYzqUnha+RBFTNm4wf7lqBl+573U+d9urTCjJZdqYIsYW51CYnU5BdpTs9DSikQjRNCE9LUI0IkTTIqSnSdv6iEBEhIgIaRFB3Pex5TR3W0SESKTjbSJYskpxfiaMkcCWuPcVwIzO9lHVZhHZBxS765e0O3akf6Ea03/NmFDM4m+fyiMrtvL0Gx/w0juVPLyiIZBY2hJPRIilDhEQnIQCIDiJpS21CHH7uokn/r17jtjOB57HOfeH1zkwabXt20EM3fGS/Dylx252SkQsg3MyePCLJ3iJpk/6faO3iMwD5gGMGTMm4GiMCUZORpRLZo7lkpljAWhsbqW6vol9dU3UN7XS3NpKU4vS3NJKc6vS1NJKc4vS3NpKSyu0qKKqtLQqrQqtqrS6y+23edmvJVbVraA4xwBo2/vYZiW+VlxV0bblA7fHH4e7vv152h/bYQwe/j291NR7O0/Xe3lqEPCwU35Wcn6V+3mVrcDouPej3HUd7VMhIlGgEKfx28uxAKjqfGA+OG0YCYncmH4uIxqhOC8zYQ8qGgP+jiW1DJgoIuNFJAOYAyxqt88i4DJ3+XzgOXVS8iJgjohkish4YCLwmo+xGmOM6YZvJQy3TeI64CmcbrULVXWdiNwElKvqIuCPwJ/dRu3dOEkFd78HcRrIm4Fru+shZYwxxl82lpQxxgxgPelWG97+gsYYY0LFEoYxxhhPLGEYY4zxxBKGMcYYTyxhGGOM8SSlekmJSCWwuZeHlwA7ExhOWA2U+4SBc68D5T5h4NxrMu9zrKqWetkxpRJGX4hIudeuZf3ZQLlPGDj3OlDuEwbOvYb1Pq1KyhhjjCeWMIwxxnhiCeND84MOIEkGyn3CwLnXgXKfMHDuNZT3aW0YxhhjPLEShjHGGE9SPmGIyGwRWS8iG0Tkex1szxSRB9ztS0VkXNy277vr14vI2cmMuzd6e68iMk5E6kRkpfu6Ldmx94SH+zxZRFaISLOInN9u22Ui8o77uqz9sWHTx3ttiftM208tECoe7vN6EXlDRFaLyLMiMjZuW6p9pl3da7CfqbqzZKXiC2dY9X8DE4AMYBUwud0+XwZuc5fnAA+4y5Pd/TOB8e550oK+J5/udRywNuh7SOB9jgOmAHcD58etHwxsdH8WuctFQd+TH/fqbqsJ+h4SeJ+nATnu8pfivrup+Jl2eK9h+ExTvYQxHdigqhtVtRG4Hziv3T7nAXe5y38FPirOBLrnAferaoOqvgtscM8XVn251/6k2/tU1U2quhpobXfs2cDTqrpbVfcATwOzkxF0L/XlXvsTL/f5vKrud98uwZmFE1LzM+3sXgOX6gljJLAl7n2Fu67DfVS1GdgHFHs8Nkz6cq8A40XkdRF5QURO8jvYPujL55KKn2lXskSkXESWiMinEhtaQvX0Pq8CnujlsUHry71CwJ9pcmYON2G3DRijqrtE5DjgMRE5UlWrgg7M9MlYVd0qIhOA50Rkjar+O+ig+kJELgHKgFOCjsVvndxroJ9pqpcwtgKj496Pctd1uI+IRIFCYJfHY8Ok1/fqVrvtAlDV5Th1rIf5HnHv9OVzScXPtFOqutX9uRFYDExLZHAJ5Ok+ReQM4AbgXFVt6MmxIdKXew3+Mw26EcjPF04JaiNOo3WsgenIdvtcy4ENwQ+6y0dyYKP3RsLd6N2Xey2N3RtOY9xWYHDQ99Tb+4zb904ObvR+F6dxtMhdDuV9JuBei4BMd7kEeId2jatheXn87k7D+UNmYrv1KfeZdnGvgX+mgf8DJuED+jjwtvsB3OCuuwkncwNkAQ/hNGq/BkyIO/YG97j1wMeCvhe/7hX4LLAOWAmsAM4J+l76eJ/H49QN1+KUFtfFHXule/8bgCuCvhe/7hX4CLDG/YW0Brgq6Hvp430+A3zgfkdXAotS+DPt8F7D8Jnak97GGGM8SfU2DGOMMQliCcMYY4wnljCMMcZ4YgnDGGOMJ5YwjDHGeGIJw4SKO3Lu2k62LRaRPs9zLCKXi8gtPdj/VBH5Ry+vVdPD/X8kIt/qzbU8nv9yERnRh+OnisjHExmT6T8sYRgzsFwO9DphAFNxniMwA5AlDBNGURG5R0TeFJG/ikhO+x1EZK6IrBGRtSJys4f1V4jI2yLyGjCro4uKyHQRedUdhPEVEZnUwT55IvIn9xqrReSzXV3X3fbfIrLKHTBuqLtunIg8FzfnwZiu/kFEZKiIPOqeZ5WIfMRdf717zbUi8vW4c78pIneIyDoR+aeIZLvzZZQB97jzKWSLyHHugJPLReQpERnunmOxiNwsIq+5/24niUgGzgNmF7rHX9hVzCYFBf3Uo73sFf/Cmd9BgVnu+4XAt9zlxTi/8EYA7+EMaRIFngM+1cX64XHrM4B/Abd0cO0CIOounwE87C6fCvzDXb4Z+HXcMUWdXdfdrrhPzgM/A37gLv8duMxdvhJ4zF3+Uex+28X2APB1dzkNZxyw43Ce+M0F8nCe1p/m/hs2A1Pd/R8ELon/N3SX04FXgFL3/YXAwrj9fuEufxx4xl2+vKN/O3sNjJeNVmvCaIuq/std/gvwVeB/47YfDyxW1UoAEbkHOBnnl3NH62m3/gE6HlyxELhLRCa650rvYJ8zcMbhAkBV94jIyZ1c9zGgEYi1fywHznSXTwA+4y7/GSeZdOV04FL3mi3APhE5EXhUVWvd6z4CnAQsAt5V1ZVx1x3XwTknAUcBT7vToqThjFwc80g3x5sBxhKGCaP249Uka/yaHwPPq+qnxZm+dnECztmkqrH4W0je/7mGuOUWILuDfQRn7KkTujlHMuM2IWZtGCaMxohI7JfYRcDL7ba/BpwiIiUikgbMBV7oYv1Sd32xiKQDn+vkuoV8ONT05Z3s8zTOqL8AiEhRF9ftyit8WFK5GHipm/2fxZmuExFJE5FC95hPiUiOiOQCn/Zwnmog311eD5TG/q1FJF1EjuzB8WaAsYRhwmg9cK2IvInTRvCH+I2qug34HvA8zsidy1X1b92s/xHwKk77xZudXPdnwE9E5HU6/4v6v4Ait5F5FXBaZ9ft5h6/AlwhIquBzwNf62b/rwGnicganCqiyaq6AmdY89dwkuICVX29m/PcCdwmIitxqqDOB25272UlzoioXXkemGyN3gOTjVZrjDHGEythGGOM8cQShjHGGE8sYRhjjPHEEoYxxhhPLGEYY4zxxBKGMcYYTyxhGGOM8cQShjHGGE/+P0cF9B/+C/1zAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "xspace = np.linspace(0, 0.26, 1000)\n",
    "plt.xlabel('blood alcohol content')\n",
    "plt.ylabel('programming skill')\n",
    "plt.plot(xspace, ballmer_function(xspace))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "def productivity(state):\n",
    "    p = 1\n",
    "    p *= state['alertness']\n",
    "    p *= 1 - crippling_anxiety(state['alertness'])\n",
    "    p *= ballmer_function(state['intoxication'])\n",
    "    return p"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'alertness': 1.7179066415679574,\n",
       " 'hypertension': 0.3555231469621428,\n",
       " 'intoxication': 0.011052524897623732,\n",
       " 'time_since_slept': 1,\n",
       " 'time_elapsed': 1,\n",
       " 'work_done': 0}"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "state"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.1769041833292344"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "productivity(state)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "def work(state):\n",
    "    state['work_done'] += productivity(state)\n",
    "    half_hour_passed(state)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'alertness': 1.69806633583388,\n",
       " 'hypertension': 0.34485745255327854,\n",
       " 'intoxication': 0.01072094915069502,\n",
       " 'time_since_slept': 2,\n",
       " 'time_elapsed': 2,\n",
       " 'work_done': 1.1769041833292344}"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "work(state)\n",
    "state"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Hopefully, the powers of Reinforcement Learning will help us improve this state!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Writing the environment class\n",
    "\n",
    "To write own OpenAI gym environment, you have to:\n",
    "- Create a class that inherits from `gym.Env`\n",
    "- Make sure that it has `action_space` and `observation_space` attributes defined\n",
    "- Make sure it has `reset()`, `step()`, `close()` and `render()` functions defined. See our exploration of MountainCar above for an intuition on how this functions are supposed to work \n",
    "\n",
    "Let's do that, shall we?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, we need to decide what the action and observation spaces of HeartPole are going to look like.\n",
    "\n",
    "Every half an hour of working you stop and do one of the following:\n",
    "\n",
    "1. Nothing\n",
    "2. Drink coffee\n",
    "3. Drink beer\n",
    "4. Go to sleep\n",
    "\n",
    "If you've chosen 1-3, you immediately go back to work, if you've chosen sleep, you sleep for 16 half-hours and go back to work"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "def do_nothing(state):\n",
    "    pass\n",
    "\n",
    "def sleep(state):\n",
    "    \"\"\"Have 16 half-hours of healthy sleep\"\"\"\n",
    "    for hh in range(16):\n",
    "        half_hour_passed(state)\n",
    "    wakeup(state)\n",
    "\n",
    "actions = [do_nothing, drink_coffee, drink_beer, sleep]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So, it is a 4-way discrete action space"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "heartpole_action_space = gym.spaces.Discrete(len(actions))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As for the observations, we observe the state of the patient, which is the following:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'alertness': 1.69806633583388,\n",
       " 'hypertension': 0.34485745255327854,\n",
       " 'intoxication': 0.01072094915069502,\n",
       " 'time_since_slept': 2,\n",
       " 'time_elapsed': 2,\n",
       " 'work_done': 1.1769041833292344}"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "state"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So, it's a `Box` of shape `(6,)`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "observations = ['alertness', 'hypertension', 'intoxication',\n",
    "                    'time_since_slept', 'time_elapsed', 'work_done']\n",
    "\n",
    "def make_heartpole_obs_space():\n",
    "    lower_obs_bound = {\n",
    "        'alertness': - np.inf,\n",
    "        'hypertension': 0,\n",
    "        'intoxication': 0,\n",
    "        'time_since_slept': 0,\n",
    "        'time_elapsed': 0,\n",
    "        'work_done': - np.inf\n",
    "    }\n",
    "    higher_obs_bound = {\n",
    "        'alertness': np.inf,\n",
    "        'hypertension': np.inf,\n",
    "        'intoxication': np.inf,\n",
    "        'time_since_slept': np.inf,\n",
    "        'time_elapsed': np.inf,\n",
    "        'work_done': np.inf\n",
    "    }\n",
    "\n",
    "    low = np.array([lower_obs_bound[o] for o in observations])\n",
    "    high = np.array([higher_obs_bound[o] for o in observations])\n",
    "    shape = (len(observations),)\n",
    "    return gym.spaces.Box(low,high,shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Aaaaaaaaaaand, *voila!*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "class HeartPole(gym.Env):\n",
    "    def __init__(self, heart_attack_proclivity=0.5):\n",
    "        self.actions = actions\n",
    "        self.observations = observations\n",
    "        self.action_space = heartpole_action_space\n",
    "        self.observation_space = make_heartpole_obs_space()\n",
    "        self.heart_attack_proclivity = heart_attack_proclivity\n",
    "        self.log = ''\n",
    "        \n",
    "    def observation(self):\n",
    "        return np.array([self.state[o] for o in self.observations])\n",
    "        \n",
    "    def reset(self):\n",
    "        self.state = {\n",
    "            'alertness': 0,\n",
    "            'hypertension': 0,\n",
    "            'intoxication': 0,\n",
    "            'time_since_slept': 0,\n",
    "            'time_elapsed': 0,\n",
    "            'work_done': 0\n",
    "        }\n",
    "        \n",
    "        wakeup(self.state)\n",
    "        return self.observation()\n",
    "        \n",
    "    def step(self, action):\n",
    "        if self.state['time_elapsed'] == 0:\n",
    "            old_score = 0\n",
    "        else:\n",
    "            old_score = self.state['work_done'] / self.state['time_elapsed']\n",
    "        \n",
    "        # Do selected action\n",
    "        self.actions[action](self.state)\n",
    "        self.log += f'Chosen action: {self.actions[action].__name__}\\n'\n",
    "        \n",
    "        # Do work\n",
    "        work(self.state)\n",
    "        \n",
    "        new_score = self.state['work_done'] / self.state['time_elapsed']\n",
    "        \n",
    "        reward = new_score - old_score\n",
    "        \n",
    "        if heart_attack_occured(self.state, self.heart_attack_proclivity):\n",
    "            # We would like to avoid this\n",
    "            reward -= 100\n",
    "            \n",
    "            # A heart attack is like purgatory - painful, but cleansing\n",
    "            # You can tell I am not a doctor\n",
    "            self.state['hypertension'] = 0\n",
    "            \n",
    "        self.log += str(self.state) + '\\n'\n",
    "        return self.observation(), reward, False, {}\n",
    "    \n",
    "    def close(self):\n",
    "        pass\n",
    "        \n",
    "    def render(self, mode=None):\n",
    "        print(self.log)\n",
    "        self.log = ''"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And that's it! Let's do what we did with MountainCar and test this Env with a random agent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/vadim0x60/.pyenv/versions/3.7.4/lib/python3.7/site-packages/gym/logger.py:30: UserWarning: \u001b[33mWARN: Box bound precision lowered by casting to float32\u001b[0m\n",
      "  warnings.warn(colorize('%s: %s'%('WARN', msg % args), 'yellow'))\n"
     ]
    }
   ],
   "source": [
    "heartpole = HeartPole()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7722006353586487, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 17, 'work_done': 0.6580434292110483}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7869557267550472, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 34, 'work_done': 1.3273907127168645}\n",
      "\n",
      "Chosen action: do_nothing\n",
      "{'alertness': 0.7671154210209697, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 2, 'time_elapsed': 35, 'work_done': 1.9829524592255678}\n",
      "\n",
      "Chosen action: do_nothing\n",
      "{'alertness': 0.7452341500848393, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 3, 'time_elapsed': 36, 'work_done': 2.6232265437509885}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0979404225554534, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 53, 'work_done': 3.518453533440179}\n",
      "\n"
     ]
    }
   ],
   "source": [
    "observation = heartpole.reset()\n",
    "heartpole.render()\n",
    "\n",
    "for step in range(5):\n",
    "    observation, reward, done, _ = heartpole.step(heartpole.action_space.sample())\n",
    "    heartpole.render()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Registering the environment\n",
    "\n",
    "The only difference left between \"official\" gym environments and our `HeartPole` is that bundled environemnts have an id string like `CartPole-v0` and can be created with `gym.make`. If you insist, this too can be rectified. For that you need to put your environment into a Python file and specify the entry point in the format of `module:Class`.\n",
    "\n",
    "In the [source code attached to this tutorial](github.com/vadim0x60/heartpole) we have created a package installable with"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install heartpole"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "See \"[How to upload your python package to PyPi](https://medium.com/@joel.barmettler/how-to-upload-your-python-package-to-pypi-65edc5fe9c56)\" if you want to learn how to do it"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Making it official\n",
    "gym.register('HeartPole-v0', entry_point='heartpole:HeartPole')\n",
    "heartpole = gym.make('HeartPole-v0')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.956015130507798, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 17, 'work_done': 0.7952518047425368}\n",
      "\n",
      "Chosen action: drink_beer\n",
      "{'alertness': 0.7871835207383427, 'hypertension': 0.16689157358401005, 'intoxication': 0.02498348341360805, 'time_since_slept': 2, 'time_elapsed': 18, 'work_done': 1.3547797157628843}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1283641898890902, 'hypertension': 0.09943834604193133, 'intoxication': 0.01488581008414216, 'time_since_slept': 1, 'time_elapsed': 35, 'work_done': 2.1969586550340936}\n",
      "\n",
      "Chosen action: do_nothing\n",
      "{'alertness': 1.1085238841550127, 'hypertension': 0.09645519566067338, 'intoxication': 0.014439235781617894, 'time_since_slept': 2, 'time_elapsed': 36, 'work_done': 3.0307809359414644}\n",
      "\n",
      "Chosen action: do_nothing\n",
      "{'alertness': 1.0866426132188822, 'hypertension': 0.09356153979085317, 'intoxication': 0.014006058708169357, 'time_since_slept': 3, 'time_elapsed': 37, 'work_done': 3.854774107432126}\n",
      "\n"
     ]
    }
   ],
   "source": [
    "observation = heartpole.reset()\n",
    "heartpole.render()\n",
    "\n",
    "for step in range(5):\n",
    "    observation, reward, done, _ = heartpole.step(heartpole.action_space.sample())\n",
    "    heartpole.render()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Testing the environment"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: keras-rl in /home/vadim0x60/.pyenv/versions/3.7.4/lib/python3.7/site-packages (0.4.2)\n",
      "Requirement already satisfied: keras>=2.0.7 in /home/vadim0x60/.pyenv/versions/3.7.4/lib/python3.7/site-packages (from keras-rl) (2.3.1)\n",
      "Requirement already satisfied: keras-applications>=1.0.6 in /home/vadim0x60/.pyenv/versions/3.7.4/lib/python3.7/site-packages (from keras>=2.0.7->keras-rl) (1.0.8)\n",
      "Requirement already satisfied: pyyaml in /home/vadim0x60/.pyenv/versions/3.7.4/lib/python3.7/site-packages (from keras>=2.0.7->keras-rl) (5.1.2)\n",
      "Requirement already satisfied: keras-preprocessing>=1.0.5 in /home/vadim0x60/.pyenv/versions/3.7.4/lib/python3.7/site-packages (from keras>=2.0.7->keras-rl) (1.1.0)\n",
      "Requirement already satisfied: h5py in /home/vadim0x60/.pyenv/versions/3.7.4/lib/python3.7/site-packages (from keras>=2.0.7->keras-rl) (2.10.0)\n",
      "Requirement already satisfied: numpy>=1.9.1 in /home/vadim0x60/.pyenv/versions/3.7.4/lib/python3.7/site-packages (from keras>=2.0.7->keras-rl) (1.19.0)\n",
      "Requirement already satisfied: six>=1.9.0 in /home/vadim0x60/.pyenv/versions/3.7.4/lib/python3.7/site-packages (from keras>=2.0.7->keras-rl) (1.12.0)\n",
      "Requirement already satisfied: scipy>=0.14 in /home/vadim0x60/.pyenv/versions/3.7.4/lib/python3.7/site-packages (from keras>=2.0.7->keras-rl) (1.3.1)\n",
      "\u001b[33mYou are using pip version 19.0.3, however version 20.2.3 is available.\n",
      "You should consider upgrading via the 'pip install --upgrade pip' command.\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "!pip install keras-rl"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To test this environment, let us use `keras-rl` to build a [SARSA](https://en.wikipedia.org/wiki/State–action–reward–state–action) agent with a simple feed-forward Q-network. Feel free to play around with the model and swap out the layers if you know what you're doing or just copy and paste the model [from one of the examples](https://github.com/keras-rl/keras-rl/blob/master/examples) if you don't. This model is from `sarsa-cartpole` [example](https://github.com/keras-rl/keras-rl/blob/master/examples/sarsa_cartpole.py)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "from keras.models import Sequential\n",
    "from keras.layers import Dense, Activation, Flatten\n",
    "from keras.optimizers import Adam\n",
    "\n",
    "from rl.agents import SARSAAgent\n",
    "from rl.policy import BoltzmannQPolicy\n",
    "\n",
    "def create_sarsa_model_for(env):\n",
    "    # We build a very simple model.\n",
    "    model = Sequential()\n",
    "    model.add(Flatten(input_shape=(1,) + env.observation_space.shape))\n",
    "    model.add(Dense(16))\n",
    "    model.add(Activation('relu'))\n",
    "    model.add(Dense(16))\n",
    "    model.add(Activation('relu'))\n",
    "    model.add(Dense(16))\n",
    "    model.add(Activation('relu'))\n",
    "    model.add(Dense(env.action_space.n))\n",
    "    model.add(Activation('linear'))\n",
    "    print(model.summary())\n",
    "    \n",
    "    # SARSA does not require a memory.\n",
    "    policy = BoltzmannQPolicy()\n",
    "    sarsa = SARSAAgent(model=model, \n",
    "                       nb_actions=env.action_space.n, \n",
    "                       nb_steps_warmup=10, policy=policy)\n",
    "    sarsa.compile(Adam(lr=1e-3), metrics=['mae'])\n",
    "    \n",
    "    return sarsa"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /home/vadim0x60/.pyenv/versions/3.7.4/lib/python3.7/site-packages/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "If using Keras pass *_constraint arguments to layers.\n",
      "Model: \"sequential_1\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "flatten_1 (Flatten)          (None, 6)                 0         \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              (None, 16)                112       \n",
      "_________________________________________________________________\n",
      "activation_1 (Activation)    (None, 16)                0         \n",
      "_________________________________________________________________\n",
      "dense_2 (Dense)              (None, 16)                272       \n",
      "_________________________________________________________________\n",
      "activation_2 (Activation)    (None, 16)                0         \n",
      "_________________________________________________________________\n",
      "dense_3 (Dense)              (None, 16)                272       \n",
      "_________________________________________________________________\n",
      "activation_3 (Activation)    (None, 16)                0         \n",
      "_________________________________________________________________\n",
      "dense_4 (Dense)              (None, 4)                 68        \n",
      "_________________________________________________________________\n",
      "activation_4 (Activation)    (None, 4)                 0         \n",
      "=================================================================\n",
      "Total params: 724\n",
      "Trainable params: 724\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "None\n"
     ]
    }
   ],
   "source": [
    "sarsa = create_sarsa_model_for(heartpole)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Time to train our agent! Not that SARSA trains in _episodes_ - it takes actions in the environment until the `done` variable becomes true. When it does, the episode is considered to have finished and updates are made to the policy to perform better next episode.\n",
    "\n",
    "Our `HeartPole` environment is infinite and never returns `done=True` (why would you ever stop working?). That's why we set `nb_max_episode_steps` to `1000` so that the agent terminates the episode after 1000 steps."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training for 80000 steps ...\n",
      "  1000/80000: episode: 1, duration: 3.096s, episode steps: 1000, steps per second: 323, episode reward: -399.952, mean reward: -0.400 [-100.000, 0.063], mean action: 2.911 [0.000, 3.000], mean observation: 1327.211 [-52.797, 15704.000], loss: 20.226103, mae: 143.311134, mean_q: -21.029920\n",
      "  2000/80000: episode: 2, duration: 3.606s, episode steps: 1000, steps per second: 277, episode reward: -99.952, mean reward: -0.100 [-99.993, 0.059], mean action: 2.980 [0.000, 3.000], mean observation: 1459.644 [0.000, 16840.000], loss: 5.012272, mae: 156.032003, mean_q: -0.189455\n",
      "  3000/80000: episode: 3, duration: 4.046s, episode steps: 1000, steps per second: 247, episode reward: -1000.176, mean reward: -1.000 [-100.008, 1.390], mean action: 2.251 [0.000, 3.000], mean observation: 680.572 [-2128.402, 10488.000], loss: 52.136068, mae: 270.010603, mean_q: -68.822053\n",
      "  4000/80000: episode: 4, duration: 4.089s, episode steps: 1000, steps per second: 245, episode reward: -199.952, mean reward: -0.200 [-100.000, 0.053], mean action: 3.000 [3.000, 3.000], mean observation: 1486.397 [0.000, 17000.000], loss: 10.069900, mae: 364.138416, mean_q: -8.081340\n",
      "  5000/80000: episode: 5, duration: 3.937s, episode steps: 1000, steps per second: 254, episode reward: -599.953, mean reward: -0.600 [-100.000, 0.847], mean action: 2.728 [0.000, 3.000], mean observation: 1377.987 [-293.150, 12728.000], loss: 43.396606, mae: 233.629282, mean_q: -38.202635\n",
      "  6000/80000: episode: 6, duration: 3.086s, episode steps: 1000, steps per second: 324, episode reward: -799.963, mean reward: -0.800 [-100.000, 0.939], mean action: 1.866 [0.000, 3.000], mean observation: 213.170 [-20.057, 3592.000], loss: 40.802704, mae: 18.055665, mean_q: -13.796689\n",
      "  7000/80000: episode: 7, duration: 3.086s, episode steps: 1000, steps per second: 324, episode reward: -299.954, mean reward: -0.300 [-100.000, 0.049], mean action: 2.410 [1.000, 3.000], mean observation: 592.105 [-8.905, 7576.000], loss: 15.198839, mae: 48.314196, mean_q: -11.497877\n",
      "  8000/80000: episode: 8, duration: 3.154s, episode steps: 1000, steps per second: 317, episode reward: -99.953, mean reward: -0.100 [-100.000, 0.044], mean action: 2.743 [1.000, 3.000], mean observation: 958.983 [-2.117, 12920.000], loss: 5.086159, mae: 100.722020, mean_q: 0.319205\n",
      "  9000/80000: episode: 9, duration: 3.237s, episode steps: 1000, steps per second: 309, episode reward: 0.048, mean reward: 0.000 [-0.616, 0.855], mean action: 2.990 [0.000, 3.000], mean observation: 1472.483 [0.000, 16920.000], loss: 0.003635, mae: 166.539033, mean_q: 1.378480\n",
      " 10000/80000: episode: 10, duration: 3.316s, episode steps: 1000, steps per second: 302, episode reward: -1000.159, mean reward: -1.000 [-100.000, 0.665], mean action: 2.472 [0.000, 3.000], mean observation: 1175.988 [-1684.488, 10520.000], loss: 78.186675, mae: 136.626915, mean_q: -45.971638\n",
      " 11000/80000: episode: 11, duration: 3.729s, episode steps: 1000, steps per second: 268, episode reward: -399.916, mean reward: -0.400 [-100.000, 0.687], mean action: 1.972 [0.000, 3.000], mean observation: 577.913 [-7.725, 7208.000], loss: 24.609802, mae: 48.075405, mean_q: -22.993989\n",
      " 12000/80000: episode: 12, duration: 3.855s, episode steps: 1000, steps per second: 259, episode reward: -99.875, mean reward: -0.100 [-100.000, 1.328], mean action: 1.836 [0.000, 3.000], mean observation: 675.090 [-0.271, 7544.000], loss: 5.332828, mae: 24.347830, mean_q: -4.479849\n",
      " 13000/80000: episode: 13, duration: 3.755s, episode steps: 1000, steps per second: 266, episode reward: -399.907, mean reward: -0.400 [-100.007, 0.071], mean action: 1.847 [0.000, 3.000], mean observation: 655.303 [-9.335, 8760.000], loss: 20.533913, mae: 38.554619, mean_q: -22.892896\n",
      " 14000/80000: episode: 14, duration: 3.802s, episode steps: 1000, steps per second: 263, episode reward: -99.968, mean reward: -0.100 [-100.000, 0.913], mean action: 2.738 [0.000, 3.000], mean observation: 1353.795 [-21.201, 15528.000], loss: 5.081051, mae: 139.492669, mean_q: -4.424322\n",
      " 15000/80000: episode: 15, duration: 3.863s, episode steps: 1000, steps per second: 259, episode reward: -199.955, mean reward: -0.200 [-100.001, 1.325], mean action: 2.773 [0.000, 3.000], mean observation: 1334.044 [-9.174, 15640.000], loss: 10.149931, mae: 108.614474, mean_q: -18.824575\n",
      " 16000/80000: episode: 16, duration: 3.902s, episode steps: 1000, steps per second: 256, episode reward: 0.048, mean reward: 0.000 [-0.498, 0.610], mean action: 2.974 [0.000, 3.000], mean observation: 1451.367 [0.000, 16792.000], loss: 0.012373, mae: 146.534260, mean_q: -0.095489\n",
      " 17000/80000: episode: 17, duration: 3.939s, episode steps: 1000, steps per second: 254, episode reward: -299.951, mean reward: -0.300 [-100.000, 0.967], mean action: 2.905 [0.000, 3.000], mean observation: 1424.668 [0.000, 16360.000], loss: 15.053367, mae: 81.096580, mean_q: -16.865607\n",
      " 18000/80000: episode: 18, duration: 4.034s, episode steps: 1000, steps per second: 248, episode reward: -399.940, mean reward: -0.400 [-100.000, 0.595], mean action: 2.628 [0.000, 3.000], mean observation: 1117.157 [-0.025, 14136.000], loss: 20.432317, mae: 36.650888, mean_q: 0.513755\n",
      " 19000/80000: episode: 19, duration: 4.078s, episode steps: 1000, steps per second: 245, episode reward: -1500.421, mean reward: -1.500 [-100.005, 0.062], mean action: 1.683 [0.000, 3.000], mean observation: 417.384 [-1655.466, 3928.000], loss: 88.058300, mae: 64.167429, mean_q: -55.269193\n",
      " 20000/80000: episode: 20, duration: 4.120s, episode steps: 1000, steps per second: 243, episode reward: -199.943, mean reward: -0.200 [-100.001, 0.059], mean action: 2.681 [0.000, 3.000], mean observation: 1105.648 [0.000, 14456.000], loss: 10.007287, mae: 427.190445, mean_q: -28.105031\n",
      " 21000/80000: episode: 21, duration: 4.154s, episode steps: 1000, steps per second: 241, episode reward: 0.048, mean reward: 0.000 [-0.649, 0.769], mean action: 2.994 [0.000, 3.000], mean observation: 1480.647 [0.000, 16968.000], loss: 0.010471, mae: 535.967341, mean_q: 11.394908\n",
      " 22000/80000: episode: 22, duration: 4.216s, episode steps: 1000, steps per second: 237, episode reward: 0.048, mean reward: 0.000 [-0.001, 0.036], mean action: 3.000 [3.000, 3.000], mean observation: 1485.769 [0.000, 17000.000], loss: 0.003345, mae: 527.625821, mean_q: 7.422901\n",
      " 23000/80000: episode: 23, duration: 4.467s, episode steps: 1000, steps per second: 224, episode reward: -99.953, mean reward: -0.100 [-100.000, 0.056], mean action: 3.000 [3.000, 3.000], mean observation: 1485.691 [0.000, 17000.000], loss: 5.067072, mae: 374.958937, mean_q: -12.792985\n",
      " 24000/80000: episode: 24, duration: 4.419s, episode steps: 1000, steps per second: 226, episode reward: -199.952, mean reward: -0.200 [-100.000, 0.656], mean action: 2.990 [0.000, 3.000], mean observation: 1475.412 [0.000, 16936.000], loss: 10.024324, mae: 327.276926, mean_q: -12.105377\n",
      " 25000/80000: episode: 25, duration: 4.391s, episode steps: 1000, steps per second: 228, episode reward: 0.048, mean reward: 0.000 [-0.222, 0.453], mean action: 2.993 [0.000, 3.000], mean observation: 1474.947 [0.000, 16936.000], loss: 0.021320, mae: 311.424805, mean_q: -0.092996\n",
      " 26000/80000: episode: 26, duration: 4.444s, episode steps: 1000, steps per second: 225, episode reward: -400.121, mean reward: -0.400 [-100.000, 0.727], mean action: 2.447 [0.000, 3.000], mean observation: 1249.076 [-1835.334, 14040.000], loss: 27.281783, mae: 144.469678, mean_q: -20.727492\n",
      " 27000/80000: episode: 27, duration: 4.522s, episode steps: 1000, steps per second: 221, episode reward: -199.949, mean reward: -0.200 [-100.000, 0.835], mean action: 2.645 [0.000, 3.000], mean observation: 1181.442 [-4.394, 15096.000], loss: 10.103268, mae: 64.858302, mean_q: -11.084994\n",
      " 28000/80000: episode: 28, duration: 4.577s, episode steps: 1000, steps per second: 218, episode reward: 0.051, mean reward: 0.000 [-0.557, 0.883], mean action: 2.838 [0.000, 3.000], mean observation: 1344.387 [0.000, 16120.000], loss: 0.041715, mae: 60.559330, mean_q: -0.572536\n",
      " 29000/80000: episode: 29, duration: 4.585s, episode steps: 1000, steps per second: 218, episode reward: -99.951, mean reward: -0.100 [-100.000, 0.577], mean action: 2.943 [0.000, 3.000], mean observation: 1414.424 [0.000, 16568.000], loss: 5.067880, mae: 34.111694, mean_q: -7.287225\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " 30000/80000: episode: 30, duration: 4.611s, episode steps: 1000, steps per second: 217, episode reward: 0.127, mean reward: 0.000 [-0.017, 0.038], mean action: 1.468 [0.000, 3.000], mean observation: 578.861 [-0.607, 5816.000], loss: 0.077640, mae: 0.878530, mean_q: -0.717650\n",
      " 31000/80000: episode: 31, duration: 4.659s, episode steps: 1000, steps per second: 215, episode reward: -199.942, mean reward: -0.200 [-100.001, 0.738], mean action: 2.587 [0.000, 3.000], mean observation: 1041.576 [0.000, 13880.000], loss: 10.032755, mae: 36.345683, mean_q: -1.750831\n",
      " 32000/80000: episode: 32, duration: 4.188s, episode steps: 1000, steps per second: 239, episode reward: -99.953, mean reward: -0.100 [-100.000, 1.267], mean action: 2.692 [0.000, 3.000], mean observation: 951.110 [-17.313, 12280.000], loss: 5.100996, mae: 19.775666, mean_q: -1.676124\n",
      " 33000/80000: episode: 33, duration: 4.071s, episode steps: 1000, steps per second: 246, episode reward: -99.906, mean reward: -0.100 [-100.000, 0.962], mean action: 1.775 [0.000, 3.000], mean observation: 794.172 [-0.217, 7288.000], loss: 5.143902, mae: 1.107175, mean_q: -0.569459\n",
      " 34000/80000: episode: 34, duration: 4.109s, episode steps: 1000, steps per second: 243, episode reward: -99.946, mean reward: -0.100 [-100.000, 0.059], mean action: 2.712 [0.000, 3.000], mean observation: 1117.105 [-0.145, 14568.000], loss: 5.030381, mae: 17.563490, mean_q: -0.418953\n",
      " 35000/80000: episode: 35, duration: 4.099s, episode steps: 1000, steps per second: 244, episode reward: -400.148, mean reward: -0.400 [-100.000, 0.704], mean action: 2.023 [0.000, 3.000], mean observation: 733.783 [-1397.720, 9160.000], loss: 20.310219, mae: 38.344929, mean_q: -3.680232\n",
      " 36000/80000: episode: 36, duration: 7.121s, episode steps: 1000, steps per second: 140, episode reward: -899.976, mean reward: -0.900 [-100.002, 1.059], mean action: 2.088 [0.000, 3.000], mean observation: 801.437 [-404.699, 7096.000], loss: 53.266104, mae: 107.642112, mean_q: -60.639623\n",
      " 37000/80000: episode: 37, duration: 7.522s, episode steps: 1000, steps per second: 133, episode reward: -2200.213, mean reward: -2.200 [-100.001, 0.783], mean action: 1.767 [0.000, 3.000], mean observation: 97.194 [-482.060, 1912.000], loss: 122.262799, mae: 42.928820, mean_q: -38.121422\n",
      " 38000/80000: episode: 38, duration: 7.639s, episode steps: 1000, steps per second: 131, episode reward: -900.352, mean reward: -0.900 [-100.016, 0.631], mean action: 1.215 [0.000, 3.000], mean observation: 216.267 [-1475.584, 3976.000], loss: 59.476928, mae: 126.089019, mean_q: -116.134850\n",
      " 39000/80000: episode: 39, duration: 7.835s, episode steps: 1000, steps per second: 128, episode reward: 0.065, mean reward: 0.000 [-0.788, 0.925], mean action: 2.522 [0.000, 3.000], mean observation: 1007.351 [0.000, 13528.000], loss: 0.372569, mae: 123.443547, mean_q: -28.017373\n",
      " 40000/80000: episode: 40, duration: 7.896s, episode steps: 1000, steps per second: 127, episode reward: -700.120, mean reward: -0.700 [-100.010, 0.059], mean action: 2.640 [1.000, 3.000], mean observation: 927.773 [-2143.225, 14120.000], loss: 36.770335, mae: 324.406979, mean_q: -81.770475\n",
      " 41000/80000: episode: 41, duration: 8.064s, episode steps: 1000, steps per second: 124, episode reward: -99.952, mean reward: -0.100 [-100.000, 0.052], mean action: 2.986 [0.000, 3.000], mean observation: 1469.711 [0.000, 16904.000], loss: 5.027932, mae: 223.221633, mean_q: 0.741529\n",
      " 42000/80000: episode: 42, duration: 8.150s, episode steps: 1000, steps per second: 123, episode reward: -99.952, mean reward: -0.100 [-100.000, 0.943], mean action: 2.988 [0.000, 3.000], mean observation: 1472.584 [0.000, 16920.000], loss: 5.060635, mae: 207.491477, mean_q: -4.727491\n",
      " 43000/80000: episode: 43, duration: 8.304s, episode steps: 1000, steps per second: 120, episode reward: -100.032, mean reward: -0.100 [-100.000, 0.614], mean action: 2.542 [0.000, 3.000], mean observation: 1270.252 [-676.995, 14536.000], loss: 5.888686, mae: 123.657180, mean_q: -14.747265\n",
      " 44000/80000: episode: 44, duration: 8.461s, episode steps: 1000, steps per second: 118, episode reward: -399.952, mean reward: -0.400 [-100.000, 0.535], mean action: 2.951 [0.000, 3.000], mean observation: 1435.879 [0.000, 16696.000], loss: 20.667230, mae: 65.002406, mean_q: -17.125092\n",
      " 45000/80000: episode: 45, duration: 8.597s, episode steps: 1000, steps per second: 116, episode reward: -199.889, mean reward: -0.200 [-100.000, 0.057], mean action: 1.487 [0.000, 3.000], mean observation: 501.343 [-1.541, 5432.000], loss: 10.067099, mae: 0.690186, mean_q: -0.589879\n",
      " 46000/80000: episode: 46, duration: 8.853s, episode steps: 1000, steps per second: 113, episode reward: 0.105, mean reward: 0.000 [-0.368, 0.440], mean action: 1.482 [0.000, 3.000], mean observation: 441.566 [-1.047, 4760.000], loss: 0.014937, mae: 0.609817, mean_q: -0.629077\n",
      " 47000/80000: episode: 47, duration: 8.822s, episode steps: 1000, steps per second: 113, episode reward: -199.885, mean reward: -0.200 [-100.000, 0.962], mean action: 1.497 [0.000, 3.000], mean observation: 475.884 [-0.917, 5176.000], loss: 10.031638, mae: 0.677624, mean_q: -0.706819\n",
      " 48000/80000: episode: 48, duration: 8.999s, episode steps: 1000, steps per second: 111, episode reward: -399.876, mean reward: -0.400 [-100.002, 0.950], mean action: 1.450 [0.000, 3.000], mean observation: 453.515 [-0.538, 4696.000], loss: 20.043586, mae: 0.754379, mean_q: -0.730525\n",
      " 49000/80000: episode: 49, duration: 9.412s, episode steps: 1000, steps per second: 106, episode reward: -299.891, mean reward: -0.300 [-100.000, 1.388], mean action: 1.455 [0.000, 3.000], mean observation: 452.806 [-0.962, 4616.000], loss: 15.022283, mae: 0.796071, mean_q: -0.833389\n",
      " 50000/80000: episode: 50, duration: 9.147s, episode steps: 1000, steps per second: 109, episode reward: -299.885, mean reward: -0.300 [-100.001, 0.057], mean action: 1.541 [0.000, 3.000], mean observation: 540.428 [-0.791, 5544.000], loss: 15.039570, mae: 0.859887, mean_q: -0.914670\n",
      " 51000/80000: episode: 51, duration: 9.257s, episode steps: 1000, steps per second: 108, episode reward: -199.951, mean reward: -0.200 [-99.996, 0.571], mean action: 2.924 [0.000, 3.000], mean observation: 1390.455 [0.000, 16424.000], loss: 10.017972, mae: 69.432987, mean_q: 0.330033\n",
      " 52000/80000: episode: 52, duration: 9.513s, episode steps: 1000, steps per second: 105, episode reward: -99.970, mean reward: -0.100 [-100.000, 0.054], mean action: 2.803 [0.000, 3.000], mean observation: 1419.789 [-22.063, 15944.000], loss: 5.202616, mae: 62.738732, mean_q: -1.070223\n",
      " 53000/80000: episode: 53, duration: 9.668s, episode steps: 1000, steps per second: 103, episode reward: -99.951, mean reward: -0.100 [-99.997, 0.271], mean action: 2.845 [0.000, 3.000], mean observation: 1316.194 [-0.813, 15976.000], loss: 5.042623, mae: 38.848761, mean_q: 4.777733\n",
      " 54000/80000: episode: 54, duration: 9.697s, episode steps: 1000, steps per second: 103, episode reward: -99.958, mean reward: -0.100 [-100.000, 1.207], mean action: 1.686 [0.000, 3.000], mean observation: 444.092 [-146.626, 7112.000], loss: 5.313911, mae: 13.447289, mean_q: -5.321116\n",
      " 55000/80000: episode: 55, duration: 9.825s, episode steps: 1000, steps per second: 102, episode reward: -1200.198, mean reward: -1.200 [-100.000, 0.047], mean action: 1.677 [0.000, 3.000], mean observation: 554.211 [-1315.848, 6216.000], loss: 85.008870, mae: 80.373439, mean_q: -36.328533\n",
      " 56000/80000: episode: 56, duration: 10.113s, episode steps: 1000, steps per second: 99, episode reward: -1800.132, mean reward: -1.800 [-100.028, 1.189], mean action: 1.043 [0.000, 3.000], mean observation: 103.118 [-241.493, 1400.000], loss: 91.123695, mae: 26.334177, mean_q: -16.462129\n",
      " 57000/80000: episode: 57, duration: 10.441s, episode steps: 1000, steps per second: 96, episode reward: -1300.024, mean reward: -1.300 [-100.004, 0.066], mean action: 0.917 [0.000, 3.000], mean observation: 110.729 [-113.189, 2024.000], loss: 69.795864, mae: 25.949245, mean_q: -27.363405\n",
      " 58000/80000: episode: 58, duration: 10.112s, episode steps: 1000, steps per second: 99, episode reward: -399.959, mean reward: -0.400 [-100.000, 0.065], mean action: 2.080 [0.000, 3.000], mean observation: 352.688 [-59.680, 7496.000], loss: 20.299455, mae: 60.048381, mean_q: -29.461342\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " 59000/80000: episode: 59, duration: 10.329s, episode steps: 1000, steps per second: 97, episode reward: 0.047, mean reward: 0.000 [-0.460, 0.543], mean action: 2.988 [0.000, 3.000], mean observation: 1457.854 [-0.159, 16840.000], loss: 0.024949, mae: 199.554831, mean_q: 2.563879\n",
      " 60000/80000: episode: 60, duration: 10.450s, episode steps: 1000, steps per second: 96, episode reward: -99.952, mean reward: -0.100 [-100.000, 0.714], mean action: 2.652 [0.000, 3.000], mean observation: 829.917 [-13.258, 11576.000], loss: 5.165361, mae: 111.167975, mean_q: -17.184062\n",
      " 61000/80000: episode: 61, duration: 10.621s, episode steps: 1000, steps per second: 94, episode reward: -99.952, mean reward: -0.100 [-100.000, 0.068], mean action: 2.993 [0.000, 3.000], mean observation: 1478.268 [0.000, 16952.000], loss: 5.056249, mae: 201.940718, mean_q: -6.883070\n",
      " 62000/80000: episode: 62, duration: 10.640s, episode steps: 1000, steps per second: 94, episode reward: -99.952, mean reward: -0.100 [-100.000, 0.916], mean action: 2.977 [0.000, 3.000], mean observation: 1459.622 [0.000, 16840.000], loss: 5.031829, mae: 59.007145, mean_q: -4.204816\n",
      " 63000/80000: episode: 63, duration: 10.771s, episode steps: 1000, steps per second: 93, episode reward: 0.048, mean reward: 0.000 [-0.027, 0.058], mean action: 2.985 [1.000, 3.000], mean observation: 1462.945 [0.000, 16856.000], loss: 0.074707, mae: 56.055127, mean_q: -2.710857\n",
      " 64000/80000: episode: 64, duration: 10.954s, episode steps: 1000, steps per second: 91, episode reward: -99.952, mean reward: -0.100 [-100.000, 0.954], mean action: 2.973 [0.000, 3.000], mean observation: 1448.693 [0.000, 16776.000], loss: 5.038747, mae: 35.191689, mean_q: -2.590885\n",
      " 65000/80000: episode: 65, duration: 11.218s, episode steps: 1000, steps per second: 89, episode reward: 0.057, mean reward: 0.000 [-0.564, 0.651], mean action: 2.243 [0.000, 3.000], mean observation: 748.477 [-1.271, 8792.000], loss: 0.059323, mae: 11.201503, mean_q: 0.197449\n",
      " 66000/80000: episode: 66, duration: 11.273s, episode steps: 1000, steps per second: 89, episode reward: -99.950, mean reward: -0.100 [-100.000, 1.271], mean action: 2.744 [0.000, 3.000], mean observation: 1160.458 [-0.729, 14856.000], loss: 5.044860, mae: 47.286041, mean_q: -3.992982\n",
      " 67000/80000: episode: 67, duration: 11.325s, episode steps: 1000, steps per second: 88, episode reward: -99.951, mean reward: -0.100 [-99.999, 0.639], mean action: 2.918 [0.000, 3.000], mean observation: 1407.344 [0.000, 16504.000], loss: 5.062954, mae: 55.352825, mean_q: -0.102303\n",
      " 68000/80000: episode: 68, duration: 11.356s, episode steps: 1000, steps per second: 88, episode reward: 0.048, mean reward: 0.000 [-0.028, 0.044], mean action: 2.974 [0.000, 3.000], mean observation: 1451.735 [0.000, 16792.000], loss: 0.022526, mae: 53.808183, mean_q: -0.170187\n",
      " 69000/80000: episode: 69, duration: 11.520s, episode steps: 1000, steps per second: 87, episode reward: -299.992, mean reward: -0.300 [-100.000, 0.389], mean action: 2.584 [0.000, 3.000], mean observation: 1197.054 [-194.336, 14696.000], loss: 15.253619, mae: 68.669184, mean_q: -4.175360\n",
      " 70000/80000: episode: 70, duration: 11.761s, episode steps: 1000, steps per second: 85, episode reward: -499.994, mean reward: -0.500 [-100.000, 0.487], mean action: 2.537 [0.000, 3.000], mean observation: 1210.022 [-227.323, 12440.000], loss: 26.999148, mae: 83.379527, mean_q: -13.954548\n",
      " 71000/80000: episode: 71, duration: 11.768s, episode steps: 1000, steps per second: 85, episode reward: -99.889, mean reward: -0.100 [-100.000, 0.248], mean action: 1.594 [0.000, 3.000], mean observation: 474.756 [-0.801, 4920.000], loss: 5.083488, mae: 3.112450, mean_q: -3.061077\n",
      " 72000/80000: episode: 72, duration: 12.001s, episode steps: 1000, steps per second: 83, episode reward: -99.898, mean reward: -0.100 [-100.000, 0.040], mean action: 1.569 [0.000, 3.000], mean observation: 476.820 [-1.352, 5208.000], loss: 5.016193, mae: 1.076286, mean_q: -1.201834\n",
      " 73000/80000: episode: 73, duration: 12.224s, episode steps: 1000, steps per second: 82, episode reward: -299.896, mean reward: -0.300 [-100.000, 1.161], mean action: 1.553 [0.000, 3.000], mean observation: 431.065 [-0.641, 4728.000], loss: 15.059831, mae: 1.179646, mean_q: -1.306725\n",
      " 74000/80000: episode: 74, duration: 12.159s, episode steps: 1000, steps per second: 82, episode reward: -199.894, mean reward: -0.200 [-100.001, 0.651], mean action: 1.612 [0.000, 3.000], mean observation: 513.656 [-1.180, 5496.000], loss: 10.012646, mae: 1.193858, mean_q: -1.339436\n",
      " 75000/80000: episode: 75, duration: 12.332s, episode steps: 1000, steps per second: 81, episode reward: -499.903, mean reward: -0.500 [-100.001, 0.044], mean action: 1.572 [0.000, 3.000], mean observation: 459.918 [-0.962, 5064.000], loss: 25.016136, mae: 1.339443, mean_q: -1.373723\n",
      " 76000/80000: episode: 76, duration: 12.452s, episode steps: 1000, steps per second: 80, episode reward: -99.904, mean reward: -0.100 [-100.000, 0.057], mean action: 1.587 [0.000, 3.000], mean observation: 479.436 [-0.618, 5176.000], loss: 5.030677, mae: 1.328326, mean_q: -1.520614\n",
      " 77000/80000: episode: 77, duration: 12.608s, episode steps: 1000, steps per second: 79, episode reward: 0.110, mean reward: 0.000 [-0.205, 0.291], mean action: 1.606 [0.000, 3.000], mean observation: 521.235 [-1.067, 5592.000], loss: 0.004904, mae: 1.257675, mean_q: -1.598356\n",
      " 78000/80000: episode: 78, duration: 12.678s, episode steps: 1000, steps per second: 79, episode reward: -99.890, mean reward: -0.100 [-99.989, 0.055], mean action: 1.612 [0.000, 3.000], mean observation: 522.019 [-0.797, 5544.000], loss: 5.027337, mae: 1.340443, mean_q: -1.603227\n",
      " 79000/80000: episode: 79, duration: 12.858s, episode steps: 1000, steps per second: 78, episode reward: -199.881, mean reward: -0.200 [-100.000, 0.848], mean action: 1.510 [0.000, 3.000], mean observation: 472.823 [-2.077, 4872.000], loss: 10.015758, mae: 1.350353, mean_q: -1.646644\n",
      " 80000/80000: episode: 80, duration: 12.903s, episode steps: 1000, steps per second: 78, episode reward: -299.880, mean reward: -0.300 [-100.000, 0.895], mean action: 1.520 [0.000, 3.000], mean observation: 536.794 [-0.713, 5704.000], loss: 15.007710, mae: 1.456829, mean_q: -1.738174\n",
      "done, took 599.130 seconds\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.callbacks.History at 0x7fb6ec6dd5d0>"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Okay, now it's time to learn something! We visualize the training here for show, but this\n",
    "# slows down training quite a lot. You can always safely abort the training prematurely using\n",
    "# Ctrl + C.\n",
    "sarsa.fit(heartpole, nb_steps=80000,\n",
    "          nb_max_episode_steps=1000, visualize=False, verbose=2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have an agent! Let us see how well it works."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Testing for 5 episodes ...\n",
      "Episode 1: reward: -299.951, steps: 1000\n",
      "Episode 2: reward: -199.951, steps: 1000\n",
      "Episode 3: reward: -199.950, steps: 1000\n",
      "Episode 4: reward: -99.950, steps: 1000\n",
      "Episode 5: reward: -99.950, steps: 1000\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.callbacks.History at 0x7fb76a766190>"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sarsa.test(heartpole, nb_episodes=5,\n",
    "           nb_max_episode_steps=1000,\n",
    "           visualize=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And an example"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "IOPub data rate exceeded.\n",
      "The notebook server will temporarily stop sending output\n",
      "to the client in order to avoid crashing it.\n",
      "To change this limit, set the config variable\n",
      "`--NotebookApp.iopub_data_rate_limit`.\n",
      "\n",
      "Current values:\n",
      "NotebookApp.iopub_data_rate_limit=1000000.0 (bytes/sec)\n",
      "NotebookApp.rate_limit_window=3.0 (secs)\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Chosen action: sleep\n",
      "{'alertness': 1.016310273755929, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 6929, 'work_done': 336.59534552171453}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7146819647621933, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 6946, 'work_done': 337.208879898509}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7763801844748786, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 6963, 'work_done': 337.87013009923703}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1860438875219204, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 6980, 'work_done': 338.82444110057446}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.2647538897991435, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 6997, 'work_done': 339.82942287602066}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8637719903662721, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7014, 'work_done': 340.556827318078}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7812411283515819, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7031, 'work_done': 341.22180230504716}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.2050444758388488, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7048, 'work_done': 342.1885335136106}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8787041768115926, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7065, 'work_done': 342.92706365552226}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8125861240510552, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7082, 'work_done': 343.61593191841433}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0313466905190944, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7099, 'work_done': 344.4649514268901}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.029980669998459, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7116, 'work_done': 345.3130097906448}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1388554573990899, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7133, 'work_done': 346.23597484722353}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9355875843344917, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7150, 'work_done': 347.01638637791956}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1647076309867888, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7167, 'work_done': 347.95661158628735}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1414932790554013, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7184, 'work_done': 348.88134734325774}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0098065992135867, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7201, 'work_done': 349.7151504663884}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.2713367870414756, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7218, 'work_done': 350.7242751532035}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0172145990603578, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7235, 'work_done': 351.5633259828579}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7421970493742436, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7252, 'work_done': 352.1982388603339}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1373197476439274, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7269, 'work_done': 353.1201720422975}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0278450881629466, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7286, 'work_done': 353.9667267424995}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.089283457091733, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7303, 'work_done': 354.8560201033685}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7720547514750373, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7320, 'work_done': 355.5139515342134}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.6860946020917412, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7337, 'work_done': 356.10511075751185}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.828392739759997, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7354, 'work_done': 356.8059440393351}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.6884704610269021, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7371, 'work_done': 357.3989690602627}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1890132544055663, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7388, 'work_done': 358.3552287939743}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8442687013773515, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7405, 'work_done': 359.068021754879}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.2534825033437846, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7422, 'work_done': 360.0658752801992}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.145853269944537, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7439, 'work_done': 360.9935330456524}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.001861328897193, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7456, 'work_done': 361.82169108852105}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7172077032109708, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7473, 'work_done': 362.4371944051048}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8373649627442618, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7490, 'work_done': 363.1447938242582}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9095788603143631, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7507, 'work_done': 363.9061560548682}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9482009855020899, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7524, 'work_done': 364.6957437722665}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7352273064879299, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7541, 'work_done': 365.3252562002646}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.2176812031283017, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7558, 'work_done': 366.30018225004017}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.754983395385694, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7575, 'work_done': 366.94497593554996}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1112427745995945, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7592, 'work_done': 367.8492769058207}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.826159824936324, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7609, 'work_done': 368.54842341751026}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9146819359938831, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7626, 'work_done': 369.3135366945049}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7477475643092184, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7643, 'work_done': 369.9527430478452}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1005101347502106, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7660, 'work_done': 370.8497270618979}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0561616460473042, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7677, 'work_done': 371.71611471426246}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0759499524501794, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7694, 'work_done': 372.59622569604966}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7722031014249515, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7711, 'work_done': 373.25427101847606}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1051550434793114, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7728, 'work_done': 374.1544259412458}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8448562197072405, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7745, 'work_done': 374.86766036698015}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9155068760381379, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7762, 'work_done': 375.6333794084843}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.278852298877717, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7779, 'work_done': 376.6472155651761}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8875620246410134, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7796, 'work_done': 377.392320077855}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.6965151819833146, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7813, 'work_done': 377.9916544108191}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.6821353285971531, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7830, 'work_done': 378.579701878524}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8708529223818838, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7847, 'work_done': 379.3123888704625}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.6973799159293516, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7864, 'work_done': 379.91240062828666}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7565201204085292, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7881, 'work_done': 380.55837950277305}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7135085144891805, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7898, 'work_done': 381.17099867328966}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8465461430098711, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7915, 'work_done': 381.88550246742324}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7514666356383766, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7932, 'work_done': 382.52758198426864}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9285381157168577, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7949, 'work_done': 383.3028472775101}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.784342457594548, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7966, 'work_done': 383.9701960125481}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1203619414563923, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 7983, 'work_done': 384.88068661515445}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1010602334429984, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8000, 'work_done': 385.7780464990164}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1989516606941957, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8017, 'work_done': 386.7408077975985}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9057657900171963, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8034, 'work_done': 387.499362962707}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9253773630530656, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8051, 'work_done': 388.27231674066513}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.711664967478951, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8068, 'work_done': 388.88349751153083}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.6893229706593705, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8085, 'work_done': 389.4771917483517}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0288231938754617, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8102, 'work_done': 390.3244352915511}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.6899187278299022, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8119, 'work_done': 390.9185971092289}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1745778989812155, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8136, 'work_done': 391.86535657631663}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.198463560375526, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8153, 'work_done': 392.82779931714487}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9836776336693966, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8170, 'work_done': 393.64297325749266}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.6991462098746456, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8187, 'work_done': 394.24436824889705}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7562443534394068, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8204, 'work_done': 394.89013447700177}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.819312062635316, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8221, 'work_done': 395.58410098695305}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7579296260348316, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8238, 'work_done': 396.23116649041265}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1118114522303952, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8255, 'work_done': 397.1358541873754}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0671566147489833, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8272, 'work_done': 398.00988095263716}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8908346973862038, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8289, 'work_done': 398.7574096328736}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9862758182847531, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8306, 'work_done': 399.5744442821404}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9772288346627865, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8323, 'work_done': 400.3849920485114}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9114924430076218, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8340, 'work_done': 401.1477616331284}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.6826758049084403, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8357, 'work_done': 401.73623406557925}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9140133197627004, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8374, 'work_done': 402.5008562434481}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9728637157426745, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8391, 'work_done': 403.3082663056803}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8122413693234947, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8408, 'work_done': 403.9968729688943}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.222967857337102, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8425, 'work_done': 404.97521170163475}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.142969196074714, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8442, 'work_done': 405.9009372607033}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.841090804326863, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8459, 'work_done': 406.6113409333972}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7947792398093603, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8476, 'work_done': 407.2866623920747}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8730671215950748, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8493, 'work_done': 408.020998764226}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.120134697906314, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8510, 'work_done': 408.93133543230795}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.771089079027219, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8527, 'work_done': 409.5885253781686}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1188804193136732, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8544, 'work_done': 410.49801211639215}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9923930403340588, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8561, 'work_done': 411.3194204787277}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.790625018371311, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8578, 'work_done': 411.99157138471804}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.2663906090957553, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8595, 'work_done': 412.99758462009333}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7247742907155835, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8612, 'work_done': 413.61897864724574}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8907980044786339, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8629, 'work_done': 414.36648016246676}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.2597686536000567, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8646, 'work_done': 415.3683145555656}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0321281734254992, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8663, 'work_done': 416.2178836870799}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1281289070801204, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8680, 'work_done': 417.13362614272927}\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7026415482694842, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8697, 'work_done': 417.7377565845135}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.2312697918067868, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8714, 'work_done': 418.72143564098855}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.147473005001727, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8731, 'work_done': 419.65017742335954}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0310533496839347, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8748, 'work_done': 420.49899057865724}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1245194954752065, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8765, 'work_done': 421.41229472309567}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1540187487924551, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8782, 'work_done': 422.34540892825186}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9651614142601461, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8799, 'work_done': 423.14727014595724}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0285481067050641, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8816, 'work_done': 423.9943199827063}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0333563256645222, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8833, 'work_done': 424.84475253645155}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9215660707920775, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8850, 'work_done': 425.6149156795406}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8047607711636845, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8867, 'work_done': 426.29783946679436}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0292051555001958, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8884, 'work_done': 427.14535193842005}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.771696638823766, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8901, 'work_done': 427.8030084184935}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9047784015528387, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8918, 'work_done': 428.5608361085409}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.177044890693104, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8935, 'work_done': 429.5092239059067}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8025870824607314, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8952, 'work_done': 430.1904940221263}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9816798406311931, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8969, 'work_done': 431.00423599601504}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7397582664698892, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 8986, 'work_done': 431.637260358937}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9924224710770309, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9003, 'work_done': 432.4586897393378}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.038313813473394, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9020, 'work_done': 433.31260321427214}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8812726310754646, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9037, 'work_done': 434.0530416474497}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9375439214680075, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9054, 'work_done': 434.83487908648266}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.757587346204688, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9071, 'work_done': 435.481680755438}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.6903872750501588, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9088, 'work_done': 436.07621026355025}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.2460443164905424, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9105, 'work_done': 437.06933592635636}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7151819001756443, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9122, 'work_done': 437.68326013128393}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0987642243180733, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9139, 'work_done': 438.5790506051447}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.2643792900189679, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9156, 'work_done': 439.5837961779525}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1005887586315044, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9173, 'work_done': 440.48083391944687}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9045066923862131, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9190, 'work_done': 441.23846138068774}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.071531980432275, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9207, 'work_done': 442.1155183316167}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.912653164659372, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9224, 'work_done': 442.8791411286723}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1471585107147957, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9241, 'work_done': 443.8076724979757}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1767484481992636, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9258, 'work_done': 444.75586473235364}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9143517272361835, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9275, 'work_done': 445.5207354847807}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8003398900325961, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9292, 'work_done': 446.20029489867835}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.903542541455143, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9309, 'work_done': 446.9572117067232}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.265710543791912, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9326, 'work_done': 447.96279647746553}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7211036013997357, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9343, 'work_done': 448.5813342822407}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.818815587335487, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9360, 'work_done': 449.2749248163689}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.037057328952906, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9377, 'work_done': 450.12795670126553}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0338452959492281, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9394, 'work_done': 450.9787328954631}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8900948658344097, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9411, 'work_done': 451.7257137893686}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8252821321732811, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9428, 'work_done': 452.4241969683}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1025029256481516, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9445, 'work_done': 453.32254218005954}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8969653216723072, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9462, 'work_done': 454.0746049300959}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0403027546034025, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9479, 'work_done': 454.9299129977813}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7350579371476673, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9496, 'work_done': 455.5592940649249}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9999265402251143, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9513, 'work_done': 456.3860748336249}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0543700374889025, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9530, 'work_done': 457.25121440640214}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8258656992003506, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9547, 'work_done': 457.9501386467341}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9809863201888784, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9564, 'work_done': 458.76338327298885}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1429942735638143, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9581, 'work_done': 459.68912564405895}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.2585149135479485, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9598, 'work_done': 460.69016715518234}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1029815894388955, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9615, 'work_done': 461.58883914622777}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9512055408033138, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9632, 'work_done': 462.380606597463}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7669981923723347, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9649, 'work_done': 463.0346531459712}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9212449150405491, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9666, 'work_done': 463.804580970432}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1172166419975897, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9683, 'work_done': 464.7129395538668}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.2019625906632612, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9700, 'work_done': 465.67766420957577}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0637056549422501, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9717, 'work_done': 466.54929707036376}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.6987851345313197, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9734, 'work_done': 467.1504093445414}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.099520005975643, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9751, 'work_done': 468.0467165981427}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9000656020331161, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9768, 'work_done': 468.80106872092387}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7848830505039427, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9785, 'work_done': 469.46883100811016}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.2185006631300273, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9802, 'work_done': 470.4442866502925}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8038473646822888, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9819, 'work_done': 471.1265156768846}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7244590678268593, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9836, 'work_done': 471.7476645329386}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8150523238316562, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9853, 'work_done': 472.4384033620303}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9994897716421793, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9870, 'work_done': 473.2648730769327}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.04118783224721, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9887, 'work_done': 474.1208013775433}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9620656177988777, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9904, 'work_done': 474.9204279276637}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8412000910190647, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9921, 'work_done': 475.6309138061932}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1351233367075946, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9938, 'work_done': 476.55136990737225}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8436121881666772, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9955, 'work_done': 477.2636694648359}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0851482167263378, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9972, 'work_done': 478.1501206160903}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7049024706106188, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 9989, 'work_done': 478.75601914512765}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9788762721167725, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10006, 'work_done': 479.56774979108707}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0338208284065082, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10023, 'work_done': 480.41850879148836}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8488078955224252, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10040, 'work_done': 481.1347104305107}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.2092784986291727, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10057, 'work_done': 482.10419324191696}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8874351491084502, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10074, 'work_done': 482.84920372133644}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.2473033476428859, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10091, 'work_done': 483.843130974234}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0415847058334826, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10108, 'work_done': 484.699337318547}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8001247518013097, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10125, 'work_done': 485.3787328966696}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.830913774142244, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10142, 'work_done': 486.08146921946917}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8317800116456444, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10159, 'work_done': 486.784859096731}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.6837751644487702, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10176, 'work_done': 487.37419575399775}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.783300307644003, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10193, 'work_done': 488.040747065949}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9896442293246455, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10210, 'work_done': 488.86019132304375}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0001394015008367, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10227, 'work_done': 489.6871236664668}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7008590349323365, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10244, 'work_done': 490.289859420838}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.2038600115692282, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10261, 'work_done': 491.25581981702487}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.2442094430339778, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10278, 'work_done': 492.24777630307267}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.856204353281844, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10295, 'work_done': 492.96952188893744}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9156972277649756, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10312, 'work_done': 493.73538068400126}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0193757485046495, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10329, 'work_done': 494.5759595762372}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.700881668667144, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10346, 'work_done': 495.17871304386944}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.093132516409512, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10363, 'work_done': 496.0706473660527}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7615248296470942, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10380, 'work_done': 496.7204825933896}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1563478581045734, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10397, 'work_done': 497.65514935531064}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9997313899117988, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10414, 'work_done': 498.4817911500961}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7185747330557236, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10431, 'work_done': 499.0983595910002}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9514293187318927, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10448, 'work_done': 499.89028929397267}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.92449775521519, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10465, 'work_done': 500.6625993479716}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.068831826986955, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10482, 'work_done': 501.5377869493098}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.124866503662282, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10499, 'work_done': 502.45132568586945}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9246284069827533, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10516, 'work_done': 503.2237313672334}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1264441678716177, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10533, 'work_done': 504.1383362064765}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8880797404727931, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10550, 'work_done': 504.8838243812821}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.2037837119294637, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10567, 'work_done': 505.8497351082744}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0461632310295836, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10584, 'work_done': 506.70914586547775}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.814828105077922, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10601, 'work_done': 507.3997146858096}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0234105352065839, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10618, 'work_done': 508.2431429420588}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.2725676190703934, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10635, 'work_done': 509.2530405812485}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9001403403632807, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10652, 'work_done': 510.0074478645341}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.264604129385755, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10669, 'work_done': 511.01233521485904}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.240454922737935, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10686, 'work_done': 512.001895773585}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8945529924349888, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10703, 'work_done': 512.7521755224128}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.28091359197038, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10720, 'work_done': 513.7673004635401}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9647784054059129, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10737, 'work_done': 514.5688853472035}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9758755081954342, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10754, 'work_done': 515.3784608675201}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8667451769916643, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10771, 'work_done': 516.1080848409263}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9169340077186899, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10788, 'work_done': 516.8748514407057}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8806822208221476, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10805, 'work_done': 517.6148513567617}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9149470968837831, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10822, 'work_done': 518.3801593635723}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8481818655837725, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10839, 'work_done': 519.0958911766173}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.833839013307579, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10856, 'work_done': 519.8008338237854}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0706990322259289, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10873, 'work_done': 520.6773143408096}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1682738492069702, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10890, 'work_done': 521.6199040126799}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.275219069783271, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10907, 'work_done': 522.6314649536669}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9655093919925568, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10924, 'work_done': 523.4335771975568}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9637827656845911, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10941, 'work_done': 524.2344435641263}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9001849715537746, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10958, 'work_done': 524.9888837867315}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8537650445671152, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10975, 'work_done': 525.7088024368213}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9564811843978804, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 10992, 'work_done': 526.5043915580045}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1588253245656042, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11009, 'work_done': 527.4407078967391}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.231811277243818, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11026, 'work_done': 528.4247344690522}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9810467442402406, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11043, 'work_done': 529.2380224326174}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9789668640779514, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11060, 'work_done': 530.0498181036562}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0856913426268398, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11077, 'work_done': 530.9366428414021}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.2018761453321782, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11094, 'work_done': 531.9013111694584}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.2776655437646622, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11111, 'work_done': 532.9144046573465}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1829608007236663, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11128, 'work_done': 533.8666892826079}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8158065272723394, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11145, 'work_done': 534.5579998855732}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7890143576237979, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11162, 'work_done': 535.2289204881199}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1133346010503573, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11179, 'work_done': 536.1346435138267}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.047212846409434, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11196, 'work_done': 536.9947880345469}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9696037668620316, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11213, 'work_done': 537.7998515009448}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.780849257250226, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11230, 'work_done': 538.4645264009554}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1772492247676203, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11247, 'work_done': 539.4130489810365}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9090686037912952, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11264, 'work_done': 540.1740357864804}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8141021353593615, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11281, 'work_done': 540.8640540786267}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7443281195195632, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11298, 'work_done': 541.5006161649842}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.2205976550755568, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11315, 'work_done': 542.4774260149014}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1769045547531947, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11332, 'work_done': 543.425721236215}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9808821991073857, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11349, 'work_done': 544.2388911825022}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.978624174237797, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11366, 'work_done': 545.0504408661586}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0652638148090512, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11383, 'work_done': 545.923155037965}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9170002394588626, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11400, 'work_done': 546.689970241436}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7200616444164216, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11417, 'work_done': 547.307696777966}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9816727829148573, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11434, 'work_done': 548.1214336911696}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.2510413540567704, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11451, 'work_done': 549.1177376511531}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7489187989329915, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11468, 'work_done': 549.7578491560776}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1340101352540015, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11485, 'work_done': 550.677556063191}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.803108999790441, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11502, 'work_done': 551.3592233335819}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9931149122223522, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11519, 'work_done': 552.1811471555276}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9043812630141379, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11536, 'work_done': 552.9386821787594}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1323691842433934, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11553, 'work_done': 553.8572840150058}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0376102009541814, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11570, 'work_done': 554.710703867661}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7720088301610114, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11587, 'work_done': 555.3686000427166}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.124472285524896, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11604, 'work_done': 556.2818722683979}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8110692953986414, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11621, 'work_done': 556.9695893616671}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1294548697687137, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11638, 'work_done': 557.8862265583193}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8435870935069673, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11655, 'work_done': 558.5985072538576}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9072826578733688, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11672, 'work_done': 559.3581795244577}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0555359275305256, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11689, 'work_done': 560.2241313903121}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0861357813360062, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11706, 'work_done': 561.1112617682459}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1605207423269366, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11723, 'work_done': 562.0487058556926}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.954426471192235, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11740, 'work_done': 562.8428074080629}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9640609857739397, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11757, 'work_done': 563.6438745822712}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8802996858514002, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11774, 'work_done': 564.3835903318837}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.2263573943054247, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11791, 'work_done': 565.3641122095212}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8402875267897599, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11808, 'work_done': 566.0739115686514}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.261242215424889, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11825, 'work_done': 567.076677171635}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.2272882139141748, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11842, 'work_done': 568.0577979035357}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.2553645070283925, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11859, 'work_done': 569.0568446760027}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9740082611138139, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11876, 'work_done': 569.8650779426907}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9091303585954031, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11893, 'work_done': 570.6261101881634}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9164656559616865, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11910, 'work_done': 571.3925330598611}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.2217479509169318, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11927, 'work_done': 572.3700851337777}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7136143525732471, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11944, 'work_done': 572.9827868618544}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9075278833240153, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11961, 'work_done': 573.7426396762679}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0825183787163948, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11978, 'work_done': 574.6272806756905}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0416437809891608, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 11995, 'work_done': 575.4835284033567}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1007645942593018, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12012, 'work_done': 576.3806862950114}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.2742299645086737, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12029, 'work_done': 577.3916270385463}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7927806241647123, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12046, 'work_done': 578.0654236066214}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1690448527585466, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12063, 'work_done': 579.0085239359429}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1628304565778824, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12080, 'work_done': 579.9475029258706}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7465600753475929, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12097, 'work_done': 580.5857912700126}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.73194713651471, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12114, 'work_done': 581.2127585744207}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8470333383325571, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12131, 'work_done': 581.927628196089}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9460525205546167, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12148, 'work_done': 582.7156558171025}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1938624934470279, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12165, 'work_done': 583.6750918648205}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9312045039984292, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12182, 'work_done': 584.4523051600366}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8869286637515506, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12199, 'work_done': 585.196940220837}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0221858136617714, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12216, 'work_done': 586.039504058273}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.040295216193446, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12233, 'work_done': 586.8948068423388}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7359472114869472, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12250, 'work_done': 587.5248775533657}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.6965975330952483, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12267, 'work_done': 588.124276405902}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9680229093151607, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12284, 'work_done': 588.9282009163658}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.2657847533630435, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12301, 'work_done': 589.9338324494631}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.950817658096778, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12318, 'work_done': 590.7253186328045}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7908159921694479, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12335, 'work_done': 591.3976153761391}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.194490182921921, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12352, 'work_done': 592.357462010597}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1213165619528063, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12369, 'work_done': 593.2685991001293}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8693482068398097, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12386, 'work_done': 594.0001645416602}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0327236405704465, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12403, 'work_done': 594.8501523546737}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1462707774288825, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12420, 'work_done': 595.7780896176596}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7254479039151827, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12437, 'work_done': 596.4000074926754}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.2748711446297192, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12454, 'work_done': 597.4113503133766}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1916432488450028, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12471, 'work_done': 598.3693336738}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0216600604299837, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12488, 'work_done': 599.2115263020265}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.2613500317603326, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12505, 'work_done': 600.2143600096844}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1339742903907846, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12522, 'work_done': 601.1340427865457}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7297133210716069, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12539, 'work_done': 601.7592755662774}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1904247166985802, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12556, 'work_done': 602.716460612561}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.271347623119475, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12573, 'work_done': 603.7255921066369}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.807385296502073, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12590, 'work_done': 604.4105111335437}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1023369130612863, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12607, 'work_done': 605.3087429940188}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9629134667293655, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12624, 'work_done': 606.1089818056238}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.128659991645849, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12641, 'work_done': 607.0250826944775}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0991820038548608, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12658, 'work_done': 607.9211588539594}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7087304749063614, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12675, 'work_done': 608.5300486065809}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9728867890049101, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12692, 'work_done': 609.337475267497}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.2521831240298493, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12709, 'work_done': 610.3345042406459}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.2451280255716646, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12726, 'work_done': 611.327046188006}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.006494164145424, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12743, 'work_done': 612.1584979441208}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0239076761784645, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12760, 'work_done': 613.0022769674007}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.111484175776485, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12777, 'work_done': 613.9067421134084}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8469365820819685, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12794, 'work_done': 614.6215390867106}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9143362646964872, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12811, 'work_done': 615.3863984818764}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0183352370889227, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12828, 'work_done': 616.226241831708}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0502365791907007, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12845, 'work_done': 617.0884984226825}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9297408370516326, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12862, 'work_done': 617.8646426206508}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.162849542541379, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12879, 'work_done': 618.8036342869797}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8973289710845025, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12896, 'work_done': 619.5559656932927}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1039351255392695, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12913, 'work_done': 620.4552884496455}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8905534540396537, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12930, 'work_done': 621.2026089073814}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7393400861362136, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12947, 'work_done': 621.8353093197248}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1368146750491939, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12964, 'work_done': 622.756902973669}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.135439407565126, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12981, 'work_done': 623.6775717235099}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1617282118638397, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 12998, 'work_done': 624.6158184356825}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0684949422582521, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13015, 'work_done': 625.4907726587066}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8284721782520084, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13032, 'work_done': 626.191665928198}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9802206189283738, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13049, 'work_done': 627.0043612938978}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8155359419474373, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13066, 'work_done': 627.6954667765808}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.873482971438936, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13083, 'work_done': 628.4301127881138}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.817960225592832, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13100, 'work_done': 629.1230554336535}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9684919988657004, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13117, 'work_done': 629.9273179767104}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7404316515169015, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13134, 'work_done': 630.5608639119079}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0745565754491733, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13151, 'work_done': 631.4400123035047}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.2484259079568933, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13168, 'work_done': 632.4346538048417}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9872892497333494, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13185, 'work_done': 633.2524137383185}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.20558881553077, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13202, 'work_done': 634.2194990318054}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8444834386980327, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13219, 'work_done': 634.9324533569901}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9531079752153593, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13236, 'work_done': 635.7255997659026}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0081986593107304, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13253, 'work_done': 636.5582618523963}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9711169231159421, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13270, 'work_done': 637.3644148783995}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7861736882891854, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13287, 'work_done': 638.0331642402929}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9623716233615511, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13304, 'work_done': 638.8330117895739}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0779999456957174, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13321, 'work_done': 639.7145379402418}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1478766622670378, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13338, 'work_done': 640.6435497451176}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.6874910374259557, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13355, 'work_done': 641.2358057454386}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7221175635963931, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13372, 'work_done': 641.8551328076179}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8980410888423417, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13389, 'work_done': 642.6079902165735}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.2723226977618414, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13406, 'work_done': 643.6177340890953}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9749415453148061, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13423, 'work_done': 644.4266383546553}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9407987869821589, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13440, 'work_done': 645.2108459783559}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1516025344164424, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13457, 'work_done': 646.142347763863}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0095276392500594, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13474, 'work_done': 646.9759529806086}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.2223267520611996, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13491, 'work_done': 647.953878357249}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.878256342232332, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13508, 'work_done': 648.6920756067299}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1551168755269594, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13525, 'work_done': 649.6259220225955}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8981844539053265, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13542, 'work_done': 650.3788853123925}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.6966589215751949, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13559, 'work_done': 650.9783322600389}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9209897063508228, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13576, 'work_done': 651.7480730683135}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7658887040010204, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13593, 'work_done': 652.4012664739103}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7927226308775593, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13610, 'work_done': 653.0750187814247}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.225411888105877, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13627, 'work_done': 654.0549320596493}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7638603832523665, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13644, 'work_done': 654.706565099322}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.224814350984142, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13661, 'work_done': 655.6860936034785}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1866946069796651, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13678, 'work_done': 656.6408319019437}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8034267069079634, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13695, 'work_done': 657.3227409024084}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9919273252700287, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13712, 'work_done': 658.1438166415469}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.2437771053039934, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13729, 'work_done': 659.1354974768108}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9159971117524166, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13746, 'work_done': 659.9015764240168}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0366870674230417, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13763, 'work_done': 660.7543484364437}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.19932580451651, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13780, 'work_done': 661.7173538666385}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9650769575284963, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13797, 'work_done': 662.5191541536449}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1903361392809055, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13814, 'work_done': 663.4762811501922}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8617479075716818, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13831, 'work_done': 664.2021733759522}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1068595883450862, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13848, 'work_done': 665.1034903039242}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1932427552368867, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13865, 'work_done': 666.0625208403945}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0213804336173915, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13882, 'work_done': 666.9045160061678}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0393574435459816, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13899, 'work_done': 667.7591613863763}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1794675379604258, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13916, 'work_done': 668.7091463474825}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7678233228044368, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13933, 'work_done': 669.3638272092619}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.038891147020023, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13950, 'work_done': 670.2181456104288}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0343528292302038, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13967, 'work_done': 671.0692784196759}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.2819268229963157, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 13984, 'work_done': 672.0850363200417}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8420776646229681, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14001, 'work_done': 672.796182211618}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.2417048707593774, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14018, 'work_done': 673.7865409479826}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7681411684138756, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14035, 'work_done': 674.4414661121543}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1611353077298794, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14052, 'work_done': 675.3793187688162}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8268204808300119, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14069, 'work_done': 676.0789644670103}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1590408971572166, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14086, 'work_done': 677.0154242494561}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8172449876895496, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14103, 'work_done': 677.7078250145859}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1659169635200242, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14120, 'work_done': 678.6488524832782}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.2714131439856131, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14137, 'work_done': 679.6580251369162}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.2254919840628753, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14154, 'work_done': 680.6379899825391}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8450300650619302, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14171, 'work_done': 681.3513550212507}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1999865829413425, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14188, 'work_done': 682.3147915026046}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9828603147813995, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14205, 'work_done': 683.1293797392345}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1089020164021046, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14222, 'work_done': 684.0320878587829}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.6917811056320576, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14239, 'work_done': 684.6277109115354}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9487352509326714, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14256, 'work_done': 685.4176863971916}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7180690541530688, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14273, 'work_done': 686.0338608818433}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0849193489237312, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14290, 'work_done': 686.9201545814237}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.6842927431208774, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14307, 'work_done': 687.5098980333643}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9695217241056966, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14324, 'work_done': 688.3149024069212}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.6875463978306645, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14341, 'work_done': 688.907201880011}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1715298398743308, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14358, 'work_done': 689.8519467990872}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8978980563927229, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14375, 'work_done': 690.6046985677951}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7731755925424536, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14392, 'work_done': 691.2634903752177}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.042116912805301, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14409, 'work_done': 692.1200695056837}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0863690096319605, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14426, 'work_done': 693.0073602512089}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0407291035945594, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14443, 'work_done': 693.8629671177656}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1836956988502394, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14460, 'work_done': 694.8157350370358}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0605280556646073, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14477, 'work_done': 695.6851605734446}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8062343126377514, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14494, 'work_done': 696.3692047798671}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9070526232810929, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14511, 'work_done': 697.1287076770964}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.214666922107812, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14528, 'work_done': 698.1016837724059}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1752170354512428, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14545, 'work_done': 699.0488652864173}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0167480045512711, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14562, 'work_done': 699.8875860357387}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0442249435313307, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14579, 'work_done': 700.7456409536192}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0519912634069095, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14596, 'work_done': 701.609121989678}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8746893862699429, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14613, 'work_done': 702.3446660550304}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8469738174262496, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14630, 'work_done': 703.0594909863408}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1587102502939282, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14647, 'work_done': 703.9957307477451}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.227164776143563, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14664, 'work_done': 704.9767720810914}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.6829690494232461, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14681, 'work_done': 705.5654750612312}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8132826155617128, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14698, 'work_done': 706.2548717394192}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.830498970659105, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14715, 'work_done': 706.9572950421361}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0109712313591854, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14732, 'work_done': 707.7919241748177}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8523197192967957, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14749, 'work_done': 708.5107596766206}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1268115930494056, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14766, 'work_done': 709.4256126930843}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8287967687993676, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14783, 'work_done': 710.1267510607837}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9907624205786332, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14800, 'work_done': 710.9469945443669}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0881156636283822, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14817, 'work_done': 711.8354857766556}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7497784816843969, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14834, 'work_done': 712.4762614754708}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.6862550819856832, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14851, 'work_done': 713.0675467612223}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.132204080518796, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14868, 'work_done': 713.9860373652036}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0262777282023732, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14885, 'work_done': 714.8314876746184}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1382070296772808, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14902, 'work_done': 715.7540171284745}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9246526156912855, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14919, 'work_done': 716.5264405283356}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0882245574002591, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14936, 'work_done': 717.4150065741444}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8009646412948048, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14953, 'work_done': 718.0950417007853}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0091726709163804, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14970, 'work_done': 718.9283950565372}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0602427085723913, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 14987, 'work_done': 719.7976222334455}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.069149058499757, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15004, 'work_done': 720.673029568082}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7407420181570894, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15021, 'work_done': 721.306815866623}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.2670129072942053, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15038, 'work_done': 722.313221031397}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8811166082188062, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15055, 'work_done': 723.0535435894864}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0423980863576177, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15072, 'work_done': 723.9103196364738}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0560149311251847, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15089, 'work_done': 724.7766051181897}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7573756427985933, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15106, 'work_done': 725.4232435904132}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.778806968448669, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15123, 'work_done': 726.0863540033155}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.2232520197286298, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15140, 'work_done': 727.0648759075065}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9510384798169789, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15157, 'work_done': 727.8565222215253}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.87092808158969, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15174, 'work_done': 728.589265219663}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.087676901134788, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15191, 'work_done': 729.4774549724672}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7052458374754251, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15208, 'work_done': 730.0836219311927}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.2305314852235612, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15225, 'work_done': 731.0668269969948}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.718672647838015, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15242, 'work_done': 731.6834717137332}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8874332052806995, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15259, 'work_done': 732.4284807524624}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.02819592244138, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15276, 'work_done': 733.275282562837}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0109573292958396, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15293, 'work_done': 734.1099018377519}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.881163370807001, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15310, 'work_done': 734.8502591261099}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9742656814261409, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15327, 'work_done': 735.6586774921863}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.123445511305517, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15344, 'work_done': 736.5712553448626}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1425926156826918, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15361, 'work_done': 737.4967284201324}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9404589706214551, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15378, 'work_done': 738.2806887177967}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9487402861335429, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15395, 'work_done': 739.0706678576345}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9618764197979908, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15412, 'work_done': 739.8701577554436}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1055532527503604, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15429, 'work_done': 740.7705842196611}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.966932281801924, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15446, 'work_done': 741.5737225836427}\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Chosen action: sleep\n",
      "{'alertness': 0.8394872355361667, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15463, 'work_done': 742.2829197264598}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8712519479510732, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15480, 'work_done': 743.0159040437673}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.2742459595131856, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15497, 'work_done': 744.026854819336}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0142887925311792, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15514, 'work_done': 744.86383486629}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.018495065022833, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15531, 'work_done': 745.7037912187682}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0265320469103905, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15548, 'work_done': 746.5494207725329}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7717685052556666, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15565, 'work_done': 747.2071324322527}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.209878227998537, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15582, 'work_done': 748.1770045184155}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8046633713451842, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15599, 'work_done': 748.859854229755}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7431380165394957, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15616, 'work_done': 749.4954954278666}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1493440428390278, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15633, 'work_done': 750.425488393589}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9794980701844671, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15650, 'work_done': 751.2376653095087}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.743509570535834, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15667, 'work_done': 751.8735940439037}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.2390351897801757, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15684, 'work_done': 752.8622473650146}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0888508129344796, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15701, 'work_done': 753.7512436003547}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7570415431846662, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15718, 'work_done': 754.3976245042801}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.2467948757564722, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15735, 'work_done': 755.3912280921259}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8495418603969762, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15752, 'work_done': 756.1079804434631}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8697337257817728, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15769, 'work_done': 756.8398332860311}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0959432927301973, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15786, 'work_done': 757.73369338896}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0126940120643761, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15803, 'work_done': 758.5695437100242}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1107080707707944, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15820, 'work_done': 759.4734809681851}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8594868570087265, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15837, 'work_done': 760.1976827837102}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9739562353031094, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15854, 'work_done': 761.0058786388332}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1385355774674737, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15871, 'work_done': 761.9286288217831}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.195903479672175, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15888, 'work_done': 762.8893994584677}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.146812050309089, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15905, 'work_done': 763.8176989919767}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7880173792686281, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15922, 'work_done': 764.4878577654728}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1783383427794756, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15939, 'work_done': 765.4370985231917}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8661697221332973, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15956, 'work_done': 766.1662930751224}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7363070696869136, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15973, 'work_done': 766.7966428139958}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9547607078045446, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 15990, 'work_done': 767.5909864216615}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1159042027457273, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 16007, 'work_done': 768.498454488065}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1109314640302625, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 16024, 'work_done': 769.4025437115699}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.2712467311179727, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 16041, 'work_done': 770.411611823374}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8723682054105276, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 16058, 'work_done': 771.1454276928434}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.823706609811359, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 16075, 'work_done': 771.8427196983661}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8073372491229288, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 16092, 'work_done': 772.5276022121872}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8519345307748065, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 16109, 'work_done': 773.2461489648665}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7002194485557468, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 16126, 'work_done': 773.848384133561}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9202864285300566, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 16143, 'work_done': 774.6176094968112}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9662828908700366, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 16160, 'work_done': 775.4202796173856}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8277944840014542, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 16177, 'work_done': 776.120661082096}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7330014982826624, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 16194, 'work_done': 776.7484467227729}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8940218062976046, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 16211, 'work_done': 777.4983336688349}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9153797262220709, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 16228, 'work_done': 778.2639593536643}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.831949672623709, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 16245, 'work_done': 778.967477215605}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9305196118874925, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 16262, 'work_done': 779.7441903172537}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8776716857849863, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 16279, 'work_done': 780.4819528964024}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.2736557317653678, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 16296, 'work_done': 781.492533423443}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8541545041814783, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 16313, 'work_done': 782.2127438560805}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.044202835133661, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 16330, 'work_done': 783.0707833028918}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1746576480267057, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 16347, 'work_done': 784.0175954385384}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9904061727274196, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 16364, 'work_done': 784.8375843308525}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0707577904077015, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 16381, 'work_done': 785.7141055175456}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.2415208470530947, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 16398, 'work_done': 786.7043467753156}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0505589301415208, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 16415, 'work_done': 787.5668283736637}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9297379771672813, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 16432, 'work_done': 788.34297048217}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8209059261657786, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 16449, 'work_done': 789.0381436295463}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1264244558216994, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 16466, 'work_done': 789.9527351531224}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8499712424788641, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 16483, 'work_done': 790.6698096219574}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8436480592950667, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 16500, 'work_done': 791.3821361410194}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7578779807374446, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 16517, 'work_done': 792.0291618371004}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8029531525726301, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 16534, 'work_done': 792.7107105215872}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8575979842167625, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 16551, 'work_done': 793.4334992445469}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8590550711669053, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 16568, 'work_done': 794.1573781095641}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7636275927124819, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 16585, 'work_done': 794.8088320092978}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9519702182274946, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 16602, 'work_done': 795.6011538413933}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1025439129416352, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 16619, 'work_done': 796.499527037484}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8403552627367065, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 16636, 'work_done': 797.2093773608312}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.1645969931173934, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 16653, 'work_done': 798.1495291498856}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9591753321385971, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 16670, 'work_done': 798.9470671042734}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8538986426333456, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 16687, 'work_done': 799.6670858498644}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8424878566441558, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 16704, 'work_done': 800.378540180576}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8844882081533402, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 16721, 'work_done': 801.1213654424333}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.7810715960506831, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 16738, 'work_done': 801.7862106091231}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.0562823671150146, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 16755, 'work_done': 802.6526823256647}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.122459828541292, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 16772, 'work_done': 803.5645932910676}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.8554136287909352, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 16789, 'work_done': 804.2857468128299}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9754497273257554, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 16806, 'work_done': 805.0950163461138}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 0.9428693956128831, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 16823, 'work_done': 805.880730362802}\n",
      "\n",
      "Chosen action: sleep\n",
      "{'alertness': 1.27049255500817, 'hypertension': 0.0, 'intoxication': 0.0, 'time_since_slept': 1, 'time_elapsed': 16840, 'work_done': 806.8893245750555}\n",
      "\n",
      "Episode 1: reward: 0.048, steps: 1000\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.callbacks.History at 0x7fb6ec6dd8d0>"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sarsa.test(heartpole, nb_episodes=1,\n",
    "           nb_max_episode_steps=1000,\n",
    "           visualize=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that our agent has learned:\n",
    "- That you have to sleep once in a while to keep alertness levels from going deeply negative\n",
    "- That `do_nothing` is a good strategy most of the time (no need to drink something every half an hour)\n",
    "- That drinking too much coffee increases the risk of a heart attack and should be avoided \n",
    "\n",
    "The last part is _very not-trivial_ (try reducing the number of trainign iterations from 300000 to, say 30000 and your agent will probably be overdosing on coffee heaving learnt that it increases productivity, but not yet learnt that it's dangerous). _We_ know that the causal link between coffee and heart attacks exists because we programmed it in. But this link is rather hard to observe from he outside: it doesn't manifest immediately upon drinking coffee (in fact, it can take a lot of time to manifest) _and_ it is has a large component of randomness! \n",
    "\n",
    "Yet we see that: \n",
    "- SARSA is sophisticated enough to find such dependencies\n",
    "- It is fairly easy to use out of the box with libraries like [keras-rl](keras-rl.readthedocs.io)\n",
    "\n",
    "And the final score is\n",
    "\n",
    "| AI | Heart attacks |\n",
    "| -- | ------------- |\n",
    "| 1  | 0 |"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Homework\n",
    "\n",
    "Make *HeartPole-v1*! Add ~~cannabis~~ some new actions that can modify your health and productivity! "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}