{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Deep Learning Models\n", "\n", "This is not a deep learning course, but by request, I will introduce simple examples to get started in deep learning and highlight some aspects that I find interesting. In particular, strategies for optimization and specialized architectures are not covered.\n", "\n", "**Note**: This is a version for those of you on a Mac with AMD GPU - we can use the PlaidML backend to take advantage of the GPU. Note that this works with the standalone `keras`, not the tensorflow version." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "First install\n", "\n", "```\n", "python3 -m pip install -U --quiet plaidml-keras plaidbench\n", "```\n", "\n", "Then run and choose the GPU as default device\n", "```\n", "plaidml-setup\n", "```" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import os\n", "os.environ['KERAS_BACKEND'] = 'plaidml.keras.backend'\n", "os.environ['RUNFILES_DIR'] = '/usr/local/share/plaidml'\n", "os.environ['PLAIDML_NATIVE_PATH'] = '/usr/local/lib/libplaidml.dylib'" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "%matplotlib inline" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "import warnings\n", "warnings.simplefilter('ignore', RuntimeWarning)" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "import matplotlib.pyplot as plt\n", "import pandas as pd\n", "import numpy as np\n", "import seaborn as sns" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "Using plaidml.keras.backend backend.\n" ] } ], "source": [ "import tensorflow as tf\n", "import keras" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "from keras.utils import plot_model" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [], "source": [ "import tensorflow_datasets as tfds" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Building models" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Prepare data" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "import seaborn as sns" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [], "source": [ "iris = sns.load_dataset('iris')\n", "iris.species = iris.species.astype('category').cat.codes\n", "iris = iris.sample(frac=1)" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [], "source": [ "X_train, y_train = iris.iloc[:, :-1].values, iris.iloc[:, -1].values" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [], "source": [ "X_train = X_train.astype('float32')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Pre-process data\n", "\n", "If you need to pre-process your data, see https://keras.io/api/preprocessing/. We will not do any pre-processing for simplicity." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Sequential API\n", "\n", "If the entire pipeline is a single chain of layers, the Sequential API is the simplest to use." ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [], "source": [ "from keras.models import Sequential, Model\n", "from keras.layers import Dense, Input, concatenate" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Set random seed for reproducibility" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [], "source": [ "np.random.seed(0)\n", "tf.random.set_seed(0)" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(4,)" ] }, "execution_count": 14, "metadata": {}, "output_type": "execute_result" } ], "source": [ "X_train[0].shape" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "INFO:plaidml:Opening device \"metal_amd_radeon_pro_560x.0\"\n" ] } ], "source": [ "model = Sequential()\n", "model.add(Dense(8, input_shape=X_train[0].shape))\n", "model.add(Dense(4, activation='relu'))\n", "model.add(Dense(3, activation='softmax'))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Alternative model specification**\n", "\n", "```python\n", "model = Sequential(\n", " Dense(8, input_shape=(4,)),\n", " Dense(4, activation='relu'),\n", " Dense(3, activation='softmax')\n", ")\n", "```" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [], "source": [ "model.compile(\n", " optimizer='adam',\n", " loss='sparse_categorical_crossentropy',\n", " metrics=['accuracy']\n", ")" ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "_________________________________________________________________\n", "Layer (type) Output Shape Param # \n", "=================================================================\n", "dense_1 (Dense) (None, 8) 40 \n", "_________________________________________________________________\n", "dense_2 (Dense) (None, 4) 36 \n", "_________________________________________________________________\n", "dense_3 (Dense) (None, 3) 15 \n", "=================================================================\n", "Total params: 91\n", "Trainable params: 91\n", "Non-trainable params: 0\n", "_________________________________________________________________\n" ] } ], "source": [ "model.summary()" ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [], "source": [ "hist = model.fit(\n", " X_train,\n", " y_train,\n", " validation_split = 0.3,\n", " epochs=50, \n", " verbose=0)" ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAsIAAAD4CAYAAADmbIA7AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAABcfklEQVR4nO3dd1xV9f/A8deHrYIgCKLgwIkDV7i3peLImTnaQxtWpn0r2/tX3+pbaZkjM8tKMzMzc5S5zdxb3BMcIAoOQBmf3x/noowLXODChXvfz8fjPi733HPOfR+j49vPfX/eH6W1RgghhBBCCEfjZOsAhBBCCCGEsAVJhIUQQgghhEOSRFgIIYQQQjgkSYSFEEIIIYRDkkRYCCGEEEI4JBdbfXDlypV1rVq1bPXxQghRaNu2bbugtfa3dRwlSe7ZQoiyLLf7ts0S4Vq1arF161ZbfbwQQhSaUuqkrWMoaXLPFkKUZbndt6U0QgghhBBCOCRJhIUQQgghhEOSRFgIIYQQQjgkm9UICyFsJyUlhaioKJKTk20dSqnm4eFBcHAwrq6utg5FCCFEMZBEWAgHFBUVhZeXF7Vq1UIpZetwSiWtNXFxcURFRRESEmLrcIQQQhQDKY0QwgElJyfj5+cnSXAelFL4+fnJqLkQQtgxSYSFcFCSBOdP/oyEEMK+lanSiFkbjlPTrwJdG/jLX1BCCCGEcDhXklP4Y/dZQqtWpHl1H1uHk8P2U5dYfTAWtM7xXosalQqdwyWnpDF38ylq+3vSub711jMqM4lwalo6czaf5uD5K7Sr7cdLfUJpGuxj67CEEIXk6enJ1atXbR2GEEKUCVeSU5i14QQz1h8nISkFgG4N/Bl7R/1SkRBvO3mJiX8fZu2hWACy57oZeXHTYG+evaMe3RoEWJQQZyTAX64+SsyV69zbtoZjJsIuzk78/nRH5mw+xcS/D9P/iw30b1aN53s1oLpveVuHJ4QQwtGkpcKaD6D5PeArEypF8cieAN/RMIDHutRhy4mLfLX2GAMnb7BpQpw5Afat4MZLvUO5r11NyrtlTTFT0tL5dXs0n686zMOztuabEGdPgFuH+DJxeAva1fGzavxKmxm6zrGTUhHARMAZmKG1/iDb+zWAbwEf0z4TtNZL8jpneHi4LuxynVeSU5i25hgz1h8jLV1zX9tajL2jHt7lpMWREJaIjIykYcOGNo0hY0RYa80LL7zA0qVLUUrx6quvMmzYMM6ePcuwYcO4fPkyqampTJkyhfbt2/PII4+wdetWlFI8/PDDjBs3rljjNPdnpZTaprUOL9YPLmWKcs+2W2d2wPSu4FsHHl0B5X0B2HcmgeV7zzGuR30p4xNZHL9wja/XHyPm8nWL9tfA5uMXbybAY2+vT1iw9833r15P5buNJ/hq7TEuJabQqlYlKpV3K6boc7p47QZbT17Ct4Ibj3WubTYBzi5zQnz6YhINq1akeqVyOfbbeTr+ZgI87o76RU6Ac7tv5zsirJRyBiYDPYAoYItSapHWen+m3V4F5mmtpyilGgFLgFpFijgPXh6u/KdXA+5tW5PPVhxi1j/HWbz7DG/1b0xEk0C58QhRAG/9vo/9Zy5b9ZyNqlXkjTsbW7TvggUL2LlzJ7t27eLChQu0atWKzp078+OPP9KrVy9eeeUV0tLSSExMZOfOnURHR7N3714A4uPjrRq3EAUSd9R4vngM5t4D9y8EF3d+2HSKHzedomfjQJoEeed5CuEYjl+4xucrD7NwRzRuLk7U8qtg8bHt6/jxZNe6WRLgDJ7uLjzZtS73t6vFdxtPsGTPWa5cTLRm6HlycVa5jgDnxtXZibtbVWdQyyB+3R7NnC2nOGUm5kbVKjKxcx2rjwBnZ0nUrYEjWutjAEqpucAAIHMirIGKpp+9gTPWDDI3gd4efDCkKfe0qcmEBbt54oft9GhUhXcGNCHQ26MkQhBCFNH69esZMWIEzs7OVKlShS5durBlyxZatWrFww8/TEpKCgMHDqR58+bUrl2bY8eO8fTTT9O3b1969uxp6/CFI7t43Hju/zksegoWPQ2DprE3OgGAP/aclUTYwWVPgB/pGMLoznXw93K36udkJMRPdq1r1fMWp4yE+O5W1W0ahyWJcBBwOtPrKKBNtn3eBP5USj0NVADuMHcipdRoYDRAjRo1ChprrsKCvfltTAe+Xn+cT1cc4o5P1vBiRAPuaVMTJycZHRYiL5aO3Ja0zp07s3btWv744w8efPBBxo8fz/3338+uXbtYvnw5U6dOZd68ecycOdPWoQpHdfEoVAyClvfBlXOw6l1SfUI4cLY5AH/sPssLvRrIt5QFsPpgDO/+EUlKWrqtQykyrSHqUmKxJsCi6Kw1WW4EMEtr/T+lVDtgtlKqidY6y2+y1no6MB2MejMrfTZgTKZ7rEsdIpoE8sqve3ntt30s2BHN6/0a0aJGJWt+lBDCijp16sS0adN44IEHuHjxImvXruWjjz7i5MmTBAcHM2rUKK5fv8727dvp06cPbm5uDBkyhAYNGnDvvffaOnzhyOKOgm9t4+fO/4GLx3BZ+wG99ZNcrDeQdYcvsO/MZRkVtlDc1es8N28XXh4utCgFXRCs4c5mVXmwfYgkwKWYJYlwNJB53DrYtC2zR4AIAK31RqWUB1AZiLFGkAVR068Csx9pzS/bo/lg6QEGffkP/ZtV44WIBgRXku4SQpQ2gwYNYuPGjTRr1gylFB9++CGBgYF8++23fPTRR7i6uuLp6cl3331HdHQ0Dz30EOnpxr+x33//fRtHLxzaxWMQ2tf4WSm4cyLnTx/mw7jpnGrajYijquTKI9JSID3Vwp0VuBZT+aDWOftm5SYtFdJTbr58Z+FObiQnMv2hdtSv4lU88ZU0Z3dwKsG1y1Kvgy7B0XTlDC4lNzmvOOTbNUIp5QIcAm7HSIC3ACO11vsy7bMU+ElrPUsp1RD4GwjSeZy8JGYgX72eyrQ1R5m+9hgaeKRjCE92rYOXh3SXEI6tNHSNKCuka4RBukZkk5wAH9SAO96Cjs/e3PzOzxu4b98j1KyQxgOVvuPExWTWPN+1eMsjYiJhWmdIu2H5Mbe/AZ3GWzeOzV/B+s/gkeXgHZz3vldjYXIrSLpk3RhKG/+G8OBiqFC5+D9r03RYNgF0WvF/VgYnF+jzEYQ/XHKfWUiF7hqhtU5VSj0FLMdojTZTa71PKfU2sFVrvQh4DvhKKTUOY+Lcg3klwYW29xfwqwdVm1q0u6e7C8/1bMCI1jX4ePlBpqw+yrwtp3muZwOGtaqOs9QPCyGEKIyLx4xnvzpZNm85r6npPYD7E6YysJ0n4xdfLP7yiFMbjSS48/PgZkE3gsjfYeNkaDcGXKz0lf3BpbD0BWM0cssMuOPNvPffNstIgru+xLU0Z75ef5yK5Vy5r21N+/m7OfUGrP8E5o6E+xcV3yg8wIElxp9/7a5Qu0vxfU52R1fCH/8B7xpQz+z0sFLPohphU0/gJdm2vZ7p5/1AB+uGlk1aKqx4C+JPGc3Lu78KFatadGg1n3J8Mqw5D3aoxbuLI3n51z3M3XKKt/o3lvphIYQQBZfROs33ViJ8IzWdA2ev4NMgEBLg9pouODuVQHlEzAFw84Rur1hWllC1OcweCHsXQPMRRf/8s7tg/iPowGYku1Wi3LZZ0OVFcM3ZGxYwyji2fg11boeuE3jhx+38deM8vz/WEedAOymJyODfAH5+AH57EgbPKJ4yiTM74JdHoFpzGP4juJVgGWirUfBNBPz8IDy8DAKblNxnW0kJFq4UkbMLPLYW2j8Fe+bB5y1h1ftw45rFp2ga7MNPj7Vl4vDmnEtIZtCX//DC/F3EXbWssbUQQggB3GqdVqnWzU2Hzl/hRlo6AYFBAHjry7Sv48cfu89SHF+S3hSzHwIaWl6bW7srVG4Am6beWve2sBKi0T8OI9GlIkMSxvLQ4fbGSO+e+bkfE7kIrpyFNo+zZM9Z/th9lrF31KOBvSXBAI0HGqPje3+B1f9n/fMnRMGPw6G8H4z4qWSTYAB3Txg5D9y94MdhRveUMqbsJMIA5Xyg57swZjPU62ksbTmpJez4HtItq4lRSjGgeRAr/9OVxzrXZsH2aLp9vJpv/zlBqh20axFCCFECMlqnZUo89pj6B9esbppfnhhH37CqnLqYyD4rL1qTRUykkQhbSiloMxrO7oSoLYX+2NTEBOK/HkzilXgGxj9LkntlVM2ORKZX5+Kqz3NPsjdNA9/axFXtxGsL9xIW5M1jnWsXOo5Sr8Oz0OI+WPsR7PzReue9fsVIPm9cM5JRryrWO3dBVKwGI38y/gGUEU8ZYq32aSXLNwTu/hZO/QvLX4HfxsC/U6DHW1DXshoVT3cXXurTkKHh1Xlz0T7eWLSPn7ac5p2BTbitppRLCCGEyEPm1mkme6ITqOjhQmBgNWNDYhy9GgfyysK9xVcecTUWEi9AQCOLdk9P16w5FMvF1I7c6eLF2aWf8G+LDwv+sYnJNFr7BK1SD/GG1xuMv2sgPRtVISU9nTlT7uLBuE/5c+kCevYZkvXAMzvg9CZSe77Pq7/t50pyKh8PbYaLc9kalysQpaDfp0Zp56JnwLs6hHQq2jnTUmH+w8Y/gu6ZB1Us++9fbKo2hbtmwtwR8MsoGDYbnJxtG5OFymYinKFGW2N9972/wN9vw/dDoHY36PG2xRPq6gZ4MvuR1izde453Fu9nyJR/uDs8mBcjQvHzlL5/QgghzMjcOs1kb3QCTYK8URkdAhLjqFTB7WZ5RLEsrhFjWuTVPzTP3dLTNUv3nmPi34c4dP6qcQkunXgwejn/O9aPGAo2APSGy7e0c9nKvtve4p1+Y28uXuXu5MyIR5/j6kczSN04le99w7m3bc2bx6X9O4105/L0W1udg/HneCGigX2WRGTn7Ap3fwdf94Sf7oFOzxmtxworeisc/tNIsC0cACx2DSIg4gNj0t6C0VCtRfF8TpXGUKeb1U5XthNhMP6lFXYXNLwTtnwNaz802sg0HWZMqPPJf+k+pRR9wqrSpb4/k1Ye5ut1x1m29xzPR4QysnUN+5nBKoRwCEqpCGAiRqefGVrrD7K9XxOYCfgDF4F7tdZRJR5oWZWcYIzCZhoRzpgo91CHWuBaHlw8IDEOgL5hVZmwYE/xdI+IiTSecxkRzp4A1w3wZOLw5rSq5YtzQh1cvlnK312OcKXdixZ/ZIUdM/Besxzd7ika93o2x/vu5bxwbvMQvTZ+QaeFq4Gu3B1enT827qLv7p/5MbU7Hr6+fDOoHl3r+xf8msuqcj7G6O03feGv1/PdPV8dx5e+tmVtHjNGvjd+AXvzqBMvivCHJRE2y8Ud2j0JzUca7Ur+nQr7f4Pur0CbJ4zJdvmo4O7CS70bMvS2YF5buI/XFu7l562n+b9BYbIykBA25OnpydWrV82+d+LECfr168fevXtLOKrSSSnlDEwGegBRwBal1CJTd58MHwPfaa2/VUp1B94H7iv5aMsoM63TMibKNQnyNgZoyvtB4kWA4i2PiNkP5XzBMyDLZnMJ8KQRLegbVvXW4I5PKNSPwGvPbLx6vGRZK7WDy2Dt6xDaD9Xj7Vx3c2kzCv3vF7xS5R/GLKzM5ysPM/TaXNxcUwkd8BwLw9s45tLTlWrB2F2Qkli08zg5W9YqzxZ6vQddXyq+hT2crbsWhP0kwhnK+RilEa0ehSUvwJ+vGqUT/T+HwDCLTlE3wIsfR7Vh0a4zvLM4kgGTN/Bwh1qM61Gf8m7290cmhLArrYEjWutjAEqpucAAIHMi3AjIWE1hFbCwJAMs88y0TsuYKBeWkeiW9705Ilys5RExkcZosOmc2RPgOv4VcibAmbV5DA4ttayV2tldRl1qYFMYPD3vGlCfGqgGfehzcjkDmzzI2cspPO28Fl3tdtq2aluEC7YDzi7gXNHWURQvd09bR2Ax+83qfGrAiDmw71ejXmV6V+gwFjq/YFFT64zuEl3rB/DBski+WnecJXvO8e6gJnRrEJDv8UKUGUsnwLk91j1nYBj0/iDXtydMmED16tUZM2YMAG+++SYuLi6sWrWKS5cukZKSwrvvvsuAAQMK9LHJyck88cQTbN26FRcXFz755BO6devGvn37eOihh7hx4wbp6en88ssvVKtWjbvvvpuoqCjS0tJ47bXXGDZsWJEuu5QIAk5neh0FtMm2zy5gMEb5xCDASynlp7WOK5kQyzgzrdP2RCfg5eFCTT9TF4nyfjcTYSim8gitIfYANB1mNgGeOLw5/ZpWy7u8L3MrtWbDc2/BlhBtdAQoV8noEGDJaGSbx1EHFvNZoyNGd43556HNF4W6VCGKix1P08T4H7rJYKPdWtNhsO5/MLUDHF9n8Sm8y7vy/uCmzHusHeXcnHnomy08PWcHsVek97AQhTVs2DDmzZt38/W8efN44IEH+PXXX9m+fTurVq3iueeeK3Dv1cmTJ6OUYs+ePcyZM4cHHniA5ORkpk6dytixY9m5cydbt24lODiYZcuWUa1aNXbt2sXevXuJiIiw9mWWZv8BuiildgBdgGggRw9KpdRopdRWpdTW2NjYko6x9DLTOm1vdAJhQd63RnuzJcK9Ggfi7KRYvPus9eK4HA3XL3PJqy59Jq1jzI/bSUvXTBzenD/HdWFA86D857hY0krt+lWYM8x4vmceeAVaFl+tjhDQ2GiXZmqZVmomdglhYr8jwpmV94WBX0LYUPh9LHzbz/i557sW/w/dOsSXP57pyNTVx5i86ghrDsYwoXdDhreqfnO2rBBlUh4jt8WlRYsWxMTEcObMGWJjY6lUqRKBgYGMGzeOtWvX4uTkRHR0NOfPnycw0MK/dIH169fz9NNPAxAaGkrNmjU5dOgQ7dq147333iMqKorBgwdTr149wsLCeO6553jxxRfp168fnToVsZ1R6RENZJ4lHGzadpPW+gzGiDBKKU9giNY6PvuJtNbTgekA4eHhxbgiRBlz8VjuE+UyZEuEM8ojluw5y4sRViqPME2Um7THhdMXEy0bATan6XBY8bYxKly9ddb30tOMcojz+41etVUaW37ejCT797HG64gPimdlNSGKwLF+I+t0gzGbjKUf9y+Cz8ON9dbTUiw63N3FmbF31GPps51oVK0iL/+6h6HTNnLw3JViDlwI+zN06FDmz5/PTz/9xLBhw/jhhx+IjY1l27Zt7Ny5kypVqpCcnGyVzxo5ciSLFi2iXLly9OnTh5UrV1K/fn22b99OWFgYr776Km+/nfvEnzJmC1BPKRWilHIDhgOLMu+glKqslMq4/7+E0UFCWCpbD+EsE+UylPeD5Hij36uJ1RfXMLVO++W0Fy/3bWjZCLA57p7Q8j6jlHBSi6yPz8Lg8HLo8yHUK8Robtjd4OFjLAHdfGTBjxeimDlWIgzG2ufdXoYnNxp9iJe/bLRbO7HB4lPU8fdkzqi2fHRXU47GXqXvpHV8tPwAySmWrW4nhDDKI+bOncv8+fMZOnQoCQkJBAQE4OrqyqpVqzh58mSBz9mpUyd++OEHAA4dOsSpU6do0KABx44do3bt2jzzzDMMGDCA3bt3c+bMGcqXL8+9997L888/z/bt2619iTahtU4FngKWA5HAPK31PqXU20qp/qbdugIHlVKHgCrAezYJtiwy0zotx0Q5MDo5gLHalklGecQfe6xTHnHt9B7OaV/C6tZkZOsaRTtZ+2eg2UgIui3ro2Z76Ps/YwJ6YbiVNyar3zkRPKT7kih9HKM0why/OnDPz3DgD1j2EszqA03uMjpOeAfle7hSiqHh1ekeGsD/LTnA5FVHWbz7LF/e05LG1eR/diHy07hxY65cuUJQUBBVq1blnnvu4c477yQsLIzw8HBCQ/NeIMCcJ598kieeeIKwsDBcXFyYNWsW7u7uzJs3j9mzZ+Pq6kpgYCAvv/wyW7Zs4fnnn8fJyQlXV1emTJlSDFdpG1rrJcCSbNtez/TzfKCYmnzaOTOt03JMlAOjJA+M8ghPo1euNbtHaK2JObqDaKrz3yFNi15q4VUFBk4u2jly06h//vsIYSOOmwiDUb/UsB/U6Q7rP4UNE+HgEmPFl3ZPWdRdws/Tnf/d3YwhLYN47udd3DVlI58Oa0ZEk6olcAFClG179tzqVlG5cmU2btxodr/ceggD1KpV62YPYQ8PD7755psc+0yYMIEJEyZk2darVy969epVmLCFI7vZOu3WiHCOiXJglEZAljphsF73iDmbjjP4xkmSQ0YQXKl8/gcIIcxyvNIIc9zKGwtvPLXZSIpXvgNftoEDS4z2NBZoX7cyv43pQINALx7/fjuT/j5c4BnvQgghSrmbrdNCgFsT5cKyJ7UZiXDSxSybrVEecfpiIrOXrMFDpRDatHX+BwghciWJcGaVasHwH+C+hcbymHNHwOyBEG1Z7WBARQ/mjm7L4BZBfPLXIZ6es4OkG1I3LIQ17Nmzh+bNm2d5tGmTvT2uEMUsW+s0sxPlINcR4czlEYUZLNFaM2HBbuqZWkWrKuaXVhZCWMaxSyNyU6cbPL4etsyANR/CV90gtB90fxUCGuZ5qIerM/+7uxn1A73477IDnIxL5Kv7wwn0zr/MQoiSpLUuU0uchoWFsXPnzhL9TPlWR+SQrXWa2YlykLVGOJuilEf8uPkUG47EsTDsOhwG/AteSy+EuEVGhHPj7AptnzDWBO/6MhxbA1+2g19G3aoRy4VSise71OGr+8I5FnuVodP+ISHJshZtQpQEDw8P4uLiJNHLg9aauLg4PDzkH7Eik2yt08xOlAOjQ5FrBUi8SHaFLY9ISErhg6UHaF/Hj2ZuZ4xvMS1Z4U0IkSsZEc6PR0Xo+iK0HgUbPoNN02HvL0Yrme6v5NkO5o5GVfjukdbcPe1fJvyymy/vaVmmRuCE/QoODiYqKgpZLSxvHh4eBAcH2zoMUVqYaZ1mdqJchmyLamTIvLhGQbpHzFx/nCvJqbzatxHq10gIkLIIIYpKEmFLlfc1Wqu1fRLW/Bc2Tzeaj/d8F5renev67LfV9OX5Xg34YOkBvv/3JPe1q1WycQthhqurKyEhIbYOQ4iyJVvrNLMrymVW3tdsIgwFL49ISEph5objRDQOpFGAB8QdhtA+hbkKIUQmUhpRUF6B0O9TGL0KfKrDr6NhVr+bS12aM7pTbbo28OedxZHsNdWTCSGEKGOytU47EXeNG2npNKxa0fz+uYwIQ8HLIzJGg5+5vR7EHYH0VBkRFsIKJBEurGot4JEV0O8ziNkHUzvCn6+ZXa7ZyUnxv6HNqFTBlad+3M7V66k5zyeEEKJ0y9Y6LTo+CYDqvuXM759HIpy5PCK/Wv0so8HVKt5cWjm/ydtCiPxJIlwUTk4Q/hA8tQ2ajYB/JsHyV8zu6ufpzqThLTh1MZGXF+yRSUpCCFHWZGuddsaUCFfzySsRzjlZLkPfsKqcjEtk35nLeX5sltFgML6BdHIBv3oFvwYhRBaSCFtDBT8Y8AW0HQObp8H278zu1qa2H+PuqM+iXWf4acvpEg5SCCFEkWRrnXYmPglnJ0WAVy6dRcr7wfXLkHrD7NuWlEfkGA0GIxH2rQMuboW+FCGEQRJha+rxNtTuBovHw6lNZnd5sltdOtatzBuL9nHo/JUSDlAIIUShZWuddiY+mcCKHjg75dL1IaOXcJL5UWFLyiNyjAaDURohZRFCWIUkwtbk7AJDvzEm0f10LyRE59zFSfHpsOa4OTsxdXXe/YiFEEKUEmZap0VfSiIot7IIyHV1ucwyyiNmrDvO9dSsK5GaHQ2+kQiXTshEOSGsRBJhaytXCYbPgZQkmDvSeM7G38udfs2qsmzfOa7JxDkhhCj9srVOA2OyXDWfPBZcsSAR7tesGm1CfHlvSSRdP1rN7H9P3kyIzY4GXzgIaBkRFsJKJBEuDgGhMOQrOLsLFj0DZr7yGtwymMQbaSzfd84GAQohhCiQbK3T0tI15y4n5z5RDixKhD3dXZg7ui3fP9KGIJ9yvLZwL10/Ws3M9cdzjgbDrVadMiIshFVIIlxcGvQ2Vp7bMw/++TzH2+E1K1Hdtxy/7shZPiGEEKKUydY6LeZKMmnpmqBKRUuEAZRSdKxXmZ8fb3czIX578f6co8Fg1Ac7u4OvLIgjhDVYtLKcUioCmAg4AzO01h9ke/9ToJvpZXkgQGvtY8U4y6ZO/4EzO2Hlu9BkCHgH3XxLKcWgFsF8vvIw5xKSCfTO4+s1IYQQtlXQ1mlwa7JcHi3UMstIiDvU9WPDkTguJt7IOhoMxoiwfwNwci7wJQghcsp3RFgp5QxMBnoDjYARSqks38lorcdprZtrrZsDnwMLiiHWskcpiHgf0LDmgxxvD2oRhNawcKeMCgshRKmWrWNEdHwyQN6T5Zxdwb1iviPC2WUkxP2bVcv6htZwbo/UBwthRZaURrQGjmitj2mtbwBzgQF57D8CmGON4OyCTw0IfwR2/AAXDmd5K6RyBVrW8GHB9ihZYEMIIUorrSH2gDESa5IxIlw1v2/zyvtaPCKcr5j9cPU81OxgnfMJISxKhIOAzKs/RJm25aCUqgmEACuLHpod6fQcuHgYJRLZDG4ZzKHzV/NdWUgIIYSNXD5jLIyRaST2THwSFT1c8PJwzfvYPJZZLrDDfxnPde+wzvmEEFafLDccmK+1TjP3plJqtFJqq1Jqa2xsrJU/uhTz9If2T8H+hXBmR5a3+jWtipuzk0yaE0KI0spMp4Yz8Ul51wdnsGYifGQFVGmSZb6JEKJoLEmEo4HqmV4Hm7aZM5w8yiK01tO11uFa63B/f3/Lo7QH7Z6Ccr7w99tZNvuUd6N7aAC/7YwmNS3dRsEJIeyJUipCKXVQKXVEKTXBzPs1lFKrlFI7lFK7lVJ9bBFnmRGz33j2D725KSq/xTQylPezTmlE8mU4tVFGg4WwMksS4S1APaVUiFLKDSPZXZR9J6VUKFAJ2GjdEO2ER0WjROLoSji+Nstbg1oGceHqDdYdvmCj4IQQ9sKSCc7Aq8A8rXULjHv6lyUbZRkTEwleVW91gcAGI8LH10B6KtTrUfRzCSFuyjcR1lqnAk8By4FIjJvnPqXU20qp/pl2HQ7M1TLrK3etHjXa76x4K8siG90aBOBT3pUFUh4hhCg6SyY4ayCjL5c3cKYE4yt7YvZnGQ2+kpzC5eRUCxNhX0i5ZnaV0QI5/JfRgaJ6m6KdRwiRhUU1wlrrJVrr+lrrOlrr90zbXtdaL8q0z5ta6xxfwYlMXD2g6wSI3goH/ri52c3Fif7NqvHnvnNcTk6xYYBCCDtgyQTnN4F7lVJRwBLgaXMncth5HZmlp0HswSz1wWcTTK3T8lpMI8PNRTWKUB6htVEfXLuL0ZJNCGE1srJcSWs2Evzqwcp3jBusyaAWQVxPTWfZHllyWQhR7EYAs7TWwUAfYLZSKsffBw49ryPDpROQmpSlY0S0qXVakI8FCyFZuLpcnmL2w+VoqCtlEUJYmyTCJc3ZBW5/zehJuWf+zc3Nq/tQu3IFftkeZcPghBB2wJIJzo8A8wC01hsBD6ByiURX1uTSMQLyWVUugzUSYWmbJkSxkUTYFhr2B986sPP7m5uMJZeD2HT8ImcTilhLJoRwZJZMcD4F3A6glGqIkQg7aO1DPmJNiXC2xTScnRQBXiU0InxkBQQ0lrZpQhQDSYRtQSloejccX2c0ajfp2TgQgLWH5O8jIUThWDjB+TlglFJqF0bLywdlonMuYiLBpya4e97cdCY+mcCKHjg7qfyPL2qNcEbbNOkWIUSxkETYVsKGAhr2Lri5qX4VT6pUdGettFETQhRBfhOctdb7tdYdtNbNtNbNtdZ/2jbiUiwmMktZBBg1whb1EAbw8AFU4UeEpW2aEMVKEmFb8asD1VrCnnk3Nyml6FTPn/WHL5CWLoMzQghhU6k34MKhLBPlAKIvJVHNkolyYMwLKedT+ERY2qYJUawkEbalsKFwdhfEHrq5qXN9fxKSUtgdFV+4c148Blu+ztKnWAghRCFcPGqMxmZKhNPSNecuJ1s2US5DYRfVkLZpQhQ7SYRtqclgUE6w5+ebmzrWrYxSFG6VuSvn4dsB8Md4OL3JioEKIYQDylhaOVMiHHMlmbR0XTKJsLRNE6LYSSJsS16BENLZSIRNI7i+FdwIC/Iu+IS5G9dgzjBIvAAuHrB7Xv7HCCGEyF1MJChno/e7SUbrNIsW08hQ3q9wk+WkbZoQxU4SYVsLGwqXjkP09pubOtfzZ8fpeMtXmUtPg/mPGGUWd82E0L6w71dIk1XqhBCi0GIijfkcrrfqgaPjTavKFWhE2LdwI8LSNk2IYieJsK01vBOc3bNMmutUrzJp6Zp/jlhw49Qalk2AQ0uh94fQoDeE3Q1JF+HI38UYuBBC2LmYyBwT5TJGhKt6WzhZDm6VRhRk7oa0TROiREgibGse3lC/l9FGLS0VgJY1K+Hp7sLawxaUR/z7JWyeDu2egtajjG11b4dyvlmSayGEEAWQkmRMPs7WOu1MfBIVPVzw8ijA5LVyvpB2HVISLT9G2qYJUSIkES4Nmt4N12KMGx/g6uxEuzp+rD0US5497vf/BstfMVaq6/HOre3OrtB4EBxYAtevFHPwQghhh2IPAtrsiHCBJspB4VaXO7EBXMtL2zQhipkkwqVB3R7g7g175t/c1LleZaIuJXEiLpcRhIRoWPAYBLeCwdPBKdt/yqZ3Q2oSHPijGAMXQgg7FZOxtHK2HsLxyQWrD4bCJcJxR8CvrrRNE6KYSSJcGrh6QKM7IfJ34+s4jH7CkMdyyxu/gPQUGDIDXM3clKu3AZ8a0j1CCCEKI2Y/OLuBb+0sm6MvJZbMiPDFozk+WwhhfZIIlxZhd8ONK3BoGQA1/SpQw7c868zVCV+7ANtmGcdUqmn+fEoZHSmOrYKrMcUXtxBC2KOYSKjcwFgZzuRKcgqXk1OLkAhb2EItLQXiTxkdK4QQxUoS4dKiVkfwDITdtxbX6Fy/MhuPxnEjNT3rvpumGiPHHZ/N+5xhQ0GnGxPxhBBCWM5Mx4izCUbrNIuXV85Q3td4tnREOP6UMVFORoSFKHaSCJcWTs4Qdhcc/hMSogCjn/C1G2lsO3np1n7Jl2HTdKPtmn+DvM8Z0BCqhEn3CCGEKIjky3A5KkciHG1qnRZckMU0ADx8jFVELU2ELx43nn1lRFiI4iaJcGnS6lFwcYdfHoW0VNrV8cPFSWUtj9j6NVxPgE7jLTtn06EQvQ3ijhZPzEIIYW9iDxjPZlqnAQUvjXByMlqoWZwIm+7XMiIsRLGTRLg08Q2Bfp8aTdTXfICXhysta1S61U84JQk2ToY6t0O1Fpads8ldgDKWcRZCCJG/mP3Gs5nWac5OigCvApZGwK1FNSxx8Ri4eYJnQME/RwhRIJIIlzZN74bm98Laj+HoKjrVq8ze6MtcuHodts+Ga7HQ6TnLz+cdZNQf755XsFWNhBDCUcVEGomod/Usm8/EJxNY0QNnJ1Xwc5b3s3yyXJypY4QqxOcIIQpEEuHSqM+HULk+LBhNd9N9+J9DZ2DDRKjeFmq2L9j5woYaX7Wd2W79WIUQwt7E7DfmYGTrzx4dn1TwHsIZyhewNELKIoQoEZIIl0ZuFWDoLLh+mUYb/4NvOWeubZlrTN7o9FzBRwkaDTD6Ye6W8gghhMiXmY4RkLGqXCHKIsDy0ghpnSZEiZJEuLSq0gh6/xd1fDX/5/8nbc58R6p/48KtO1/OB+r3gr3zjZusEEII867GGiVo2SbKpaVrziUkF3yiXIaMRDi/EjVpnSZEiZJEuDRr+QA0GUJEzNfUVmdYVmlk4WvGmo0wbu5HV1o3RiGEsCexpqWVs40Ix1xJJjVdFy0RTk+F65fz3k9apwlRoiQRLs2Ugn6fgW9tzrrW5NVDdbiSXMgR3bo9jBvxrjlWDVEIIexK7EHj2T80y+aM1mmFrxG2cJllaZ0mRImSRLi086gIj63l4rBFxCen8/2/pwp3Hhc3o5XagSWQdCn//YUQZZZSKkIpdVApdUQpNcHM+58qpXaaHoeUUvE2CLN0ykhUK2RtXRYdb6wqF1TQxTQyWLrMctxRaZ0mRAmSRLgscPeicd1adKpXma/XHyM5Ja1w52k2HNKuw76FVg1PCFF6KKWcgclAb6ARMEIplaXgVWs9TmvdXGvdHPgckHXYMyTFg5sXOLtk2ZwxIlzVuwiT5cCCEeFj0jpNiBIkiXAZ8lS3uly4eoN5W08X7gTVWhhf9+2aa93AhBClSWvgiNb6mNb6BjAXGJDH/iMAqZnKkBwPHt45Np+JT6KihwteHq6FO295X+M5vxFhaZ0mRImSRLgMaR3iS3jNSkxbc4yUtPSCn0ApY1T49L+y5LIQ9isIyPyv5SjTthyUUjWBEMDsLFql1Gil1Fal1NbY2Fhzu9if5ASj0042Ruu0QpZFgGUjwtI6TYgSZ1EinF+9mWmfu5VS+5VS+5RSP1o3TAGglGJMt7pExyfx284zhTtJ2N2Agt0/WTU2IUSZNByYr7U2W2+ltZ6utQ7XWof7+/uXcGg2khQPHj45Np+7nExgYcsiANy9wMkl70RYWqcJUeLyTYQtqTdTStUDXgI6aK0bA89aP1QB0LWBP42qVuTL1UdISy/EksneQVC7q9E9Ir0Qo8pCiNIuGsi8NnCwaZs5w5GyiKyS482OCF+6loJvebfCn1ep/BfVkNZpQpQ4S0aELak3GwVM1lpfAtBax1g3TJEhY1T4WOw1lu87V7iTNBthjDyc2mjd4IQQpcEWoJ5SKkQp5YaR7C7KvpNSKhSoBMiNILNcRoQTklLwLl/I+uAMPjWMVetyI63ThChxliTCltSb1QfqK6U2KKX+VUpFmDuRQ9abFYOIJoHUrlyByauOoPNbpcichv2M9jzSU1gIu6O1TgWeApYDkcA8rfU+pdTbSqn+mXYdDszVhbqJ2DEzI8IpaelcvZ6KT7kijAgD1OkO0VtznzAnrdOEKHHWmiznAtQDumLMQP5KKeWTfSeHrDcrBs5Oise71mHfmcu8tGAP/xy9QGpBJs+5VYBGA4w2ailJxRanEMI2tNZLtNb1tdZ1tNbvmba9rrVelGmfN7XWZud8OKzUG5CSmGNEOCHJWMioUoUijgjX7QE6PfcVPqV1mhAlzpJE2JJ6syhgkdY6RWt9HDiEkRiLYjKoRRCDWwTx645oRn61iVbvreA/P+/ir/3nLesz3Gw43LgCB/4o/mCFEKIsSI43nrONCMcn3gDAu1wRE+GgllDOF46sMP++tE4TosRZkghbUm+2EGM0GKVUZYxSiWPWC1Nk5+rsxCfDmrPj9R5MvbclXRsE8Oe+c4z6bitt3/+btYfyKT2p2RG8q0t5hBBCZEhOMJ6z9RGOTzRGhH2KMlkOwMkZ6t5uJMLZJytL6zQhbCLfRNjCerPlQJxSaj+wCnhea53P8jnCGsq7uRDRpCqfDmvOttd6MPuR1gRW9ODBbzYzY92x3GuInZyg6TDjK7orhZx0J4QQ9iQp3njOVhpxMxEu6ogwGOUR12Lh7M6s26V1mhA2YVGNcH71ZtowXmvdSGsdprWWpctswNXZiU71/Pnlifb0ahzIu39E8p+fd+deKtF8JGgNK98p2UCFEKI0yqU04pKpNKJSUUeEwRgRRuUsj7ho+hJVWqcJUaJkZTk7VMHdhckjWzLujvr8sj2K4dP/5fzl5Jw7+tWBTuNhx/ew/7eSD1QIIUqTXEaEMybLFbl9GkCFysZy94f/yrr9ZiIsI8JClCRJhO2Uk5Ni7B31mHrvbRw6f4U7P1/PrtPxOXfs+hJUawmLnoGE3HruCyGEA8h1slwKTgq83F2s8zn1euRsoyat04SwCUmE7VxEk0AWPNkeNxcnHp61JefIsLMrDJlhTNT49TFZbU4I4bhyqxFOuoFPeTecnKzU1sxcGzVpnSaETUgi7ABCAysy66FWJKWk8fScHTl7DvvVgd7/hRPrYOPntglSCCFsLTkeXMuDS9Za4EuJKdaZKJfBXBs1aZ0mhE1IIuwg6gZ48d6gJmw+fpFPVxzKuUOLe6Fhf/j7HTizs8TjE0IIm8tteeVEKyyvnJmTs7HKXEYbNWmdJoTNSCLsQAa1CGZ4q+pMXnWU1Qdjsr6pFNw5ESr4wy+Pwo1rtglSCCFsJTk+Rw9hMJVGWHNEGIw64Yw2atI6TQibkUTYwbzZvzGhgV6M+2knZxOyLa9c3hcGTYW4I7BsgtFaTQghHEVyQo6JcgCXrqVYp3VaZnUytVGT1mlC2Iwkwg7Gw9WZyfe05EZqOk//uIOU7PXCtbtAh7Gw/Ttj8lyKmbZrQghhj3IrjUiycmkEgKf/rTZq0jpNCJuRRNgB1fH35P8Gh7H15CU+/vNgzh3ueBO6vQK7f4JZfeDy2RKPUQghSlxyfI4R4ZS0dK5eT8WnnJVHhOFWG7WordI6TQgbkUTYQQ1oHsTINjWYtuYYM9cfz7oUs1LQ5QUY9j3EHIDpXY0btRBC2DMzI8IZi2lUqmDlEWG41UZt/0JpnSaEjUgi7MBe79eIOxoG8Pbi/Tz+/TYSElOy7tDwTnj0L3Bxh2/6wM45tglUCCGKW1oq3LhiZjENY3llb2tPloNbbdTSbkhZhBA2IomwA/Nwdear+8N5tW9D/o6Moc+kdew4dSnrTlUaw6hVUL01LHwcFo83JpQIIYQ9ybivZV9MwzRA4GPtyXJwq40aSOs0IWxEEmEHp5Ti0U61+fnxdgAMnbqRr9Yey1oqUcEP7vsV2j0FW2fC5Dawb6F0lRBC2I88llcGrN8+LUO9HsazjAgLYROSCAsAWtSoxJJnOnF7wwDeWxLJo99u5dK1G7d2cHaFXu/Bo39Dhcrw8wPw4zC4dNJ2QQshhLVkJMLZ+ghfMpVGWL19WobQvnDbg1CvZ/GcXwiRJ0mExU3e5V2Zeu9tvHlnI9YdvkCfSevYcuJi1p2Cb4NRq6Hne8aSzF+2hQ0TISXJ7DmFEKJMSIo3nnOZLGf19mkZ3L2MxYykY4QQNiGJsMhCKcWDHUL45Yn2uLk4MXz6v0xedYT09ExlEM4u0P4pGLMZQrrAX6/D/xrA0hchJtJ2wQshRGHlURrhpMDL3aXEQxJCFD9JhIVZYcHeLH66I72bBPLR8oM88M1mYq9cz7qTT3UYMQce/MNoA7R1pjFC/HVPo8OEjBILYRNKqQil1EGl1BGl1IRc9rlbKbVfKbVPKfVjScdY6uQyIhyfdAPvcq44OUlrMyHskSTCIldeHq58PqIF7w8OY/Pxi/SZtI71hy9k3UkpqNUR7voaxh+Anu9CYpzRYeLTJrDje0hPN/8BQgirU0o5A5OB3kAjYIRSqlG2feoBLwEdtNaNgWdLOs5SJ5cR4UuJxbC8shCi1JBEWORJKcWI1jX47akOVPRw4d6vN/Hu4v0kp6Tl3LmCH7R/Gp7aCg8sNtoB/TYGZvaEMztLPHYhHFRr4IjW+pjW+gYwFxiQbZ9RwGSt9SUArXVMCcdY+iTFg7M7uJbLsjkhsRiWVxZClBqSCAuLhAZWZPHTnbivbU1mrD/OwMkbOHDusvmdlYKQTvDQMhg4BS6dMFanWzwOEi+aP0YIYS1BwOlMr6NM2zKrD9RXSm1QSv2rlIoosehKKzPLK4NRGlFsrdOEEDYnibCwWDk3Z94Z2IRvHmzFhavX6f/5BmasO5Z1Il1mTk7QfKQxQtzmMdg2Cz6/DbZ+A+lmRpSFECXFBagHdAVGAF8ppXyy76SUGq2U2qqU2hobG1uyEZa05IQc9cFgTJaT0ggh7JckwqLAuoUGsPzZznRp4M+7f0Ry38xNnE3IY2JcOR/o/V94bB34h8LiZ2F6FzixoaRCFsKRRAPVM70ONm3LLApYpLVO0VofBw5hJMZZaK2na63Dtdbh/v7+xRZwqZAUn6OHMBiJsJRGCGG/JBEWheLn6c70+27jg8FhbD8ZT69P17J495m8DwpsAg8tgbtmQuIlmNUHfn4Q4k/nfZwQoiC2APWUUiFKKTdgOLAo2z4LMUaDUUpVxiiVOFaCMZY+ZkojUtLSuXo9FZ9yMiIshL2SRFgUmlKK4a1rsGRsJ0L8PXnqxx2M/2knl5NT8joImgyBp7ZAlwlwcCl80QpWvQ83EksueCHslNY6FXgKWA5EAvO01vuUUm8rpfqbdlsOxCml9gOrgOe11nG2ibiUSIrPdTENHxkRFsJuSSIsiiykcgXmP96OsbfXY+HOaHp/to7Nx/OZFOdWHrq9ZNQPN+gNaz6Aya1h7wLQudQcCyEsorVeorWur7Wuo7V+z7Ttda31ItPPWms9XmvdSGsdprWea9uISwEzI8LxpuWVJREWwn5JIiyswtXZiXE96vPz4+1xdlIMm76RD5cdICUtnx7CPtVh6Dfw4BJjNGb+QzCrH5zbUyJxCyEE6emQfDnnYhqJGSPCUhohhL2SRFhY1W01K7FkbCeG3hbMl6uPctfUjZyMu5b/gbU6wGNroO8nELMfpnU22q1dc+xva4UQJeB6AqDNLq8MSPs0IeyYJMLC6jzdXfjwrmZMHtmSY7FX6TtpPQt3ZJ+0boaTM7R6BJ7ZDq1Hw7Zv4ZNQ+H4IbPkaLuczGU8IIQoj1+WVjURY2qcJYb8kERbFpm/Tqiwd24nQQC+e/Wkn43/aydXrqfkfWK6S0W7tiX+MhDjuKPwxHj5paCzMseYjuGrnPU2FECUnOcF4zqVGWNqnCWG/JBEWxSq4Unnmjm57cyJd30nr2HHqkmUHB4RCr/fgmR3w5Ca4/XVQzrDqXaP1miTDQghrSI43nrP1EY5PTMFJgZe7S8nHJIQoERYlwkqpCKXUQaXUEaXUBDPvP6iUilVK7TQ9HrV+qKKscjFNpJs7uh0pqekMmfIP7y+JJDnFwtXllDKS4k7Pwai/jYl18afh+0G3vtIUQojCyrU04gbe5VxxclIlHpIQomTkmwgrpZyByUBvoBEwQinVyMyuP2mtm5seM6wcp7ADrUN8WTauM8NaVWfa2mP0mbiObSfzabNmTq0OMPx7iDkAPwyF61etH6wQwnFkjAhnK424JMsrC2H3LBkRbg0c0Vof01rfAOYCA4o3LGGvKnq48v7gpnz/SBuup6Zz19SNvP37fpJuWDg6nKHuHcYKddHbYO4ISEkunoCFEPYvlxHhBFleWQi7Z0kiHARkXgM3yrQtuyFKqd1KqflKqepm3kcpNVoptVUptTU2Vuo7HVnHepVZPq4z97WtycwNx4mYuJYtJwo4OtyoPwz8Eo6vNZZqTstjRTshhMhNcjw4uYBbhSyb45NuSOs0IeyctSbL/Q7U0lo3Bf4CvjW3k9Z6utY6XGsd7u/vb6WPFmWVp7sLbw9owtzRbUnXmrunbeT9pZFcTy3A6HCz4dDnYzi0FH59DNILOLIshBAZyyurrLXA8VIaIYTdsyQRjgYyj/AGm7bdpLWO01pfN72cAdxmnfCEI2hb24+lYzszvFUNpq05xoAvNrD/zGXLT9B6FNzxFuz9BX5+AFKv53+MEEJkSE7IUR8MRiIspRFC2DdLEuEtQD2lVIhSyg0YDizKvINSqmqml/2BSOuFKByBp7sL7w8O45sHWxF37QYDJq9n8qojpOa3RHOGjs9Cr/ch8nfTBLorxRqvEMKOJMfnqA9OSUvn6vVUfMrJiLAQ9izfRFhrnQo8BSzHSHDnaa33KaXeVkr1N+32jFJqn1JqF/AM8GBxBSzsW7fQAP58tjM9GwXy0fKDDJ22kUPnLUxq2z0JA6fCifXw3QBILERHCiGE40mKzzEinGBaVc5HRoSFsGsW1QhrrZdoretrretord8zbXtda73I9PNLWuvGWutmWutuWusDxRm0sG+VKrjxxcgWTBzenBMXrtF30jo++euQZbXDzUfAsO/h3F6YGQEJFiztLIRwbMnxZhbTMFaVk0RYCPsmK8uJUkkpxYDmQawY34V+Tasx6e/D9J20nq2WdJYI7QP3LYDLZ4xk+MKR4g9YCFF2ZUyWyyQ+MWNEWEojhLBnkgiLUs3P051PhzVn1kOtSLqRxl1TN/Lqwj1cSc6nVVqtjvDgYkhJhBndYc/8kglYCFG2aG12stzNRFjapwlh1yQRFmVC1wYB/DmuMw93COGHTafo8clalu87l/dB1ZrDoyugcgP45RGY/zAkXSqReIUQZcT1K6DTzCyvLDXCQjgCSYRFmVHB3YXX72zEr092wKe8K4/N3sbjs7dx/nIeq8r5hsBDS6H7q7D/N/iyPRxdVXJBCyFKt1yWV75VIyylEULYM0mERZnTvLoPvz/dkRciGrDqYAx3/G8N3/97kvR0bf4AZxfo/Dw88pexctTsgbB0grRYE0IYZRFgtkbYSYGXu0vJxySEKDGSCIsyydXZiSe71mX5s51pWt2bVxfu5e5pGzkWezX3g4JawmNrofVo2DQFPq4PCx6DY6tlRTohHFVSvPGcfUQ46Qbe5VxxclI5DhFC2A9JhEWZVqtyBb5/pA0fD23G4Zir9J64jq/XH899dNitPPT5CB5dCWFD4eASo+fwZ2Gw4i24cLhkL0AIYVsZpRFmRoRleWUh7J8kwqLMU0px123B/DWuMx3rVuadxfsZPv1fTsZdy/2g4Nug/yT4zyEY8jUENIINn8EX4fDjcDi9pcTiF8LalFIRSqmDSqkjSqkJZt5/UCkVq5TaaXo8aos4S4WMEeEcfYRleWUhHIEkwsJuBFT0YMYD4Xw8tBmR5y4T8dk6Zm88kfvoMIBrOQi7C+6dD+MjoetLcPpf+PoOmNUPjq402isJUUYopZyByUBvoBEwQinVyMyuP2mtm5seM0o0yNIkt8lySTekdZoQDkASYWFXMkaH/xzXmdYhvrz22z6Gf/UvkWcv53+wVyB0nQDP7oWe70HcEZg9CKZ3he3fQewhSYpFWdAaOKK1Pqa1vgHMBQbYOKbSKykelBO4eWXZHJ+YIh0jhHAAkggLu1TVuxyzHmrFf4eEcfj8FfpOWserC/dw6dqN/A9294T2T8HYXXDnJLh+GRY9DZNbwYch8MNQWPsRHF8rk+xEaRQEnM70Osq0LbshSqndSqn5Sqnq5k6klBqtlNqqlNoaGxtbHLHaXsbyyk5Z/zo0EmEZERbC3klfGGG3lFIMa1WDXo0D+WzFYWb/e5Lfd51l3B31uLdtTVyc8/l3oIs73PYAtLgP4g7D6c1wehNEbYHDfxr7VG8Lg6dDpZrFf0FCWM/vwByt9XWl1GPAt0D37DtpracD0wHCw8Pt8+uQ5IQcE+VS0tK5ej0Vn3IyIiyEvZMRYWH3fMq78Wb/xix5phNNgiry5u/76TNpHf8cvWDZCZycwL8BtLwPBnwBYzbBiyeg/+dwfh9M7ShLOIvSJBrIPMIbbNp2k9Y6Tmt93fRyBnBbCcVW+iTF56gPTpBV5YRwGJIIC4fRINCL7x9pw7T7biMpJY2RX23imTk7iMlrZbrclKsELe+HJ9ZDQENjCecFoyHZglpkIYrXFqCeUipEKeUGDAcWZd5BKVU108v+QGQJxle6JMebbZ0GkggL4QgkERYORSlFr8aB/DWuC890r8uyvefo/r81zFx/nNS09IKfsFIteHCJ0W1iz8/G6PDpzVaPWwhLaa1TgaeA5RgJ7jyt9T6l1NtKqf6m3Z5RSu1TSu0CngEetE20pYCZEWFZXlkIxyGJsHBIHq7OjO/ZgOXjOtOyZiXeXryfO7/YwLaTFwt+MmcXo9vEQ8sADTN7wZ+vQUqS1eMWwhJa6yVa6/pa6zpa6/dM217XWi8y/fyS1rqx1rqZ1rqb1vqAbSO2oYzJcpncHBGW9mlC2D1JhIVDC6lcgW8fasWUe1oSn3iDIVM2Mn7eTmKuFKJcokYbeHy9Mbnun0kwpQOc/Mf6QQshrENrY0Q4e2mE1AgL4TAkERYOTylF77CqrBjfhSe61uH3XWfo/vEavlp7jJSClkt4eBsr1t3/G6Snwje9YcnzcP1q8QQvhCi8lERIT5HSCCEcmCTCQphUcHfhxYhQ/hzXhVa1KvHekkgiPlvLusOF6J9auys8uRHaPAGbv4Iv28G+XyEtxepxCyEK6ebyyj5ZNscnpuCkwMtdOowKYe8kERYim5DKFfjmodZ8/UA4qema+77ezKjvtnIkpoCjum4VoPcH8PAycPWAnx+ETxrBX29A3NFiiV0IUQDJCcazmeWVvcu54uSkSj4mIUSJkkRYiFzc3rAKf47rzPO9GrDxaBy9PlvLSwv2FLzdWo228MRGGPETBIfDP5/D5y1hVj+j//CNa8VzAUKIvCXHG89mRoQrSVmEEA5BvvcRIg/uLs6M6VaX4a2q8/nKI/yw6SQLd0QzqlMIo7vUwdPSr06dXaBBhPG4fBZ2fg/bZxv9h108oE53CO0HDXpDed/ivSghhCGjNCJHjXAK3jJRTgiHIImwEBbw83Tnzf6NeahDLT5afpBJK4/ww6ZTPNW9LiPb1MDdxdnyk1WsCp2fh47PwckNEPk7HPgDDi4B5Qw120Oz4dD8HlDy1awQxSa3EeGkG/h7upd4OEKIkielEUIUQE2/CnwxsiW/jelA/SpevPX7frp9tJqftpwq+IIcTk4Q0gn6fAjj9sLo1dBxHFyNgd/GwOr3i+UahBAmNyfL5ewjLB0jhHAMkggLUQjNqvswZ3Rbfni0Df4VPXjxlz30+HQti3adIT1dF/yESkG1FnD7a/Dkv0Yv4jX/hXWfWD94IYTh5oiwuURYSiOEcASSCAtRBB3qVmbhk+2Zft9tuDk78cycHfSZtI6VB86jdSESYjBGiu+cCGFD4e+3YOOX1g1aCGFIigd3b3C6VdqUkpbO1eup+JSTEWEhHIHUCAtRREopejYO5PaGVVi8+wyf/HWIh2dtJbxmJV6ICKV1SCEmvzk5w8CpkJoMy18y2q+FP2z94IVwZMkJUC7raHCCrConhEOREWEhrMTZSTGgeRArxnfh3YFNOHUxkbunbeShbzaz70xCIU7oAkNmQr2esHg87Jxj/aCFcGTJ8WZbp4EkwkI4CkmEhbAyV2cn7m1bkzXPd2NC71C2n4qn76T1PDNnB6cvJhbsZC5ucPdsCOkMvz1p9B0WQlhHUrwsryyEg7MoEVZKRSilDiqljiilJuSx3xCllFZKhVsvRCHKpnJuzjzepQ5rX+jGk13r8Of+c3T/32re+n0fF6/dsPxErh4wYg7UaAcLRsGO74svaCEcSV4jwuVkRFgIR5BvIqyUcgYmA72BRsAIpVQjM/t5AWOBTdYOUoiyzLucKy9EhLL6P90Y0jKYb/85QZcPVzF51RGSbqRZdhK3CnDPzxDSxWittmla8QYthCMwNyIsNcJCOBRLRoRbA0e01se01jeAucAAM/u9A/wXKOD6s0I4hkBvDz4Y0pTlz3amTW0/Plp+kK4fr+Lbf06QnGJBQuxWAUb+ZKxAt/QFWPe/4g9aCHuWHJ+jddp50xLqvhWkNEIIR2BJ14gg4HSm11FAm8w7KKVaAtW11n8opZ7P7URKqdHAaIAaNWoUPFoh7EC9Kl7MeCCczccv8uGyA7yxaB+frzzMQx1CuK9dTSp65DES5eIOQ2fBwifg77fhxjXo/pqsQCdKp4PL4NweW0dhnk43urJkK404eO4KQT7l8Mrr/0MhhN0ocvs0pZQT8AnwYH77aq2nA9MBwsPDC9lkVQj70DrEl/lPtGfz8YtMXnWEj5YfZOrqo9zfviYPdQihcm5LvDq7wqBp4FreGBW+fhVufx3cPUv2AoTIz8E/YPt3to4id8oZqjTJsiny7GUaVvWyUUBCiJJmSSIcDVTP9DrYtC2DF9AEWK2MUalAYJFSqr/Wequ1AhXCXrUO8aV1SGv2Rifw5eojfLn6KDPXn+D+9jV5rHMd81/ROjkbi264e8HGL2DLDKjaDGq2Nx412kH5QvQvFsKa+n0GfT+1dRS5UyrLYhrJKWkcu3CNXo0DbRiUEKIkWZIIbwHqKaVCMBLg4cDIjDe11glA5YzXSqnVwH8kCRaiYJoEefPlPbdxJOYqX6w8zPS1x/h+40ke6hDCqE618c4+eUcp6Pku1O8Fx9fCyX9g81dGYgwQ3Br6fGgs3SyELWRKMsuCIzFXSUvXNKxa0dahCCFKSL6JsNY6VSn1FLAccAZmaq33KaXeBrZqrRcVd5BCOJK6AZ58NrwFY7rV5bMVh/li1RG+3XiCRzvW5qGOtbLWECtl9BgO6Wy8TkmGMzvg5HrYPAO+6g7txkDXl8GtvG0uSIgyIvLsZQBCpTRCCIdhUR9hrfUSrXV9rXUdrfV7pm2vm0uCtdZdZTRYiKKrV8WLyfe0ZMkznWhX249PVxyi/fsreev3fZyKy2VhDlcPqNkOOj8PYzZBy/vhn89hSjs4uqpgAaQkQeLFol+IKHHS+71wIs9ewcPViVp+FWwdihCihBR5spwQong1qlaR6feHszc6gRnrjjF740m+/ecEPRpV4ZGOtWlVqxLKXNeIcj5GHXHY3fD7MzB7IDQbCR3GQuV65r+2Tr0OR1fC3l/gwBJITYJanaDJEGh4p9QdlwGZer/3wOjys0UptUhrvT/bftL7PZsD5y7ToIoXzk7ShUUIRyGJsBBlRJMgbz4b3oIJvRvy3cYT/Lj5FMv3nScsyJuHO9aib1g13FzMfMlTqwM8vgHWfgQbPoNdP4KLBwQ0gsAmUCUMvKrAoT8h8ne4ngDlKkHToVDBH/YuMBLpP56DurebkuL+xuizKI1u9n4HUEpl9H7fn22/jN7vuba8dCRaayLPXpaJckI4GEmEhShjAr09eCEilKe71+OX7VHM3HCccT/t4v+WHOD+tjUZ2aYGftlbr7l6wO2vQcv74NS/Rm/Xc3sgcvGt9lZuXtCwn5Ho1u5qtGkD6PYKnN0Fe+cbSfGhZRDwGQybDX51SvLShWWk93shxFy5zqXEFEIDpT5YCEciibAQZVQ5N2fubVuTka1rsPZwLDM3nOB/fx3i81VHGNQ8iIc61iI0MNvs90q1jEez4cZrreHKWYg/BVWbmx/lVQqqNTced7xtJMK/PQnTu8LAKUbyLMoM6f1u3n7TRDnpGCGEY5FEWIgyzslJ0bVBAF0bBHAk5grfbDjBgu3R/LT1NJ3r+/NY59q0r+Nnvo5YKahYzXhY9mEQ2gcC18K8B+Cne4ya4+6vg7PcTkoJ6f1eCAfOXgEgVBJhUcqlpKQQFRVFcnKyrUMplTw8PAgODsbV1bLVIeVvLiHsSN0AL94bFMbzvRrww6ZTfLPhBPfM2ESToIqM6lSbvmFVcXG2qFlM3nxqwMPLYNlLsGEiRG2Du2YatcbC1qT3eyFEnr1MkE85vMvJ0sqidIuKisLLy4tatWqZH+BwYFpr4uLiiIqKIiQkxKJjrPA3ohCitPEp78aYbnVZ/2I3PhgcRuKNNMbO3UmXj1bz9frjXElOKfqHuLhDv0+M5Z6jt8G0TrB7HqSnF/3cotC01qlARu/3SGBeRu93pVR/20ZXeh04J0sri7IhOTkZP79cvuVzcEop/Pz8CjRaLomwEHbMw9WZ4a1rsGJcF766P5xqPh68s3g/7d9fyXt/7CfqUi79iAui2XAY9Td4BcKCUfD1HXBKOnLZkvR+L5jklDSOxl7LWVMvRCklSXDuCvpnI4mwEA7AyUnRo1EVfn68Pb+N6UC30ABmbjhBl49WM+bH7Ww/daloH1ClMYxabUyeu3wGZvaEnx8yJuEJUcrJ0spCOC5JhIVwMM2q+zBpRAvWvdCNRzuGsPZQLIO//IdBX27gj91nSU0rZGmDkxM0HwlPb4MuL8LBpfB5OCx5wWjZJiUTopSSpZWFcFwyWU4IB1XNpxwv9WnIM7fX4+etp/nmnxOM+XE7QT7leLB9LYa1rk5Fj0JMHHKrAN1eNpZ3/vsd2PYNbJ4GnlUgtB806g81O0qXCVFqHDgnSysL4ajkbyIhHFwFdxce7BDCfe1q8XfkeWasP857SyL5bMUhhoZX5542NahXpRAjZd7BMHga9PnQtGrdItg1B7Z+DeV8jWS51aNGCzchbCjyrCytLMqmt37fx/4zl616zkbVKvLGnY3z3GfgwIGcPn2a5ORkxo4dy+jRo1m2bBkvv/wyaWlpVK5cmb///purV6/y9NNPs3XrVpRSvPHGGwwZMsSq8RaVJMJCCACcnRQ9GwfSs3Ege6MT+Hr9cX7YdJJZ/5ygVa1KjGhdgz5hVfFwdS7YiT28jeWamw6FG4lw9G/YMgOW/MfoNtHvU3AtVzwXJUQ+ZGllIQpu5syZ+Pr6kpSURKtWrRgwYACjRo1i7dq1hISEcPHiRQDeeecdvL292bNnDwCXLhVxPkoxkERYCJFDkyBvPh3WnFf7NuSX7VHM2Xya8fN28dbv+xnSMpiRbapTN6AQo8Ru5aHhndCgL6z9CFa/D+f3wrDvjRXvhChhGUsry0Q5URblN3JbXCZNmsSvv/4KwOnTp5k+fTqdO3e+2bvX19cXgBUrVjB37tybx1WqVKnkg82HTJYTQuTKz9Od0Z3r8Pf4Lvz4aBs61qvM7H9PcMcnaxk2bSOLdp3hempawU/s5ARdX4SRPxmdJaZ1gSMrrH8BQuQjY2nl0ECZKCeEJVavXs2KFSvYuHEju3btokWLFjRv3tzWYRWaJMJCiHw5OSna163M5JEt2fjS7bwYEcqZhCSembOD9u+v5IOlBzgVV4iexPV7wejVUDEIvr/LmFx3ejMkXrT6NQhhjiytLETBJCQkUKlSJcqXL8+BAwf4999/SU5OZu3atRw/fhzgZmlEjx49mDx58s1jpTRCCFHmVfZ054mudXisc23WHbnAj5tO8tW6Y0xdc5TO9f25t00NuocGWL6Us29tePQv+H0srPvYeAB4+IBfHfCrC1WaQEgnCGwKTgWsURYiD7K0shAFExERwdSpU2nYsCENGjSgbdu2+Pv7M336dAYPHkx6ejoBAQH89ddfvPrqq4wZM4YmTZrg7OzMG2+8weDBg219CVlIIiyEKBQnJ0WX+v50qe/PuYRk5m45xdzNpxk9exvVvD0Y0boGw1pXJ8DLI/+TuVWAwV9B15fgwmG4eBTijkDcUTixAXb/ZOzn4W20XgvpbCTG/qGSGIsikaWVhSgYd3d3li5dava93r17Z3nt6enJt99+WxJhFZokwkKIIgv09uDZO+rzVLe6rIiM4YdNJ/nfX4eY+PdhejSqwt3h1elUr3Leo8RKmUaA6+R878o5OL4Ojq+BE+vg4B/GdhcP8G8AAY1uPao2Bc+A4rlQYVcyllbu2Ug6RgjhqCQRFkJYjYuzExFNAoloEsjxC9f4cdNJftkezdK956hS0Z3BLYMZelswtf09C3Zir8BbLdjAmGB3YoPRcSJmPxxbbfQoBlDO0OJe6PKC0ctYiFzI0spCCEmEhRDFIqRyBV7p24jne4Wy8sB5ft4axbQ1R5my+ijhNSsxoEUQEY0D8fdyL/jJfWpA8xpZtyVehJhI2P+bsZrdrrnGgh2dxkOFyta5KGFXZGllIYQkwkKIYuXm4kREk6pENKlKzOVkFuyIZv62KF5buJc3fttLmxA/+jatSkSTQCp7FiIpzlDeF2p1MB7tn4LVH8CmKbD9W2g3Blo/BhX8rHdhosyTpZWFEJIICyFKTEBFDx7vYnScOHj+Ckt2n2XxnrO8unAvr/+2l3Z1/Bh6W3UimgQWfAW7zHxqwMAvocNYWPkurPmv8agUAkG3mR4tjS4UbuWtd4GiTJGllYUQkggLIUqcUorQwIqEBlZkXI/6HDx/hT92n+W3nWd49qedVPzNhUEtghjWqgaNqhWhftO/AQybDWd3Gwt2RG+DUxth73xTIM4waCo0vds6FybKDK01B85doWejKrYORQhhQ5IICyFsKktSfEd9/j0Wx9wtp5mz+TTfbjxJ02Bv7g6vzp1Nq+FdvpC9Xqs2NR4ZrpyD6O1GYhzYNPfjhN3RWvPX/vNM/PswF6/d4LaapW/JVyFEyZFEWAhRamSsYNe+bmUuXbvBwp3R/LTlNK8u3Mvbi/fTs1EV7rotmE71/Iv2dbZXIIT2MR7CIWROgPeduUwtv/L8b2gzBrUIsnVoQtg1T09Prl69auswciWJsBCiVKpUwY2HOoTwYPta7Dtzmfnboli4M5rFu88S4OXOoJZBDGgWRMOqXiglNZ7CvNwS4AHNq1m++qEQpdXSCXBuj3XPGRgGvT+w7jlLMUmEhRClmlKKJkHeNAny5qU+oaw6EMP8bdHMWHecaWuOUdu/Av3CqtK3aTUaBEobLGGQBFiI4jFhwgSqV6/OmDFjAHjzzTdxcXFh1apVXLp0iZSUFN59910GDBiQ77muXr3KgAEDzB733Xff8fHHH6OUomnTpsyePZvz58/z+OOPc+zYMQCmTJlC+/bti3Q9SmtdpBMUVnh4uN66datNPlsIUfbFXb3O8n3nWbz7DP8eiyNdQ70AT/qEVaVX48BiHSlWSm3TWocXy8mtQCkVAUwEnIEZWusPsr3/ODAGSAOuAqO11vvzOmdZuWebS4Cf7l5PEmBhNyIjI2nYsKHNPn/Hjh08++yzrFmzBoBGjRqxfPlyvL29qVixIhcuXKBt27YcPnwYpVSepRGpqakkJibmOG7//v0MGjSIf/75h8qVK3Px4kV8fX0ZNmwY7dq149lnnyUtLY2rV6/i7e2d47zm/oxyu2/LiLAQokzy83RnZJsajGxTg9gr11m27xyLd51h0srDTPz7MDV8y9OzURV6Ng7ktpqVHKZFllLKGZgM9ACigC1KqUXZEt0ftdZTTfv3Bz4BIko8WCuSEWAhSkaLFi2IiYnhzJkzxMbGUqlSJQIDAxk3bhxr167FycmJ6Ohozp8/T2Bg3suXa615+eWXcxy3cuVKhg4dSuXKxmJIvr6+AKxcuZLvvvsOAGdnZ7NJcEFZlAgXx+iCEEJYi7+XO/e1rcl9bWsSe+U6KyLP8+e+c3y38SQz1h/Hr4IbPRtXoW9YNdrW9rX3xKg1cERrfQxAKTUXGADcvCdrrS9n2r8CYJuvBq1AEmAhSt7QoUOZP38+586dY9iwYfzwww/Exsaybds2XF1dqVWrFsnJyfmep7DHWVO+ibCjji4IIcomfy93RrSuwYjWNbiSnMKaQ7Es23uO33aeYc7m0/hVcCOiSSD9mlajdYivPY4UBwGnM72OAtpk30kpNQYYD7gB3UsmNOuRBFgI2xk2bBijRo3iwoULrFmzhnnz5hEQEICrqyurVq3i5MmTFp0nISHB7HHdu3dn0KBBjB8/Hj8/v5ulEbfffjtTpkzJtzSiICwZEXao0QUhhP3w8nClX9Nq9GtajeSUNFYfjOH33WdZsD2aHzadwt/LnfcGNqFn47y/vrNHWuvJwGSl1EjgVeCB7PsopUYDowFq1KhR4M/45K9DLN1ztoiRmnfteipnEpIlARbCBho3bsyVK1cICgqiatWq3HPPPdx5552EhYURHh5OaGioRefJ7bjGjRvzyiuv0KVLF5ydnWnRogWzZs1i4sSJjB49mq+//hpnZ2emTJlCu3btinQtliTCVhtdKOpNVQghCsvD1ZmIJlWJaFKVxBuprDwQwx+7z1LNp5ytQ7O2aKB6ptfBpm25mQtMMfeG1no6MB2MyXIFDSTAy516VTwLephFlFKMbxDAQEmAhbCJPXtutW2rXLkyGzduNLtfXj2E8zrugQce4IEHsv77vEqVKvz222+FiDZ3VpssZ8noQlFvqkIIYQ3l3VxujhTboS1APaVUCEYCPBwYmXkHpVQ9rfVh08u+wGGKwb1ta3Jv25rFcWohhLAKSxJhq40uCCGEKF5a61Sl1FPAcowJzjO11vuUUm8DW7XWi4CnlFJ3ACnAJcwMXAghhLXs2bOH++67L8s2d3d3Nm3aZKOIbrEkES41owtCCCHyp7VeAizJtu31TD+PLfGghBBWo7UuUytqhoWFsXPnzhL5rIKuj5FvIiyjC0IIIYQQpYOHhwdxcXH4+fmVqWS4JGitiYuLw8PDw+JjLKoRltEFIYQQQgjbCw4OJioqitjYWFuHUip5eHgQHBxs8f6yspwQQgghRBnh6upKSEiIrcOwG9JzRgghhBBCOCRJhIUQQgghhEOSRFgIIYQQQjgkVdA2E1b7YKViAcsWo86qMnDByuGUJvZ+fWD/12jv1wf2f435XV9NrbV/SQVTGsg9O0/2fo32fn1g/9co15fLfdtmiXBhKaW2aq3DbR1HcbH36wP7v0Z7vz6w/2u09+srSY7wZ2nv12jv1wf2f41yfbmT0gghhBBCCOGQJBEWQgghhBAOqSwmwtNtHUAxs/frA/u/Rnu/PrD/a7T36ytJjvBnae/XaO/XB/Z/jXJ9uShzNcJCCCGEEEJYQ1kcERZCCCGEEKLIJBEWQgghhBAOqcwkwkqpCKXUQaXUEaXUBFvHYw1KqZlKqRil1N5M23yVUn8ppQ6bnivZMsaiUEpVV0qtUkrtV0rtU0qNNW23p2v0UEptVkrtMl3jW6btIUqpTabf15+UUm62jrUolFLOSqkdSqnFptf2dn0nlFJ7lFI7lVJbTdvs5vfUVuztvm3v92yw//u23LPt5vqsds8uE4mwUsoZmAz0BhoBI5RSjWwblVXMAiKybZsA/K21rgf8bXpdVqUCz2mtGwFtgTGm/272dI3Xge5a62ZAcyBCKdUW+C/wqda6LnAJeMR2IVrFWCAy02t7uz6Ablrr5pl6UdrT72mJs9P79izs+54N9n/flnu2fVwfWOmeXSYSYaA1cERrfUxrfQOYCwywcUxFprVeC1zMtnkA8K3p52+BgSUZkzVprc9qrbebfr6C8T9lEPZ1jVprfdX00tX00EB3YL5pe5m+RqVUMNAXmGF6rbCj68uD3fye2ojd3bft/Z4N9n/flns2UMavLw+F+h0tK4lwEHA60+so0zZ7VEVrfdb08zmgii2DsRalVC2gBbAJO7tG01dQO4EY4C/gKBCvtU417VLWf18/A14A0k2v/bCv6wPjL8I/lVLblFKjTdvs6vfUBhzlvm23vyf2et+We3aZvz6w4j3bpTiiE9ahtdZKqTLf304p5Qn8Ajyrtb5s/OPUYA/XqLVOA5orpXyAX4FQ20ZkPUqpfkCM1nqbUqqrjcMpTh211tFKqQDgL6XUgcxv2sPvqSh+9vR7Ys/3bbln2wWr3bPLyohwNFA90+tg0zZ7dF4pVRXA9Bxj43iKRCnlinEz/UFrvcC02a6uMYPWOh5YBbQDfJRSGf/QLMu/rx2A/kqpExhfbXcHJmI/1weA1jra9ByD8Rdja+z097QEOcp92+5+Txzlvi337LLLmvfsspIIbwHqmWY9ugHDgUU2jqm4LAIeMP38APCbDWMpElNd0tdApNb6k0xv2dM1+ptGFVBKlQN6YNTUrQLuMu1WZq9Ra/2S1jpYa10L4/+7lVrre7CT6wNQSlVQSnll/Az0BPZiR7+nNuIo9227+j2x9/u23LOBMnx9UAz3bK11mXgAfYBDGLU8r9g6Hitd0xzgLJCCUbPzCEYtz9/AYWAF4GvrOItwfR0x6nh2AztNjz52do1NgR2ma9wLvG7aXhvYDBwBfgbcbR2rFa61K7DY3q7PdC27TI99GfcXe/o9teGfrV3dt+39nm26Rru+b8s9u+xfn7Xv2bLEshBCCCGEcEhlpTRCCCGEEEIIq5JEWAghhBBCOCRJhIUQQgghhEOSRFgIIYQQQjgkSYSFEEIIIYRDkkRYCCGEEEI4JEmEhRBCCCGEQ/p/+9RFjopZgUIAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "fig, axes = plt.subplots(1,2,figsize=(12, 4))\n", "for ax, measure in zip(axes, ['loss', 'acc']):\n", " ax.plot(hist.history[measure], label=measure)\n", " ax.plot(hist.history['val_' + measure], label='val_' + measure)\n", " ax.legend()\n", "pass" ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [], "source": [ "plot_model(model)" ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [], "source": [ "from IPython.display import Image" ] }, { "cell_type": "code", "execution_count": 22, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAJMAAAFgCAYAAAC/lEsWAAAABmJLR0QA/wD/AP+gvaeTAAAgAElEQVR4nO3de0yUV/4G8GcYwKpAbStVjFIcvFBvKDjiddWldGvQ4lYRxFu9Jkur2y7qum7XGtd0ozW19S6kybbdagGFaKOxrQFRC1GRSrey2lihgtJ4F4Yijszz+8Pw/hiZGQY4MEz9fpJJnPOe+b5nDo/znrm9oyNJCNFy6R6uHoH47ZAwCWUkTEIZCZNQxrOlBfLy8vDBBx+oGItwofT09BbXaPEjU2lpKfbt29figQjXKCsrU/b3a/EjUx0VyRZtLy0tDXFxcUpqyZpJKCNhEspImIQyEiahjIRJKCNhEspImIQyEiahjIRJKCNhEspImIQyEiahjIRJKCNhEsoo+whKcxw6dAgVFRXa9dLSUrz55pvo1KkTzpw5g0uXLtm83ciRI9G7d28AQGVlJfbs2YPi4mL06dMHCQkJ6NSpk9ZXVR0AuHXrFg4cOIArV65gyJAhePnll+Hj42PV5+7du/j4449x5coVREdHIzIyEnq9vsl9nNmXM2NuU2yh1NRUNqfM//73P+p0OgLQLvHx8SRJi8XC4OBgq231L2fPniVJXrhwgd27d2ffvn3p7e1NAAwODmZ5ebnSOiT53XffcdCgQczLy2NVVRU3bNjAIUOG8Nq1a1qfW7duMTg4mHPmzOHvf/97enh4cMSIEVb325k+zuzLmTE7o7l/PxvSXBamxYsXMzs7m1euXNEu1dXVJMmvv/6ay5YtY3FxMWtqarTL119/zaCgIK3GpEmTWFhYSJK8fv06Fy1aRABcsGCB0jq1tbUMDQ3lypUrre7DiBEjGBUVpV3fuXMnb926pV1ft24dAfDkyZNO93F2X42N2VluH6by8nJGRESwtLTU5vbc3FzW1tY2aH/jjTeYlJREkszPz+d//vMfq+3Xrl2jh4cHQ0JClNb59ttvCYB79+616vfmm28SAPPz81lTU8PLly9bbS8pKSEAfv/99yTpVB9n9uXMmJ2lMkwuWYBv3boVp06dQq9evWAwGPDvf/8brPfF4lGjRsHDw3poFosFGRkZeO211wAAQUFBSEhIsOoTEBCA8PBwPPPMM0rrXLx4EQCsxggARqMRAHDy5El4e3tr668633//PSZPnozBgwcDgFN9nNmXM2N2BZeEafz48VixYgXGjh2LsrIyzJ8/Hy+//DJqa2vt3ubbb7+FTqfDqFGjAADPPfccdDpdg36lpaWYNGmS0jodO3YEAOTn51v1CQ4OBgBcuXLFqp0k0tLSsGrVKuzcudPmOOz1cWZfzb3vra6lj20tfZg8d+4cQ0JCCID/+te/7PZbunQp33jjDYe1cnJy2LNnT1ZWViqtc+XKFXp7ezM8PJwWi0Xrd+jQIQLgli1btDaTycTFixezU6dOBMAuXbrw9OnTVvUd9WnKvpp6321x+zXT46qqqtizZ08OGTLE5naLxcIePXowKyvLbo2HDx9y/PjxLCgosNunJXU2btxIAJw3bx4PHTrETZs2cfjw4QTAnJycBnVqa2u5efNm6vV6hoeH29yXvT5N3Zcz992e31yYyEeL4o4dO9rcduLECXbt2pUPHz60e/u33nqLBw4ccLiPltY5duwYV69ezb/97W8sLCzk9OnT+fTTT7OiosJuvdjYWHp4ePD+/ftN6tOUfTlz3+35TYZp69atDA0Ntbntz3/+MxcuXGj3trt37+Ynn3zS6D5U1SHJy5cv09vbmx999JHDfsnJyezatWuL+jjaV1PGbIvbP5uzJTMzEzExMQ3aSWLfvn2YNm2a3duRxNy5c63ac3JyWqUOADx48ABxcXHo378/EhMTHd6vH374AVOmTGl2H0f7asqY20Kbv53y448/YseOHZg3bx6GDRsGADh//jyqqqrwzjvvNOifl5cHk8mEyMjIBtuOHj2KDRs2YPbs2di2bRsAoLa2FkVFRRg0aBDGjx+vvE5VVRUSExPRu3dvbN26FZ6ej6awuroaH3zwAWJiYjBo0CAAj94S+e677/Dll1863ac+e/tq6pjbTEsf25r6MHn27Fk+/fTTBMCJEyfyr3/9Kzds2MBff/3VZv+33nqLs2fPtlmnc+fONt8meeqpp6xeZVZR5+bNm/z44485evRoZmRkNKhjMpk4bNgw6nQ6Go1G/uMf/+BHH31k9ezKmT7O7Kup990Rt18z3b9/nz/++CPLysoa7Xv58mXevHmzucNTViczM5M//fRTo/3u3LnDqqqqFvVxdl8qqAyTSz410KFDB/Tt29epvo+/YtxcLa0zdepUp/p16dKlxX2c3Vd7024W4ML9SZiEMhImoYyESSgjYRLKSJiEMhImoYyESSgjYRLKSJiEMhImoYyESSgjYRLKSJiEMhImoYyESSij7MNxsbGxqkqJNlRWVqasVosfmXr16oXp06erGEu7dOPGDRw/ftzVw2g1PXv2VPb305HyU/SO1P0em0xTo+Sn6IU6EiahjIRJKCNhEspImIQyEiahjIRJKCNhEspImIQyEiahjIRJKCNhEspImIQyEiahjIRJKCNhEspImIQyEiahjIRJKCNhEspImIQyEiahjIRJKCNhEspImIQyEiahjIRJKCNhEspImIQyEiahjIRJKCNhEspImIQyLvnB5/aqrKwM8+bNQ21trdZ28+ZNeHp6YsKECVZ9+/fvj927d7fxCNs3CVM9PXv2RElJCS5fvtxgW05OjtX1cePGtdWw3IYc5h4zd+5ceHl5NdovPj6+DUbjXiRMj5k1axbMZrPDPgMGDMDAgQPbaETuQ8L0mD59+mDIkCHQ6XQ2t3t5eWHevHltPCr3IGGyYe7cudDr9Ta3PXz4EDNmzGjjEbkHCZMNM2fOhMViadCu0+kQERGBoKCgth+UG5Aw2dCjRw+MHj0aHh7W06PX6zF37lwXjar9kzDZMWfOnAZtJDFt2jQXjMY9SJjsiI2NtXpk0uv1eOmll/D888+7cFTtm4TJjmeeeQYvv/yythAnidmzZ7t4VO2bhMmB2bNnawtxT09PvPrqqy4eUfsmYXLg1VdfRYcOHbR/+/n5uXhE7VuD9+bKysqQm5vrirG0S2FhYcjNzUXv3r2Rlpbm6uG0GzZfa+NjUlNTCUAucnF4sSHN7mGOpFxIPHjwACtXrnT5ONrLJTU11V5kZM3UGC8vL6xdu9bVw3ALEiYndOzY0dVDcAsSJqGMhEkoI2ESykiYhDISJqGMhEkoI2ESykiYhDISJqGMhEkoI2ESykiYhDKtcuIKk8mE7OxsnDx5Ehs2bGiNXbS6X375BRcuXGhw9hNnHD9+HFevXrVq8/Lygr+/P3r06IG+ffsqGmX70iqPTEeOHMGyZcvwxRdftEb5VnXjxg0sX74cBoMBmZmZzaoxZMgQ/PTTT0hISMDrr7+OiooK3LhxA19++SXi4uLQu3dvvPPOO42e08Dt8DF1n7RsqRkzZtBgMLS4Tls7ffo0CwsLCYDLli1rdp3S0lIC4IsvvmjVbrFYmJ6eTj8/P0ZFRbGioqKlQ25TDvKR1mrnZ/Lw8GjwjVh3YDQa8eDBgxbXsfflA51Oh+nTp6O2thbx8fEYN24cTp8+DW9v7xbv09WUhen27dvYt28fSkpKMHz4cJBscCaRa9eu4ciRIygrK8OYMWMQGRmpbSstLUVGRgaWLl2KoqIiHDhwAIGBgZg1a5YWSpLIycnBuXPnoNfrERISgqioKKfqq3Lz5k2kpKRgwYIF6NatW7PrxMXF4dNPP8Xhw4dx+vRpjB07FoCbz1ETHsbsunDhAo1GI3Nzc2k2m7l792526NCB/fr10/pkZWVx8eLFLCgoYFpaGn18fJiYmEiSPHjwIP39/QmAmzdv5vz58zl58mQC4HvvvafVWL16NVNSUkiSZ86c4YgRI5yq31Q1NTV2D3MpKSkEwC1btjisce/ePZuHufrWrVtndR/dYY4cHeaUhCkiIoIrVqzQrlssFhoMBi1MlZWVNBgMNJlMWp+FCxcSAPPy8kiSq1atIgAePXpU6xMWFsbw8HCtZteuXZmdna1tX79+vdP1m8JRmEwmE/fs2dPoWseZMGVkZBAAJ02a5DZz1KprpqysLJw6dQrvvvuu1qbT6WA0GnHu3DkAwN69e1FdXY2VK1dqfcrLyxEcHIxLly5h5MiR2uesQ0JCtD4DBgzAV199pdXs378/4uLikJycjJiYGCxfvtzp+qp07twZM2fOVFLLZDJpNX8Lc9TiMBUWFgIABg0aZNVef710/vx5BAQEYPv27U2qrdfrQVK7vm3bNsTGxmLq1KmIjIzE559/jm7dujW7vqsVFBQAACIiIn4Tc9Tip1sVFRUAgFOnTjXYVhcovV6Pixcvtvh1laFDh6KgoACJiYk4duwYwsLCcPv2bWX12xJJnDhxAnq9HlFRUb+JOWpxmAYPHgzg0eHOntDQUFRVVWHXrl1W7Xfv3sWOHTuc2k9NTQ0+++wz+Pr6Yvv27Th06BDKy8uRkZGhpH5be/vtt3H27Fm8//77CA0N/W3MURMWWDaZzWaGhITQx8eHOTk5JMmrV68yICCAPj4+LCwspMlkYq9evejt7c2NGzeyqKiIqampjI2N1RaySUlJBMDLly9rtaOjo+nr60uLxcLq6mqOHj2aFouF5KPFpr+/PzMzM3n//v1G6zfFL7/8QgBcsmRJg235+fk0Go1Wi1xb6l74DAoKsmovLi5mYmIidTodly5dqrU7cx/awxy1+rO54uJiGo1GAqDBYGBCQgKnTJnCsWPHcufOnayurmZRURH79eunfVd94MCBLCgoIEkeO3aMBoOBALho0SKWl5dz79699PPzIwCuXbuWlZWVDAgIYHx8PNPT07lp0yauWbNGG4Oj+k1x+PBhxsXFEQCff/55pqSksLy8XNu+f/9+6nQ67em3LQcPHuSECRO0sYwaNYpRUVGMjo5mTEwMk5KSeObMmQa3c4c5avUw1bl+/br21LOystJmn5KSEv7888/Nqm82m1lTU+Pw9i2p76x79+61av32PEdt9naKv7+/9m8fHx+bfV544YVm1/f0fDTcwMBAu31s1U9MTGy09pIlSzB06FCnxtHa52lyxRyp8ET8dsrEiRMb7VP/P4JoniciTLGxsa4ewhPB/d7WF+2WhEkoI2ESykiYhDISJqGMhEkoI2ESykiYhDISJqGMhEkoI2ESykiYhDISJqGMhEkoY/cjKPLbasKWvLw8u9vshikuLq5VBiN+u3RkvW/wiQbS0tIQFxcHmaZGpcuaSSgjYRLKSJiEMhImoYyESSgjYRLKSJiEMhImoYyESSgjYRLKSJiEMhImoYyESSgjYRLKSJiEMhImoYyESSgjYRLKSJiEMhImoYyESSgjYRLKSJiEMhImoYyESSgjYRLKSJiEMhImoYyESSgjYRLKSJiEMhImoYyESSjzRPysqrNu3LiBzMxMq7b8/HwAQHJyslW7j48PEhIS2mxs7kBOQ1hPTU0N/P39UVVVBb1eDwAgCZLw8Pj/B3Gz2Yy5c+fik08+cdVQ2yM5DWF9HTp0QGxsLDw9PWE2m2E2m/Hw4UPU1tZq181mMwDIo5INEqbHJCQk4MGDBw77dOnSBZGRkW00IvchYXrMxIkT4e/vb3e7l5cXZs+eDU9PWW4+TsL0GA8PDyQkJMDb29vmdrPZjJkzZ7bxqNyDhMmGmTNn2j3UBQQEYNSoUW08IvcgYbIhIiICL7zwQoN2Ly8vzJs3DzqdzgWjav8kTHbMmTMHXl5eVm1yiHNMwmTHrFmztJcB6vTp0wdDhgxx0YjaPwmTHSEhIRgwYIB2SPPy8sL8+fNdPKr2TcLkwNy5c7VXws1mM2bMmOHiEbVvEiYH4uPjUVtbCwAIDw9Hnz59XDyi9k3C5MALL7wAo9EI4NGjlGgEH5OamkoAcpGLw4sNaXbfE0hNTbW36YlSUVGBHTt2YNWqVa4eSruQl5eHDz/80OY2u2GSxeb/Gz9+PPr27evqYbQb9sIkayYnSJCcI2ESykiYhDISJqGMhEkoI2ESykiYhDISJqGMhEkoI2ESykiYhDISJqGMhEkoI2ESyrTKd5xNJhOys7Nx8uRJbNiwoTV20WoqKyuxZ88eFBcXo0+fPkhISECnTp2aVOP48eO4evWqVZuXlxf8/f3Ro0eP3+6nEOx90rIl0tPTGRQUxMDAwBbVaWsXLlxg9+7d2bdvX3p7exMAg4ODWV5e3qQ6d+7c4T//+U8CoLe3N3ft2sUdO3YwKSmJw4YNY1BQEP/+97/zwYMHrXRPWo+DfKS1SphIcsaMGTQYDC2u05YmTZrEwsJCkuT169e5aNEiAuCCBQuaXKu0tJQA+OKLL1q1WywWpqen08/Pj1FRUayoqFAy9rbiKEyttmby8PCwOkFWe3f27FnMmjVL+5Klv78/1q1bBw8PD+Tm5ja5np+fn812nU6H6dOnIzk5Gd988w3GjRvX6Cl83IWyNdPt27exb98+lJSUYPjw4SDZ4Dv5165dw5EjR1BWVoYxY8ZYneOotLQUGRkZWLp0KYqKinDgwAEEBgZi1qxZWihJIicnB+fOnYNer0dISAiioqKcqt+YoKAghIWFWbUFBAQgPDzc6vQ5N2/eREpKChYsWIBu3bo1aY7qi4uLw6efforDhw/j9OnTGDt2bKP3wdVz1KgmPIzZdeHCBRqNRubm5tJsNnP37t3s0KED+/Xrp/XJysri4sWLWVBQwLS0NPr4+DAxMZEkefDgQfr7+xMAN2/ezPnz53Py5MkEwPfee0+rsXr1aqakpJAkz5w5wxEjRjhVvyW6d+/OdevWaddTUlIIgFu2bHF4u3v37tk8zNW3bt06q/voDnPU6mumiIgIrlixQrtusVhoMBi0MFVWVtJgMNBkMml9Fi5cSADMy8sjSa5atYoAePToUa1PWFgYw8PDtZpdu3Zldna2tn39+vVO12+OnJwc9uzZk5WVlVqbyWTinj17Gl3rOBOmjIwMAuCkSZPcZo4chanFh7msrCycOnUK7777rtam0+lgNBpx7tw5AMDevXtRXV2NlStXan3Ky8sRHByMS5cuYeTIkejYsSOAR9/xrzNgwAB89dVXWs3+/fsjLi4OycnJiImJwfLly52u31S1tbVYs2YNDh48CB8fH629c+fOys6EYjKZtJruOEePa3GYCgsLAQCDBg2yaq+/Xjp//jwCAgKwffv2JtXW6/VgvZMBb9u2DbGxsZg6dSoiIyPx+eefo1u3bs2u78jy5cvxl7/8BcOGDVNW83EFBQUAHp0Pyh3n6HEtfrpVUVEBADh16lSDbXWB0uv1uHjxYoNT1DTV0KFDUVBQgMTERBw7dgxhYWG4ffu2svp1kpOTMWzYMLz66qtK6tlCEidOnIBer0dUVJTbzZEtLQ7T4MGDATw63NkTGhqKqqoq7Nq1y6r97t272LFjh1P7qampwWeffQZfX19s374dhw4dQnl5OTIyMpTUr5OZmQmSDc4tkJOT06Q6jXn77bdx9uxZvP/++wgNDXWrObKrCQssm8xmM0NCQujj48OcnByS5NWrVxkQEEAfHx8WFhbSZDKxV69e9Pb25saNG1lUVMTU1FTGxsZqC9mkpCQC4OXLl7Xa0dHR9PX1pcViYXV1NUePHk2LxULy0WLT39+fmZmZvH//fqP1nfHNN98wIiKCW7du1S4ffvghlyxZoj17y8/Pp9FotFrk2lJYWEgADAoKsmovLi5mYmIidTodly5dqrU7cx/awxy1+rO54uJiGo1GAqDBYGBCQgKnTJnCsWPHcufOnayurmZRURH79eunnfhg4MCBLCgoIEkeO3aMBoOBALho0SKWl5dz79699PPzIwCuXbuWlZWVDAgIYHx8PNPT07lp0yauWbNGG4Oj+s44e/YsO3fubPMkDU899RRv3bpFkty/fz91Op329NuWgwcPcsKECdrtR40axaioKEZHRzMmJoZJSUk8c+ZMg9u19zki2/DtlOvXr2tPPes/na6vpKSEP//8c7Pqm81m1tTUOLx9S+o76969e61avz3PUau+NFBf/ZOx1386XZ+ts9g6q+6V6MDAQLt9bNVPTExstPaSJUswdOhQp8Zh760SVVwxRyo8EafZnzhxYqN9HP0qgXDOExGm2NhYVw/hieA+b+uLdk/CJJSRMAllJExCGQmTUEbCJJSRMAllJExCGQmTUEbCJJSRMAllJExCGQmTUEbCJJSx+xEU+bl10VQNwjR69Gj5rbl66n5fTeakcTqy3jf4RANpaWmIi4uDTFOj0mXNJJSRMAllJExCGQmTUEbCJJSRMAllJExCGQmTUEbCJJSRMAllJExCGQmTUEbCJJSRMAllJExCGQmTUEbCJJSRMAllJExCGQmTUEbCJJSRMAllJExCGQmTUEbCJJSRMAllJExCGQmTUEbCJJSRMAllJExCGQmTUOaJ+CVMZ5nNZphMJqu2qqoqAMCdO3es2nU6Hbp06dJmY3MHEqZ6bt26hZ49e6K2trbBtmeffdbq+oQJE5Cdnd1WQ3MLcpirp3v37vjd734HDw/H06LT6TBz5sw2GpX7kDA9Zs6cOY2eadjDwwPTpk1roxG5DwnTY6ZNmwa9Xm93u16vxyuvvILnnnuuDUflHiRMj/Hz88Mrr7wCT0/by0mSmD17dhuPyj1ImGyYPXu2zUU4AHh7e2Py5MltPCL3IGGyYcqUKejUqVODdk9PT/zxj3+Ej4+PC0bV/kmYbHjqqafw2muvwcvLy6r94cOHmDVrlotG1f5JmOxISEiA2Wy2avPz80NUVJSLRtT+SZjseOmll6xeqPTy8kJ8fDy8vb1dOKr2TcJkh6enJ+Lj47VDndlsRkJCgotH1b5JmByYOXOmdqjr1q0bxo0b5+IRtW8SJgfGjBmDHj16AHj0ynhjb7M86Rq8MpeXl4cPPvjAFWNpl3x9fQEA3333HWJjY108mvYjPT29QVuD/2qlpaXYt29fmwzIHQQGBsLX1xfPPPOMq4fSLpSVldnNh92PoNhK3pMqLS0NM2bMcPUw2oW6H3O0RRYBTpAgOUfCJJSRMAllJExCGQmTUEbCJJSRMAllJExCGQmTUEbCJJSRMAllJExCGQmTUEbCJJRplbOgmEwmZGdn4+TJk9iwYUNr7KLV3L17Fx9//DGuXLmC6OhoREZGOvy6uC3Hjx/H1atXrdq8vLzg7++PHj16oG/fviqH3G60yiPTkSNHsGzZMnzxxRetUb7V3L59G8OHD0dhYSF++OEHTJo0CaNHj25ynSFDhuCnn35CQkICXn/9dVRUVODGjRv48ssvERcXh969e+Odd95p8FUqt8fHpKam0kZzk82YMYMGg6HFddrSzp07eevWLe36unXrCIAnT55scq3S0lIC4IsvvmjVbrFYmJ6eTj8/P0ZFRbGioqLF425LDvKR1mprJg8PD7f6AP6DBw/whz/8weq7cnPnzgXw6MuXTWXvNjqdDtOnT0dycjK++eYbjBs3Dg8ePGjeoNsZZWum27dvY9++fSgpKcHw4cNBssF5jq5du4YjR46grKwMY8aMQWRkpLattLQUGRkZWLp0KYqKinDgwAEEBgZi1qxZWihJIicnB+fOnYNer0dISIjVN2wd1W+Mt7c3evfubdX2/fffY/LkyRg8eLDWdvPmTaSkpGDBggXo1q1bk+aovri4OHz66ac4fPgwTp8+jbFjxzZ6H1w9R41qwsOYXRcuXKDRaGRubi7NZjN3797NDh06sF+/flqfrKwsLl68mAUFBUxLS6OPjw8TExNJkgcPHqS/vz8BcPPmzZw/fz4nT55MAHzvvfe0GqtXr2ZKSgpJ8syZMxwxYoRT9ZvKYrEwNTWVAwYMYGlpqdW2lJQUAuCWLVsc1rh3757Nw1x9dYfRuvvoDnPk6DCnJEwRERFcsWKFdt1isdBgMGhhqqyspMFgoMlk0vosXLiQAJiXl0eSXLVqFQHw6NGjWp+wsDCGh4drNbt27crs7Gxt+/r1652u7yyTycTFixezU6dOBMAuXbrw9OnTVtv37NnT6FrHmTBlZGQQACdNmuQ2c+QoTC0+zGVlZeHUqVN49913tTadTgej0Yhz584BAPbu3Yvq6mqsXLlS61NeXo7g4GBcunQJI0eORMeOHQEAISEhWp8BAwbgq6++0mr2798fcXFxSE5ORkxMDJYvX+50fWd17twZycnJ2LVrF7Zs2YLly5fjT3/6E/Lz87Xtqs5nWXdm386dO7vVHNnT4jAVFhYCAAYNGmTVXn+9dP78eQQEBGD79u1Nqq3X60FSu75t2zbExsZi6tSpiIyMxOeff45u3bo1u74jHh4eeOutt5Cbm4v9+/ejpqYGHTp0UFYfAAoKCgAAERERbjlHj2vx062KigoAwKlTpxpsqwuUXq/HxYsXW/y6ytChQ1FQUIDExEQcO3YMYWFhuH37trL6tkRFReHZZ59VHiSSOHHiBPR6PaKiotx6juq0OEx1z3SysrLs9gkNDUVVVRV27dpl1X737l3s2LHDqf3U1NTgs88+g6+vL7Zv345Dhw6hvLwcGRkZSurb88MPP2DKlCktqmHL22+/jbNnz+L9999HaGioW8+RpgkLLJvMZjNDQkLo4+PDnJwckuTVq1cZEBBAHx8fFhYW0mQysVevXvT29ubGjRtZVFTE1NRUxsbGagvZpKQkAuDly5e12tHR0fT19aXFYmF1dTVHjx5Ni8VC8tFi09/fn5mZmbx//36j9Rvz66+/cv369fzvf/+rtd28eZPjxo3j3bt3tbb8/HwajUarRa4thYWFBMCgoCCr9uLiYiYmJlKn03Hp0qVauzP3wdVzRLbBs7ni4mIajUYCoMFgYEJCAqdMmcKxY8dy586drK6uZlFREfv160cABMCBAweyoKCAJHns2DEaDAYC4KJFi1heXs69e/fSz8+PALh27VpWVlYyICCA8fHxTE9P56ZNmxQcRAkAAAFSSURBVLhmzRptDI7qO8NkMnHYsGHU6XQ0Go38xz/+wY8++oiVlZVW/fbv30+dTqc9/bbl4MGDnDBhgjaWUaNGMSoqitHR0YyJiWFSUhLPnDnT4HbtfY7INghTnevXr2tPPR//I9QpKSnhzz//3Kz6ZrOZNTU1Dm/fkvokeefOHVZVVTnsc+/evWbXd0Z7nqNWfWmgPn9/f+3f9s5I+8ILLzS7ft25uQMDA+32sVU/MTGx0dpLlizB0KFDnfpxnea8vdIUrpgjFZ6IH+KZOHFio33q/0cQzfNEhElO0tU23OdtfdHuSZiEMhImoYyESSgjYRLKSJiEMhImoYyESSgjYRLKSJiEMhImoYyESSgjYRLKSJiEMnY/giIf2xC2lJWV2d3W4JGpV69emD59eqsOSLivnj172s2Hjqz3DT4hmi9d1kxCGQmTUEbCJJSRMAll/g8mzskY2I5WJgAAAABJRU5ErkJggg==\n", "text/plain": [ "" ] }, "execution_count": 22, "metadata": {}, "output_type": "execute_result" } ], "source": [ "Image('model.png')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Functional API\n", "\n", "The functional API provides flexibility, allowing you to build models with multiple inputs, multiple outputs, or branch and merge architectures." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Set random seed for reproducibility" ] }, { "cell_type": "code", "execution_count": 23, "metadata": {}, "outputs": [], "source": [ "np.random.seed(0)\n", "tf.random.set_seed(0)" ] }, { "cell_type": "code", "execution_count": 24, "metadata": {}, "outputs": [], "source": [ "input = Input(shape=(4,))\n", "x = Dense(8)(input)\n", "x = Dense(4, activation='relu')(x)\n", "output = Dense(3, activation='softmax')(x)\n", "model = Model(inputs=[input], outputs=[output])" ] }, { "cell_type": "code", "execution_count": 25, "metadata": {}, "outputs": [], "source": [ "model.compile(\n", " optimizer='adam',\n", " loss='sparse_categorical_crossentropy',\n", " metrics=['accuracy']\n", ")" ] }, { "cell_type": "code", "execution_count": 26, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "_________________________________________________________________\n", "Layer (type) Output Shape Param # \n", "=================================================================\n", "input_1 (InputLayer) (None, 4) 0 \n", "_________________________________________________________________\n", "dense_4 (Dense) (None, 8) 40 \n", "_________________________________________________________________\n", "dense_5 (Dense) (None, 4) 36 \n", "_________________________________________________________________\n", "dense_6 (Dense) (None, 3) 15 \n", "=================================================================\n", "Total params: 91\n", "Trainable params: 91\n", "Non-trainable params: 0\n", "_________________________________________________________________\n" ] } ], "source": [ "model.summary()" ] }, { "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [], "source": [ "hist = model.fit(\n", " X_train,\n", " y_train,\n", " validation_split = 0.3,\n", " epochs=50, \n", " verbose=0)" ] }, { "cell_type": "code", "execution_count": 28, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAsIAAAD4CAYAAADmbIA7AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAABcfklEQVR4nO3dd1xV9f/A8deHrYIgCKLgwIkDV7i3peLImTnaQxtWpn0r2/tX3+pbaZkjM8tKMzMzc5S5zdxb3BMcIAoOQBmf3x/noowLXODChXvfz8fjPi733HPOfR+j49vPfX/eH6W1RgghhBBCCEfjZOsAhBBCCCGEsAVJhIUQQgghhEOSRFgIIYQQQjgkSYSFEEIIIYRDkkRYCCGEEEI4JBdbfXDlypV1rVq1bPXxQghRaNu2bbugtfa3dRwlSe7ZQoiyLLf7ts0S4Vq1arF161ZbfbwQQhSaUuqkrWMoaXLPFkKUZbndt6U0QgghhBBCOCRJhIUQQgghhEOSRFgIIYQQQjgkm9UICyFsJyUlhaioKJKTk20dSqnm4eFBcHAwrq6utg5FCCFEMZBEWAgHFBUVhZeXF7Vq1UIpZetwSiWtNXFxcURFRRESEmLrcIQQQhQDKY0QwgElJyfj5+cnSXAelFL4+fnJqLkQQtgxSYSFcFCSBOdP/oyEEMK+lanSiFkbjlPTrwJdG/jLX1BCCCGEcDhXklP4Y/dZQqtWpHl1H1uHk8P2U5dYfTAWtM7xXosalQqdwyWnpDF38ylq+3vSub711jMqM4lwalo6czaf5uD5K7Sr7cdLfUJpGuxj67CEEIXk6enJ1atXbR2GEEKUCVeSU5i14QQz1h8nISkFgG4N/Bl7R/1SkRBvO3mJiX8fZu2hWACy57oZeXHTYG+evaMe3RoEWJQQZyTAX64+SsyV69zbtoZjJsIuzk78/nRH5mw+xcS/D9P/iw30b1aN53s1oLpveVuHJ4QQwtGkpcKaD6D5PeArEypF8cieAN/RMIDHutRhy4mLfLX2GAMnb7BpQpw5Afat4MZLvUO5r11NyrtlTTFT0tL5dXs0n686zMOztuabEGdPgFuH+DJxeAva1fGzavxKmxm6zrGTUhHARMAZmKG1/iDb+zWAbwEf0z4TtNZL8jpneHi4LuxynVeSU5i25hgz1h8jLV1zX9tajL2jHt7lpMWREJaIjIykYcOGNo0hY0RYa80LL7zA0qVLUUrx6quvMmzYMM6ePcuwYcO4fPkyqampTJkyhfbt2/PII4+wdetWlFI8/PDDjBs3rljjNPdnpZTaprUOL9YPLmWKcs+2W2d2wPSu4FsHHl0B5X0B2HcmgeV7zzGuR30p4xNZHL9wja/XHyPm8nWL9tfA5uMXbybAY2+vT1iw9833r15P5buNJ/hq7TEuJabQqlYlKpV3K6boc7p47QZbT17Ct4Ibj3WubTYBzi5zQnz6YhINq1akeqVyOfbbeTr+ZgI87o76RU6Ac7tv5zsirJRyBiYDPYAoYItSapHWen+m3V4F5mmtpyilGgFLgFpFijgPXh6u/KdXA+5tW5PPVhxi1j/HWbz7DG/1b0xEk0C58QhRAG/9vo/9Zy5b9ZyNqlXkjTsbW7TvggUL2LlzJ7t27eLChQu0atWKzp078+OPP9KrVy9eeeUV0tLSSExMZOfOnURHR7N3714A4uPjrRq3EAUSd9R4vngM5t4D9y8EF3d+2HSKHzedomfjQJoEeed5CuEYjl+4xucrD7NwRzRuLk7U8qtg8bHt6/jxZNe6WRLgDJ7uLjzZtS73t6vFdxtPsGTPWa5cTLRm6HlycVa5jgDnxtXZibtbVWdQyyB+3R7NnC2nOGUm5kbVKjKxcx2rjwBnZ0nUrYEjWutjAEqpucAAIHMirIGKpp+9gTPWDDI3gd4efDCkKfe0qcmEBbt54oft9GhUhXcGNCHQ26MkQhBCFNH69esZMWIEzs7OVKlShS5durBlyxZatWrFww8/TEpKCgMHDqR58+bUrl2bY8eO8fTTT9O3b1969uxp6/CFI7t43Hju/zksegoWPQ2DprE3OgGAP/aclUTYwWVPgB/pGMLoznXw93K36udkJMRPdq1r1fMWp4yE+O5W1W0ahyWJcBBwOtPrKKBNtn3eBP5USj0NVADuMHcipdRoYDRAjRo1ChprrsKCvfltTAe+Xn+cT1cc4o5P1vBiRAPuaVMTJycZHRYiL5aO3Ja0zp07s3btWv744w8efPBBxo8fz/3338+uXbtYvnw5U6dOZd68ecycOdPWoQpHdfEoVAyClvfBlXOw6l1SfUI4cLY5AH/sPssLvRrIt5QFsPpgDO/+EUlKWrqtQykyrSHqUmKxJsCi6Kw1WW4EMEtr/T+lVDtgtlKqidY6y2+y1no6MB2MejMrfTZgTKZ7rEsdIpoE8sqve3ntt30s2BHN6/0a0aJGJWt+lBDCijp16sS0adN44IEHuHjxImvXruWjjz7i5MmTBAcHM2rUKK5fv8727dvp06cPbm5uDBkyhAYNGnDvvffaOnzhyOKOgm9t4+fO/4GLx3BZ+wG99ZNcrDeQdYcvsO/MZRkVtlDc1es8N28XXh4utCgFXRCs4c5mVXmwfYgkwKWYJYlwNJB53DrYtC2zR4AIAK31RqWUB1AZiLFGkAVR068Csx9pzS/bo/lg6QEGffkP/ZtV44WIBgRXku4SQpQ2gwYNYuPGjTRr1gylFB9++CGBgYF8++23fPTRR7i6uuLp6cl3331HdHQ0Dz30EOnpxr+x33//fRtHLxzaxWMQ2tf4WSm4cyLnTx/mw7jpnGrajYijquTKI9JSID3Vwp0VuBZT+aDWOftm5SYtFdJTbr58Z+FObiQnMv2hdtSv4lU88ZU0Z3dwKsG1y1Kvgy7B0XTlDC4lNzmvOOTbNUIp5QIcAm7HSIC3ACO11vsy7bMU+ElrPUsp1RD4GwjSeZy8JGYgX72eyrQ1R5m+9hgaeKRjCE92rYOXh3SXEI6tNHSNKCuka4RBukZkk5wAH9SAO96Cjs/e3PzOzxu4b98j1KyQxgOVvuPExWTWPN+1eMsjYiJhWmdIu2H5Mbe/AZ3GWzeOzV/B+s/gkeXgHZz3vldjYXIrSLpk3RhKG/+G8OBiqFC5+D9r03RYNgF0WvF/VgYnF+jzEYQ/XHKfWUiF7hqhtU5VSj0FLMdojTZTa71PKfU2sFVrvQh4DvhKKTUOY+Lcg3klwYW29xfwqwdVm1q0u6e7C8/1bMCI1jX4ePlBpqw+yrwtp3muZwOGtaqOs9QPCyGEKIyLx4xnvzpZNm85r6npPYD7E6YysJ0n4xdfLP7yiFMbjSS48/PgZkE3gsjfYeNkaDcGXKz0lf3BpbD0BWM0cssMuOPNvPffNstIgru+xLU0Z75ef5yK5Vy5r21N+/m7OfUGrP8E5o6E+xcV3yg8wIElxp9/7a5Qu0vxfU52R1fCH/8B7xpQz+z0sFLPohphU0/gJdm2vZ7p5/1AB+uGlk1aKqx4C+JPGc3Lu78KFatadGg1n3J8Mqw5D3aoxbuLI3n51z3M3XKKt/o3lvphIYQQBZfROs33ViJ8IzWdA2ev4NMgEBLg9pouODuVQHlEzAFw84Rur1hWllC1OcweCHsXQPMRRf/8s7tg/iPowGYku1Wi3LZZ0OVFcM3ZGxYwyji2fg11boeuE3jhx+38deM8vz/WEedAOymJyODfAH5+AH57EgbPKJ4yiTM74JdHoFpzGP4juJVgGWirUfBNBPz8IDy8DAKblNxnW0kJFq4UkbMLPLYW2j8Fe+bB5y1h1ftw45rFp2ga7MNPj7Vl4vDmnEtIZtCX//DC/F3EXbWssbUQQggB3GqdVqnWzU2Hzl/hRlo6AYFBAHjry7Sv48cfu89SHF+S3hSzHwIaWl6bW7srVG4Am6beWve2sBKi0T8OI9GlIkMSxvLQ4fbGSO+e+bkfE7kIrpyFNo+zZM9Z/th9lrF31KOBvSXBAI0HGqPje3+B1f9n/fMnRMGPw6G8H4z4qWSTYAB3Txg5D9y94MdhRveUMqbsJMIA5Xyg57swZjPU62ksbTmpJez4HtItq4lRSjGgeRAr/9OVxzrXZsH2aLp9vJpv/zlBqh20axFCCFECMlqnZUo89pj6B9esbppfnhhH37CqnLqYyD4rL1qTRUykkQhbSiloMxrO7oSoLYX+2NTEBOK/HkzilXgGxj9LkntlVM2ORKZX5+Kqz3NPsjdNA9/axFXtxGsL9xIW5M1jnWsXOo5Sr8Oz0OI+WPsR7PzReue9fsVIPm9cM5JRryrWO3dBVKwGI38y/gGUEU8ZYq32aSXLNwTu/hZO/QvLX4HfxsC/U6DHW1DXshoVT3cXXurTkKHh1Xlz0T7eWLSPn7ac5p2BTbitppRLCCGEyEPm1mkme6ITqOjhQmBgNWNDYhy9GgfyysK9xVcecTUWEi9AQCOLdk9P16w5FMvF1I7c6eLF2aWf8G+LDwv+sYnJNFr7BK1SD/GG1xuMv2sgPRtVISU9nTlT7uLBuE/5c+kCevYZkvXAMzvg9CZSe77Pq7/t50pyKh8PbYaLc9kalysQpaDfp0Zp56JnwLs6hHQq2jnTUmH+w8Y/gu6ZB1Us++9fbKo2hbtmwtwR8MsoGDYbnJxtG5OFymYinKFGW2N9972/wN9vw/dDoHY36PG2xRPq6gZ4MvuR1izde453Fu9nyJR/uDs8mBcjQvHzlL5/QgghzMjcOs1kb3QCTYK8URkdAhLjqFTB7WZ5RLEsrhFjWuTVPzTP3dLTNUv3nmPi34c4dP6qcQkunXgwejn/O9aPGAo2APSGy7e0c9nKvtve4p1+Y28uXuXu5MyIR5/j6kczSN04le99w7m3bc2bx6X9O4105/L0W1udg/HneCGigX2WRGTn7Ap3fwdf94Sf7oFOzxmtxworeisc/tNIsC0cACx2DSIg4gNj0t6C0VCtRfF8TpXGUKeb1U5XthNhMP6lFXYXNLwTtnwNaz802sg0HWZMqPPJf+k+pRR9wqrSpb4/k1Ye5ut1x1m29xzPR4QysnUN+5nBKoRwCEqpCGAiRqefGVrrD7K9XxOYCfgDF4F7tdZRJR5oWZWcYIzCZhoRzpgo91CHWuBaHlw8IDEOgL5hVZmwYE/xdI+IiTSecxkRzp4A1w3wZOLw5rSq5YtzQh1cvlnK312OcKXdixZ/ZIUdM/Besxzd7ika93o2x/vu5bxwbvMQvTZ+QaeFq4Gu3B1enT827qLv7p/5MbU7Hr6+fDOoHl3r+xf8msuqcj7G6O03feGv1/PdPV8dx5e+tmVtHjNGvjd+AXvzqBMvivCHJRE2y8Ud2j0JzUca7Ur+nQr7f4Pur0CbJ4zJdvmo4O7CS70bMvS2YF5buI/XFu7l562n+b9BYbIykBA25OnpydWrV82+d+LECfr168fevXtLOKrSSSnlDEwGegBRwBal1CJTd58MHwPfaa2/VUp1B94H7iv5aMsoM63TMibKNQnyNgZoyvtB4kWA4i2PiNkP5XzBMyDLZnMJ8KQRLegbVvXW4I5PKNSPwGvPbLx6vGRZK7WDy2Dt6xDaD9Xj7Vx3c2kzCv3vF7xS5R/GLKzM5ysPM/TaXNxcUwkd8BwLw9s45tLTlWrB2F2Qkli08zg5W9YqzxZ6vQddXyq+hT2crbsWhP0kwhnK+RilEa0ehSUvwJ+vGqUT/T+HwDCLTlE3wIsfR7Vh0a4zvLM4kgGTN/Bwh1qM61Gf8m7290cmhLArrYEjWutjAEqpucAAIHMi3AjIWE1hFbCwJAMs88y0TsuYKBeWkeiW9705Ilys5RExkcZosOmc2RPgOv4VcibAmbV5DA4ttayV2tldRl1qYFMYPD3vGlCfGqgGfehzcjkDmzzI2cspPO28Fl3tdtq2aluEC7YDzi7gXNHWURQvd09bR2Ax+83qfGrAiDmw71ejXmV6V+gwFjq/YFFT64zuEl3rB/DBski+WnecJXvO8e6gJnRrEJDv8UKUGUsnwLk91j1nYBj0/iDXtydMmED16tUZM2YMAG+++SYuLi6sWrWKS5cukZKSwrvvvsuAAQMK9LHJyck88cQTbN26FRcXFz755BO6devGvn37eOihh7hx4wbp6en88ssvVKtWjbvvvpuoqCjS0tJ47bXXGDZsWJEuu5QIAk5neh0FtMm2zy5gMEb5xCDASynlp7WOK5kQyzgzrdP2RCfg5eFCTT9TF4nyfjcTYSim8gitIfYANB1mNgGeOLw5/ZpWy7u8L3MrtWbDc2/BlhBtdAQoV8noEGDJaGSbx1EHFvNZoyNGd43556HNF4W6VCGKix1P08T4H7rJYKPdWtNhsO5/MLUDHF9n8Sm8y7vy/uCmzHusHeXcnHnomy08PWcHsVek97AQhTVs2DDmzZt38/W8efN44IEH+PXXX9m+fTurVq3iueeeK3Dv1cmTJ6OUYs+ePcyZM4cHHniA5ORkpk6dytixY9m5cydbt24lODiYZcuWUa1aNXbt2sXevXuJiIiw9mWWZv8BuiildgBdgGggRw9KpdRopdRWpdTW2NjYko6x9DLTOm1vdAJhQd63RnuzJcK9Ggfi7KRYvPus9eK4HA3XL3PJqy59Jq1jzI/bSUvXTBzenD/HdWFA86D857hY0krt+lWYM8x4vmceeAVaFl+tjhDQ2GiXZmqZVmomdglhYr8jwpmV94WBX0LYUPh9LHzbz/i557sW/w/dOsSXP57pyNTVx5i86ghrDsYwoXdDhreqfnO2rBBlUh4jt8WlRYsWxMTEcObMGWJjY6lUqRKBgYGMGzeOtWvX4uTkRHR0NOfPnycw0MK/dIH169fz9NNPAxAaGkrNmjU5dOgQ7dq147333iMqKorBgwdTr149wsLCeO6553jxxRfp168fnToVsZ1R6RENZJ4lHGzadpPW+gzGiDBKKU9giNY6PvuJtNbTgekA4eHhxbgiRBlz8VjuE+UyZEuEM8ojluw5y4sRViqPME2Um7THhdMXEy0bATan6XBY8bYxKly9ddb30tOMcojz+41etVUaW37ejCT797HG64gPimdlNSGKwLF+I+t0gzGbjKUf9y+Cz8ON9dbTUiw63N3FmbF31GPps51oVK0iL/+6h6HTNnLw3JViDlwI+zN06FDmz5/PTz/9xLBhw/jhhx+IjY1l27Zt7Ny5kypVqpCcnGyVzxo5ciSLFi2iXLly9OnTh5UrV1K/fn22b99OWFgYr776Km+/nfvEnzJmC1BPKRWilHIDhgOLMu+glKqslMq4/7+E0UFCWCpbD+EsE+UylPeD5Hij36uJ1RfXMLVO++W0Fy/3bWjZCLA57p7Q8j6jlHBSi6yPz8Lg8HLo8yHUK8Robtjd4OFjLAHdfGTBjxeimDlWIgzG2ufdXoYnNxp9iJe/bLRbO7HB4lPU8fdkzqi2fHRXU47GXqXvpHV8tPwAySmWrW4nhDDKI+bOncv8+fMZOnQoCQkJBAQE4OrqyqpVqzh58mSBz9mpUyd++OEHAA4dOsSpU6do0KABx44do3bt2jzzzDMMGDCA3bt3c+bMGcqXL8+9997L888/z/bt2619iTahtU4FngKWA5HAPK31PqXU20qp/qbdugIHlVKHgCrAezYJtiwy0zotx0Q5MDo5gLHalklGecQfe6xTHnHt9B7OaV/C6tZkZOsaRTtZ+2eg2UgIui3ro2Z76Ps/YwJ6YbiVNyar3zkRPKT7kih9HKM0why/OnDPz3DgD1j2EszqA03uMjpOeAfle7hSiqHh1ekeGsD/LTnA5FVHWbz7LF/e05LG1eR/diHy07hxY65cuUJQUBBVq1blnnvu4c477yQsLIzw8HBCQ/NeIMCcJ598kieeeIKwsDBcXFyYNWsW7u7uzJs3j9mzZ+Pq6kpgYCAvv/wyW7Zs4fnnn8fJyQlXV1emTJlSDFdpG1rrJcCSbNtez/TzfKCYmnzaOTOt03JMlAOjJA+M8ghPo1euNbtHaK2JObqDaKrz3yFNi15q4VUFBk4u2jly06h//vsIYSOOmwiDUb/UsB/U6Q7rP4UNE+HgEmPFl3ZPWdRdws/Tnf/d3YwhLYN47udd3DVlI58Oa0ZEk6olcAFClG179tzqVlG5cmU2btxodr/ceggD1KpV62YPYQ8PD7755psc+0yYMIEJEyZk2darVy969epVmLCFI7vZOu3WiHCOiXJglEZAljphsF73iDmbjjP4xkmSQ0YQXKl8/gcIIcxyvNIIc9zKGwtvPLXZSIpXvgNftoEDS4z2NBZoX7cyv43pQINALx7/fjuT/j5c4BnvQgghSrmbrdNCgFsT5cKyJ7UZiXDSxSybrVEecfpiIrOXrMFDpRDatHX+BwghciWJcGaVasHwH+C+hcbymHNHwOyBEG1Z7WBARQ/mjm7L4BZBfPLXIZ6es4OkG1I3LIQ17Nmzh+bNm2d5tGmTvT2uEMUsW+s0sxPlINcR4czlEYUZLNFaM2HBbuqZWkWrKuaXVhZCWMaxSyNyU6cbPL4etsyANR/CV90gtB90fxUCGuZ5qIerM/+7uxn1A73477IDnIxL5Kv7wwn0zr/MQoiSpLUuU0uchoWFsXPnzhL9TPlWR+SQrXWa2YlykLVGOJuilEf8uPkUG47EsTDsOhwG/AteSy+EuEVGhHPj7AptnzDWBO/6MhxbA1+2g19G3aoRy4VSise71OGr+8I5FnuVodP+ISHJshZtQpQEDw8P4uLiJNHLg9aauLg4PDzkH7Eik2yt08xOlAOjQ5FrBUi8SHaFLY9ISErhg6UHaF/Hj2ZuZ4xvMS1Z4U0IkSsZEc6PR0Xo+iK0HgUbPoNN02HvL0Yrme6v5NkO5o5GVfjukdbcPe1fJvyymy/vaVmmRuCE/QoODiYqKgpZLSxvHh4eBAcH2zoMUVqYaZ1mdqJchmyLamTIvLhGQbpHzFx/nCvJqbzatxHq10gIkLIIIYpKEmFLlfc1Wqu1fRLW/Bc2Tzeaj/d8F5renev67LfV9OX5Xg34YOkBvv/3JPe1q1WycQthhqurKyEhIbYOQ4iyJVvrNLMrymVW3tdsIgwFL49ISEph5objRDQOpFGAB8QdhtA+hbkKIUQmUhpRUF6B0O9TGL0KfKrDr6NhVr+bS12aM7pTbbo28OedxZHsNdWTCSGEKGOytU47EXeNG2npNKxa0fz+uYwIQ8HLIzJGg5+5vR7EHYH0VBkRFsIKJBEurGot4JEV0O8ziNkHUzvCn6+ZXa7ZyUnxv6HNqFTBlad+3M7V66k5zyeEEKJ0y9Y6LTo+CYDqvuXM759HIpy5PCK/Wv0so8HVKt5cWjm/ydtCiPxJIlwUTk4Q/hA8tQ2ajYB/JsHyV8zu6ufpzqThLTh1MZGXF+yRSUpCCFHWZGuddsaUCFfzySsRzjlZLkPfsKqcjEtk35nLeX5sltFgML6BdHIBv3oFvwYhRBaSCFtDBT8Y8AW0HQObp8H278zu1qa2H+PuqM+iXWf4acvpEg5SCCFEkWRrnXYmPglnJ0WAVy6dRcr7wfXLkHrD7NuWlEfkGA0GIxH2rQMuboW+FCGEQRJha+rxNtTuBovHw6lNZnd5sltdOtatzBuL9nHo/JUSDlAIIUShZWuddiY+mcCKHjg75dL1IaOXcJL5UWFLyiNyjAaDURohZRFCWIUkwtbk7AJDvzEm0f10LyRE59zFSfHpsOa4OTsxdXXe/YiFEEKUEmZap0VfSiIot7IIyHV1ucwyyiNmrDvO9dSsK5GaHQ2+kQiXTshEOSGsRBJhaytXCYbPgZQkmDvSeM7G38udfs2qsmzfOa7JxDkhhCj9srVOA2OyXDWfPBZcsSAR7tesGm1CfHlvSSRdP1rN7H9P3kyIzY4GXzgIaBkRFsJKJBEuDgGhMOQrOLsLFj0DZr7yGtwymMQbaSzfd84GAQohhCiQbK3T0tI15y4n5z5RDixKhD3dXZg7ui3fP9KGIJ9yvLZwL10/Ws3M9cdzjgbDrVadMiIshFVIIlxcGvQ2Vp7bMw/++TzH2+E1K1Hdtxy/7shZPiGEEKKUydY6LeZKMmnpmqBKRUuEAZRSdKxXmZ8fb3czIX578f6co8Fg1Ac7u4OvLIgjhDVYtLKcUioCmAg4AzO01h9ke/9ToJvpZXkgQGvtY8U4y6ZO/4EzO2Hlu9BkCHgH3XxLKcWgFsF8vvIw5xKSCfTO4+s1IYQQtlXQ1mlwa7JcHi3UMstIiDvU9WPDkTguJt7IOhoMxoiwfwNwci7wJQghcsp3RFgp5QxMBnoDjYARSqks38lorcdprZtrrZsDnwMLiiHWskcpiHgf0LDmgxxvD2oRhNawcKeMCgshRKmWrWNEdHwyQN6T5Zxdwb1iviPC2WUkxP2bVcv6htZwbo/UBwthRZaURrQGjmitj2mtbwBzgQF57D8CmGON4OyCTw0IfwR2/AAXDmd5K6RyBVrW8GHB9ihZYEMIIUorrSH2gDESa5IxIlw1v2/zyvtaPCKcr5j9cPU81OxgnfMJISxKhIOAzKs/RJm25aCUqgmEACuLHpod6fQcuHgYJRLZDG4ZzKHzV/NdWUgIIYSNXD5jLIyRaST2THwSFT1c8PJwzfvYPJZZLrDDfxnPde+wzvmEEFafLDccmK+1TjP3plJqtFJqq1Jqa2xsrJU/uhTz9If2T8H+hXBmR5a3+jWtipuzk0yaE0KI0spMp4Yz8Ul51wdnsGYifGQFVGmSZb6JEKJoLEmEo4HqmV4Hm7aZM5w8yiK01tO11uFa63B/f3/Lo7QH7Z6Ccr7w99tZNvuUd6N7aAC/7YwmNS3dRsEJIeyJUipCKXVQKXVEKTXBzPs1lFKrlFI7lFK7lVJ9bBFnmRGz33j2D725KSq/xTQylPezTmlE8mU4tVFGg4WwMksS4S1APaVUiFLKDSPZXZR9J6VUKFAJ2GjdEO2ER0WjROLoSji+Nstbg1oGceHqDdYdvmCj4IQQ9sKSCc7Aq8A8rXULjHv6lyUbZRkTEwleVW91gcAGI8LH10B6KtTrUfRzCSFuyjcR1lqnAk8By4FIjJvnPqXU20qp/pl2HQ7M1TLrK3etHjXa76x4K8siG90aBOBT3pUFUh4hhCg6SyY4ayCjL5c3cKYE4yt7YvZnGQ2+kpzC5eRUCxNhX0i5ZnaV0QI5/JfRgaJ6m6KdRwiRhUU1wlrrJVrr+lrrOlrr90zbXtdaL8q0z5ta6xxfwYlMXD2g6wSI3goH/ri52c3Fif7NqvHnvnNcTk6xYYBCCDtgyQTnN4F7lVJRwBLgaXMncth5HZmlp0HswSz1wWcTTK3T8lpMI8PNRTWKUB6htVEfXLuL0ZJNCGE1srJcSWs2Evzqwcp3jBusyaAWQVxPTWfZHllyWQhR7EYAs7TWwUAfYLZSKsffBw49ryPDpROQmpSlY0S0qXVakI8FCyFZuLpcnmL2w+VoqCtlEUJYmyTCJc3ZBW5/zehJuWf+zc3Nq/tQu3IFftkeZcPghBB2wJIJzo8A8wC01hsBD6ByiURX1uTSMQLyWVUugzUSYWmbJkSxkUTYFhr2B986sPP7m5uMJZeD2HT8ImcTilhLJoRwZJZMcD4F3A6glGqIkQg7aO1DPmJNiXC2xTScnRQBXiU0InxkBQQ0lrZpQhQDSYRtQSloejccX2c0ajfp2TgQgLWH5O8jIUThWDjB+TlglFJqF0bLywdlonMuYiLBpya4e97cdCY+mcCKHjg7qfyPL2qNcEbbNOkWIUSxkETYVsKGAhr2Lri5qX4VT6pUdGettFETQhRBfhOctdb7tdYdtNbNtNbNtdZ/2jbiUiwmMktZBBg1whb1EAbw8AFU4UeEpW2aEMVKEmFb8asD1VrCnnk3Nyml6FTPn/WHL5CWLoMzQghhU6k34MKhLBPlAKIvJVHNkolyYMwLKedT+ERY2qYJUawkEbalsKFwdhfEHrq5qXN9fxKSUtgdFV+4c148Blu+ztKnWAghRCFcPGqMxmZKhNPSNecuJ1s2US5DYRfVkLZpQhQ7SYRtqclgUE6w5+ebmzrWrYxSFG6VuSvn4dsB8Md4OL3JioEKIYQDylhaOVMiHHMlmbR0XTKJsLRNE6LYSSJsS16BENLZSIRNI7i+FdwIC/Iu+IS5G9dgzjBIvAAuHrB7Xv7HCCGEyF1MJChno/e7SUbrNIsW08hQ3q9wk+WkbZoQxU4SYVsLGwqXjkP09pubOtfzZ8fpeMtXmUtPg/mPGGUWd82E0L6w71dIk1XqhBCi0GIijfkcrrfqgaPjTavKFWhE2LdwI8LSNk2IYieJsK01vBOc3bNMmutUrzJp6Zp/jlhw49Qalk2AQ0uh94fQoDeE3Q1JF+HI38UYuBBC2LmYyBwT5TJGhKt6WzhZDm6VRhRk7oa0TROiREgibGse3lC/l9FGLS0VgJY1K+Hp7sLawxaUR/z7JWyeDu2egtajjG11b4dyvlmSayGEEAWQkmRMPs7WOu1MfBIVPVzw8ijA5LVyvpB2HVISLT9G2qYJUSIkES4Nmt4N12KMGx/g6uxEuzp+rD0US5497vf/BstfMVaq6/HOre3OrtB4EBxYAtevFHPwQghhh2IPAtrsiHCBJspB4VaXO7EBXMtL2zQhipkkwqVB3R7g7g175t/c1LleZaIuJXEiLpcRhIRoWPAYBLeCwdPBKdt/yqZ3Q2oSHPijGAMXQgg7FZOxtHK2HsLxyQWrD4bCJcJxR8CvrrRNE6KYSSJcGrh6QKM7IfJ34+s4jH7CkMdyyxu/gPQUGDIDXM3clKu3AZ8a0j1CCCEKI2Y/OLuBb+0sm6MvJZbMiPDFozk+WwhhfZIIlxZhd8ONK3BoGQA1/SpQw7c868zVCV+7ANtmGcdUqmn+fEoZHSmOrYKrMcUXtxBC2KOYSKjcwFgZzuRKcgqXk1OLkAhb2EItLQXiTxkdK4QQxUoS4dKiVkfwDITdtxbX6Fy/MhuPxnEjNT3rvpumGiPHHZ/N+5xhQ0GnGxPxhBBCWM5Mx4izCUbrNIuXV85Q3td4tnREOP6UMVFORoSFKHaSCJcWTs4Qdhcc/hMSogCjn/C1G2lsO3np1n7Jl2HTdKPtmn+DvM8Z0BCqhEn3CCGEKIjky3A5KkciHG1qnRZckMU0ADx8jFVELU2ELx43nn1lRFiI4iaJcGnS6lFwcYdfHoW0VNrV8cPFSWUtj9j6NVxPgE7jLTtn06EQvQ3ijhZPzEIIYW9iDxjPZlqnAQUvjXByMlqoWZwIm+7XMiIsRLGTRLg08Q2Bfp8aTdTXfICXhysta1S61U84JQk2ToY6t0O1Fpads8ldgDKWcRZCCJG/mP3Gs5nWac5OigCvApZGwK1FNSxx8Ri4eYJnQME/RwhRIJIIlzZN74bm98Laj+HoKjrVq8ze6MtcuHodts+Ga7HQ6TnLz+cdZNQf755XsFWNhBDCUcVEGomod/Usm8/EJxNY0QNnJ1Xwc5b3s3yyXJypY4QqxOcIIQpEEuHSqM+HULk+LBhNd9N9+J9DZ2DDRKjeFmq2L9j5woYaX7Wd2W79WIUQwt7E7DfmYGTrzx4dn1TwHsIZyhewNELKIoQoEZIIl0ZuFWDoLLh+mUYb/4NvOWeubZlrTN7o9FzBRwkaDTD6Ye6W8gghhMiXmY4RkLGqXCHKIsDy0ghpnSZEiZJEuLSq0gh6/xd1fDX/5/8nbc58R6p/48KtO1/OB+r3gr3zjZusEEII867GGiVo2SbKpaVrziUkF3yiXIaMRDi/EjVpnSZEiZJEuDRr+QA0GUJEzNfUVmdYVmlk4WvGmo0wbu5HV1o3RiGEsCexpqWVs40Ix1xJJjVdFy0RTk+F65fz3k9apwlRoiQRLs2Ugn6fgW9tzrrW5NVDdbiSXMgR3bo9jBvxrjlWDVEIIexK7EHj2T80y+aM1mmFrxG2cJllaZ0mRImSRLi086gIj63l4rBFxCen8/2/pwp3Hhc3o5XagSWQdCn//YUQZZZSKkIpdVApdUQpNcHM+58qpXaaHoeUUvE2CLN0ykhUK2RtXRYdb6wqF1TQxTQyWLrMctxRaZ0mRAmSRLgscPeicd1adKpXma/XHyM5Ja1w52k2HNKuw76FVg1PCFF6KKWcgclAb6ARMEIplaXgVWs9TmvdXGvdHPgckHXYMyTFg5sXOLtk2ZwxIlzVuwiT5cCCEeFj0jpNiBIkiXAZ8lS3uly4eoN5W08X7gTVWhhf9+2aa93AhBClSWvgiNb6mNb6BjAXGJDH/iMAqZnKkBwPHt45Np+JT6KihwteHq6FO295X+M5vxFhaZ0mRImSRLgMaR3iS3jNSkxbc4yUtPSCn0ApY1T49L+y5LIQ9isIyPyv5SjTthyUUjWBEMDsLFql1Gil1Fal1NbY2Fhzu9if5ASj0042Ruu0QpZFgGUjwtI6TYgSZ1EinF+9mWmfu5VS+5VS+5RSP1o3TAGglGJMt7pExyfx284zhTtJ2N2Agt0/WTU2IUSZNByYr7U2W2+ltZ6utQ7XWof7+/uXcGg2khQPHj45Np+7nExgYcsiANy9wMkl70RYWqcJUeLyTYQtqTdTStUDXgI6aK0bA89aP1QB0LWBP42qVuTL1UdISy/EksneQVC7q9E9Ir0Qo8pCiNIuGsi8NnCwaZs5w5GyiKyS482OCF+6loJvebfCn1ep/BfVkNZpQpQ4S0aELak3GwVM1lpfAtBax1g3TJEhY1T4WOw1lu87V7iTNBthjDyc2mjd4IQQpcEWoJ5SKkQp5YaR7C7KvpNSKhSoBMiNILNcRoQTklLwLl/I+uAMPjWMVetyI63ThChxliTCltSb1QfqK6U2KKX+VUpFmDuRQ9abFYOIJoHUrlyByauOoPNbpcichv2M9jzSU1gIu6O1TgWeApYDkcA8rfU+pdTbSqn+mXYdDszVhbqJ2DEzI8IpaelcvZ6KT7kijAgD1OkO0VtznzAnrdOEKHHWmiznAtQDumLMQP5KKeWTfSeHrDcrBs5Oise71mHfmcu8tGAP/xy9QGpBJs+5VYBGA4w2ailJxRanEMI2tNZLtNb1tdZ1tNbvmba9rrVelGmfN7XWZud8OKzUG5CSmGNEOCHJWMioUoUijgjX7QE6PfcVPqV1mhAlzpJE2JJ6syhgkdY6RWt9HDiEkRiLYjKoRRCDWwTx645oRn61iVbvreA/P+/ir/3nLesz3Gw43LgCB/4o/mCFEKIsSI43nrONCMcn3gDAu1wRE+GgllDOF46sMP++tE4TosRZkghbUm+2EGM0GKVUZYxSiWPWC1Nk5+rsxCfDmrPj9R5MvbclXRsE8Oe+c4z6bitt3/+btYfyKT2p2RG8q0t5hBBCZEhOMJ6z9RGOTzRGhH2KMlkOwMkZ6t5uJMLZJytL6zQhbCLfRNjCerPlQJxSaj+wCnhea53P8jnCGsq7uRDRpCqfDmvOttd6MPuR1gRW9ODBbzYzY92x3GuInZyg6TDjK7orhZx0J4QQ9iQp3njOVhpxMxEu6ogwGOUR12Lh7M6s26V1mhA2YVGNcH71ZtowXmvdSGsdprWWpctswNXZiU71/Pnlifb0ahzIu39E8p+fd+deKtF8JGgNK98p2UCFEKI0yqU04pKpNKJSUUeEwRgRRuUsj7ho+hJVWqcJUaJkZTk7VMHdhckjWzLujvr8sj2K4dP/5fzl5Jw7+tWBTuNhx/ew/7eSD1QIIUqTXEaEMybLFbl9GkCFysZy94f/yrr9ZiIsI8JClCRJhO2Uk5Ni7B31mHrvbRw6f4U7P1/PrtPxOXfs+hJUawmLnoGE3HruCyGEA8h1slwKTgq83F2s8zn1euRsoyat04SwCUmE7VxEk0AWPNkeNxcnHp61JefIsLMrDJlhTNT49TFZbU4I4bhyqxFOuoFPeTecnKzU1sxcGzVpnSaETUgi7ABCAysy66FWJKWk8fScHTl7DvvVgd7/hRPrYOPntglSCCFsLTkeXMuDS9Za4EuJKdaZKJfBXBs1aZ0mhE1IIuwg6gZ48d6gJmw+fpFPVxzKuUOLe6Fhf/j7HTizs8TjE0IIm8tteeVEKyyvnJmTs7HKXEYbNWmdJoTNSCLsQAa1CGZ4q+pMXnWU1Qdjsr6pFNw5ESr4wy+Pwo1rtglSCCFsJTk+Rw9hMJVGWHNEGIw64Yw2atI6TQibkUTYwbzZvzGhgV6M+2knZxOyLa9c3hcGTYW4I7BsgtFaTQghHEVyQo6JcgCXrqVYp3VaZnUytVGT1mlC2Iwkwg7Gw9WZyfe05EZqOk//uIOU7PXCtbtAh7Gw/Ttj8lyKmbZrQghhj3IrjUiycmkEgKf/rTZq0jpNCJuRRNgB1fH35P8Gh7H15CU+/vNgzh3ueBO6vQK7f4JZfeDy2RKPUQghSlxyfI4R4ZS0dK5eT8WnnJVHhOFWG7WordI6TQgbkUTYQQ1oHsTINjWYtuYYM9cfz7oUs1LQ5QUY9j3EHIDpXY0btRBC2DMzI8IZi2lUqmDlEWG41UZt/0JpnSaEjUgi7MBe79eIOxoG8Pbi/Tz+/TYSElOy7tDwTnj0L3Bxh2/6wM45tglUCCGKW1oq3LhiZjENY3llb2tPloNbbdTSbkhZhBA2IomwA/Nwdear+8N5tW9D/o6Moc+kdew4dSnrTlUaw6hVUL01LHwcFo83JpQIIYQ9ybivZV9MwzRA4GPtyXJwq40aSOs0IWxEEmEHp5Ti0U61+fnxdgAMnbqRr9Yey1oqUcEP7vsV2j0FW2fC5Dawb6F0lRBC2I88llcGrN8+LUO9HsazjAgLYROSCAsAWtSoxJJnOnF7wwDeWxLJo99u5dK1G7d2cHaFXu/Bo39Dhcrw8wPw4zC4dNJ2QQshhLVkJMLZ+ghfMpVGWL19WobQvnDbg1CvZ/GcXwiRJ0mExU3e5V2Zeu9tvHlnI9YdvkCfSevYcuJi1p2Cb4NRq6Hne8aSzF+2hQ0TISXJ7DmFEKJMSIo3nnOZLGf19mkZ3L2MxYykY4QQNiGJsMhCKcWDHUL45Yn2uLk4MXz6v0xedYT09ExlEM4u0P4pGLMZQrrAX6/D/xrA0hchJtJ2wQshRGHlURrhpMDL3aXEQxJCFD9JhIVZYcHeLH66I72bBPLR8oM88M1mYq9cz7qTT3UYMQce/MNoA7R1pjFC/HVPo8OEjBILYRNKqQil1EGl1BGl1IRc9rlbKbVfKbVPKfVjScdY6uQyIhyfdAPvcq44OUlrMyHskSTCIldeHq58PqIF7w8OY/Pxi/SZtI71hy9k3UkpqNUR7voaxh+Anu9CYpzRYeLTJrDje0hPN/8BQgirU0o5A5OB3kAjYIRSqlG2feoBLwEdtNaNgWdLOs5SJ5cR4UuJxbC8shCi1JBEWORJKcWI1jX47akOVPRw4d6vN/Hu4v0kp6Tl3LmCH7R/Gp7aCg8sNtoB/TYGZvaEMztLPHYhHFRr4IjW+pjW+gYwFxiQbZ9RwGSt9SUArXVMCcdY+iTFg7M7uJbLsjkhsRiWVxZClBqSCAuLhAZWZPHTnbivbU1mrD/OwMkbOHDusvmdlYKQTvDQMhg4BS6dMFanWzwOEi+aP0YIYS1BwOlMr6NM2zKrD9RXSm1QSv2rlIoosehKKzPLK4NRGlFsrdOEEDYnibCwWDk3Z94Z2IRvHmzFhavX6f/5BmasO5Z1Il1mTk7QfKQxQtzmMdg2Cz6/DbZ+A+lmRpSFECXFBagHdAVGAF8ppXyy76SUGq2U2qqU2hobG1uyEZa05IQc9cFgTJaT0ggh7JckwqLAuoUGsPzZznRp4M+7f0Ry38xNnE3IY2JcOR/o/V94bB34h8LiZ2F6FzixoaRCFsKRRAPVM70ONm3LLApYpLVO0VofBw5hJMZZaK2na63Dtdbh/v7+xRZwqZAUn6OHMBiJsJRGCGG/JBEWheLn6c70+27jg8FhbD8ZT69P17J495m8DwpsAg8tgbtmQuIlmNUHfn4Q4k/nfZwQoiC2APWUUiFKKTdgOLAo2z4LMUaDUUpVxiiVOFaCMZY+ZkojUtLSuXo9FZ9yMiIshL2SRFgUmlKK4a1rsGRsJ0L8PXnqxx2M/2knl5NT8joImgyBp7ZAlwlwcCl80QpWvQ83EksueCHslNY6FXgKWA5EAvO01vuUUm8rpfqbdlsOxCml9gOrgOe11nG2ibiUSIrPdTENHxkRFsJuSSIsiiykcgXmP96OsbfXY+HOaHp/to7Nx/OZFOdWHrq9ZNQPN+gNaz6Aya1h7wLQudQcCyEsorVeorWur7Wuo7V+z7Ttda31ItPPWms9XmvdSGsdprWea9uISwEzI8LxpuWVJREWwn5JIiyswtXZiXE96vPz4+1xdlIMm76RD5cdICUtnx7CPtVh6Dfw4BJjNGb+QzCrH5zbUyJxCyEE6emQfDnnYhqJGSPCUhohhL2SRFhY1W01K7FkbCeG3hbMl6uPctfUjZyMu5b/gbU6wGNroO8nELMfpnU22q1dc+xva4UQJeB6AqDNLq8MSPs0IeyYJMLC6jzdXfjwrmZMHtmSY7FX6TtpPQt3ZJ+0boaTM7R6BJ7ZDq1Hw7Zv4ZNQ+H4IbPkaLuczGU8IIQoj1+WVjURY2qcJYb8kERbFpm/Tqiwd24nQQC+e/Wkn43/aydXrqfkfWK6S0W7tiX+MhDjuKPwxHj5paCzMseYjuGrnPU2FECUnOcF4zqVGWNqnCWG/JBEWxSq4Unnmjm57cyJd30nr2HHqkmUHB4RCr/fgmR3w5Ca4/XVQzrDqXaP1miTDQghrSI43nrP1EY5PTMFJgZe7S8nHJIQoERYlwkqpCKXUQaXUEaXUBDPvP6iUilVK7TQ9HrV+qKKscjFNpJs7uh0pqekMmfIP7y+JJDnFwtXllDKS4k7Pwai/jYl18afh+0G3vtIUQojCyrU04gbe5VxxclIlHpIQomTkmwgrpZyByUBvoBEwQinVyMyuP2mtm5seM6wcp7ADrUN8WTauM8NaVWfa2mP0mbiObSfzabNmTq0OMPx7iDkAPwyF61etH6wQwnFkjAhnK424JMsrC2H3LBkRbg0c0Vof01rfAOYCA4o3LGGvKnq48v7gpnz/SBuup6Zz19SNvP37fpJuWDg6nKHuHcYKddHbYO4ISEkunoCFEPYvlxHhBFleWQi7Z0kiHARkXgM3yrQtuyFKqd1KqflKqepm3kcpNVoptVUptTU2Vuo7HVnHepVZPq4z97WtycwNx4mYuJYtJwo4OtyoPwz8Eo6vNZZqTstjRTshhMhNcjw4uYBbhSyb45NuSOs0IeyctSbL/Q7U0lo3Bf4CvjW3k9Z6utY6XGsd7u/vb6WPFmWVp7sLbw9owtzRbUnXmrunbeT9pZFcTy3A6HCz4dDnYzi0FH59DNILOLIshBAZyyurrLXA8VIaIYTdsyQRjgYyj/AGm7bdpLWO01pfN72cAdxmnfCEI2hb24+lYzszvFUNpq05xoAvNrD/zGXLT9B6FNzxFuz9BX5+AFKv53+MEEJkSE7IUR8MRiIspRFC2DdLEuEtQD2lVIhSyg0YDizKvINSqmqml/2BSOuFKByBp7sL7w8O45sHWxF37QYDJq9n8qojpOa3RHOGjs9Cr/ch8nfTBLorxRqvEMKOJMfnqA9OSUvn6vVUfMrJiLAQ9izfRFhrnQo8BSzHSHDnaa33KaXeVkr1N+32jFJqn1JqF/AM8GBxBSzsW7fQAP58tjM9GwXy0fKDDJ22kUPnLUxq2z0JA6fCifXw3QBILERHCiGE40mKzzEinGBaVc5HRoSFsGsW1QhrrZdoretrretord8zbXtda73I9PNLWuvGWutmWutuWusDxRm0sG+VKrjxxcgWTBzenBMXrtF30jo++euQZbXDzUfAsO/h3F6YGQEJFiztLIRwbMnxZhbTMFaVk0RYCPsmK8uJUkkpxYDmQawY34V+Tasx6e/D9J20nq2WdJYI7QP3LYDLZ4xk+MKR4g9YCFF2ZUyWyyQ+MWNEWEojhLBnkgiLUs3P051PhzVn1kOtSLqRxl1TN/Lqwj1cSc6nVVqtjvDgYkhJhBndYc/8kglYCFG2aG12stzNRFjapwlh1yQRFmVC1wYB/DmuMw93COGHTafo8clalu87l/dB1ZrDoyugcgP45RGY/zAkXSqReIUQZcT1K6DTzCyvLDXCQjgCSYRFmVHB3YXX72zEr092wKe8K4/N3sbjs7dx/nIeq8r5hsBDS6H7q7D/N/iyPRxdVXJBCyFKt1yWV75VIyylEULYM0mERZnTvLoPvz/dkRciGrDqYAx3/G8N3/97kvR0bf4AZxfo/Dw88pexctTsgbB0grRYE0IYZRFgtkbYSYGXu0vJxySEKDGSCIsyydXZiSe71mX5s51pWt2bVxfu5e5pGzkWezX3g4JawmNrofVo2DQFPq4PCx6DY6tlRTohHFVSvPGcfUQ46Qbe5VxxclI5DhFC2A9JhEWZVqtyBb5/pA0fD23G4Zir9J64jq/XH899dNitPPT5CB5dCWFD4eASo+fwZ2Gw4i24cLhkL0AIYVsZpRFmRoRleWUh7J8kwqLMU0px123B/DWuMx3rVuadxfsZPv1fTsZdy/2g4Nug/yT4zyEY8jUENIINn8EX4fDjcDi9pcTiF8LalFIRSqmDSqkjSqkJZt5/UCkVq5TaaXo8aos4S4WMEeEcfYRleWUhHIEkwsJuBFT0YMYD4Xw8tBmR5y4T8dk6Zm88kfvoMIBrOQi7C+6dD+MjoetLcPpf+PoOmNUPjq402isJUUYopZyByUBvoBEwQinVyMyuP2mtm5seM0o0yNIkt8lySTekdZoQDkASYWFXMkaH/xzXmdYhvrz22z6Gf/UvkWcv53+wVyB0nQDP7oWe70HcEZg9CKZ3he3fQewhSYpFWdAaOKK1Pqa1vgHMBQbYOKbSKykelBO4eWXZHJ+YIh0jhHAAkggLu1TVuxyzHmrFf4eEcfj8FfpOWserC/dw6dqN/A9294T2T8HYXXDnJLh+GRY9DZNbwYch8MNQWPsRHF8rk+xEaRQEnM70Osq0LbshSqndSqn5Sqnq5k6klBqtlNqqlNoaGxtbHLHaXsbyyk5Z/zo0EmEZERbC3klfGGG3lFIMa1WDXo0D+WzFYWb/e5Lfd51l3B31uLdtTVyc8/l3oIs73PYAtLgP4g7D6c1wehNEbYHDfxr7VG8Lg6dDpZrFf0FCWM/vwByt9XWl1GPAt0D37DtpracD0wHCw8Pt8+uQ5IQcE+VS0tK5ej0Vn3IyIiyEvZMRYWH3fMq78Wb/xix5phNNgiry5u/76TNpHf8cvWDZCZycwL8BtLwPBnwBYzbBiyeg/+dwfh9M7ShLOIvSJBrIPMIbbNp2k9Y6Tmt93fRyBnBbCcVW+iTF56gPTpBV5YRwGJIIC4fRINCL7x9pw7T7biMpJY2RX23imTk7iMlrZbrclKsELe+HJ9ZDQENjCecFoyHZglpkIYrXFqCeUipEKeUGDAcWZd5BKVU108v+QGQJxle6JMebbZ0GkggL4QgkERYORSlFr8aB/DWuC890r8uyvefo/r81zFx/nNS09IKfsFIteHCJ0W1iz8/G6PDpzVaPWwhLaa1TgaeA5RgJ7jyt9T6l1NtKqf6m3Z5RSu1TSu0CngEetE20pYCZEWFZXlkIxyGJsHBIHq7OjO/ZgOXjOtOyZiXeXryfO7/YwLaTFwt+MmcXo9vEQ8sADTN7wZ+vQUqS1eMWwhJa6yVa6/pa6zpa6/dM217XWi8y/fyS1rqx1rqZ1rqb1vqAbSO2oYzJcpncHBGW9mlC2D1JhIVDC6lcgW8fasWUe1oSn3iDIVM2Mn7eTmKuFKJcokYbeHy9Mbnun0kwpQOc/Mf6QQshrENrY0Q4e2mE1AgL4TAkERYOTylF77CqrBjfhSe61uH3XWfo/vEavlp7jJSClkt4eBsr1t3/G6Snwje9YcnzcP1q8QQvhCi8lERIT5HSCCEcmCTCQphUcHfhxYhQ/hzXhVa1KvHekkgiPlvLusOF6J9auys8uRHaPAGbv4Iv28G+XyEtxepxCyEK6ebyyj5ZNscnpuCkwMtdOowKYe8kERYim5DKFfjmodZ8/UA4qema+77ezKjvtnIkpoCjum4VoPcH8PAycPWAnx+ETxrBX29A3NFiiV0IUQDJCcazmeWVvcu54uSkSj4mIUSJkkRYiFzc3rAKf47rzPO9GrDxaBy9PlvLSwv2FLzdWo228MRGGPETBIfDP5/D5y1hVj+j//CNa8VzAUKIvCXHG89mRoQrSVmEEA5BvvcRIg/uLs6M6VaX4a2q8/nKI/yw6SQLd0QzqlMIo7vUwdPSr06dXaBBhPG4fBZ2fg/bZxv9h108oE53CO0HDXpDed/ivSghhCGjNCJHjXAK3jJRTgiHIImwEBbw83Tnzf6NeahDLT5afpBJK4/ww6ZTPNW9LiPb1MDdxdnyk1WsCp2fh47PwckNEPk7HPgDDi4B5Qw120Oz4dD8HlDy1awQxSa3EeGkG/h7upd4OEKIkielEUIUQE2/CnwxsiW/jelA/SpevPX7frp9tJqftpwq+IIcTk4Q0gn6fAjj9sLo1dBxHFyNgd/GwOr3i+UahBAmNyfL5ewjLB0jhHAMkggLUQjNqvswZ3Rbfni0Df4VPXjxlz30+HQti3adIT1dF/yESkG1FnD7a/Dkv0Yv4jX/hXWfWD94IYTh5oiwuURYSiOEcASSCAtRBB3qVmbhk+2Zft9tuDk78cycHfSZtI6VB86jdSESYjBGiu+cCGFD4e+3YOOX1g1aCGFIigd3b3C6VdqUkpbO1eup+JSTEWEhHIHUCAtRREopejYO5PaGVVi8+wyf/HWIh2dtJbxmJV6ICKV1SCEmvzk5w8CpkJoMy18y2q+FP2z94IVwZMkJUC7raHCCrConhEOREWEhrMTZSTGgeRArxnfh3YFNOHUxkbunbeShbzaz70xCIU7oAkNmQr2esHg87Jxj/aCFcGTJ8WZbp4EkwkI4CkmEhbAyV2cn7m1bkzXPd2NC71C2n4qn76T1PDNnB6cvJhbsZC5ucPdsCOkMvz1p9B0WQlhHUrwsryyEg7MoEVZKRSilDiqljiilJuSx3xCllFZKhVsvRCHKpnJuzjzepQ5rX+jGk13r8Of+c3T/32re+n0fF6/dsPxErh4wYg7UaAcLRsGO74svaCEcSV4jwuVkRFgIR5BvIqyUcgYmA72BRsAIpVQjM/t5AWOBTdYOUoiyzLucKy9EhLL6P90Y0jKYb/85QZcPVzF51RGSbqRZdhK3CnDPzxDSxWittmla8QYthCMwNyIsNcJCOBRLRoRbA0e01se01jeAucAAM/u9A/wXKOD6s0I4hkBvDz4Y0pTlz3amTW0/Plp+kK4fr+Lbf06QnGJBQuxWAUb+ZKxAt/QFWPe/4g9aCHuWHJ+jddp50xLqvhWkNEIIR2BJ14gg4HSm11FAm8w7KKVaAtW11n8opZ7P7URKqdHAaIAaNWoUPFoh7EC9Kl7MeCCczccv8uGyA7yxaB+frzzMQx1CuK9dTSp65DES5eIOQ2fBwifg77fhxjXo/pqsQCdKp4PL4NweW0dhnk43urJkK404eO4KQT7l8Mrr/0MhhN0ocvs0pZQT8AnwYH77aq2nA9MBwsPDC9lkVQj70DrEl/lPtGfz8YtMXnWEj5YfZOrqo9zfviYPdQihcm5LvDq7wqBp4FreGBW+fhVufx3cPUv2AoTIz8E/YPt3to4id8oZqjTJsiny7GUaVvWyUUBCiJJmSSIcDVTP9DrYtC2DF9AEWK2MUalAYJFSqr/Wequ1AhXCXrUO8aV1SGv2Rifw5eojfLn6KDPXn+D+9jV5rHMd81/ROjkbi264e8HGL2DLDKjaDGq2Nx412kH5QvQvFsKa+n0GfT+1dRS5UyrLYhrJKWkcu3CNXo0DbRiUEKIkWZIIbwHqKaVCMBLg4cDIjDe11glA5YzXSqnVwH8kCRaiYJoEefPlPbdxJOYqX6w8zPS1x/h+40ke6hDCqE618c4+eUcp6Pku1O8Fx9fCyX9g81dGYgwQ3Br6fGgs3SyELWRKMsuCIzFXSUvXNKxa0dahCCFKSL6JsNY6VSn1FLAccAZmaq33KaXeBrZqrRcVd5BCOJK6AZ58NrwFY7rV5bMVh/li1RG+3XiCRzvW5qGOtbLWECtl9BgO6Wy8TkmGMzvg5HrYPAO+6g7txkDXl8GtvG0uSIgyIvLsZQBCpTRCCIdhUR9hrfUSrXV9rXUdrfV7pm2vm0uCtdZdZTRYiKKrV8WLyfe0ZMkznWhX249PVxyi/fsreev3fZyKy2VhDlcPqNkOOj8PYzZBy/vhn89hSjs4uqpgAaQkQeLFol+IKHHS+71wIs9ewcPViVp+FWwdihCihBR5spwQong1qlaR6feHszc6gRnrjjF740m+/ecEPRpV4ZGOtWlVqxLKXNeIcj5GHXHY3fD7MzB7IDQbCR3GQuV65r+2Tr0OR1fC3l/gwBJITYJanaDJEGh4p9QdlwGZer/3wOjys0UptUhrvT/bftL7PZsD5y7ToIoXzk7ShUUIRyGJsBBlRJMgbz4b3oIJvRvy3cYT/Lj5FMv3nScsyJuHO9aib1g13FzMfMlTqwM8vgHWfgQbPoNdP4KLBwQ0gsAmUCUMvKrAoT8h8ne4ngDlKkHToVDBH/YuMBLpP56DurebkuL+xuizKI1u9n4HUEpl9H7fn22/jN7vuba8dCRaayLPXpaJckI4GEmEhShjAr09eCEilKe71+OX7VHM3HCccT/t4v+WHOD+tjUZ2aYGftlbr7l6wO2vQcv74NS/Rm/Xc3sgcvGt9lZuXtCwn5Ho1u5qtGkD6PYKnN0Fe+cbSfGhZRDwGQybDX51SvLShWWk93shxFy5zqXEFEIDpT5YCEciibAQZVQ5N2fubVuTka1rsPZwLDM3nOB/fx3i81VHGNQ8iIc61iI0MNvs90q1jEez4cZrreHKWYg/BVWbmx/lVQqqNTced7xtJMK/PQnTu8LAKUbyLMoM6f1u3n7TRDnpGCGEY5FEWIgyzslJ0bVBAF0bBHAk5grfbDjBgu3R/LT1NJ3r+/NY59q0r+Nnvo5YKahYzXhY9mEQ2gcC18K8B+Cne4ya4+6vg7PcTkoJ6f1eCAfOXgEgVBJhUcqlpKQQFRVFcnKyrUMplTw8PAgODsbV1bLVIeVvLiHsSN0AL94bFMbzvRrww6ZTfLPhBPfM2ESToIqM6lSbvmFVcXG2qFlM3nxqwMPLYNlLsGEiRG2Du2YatcbC1qT3eyFEnr1MkE85vMvJ0sqidIuKisLLy4tatWqZH+BwYFpr4uLiiIqKIiQkxKJjrPA3ohCitPEp78aYbnVZ/2I3PhgcRuKNNMbO3UmXj1bz9frjXElOKfqHuLhDv0+M5Z6jt8G0TrB7HqSnF/3cotC01qlARu/3SGBeRu93pVR/20ZXeh04J0sri7IhOTkZP79cvuVzcEop/Pz8CjRaLomwEHbMw9WZ4a1rsGJcF766P5xqPh68s3g/7d9fyXt/7CfqUi79iAui2XAY9Td4BcKCUfD1HXBKOnLZkvR+L5jklDSOxl7LWVMvRCklSXDuCvpnI4mwEA7AyUnRo1EVfn68Pb+N6UC30ABmbjhBl49WM+bH7Ww/daloH1ClMYxabUyeu3wGZvaEnx8yJuEJUcrJ0spCOC5JhIVwMM2q+zBpRAvWvdCNRzuGsPZQLIO//IdBX27gj91nSU0rZGmDkxM0HwlPb4MuL8LBpfB5OCx5wWjZJiUTopSSpZWFcFwyWU4IB1XNpxwv9WnIM7fX4+etp/nmnxOM+XE7QT7leLB9LYa1rk5Fj0JMHHKrAN1eNpZ3/vsd2PYNbJ4GnlUgtB806g81O0qXCVFqHDgnSysL4ajkbyIhHFwFdxce7BDCfe1q8XfkeWasP857SyL5bMUhhoZX5542NahXpRAjZd7BMHga9PnQtGrdItg1B7Z+DeV8jWS51aNGCzchbCjyrCytLMqmt37fx/4zl616zkbVKvLGnY3z3GfgwIGcPn2a5ORkxo4dy+jRo1m2bBkvv/wyaWlpVK5cmb///purV6/y9NNPs3XrVpRSvPHGGwwZMsSq8RaVJMJCCACcnRQ9GwfSs3Ege6MT+Hr9cX7YdJJZ/5ygVa1KjGhdgz5hVfFwdS7YiT28jeWamw6FG4lw9G/YMgOW/MfoNtHvU3AtVzwXJUQ+ZGllIQpu5syZ+Pr6kpSURKtWrRgwYACjRo1i7dq1hISEcPHiRQDeeecdvL292bNnDwCXLhVxPkoxkERYCJFDkyBvPh3WnFf7NuSX7VHM2Xya8fN28dbv+xnSMpiRbapTN6AQo8Ru5aHhndCgL6z9CFa/D+f3wrDvjRXvhChhGUsry0Q5URblN3JbXCZNmsSvv/4KwOnTp5k+fTqdO3e+2bvX19cXgBUrVjB37tybx1WqVKnkg82HTJYTQuTKz9Od0Z3r8Pf4Lvz4aBs61qvM7H9PcMcnaxk2bSOLdp3hempawU/s5ARdX4SRPxmdJaZ1gSMrrH8BQuQjY2nl0ECZKCeEJVavXs2KFSvYuHEju3btokWLFjRv3tzWYRWaJMJCiHw5OSna163M5JEt2fjS7bwYEcqZhCSembOD9u+v5IOlBzgVV4iexPV7wejVUDEIvr/LmFx3ejMkXrT6NQhhjiytLETBJCQkUKlSJcqXL8+BAwf4999/SU5OZu3atRw/fhzgZmlEjx49mDx58s1jpTRCCFHmVfZ054mudXisc23WHbnAj5tO8tW6Y0xdc5TO9f25t00NuocGWL6Us29tePQv+H0srPvYeAB4+IBfHfCrC1WaQEgnCGwKTgWsURYiD7K0shAFExERwdSpU2nYsCENGjSgbdu2+Pv7M336dAYPHkx6ejoBAQH89ddfvPrqq4wZM4YmTZrg7OzMG2+8weDBg219CVlIIiyEKBQnJ0WX+v50qe/PuYRk5m45xdzNpxk9exvVvD0Y0boGw1pXJ8DLI/+TuVWAwV9B15fgwmG4eBTijkDcUTixAXb/ZOzn4W20XgvpbCTG/qGSGIsikaWVhSgYd3d3li5dava93r17Z3nt6enJt99+WxJhFZokwkKIIgv09uDZO+rzVLe6rIiM4YdNJ/nfX4eY+PdhejSqwt3h1elUr3Leo8RKmUaA6+R878o5OL4Ojq+BE+vg4B/GdhcP8G8AAY1uPao2Bc+A4rlQYVcyllbu2Ug6RgjhqCQRFkJYjYuzExFNAoloEsjxC9f4cdNJftkezdK956hS0Z3BLYMZelswtf09C3Zir8BbLdjAmGB3YoPRcSJmPxxbbfQoBlDO0OJe6PKC0ctYiFzI0spCCEmEhRDFIqRyBV7p24jne4Wy8sB5ft4axbQ1R5my+ijhNSsxoEUQEY0D8fdyL/jJfWpA8xpZtyVehJhI2P+bsZrdrrnGgh2dxkOFyta5KGFXZGllIYQkwkKIYuXm4kREk6pENKlKzOVkFuyIZv62KF5buJc3fttLmxA/+jatSkSTQCp7FiIpzlDeF2p1MB7tn4LVH8CmKbD9W2g3Blo/BhX8rHdhosyTpZWFEJIICyFKTEBFDx7vYnScOHj+Ckt2n2XxnrO8unAvr/+2l3Z1/Bh6W3UimgQWfAW7zHxqwMAvocNYWPkurPmv8agUAkG3mR4tjS4UbuWtd4GiTJGllYUQkggLIUqcUorQwIqEBlZkXI/6HDx/hT92n+W3nWd49qedVPzNhUEtghjWqgaNqhWhftO/AQybDWd3Gwt2RG+DUxth73xTIM4waCo0vds6FybKDK01B85doWejKrYORQhhQ5IICyFsKktSfEd9/j0Wx9wtp5mz+TTfbjxJ02Bv7g6vzp1Nq+FdvpC9Xqs2NR4ZrpyD6O1GYhzYNPfjhN3RWvPX/vNM/PswF6/d4LaapW/JVyFEyZFEWAhRamSsYNe+bmUuXbvBwp3R/LTlNK8u3Mvbi/fTs1EV7rotmE71/Iv2dbZXIIT2MR7CIWROgPeduUwtv/L8b2gzBrUIsnVoQtg1T09Prl69auswciWJsBCiVKpUwY2HOoTwYPta7Dtzmfnboli4M5rFu88S4OXOoJZBDGgWRMOqXiglNZ7CvNwS4AHNq1m++qEQpdXSCXBuj3XPGRgGvT+w7jlLMUmEhRClmlKKJkHeNAny5qU+oaw6EMP8bdHMWHecaWuOUdu/Av3CqtK3aTUaBEobLGGQBFiI4jFhwgSqV6/OmDFjAHjzzTdxcXFh1apVXLp0iZSUFN59910GDBiQ77muXr3KgAEDzB733Xff8fHHH6OUomnTpsyePZvz58/z+OOPc+zYMQCmTJlC+/bti3Q9SmtdpBMUVnh4uN66datNPlsIUfbFXb3O8n3nWbz7DP8eiyNdQ70AT/qEVaVX48BiHSlWSm3TWocXy8mtQCkVAUwEnIEZWusPsr3/ODAGSAOuAqO11vvzOmdZuWebS4Cf7l5PEmBhNyIjI2nYsKHNPn/Hjh08++yzrFmzBoBGjRqxfPlyvL29qVixIhcuXKBt27YcPnwYpVSepRGpqakkJibmOG7//v0MGjSIf/75h8qVK3Px4kV8fX0ZNmwY7dq149lnnyUtLY2rV6/i7e2d47zm/oxyu2/LiLAQokzy83RnZJsajGxTg9gr11m27xyLd51h0srDTPz7MDV8y9OzURV6Ng7ktpqVHKZFllLKGZgM9ACigC1KqUXZEt0ftdZTTfv3Bz4BIko8WCuSEWAhSkaLFi2IiYnhzJkzxMbGUqlSJQIDAxk3bhxr167FycmJ6Ohozp8/T2Bg3suXa615+eWXcxy3cuVKhg4dSuXKxmJIvr6+AKxcuZLvvvsOAGdnZ7NJcEFZlAgXx+iCEEJYi7+XO/e1rcl9bWsSe+U6KyLP8+e+c3y38SQz1h/Hr4IbPRtXoW9YNdrW9rX3xKg1cERrfQxAKTUXGADcvCdrrS9n2r8CYJuvBq1AEmAhSt7QoUOZP38+586dY9iwYfzwww/Exsaybds2XF1dqVWrFsnJyfmep7DHWVO+ibCjji4IIcomfy93RrSuwYjWNbiSnMKaQ7Es23uO33aeYc7m0/hVcCOiSSD9mlajdYivPY4UBwGnM72OAtpk30kpNQYYD7gB3UsmNOuRBFgI2xk2bBijRo3iwoULrFmzhnnz5hEQEICrqyurVq3i5MmTFp0nISHB7HHdu3dn0KBBjB8/Hj8/v5ulEbfffjtTpkzJtzSiICwZEXao0QUhhP3w8nClX9Nq9GtajeSUNFYfjOH33WdZsD2aHzadwt/LnfcGNqFn47y/vrNHWuvJwGSl1EjgVeCB7PsopUYDowFq1KhR4M/45K9DLN1ztoiRmnfteipnEpIlARbCBho3bsyVK1cICgqiatWq3HPPPdx5552EhYURHh5OaGioRefJ7bjGjRvzyiuv0KVLF5ydnWnRogWzZs1i4sSJjB49mq+//hpnZ2emTJlCu3btinQtliTCVhtdKOpNVQghCsvD1ZmIJlWJaFKVxBuprDwQwx+7z1LNp5ytQ7O2aKB6ptfBpm25mQtMMfeG1no6MB2MyXIFDSTAy516VTwLephFlFKMbxDAQEmAhbCJPXtutW2rXLkyGzduNLtfXj2E8zrugQce4IEHsv77vEqVKvz222+FiDZ3VpssZ8noQlFvqkIIYQ3l3VxujhTboS1APaVUCEYCPBwYmXkHpVQ9rfVh08u+wGGKwb1ta3Jv25rFcWohhLAKSxJhq40uCCGEKF5a61Sl1FPAcowJzjO11vuUUm8DW7XWi4CnlFJ3ACnAJcwMXAghhLXs2bOH++67L8s2d3d3Nm3aZKOIbrEkES41owtCCCHyp7VeAizJtu31TD+PLfGghBBWo7UuUytqhoWFsXPnzhL5rIKuj5FvIiyjC0IIIYQQpYOHhwdxcXH4+fmVqWS4JGitiYuLw8PDw+JjLKoRltEFIYQQQgjbCw4OJioqitjYWFuHUip5eHgQHBxs8f6yspwQQgghRBnh6upKSEiIrcOwG9JzRgghhBBCOCRJhIUQQgghhEOSRFgIIYQQQjgkVdA2E1b7YKViAcsWo86qMnDByuGUJvZ+fWD/12jv1wf2f435XV9NrbV/SQVTGsg9O0/2fo32fn1g/9co15fLfdtmiXBhKaW2aq3DbR1HcbH36wP7v0Z7vz6w/2u09+srSY7wZ2nv12jv1wf2f41yfbmT0gghhBBCCOGQJBEWQgghhBAOqSwmwtNtHUAxs/frA/u/Rnu/PrD/a7T36ytJjvBnae/XaO/XB/Z/jXJ9uShzNcJCCCGEEEJYQ1kcERZCCCGEEKLIJBEWQgghhBAOqcwkwkqpCKXUQaXUEaXUBFvHYw1KqZlKqRil1N5M23yVUn8ppQ6bnivZMsaiUEpVV0qtUkrtV0rtU0qNNW23p2v0UEptVkrtMl3jW6btIUqpTabf15+UUm62jrUolFLOSqkdSqnFptf2dn0nlFJ7lFI7lVJbTdvs5vfUVuztvm3v92yw//u23LPt5vqsds8uE4mwUsoZmAz0BhoBI5RSjWwblVXMAiKybZsA/K21rgf8bXpdVqUCz2mtGwFtgTGm/272dI3Xge5a62ZAcyBCKdUW+C/wqda6LnAJeMR2IVrFWCAy02t7uz6Ablrr5pl6UdrT72mJs9P79izs+54N9n/flnu2fVwfWOmeXSYSYaA1cERrfUxrfQOYCwywcUxFprVeC1zMtnkA8K3p52+BgSUZkzVprc9qrbebfr6C8T9lEPZ1jVprfdX00tX00EB3YL5pe5m+RqVUMNAXmGF6rbCj68uD3fye2ojd3bft/Z4N9n/flns2UMavLw+F+h0tK4lwEHA60+so0zZ7VEVrfdb08zmgii2DsRalVC2gBbAJO7tG01dQO4EY4C/gKBCvtU417VLWf18/A14A0k2v/bCv6wPjL8I/lVLblFKjTdvs6vfUBhzlvm23vyf2et+We3aZvz6w4j3bpTiiE9ahtdZKqTLf304p5Qn8Ajyrtb5s/OPUYA/XqLVOA5orpXyAX4FQ20ZkPUqpfkCM1nqbUqqrjcMpTh211tFKqQDgL6XUgcxv2sPvqSh+9vR7Ys/3bbln2wWr3bPLyohwNFA90+tg0zZ7dF4pVRXA9Bxj43iKRCnlinEz/UFrvcC02a6uMYPWOh5YBbQDfJRSGf/QLMu/rx2A/kqpExhfbXcHJmI/1weA1jra9ByD8Rdja+z097QEOcp92+5+Txzlvi337LLLmvfsspIIbwHqmWY9ugHDgUU2jqm4LAIeMP38APCbDWMpElNd0tdApNb6k0xv2dM1+ptGFVBKlQN6YNTUrQLuMu1WZq9Ra/2S1jpYa10L4/+7lVrre7CT6wNQSlVQSnll/Az0BPZiR7+nNuIo9227+j2x9/u23LOBMnx9UAz3bK11mXgAfYBDGLU8r9g6Hitd0xzgLJCCUbPzCEYtz9/AYWAF4GvrOItwfR0x6nh2AztNjz52do1NgR2ma9wLvG7aXhvYDBwBfgbcbR2rFa61K7DY3q7PdC27TI99GfcXe/o9teGfrV3dt+39nm26Rru+b8s9u+xfn7Xv2bLEshBCCCGEcEhlpTRCCCGEEEIIq5JEWAghhBBCOCRJhIUQQgghhEOSRFgIIYQQQjgkSYSFEEIIIYRDkkRYCCGEEEI4JEmEhRBCCCGEQ/p/+9RFjopZgUIAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "fig, axes = plt.subplots(1,2,figsize=(12, 4))\n", "for ax, measure in zip(axes, ['loss', 'acc']):\n", " ax.plot(hist.history[measure], label=measure)\n", " ax.plot(hist.history['val_' + measure], label='val_' + measure)\n", " ax.legend()\n", "pass" ] }, { "cell_type": "code", "execution_count": 29, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAALEAAAFgCAIAAAB7Xg+sAAAABmJLR0QA/wD/AP+gvaeTAAAgAElEQVR4nO3de1gTV/oH8HcIhEUCagulqKCNVVFUKBoQwVbK0q4FHlgX5CpWvLVpsbWIa/url8el9rF267NWFKG73bVVlou40upiy3KzLbeSilWUVgXLzVVELkHASOb3x9nOkx4ghCRkUvJ+/vDJnJk5cw75OnNmkswwLMsCQirM+G4AMjqYCUTDTCAaZgLRzFUnysrKPvjgA76agvji7e39xhtvcJO/2E80Njbm5OQYvEmIT+Xl5WVlZaol5oMXys7ONlR7EP/Cw8OpEhxPIBpmAtEwE4iGmUA0zASiYSYQDTOBaJgJRMNMIBpmAtEwE4iGmUA0zASiYSYQbYjPykd048aN5OTkPXv2TJs2Te8N0s6tW7euXr26fPlyTRYuLS1tbm7mJidNmrRixYqxahkAAHzxxRd3797lJhcuXOjq6jqmW9SFNvsJmUz28ccff//993pvjRbu3LmzdetWsVh86tQpDVdZsmSJlZVVdHR0dHR0W1ubhknSxVNPPVVeXh4dHb169erHH3981qxZY71FXWiTibCwsDt37ozp/61jx45puGRDQ0NcXFxvb6/mlQuFwpCQkEmTJgFAbGyslZWVNk3UANcLe3v7uLg4AHB3d/fz8xMKhWO0Rb3QcjxhZ2en33aoKiwsfPPNNzVcWCKRuLi4jHYTDMPY2NgAwMSJE0e7roaoXpDNWVtbj9Hm9Eib8YRSqSwpKRGJRBKJBAAaGxtzc3MTEhJqa2tPnz7t7OwcExNjZmYGAE1NTXl5eS+//HJJScm5c+emTp26bt06Kyurzz777Pr16yKRaP369d3d3ceOHVMoFI6OjhEREUVFRaGhoQzDHD16dMqUKcHBwdp1rK2tLT09PT4+3sHBQZPljaEXP/zwQ3l5+cWLF318fH7/+98DwH/+85/GxkYAsLS0XLlypaWlZWVlZW1t7eTJk0NCQgCgpaUlPz+/qanJx8fH39+f1HPv3r2MjAypVPrvf//74sWLiYmJ5uajeaNZFZmZmVTJYJcvXw4LCwOAI0eOsCybl5dnb28PAAcOHFi7dm1QUBAA7N27l2XZTz/9dPLkyVZWVi+99FJ8fPwLL7wAABKJ5MGDByzLurq6Tps2jdTZ1dVla2vr7e3Nsux3333n4+Njb29fVFT03XffqW8M0d/fDwCbN29WLUxPTweAgwcPDreWk5MTAAwMDBimF3V1dQDw9NNPD9eeAwcOLF++XKlU1tfXz5gx4/DhwyzL9vT0kNHo9evXuSVdXFzq6upYli0sLNywYYNMJsvKyhKJRFKplGXZv//97xMmTDA3N//www/d3NwAoKamRs1fLywsLCwsTLVk1JlgWfbixYtcJliW3b59OwAUFBSQSQ8Pj0WLFpHXsbGxDMNcunSJTO7YsQMAUlNTSVO4vyZZi/w1WZYNDQ11cnIasRmcITMhl8tPnDjR1dU13FqqmTBAL0bMxJNPPvnKK69w677wwgvkdV5eHgCkp6eTyZaWFvIWdnd3i8ViuVxOytetWwcAZWVlLMvGxMQAQG5uLsuyV65cGW6LxOBMaDOesLS0VJ0kYzTuoD5v3ryffvqJvLa2tjY3N+fOu7Zv325ubl5aWjriJhiG0aJhqqytraOioshRXBO896K4uDg5ORkAamtrGxsbf/zxR1IeFBQ0d+7cDz74gGVZADhx4gQZrmZkZPT29m7btu2VV1555ZVXWltbZ86cee3aNQCYMmUKAJCDixaDLW3GE+oJBAJ2mN+qT5gwYdq0aXfu3BmxEt0zoSPD92Lq1KlffPHF559//swzz8ycObO6upqrJCkpKT4+/uzZs4GBgQUFBa+99hoAXL582dHRMSUlZXBVZBhE/tWCQa9j9vf337p1SywWj7gk75lQQ++9uH37dn9//44dO5KTk/ft2/eHP/xBIBCoLhATEzN16tQ///nPly9fdnV1JQNGgUBQV1enUCi064UaBs1EeXl5X18fGcGZm5v39fUNuRjDMAMDA4Zs2KjovRcbNmz46aefkpOTuYslSqVSdQGhUPj6668XFRUlJSWtXbuWFLq5ufX09KSmpnKLdXR0HD58WLtOqdImE2RM19bWRia7uroA4MGDB2Syra2tv7+f2/E+fPjwypUr5HVOTs4zzzxD/prPPfdcW1vbxx9/3NPT8/HHH9+9e/fGjRv37t0DAEdHx1u3bt24ceP69es9PT0jtoesRb031dXVnp6excXFw61Fmk3+NUAvbt68qVo/cf/+/c2bN3PBysjI6OrqOn/+fGlp6b179+RyeXd3N1ly06ZNEydObGtr44Y1ERERTk5OW7du3b9//5UrV7KysjZu3Lh69WoAIH801avpo6M64NTkvKO8vJyci86fP//zzz8vLi4me9H169e3trZmZGTY2toCwO7duxUKxaZNmwQCwauvvpqUlBQZGRkcHMydCHR3dy9ZsgQA5s6dm5ubu3Llyueff56MrouKiszNzSdNmqTmTJJz9uzZiIgIAHjsscfS09NbW1tJ+cmTJxmG4Ybrqr788sv169eT7q9cufLkyZNj3Yvjx497enoCAMMwXl5e/v7+S5cudXV1tbCwAIC0tDSWZePj483NzZ988snU1NScnByhUPjss8/evXuXa/ZLL72UkpKi2pHa2trZs2eTjri6uspkMpZlP/roo6lTpwLAqlWrKioqRvwD6udcVHObNm2ysLBgWfann37q7OwcvMDt27fJi97eXtXyjo4ONaeRGhpyi1owWC9UF+7r66PmBgQE3Lt3b/BaDQ0NN2/e1HwrlMGZ0P95x5DIxYDByJUiAPjNb36jWs5dcpZKpcPVuXHjRnd3dzUbJf/X9UjrXmhI9cyZOuGvqakRi8XkMxrK9OnTR7WVEY1tJu7fv//w4UO5XC4SibSrwc/Pb7hZ3Dsx1nTvhdaqq6u3bdu2YMGC4uLif/3rXwbaqupOQ7/Hjk8//ZR81iCVSjW8Sm2E+O1FZWWljY3NxIkTs7KyxmgTBj12BAUFBQYGktfUnvBXhN9eSCSS9vZ2MzMzrS9AaWEMMzF2H0MbEu+9GN1HmvqA38dENMwEomEmEA0zgWiYCUTDTCAaZgLRMBOIhplANMwEomEmEA0zgWhDfL4y+EbtaBwrLy8nXx/k/GI/4eTkRL5raYJKS0s1+cnG+LNkyRJvb2/VEobFZ0kCAADDMJmZmatWreK7IfzD8QSiYSYQDTOBaJgJRMNMIBpmAtEwE4iGmUA0zASiYSYQDTOBaJgJRMNMIBpmAtEwE4iGmUA0zASiYSYQDTOBaJgJRMNMIBpmAtEwE4iGmUA0zASiYSYQDTOBaJgJRMNMIBpmAtEwE4iGmUA0zASime59ajZt2kSeK098/fXXc+bMsbOzI5MCgeAf//jHtGnTeGodnwz9DBnj8dhjj6WlpamWXL58mXv9xBNPmGYgwJSPHTExMcPNEgqFL774ogHbYlxM99gBAK6urleuXBnyL1BXV8c94tfUmO5+AgDi4uIEAgFVyDDMwoULTTYQYOKZiI6OHhgYoArNzc3XrFnDS3uMhEkfOwBgyZIlVVVVSqWSK2EYprGxkTwG3jSZ9H4CAOLi4hiG4SbNzMx8fHxMORCAmaBunMswTFxcHF+NMRKmngk7Ozt/f3/VkebKlSt5bI8xMPVMAEBsbCwZVAkEgt/97nePPvoo3y3iGWYCQkNDLSwsAIBl2djYWL6bwz/MBNjY2AQHBwOAUCgkL0ycTp93ZGVl6asd/JoxYwYAeHh4nDlzhu+26MfSpUu1/7yG1YFee4H0KTMzU+u3Vddjhy7bNiqJiYn9/f18t0I/dHxPcTzxP3/605+EQiHfrTAKmIn/sbKy4rsJxgIzgWiYCUTDTCAaZgLRMBOIhplANMwEomEmEA0zgWiYCUTDTCAaZgLRDPobYrlcXlRU9NVXX+3bt8+Q21Xv7t27aWlpb7755ohLlpaWNjc3c5MWFhb29vZTpkyZNWvWWDbQ0Ay6n8jPz9+8efM///lPQ250ROvXr//LX/6iyZILFy68fv16dHT0iy++2NXVdefOnc8++ywiIuKJJ554++23FQrFWDfVQHT87sZov1OzatUqsVisy0b1Ky0tbdasWQ4ODhou39jYCABz587lSpRKZXZ2tq2tbUBAQFdX19g0c3S0eF9UGXo8YWZmZmZmLIOYH3744bvvvgsKCtJ8FVtbW6qEYZiwsLC0tLQvv/xy2bJlDx480GsbeWCI8UR7e3tOTk5DQ8PixYtZllX9LV5LS0t+fn5TU5OPj4+/vz8pbGxszM3NTUhIqK2tPX36tLOzc0xMDEkSy7IlJSUXLlwQCAQuLi4BAQFq6lFPoVC8/fbbf/3rX3ft2qVa3tbWlp6eHh8f7+DgoHkfIyIijh07dvbs2crKSl9fX367pqux3kddvXpVIpF88803CoXi6NGjlpaWs2fPJrMKCws3bNggk8mysrJEIpFUKmVZNi8vz97eHgAOHDiwdu1a8p947969ZJW33norPT2dZdmqqipPT0819Yzo7bff/vrrr1mW3bJli+qxIz09HQAOHjw45FqdnZ3wy2MHZ8+ePVxT+e2aJu+LutW1XlPDbXt5eSUlJZHXSqVSLBaTTHR3d4vFYrlcTmatW7cOAMrKyliW3b59OwAUFBSQWR4eHosWLSKr29nZFRUVkfLk5GT19ahRXFy8e/du8prKhFwuP3HixHAjAzWZyM3NBYAVK1bw2zVW50yM7bGjsLCwoqKC2zkzDCORSC5cuAAAGRkZvb2927ZtI7NaW1tnzpx57dq1JUuWkK9Guri4kFnz5s07d+4cWX3OnDkRERFpaWkhISFbt25VX89wrero6Dh06FBGRsaQc62traOiorTorFwuJ6vz2DW9GNtM1NTUAMD8+fO5Em4wcfnyZUdHx5SUlBErEQgE7M/fTz906FB4eHhoaKi/v//x48cdHBw0r4ezZcsWiUSSl5dHJn/88ce+vr7c3NxJkyY9++yzmtdDkclkAODl5cVj1/RibDPR1dUFABUVFU5OTlwhiYVAIKirq1MoFOS3mhpyd3eXyWTbt28/evSoh4fH999/r0U9d+7c+fLLL7nJzs7O+/fvb9682dXVVetMsCx7/vx5gUAQEBBw7NgxvrqmF2N7WrhgwQIAKCwsHDzLzc2tp6cnNTWVK+no6Dh8+LCa2vr7+z/55BMbG5uUlJQzZ860trbm5uZqUc/nn3/epOLll1+2t7dvamoiu3HtbNmypbq6ev/+/W5ubjx2TT+0HoloMpZRKBQuLi4ikaikpIRl2ebmZkdHR5FIVFNTI5fLnZychELhe++9V1tbm5mZGR4eTkZ2iYmJAHDjxg1SSWBgoI2NjVKp7O3tXbp0qVKpZFlWqVTa29ufOnWqr69vuHo0lJSUpDrG/PbbbyUSCTfco5Cj4YwZM7iS+vp6qVTKMExCQgIpUdMkw3RtxPdlhNW1XlPDbdfX10skEgAQi8XR0dHBwcG+vr5Hjhzp7e2tra3lbi/n6uoqk8lYli0uLhaLxQCwfv361tbWjIwMcplo9+7d3d3djo6OkZGR2dnZ77///s6dO8kmhqxHc1QmTp48yTAMOS2k5OXlLV++nGzI29s7ICAgMDAwJCQkMTGxqqpKdUl+u2bsmSBu375Nzqm6u7upWQ0NDTdv3tRwiwqFor+/f8jlR1WPep2dnXqph6+u6ZgJA30uSq7VAIBIJKJmTZ8+XfN6zM3NAcDZ2XnwLKoeqVQ6XCUbN250d3dXs5XBF7C1M0ZdG2vj9n7bfn5+w83iAoqGNG4zER4ezncTfq2M5SNKZDwwE4iGmUA0zASiYSYQDTOBaJgJRMNMIBpmAtEwE4iGmUA0zASiYSYQDTOBaLp+Vl5WVqaXdiAjovU3tFh8focR0+W7d6b+zFkOwzCZmZnUoyVNE44nEA0zgWiYCUTDTCAaZgLRMBOIhplANMwEomEmEA0zgWiYCUTDTCAaZgLRMBOIhplANMwEomEmEA0zgWiYCUTDTCAaZgLRMBOIhplANMwEomEmEA0zgWiYCUTDTCAaZgLRMBOIhplANMwEomEmEG3cPvtpRBkZGd3d3aolBQUFHR0d3GRoaOhjjz1m8Hbxz3TvXbRmzZpjx45xj31WKpUMw5DnZg8MDFhbW9+5c8fS0pLXNvLDdI8dUVFRAKD42cDAwMOHD8lrgUAQHh5umoEAU95PPHz40MHBob29fci5BQUF/v7+Bm6SkTDd/YS5uXlUVBR37FD16KOPcs+gNkGmmwkAiIqKUigUVKFQKFy9erVAIOClScbAdI8dAMCy7LRp01paWqjyiooKT09PXppkDEx6P8EwTFxcHHX4cHJykkgkfDXJGJh0JmDQ4cPCwuLFF18kZ6Qmy6SPHYSLi0tdXR03eenSJVdXVx7bwztT308AwOrVq7nDx7x580w8EICZAICoqKiHDx8CgIWFxZo1a/huDv/w2AEAsHjxYplMBgD19fXTp0/nuzk8w/0EAEBcXBzLsp6enhgIAHymyzilyzNddP2s/PXXX/f29tZLN/j17rvvSqXSiRMn8t0QPYiIiNBldV0z4e3tPT6ejfPUU0/NmjWL71boh46ZwPHE/4ybQOgOM4FomAlEw0wgGmYC0TATiIaZQDTMBKJhJhANM4FomAlEw0wgGmYC0TATiGbQew3I5fKioqKvvvpq3759htzucM6cOdPV1UVeNzY2vvrqqxMmTFCzfGlpaXNzMzdpYWFhb28/ZcqUcfaZqkEzkZ+fn5SUpFQqjSETV69eDQ4O5r4tFhkZqT4QALBw4cLS0tIdO3YIhcKDBw8qlcry8vLCwsJ79+7FxMTs2rVryF+f/vro+N270X7Ha9WqVWKxWJeN6suGDRuKiop++llvb68mazU2NgLA3LlzuRKlUpmdnW1raxsQENDV1TVm7R0FLd4XVYa+T42ZmZmZGf+DmFu3bl28eHHnzp3Tpk0b1Yq2trZUCcMwYWFhAwMDkZGRy5Ytq6ysFAqF+mspDwyRifb29pycnIaGhsWLF7Msq/rLu5aWlvz8/KamJh8fH+5+D42Njbm5uQkJCbW1tadPn3Z2do6JiSFJYlm2pKTkwoULAoHAxcUlICBATT1qfPjhhxUVFU5OTk888cTOnTvXrFnDtaqtrS09PT0+Pt7BwUHzPkZERBw7duzs2bOVlZW+vr48dk0PxnofdfXqVYlE8s033ygUiqNHj1paWs6ePZvMKiws3LBhg0wmy8rKEolEUqmUZdm8vDx7e3sAOHDgwNq1a4OCggBg7969ZJW33norPT2dZdmqqipPT0819ah37ty5pKQkX19fMgL47W9/+/DhQzIrPT0dAA4ePDjkip2dnfDLYwdnz549XFN57Bqr87FjzDPh5eWVlJREXiuVSrFYTDLR3d0tFovlcjmZtW7dOgAoKytjWXb79u0AUFBQQGZ5eHgsWrSIrG5nZ1dUVETKk5OT1dejiQsXLri4uADAu+++S0rkcvmJEyeGGxmoyURubi4ArFixgveu6ZiJsT12FBYWVlRU7Nq1i0wyDCORSC5cuAAAGRkZvb2927ZtI7NaW1tnzpx57dq1JUuWWFlZAQB5qwBg3rx5586dI6vPmTMnIiIiLS0tJCRk69at6uvRpIVubm7V1dVz5szJyMggb5i1tTW51dVoyeVysrqRdE1rY5uJmpoaAJg/fz5Xwh22L1++7OjomJKSMmIlAoGA/fmM8dChQ+Hh4aGhof7+/sePH3dwcNC8nuFMmDAhJCTkb3/7m9Y1EOTXhV5eXsbTNe2M7SkAuSJUUVGhWkhiIRAI6urqBt86SD13d3eZTCaVSouLiz08PNrb27Wrh+Li4jJ79mxdamBZ9vz58wKBICAgwKi6poWxzcSCBQsAoLCwcPAsNze3np6e1NRUrqSjo+Pw4cNqauvv7//kk09sbGxSUlLOnDnT2tqam5urRT2DnTp1KiQkZFSrULZs2VJdXb1//343Nzej6po2tB6JaDKWUSgULi4uIpGopKSEZdnm5mZHR0eRSFRTUyOXy52cnIRC4XvvvVdbW5uZmRkeHk5GdomJiQBw48YNUklgYKCNjY1Sqezt7V26dKlSqWRZVqlU2tvbnzp1qq+vb7h6hlNXV/faa6/JZDIyeenSJS8vrwcPHpDJb7/9ViKRcMM9Cjkazpgxgyupr6+XSqUMwyQkJJASNU0a664RI74vI6yu9Zoabru+vp7cHkosFkdHRwcHB/v6+h45cqS3t7e2tpbbY7u6upI3qbi4WCwWA8D69etbW1szMjLIZaLdu3d3d3c7OjpGRkZmZ2e///77O3fuJJsYsh41qqurye9C/fz8/vjHP+7bt+/+/fvc3JMnTzIMQ04LKXl5edw9Er29vQMCAgIDA0NCQhITE6uqqlSX5KtrhLFngrh9+zY5p+ru7qZmNTQ03Lx5U8MtKhSK/v7+IZcfVT19fX0//PBDU1PTkHM7Ozs1rEc9XrrGGvm5KIdcqwEAkUhEzRrVLR/Mzc0BwNnZefAsqh6pVDpcJRs3bnR3d1fzYebgC9jaGaOujbVxe19+Pz+/4WZxAUVDGreZCA8P57sJv1b8f0SJjA1mAtEwE4iGmUA0zASiYSYQDTOBaJgJRMNMIBpmAtEwE4iGmUA0zASiYSbQIFp/G4fF53cYMd6+Z5WZmamvPvAuIiJi3DyLBACWLl2q9br4PLD/YRgmMzNzfDyLREc4nkA0zASiYSYQDTOBaJgJRMNMIBpmAtEwE4iGmUA0zASiYSYQDTOBaJgJRMNMIBpmAtEwE4iGmUA0zASiYSYQDTOBaJgJRMNMIBpmAtEwE4iGmUA0zASiYSYQDTOBaJgJRMNMIBpmAtEwE4g2bp/zM6KOjg7qfiw9PT337t3jJkUiEXnCvakx3fvU+Pn5FRcXDzdXIBA0NTU9/vjjBmyRsTDdY0dUVBT38HSKmZnZ008/bZqBAFPORHh4uEAgGHIWwzBxcXEGbo/xMN1MTJ48+bnnnhsyFmZmZqGhoYZvkpEw3UwAQGxsrFKppArNzc1feOGFSZMm8dIkY2DSmQgJCbG0tKQKlUplbGwsL+0xEiadiQkTJoSGhlInnJaWloGBgXw1yRiYdCYAICYmRqFQcJMWFhbh4eFWVlY8Nol3pp6J559/XvWJ9QqFIjo6msf2GANTz4SFhUVUVJRQKCSTkyZN8vf357dJvDP1TABAVFTUgwcPAMDCwiImJsbc3HSv9xOme22bo1Qqp0yZ8t///hcAzp8/7+vry3eLeIb7CTAzMyMnn46Ojj4+Pnw3h3867SfDw8P11Q5+kY9DbW1tx82zGt544w2tn0Wi034iJyenqalJlxqMxOTJk21tbZ2dnfluiH7k5OQ0NjZqvbqu46ktW7aMj/9bWVlZ46MjADDc570awvHE/4ybQOgOM4FomAlEw0wgGmYC0TATiIaZQDTMBKJhJhANM4FomAlEw0wgGmYC0TATiGbQ7x7K5fKioqKvvvpq3759htyuejU1NaWlpUKhMDAwcNq0aWqWLC0tbW5u5iYtLCzs7e2nTJkya9assW+m4Rg0E/n5+UlJSUql0kgy0dbWtn379paWltTUVE2+ULNw4cLS0tIdO3YIhcKDBw8qlcry8vLCwsJ79+7FxMTs2rVrnNyvgtUBAGRmZo5qlVWrVonFYl02qi/19fV2dnaxsbGjWot8f2nu3LlciVKpzM7OtrW1DQgI6Orq0ncztaHF+6LK0N9bNzMzMzPjfxDz4MGDVatWPfLII6mpqaNaUfUHQgTDMGFhYQMDA5GRkcuWLausrOR+LfIrZYhMtLe35+TkNDQ0LF68mGVZ1W+GtbS05OfnNzU1+fj4cD+2aWxszM3NTUhIqK2tPX36tLOzc0xMDEkSy7IlJSUXLlwQCAQuLi4BAQFq6lHj//7v/6qqqj766CNra2tqVltbW3p6enx8vIODg+Z9jIiIOHbs2NmzZysrK8mvAfjqmh6M9T7q6tWrEonkm2++USgUR48etbS0nD17NplVWFi4YcMGmUyWlZUlEomkUinLsnl5efb29gBw4MCBtWvXBgUFAcDevXvJKm+99VZ6ejrLslVVVZ6enmrqUW/q1Knm5uavvfaan5+ftbX1smXLqquryaz09HQAOHjw4JArdnZ2wi+PHZw9e/ZwTeWxa6zOx44xz4SXl1dSUhJ5rVQqxWIxyUR3d7dYLJbL5WTWunXrAKCsrIxl2e3btwNAQUEBmeXh4bFo0SKyup2dXVFRESlPTk5WX89wyHfN3d3d7969y7JsXV2do6OjSCRqampiWVYul584cWK4kYGaTOTm5gLAihUreOwaoWMmxvbYUVhYWFFRsWvXLjLJMIxEIrlw4QIAZGRk9Pb2btu2jcxqbW2dOXPmtWvXlixZQn7W7eLiQmbNmzfv3LlzZPU5c+ZERESkpaWFhIRs3bpVfT3DtUomkwFAaGjoI488AgCzZ8/+4IMPoqKiDh8+/M4771hbW0dFRWnRWblcDgDW1tY8dk0vxjYTNTU1ADB//nyuhBtMXL582dHRMSUlZcRKBAIB+/MPGA8dOhQeHh4aGurv73/8+HEHBwfN6+FMnDgRAOzs7LgS8vOYuro6zSsZjETNy8uLx67pxdieAnR1dQFARUWFaiGJhUAgqKurU733gybc3d1lMplUKi0uLvbw8Ghvb9eintmzZwNAdXU1V+Ls7GxhYWFjYzOqxqhiWfb8+fMCgSAgIIDHrunF2GZiwYIFAFBYWDh4lpubW09Pj+qpYEdHx+HDh9XU1t/f/8knn9jY2KSkpJw5c6a1tTU3N1eLeh5//PHnn3++vLycK/nxxx8VCoUuPxbdsmVLdXX1/v373dzceOyafmg9EtFkLKNQKFxcXEQiUUlJCcuyzc3NZDRXU1Mjl8udnJyEQuF7771XW1ubmZkZHjIxFBYAAAH0SURBVB5ORnaJiYkAcOPGDVJJYGCgjY2NUqns7e1dunSpUqlkWVapVNrb2586daqvr2+4etS4dOmSSCT6+uuvyWRqaurcuXMVCgXLst9++61EIuGGexRyNJwxYwZXUl9fL5VKGYZJSEggJWqaZICuscZ/3lFfXy+RSABALBZHR0cHBwf7+voeOXKkt7e3traW7MYBwNXVVSaTsSxbXFwsFosBYP369a2trRkZGeQy0e7du7u7ux0dHSMjI7Ozs99///2dO3eSTQxZz4hqamr8/f137tz5zjvvBAUFtbS0kPKTJ08yDENOCyl5eXnLly8nG/L29g4ICAgMDAwJCUlMTKyqqlJdkt+uGXsmiNu3b5Nzqu7ubmpWQ0PDzZs3NdyiQqHo7+8fcvlR1cNpbm5ub2+nCjs7O0dbz5D46pqOmTDQtW1yrQYARCIRNWv69Oma10NuIjPk51VUPVKpdLhKNm7c6O7uTl5PmTJl8AKDL2BrZ4y6NtbG7X16/Pz8hpvFBRQNadxmYtzcL8Xw+P+IEhkbzASiYSYQDTOBaJgJRMNMIBpmAtEwE4iGmUA0zASiYSYQDTOBaJgJRMNMIJqun5UfOHAgOztbL01BRkKnTISFhemrHUiPwsLCnJyctF4dnweGaDieQDTMBKJhJhANM4Fo/w8tl7NYQPV+QAAAAABJRU5ErkJggg==\n", "text/plain": [ "" ] }, "execution_count": 29, "metadata": {}, "output_type": "execute_result" } ], "source": [ "plot_model(model)\n", "Image('model.png')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Flexibility of functional API\n", "\n", "You can easily implement multiple inputs, multiple outputs or skip connections. Note that if you have multiple outputs, you probably also want multiple loss functions given as a list in the compile step (unless the same loss function is applicable to both outputs)." ] }, { "cell_type": "code", "execution_count": 30, "metadata": {}, "outputs": [], "source": [ "input = Input(shape=(4,))\n", "x1 = Dense(8)(input)\n", "x2 = Dense(4, activation='relu')(x1)\n", "x3 = concatenate([input, x2])\n", "output = Dense(3, activation='softmax')(x3)\n", "model = Model(inputs=[input], outputs=[output])" ] }, { "cell_type": "code", "execution_count": 31, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPMAAAHBCAYAAAC8M40tAAAABmJLR0QA/wD/AP+gvaeTAAAgAElEQVR4nO3deVhTZ/o38G9IgCqL1ooWFZegiCKgUkRc6ja0Y13HCriN1t1itYvLWKcudagzbnXGV61KV61aFrFi3S0IWsUFFBSQUREBRRFBIYgQyP3+4ZifkTWQ5CSH+3NdXMo5J89z5yRfzn6OhIgIjDFTF2omdAWMMd3gMDMmEhxmxkSCw8yYSMiELqA2vv76a5w7d07oMhir1GeffQZvb2+hyzCNJfO5c+cQGxsrdBmMVRAWFobMzEyhywBgIktmAOjduzdCQ0OFLoMxDRKJROgS1ExiycwYqxmHmTGR4DAzJhIcZsZEgsPMmEhwmBkTCQ4zYyLBYWZMJDjMjIkEh5kxkeAwMyYSHGbGRILDzJhIcJgZEwmTuQRSW2lpaQgMDMSqVavQpk0bocvRSmFhIfbs2YPbt2+jY8eOmDBhAho3bqxVGzExMbh7967GsKZNm2Lo0KG6LFVrx48fx6NHjzSGubm5wcXFRaCKRIRMwNixY2ns2LFavSY0NJQA0OHDh/VUlX5cv36d3nzzTerUqRNZWFgQAHJ0dKTs7Gyt2ikpKaH9+/cTAAJAmzZtoqdPn+qp6trLycmh+fPnEwCSSqUUGRlJJSUlQpdVZwAoODhY6DKIiEJEG2YioocPH+qhmtr76aeftH7N0KFDKSEhgYief/FnzJhBAGjatGlat6VSqahp06YEgPLy8rR+va68Oh8uXbpEAMjDw0OginTHmMIs6m3m5s2bC9Z3ZGQkPv/8c61eExcXh4kTJ8LNzQ0AYGdnh1WrVsHMzAxnz57VugaJRAIbGxsAQJMmTbR+vS5UNh9e1GRlZSVESaIl2m1mlUqF6OhoWFtbw9PTEwCQmZmJ8PBwzJs3D8nJyThw4ADatm2LiRMnwszs+d+1rKwsRERE4MMPP0R0dDSOHTuG1q1bY/r06WjUqBEOHjyIW7duwdraGjNmzEBhYSF27twJpVIJe3t7+Pv7IyoqCqNHj4ZEIsH27dvRqlUrjBgxosaa27dvj549e2oMs7e3h4eHB2Sy//uocnNzERQUhGnTpqFly5Zazxtjnw+v+u9//4vY2FgkJiaib9+++Mtf/gIA+P3339X337K0tMSYMWNgaWmJCxcuIDk5Ga+//jpGjRoFALh37x6OHj2KrKws9O3bF0OGDFG3n5+fj7179yIgIABHjhxBYmIiFixYoDHPTYLQ6wa1oe1qdlJSEo0dO5YA0DfffENERBEREWRnZ0cAaOPGjTR16lQaPnw4AaDVq1cTEdHPP/9Mr7/+OjVq1IjmzJlD06ZNo/fee48AkKenJ5WWlhIRkYuLC7Vp00bdX0FBAdna2pK3tzcREV2+fJn69u1LdnZ2FBUVRZcvX67X+3/zzTdp1apV6t+DgoLU28E1cXBwIABUXl5uNPMhNTWVANDbb79dY/0bN26kgQMHkkqlotu3b1P79u1p69atRERUVFRELi4uBIBu3bql8TpnZ2dKTU0lIqLIyEiaOXMmxcfHU0hICFlbW1NAQAAREf3444/UuHFjkslk9P/+3/8jd3d3AqDe1KkJjGg1W5RhJiJKTEzUCDMR0ZIlSwgAnTx5Uj2sZ8+eGttukyZNIolEQteuXVMPW7ZsGQGgbdu2qet5+Uv8op0XX2IiotGjR5ODg4NWNVcmOjqa2rRpQ4WFhephCoWC9uzZQwUFBTW+/tUwEwk/H7QJc8eOHWnu3Lka7b333nvq3yMiIggABQUFqYfdu3dP/X0pLCwkuVxOCoVCPX769OkEgM6dO0dERBMnTiQAFB4eTkREKSkpNdb1gjGFWbTbzJaWlhWGNWrUCADg7OysHta1a1dkZGSof7eysoJMJtM4VLJkyRLIZDLExMRoVUN979xYXl6O5cuXIyIiAtbW1ho1jh8/Xr3tqS1Tmg+nTp1CYGAgACA5ORmZmZm4ceOGevzw4cPRpUsXfP3116D/PQNxz549mDx5MgBg7969KC4uxuLFizF37lzMnTsX2dnZcHR0xM2bNwEArVq1AgD1KvnL88WUmNhGge5JpVL1l6AqjRs3Rps2bfDw4UOt2q5vmBcuXIjPPvsMPXr0qFc7tWGs86F169Y4fvw4fvvtNwwYMACOjo6Ii4vTaHvRokWYNm0aDh8+jGHDhuHkyZP4+OOPAQBJSUmwt7fHli1bquzjxX6CF/+aKtOu3kBKSkpw//59yOVyrV5Xny/xjh070KNHD4wcObLObeiaIedDTk4OSkpKsGzZMgQGBmLNmjV4//33IZVKK0w7ceJEtG7dGhs2bEBSUhJcXFzUO6+kUilSU1OhVCq1rsHUcJhrITY2Fs+ePcPw4cMBADKZDM+ePav2NRKJBOXl5XXqb//+/SAi9ariC9HR0XVqT1cMOR9mzpyJjIwMBAYGYtKkSepNA5VKVWFaCwsLfPLJJ4iKisKiRYswdepU9Th3d3cUFRVh27ZtGq95/Pgxtm7dqnVdxky0YS4pKQHw/DDOCwUFBQCA0tJS9bDc3FyUlJRorGKWlZUhJSVF/XtYWBgGDBig/hK/8847yM3NxQ8//ICioiL88MMPePToEdLS0pCfnw/g+SGl+/fvIy0tDbdu3UJRUVGt6j558iTWrFkDpVKJzZs3Y/PmzfjPf/6D2bNnIzExEcDz49G9evXCqVOnamzvxXt+8a8xzIc7d+5U6P+Fp0+fYv78+Rp/KPbu3YuCggKcPn0aMTExyM/Ph0KhQGFhofp1s2fPRpMmTZCbm6uxne/v7w8HBwcsXLgQ69atQ0pKCkJCQjBr1iz89a9/BQD1Z/PqaaYmR8jdb7Wl7d7s2NhY9aGpbt260W+//UanTp0iuVxOAGjGjBmUnZ1Ne/fuJVtbWwJAK1euJKVSSbNnzyapVEofffQRLVq0iMaNG0cjRozQ2HNcWFhIvXv3JgDUpUsXCg8PpzFjxtC7776r3qsaFRVFMpmMmjZtWqtDSEREcXFxZGVlpT4F8+Wf1157jR49ekRERPv27SOJRKKxB/dVJ06cUJ89BoDGjBlD+/btE3w+7N69m3r16kUASCKRkJeXFw0ZMoT69OlDLi4uZG5uTgBox44dREQ0bdo0kslk1LFjR9q2bRuFhYWRhYUFDR48WD0/XpgzZw5t2bKlwrxITk4mJycn9bxwcXGh+Ph4IiL69ttvqXXr1gSA/Pz86Pz587X6rF6AEe3NFmWY62P27Nlkbm5OREQZGRn05MmTKqfNyclR/7+4uLjC+MePH9fq8FFdVFeXLhjTfHj1tc+ePat0Oh8fH8rPz6+ynfT0dLpz506d66iMMYW5we/Nro6Dg0O14+3s7NT/f+211yqMf/kUyoCAgBr7mzVrFrp3716r2mxtbWs1nS7ocj7UxauH4Co77JiQkAC5XI6mTZtW2U67du3qVYex4zC/4unTpygrK4NCodA4tltfgwYNqnGal0MhNH3NB12Ki4vD4sWL4erqilOnTuHXX38VuiRBcZhfsnv3bhw/fhxEhL/97W+YOXNmrZeUNfH19dVJO4agz/mgSyqVChcvXkRcXByCgoLQvn17oUsSFIf5JcOHD8ewYcPUv1e2OtcQmMp88PT0RF5eHszMzEz+hA9d4DC/RKjLBI2NKc0Hk7uySY/4zxljIsFhZkwkOMyMiQSHmTGR4DAzJhIcZsZEgsPMmEhwmBkTCQ4zYyLBYWZMJDjMjIkEh5kxkTCZs9RjY2NN6jJCxgzNJJbM3t7e6N27t9BlGJ2CggIcO3YMT58+1VsfDx8+xNWrV/XWvqkbO3ZsjXdiMRQJUQ13PmdGa/jw4cjKykJ8fLzeruf99ddfMWbMGKSkpKBz58566YPpRKhJLJlZRdHR0Th06BDWrVun1wvzR44ciQ4dOojuHtNixEtmE0RE6NWrF5o3b44jR47ovb8NGzbgyy+/RGZmpknduKCB4SWzKdq7dy/i4+OxevVqg/Q3ffp0EBF+/PFHg/TH6oaXzCamtLQUXbt2xdtvv43vv//eYP3OnTsXR44cwY0bNyp93hMTHC+ZTc2mTZtw7949fPnllwbt95NPPsGdO3dw+PBhg/bLao+XzCYkPz8fnTp1wuzZs/HVV18ZvP+hQ4eirKwMJ06cMHjfrEa8ZDYlX331FSQSCRYvXixI//Pnz8fJkyfVD7BjxoXDbCLS09OxefNmrFixQrA9yn/+85/h7Oxc7YPLmXA4zCZi6dKlaN26NWbNmiVYDRKJBB999BF27dpl+o8/FSEOswm4cuUKgoODsXbtWlhYWAhay5QpU2BhYcGHqYwQ7wAzAX/605+gUChw7tw5SCQSoctBQEAAIiMjkZKSYhT1MAC8A8z4/fbbb/j999+xfv16ownOnDlzkJqaitOnTwtdCnsJL5mNWHl5Odzd3dG5c2fs27dP6HI0eHp6wtnZGbt27RK6FPYcL5mN2ffff4/U1FQEBgYKXUoFM2fORFhYGPLy8oQuhf0Ph9lIFRcXY9WqVZgzZw66dOkidDkVTJgwAebm5vj555+FLoX9D4fZSK1btw5PnjzBF198IXQplbK2toa/v79Bzw9n1eMwG6GcnBxs2LABS5YsQcuWLYUup0pTpkxBQkICEhIShC6FgcNslFasWAFra2t88sknQpdSrX79+sHJyQk7d+4UuhQGDrPRSU1NxXfffYfAwEA0btxY6HJqNGHCBPz8889QKpVCl9Lg8aEpIzN69GikpaXh8uXLJnHdcHp6OuRyOQ4ePIhhw4YJXU5DxoemjMnp06dx4MABrF271iSCDADt27dHv379eFXbCPCS2UgQEfr16wdLS0tERkYKXY5Wvv32W8ybNw8PHjyAra2t0OU0VLxkNhYhISGIjY3FunXrhC5Fa2PHjoVKpUJERITQpTRovGQ2AqWlpXBxcYG3t7fJrq4OGzYMMpkMBw4cELqUhoqXzMZg69atyMrKwj/+8Q+hS6kzPz8/HD16FPn5+UKX0mBxmAX2+PFjBAYGYv78+WjXrp3Q5dTZX/7yF5iZmeHgwYNCl9JgcZgF9s9//hMqlQp/+9vfhC6lXmxtbeHj44OQkBChS2mwOMwCunv3LjZv3ozly5ejWbNmQpdTb35+fjhx4gSePHkidCkNEodZQJ9//jlatmyJDz/8UOhSdGLEiBEgIhw9elToUhokDrNAEhISsHv3bvzrX/+CpaWl0OXoRJMmTdCvXz/ebhYIh1kgixYtwltvvSW6B8iPGDEChw4d4nO1BcBhFsDRo0dx4sQJ/POf/zSa+3rpyqhRo/D48WP88ccfQpfS4HCY9Sg6OhpxcXEaw1QqFZYuXYpRo0Zh8ODBAlWmP3K5HF27duVVbQFwmPVo//798PT0xPjx45Geng4A+Omnn5CYmCjIs6IMZcSIERxmAXCY9SgxMRFEhLCwMHTq1Akff/wxli1bhpkzZ8LFxUXo8vRm6NChuHHjBm7fvi10KQ0Kh1mPXjxgraysDGVlZfjmm2/w8OFDvPHGG3j27JnA1emPt7c3rKysTO7qL1PHYdaTx48fV3gek1KpRGlpKf71r3+hQ4cO2LFjB1QqlUAV6o+FhQX69u2L33//XehSGhQOs54kJSVVOa68vBw5OTmYM2cOPDw8cPPmTQNWZhiDBw/G77//Dr4oz3A4zHqSlJQEmUxW5fgXS+TevXujQ4cOhirLYIYMGYKcnJxq/6gx3eIw60lSUhLMzKqfvcuXL8c333xjMrcI0kaPHj3QrFkzXtU2IA6zniQkJKC0tLTCcIlEAjMzM2zfvh0rV640fGEGIpVK8fbbbyM6OlroUhoMDrOeXLt2rcIwqVQKCwsLHDhwQNCHphtK7969ERsbK3QZDQaHWQ/y8/Mr7MmWyWSwsrLC77//juHDhwtUmWH17t0b2dnZyMzMFLqUBoHDrAev7vQxNzdHy5YtceHCBfTt21egqgzvrbfegkwmw/nz54UupUHgMOvBy3uyzc3NIZfLERsbi86dOwtcmWFZWVmha9euHGYD4TDrQXJyMlQqFWQyGXr37o3z58+jTZs2QpclCC8vL95uNpAKB0KzsrJw9uxZIWoRjZMnT0KlUqFXr16YPXs2jh07JnRJOufn51er6by8vLB7926Ul5eL8hCcMalw3+yQkBD4+/sLVQ8zEbU9sys+Ph4eHh5ITk42yofGi0jV980mIv6pw09+fj5WrlwpeB36+gkODtbqG+bi4gKZTKa+6ITpD28z61jTpk2xYsUKocswGpaWlnBycsLVq1eFLkX0OMxM79zc3HjJbAAcZqZ3rq6uHGYD4DAzvXNzc0NGRgYeP34sdCmixmFmeufu7g4i4u1mPeMwM71zcHDAG2+8gYSEBKFLETUOMzOIbt268ZJZzzjMzCB4j7b+cZiZQbi6uuLq1auivIGhseAwM4Nwc3NDUVER0tLShC5FtDjMzCC6desGMzMzXtXWIw4zMwgrKys4OjpymPWIw8wMxs3Njfdo6xGHmRkM79HWr6rv0l4PCoUCUVFROHPmDNasWaOPLvTi4sWLVT5dQpub1cfExODu3bsaw8zNzWFnZ4dWrVqhU6dO9a7VFLm5uSEtLQ0KhQLW1tZClyM6egnz0aNHsWjRIqhUKpMJMxFh/PjxuHXrVqXj4+Liah1mNzc3xMTEYNmyZbCwsMCmTZugUqkQGxuLyMhI5OfnY+LEiVixYgXMzc11+TaMmpubG1QqFa5du4bevXsLXY740CuCg4OpksFa8/PzI7lcXu92DOX48eM0f/58un37NpWUlKh/jh8/Tu3bt9e6vczMTAJAXbp00RiuUqkoNDSUbG1tycfHhwoKCnT1FgyiPt8PlUpFtra2tH37dh1XxYgoRC9LZgAwMzOr8fEsxsTa2hobN26sUPOBAwfw/vvva92era1tpcMlEgnGjh2L8vJyjBs3Dv3798eFCxdgYWFRp7pNiUQi4dM69UhnYc7Ly0NYWBjS09Px1ltvgYggkUg0prl37x6OHj2KrKws9O3bF0OGDFGPy8zMRHh4OObNm4fk5GQcOHAAbdu2xcSJE9UBIyJER0fjypUrkEqlcHZ2ho+PT63ar4m3t3eFYSqVCuHh4QgLC1MPy83NRVBQEKZNm4aWLVvWuv1X+fv7Y+fOnTh8+DAuXLiAfv361fgehJ5HuuDm5sYXXOjLq8vquqxGXb9+nTw9Pens2bOkVCpp+/btZGlpSU5OTuppIiMjaebMmRQfH08hISFkbW1NAQEBREQUERFBdnZ2BIA2btxIU6dOpeHDhxMAWr16tbqNpUuXUlBQEBERXbx4kXr16lWr9usqJiaGWrVqRSqVSj0sKCiIANCmTZuqfe2TJ08qXc1+2apVqzTeoynMo/puhm3dupWaNm1a59ezKoXoJMxeXl60aNEi9e8qlYrkcrk6zIWFhSSXy0mhUKinmT59OgGgc+fOERHRkiVLCACdPHlSPU3Pnj3Jw8ND3Wbz5s0pKipKPT4wMLDW7dfFvHnzaO7cuRrDFAoF7dmzp8Zt3dqEOTw8nADQ0KFDTWYe1TfMp06dIgB07969OrfBKlX/bebIyEicP39e4yZ2EokEnp6euHLlCgBg7969KC4uxuLFi9XTZGdnw9HRETdv3kTv3r3RqFEjAICzs7N6mq5du6rvOS2RSNC5c2f4+/tjx44dGDVqFBYuXFjr9rVFRNi3bx9+/vlnjeFWVlYYP3681u1VRqFQqNs0xXlUFy9ut5uSkgJ7e3uD9NlQ1DvML7Z/unXrpjH85e3lpKQk2NvbY8uWLVq1LZVKQfR/92fevHkzfH19MXr0aAwZMgS7d+9Gy5Yt69x+df744w+Ulpbi7bff1lmbr4qPjwfw/EbxpjiP6qJFixZ44403kJKSgsGDBwtai9jUe3dzQUEBAFT6PKEXgZZKpUhNTYVSqaxXX927d0d8fDwCAgJw6tQp9OzZE3l5eTpr/2VhYWEYNWqU3p7CQEQ4ffo0pFIpfHx8THIe1ZWzszNSUlKELkN06h1mV1dXAM9Xt6vi7u6OoqIibNu2TWP448ePsXXr1lr1U1JSgl27dsHGxgZbtmzBoUOHkJ2djfDwcJ20/zIiQlhYWJ0OSdXWp59+iri4OKxbtw7u7u4mN4/qg8OsJ69uRWu7g0OpVJKzszNZW1tTdHQ0ERHdvXuX7O3tydramhISEkihUJCDgwNZWFjQ2rVrKTk5mYKDg8nX11e9I2nBggUEgNLS0tRtDxs2jGxsbEilUlFxcTH16dNHvWdZpVKRnZ0d7d+/n549e1Zj+9r4448/qEmTJlRSUlJh3KVLl8jT01NjJ1NlEhISCECFE05u375NAQEBJJFIaN68eerhtXkPxjCPdHFS0fr168ne3r5ebbAKdLM3+/bt2+Tp6UkASC6X04QJE2jEiBHUr18/+uabb6i4uJiSk5PJycmJABAAcnFxofj4eCJ6vodTLpcTAJoxYwZlZ2fT3r17ydbWlgDQypUrqbCwkOzt7WncuHEUGhpK69evp+XLl6trqK59bX3yySc0adKkSsft27ePJBKJ+vBPZSIiImjgwIHqWry9vcnHx4eGDRtGo0aNogULFtDFixcrvM4U5pEuwnzo0CECQHl5efVqh2nQTZhfyMnJUR/6KCwsrHSa9PR0unPnTp3aVyqVVFJSUu3r69P+C2lpaZSbm1vl+CdPntSr/ZoY8zzSRZjT0tLqfdiQVaDb0znt7OzU/6/qqph27drVuf0XDzBv27ZtldNU1n5AQECNbc+aNQvdu3cHgBovqKjqVE1dEWIeGVK7du3QuHFjpKSk8AUXOqS3c7ONyaBBg2qc5uU/REy/zMzM4OTkhOvXrwtdiqg0iDD7+voKXQJ7RceOHau8dpzVjelc1sREpVOnThxmHeMwM0G8OI2UXjp7jdUPh5kJomPHjnj69Cmys7OFLkU0OMxMEB07dgQAXtXWIQ4zE0SrVq1gZWXFYdYhDjMThEQigaOjY5U3UGTa4zAzwfDhKd3iMDPBcJh1i8PMBNO2bVvcuXNH6DJEg8PMBNO2bVs8evQIRUVFQpciChxmJhgHBwcAQFZWlsCViAOHmQnmxZVdGRkZAlciDhxmJphmzZrB2tqaw6wjHGYmqDZt2iAzM1PoMkShyksgQ0JCDFkHMxHnzp3TaXtt27blMOtIlWH29/c3ZB2sgXJwcODDUzpSYTXbz88PRMQ///sJDg4GAMHrMLYfXWnTpg3u3buns/YaMt5mZoJq0aIFHjx4IHQZosBhZoJq0aIF8vLyjOJJG6aOw8wE1bJlSxARcnNzhS7F5HGYmaBatGgBAMjJyRG4EtPHYWaCatmyJQDwdrMOcJiZoJo2bQpLS0teMusAh5kJzs7OjsOsAxxmJjg7OzveAaYDHGYmOFtbWxQWFgpdhsnjMDPB2djYoKCgQOgyTB6HmQnO1taWw6wDHGYmOF4y6waHmQmOt5l1g8PMBMdLZt3gMDPB8TazbnCYmeCsrKz4drs6wGFmgjM3N+dLIHWAw8wEx2HWDQ4zE5y5uTnKysp0ejuihojDzARnbm4OACgrKxO4EtPGYWaCexFmXtWuHw4zExyHWTc4zExwHGbd4DAzwfGOL93gMDPBvdjx9WIJzeqGw8wE92L1msNcPxxmJjgOs25wmJngXoRZJqvyOYasFjjMTHBKpRIymQwSiUToUkwah5kJTqlU8iq2DnCYmeA4zLrBYWaCUygUsLKyEroMk8dhZoIrLCyEra2t0GWYPN59+JKHDx9i//79GsMuXboEANixY4fGcGtra0yYMMFgtYkZh1k3OMwvsbW1xcKFC1FUVASpVArg+amGUqkUH330kXo6pVKJyZMnc5h1pKCgADY2NkKXYfJ4NfsllpaW8PX1hUwmg1KphFKpRFlZGcrLy9W/vzgmykHWHV4y6waH+RUTJkxAaWlptdM0bdoUQ4YMMVBF4ldQUMBh1gEO8ysGDRoEOzu7Ksebm5tj0qRJfLaSDnGYdYPD/AozMzNMmDABFhYWlY5XKpUYP368gasSt8LCQt5m1gEOcyXGjx9f5aq2vb09vL29DVyRuOXk5FS7NsRqh8NcCS8vL7Rr167CcHNzc0yZMoXPIdYhIsLDhw/RokULoUsxeRzmKvz1r3+tcIohr2LrXn5+PkpLS9GyZUuhSzF5HOYqTJw4scI9qTp27Ag3NzeBKhKnnJwcAOAlsw5wmKvg7OyMrl27qlepzc3NMXXqVIGrEh8Os+5wmKsxefJk9ZlgSqUSfn5+AlckPg8ePICZmRmaN28udCkmj8NcjXHjxqG8vBwA4OHhgY4dOwpckfjk5OTgjTfe4OP2OsBhrka7du3g6ekJ4PlSmukeH5bSHQm9ctPikJAQ+Pv7C1UPMxG6utf19OnTkZWVhWPHjumkvQYstMp1m+DgYEMWYrQKCgqwdetWLFmyROhSjMK5c+fw73//W2ftZWZmom3btjprryGrMsy8s+f/DBgwAJ06dRK6DKOhyzBnZGSgb9++OmuvIeNt5lrgIOtPVlYWL5l1hMPMBPPo0SMUFRXBwcFB6FJEgcPMBJORkQEAvGTWEQ4zE0xmZiYAoE2bNgJXIg4cZiaYjIwM2NnZoXHjxkKXIgocZiaYjIwM3l7WIQ4zE8zNmzf5FFkd4jAzwXCYdYvDzARBREhLS4Ojo6PQpYgGh5kJ4t69eygqKuITcnSIw8wEcfPmTQDg1Wwd4jAzQdy8eRNWVlZ48803hS5FNDjMTBAvdn7xnU51h8PMBMF7snVPL/dqUSgUiIqKwpkzZ7BmzRp9dKE3CoUCISEhSE9PR+/eveHj41Phlrs1iYmJwd27dzWGmZubw87ODq1ateKdPgBu3LiBP//5z0KXISp6WTIfPXoU8+fPxy+//KKP5vUmNTUVPXr0wJtvvonFixfjyZMn6NixI2JiYrRqx83NDbdu3cKECRPwwQcfoKCgAA8fPsTBgwfh7++PDh064IsvvqhwK8KxwWYAACAASURBVN+GQqVS4caNG3B2dha6FHGhVwQHB1Mlg7Xm5+dHcrm83u0Y0tChQ2n69Okaw6ZMmUL9+/fXuq3MzEwCQF26dNEYrlKpKDQ0lGxtbcnHx4cKCgrqVbOh6eL7cevWLQJAsbGxOqqKEVGI3m6JaGZmBjMz09okz87ORn5+vsYwS0tLlJSUaN1WVU81lEgkGDt2LMrLyzFu3Dj0798fFy5cqPJBdWKUkpICAOjcubPAlYiLzsKcl5eHsLAwpKen46233gIRVdhTee/ePRw9ehRZWVno27evxjOOMzMzER4ejnnz5iE5ORkHDhxA27ZtMXHiRPUfBSJCdHQ0rly5AqlUCmdnZ/j4+NSq/doYM2YMli9fjp9//hmTJk2CQqHA/v378Z///Ec9TW5uLoKCgjBt2rR6PVLF398fO3fuxOHDh3HhwgX069fPJOaRLly/fh2tWrVC06ZNDd63qL26rK7LatT169fJ09OTzp49S0qlkrZv306Wlpbk5OSkniYyMpJmzpxJ8fHxFBISQtbW1hQQEEBERBEREWRnZ0cAaOPGjTR16lQaPnw4AaDVq1er21i6dCkFBQUREdHFixepV69etWq/tu7fv0+dO3cmAPTpp5/SO++8Q+Hh4RrTBAUFEQDatGlTtW09efKk0tXsl61atUrjPZrCPNLFavb06dNpyJAh9WqDVRCikzB7eXnRokWL1L+rVCqSy+XqMBcWFpJcLieFQqGeZvr06QSAzp07R0RES5YsIQB08uRJ9TQ9e/YkDw8PdZvNmzenqKgo9fjAwMBat19bOTk55OjoSADI29ub7t+/rzFeoVDQnj17atzWrU2Yw8PDCQANHTrUZOaRLsLcp08f+uijj+rVBqug/tvMkZGROH/+PFasWKEeJpFI4OnpiStXrgAA9u7di+LiYixevFg9TXZ2NhwdHXHz5k307t0bjRo1AgCNPZxdu3ZV309ZIpGgc+fO8Pf3x44dOzBq1CgsXLiw1u3X1nfffYcBAwZgwIAB+P777+Hl5YWYmBj1rW2srKx09iRIhUKhbtOU5lF9paamYsKECQbrr6God5gTEhIAAN26ddMY/vL2clJSEuzt7bFlyxat2pZKpRo3W9+8eTN8fX0xevRoDBkyBLt370bLli3r3P6rfvjhBwQHB+PixYuQyWTo27cvZs+ejblz5+LgwYP1arsy8fHxAJ4/D9pU5lF9PXjwAI8ePUKXLl0ErUOM6r27uaCgAABw/vz5CuNeBFoqlSI1NbXex1W7d++O+Ph4BAQE4NSpU+jZsyfy8vJ01v5PP/2EoUOHqp97NG3aNMycORPHjx/H48eP69X2q4gIp0+fhlQqhY+Pj8nMo/q6fv06AHCY9aDeYXZ1dQXwfHW7Ku7u7igqKsK2bds0hj9+/Bhbt26tVT8lJSXYtWsXbGxssGXLFhw6dAjZ2dkIDw/XSfsAkJiYWCG0o0aNQmlpKR48eFDrdmrj008/RVxcHNatWwd3d3eTmUf1lZSUhKZNm8Le3t5gfTYYr25Fa7uDQ6lUkrOzM1lbW1N0dDQREd29e5fs7e3J2tqaEhISSKFQkIODA1lYWNDatWspOTmZgoODydfXV70jacGCBQSA0tLS1G0PGzaMbGxsSKVSUXFxMfXp04dUKhURPd/ZY2dnR/v376dnz57V2H5tTJ06ld58800qLy9XD1u5ciW5ubmph126dIk8PT01djJVJiEhgQBQ+/btNYbfvn2bAgICSCKR0Lx589TDa/MejGEe1XcH2OzZs2nAgAF1fj2rkm72Zt++fZs8PT0JAMnlcpowYQKNGDGC+vXrR9988w0VFxdTcnIyOTk5EQACQC4uLhQfH09ERKdOnSK5XE4AaMaMGZSdnU179+4lW1tbAkArV66kwsJCsre3p3HjxlFoaCitX7+eli9frq6huvZrq6ioiKZPn07dunWjf//73zRjxgwaOXKkRnj27dtHEolEffinMhERETRw4EB1Ld7e3uTj40PDhg2jUaNG0YIFC+jixYsVXmcK86i+Yfb29tb4I8Z0RjdhfiEnJ0d96KOwsLDSadLT0+nOnTt1al+pVFJJSUm1r69P+y8UFRVRcnIy5eXlVTr+yZMn9Wq/JsY8j+rz/VCpVGRra0s7duyo0+tZtXR7OufLz9m1traudJp27drVuf0XO6aqewJCZe0HBATU2PasWbPQvXt3AEDjxo2r3UFT1amauiLEPDKEtLQ0FBQUwM3NTZD+xa5BPK5+0KBBNU7DD/zWv8TERJiZmcHFxUXoUkSpQYTZ19dX6BIYnp+T4OjoWOVaG6sf07qsiZm0q1ev8iq2HnGYmcEkJiaqz0tgusdhZgZRVFSEtLQ0XjLrEYeZGcTVq1ehUqk4zHrEYWYGkZiYCGtra3To0EHoUkSLw8wM4urVq3B1dTW5W0mZEp6zzCASExN5FVvPOMxM74iID0sZAIeZ6V1GRgby8/M5zHrGYWZ6l5iYCIlEUuFuNEy3OMxM7xITE9GuXTu+ta6ecZiZ3vH2smFwmJne8Z5sw+AwM7169uwZbty4wedkG0CVl0DyQ7CZLiQlJaGsrIyXzAZQIcx9+vRBcHCwELWIxsGDB7F//35s2rSpwV+7m5CQgMaNG/MzqQ1AQvTSHdSZThQWFqJTp06YOHEiNmzYIHQ5gpozZw6SkpJw+vRpoUsRu1DeZtYDGxsbfPHFF9i8eTNu3boldDmCunDhAjw9PYUuo0HgMOvJnDlz4OjoiGXLlgldimCePXuGa9eucZgNhMOsJzKZDIGBgfjll19w7tw5ocsRxJUrV6BUKjnMBsJh1qMxY8agT58+WLJkidClCOLMmTOws7ODo6Oj0KU0CBxmPduwYQNOnz6N3377TehSDO706dMYMGAAH+Y0EA6znnl5eWHMmDFYtGgRysrKhC7HYIgIf/zxB/r37y90KQ0Gh9kA1q5di7S0NHz//fdCl2IwV69exaNHj/D2228LXUqDwWE2ALlcjlmzZmHZsmUoLCwUuhyDiIqKwuuvv86ncRoQh9lAVqxYgZKSEnz99ddCl2IQR44cUT9EnhkGh9lAmjdvjsWLF2P9+vXIzs4Wuhy9KioqQnR0NIYOHSp0KQ0Kh9mAPv30UzRr1gxffvml0KXoVWRkJEpKSvDuu+8KXUqDwmE2oEaNGmHlypX49ttvkZSUJHQ5enPkyBF4eHjA3t5e6FIaFA6zgU2ZMgVubm5YunSp0KXozZEjR/Dee+8JXUaDw2E2MDMzM/zzn/9EREQEIiMjhS5H55KSkpCens7bywLgMAvg3XffhY+PD5YsWQKxXYF6+PBhNGvWjM/HFgCHWSDr1q1DXFwcQkJChC5Fp44cOYKhQ4fyISkBcJgF4u7ujkmTJuHzzz9HSUmJ0OXoxOPHj/HHH3/wKrZAOMwCWr16NR48eICtW7cKXYpOhIeHQyKRYPjw4UKX0iBxmAXUunVrzJs3D//4xz+Ql5cndDn1FhwcjKFDh6JJkyZCl9IgcZgF9vnnn0MqlWLNmjVCl1Ivubm5iIyMhL+/v9ClNFgcZoE1adIEX3zxBTZt2oQ7d+4IXU6d7du3DxYWFryKLSC+O6cRUCqVcHFxgZeXF3bt2iV0OXUyePBg2NnZ8W2ahcN35zQG5ubmCAwMxO7duxEXFyd0OVq7f/8+YmJieBVbYLxkNhJEhH79+sHCwgJRUVFCl6OVzZs3Y+nSpXjw4AEaNWokdDkNFS+ZjYVEIsH69esRHR2NI0eOCF2OVoKDgzFq1CgOssB4yWxk/vKXvyA1NRWJiYmQyap8FJjRyMjIQIcOHfDrr79ixIgRQpfTkPGS2dj861//ws2bN7Fz506hS6mVH374AXZ2dvjzn/8sdCkNHofZyHTu3BkzZszAsmXLUFRUJHQ51SIi7Nq1C5MnT4a5ubnQ5TR4HGYjtHLlSigUCmzcuFHoUqoVGRmJW7du4YMPPhC6FAYOs1Fq0aIFFi5ciDVr1uD+/ftCl1Ol77//Ht7e3ujatavQpTBwmI3WwoUL0bRpUwQGBgpdSqWePHmCX3/9FVOnThW6FPY/HGYj1ahRI6xYsQLbt29HSkqKoLWcPHkS+fn5GsN2794NiUTCJ4oYEQ6zEZs6dSqcnZ3x97//XWN4SkoK/Pz8kJmZaZA6vvrqK9jb22P27NnqGxH+8MMPGDt2LGxtbQ1SA6sFYkbtt99+IwB0+vRpyszMpA8++IAkEgkBoCNHjhikBhcXFwJA5ubmJJFI6K233iIAFBkZaZD+Wa2E8EkjJuDtt99GZmYmsrOzoVKpoFQqYWFhgdWrV2PBggV679/e3l5jR5xMJkN5eTlatWqFefPmYebMmWjWrJne62DV4pNGjJlSqcSOHTtw9epVZGVloaSkBEqlEgCgUqmQnJxskDoKCgo0fi8rKwMR4e7du/j73/+OVq1a4cMPP8SNGzcMUg+rHIfZSJ08eRLdunXDhx9+iMePH1d4HGxZWRkuX76s9zrKyspQXFxc5fjy8nKUlpbil19+QXl5ud7rYVXjMBuhr7/+Gj4+Prhx4wZUKlWV012/fl3vt+rNy8urtg8zMzNYWFjg0KFDcHZ21mstrHocZiP06aef4rPPPqtxuuLiYmRkZOi1lpruTWZmZoYDBw6gT58+eq2D1YzDbIQkEgk2bNiAjRs3QiKRQCKRVDmtvrebqwuzRCLB7t27+QFxRoLDbMQ+/vhjhIaGQiaTVXpTeQsLC70/gO7Vk0VetnHjRvj5+em1f1Z7HGYj9/777+Po0aOwtLSscH2zIfZo5+XlwcxM82tiZmaGVatW4eOPP9Zr30w7HGYTMHjwYMTGxqJZs2YalxoaYo92Xl6exh8RqVSK2bNnY9myZXrtl2mPw2wiXF1dERsbCwcHB41Ap6am6nWPdn5+vnqbXSaTYezYsdi8ebPe+mN1x2E2IR06dMDFixfRo0cP9dJS33u08/PzUVZWBplMhkGDBmHnzp0VVruZceBPxcQ0a9YMv//+OwYNGqReYl67dk1v/T169Ajl5eVwdXVFeHg4LCws9NYXqx+tz80+d+4cvv76a33Vw2pJpVLh0qVLyMjIgKurKzp37qyXfs6cOQOFQoHBgwdzkI1IaGhohUFaL5kzMzMRFhamm4pYnZmZmaFXr17o3LlzhXOndUkmk+Htt9/mIBuJrKysKvNX53u5VvKXgQnk1KlTGDhwoF7azs/Px+uvv66Xtpn2QkJCqrwhBG8zi4C+ggyAg2xCOMyMiQSHmTGR4DAzJhIcZsZEgsPMmEhwmBkTCQ4zYyLBYWZMJDjMjIkEh5kxkeAwMyYSHGbGRILDzJhI1PkSSGZa7t+/j+vXr+vkCqvMzEzEx8cjMTERZmZm6NSpEzw9PSGRSJCVlYV+/frVv2CmNV4yi9zDhw+xcOFCyOVy7N+/v15tlZaWYtGiRXBycsIff/yBnj17ok+fPkhLS4OHhwfkcjkuXLigo8qZtjjMr9i5c6eo+k1PT8fkyZOrffhbbTx79gze3t4ICgrCiRMnsHbtWgwbNgyDBg3CkiVLcPHiRbRp0wZPnz7VUeW6I7bPtCoc5pdERkbi888/F1W/np6eOnmgW2BgIOLj47Fo0aJKV6MdHR2xbNkyFBUV1bsvXRLjZ1oV6cqVK1dq84KkpCSEhYVBy5dBoVAgJCQEoaGhyM3NRZs2bfDaa6+pxxcWFuLXX39FWFgYbt26BTs7OzRp0kQ9PjMzEz/++CN69eqFpKQkBAUF4c6dO3B1ddV4FlNN/fz3v//FoUOHsGvXLhQVFaFLly4AgKioKIwePRpKpRLNmjVDdna2+iZ59+7dQ2hoKA4ePIiysjLI5XKt69J1v9ooLy9HYGAgvLy8MHToUI1xubm5+M9//gMnJydYW1tX+vr79+9j7NixeO211xAaGgpLS8tKp3N1dUVhYSG6du0KgD9TfXym1eQvGaSl4OBgArR7WUpKCr333nuUkJBASqWSxo8fT2+88QbdunWLiIiuXLlCrq6utG/fPsrJyaH169eTtbU1/fTTT0REFBERQXZ2dgSANm7cSFOnTqXhw4cTAFq9enWt+9m4cSMNHDiQVCoV3b59m9q3b09bt24lIqLLly9T3759yc7OjqKioujy5ctERBQZGUkzZ86k+Ph4CgkJIWtrawoICNCqLl33q62SkhICQPPnz68wLigoiADQpk2bqnz94cOHCQB169at1n3yZ6qfz7Sa/IXoPcxlZWXUvXt32rFjh3pYXFwcWVhY0MGDB6mkpIScnZ1p+fLlGq+bMGECWVhYUFJSEhERLVmyhADQyZMn1dP07NmTPDw8atUPEVHHjh1p7ty56vGjR4+m9957T+N3BwcH9e+FhYUkl8tJoVCoh02fPp0A0Llz52pVl7761UZ1YVYoFLRnzx4qKCio8vVr164lADRixIha98efqX4+0+rCrPdDU4cPH8aVK1cwbNgw9bCePXuisLAQFhYWiIiIwPXr19G7d2+N17377rvYs2cPvvvuO2zYsAGNGjUCAI3tv65du+LYsWO16gd4fhdLKysrAM8fhZqZmVnhNrUvr0bt3bsXxcXFWLx4sXpYdnY2HB0dcfPmTfTu3bvGuvTVr65YWVlh/Pjx1U7z4ukZ5eXltWrz6NGj/JnWod/60nuYExISYGVlBTs7O43hLz6MF08xfHV7rX///gCAlJSUKtuWSqXq5yzV1A8AtG7dGsePH8dvv/2GAQMGwNHREXFxcRrTv/wBJCUlwd7eHlu2bKnVe62sLkP2qy8uLi4AgBs3btRqev5MddevNvQeZpVKhaKiIkRFReGdd96pML5Zs2YAnj8p48WHDQDt2rWDubl5rW/1WlM/ALBs2TJER0fj2LFjaNSoEfbt21dhmpc/AKlUitTUVCiVSo2HtWlLqH51xcPDA9bW1khLS8OtW7fg6OhY7fT8meqv3+ro/dCUq6srAGDPnj0awx89eoT9+/fDy8sLABATE6Mx/tq1a1AqlfD29tZJP7dv30ZgYCAmTZqkXo1SqVQa00okEo1VSXd3dxQVFWHbtm0a0z1+/Bhbt26tVV1C9atLb7zxBr788kuUl5drrCZW5vLly/yZ6qnfGtV6y7vmDfBKlZWVUY8ePQgAzZ49m06ePElff/01jRw5kp49e0ZERFOmTCEbGxu6c+eO+nVbtmyhTp06UUlJCRERLViwgABQWlqaepphw4aRjY0NqVSqGvtJTEwkADRw4EB68uQJxcTEkL29PTVr1owKCwupoKCAAgICyNzcnG7dukU3b96kR48ekYODA1lYWNDatWspOTmZgoODydfXV73DqKa69NWvNu7fv08AaNasWRXGXbp0iTw9PSkqKqraNpRKJfn5+REAmjFjBj19+lRjfHp6Os2cOZNiYmKIiD9TfX2mgu7NJiLKysoiHx8fkkgkJJFIaODAgZSVlaUeX1xcTHPnziUXFxf68ccf6dtvv6Vhw4ZRRkYGERGdOnWK5HK5+ouUnZ1Ne/fuJVtbWwJAK1euJKVSWWM/06ZNI5lMRh07dqRt27ZRWFgYWVhY0ODBg+nRo0cUFRVFMpmMmjZtqj5Uk5ycTE5OTgSAAJCLiwvFx8drVZeu+9XG4cOHyd/fnwBQixYtKCgoiLKzs9Xj9+3bRxKJhIKCgmrV3q5du6ht27bUsmVLGjlyJE2bNo2cnJzIz8+Prl+/zp+pnj/T6sKs9VMgXzzrRsuXAXi+SqFSqdTbVK968uQJkpKS0LZtW7Rp00br9mvTT2FhIWxsbNS/l5SUaJwE8eTJE5iZmWlMAwB37tyBRCJB27Zt61STUP3WRkFBAWxtbbV6TX5+Pq5duwZzc3M4OTnxZ2qgfqvJX6hBw8x0IyAgoMZpZs2ahe7duxugGmZI1YWZL4E0QYMGDapxmlcP5zDx4zCbIF9fX6FLYEaIr5piTCQ4zIyJBIeZMZHgMDMmEhxmxkSCw8yYSHCYGRMJDjNjIsFhZkwkOMyMiQSHmTGR4DAzJhIcZsZEgsPMmEjU+RJIvgyPMcPLysqqcpzWS2YHBweMHTu2XgUx4Tx8+LDCXTOZ6WjTpk2V+dP6tkHMtPFtn0QrlLeZGRMJDjNjIsFhZkwkOMyMiQSHmTGR4DAzJhIcZsZEgsPMmEhwmBkTCQ4zYyLBYWZMJDjMjIkEh5kxkeAwMyYSHGbGRILDzJhIcJgZEwkOM2MiwWFmTCQ4zIyJBIeZMZHgMDMmEhxmxkSCw8yYSHCYGRMJDjNjIsFhZkwkOMyMiQSHmTGR4DAzJhIcZsZEgsPMmEhwmBkTCZnQBTD9ycrKwpQpU1BeXq4elpubC5lMhoEDB2pM27lzZ2zfvt3AFTJd4jCLWJs2bZCeno60tLQK46KjozV+79+/v6HKYnrCq9kiN3nyZJibm9c43bhx4wxQDdMnDrPITZw4EUqlstppunbtChcXFwNVxPSFwyxyHTt2hJubGyQSSaXjzc3NMWXKFANXxfSBw9wATJ48GVKptNJxZWVl8PPzM3BFTB84zA3A+PHjoVKpKgyXSCTw8vJC+/btDV8U0zkOcwPQqlUr9OnTB2Zmmh+3VCrF5MmTBaqK6RqHuYH461//WmEYEeH9998XoBqmDxzmBsLX11djySyVSvGnP/0JLVq0ELAqpksc5gbi9ddfxzvvvKPeEUZEmDRpksBVMV3iMDcgkyZNUu8Ik8lkGDlypMAVMV3iMDcgI0eOhKWlpfr/tra2AlfEdEn052ZnZWXh7NmzQpdhNHr27ImzZ8+iQ4cOCAkJEbocoyGGY+0SIiKhi9CnkJAQ+Pv7C10GM3IiiEFog1nNJiL+IUJpaSkWL14seB3G8hMcHCz0V1NnGkyY2XPm5uZYuXKl0GUwPeAwN0CNGjUSugSmBxxmxkSCw8yYSHCYGRMJDjNjIsFhZkwkOMyMiQSHmTGR4DAzJhIcZsZEgsPMmEhwmBkTCQ4zYyIh+psT6IJCoUBUVBTOnDmDNWvWCF2OVh49eoQDBw4gIyMDbm5ueOedd2Btba1VGzExMbh7967GMHNzc9jZ2aFVq1bo1KmTLktmdcRL5lo4evQo5s+fj19++UXoUrRy5coVDBw4EF27dsXixYtx8+ZN9O3bF9nZ2Vq14+bmhlu3bmHChAn44IMPUFBQgIcPH+LgwYPw9/dHhw4d8MUXX9T4TCumZyRywcHBpIu36efnR3K5XAcVGUZ5eTm5u7vT4sWLNYb36tWLfHx8tG4vMzOTAFCXLl00hqtUKgoNDSVbW1vy8fGhgoKCetVtaLr6fhiBEF4y15KZmVmFJ0IYs9jYWCQkJKBHjx4aw3v16oUTJ04gLi5Oq/aquvmfRCLB2LFjsWPHDpw4cQL9+/dHaWlpnetmdcfbzFXIy8tDWFgY0tPT8dZbb4GIKjxJ8d69ezh69CiysrLQt29fDBkyRD0uMzMT4eHhmDdvHpKTk3HgwAG0bdsWEydOVP9RICJER0fjypUrkEqlcHZ2ho+PT63ar0lqaqq6j5d5enoCAM6cOQMPDw/k5uYiKCgI06ZNQ8uWLbWbSS/x9/fHzp07cfjwYVy4cAH9+vWr8T0IPY9ER9g1A/2ry2rU9evXydPTk86ePUtKpZK2b99OlpaW5OTkpJ4mMjKSZs6cSfHx8RQSEkLW1tYUEBBAREQRERFkZ2dHAGjjxo00depUGj58OAGg1atXq9tYunQpBQUFERHRxYsXqVevXrVqvzb27t1LAOizzz7TGH7mzBmN4UFBQQSANm3aVG17T548qXQ1+2WrVq3SeI/GPo+IxLWaLYp3UZ26fFheXl60aNEi9e8qlYrkcrk6zIWFhSSXy0mhUKinmT59OgGgc+fOERHRkiVLCACdPHlSPU3Pnj3Jw8ND3Wbz5s0pKipKPT4wMLDW7dckIyODLCwsyMPDg1QqlXr4oUOHNMKrUChoz549NW7r1ibM4eHhBICGDh1qEvOISFxh5tXsV0RGRuL8+fNYsWKFephEIoGnpyeuXLkCANi7dy+Ki4uxePFi9TTZ2dlwdHTEzZs30bt3b/V9tpydndXTdO3aFceOHVO32blzZ/j7+2PHjh0YNWoUFi5cWOv2a+Lg4IDAwEAsXrwYU6dOhZ+fH1JSUtR75N3d3QEAVlZWGD9+fJ3m1asUCoW6TVOYR2LDYX5FQkICAKBbt24aw1/eXk5KSoK9vT22bNmiVdtSqVRjG3bz5s3w9fXF6NGjMWTIEOzevRstW7asc/uvWrRoEXr16oXjx4/jzJkzGDduHGJjY3Hjxo0KO8Z0IT4+HgDg5eVlMvNITExn96yBFBQUAADOnz9fYdyLQEulUqSmptb7uGr37t0RHx+PgIAAnDp1Cj179kReXp7O2geAAQMG4KuvvsLq1athY2ODiIgIrFq1CjY2NvVu+2VEhNOnT0MqlcLHx8ek5pFYcJhf4erqCuD56nZV3N3dUVRUhG3btmkMf/z4MbZu3VqrfkpKSrBr1y7Y2Nhgy5YtOHToELKzsxEeHq6T9l9VWloKf39/dO7cGQEBAXVqozqffvop4uLisG7dOri7u5vkPDJ5Am+06522OziUSiU5OzuTtbU1RUdHExHR3bt3yd7enqytrSkhIYEUCgU5ODiQhYUFrV27lpKTkyk4OJh8fX3VO5IWLFhAACgtLU3d9rBhw8jGxoZUKhUVFxdTnz591DunVCoV2dnZ0f79++nZs2c1tq8NhUJBkydPJj8/P3rw4IHGuEuXLpGnp6fGTqbKJCQkEABq3769xvDbt29TQEAASSQSmjdvnnp4bd6DMcwjMe0AE8W7qE5dPqzbt2+Tp6cnASC5XE4TJkygESNGUL9+/eibb76h4uJiSk5OJicnJwJAAMjFxYXi4+OJiOjUqVMkl8sJAM2YMYOys7Np7969ZGtrSwBo5cqVVFhYSPb29jRu3DgKDQ2l9evX0/Lly9U1ghmRMQAAAWVJREFUVNd+beXm5tJ3331Hffr0ofDw8Eqn2bdvH0kkEvXhn8pERETQwIED1bV4e3uTj48PDRs2jEaNGkULFiygixcvVnidKcwjMYW5wTw4ri5v8+HDh2jcuDGsrKygUCgqvUDhzp07kEgkaNu2rdbtl5WVQaVS4f79+1W+vj7t//rrr3Bzc4NcLq92uoKCAr0+3tWY51F9vh9GJpT3ZlfDzs5O/f+qrjRq165dnduXyZ7P/uq+hJW1X5tt3lmzZmH06NG1qkPfz2kWYh41RBxmEzRo0KAap3n5DxFrGDjMJsjX11foEpgR4kNTjIkEh5kxkeAwMyYSHGbGRILDzJhIcJgZEwkOM2MiwWFmTCQ4zIyJBIeZMZHgMDMmEhxmxkSCw8yYSHCYGROJBnMJZEhIiNAlMCN07tw5oUvQmQYTZn9/f6FLYEyvRH8PMMYaiFDeZmZMJDjMjIkEh5kxkeAwMyYS/x+Xpf/JR/E3ogAAAABJRU5ErkJggg==\n", "text/plain": [ "" ] }, "execution_count": 31, "metadata": {}, "output_type": "execute_result" } ], "source": [ "plot_model(model)\n", "Image('model.png')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Hyperparameter optimization" ] }, { "cell_type": "code", "execution_count": 32, "metadata": {}, "outputs": [], "source": [ "import optuna" ] }, { "cell_type": "code", "execution_count": 33, "metadata": {}, "outputs": [], "source": [ "from keras.layers import Conv2D, MaxPooling2D, Dropout, Flatten" ] }, { "cell_type": "code", "execution_count": 34, "metadata": {}, "outputs": [], "source": [ "(X_train, y_train), (X_test, y_test) = tf.keras.datasets.mnist.load_data()\n", "X_train, X_test = X_train / 255.0, X_test / 255.0\n", "X_train = X_train.astype('float32')\n", "X_test = X_test.astype('float32')\n", "X_train = X_train.reshape(X_train.shape[0], 28, 28, 1)\n", "X_test = X_test.reshape(X_test.shape[0], 28, 28, 1)" ] }, { "cell_type": "code", "execution_count": 35, "metadata": {}, "outputs": [], "source": [ "class Objective(object):\n", " def __init__(self, X, y, \n", " max_epochs, \n", " input_shape, \n", " num_classes):\n", " self.X = X\n", " self.y = y\n", " self.max_epochs = max_epochs\n", " self.input_shape = input_shape\n", " self.num_classes = num_classes\n", " \n", " def __call__(self, trial): \n", " dropout=trial.suggest_discrete_uniform('dropout', 0.05, 0.5, 0.05)\n", " batch_size = trial.suggest_categorical('batch_size', [32, 64, 96, 128])\n", " \n", " params = dict(\n", " dropout = dropout,\n", " batch_size = batch_size\n", " )\n", " \n", " model = Sequential()\n", " model.add(Conv2D(32, kernel_size=(3, 3),\n", " activation='relu',\n", " input_shape=self.input_shape))\n", " model.add(Conv2D(64, (3, 3), activation='relu'))\n", " model.add(MaxPooling2D(pool_size=(2, 2)))\n", " model.add(Dropout(params['dropout']))\n", " model.add(Flatten())\n", " model.add(Dense(128, activation='relu'))\n", " model.add(Dropout(params['dropout']))\n", " model.add(Dense(self.num_classes, activation='softmax'))\n", " \n", " model.compile(optimizer='adam',\n", " loss='sparse_categorical_crossentropy',\n", " metrics=['accuracy']) \n", " \n", " # fit the model\n", " hist = model.fit(x=self.X, y=self.y,\n", " batch_size=params['batch_size'],\n", " validation_split=0.25,\n", " epochs=self.max_epochs)\n", " \n", " loss = np.min(hist.history['val_loss'])\n", " \n", " return loss" ] }, { "cell_type": "code", "execution_count": 36, "metadata": {}, "outputs": [], "source": [ "optuna.logging.set_verbosity(0)" ] }, { "cell_type": "code", "execution_count": 37, "metadata": {}, "outputs": [], "source": [ "N = 5\n", "max_epochs = 3\n", "input_shape = (28,28,1)\n", "num_classes = 10" ] }, { "cell_type": "code", "execution_count": 38, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Train on 45000 samples, validate on 15000 samples\n", "Epoch 1/3\n", "45000/45000 [==============================] - 19s 430us/step - loss: 0.2965 - acc: 0.9088 - val_loss: 0.0837 - val_acc: 0.9749\n", "Epoch 2/3\n", "45000/45000 [==============================] - 19s 420us/step - loss: 0.1083 - acc: 0.9680 - val_loss: 0.0569 - val_acc: 0.9825\n", "Epoch 3/3\n", "45000/45000 [==============================] - 19s 415us/step - loss: 0.0829 - acc: 0.9752 - val_loss: 0.0493 - val_acc: 0.9859\n", "Train on 45000 samples, validate on 15000 samples\n", "Epoch 1/3\n", "45000/45000 [==============================] - 23s 504us/step - loss: 0.2049 - acc: 0.9371 - val_loss: 0.0688 - val_acc: 0.9793\n", "Epoch 2/3\n", "45000/45000 [==============================] - 19s 423us/step - loss: 0.0540 - acc: 0.9838 - val_loss: 0.0469 - val_acc: 0.9861\n", "Epoch 3/3\n", "45000/45000 [==============================] - 19s 418us/step - loss: 0.0357 - acc: 0.9887 - val_loss: 0.0511 - val_acc: 0.9844\n", "Train on 45000 samples, validate on 15000 samples\n", "Epoch 1/3\n", "45000/45000 [==============================] - 25s 554us/step - loss: 0.1936 - acc: 0.9414 - val_loss: 0.0704 - val_acc: 0.9783\n", "Epoch 2/3\n", "45000/45000 [==============================] - 20s 445us/step - loss: 0.0556 - acc: 0.9826 - val_loss: 0.0522 - val_acc: 0.9836\n", "Epoch 3/3\n", "45000/45000 [==============================] - 20s 446us/step - loss: 0.0366 - acc: 0.9883 - val_loss: 0.0544 - val_acc: 0.9829\n", "Train on 45000 samples, validate on 15000 samples\n", "Epoch 1/3\n", "45000/45000 [==============================] - 23s 511us/step - loss: 0.2214 - acc: 0.9320 - val_loss: 0.0740 - val_acc: 0.9771\n", "Epoch 2/3\n", "45000/45000 [==============================] - 19s 417us/step - loss: 0.0639 - acc: 0.9811 - val_loss: 0.0544 - val_acc: 0.9835\n", "Epoch 3/3\n", "45000/45000 [==============================] - 19s 419us/step - loss: 0.0426 - acc: 0.9870 - val_loss: 0.0446 - val_acc: 0.9866\n", "Train on 45000 samples, validate on 15000 samples\n", "Epoch 1/3\n", "45000/45000 [==============================] - 29s 638us/step - loss: 0.2372 - acc: 0.9277 - val_loss: 0.0643 - val_acc: 0.9809\n", "Epoch 2/3\n", "45000/45000 [==============================] - 22s 498us/step - loss: 0.0898 - acc: 0.9722 - val_loss: 0.0489 - val_acc: 0.9865\n", "Epoch 3/3\n", "45000/45000 [==============================] - 22s 499us/step - loss: 0.0671 - acc: 0.9795 - val_loss: 0.0430 - val_acc: 0.9879\n", "CPU times: user 1min 10s, sys: 1min 11s, total: 2min 22s\n", "Wall time: 5min 16s\n" ] } ], "source": [ "%%time\n", "\n", "objective1 = Objective(X_train, y_train, max_epochs, input_shape, num_classes)\n", "study1 = optuna.create_study(direction='minimize')\n", "study1.optimize(objective1, n_trials=N)" ] }, { "cell_type": "code", "execution_count": 40, "metadata": {}, "outputs": [], "source": [ "df = study1.trials_dataframe()" ] }, { "cell_type": "code", "execution_count": 41, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
numbervaluedatetime_startdatetime_completedurationparams_batch_sizeparams_dropoutstate
000.0493472020-10-21 08:34:48.7128122020-10-21 08:35:45.7537410 days 00:00:57.0409291280.50COMPLETE
110.0468972020-10-21 08:35:45.7537772020-10-21 08:36:46.3497710 days 00:01:00.5959941280.15COMPLETE
220.0521582020-10-21 08:36:46.3498112020-10-21 08:37:51.4004440 days 00:01:05.050633960.15COMPLETE
330.0446222020-10-21 08:37:51.4004852020-10-21 08:38:52.0376580 days 00:01:00.6371731280.20COMPLETE
440.0430212020-10-21 08:38:52.0377012020-10-21 08:40:05.7072990 days 00:01:13.669598640.45COMPLETE
\n", "
" ], "text/plain": [ " number value datetime_start datetime_complete \\\n", "0 0 0.049347 2020-10-21 08:34:48.712812 2020-10-21 08:35:45.753741 \n", "1 1 0.046897 2020-10-21 08:35:45.753777 2020-10-21 08:36:46.349771 \n", "2 2 0.052158 2020-10-21 08:36:46.349811 2020-10-21 08:37:51.400444 \n", "3 3 0.044622 2020-10-21 08:37:51.400485 2020-10-21 08:38:52.037658 \n", "4 4 0.043021 2020-10-21 08:38:52.037701 2020-10-21 08:40:05.707299 \n", "\n", " duration params_batch_size params_dropout state \n", "0 0 days 00:00:57.040929 128 0.50 COMPLETE \n", "1 0 days 00:01:00.595994 128 0.15 COMPLETE \n", "2 0 days 00:01:05.050633 96 0.15 COMPLETE \n", "3 0 days 00:01:00.637173 128 0.20 COMPLETE \n", "4 0 days 00:01:13.669598 64 0.45 COMPLETE " ] }, "execution_count": 41, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df.head()" ] }, { "cell_type": "code", "execution_count": 42, "metadata": {}, "outputs": [], "source": [ "from optuna.visualization import plot_param_importances" ] }, { "cell_type": "code", "execution_count": 44, "metadata": {}, "outputs": [ { "data": { "text/html": [ " \n", " " ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/vnd.plotly.v1+json": { "config": { "plotlyServerURL": "https://plot.ly" }, "data": [ { "cliponaxis": false, "hovertemplate": [ "dropout (DiscreteUniformDistribution): 0.2639676867095813", "batch_size (CategoricalDistribution): 0.7360323132904186" ], "marker": { "color": [ "rgb(8,48,107)", "rgb(66,146,198)" ] }, "orientation": "h", "text": [ "0.2639676867095813", "0.7360323132904186" ], "textposition": "outside", "texttemplate": "%{text:.2f}", "type": "bar", "x": [ 0.2639676867095813, 0.7360323132904186 ], "y": [ "dropout", "batch_size" ] } ], "layout": { "showlegend": false, "template": { "data": { "bar": [ { "error_x": { "color": "#2a3f5f" }, "error_y": { "color": "#2a3f5f" }, "marker": { "line": { "color": "#E5ECF6", "width": 0.5 } }, "type": "bar" } ], "barpolar": [ { "marker": { "line": { "color": "#E5ECF6", "width": 0.5 } }, "type": "barpolar" } ], "carpet": [ { "aaxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "baxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "type": "carpet" } ], "choropleth": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "choropleth" } ], "contour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "contour" } ], "contourcarpet": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "contourcarpet" } ], "heatmap": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmap" } ], "heatmapgl": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmapgl" } ], "histogram": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "histogram" } ], "histogram2d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2d" } ], "histogram2dcontour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2dcontour" } ], "mesh3d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "mesh3d" } ], "parcoords": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "parcoords" } ], "pie": [ { "automargin": true, "type": "pie" } ], "scatter": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatter" } ], "scatter3d": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatter3d" } ], "scattercarpet": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattercarpet" } ], "scattergeo": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergeo" } ], "scattergl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergl" } ], "scattermapbox": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattermapbox" } ], "scatterpolar": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolar" } ], "scatterpolargl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolargl" } ], "scatterternary": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterternary" } ], "surface": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "surface" } ], "table": [ { "cells": { "fill": { "color": "#EBF0F8" }, "line": { "color": "white" } }, "header": { "fill": { "color": "#C8D4E3" }, "line": { "color": "white" } }, "type": "table" } ] }, "layout": { "annotationdefaults": { "arrowcolor": "#2a3f5f", "arrowhead": 0, "arrowwidth": 1 }, "coloraxis": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "colorscale": { "diverging": [ [ 0, "#8e0152" ], [ 0.1, "#c51b7d" ], [ 0.2, "#de77ae" ], [ 0.3, "#f1b6da" ], [ 0.4, "#fde0ef" ], [ 0.5, "#f7f7f7" ], [ 0.6, "#e6f5d0" ], [ 0.7, "#b8e186" ], [ 0.8, "#7fbc41" ], [ 0.9, "#4d9221" ], [ 1, "#276419" ] ], "sequential": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "sequentialminus": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ] }, "colorway": [ "#636efa", "#EF553B", "#00cc96", "#ab63fa", "#FFA15A", "#19d3f3", "#FF6692", "#B6E880", "#FF97FF", "#FECB52" ], "font": { "color": "#2a3f5f" }, "geo": { "bgcolor": "white", "lakecolor": "white", "landcolor": "#E5ECF6", "showlakes": true, "showland": true, "subunitcolor": "white" }, "hoverlabel": { "align": "left" }, "hovermode": "closest", "mapbox": { "style": "light" }, "paper_bgcolor": "white", "plot_bgcolor": "#E5ECF6", "polar": { "angularaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "radialaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "scene": { "xaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "yaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "zaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" } }, "shapedefaults": { "line": { "color": "#2a3f5f" } }, "ternary": { "aaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "baxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "caxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "title": { "x": 0.05 }, "xaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 }, "yaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 } } }, "title": { "text": "Hyperparameter Importances" }, "xaxis": { "title": { "text": "Importance" } }, "yaxis": { "title": { "text": "Hyperparameter" } } } }, "text/html": [ "
\n", " \n", " \n", "
\n", " \n", "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plot_param_importances(study1)" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.5" } }, "nbformat": 4, "nbformat_minor": 2 }