"
],
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAVMAAAEmCAYAAADfpHMGAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3debxd0/3/8df7JkhIYoqpitAaSlpEzVOoaoxBDV9SNWurtOVbRUsNxVeraiolqFkbQ81q+CE1tSoh5qKamEkQQ0jJ8Pn9sdbhiOScc+/d555z7n0/PfbDOXvvs/bnnuR+stbaa6+liMDMzDqnrdEBmJl1B06mZmYFcDI1MyuAk6mZWQGcTM3MCuBkamZWACdTaxhJfSXdKOldSVd1opwRkm4vMrZGkbShpGcaHYe1nzzO1KqRtBtwCLAS8D4wDjghIu7rZLm7AwcB60XE9E4H2uQkBbB8RPy70bFY8VwztYokHQKcBpwILAYsDZwNDC+g+GWAZ3tCIq2FpN6NjsE6ISK8eZvtBswPTAF2qnDOPKRk+2reTgPmyceGAi8D/wtMBF4D9srHjgU+Bqbla+wDHANcVlb2ICCA3vn9nsB/SLXj8cCIsv33lX1uPeAh4N38//XKjo0GfgXcn8u5HRg4h5+tFP/PyuLfDtgSeBZ4G/h52flrAX8H3snn/h6YOx+7J/8sH+Sfd5ey8g8DXgcuLe3Ln/lSvsaQ/P4LwCRgaKP/bnj7/OaaqVWyLtAHuLbCOb8A1gFWA1YlJZQjy44vTkrKS5IS5lmSFoyIo0m13VER0S8iLqgUiKT5gDOALSKiPylhjpvNeQsBN+dzFwZ+B9wsaeGy03YD9gIWBeYGflrh0ouTvoMlgV8C5wHfAdYANgSOkrRsPncGcDAwkPTdfQM4ACAiNsrnrJp/3lFl5S9EqqXvX37hiHielGgvkzQvcCFwcUSMrhCvNYiTqVWyMPBmVG6GjwCOi4iJETGJVOPcvez4tHx8WkTcQqqVrdjBeGYCgyX1jYjXIuLJ2ZyzFfBcRFwaEdMj4k/Av4Btys65MCKejYipwJWkfwjmZBqpf3ga8GdSojw9It7P13+K9I8IETE2Iv6RrzsBOBfYuIaf6eiI+CjH8xkRcR7wb+BBYAnSP17WhJxMrZK3gIFV+vK+ALxQ9v6FvO+TMmZJxh8C/dobSER8QGoafx94TdLNklaqIZ5STEuWvX+9HfG8FREz8utSsnuj7PjU0uclrSDpJkmvS3qPVPMeWKFsgEkR8d8q55wHDAbOjIiPqpxrDeJkapX8HfiI1E84J6+SmqglS+d9HfEBMG/Z+8XLD0bEbRHxTVIN7V+kJFMtnlJMr3Qwpvb4Aymu5SNiAPBzQFU+U3E4jaR+pH7oC4BjcjeGNSEnU5ujiHiX1E94lqTtJM0raS5JW0j6TT7tT8CRkhaRNDCff1kHLzkO2EjS0pLmB44oHZC0mKThue/0I1J3wczZlHELsIKk3ST1lrQLsDJwUwdjao/+wHvAlFxr/sEsx98AlmtnmacDYyJiX1Jf8DmdjtLqwsnUKoqIU0hjTI8k3Ul+CTgQuC6fcjwwBngMeBx4OO/ryLXuAEblssby2QTYluN4lXSHe2M+n6yIiLeArUkjCN4i3YnfOiLe7EhM7fRT0s2t90m15lGzHD8GuFjSO5J2rlaYpOHAMD79OQ8BhkgaUVjEVhgP2jczK4BrpmZmBXAyNTMrgJOpmVkBnEzNzArgiRW60MCBA2OZZQY1Oowe64OPZ1Q/yermmSfGvRkRixRRVq8By0RM/9wDY58TUyfdFhHDirhmNU6mXWiZZQZx/4NjGh1Gj/Xw+MmNDqFHW3+FhWZ9Mq3DYvpU5lmx6ugy/jvurGpPoBXGzXwzaz0StPWqvtVcnHpJekTSTfn9RZLGSxqXt0rzNwCumZpZq1KhdcEfA08DA8r2HRoRV9dagGumZtaapOpbTcXoi6TZxs7vTDhOpmbWgmpu5g+UNKZs2382hZ1Geux41rkeTpD0mKRTJc1TLSI3882s9Yham/lvRsTX51iMtDUwMSLGShpadugI0lSNcwMjSZN0H1fpQq6ZmlkLqqGJX1szf31gW0kTSJN/byrpsjz5eOT5Yy8krSBRkZOpmbWmAu7mR8QREfHFiBgE/A9wV0R8R9ISAJJEms/3iWpluZlvZi1IRd/Nn9XlkhZJF2IcaYWHipxMzaz1iJrv1tcqL1Q4Or/etL2fdzI1sxYkaGuu9NVc0ZiZ1aqt2JppZzmZmlnrqX1oVJdxMjWzFqR2PXvfFZxMzaw1FXwDqrOcTM2sNbmZb2bWSXIz38ysGG7mm5l1Vt2fgGo3J1Mzaz3CzXwzs85zzdTMrBhN1mfaXKndzKxW9V1Qb1lJD0r6t6RRkuauGk4nfhQzs8ZQbuZX22pXWlCv5NfAqRHxZWAysE+1ApxMzaw11WlBvTwh9KZAaWXSi0kTRFfkPlMzazkC2toKqwuWFtTrn98vDLwTEdPz+5eBJasV4pqpmbUe1bhVWZ20fEG9zobkmqmZtSCh2prxFVcn5dMF9bYE+gADgNOBBST1zrXTLwKvVLuQa6Zm1pLa2tqqbtXMYUG9EcDdwI75tD2A66vG0/EfxcyscSRV3TrhMOAQSf8m9aFeUO0DbuabWev5tE+0MLMsqPcfYK32fN7J1MxajlCRd/ML4WRqZi2pk834wjmZmllLcjI1M+ssgbzUs5lZ56j2caZdxsnUzFqSk6mZWWe5mW9mVgzXTM3MCtBsybS5Rr1aU3rppZf41mabsPrXVmbIqqvw+zNOb3RI3d6JRxzIVuuswHe2Wu+TfSNPO4HvbrMBe2y7ET/ZawcmvfFaAyNsLCHUVn3rSk6mVlXv3r056Ten8MhjT/G3+/7BueecxdNPPdXosLq1LXfYjd9dcNVn9o3Y9yAuufE+Lr7hHtbf5FtceNbJDYquCajuz+a3m5OpVbXEEkuw+pAhAPTv35+VVvoKr75adUYy64TV1lyPAfMv+Jl98/Ub8MnrqR9+2HTN3K7WbMnUfabWLi9MmMC4cY+w5lprNzqUHunc3x3Prdf9mfn6D+DMS29odDgN1Wx38+taM5W0uKQ/S3pe0lhJt0haQdIqku6S9Iyk5yQdldddQdKekn4/m7ImSBo4y749JU2SNK5sWzkfWyFf7zlJD0u6UtIuZedNydcfJ+kSSUNLKxPmz28n6TFJT0t6XNJ2ZccukvSKpHny+4GSJtTpa2waU6ZMYdedv83Jp5zGgAEDqn/ACve9Q47k2nueYPNtduKaS89rdDgNVUTNVFIfSf+U9KikJyUdm/dfJGl8Wb5YrVpZdUumOTleC4yOiC9FxBrAEcBiwA3ASRGxIrAqsB5wQAcvNSoiVivbnpLUB7gZ+ENELB8RQ4CzgSdL5wFjgBH5/XdniX1V4LfA8Ij4CrAt8FtJXys7bQawdwdjbjnTpk1j152/zS67jmC77XdodDg93ubb7sTo229sdBgNU0sirbGZ/xGwaUSsCqwGDJO0Tj52aFleGVetoHrWTDcBpkXEOaUdEfEosAJwf0Tcnvd9CBwIHF7gtXcD/h4Rn/xti4jREfFEjZ//KXBiRIzPnx0P/B9waNk5pwEHS+r2XSURwff324cVV/oKPz74kEaH02O9NOH5T17f+/9uYZnllm9gNI1X0Ez7ERFT8tu58hYdiaeeiWAwMLtFqlaZdX9EPC+pn6SOtB13kbRB2ft1K1y7VquQaqblxgA/LHv/InAfsDswxypCXsBrf4Clll66EyE1zgP3388Vl1/K4MFfZe01Umvn2ONPZNgWWzY4su7r6IP35ZF/3s87k99iuw1XYZ8fHc7f/3YHL47/N21tbSz+haU49NhTGh1mY9XWZTpQ0piy9yMjYuRnipF6kfLFl4GzIuJBST8ATpD0S+BO4PCI+KjShbpDrWpURBxYvqML7+L9H2ltmJvndEL+gxsJsMYaX+/Qv3iNtv4GGzB1WkuG3rKOPfX8z+3bZqfdGxBJ86rx97zagnpExAxgNUkLANdKGkzqknwdmJv0+3sYcFylcurZzH8SWGM2+5+adb+k5YApEfFena9dq8/FmN8/Wb4jIp4DxgE7d+JaZtZOErS1qerWHhHxDmkhvWER8VruAvgIuJAaljCpZzK9C5hHZetU5xs4zwAbSNos7+sLnAH8psBrXwGsJ2mrsmtvlP/FqcVvgSMkDcqfHQT8HJhdu+oEUh+rmXWZYm5ASVok10hLueibwL8kLZH3CdgOqHq/pW7N/IgISdsDp0k6DPgvMAH4CTAcOFPSWUAv4FKgfDjUnuVDkYDS3bXHJM3Mr68EHuPzfaYHRMQDkrbO1z4NmJbP/XGNsY/LMd8oaa78+Z/N7o5eRDwp6WFgSC1lm1kxCurNWwK4OPebtgFXRsRNeejmIqSe2XHA96vGE+G+sK6yxhpfj/sfHFP9RKuLh8dPbnQIPdr6Kyw0tlr/Za36LLFCDNrjzKrnPfPrYYVds5rucAPKzHoYQbv7ROvNydTMWlKzTU3gZGpmrUeumZqZdZpovsmhnUzNrAV5dVIzs0K4mW9m1lnyDSgzs05zn6mZWUHczDczK0CTVUydTM2sBcnNfDOzThPtn2Kv3pxMzawlNVnFtL6rk5qZ1UudVyddVtKDkv4taZSkuauV5WRqZi2nwJn257Q66a+BUyPiy8BkYJ9qBTmZmllLKqJmWmF10k2Bq/P+i0mz7VfkZGpmLUmqvpFXJy3b9v98OeolaRwwEbgDeB54JyKm51NeBpasFo9vQJlZ66l9Cr52r04KrNSRkJxMzazlqA6zRkXEO5LuBtYFFpDUO9dOvwi8Uu3zbuabWUuqsZlfpYzZrk76NGnJ5x3zaXsA11cra441U0lnkjpiZysiflQ9VDOz+uhVzKD9Oa1O+hTwZ0nHA48AF1QrqFIz38tomllTUkGPk0bEY8Dqs9n/H2Ct9pQ1x2QaEReXv5c0b0R82J7CzczqpcmeJq3eZypp3Vzl/Vd+v6qks+semZlZBQUN2i8unhrOOQ34FvAWQEQ8CmxUz6DMzCoR+Y5+lf+6Uk1DoyLipVn6J2bUJxwzs9o0WzO/lmT6kqT1gJA0F/Bj0tABM7PGUGtOwfd94HTS41SvArcBP6xnUGZmlQhoa7I5+Kom04h4ExjRBbGYmdWsyXJpTXfzl5N0o6RJkiZKul7Scl0RnJnZ7BQ4BV9harmbfwVwJelJgS8AVwF/qmdQZmbVtElVty6Np4Zz5o2ISyNiet4uA/rUOzAzs0pUw9aVKj2bv1B++VdJhwN/Jj2rvwtwSxfEZmY2W6KwZ/MLU+kG1FhS8ixF/L2yYwEcUa+gzMwqqnEm/a5U6dn8ZbsyEDOz9miyXFrbE1CSBgMrU9ZXGhGX1CsoM7NKimrmS1oKuARYjNTiHhkRp0s6BtgPmJRP/XlEVOzerJpMJR0NDCUl01uALYD7cgBmZg1RUDN/OvC/EfGwpP7AWEl35GOnRsRvay2olrv5OwLfAF6PiL2AVYH52xuxmVmRiribHxGvRcTD+fX7pEflqy6eNzu1JNOpETETmC5pAGkFv6U6cjEzsyJIqZlfbaOG1Uk/LVODSBNFP5h3HSjpMUl/lLRgtZhq6TMdk9dIOY90h38K8PcaPmdmVjc1NvOrrk6ay+oHXAP8JCLek/QH4FekftRfAacAe1cqo5Zn8w/IL8+RdCswIE/1b2bWMEXdzc+z4V0DXB4RfwGIiDfKjp8H3FStnEqD9odUOlbqZzAz62qSirqbL9JieU9HxO/K9i8REa/lt9sDT1Qrq1LN9JQKxwLYtIZYrcxH02cyYdIHjQ6jx/rGzkc1OgQrUEF389cHdgcelzQu7/s5sKuk1Ui5bgKffWhptioN2t+k83GamdVHLXfPq4mI+5j9jf92PzJf06B9M7Nm0mrP5puZNa0my6VOpmbWeqTC+kwLU8tM+5L0HUm/zO+XlrRW/UMzM5uzXm3Vt65Uy+XOBtYFds3v3wfOqltEZmZVlBbUa6aZ9mtp5q8dEUMkPQIQEZMlzV3nuMzMKuriimdVtSTTaZJ6kcZbIWkRYGZdozIzq6CoQftFqiW5nwFcCywq6QTS9Hsn1jUqM7Mq0k2oyltXquXZ/MsljSVNwydgu4h4uu6RmZlV0GQV05omh14a+BC4sXxfRLxYz8DMzOakVQft38ynC+v1AZYFngFWqWNcZmZzphasmUbEV8vf59mkDpjD6WZmXUI1zaXfddr9BFReK2XtegRjZlYLAb2bbGxULX2mh5S9bQOGAK/WLSIzsxoU8ThphdVJFwJGAYNIU/DtHBGTK5VVS27vX7bNQ+pDHd7R4M3MOis9AVV9q0FpddKVgXWAH0paGTgcuDMilgfuzO8rqlgzzYP1+0fET2sKy8ysK6iYu/l5Nv3X8uv3JZVWJx1OWuIe4GJgNHBYpbIqLVvSOyKmS1q/0xGbmRWoVDOtwUBJY8rej4yIkbMt87Orky5WtmzJ66RugIoq1Uz/SeofHSfpBuAq4JM1N0oLT5mZNUKNXaYdXZ30k2MREZKiWhm13M3vA7xFWvOpNN40ACdTM2sIIXoV9Lzo7FYnBd4oLaonaQlgYrVyKiXTRfOd/Cf4NImWVM3SZmZ1U9Cg/TmtTgrcAOwBnJT/f321siol015AP2a/2JSTqZk1VEHzlc5pddKTgCsl7QO8AOxcraBKyfS1iDius5GamRWtqGfzK6xOCmlyp5pVSqbN9ayWmVmZJlsCqmIybVdWNjPrKqKFZtqPiLe7MhAzs5qpsD7TwnipZzNrOaUF9ZqJk6mZtaTmSqVOpmbWkkRbk80O7WRqZi2npW5AmZk1syLmMy2Sk6mZtR7fzTcz6zw3883MCuJmvplZAZrsZn7T1ZTNzKpKzXxV3aqWI/1R0kRJT5TtO0bSK5LG5W3LWmJyMjWzliRV32pwETBsNvtPjYjV8nZLLQW5mW9mLUiF3M2PiHvy2k+d5pqpmbWcopr5FRwo6bHcDbBgLR9wMjWz1lNDEz9XXAdKGlO27V9D6X8AvgSsRloG+pRaQnIz36q65PyzuOryi4gIdhqxF3vs98NGh9QjtLWJ+y//Ga9OfJdv//gcAI754Tbs8M3VmTFjJuddfS9n/+lvDY6ycWps5te0Omm5iHij9FrSecBNtXzOydQqevZfT3LV5Rdx5c1/Y66552a/3bZj6GbDWGbZLzU6tG7vwN024Znxb9B/vj4A7L7tOnxx8QVYdftfEREssmC/BkfYOGkKvjqVnVclzW+3Jy0qWpWb+VbRf557hq+tviZ9552X3r17s+a6G3DHLTc0Oqxub8lFF2DYBqtw4bUPfLJv/5024MSRfyUirWc5afKURoXXFFTDf1XLkP4E/B1YUdLLeQG930h6XNJjwCbAwbXE45qpVbT8Sitz2q+PY/Lbb9GnT1/uuet2Bn9t9UaH1e2dfOi3+cXp19Fv3j6f7Fv2i4uw4+ZrsO2mq/Lm5Pf5399czfMvTmpglI1V0N38XWez+4KOlNVSNVNJM/Ig2ickXSVp3ry/t6RJkk6a5fzRkr6eX0/I/9o8LukpScdL6lN27iqS7pL0jKTnJB2V19RG0p65/NIg3kvy/oskjS/b/6Ou+za6xpeWX4l9DziYfXcdzn4jtmOlVb5KW69ejQ6rW9tiw8FMfPt9Hnn6pc/sn2fu3nz08TQ2GPEbLvzLA5x79IgGRdh4pWZ+ta0rtVQyBabmQbSDgY+B7+f93wSeBXYqJcA52CQivgqsBSwHnAsgqS9wA3BSRKwIrAqsBxxQ9tlRZYN4v1u2/9Cy/WcU8UM2mx1324NrbruPy669nfnnX5BBy3250SF1a+uuthxbb/xV/nXzsVxy0l4MXXMF/nj8d3nljclcd+ejAFx/16MMXn7JBkfaSLU08rs2m7ZaMi13L1D6rd4VOB14EVi32gcjYgopEW8naSFgN+D+iLg9H/8QOBA4vA5xt5y33pwIwKsvv8Qdt1zP1tvv3OCIurdfnnkDXx52FCttdTTfPfxCRj/0LHsfeQk3jn6MjddcHoAN11ief784scGRNlANtdKurpm2ZJ+ppN7AFsCtuam+GfA9YAFSYn2gwscBiIj3JI0HlgdWAcbOcvx5Sf0kDci7dpG0QX59ekRcmF+fLOnI/Hr3iHh8llj3B/YH+MKSS7XzJ20OP953BO9Mfpvec83FUSf+jgHzL9DokHqk3/7xDi48cQ8OGrEpH0z9iB8cd0WjQ2oYL6jXeX0ljcuv7yV1FG8L3B0RUyVdAxwl6ScRMaOG8trzpzEqIg6czf5DI+LqOX0oIkYCIwEGrzok2nG9pnHZdXc0OoQe696xz3Hv2OcAeHfKVHb40TkNjqh5NFcqbb1kOjUiVivfIWlXYANJE/KuhYFNgYoZQFJ/YBCpr/UpYKNZji8HTMk12EKCN7PiNNvvZSv3mZKb4BsCS0fEoIgYBPyQ1NSv9Ll+wNnAdRExGbiclJA3y8f7AmcAv6lj+GbWCQXNGlWYlk6mpKcT7oqIj8r2XQ9sI2me2Zx/d5638J+km1XfA4iIqcBw4EhJzwCPAw8Bv69n8GbWcaph60ot1cyPiH6zvL8YuHiWfW8Di+S3Q8v2D6pS9uPl589y7CLSvIez7t+zSshmVgei+Zr5LZVMzcyAT2aNaiZOpmbWkposlzqZmlkrkpv5ZmZFaLJc2vJ3882sB6rlTn4tuXYOq5MuJOmOPOHRHV62xMy6NUlVtxpcxOdXJz0cuDMilgfupMY5OpxMzawlFTFoPyLuAd6eZfdwPh1yeTGwXS3xuM/UzFpSjV2mAyWNKXs/Ms+XUcliZcuWvA4sVsuFnEzNrPWo5kH77V5Qr1xEhKSaJihyM9/MWk56Aqpuz+a/IWkJSIvrATVNHOtkamYtqY7P5t8A7JFf70Ga76MqN/PNrCUVMWg/r046lNS3+jJwNHAScGVeqfQFoKalJZxMzawlFTFofw6rkwJ8o71lOZmaWUtqsgegnEzNrPV4Cj4zsyJ4Cj4zs2I0WS51MjWzVuQp+MzMCtFkudTJ1MxaTyMWzKvGydTMWpKb+WZmBWiyXOpkamatqclyqZOpmbWg2qfg6zJOpmbWckpT8DUTJ1Mza0lNlkudTM2sNbUVVDWVNAF4H5gBTO/ozPxOpmbWmoqtmm4SEW92pgAnUzNrSc3WzPeyJWbWcqTUzK+2kVcnLdv2n01xAdwuaewcjtfENVMza021VU1rWZ10g4h4RdKiwB2S/hUR97Q3HNdMzawlFbWgXkS8kv8/EbgWWKsj8TiZmlkLqt7Er+Vuv6T5JPUvvQY2B57oSERu5ptZyylw0P5iwLX5aarewBURcWtHCnIyNbMeKyL+A6xaRFlOpmbWkooatF8UJ1Mzaz1eUM/MrPM8076ZWUE8BZ+ZWQGaLJc6mZpZa2qyXOpkamatqdma+YqIRsfQY0iaBLzQ6Dg6YSDQqWnKrFNa/ftfJiIWKaIgSbeSvo9q3oyIYUVcsxonU6uZpDEdnTjXOs/ff3Pzs/lmZgVwMjUzK4CTqbXHyEYH0MP5+29i7jM1MyuAa6ZmZgVwMjUzK4CTqXU5Sf57Z92O/1Jbl5E0BCAiZjqh1o+kAY2OoSfyX2jrSkdKuhOcUOtF0jeBwyT1a3QsPY3/MltX2hmYLOlGcEItmqRhwCnAHRExpdHx9DT+i2x1JWl9SUMkLRgR04GdgA8k/RWcUIsiaR3gbODIiBgtaSlJm/u77Tr+oq1uJC0JXASMBi6TdALwNWA/4FVJV4MTamfl724B4GngFUmDgRuAZSNiZkOD60E8aN/qStL+wDbArcCawBRgdeBi4GTg1ojYpXERtjZJmwMbR8QvJO0HbAIMAS6IiJMbG13P4vlMrXCSFgb6RsTLETEyzzu5CnA18FdgOLAwaTrCtSV9ISJebVjALUrSt4ATgcMAIuI8SW8A8wHjJA2IiPcaGWNP4mRqhZK0NXBkfj0emAD8AtiTdANqOnB9RMyQdC3QFhETGxNt65K0LvBnYHhE3CNpaWDFiLhB0iLAd4AFJN0REe80NNgewsnUCpOH5ZwC7AM8Tuof3QcYBewC9AJ2BPrmX/JWnui4IZSq+QK+QOqLni/3TV8DnA4QERdImg7sCnws6YZwf17duc/UCiPpF8D4iLiibN9CwO+A/0TEcZKOBBYFjoiIDxoUasuTNA+wBbA9sDnwq4g4W1KviJiRzxkB3O0ulK7hmqkVaTFgEeCTZBoRb0v6M/Dd/P54SQs5kbafpA2AtYH3gVci4jpJM4H5gXcBcvfJXBExLSIub2C4PY6Ho1inSPqapIPz21vTLs0/y2ljgSUkDYSUYLsyxu5A0pbApUBf4EvAaZJOA24kjYzYWNIBABExrWGB9mBOptZhklYgLWq2uaQ9gbuBlYGfSVqw7NRhpL9rU7s8yG4g31w6AdgrIo6PiMNIw8w2B06JiGuBW4D1JO3TwFB7NDfzrUPyo4s/BQ4i9YkeRBpDujPpLvPxkvqQBpLvDezipn2HzQSejYjRkPpLI+IdSesB/5B0E3A7EMCDjQuzZ3MytXbL4xt/CxwQEU9LeoF0h/nA/P/hfFpz+hj4dkQ83ah4W5Ekld2BnwqsLGmTiLg7Ij6S1Ccn1LuBxSLiQ9+1bywnU2uXnEhHAX8D7pXUln+R782nHAj0j4g/AvfOqRyran5JU4GFIuI1SVcCQyW9HhFPR8R/83lTgf6NC9NK3GdqNcsDxc8AdgPeJjXvFweIiKmk5HkmMCI/2mgdkG82XUx6Yux2STuQvttFSd/tsHzeHqRWwF0ArpU2lseZWk0krQwMBl6LiHvz+NGRwIvAyRHxWj5vXtLwneci4uWGBdyi8rP2pT7o14CVgONJD0M8A2xIeprsUWA5YO+IeKIhwdpnOJlaVbkmdARwCXAt8F5ETM9DoM4HXqIsoVrHSBpKmmVrh4h4WFLv/D1vApwL/CyPLR1AalW2eZhZ83Az3yrKfaSnkybT+GNEvJ1/wXtFxLvAvqRHG4+RtHgjY+0G5iH1f5Z+LyPfiLobOAo4PI/VfT8i3nEibS5OplbNNsBREfEP8t+X/As+I998ek6qvzAAAAhdSURBVBf4ASkRuJnTAZLWkLRxRNwGfA+4UtKW+bHQXnm+0v8AL0TEm+4bbU5OpjZbeUINSE/blAR85kbH0pL6RsRk0oDyN7oyxm5kY+AESRtGxNXAz4Df54Q6PU/wPBiYKalv2Z+NNREPjbLPkbQR8F/gn6T5RzeQdFdEvFmaSENSb9JEG1cBU11b6riI+J2kGaQFB0+IiKtzvvy9pJ2BpYCfACPyqAlrQk6m9hn5bvI5pPkwAcaQptLbNg8KL02btzNpirdruj7K1idpEDAtIl4BiIjT8z9Qv5BEWUK9G3iFdFPqqUbFa9X5br59QtJWwDHAjyPigTzJ8H+BrYChpPlI7wMWIq3jtLOH5bRPbqKvCDwEjCfdWHokIl7Mx38KfAs4Lg9B24Q0Q9SzjYrZauNkagDkO/E3Ag9GxIGSFgPuBw7PtaS1SGsLbUAa/3iha0odJ+kC0nSFbaSxuu/nCUzIk5X8APhRRDzQuCitPZxMDQBJ/YBvk250vAlsDVwaESMbGlg3Uho3ml9/hzSk7FxSjb9UU70GuJK0MN5DEfFCg8K1dnKfqZWGOk2RdD0wjfT0zcvliTQ/0vgecJeXD26/vKTL3pIeI93Yu57UH/0IaWFBSC2BFUjLkQzxvKStxTXTHkzSYqXhTKVZinINdRtgfWBsRFwoaUfSI43bRMRzDQy5JeUnyI4jTe68KLAM8EtgedIwqFWAH0TE9fn8hSPirQaFax3kmmkPlG+CLAz8U9IREXFFTqSlGuqtpKn01pY0ivTLv70TafvlOQxuIa0ieqOkpUjTF65Kmnu0F+mhiOtLy42QJpGxFuNB+z1QJG+Spss7RtJOpf15HOlk0gTPjwL9gP08H2nH5Ec+twFOUlrH/iXScteLRcTrpJt+B0lasNSs95jd1uSaaQ8kqRfpd/ZGSdOAP+Ra6ZWkWd0hDYcaAOwaEe81KtbuICJuVlr4bqyk20jrOJUWu7uENCjfv4stzn2mPYikFUlPK72Yn/cmImbmPr0/AIdFxJWS9if16W3q8Y3FkbQZaXmRxSNiYl5+5CNJ/SPi/UbHZ53jZNpD5OnyDgMWAP4vIl4qq6GWEuoZwD+AdYEdI+LRxkXcPUnagtRnuklETGx0PFYcJ9MeJA+83w7oA5yWa6hzATPz8/bDgF8Du0fEY42MtTuTNBw4Gvg6uQu7wSFZAZxMuzml5ZjnB94BngcGAfsDcwNnRMSEfN6BwOvATfHp+kJWJ5L6RcSURsdhxXEy7cYkbQucSBoUHsBiwJakZ+v3BuYCDiU9+XQ6sGVEPNKYaM1am5NpNyVpfdKSIt+JiLF539mkNYSGkmqrewGbkcaRfisiHm9MtGatz+NMu6/FScuMjJU0D0BEHECa9el60nPgN5AebXQiNesk10y7GUlfJz29NBxYMyK+lffPFRHT8g2na4HvR8TLpeE5DQzZrFtwzbQbyXfjzwGWBW4CXpI0PK/VNC3PWjSN9Oc+P4ATqVkx/NRFNyFpY+BMYLeIeEhSH9IibJuQnv/+S6RVRXciTf02qXHRmnU/buZ3E5IOAWbk5S/mjoiP87LAewMrk2qrD5DmKR3hcaRmxXLNtMWVps4jJct38+5puWn/pqQzgK+S7uI/AZwfEc83KFyzbst9pi2u7OmZa4F1JK1R2pf7SP9LWm7kroi43YnUrD6cTLuPB0nDnnbJCXVm7iP9H9Lid54j06yO3GfajUhaEtgH+AZpSYypwI6kSUu8iqhZHTmZdjOS+gJrkJ5seg2429PomdWfk6mZWQHcZ2pmVgAnUzOzAjiZmpkVwMnUzKwATqZmZgVwMjUzK4CTqdWNpBmSxkl6QtJVkubtRFkXSdoxvz5f0soVzh0qab0OXGNCnhympv2znNOu9ZwkHSPpp+2N0ZqXk6nV09SIWC0iBgMfA98vPyipQxPtRMS+EfFUhVOGAu1Opmad4WRqXeVe4Mu51nivpBuApyT1knSypIckPSbpe5Bmw5L0e0nPSPp/wKKlgiSNzisKIGmYpIclPSrpTkmDSEn74Fwr3lDSIpKuydd4KK+PhaSFJd0u6UlJ55NWKKhI0nWSxubP7D/LsVPz/jslLZL3fUnSrfkz90paqYgv05qPp+Czuss10C2AW/OuIcDgiBifE9K7EbFmXqvqfkm3A6sDK5LmYl0MeAr44yzlLgKcB2yUy1ooIt6WdA4wJSJ+m8+7Ajg1Iu6TtDRwG/AV0tr190XEcZK2Is1rUM3e+Rp9gYckXRMRbwHzAWMi4mBJv8xlHwiMJC0R85yktYGzgU078DVak3MytXrqK2lcfn0vcAGp+f3PiBif928OfK3UH0paTmV5YCPgTxExA3hV0l2zKX8d4J5SWRExp5mxNgNWlj6peA6Q1C9fY4f82ZslTa7hZ/qRpO3z66VyrG8BM4FRef9lwF/yNdYDriq79jw1XMNakJOp1dPUiFitfEdOKh+U7wIOiojbZjlvywLjaAPWyXO7zhpLzSQNJSXmdSPiQ0mjgT5zOD3ydd+Z9Tuw7sl9ptZotwE/yKumImkFSfMB95DmZu0laQnSWlaz+gewkaRl82cXyvvfB/qXnXc7cFDpjaRScrsH2C3v2wJYsEqs8wOTcyJdiVQzLmkjTXdILvO+iHgPGJ/X3Sr1A69a5RrWopxMrdHOJ/WHPizpCeBcUovpWuC5fOwS4O+zfjAiJgH7k5rUj/JpM/tGYPvSDSjgR8DX8w2up/h0VMGxpGT8JKm5/2KVWG8Fekt6GjiJlMxLPgDWyj/DpsBxef8IYJ8c35OkJbitG/IUfGZmBXDN1MysAE6mZmYFcDI1MyuAk6mZWQGcTM3MCuBkamZWACdTM7MC/H/jngNN9kXkGwAAAABJRU5ErkJggg==\n"
},
"metadata": {
"needs_background": "light"
}
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "vAmd6TsM_U3o"
},
"source": [
"# Model Evaluation using Test set\n"
]
},
{
"cell_type": "code",
"execution_count": 59,
"metadata": {
"id": "xqfW4-FH_U3o"
},
"outputs": [],
"source": [
"from sklearn.metrics import jaccard_score\n",
"from sklearn.metrics import f1_score\n",
"from sklearn.metrics import log_loss\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "dZ1iEibp_U3o"
},
"source": [
"First, download and load the test set:\n"
]
},
{
"cell_type": "code",
"execution_count": 60,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "_xH0BoP8_U3o",
"outputId": "21e99cb2-2d10-49ee-e2cf-a744c7f9eac3"
},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"--2022-08-21 16:46:17-- https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_test.csv\n",
"Resolving s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)... 67.228.254.196\n",
"Connecting to s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)|67.228.254.196|:443... connected.\n",
"HTTP request sent, awaiting response... 200 OK\n",
"Length: 3642 (3.6K) [text/csv]\n",
"Saving to: ‘loan_test.csv’\n",
"\n",
"loan_test.csv 100%[===================>] 3.56K --.-KB/s in 0s \n",
"\n",
"2022-08-21 16:46:18 (645 MB/s) - ‘loan_test.csv’ saved [3642/3642]\n",
"\n"
]
}
],
"source": [
"!wget -O loan_test.csv https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_test.csv"
]
},
{
"cell_type": "markdown",
"metadata": {
"button": false,
"new_sheet": false,
"run_control": {
"read_only": false
},
"id": "YHRsUnXk_U3o"
},
"source": [
"### Load Test set for evaluation\n"
]
},
{
"cell_type": "code",
"execution_count": 61,
"metadata": {
"button": false,
"new_sheet": false,
"run_control": {
"read_only": false
},
"colab": {
"base_uri": "https://localhost:8080/",
"height": 206
},
"id": "ylfA1W95_U3o",
"outputId": "85b89669-ea74-46ea-a601-b08bb658fa5b"
},
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
" Unnamed: 0 Unnamed: 0.1 loan_status Principal terms effective_date \\\n",
"0 1 1 PAIDOFF 1000 30 9/8/2016 \n",
"1 5 5 PAIDOFF 300 7 9/9/2016 \n",
"2 21 21 PAIDOFF 1000 30 9/10/2016 \n",
"3 24 24 PAIDOFF 1000 30 9/10/2016 \n",
"4 35 35 PAIDOFF 800 15 9/11/2016 \n",
"\n",
" due_date age education Gender \n",
"0 10/7/2016 50 Bechalor female \n",
"1 9/15/2016 35 Master or Above male \n",
"2 10/9/2016 43 High School or Below female \n",
"3 10/9/2016 26 college male \n",
"4 9/25/2016 29 Bechalor male "
],
"text/html": [
"\n",
" \n",
"
\n",
"
\n",
"\n",
"
\n",
" \n",
" \n",
" \n",
" Unnamed: 0 \n",
" Unnamed: 0.1 \n",
" loan_status \n",
" Principal \n",
" terms \n",
" effective_date \n",
" due_date \n",
" age \n",
" education \n",
" Gender \n",
" \n",
" \n",
" \n",
" \n",
" 0 \n",
" 1 \n",
" 1 \n",
" PAIDOFF \n",
" 1000 \n",
" 30 \n",
" 9/8/2016 \n",
" 10/7/2016 \n",
" 50 \n",
" Bechalor \n",
" female \n",
" \n",
" \n",
" 1 \n",
" 5 \n",
" 5 \n",
" PAIDOFF \n",
" 300 \n",
" 7 \n",
" 9/9/2016 \n",
" 9/15/2016 \n",
" 35 \n",
" Master or Above \n",
" male \n",
" \n",
" \n",
" 2 \n",
" 21 \n",
" 21 \n",
" PAIDOFF \n",
" 1000 \n",
" 30 \n",
" 9/10/2016 \n",
" 10/9/2016 \n",
" 43 \n",
" High School or Below \n",
" female \n",
" \n",
" \n",
" 3 \n",
" 24 \n",
" 24 \n",
" PAIDOFF \n",
" 1000 \n",
" 30 \n",
" 9/10/2016 \n",
" 10/9/2016 \n",
" 26 \n",
" college \n",
" male \n",
" \n",
" \n",
" 4 \n",
" 35 \n",
" 35 \n",
" PAIDOFF \n",
" 800 \n",
" 15 \n",
" 9/11/2016 \n",
" 9/25/2016 \n",
" 29 \n",
" Bechalor \n",
" male \n",
" \n",
" \n",
"
\n",
"
\n",
"
\n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
"\n",
" \n",
"
\n",
"
\n",
" "
]
},
"metadata": {},
"execution_count": 61
}
],
"source": [
"test_df = pd.read_csv('loan_test.csv')\n",
"test_df.head()"
]
},
{
"cell_type": "markdown",
"source": [
"## preprocess data"
],
"metadata": {
"id": "TFOZgoQ5Ukjg"
}
},
{
"cell_type": "code",
"source": [
"test_df['due_date'] = pd.to_datetime(test_df['due_date'])\n",
"test_df['effective_date'] = pd.to_datetime(test_df['effective_date'])\n",
"test_df['dayofweek'] = test_df['effective_date'].dt.dayofweek\n",
"test_df['weekend'] = test_df['dayofweek'].apply(lambda x: 1 if (x>3) else 0)\n",
"test_df['Gender'].replace(to_replace=['male','female'], value=[0,1],inplace=True)\n",
"Feature = test_df[['Principal','terms','age','Gender','weekend']]\n",
"Feature = pd.concat([Feature,pd.get_dummies(test_df['education'])], axis=1)\n",
"Feature.drop(['Master or Above'], axis = 1,inplace=True)\n",
"X = Feature\n",
"y_test = test_df['loan_status'].values\n",
"# normalize date\n",
"X_test= preprocessing.StandardScaler().fit(X).transform(X)"
],
"metadata": {
"id": "j27bTB5rTkNu"
},
"execution_count": 62,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"## predict and evaluate with test data"
],
"metadata": {
"id": "JRlvNZ1_U9Ii"
}
},
{
"cell_type": "code",
"execution_count": 63,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "6zXrtRUP_U3p",
"outputId": "1ef267ba-deae-407a-cc39-2eb6084ce8cc"
},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"KNN f1: 0.6736355806123249\n",
"KNN Jaccard: 0.6862745098039216\n",
"\n",
"DT f1: 0.6304176516942475\n",
"DT Jaccard: 0.7407407407407407\n",
"\n",
"SVM f1: 0.7583503077293734\n",
"SVM Jaccard: 0.78\n",
"\n",
"LR f1: 0.6604267310789049\n",
"LR Jaccard: 0.7358490566037735\n",
"LR log loss: 0.5672153379912981\n",
"\n"
]
}
],
"source": [
"# KNN\n",
"yhat_knn = neigh.predict(X_test)\n",
"print(\"KNN f1:\", f1_score(y_test, yhat_knn, average='weighted'))\n",
"print(\"KNN Jaccard:\", jaccard_score(y_test, yhat_knn, pos_label='PAIDOFF'))\n",
"print()\n",
"# Decision Tree\n",
"yhat_dt = loanTree.predict(X_test)\n",
"print(\"DT f1:\", f1_score(y_test, yhat_dt, average='weighted'))\n",
"print(\"DT Jaccard:\", jaccard_score(y_test, yhat_dt, pos_label='PAIDOFF'))\n",
"print()\n",
"# SVM\n",
"yhat_svm = clf.predict(X_test)\n",
"print(\"SVM f1:\", f1_score(y_test, yhat_svm, average='weighted'))\n",
"print(\"SVM Jaccard:\", jaccard_score(y_test, yhat_svm, pos_label='PAIDOFF'))\n",
"print()\n",
"# LR\n",
"yhat_lr = LR.predict(X_test)\n",
"yhat_prob = LR.predict_proba(X_test)\n",
"print(\"LR f1:\", f1_score(y_test, yhat_lr, average='weighted'))\n",
"print(\"LR Jaccard:\", jaccard_score(y_test, yhat_lr, pos_label='PAIDOFF'))\n",
"print(\"LR log loss:\", log_loss(y_test, yhat_prob))\n",
"print()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "v1C_0my0_U3p"
},
"source": [
"# Report\n",
"\n",
"You should be able to report the accuracy of the built model using different evaluation metrics:\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "w-TO5vJJ_U3p"
},
"source": [
"| Algorithm | Jaccard | F1-score | LogLoss |\n",
"| ------------------ | ------- | -------- | ------- |\n",
"| KNN | 0.69 | 0.67 | NA |\n",
"| Decision Tree | 0.74 | 0.63 | NA |\n",
"| SVM | 0.78 | 0.76 | NA |\n",
"| LogisticRegression | 0.74 | 0.66 | 0.57 |\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"button": false,
"new_sheet": false,
"run_control": {
"read_only": false
},
"id": "wYo-zN_C_U3p"
},
"source": [
"Want to learn more? \n",
"\n",
"IBM SPSS Modeler is a comprehensive analytics platform that has many machine learning algorithms. It has been designed to bring predictive intelligence to decisions made by individuals, by groups, by systems – by your enterprise as a whole. A free trial is available through this course, available here: SPSS Modeler \n",
"\n",
"Also, you can use Watson Studio to run these notebooks faster with bigger datasets. Watson Studio is IBM's leading cloud solution for data scientists, built by data scientists. With Jupyter notebooks, RStudio, Apache Spark and popular libraries pre-packaged in the cloud, Watson Studio enables data scientists to collaborate on their projects without having to install anything. Join the fast-growing community of Watson Studio users today with a free account at Watson Studio \n",
"\n",
"Thanks for completing this lesson! \n",
"\n",
"\n",
"Saeed Aghabozorgi , PhD is a Data Scientist in IBM with a track record of developing enterprise level applications that substantially increases clients’ ability to turn data into actionable knowledge. He is a researcher in data mining field and expert in developing advanced analytic methods like machine learning and statistical modelling on large datasets.
\n",
"\n",
" \n",
"\n",
"## Change Log\n",
"\n",
"| Date (YYYY-MM-DD) | Version | Changed By | Change Description |\n",
"| ----------------- | ------- | ------------- | ------------------------------------------------------------------------------ |\n",
"| 2020-10-27 | 2.1 | Lakshmi Holla | Made changes in import statement due to updates in version of sklearn library |\n",
"| 2020-08-27 | 2.0 | Malika Singla | Added lab to GitLab |\n",
"\n",
" \n",
"\n",
"## © IBM Corporation 2020. All rights reserved. \n",
"\n",
" \n"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python",
"language": "python",
"name": "conda-env-python-py"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.12"
},
"colab": {
"name": "MachineLearningwithPython-Project.ipynb",
"provenance": [],
"collapsed_sections": [],
"toc_visible": true
}
},
"nbformat": 4,
"nbformat_minor": 0
}