{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Vectors\n", "\n", "Informally, we think of a vector as an object that has magnitude and direction. More formally, we think of an $n$-dimensional vector as an ordered tuple of numbers $(x_1, x_2, \\ldots, x_n)$ that follows the rules of scalar multiplication and vector addition." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "collapsed": true }, "outputs": [], "source": [ "%matplotlib inline" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import numpy as np\n", "import matplotlib.pyplot as plt" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Vector space\n", "\n", "A vector space is a collection of vectors which is closed under addition and scalar multiplication. \n", "\n", "Examples:\n", "\n", "- Euclidean plane $\\mathbb{R}^2$ is a familiar vector space\n", "- The vector $\\pmatrix{0 & 0}$ is a trivial vector space that is a **vector subspace** of Euclidean space.\n", "- Polynomial functions of order $k$ is a vector space\n", "\n", "Polynomials of order 3 have the form $ax^3 + bx^2 + cx + d$, and can be represented as the vector\n", "\n", "$$\n", "\\pmatrix{a \\\\ b \\\\ c \\\\ d}\n", "$$\n", "\n", "- The space of all continuous functions is a vector space\n", "\n", "Consider two continuous functions, say, $f(x) = x^2$ and $g(x) = x^3)$. Scalar multiplication $(2 f)(x) = 2x^2$ and addition $(f + g)(x) = x^2 + x^3$ are well defined and the result is a continuous function, so the space of all continuous functions is also a vector space. In this case, it is an infinite-dimensional vector space. \n", "\n", "Vector spaces are important because the theorems of linear algebra apply to all vector spaces, not just Euclidean space." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Column vectors\n", "\n", "When we describe a vector $x$, we mean the column vector. The row vector is denoted $x^T$." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[0.17177801],\n", " [0.21054051],\n", " [0.21676041],\n", " [0.73357656],\n", " [0.36831598]])" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "x = np.random.random((5,1))\n", "x" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[0.17177801, 0.21054051, 0.21676041, 0.73357656, 0.36831598]])" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "x.T" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Dimensions" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "5" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "len(x)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Length\n", "\n", "The length of a vector is the Euclidean norm (i.e. Pythagoras theorem)" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0.8914097289661268" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "np.linalg.norm(x)" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0.8914097289661268" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "np.sqrt(np.sum(x**2))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Direction" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[0.19270377],\n", " [0.23618826],\n", " [0.24316585],\n", " [0.82293982],\n", " [0.41318371]])" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "n = x/np.linalg.norm(x)\n", "n" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "1.0" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "np.linalg.norm(n)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Norms and distances\n", "\n", "Recall that the 'norm' of a vector $v$, denoted $||v||$ is simply its length. For a vector with components \n", "$$v = \\left(v_1,...,v_n\\right)$$\n", "the norm of $v$ is given by:\n", " \n", "$$||v|| = \\sqrt{v_1^2+...+v_n^2}$$\n", "\n", "The distance between two vectors is the length of their difference:\n", " \n", "$$d(v,w) = ||v-w||$$" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "collapsed": true }, "outputs": [], "source": [ "u = np.array([3,0]).reshape((-1,1))\n", "v = np.array([0,4]).reshape((-1,1))" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "5.0" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "np.linalg.norm(u - v)" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "5.0" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "np.linalg.norm(v - u)" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "5.0" ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "np.sqrt(np.sum((u - v)**2))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Vector operations" ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "collapsed": true }, "outputs": [], "source": [ "x = np.arange(3).reshape((-1,1))\n", "y = np.arange(3).reshape((-1,1))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Scalar multiplication" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[0],\n", " [3],\n", " [6]])" ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "3 * x" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Vector addition" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[0],\n", " [2],\n", " [4]])" ] }, "execution_count": 16, "metadata": {}, "output_type": "execute_result" } ], "source": [ "x + y" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Linear operations" ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[ 0],\n", " [ 7],\n", " [14]])" ] }, "execution_count": 17, "metadata": {}, "output_type": "execute_result" } ], "source": [ "3*x + 4*y" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Transposition" ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[0, 1, 2]])" ] }, "execution_count": 18, "metadata": {}, "output_type": "execute_result" } ], "source": [ "x.T" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Dot product\n", "\n", "The dot product of two vectors $u$ and $v$ is written as $u \\cdot v$ and its value is given by $u^Tv$. The dot product of two $n$ dimensional vectors $v$ and $w$ is given by:\n", "\n", "$$u \\cdot v = u_1v_1+...+u_nv_n$$\n", "\n", "I.e. the dot product is just the sum of the product of the components.\n", "\n", "The inner product $\\langle u,v \\rangle$ of two vectors is a generalization of the dot product. It is any function that takes two vectors, returns a scalar (here we just consider inner products that return real numbers), and obeys the following properties:\n", "\n", "- symmetry $\\langle u,v \\rangle = \\langle v,u \\rangle$\n", "- positive definite \n", " - $\\langle v,v \\rangle \\ge 0$\n", " - $\\langle v,v \\rangle = 0 \\implies v = 0$\n", "- bilinear\n", " - $\\langle au,v \\rangle = a \\langle u,v \\rangle$\n", " - $\\langle u + v,w \\rangle = \\langle u,w \\rangle + \\langle v,w \\rangle$\n", " - Linearity also applies to second argument because of symmetry\n", "\n", "Any inner product determines a norm via:\n", "\n", "$$||v|| = \\langle v,v \\rangle^{\\frac12}$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Calculating dot products" ] }, { "cell_type": "code", "execution_count": 19, "metadata": { "collapsed": true }, "outputs": [], "source": [ "u = np.array([3,3]).reshape((-1,1))\n", "v = np.array([2,0]).reshape((-1,1))" ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[6]])" ] }, "execution_count": 20, "metadata": {}, "output_type": "execute_result" } ], "source": [ "np.dot(u.T, v)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You can also use the `@` operator to do matrix multiplication" ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[6]])" ] }, "execution_count": 21, "metadata": {}, "output_type": "execute_result" } ], "source": [ "u.T @ v" ] }, { "cell_type": "code", "execution_count": 22, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "6" ] }, "execution_count": 22, "metadata": {}, "output_type": "execute_result" } ], "source": [ "np.sum(u * v)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Geometry of dot product\n", "\n", "Geometrically, the dot product is the product of the length of $v$ and the length of the projection of $u$ onto the unit vector $\\widehat{v}$. \n", "\n", "$$\n", "u \\cdot v = \\lvert u \\rvert \\lvert v \\rvert \\cos \\theta\n", "$$" ] }, { "cell_type": "code", "execution_count": 23, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl4VFW2/vHvElDvFRSVKIgiduttBRWUCNhOtDjg0NA8\nSouKsxccaHFoh59tO88TDigYJlERQURABiEYFEVAwhQJOICioCgRNQ4oEFi/P3a1F2MwBamqU8P7\neZ48VKqOqXW66LysffbZ29wdERGRdLNN1AWIiIhURQElIiJpSQElIiJpSQElIiJpSQElIiJpSQEl\nIiJpSQElIiJpSQElIiJpSQElIiJpqXZUb9ygQQNv2rRpVG8vIiIRmTNnzlfunlfdcZEFVNOmTSku\nLo7q7UVEJCJm9kk8x2mIT0RE0pICSkRE0pICSkRE0pICSkRE0pICSkRE0pICSkRE0lK1AWVm25vZ\nO2a2wMxKzey2Ko7ZzsyGm9kSM5tlZk2TUayIiOSOeDqotcCx7t4CaAl0MLO2lY65CPjG3fcFegP3\nJbZMERHJNdUGlAc/xL6tE/vySod1AobEHo8E2puZJaxKERGJ1LJlcOqpMG9e6t4zrmtQZlbLzOYD\nq4BCd59V6ZDGwHIAd68AyoFdq/g53c2s2MyKy8rKala5iIgk3YYN8Mgj0Lw5vPEGLFmSuveOK6Dc\nfYO7twT2BFqb2YFb82buXuDu+e6en5dX7TJMIiISoZISOPxwuOoqaNcOSkuhS5fUvf8WzeJz92+B\nqUCHSi99BuwFYGa1gZ2A1YkoUEREUuvnn+Gmm6BVqzC0N2wYjBsHTZqkto54ZvHlmVn92OP/Ao4H\n3qt02FjgvNjj04Eid698nUpERNLctGnQogXcdRecfTYsXgxdu0IUswri6aAaAVPNrASYTbgGNc7M\nbjezjrFjBgK7mtkS4GrghuSUKyIiyVBeDpdcAsccA+vXw+TJ8PTTsOtvZhOkTrXbbbh7CXBIFc/f\nvMnjn4EUjkyKiEiijB4Nl18OX3wB11wDt90GO+wQdVVaSUJEJGetXAmnnw6dO0NeHsyaBQ8+mB7h\nBAooEZGc4w4DBsABB4TJD/fcA7NnQ35+1JX9WmQ76oqISOp9+CF07w6vvx6mjhcUwH77RV1V1dRB\niYjkgPXr4d574aCDwmoQ/ftDUVH6hhOogxIRyXrFxXDxxbBgAZx2Gjz+ODRqFHVV1VMHJSKSpX78\nEf75T2jTBsrK4OWXYeTIzAgnUAclIpKVCguhRw/4+OPw5333wU47RV3VllEHJSKSRVavhvPPhxNO\ngDp1wgKv/fplXjiBAkpEJCu4wwsvhKnjQ4eGtfQWLICjj466sq2nIT4RkQz36adw2WUwfjy0bg2v\nvRZm62U6dVAiIhlqwwbo0yfs1TR1KvTuDW+/nR3hBOqgREQyUmkp/O//wowZcOKJ4TpT06ZRV5VY\n6qBERDLI2rVwyy1wyCHwwQfw3HMwcWL2hROogxIRyRjTp4euafFi6NYNHn44LPKardRBiYikue++\nC9thHHkkrFkTOqZnn83ucAIFlIhIWnvlFWjWDPr2hSuvhIULoUOHqKtKDQWUiEga+vJLOOMM6NgR\ndt45TIbo3Rvq1o26stRRQImIpBF3GDw43HA7ejTceSfMmRPW09sS7dq1o2fPnr967vzzz+fUU09N\nYLXJpUkSIiJpYunSsG7ea6/BUUeFvZr23z/qqqKjDkpEJGIVFfDAA+EG29mzwz1Nr7+e2+EE6qBE\nRCI1bx5cdFH4s1MneOIJaNw46qrSgzooEZEIrFkD118Phx0GK1eGfZpefjlx4bTNNtvg7r96bv36\n9Yn54SmigBIRSbGiIjj4YLj/frjgAli0KOx0a5a498jLy2PlypW/em7BggWJe4MUUECJiKTIN9+E\n4bz27UMYFRVB//5hGnmiHXvssUycOJGxY8fy/vvvc/XVV7N8+fLEv1ESKaBERJLMHV58MUwdHzIE\nbrgBSkrgL39J3nteeOGFv3wdccQR1KtXj86dOyfvDZPAKo9Rpkp+fr4XFxdH8t4iIqmyYkVYpmjs\nWGjVCgYMgJYto64qWmY2x93zqztOHZSISBJs3BiWJ2rWDAoL4cEHYeZMhdOW0DRzEZEEW7w4rDo+\nfTocdxw89RT84Q9RV5V5qu2gzGwvM5tqZovMrNTMelVxTDszKzez+bGvm5NTrohI+lq3Du64I3RJ\nixbB00/D5MkKp60VTwdVAVzj7nPNrB4wx8wK3X1RpePedPfMWeRJRCSBZs6Eiy8OO9127QqPPAK7\n7x51VZmt2g7K3Ve6+9zY4++BxYDucxYRAb7/Hnr1gj//GcrLw/YYw4YpnBJhiyZJmFlT4BBgVhUv\nH25mC8xsopk138x/393Mis2suKysbIuLFRFJJxMmwIEHwuOPh5l6ixZBBi0WnvbiDigzqwu8BFzp\n7t9VenkusLe7twAeB0ZX9TPcvcDd8909Py/bt4IUkaxVVgZnnQWnnBL2Z5o+PYRUvXpRV5Zd4goo\nM6tDCKeh7j6q8uvu/p27/xB7PAGoY2YNElqpiEjE3MNW6wccENbOu/VWmDsXDj886sqyU7WTJMzM\ngIHAYnd/eDPHNAS+dHc3s9aE4Fud0EpFRCL08cdhr6bCwhBIAwaEe5wkeeKZxXcEcA7wrpnNjz13\nI9AEwN37AacDl5pZBfAT0NWjWqJCRCSBNmyARx+Ff/8bttkmbIdxySXhsSRXtQHl7m8Bv7vGrrv3\nAfokqigRkXSwYEG44Xb27DD54cknYa+9oq4qd+jfACIilfz0E9x4I+TnwyefwPDhYS09hVNqaakj\nEZFNvPFG6Jo+/DDs1fTgg7DLLlFXlZvUQYmIAN9+C927Q7t24bpTYSEMGqRwipICSkRy3qhRYUbe\nwIFw7bXw7rthkVeJlob4RCRnff459OwJL78cFnh95ZWwZ5OkB3VQIpJzNm6EgoLQNU2cCPfdB++8\no3BKN+qgRCSnvP9+uNY0bVrYcr2gAPbdN+qqpCrqoEQkJ6xfD3ffDS1aQElJuN702msKp3SmDkpE\nst7s2WGvppIS6NIFHnsMGjaMuiqpjjooEclaP/4IV18NbdvCV1/B6NEwYoTCKVOogxKRrDRpUlgz\nb9kyuPRSuOce2GmnqKuSLaEOSkSyyldfwbnnQocOsP328OabYQ09hVPmUUCJSFZwh+efD3s1DRsW\nVh+fNw+OPDLqymRraYhPRDLeJ5+EYbyJE6FNG+jfHw46KOqqpKbUQYlIxtqwIczIa9483Nf06KNh\n+3WFU3ZQByUiGWnhwjB1fNYsOOkk6NsX9t476qokkdRBiUhG+flnuPlmOOQQWLoUhg6F8eMVTtlI\nHZSIZIy33gp7Nb33HpxzDjz8MDRoEHVVkizqoEQk7ZWXw2WXwVFHhd1uX30VnnlG4ZTtFFAiktbG\njAmTIJ56Cq66Klx7OvHEqKuSVFBAiUha+uKLsG7e3/4Gu+4KM2eGIb26daOuTFJFASUiacU9bLV+\nwAFhA8G77oLiYjjssKgrk1TTJAkRSRtLloS9mqZOhaOPDns1/elPUVclUVEHJSKRq6gIu9oedBDM\nnRuCaepUhVOuUwclIpGaOzfccDtvHnTuDH36wB57RF2VpAN1UCISiTVr4Nprw7WlL76Al16CUaMU\nTvJ/1EGJSMpNmQI9esBHH4VrTvfdB/XrR12VpBt1UCKSMl9/DRdcAMcfD7Vqweuvh/ubFE5SlWoD\nysz2MrOpZrbIzErNrFcVx5iZPWZmS8ysxMwOTU65IpKJ3GH48DB1/Lnn4MYbYcECOOaYqCuTdBbP\nEF8FcI27zzWzesAcMyt090WbHHMSsF/sqw3QN/aniOS45cvDMkXjxkF+PkyeDC1aRF2VZIJqOyh3\nX+nuc2OPvwcWA40rHdYJeMaDmUB9M2uU8GpFJGNs3AhPPAHNmkFRUVgFYuZMhZPEb4smSZhZU+AQ\nYFallxoDyzf5fkXsuZU1qE1EMtSiRWHV8bffhhNOgH79YJ99oq5KMk3ckyTMrC7wEnClu3+3NW9m\nZt3NrNjMisvKyrbmR4hIGlu7Fm67DVq2DFtiPPNMWHlc4SRbI66AMrM6hHAa6u6jqjjkM2CvTb7f\nM/bcr7h7gbvnu3t+Xl7e1tQrImlqxgw49FC49dawyOvixWHPJrOoK5NMFc8sPgMGAovd/eHNHDYW\nODc2m68tUO7uGt4TyQHffw//+AcccUR4PH582OV2t92irkwyXTzXoI4AzgHeNbP5seduBJoAuHs/\nYAJwMrAEWANckPhSRSTdjB8Pl1wCn30WQurOO6FevairkmxRbUC5+1vA7zbp7u7A5YkqSkTS26pV\n0KsXvPBC2EzwxRehbduoq5Jso5UkRCRu7jBkSLjhdtQouP32sNirwkmSQWvxiUhcPvoorJ83ZUq4\n3tS/fwgqkWRRByUiv6uiAh56CA48EGbNgiefhGnTFE6SfOqgRGSz5s8PezXNmQN//WsIpz33jLoq\nyRXqoETkN376Cf7f/wtr561YASNGwJgxCidJLXVQIvIrU6eGPZqWLIELL4QHHoBddom6KslF6qBE\nBIBvvgnr5x17bJitN2UKDByocJLoKKBEcpw7jBwZJj0MHgzXXQclJdC+fdSVSa7TEJ9IDvvsM7j8\n8nB96ZBDYMKEsJ6eSDpQByWSgzZuDFtgNGsGkybB/ffDO+8onCS9qIMSyTHvvReuNb31VhjGe+op\n+OMfo65K5LfUQYnkiHXrwmKuLVpAaSkMGgSFhQonSV/qoERywKxZ4YbbhQvh73+HRx+Fhg2jrkrk\n96mDEsliP/wAV14Jhx8O334LY8fC8OEKJ8kM6qBEstSrr4bFXT/9NMzUu/tu2HHHqKsSiZ86KJEs\nU1YG3brBSSfBDjuEyRB9+iicJPMooESyhDs891y44XbECLjlFpg3L2yNIZKJNMQnkgWWLQtbr0+a\nFDYPHDAg7HQrksnUQYlksA0b4JFHQhhNnw6PPx6G9BROkg3UQYlkqJKSMHV89mw4+WTo2xeaNIm6\nKpHEUQclkmF+/hluuglatQpDe8OGwbhxCifJPuqgRDLItGlhmaIPPoDzzgtbse+6a9RViSSHOiiR\nDFBeHiZBHHMMrF8PkyfD008rnCS7KaBE0tzo0WHqeP/+cM018O67cPzxUVclknwKKJE0tXIlnH46\ndO4Mu+0W1tN78MFw861ILlBAiaQZ93Af0wEHhMkP99wTZurl50ddmUhqaZKESBr58EPo3h1efz1c\nb+rfH/bbL+qqRKKhDkokDaxfD/feCwcdFJYn6t8fiooUTpLb1EGJRKy4ONxwu2ABnHZaWA2iUaOo\nqxKJXrUdlJkNMrNVZrZwM6+3M7NyM5sf+7o58WWKZJ8ff4R//hPatIFVq2DUKBg5UuEk8h/xdFBP\nA32AZ37nmDfd/dSEVCSSAwoLw15NH38c/rz3XqhfP+qqRNJLtR2Uu08Dvk5BLSJZb/XqsALECSdA\nnTrwxhvQr5/CSaQqiZokcbiZLTCziWa22XWUzay7mRWbWXFZWVmC3lok/bmHNfMOOACefx7+9a9w\nzenoo6OuTCR9JWKSxFxgb3f/wcxOBkYDVc49cvcCoAAgPz/fE/DeImnv00/hsstg/Hg47DCYMgUO\nPjjqqkTSX407KHf/zt1/iD2eANQxswY1rkwkw23YEGbkNW8OU6dC794wY4bCSSReNe6gzKwh8KW7\nu5m1JoTe6hpXJpLBSkvD1PGZM+HEE8N1pqZNo65KJLNUG1BmNgxoBzQwsxXALUAdAHfvB5wOXGpm\nFcBPQFd31/Cd5KS1a+Huu8PyRDvuCM8+C2efDWZRVyaSeaoNKHc/s5rX+xCmoYvktOnTw15NixdD\nt27w8MOQlxd1VSKZS0sdidTQd9/B5ZfDkUeGm28nTgydk8JJpGYUUCI18Mor0KwZ9O0LvXqFa08d\nOkRdlUh2UECJbIUvv4QzzoCOHWHnncPsvEcegbp1o65MJHsooES2gDsMHhxuuB09Gu68E+bMCevp\niUhiaTVzkTgtXRr2aioqgqOOgoIC2H//qKsSyV7qoESqUVEBDzwQ9moqLg73NL3+usJJJNnUQYn8\njrlzww238+ZBp07wxBPQuHHUVYnkBnVQIlVYswauvx5at4aVK8M+TS+/rHASSSV1UCKVFBWFa01L\nl4bu6f77w0w9EUktdVAiMV9/DRddBO3bh6WJioqgf3+Fk0hUFFCS89zhxRfDDbdDhsANN0BJCfzl\nL1FXJpLbNMQnOW3FirBM0dix0KoVvPoqtGwZdVUiAuqgJEdt3AhPPhm6psJCePDBsDWGwkkkfaiD\nkpyzeHFYdXz6dDjuOHjqKfjDH6KuSkQqUwclOWPdOrjjjtAlLVoETz8NkycrnETSlTooyQkzZoSu\nqbQUunYNC7vuvnvUVYnI71EHJVnt++/hiivgiCOgvDxsjzFsmMJJJBOog5KsNWECXHopLF8eZurd\nfTfUqxd1VSISL3VQknVWrYKzzoJTToEddoC33oLHH1c4iWQaBZRkDXd45pmwV9PIkXDrrWGR1z//\nOerKRGRraIhPssLHH0OPHuGepsMPhwEDwj1OIpK51EFJRquogIcfhgMPDDP1+vQJQ3oKJ5HMpw5K\nMtaCBWG18eJiOPXUsDLEXntFXZWIJIo6KMk4P/0EN94I+fnw6afwwgthLT2Fk0h2UQclGeWNN8IN\ntx9+COefDw89BLvsEnVVIpIM6qAkI3z7bdhEsF27cN2psBAGD1Y4iWQzBZSkvVGjwqSHgQPh2mth\n4cKwyKuIZDcN8Una+vxz6NkTXn45LPD6yithzyYRyQ3VdlBmNsjMVpnZws28bmb2mJktMbMSMzs0\n8WVKLtm4EQoKwg23EyfCvffCO+8onERyTTxDfE8DHX7n9ZOA/WJf3YG+NS9LctX774et1nv0CIH0\n7rtw/fVQp07UlYlIqlUbUO4+Dfj6dw7pBDzjwUygvpk1SlSBkhvWrw+LubZoASUl4XrTa6/BvvtG\nXZmIRCUR16AaA8s3+X5F7LmVlQ80s+6ELosmTZok4K0lG8yeDRddFLqlLl3gscegYcOoqxKRqKV0\nFp+7F7h7vrvn5+XlpfKtJQ39+CNcfTW0bQurV8Po0TBihMJJRIJEdFCfAZvew79n7DmRzZo0CS65\nBJYtC3s23XMP7LRT1FWJSDpJRAc1Fjg3NpuvLVDu7r8Z3hMB+OorOPdc6NABttsOpk0La+gpnESk\nsmo7KDMbBrQDGpjZCuAWoA6Au/cDJgAnA0uANcAFySpWMpd72Gq9V6+wKsS//x3W09t++6grE5F0\nVW1AufuZ1bzuwOUJq0iyziefhGG8iROhTRvo3x8OOijqqkQk3WmpI0maDRvg0UehefMwlPfoozB9\nusJJROKjpY4kKRYuDHs1zZoFJ50EffvC3ntHXZWIZBJ1UJJQP/8MN98MhxwCS5fC0KEwfrzCSUS2\nnDooSZg33wx7Nb3/PpxzTtiKvUGDqKsSkUylDkpqrLw8TII4+ujQQb36KjzzjMJJRGpGASU1MmZM\nmARRUABXXRWuPZ14YtRViUg2UEDJVvnii7Bu3t/+Fna1nTEjDOnVrRt1ZSKSLRRQskXcw0rjBxwQ\nNhC86y6YMwdat466MhHJNpokIXFbsgS6d4epU8P1poIC+NOfoq5KRLKVOiip1vr1cN994QbbOXPg\nqadCSCmcRCSZ1EHJ75ozJ9xwO38+dO4MffrAHntEXZWI5AJ1UFKlNWvg2mvDtaUvv4SXXoJRoxRO\nIpI66qDkN6ZMgR494KOPwo23998P9etHXZWI5Bp1UPKLr7+GCy6A44+HWrXCdaaCAoWTiERDASW4\nw/DhYer4c8+FfZoWLIB27aKuTERymYb4ctzy5XDZZTBuHOTnw+TJ0KJF1FWJiKiDylkbN8ITT0Cz\nZlBUBA89FFaDUDiJSLpQB5WDFi0Kkx/efhtOOAH69YN99om6KhGRX1MHlUPWroXbboOWLeG998KK\n46++qnASkfSkDipHzJgRbrhdtAjOOgt694bddou6KhGRzVMHleW+/x7+8Q844ojwePz4sMutwklE\n0p0CKouNGxcmQTzxRAip0lI4+eSoqxIRiY8CKgutWgVnngl//SvstFOYDPHoo1CvXtSViYjETwGV\nRdxhyJBww+2oUXD77TB3LrRtG3VlIiJbTpMkssRHH4X186ZMCdeb+vcPQSUikqnUQWW4iopwk+2B\nB8KsWfDkkzBtmsJJRDKfOqgMNn9+mDo+Z0643vTkk7DnnlFXJSKSGOqgMtBPP8ENN4S185YvhxEj\nYMwYhZOIZJe4AsrMOpjZ+2a2xMxuqOL1882szMzmx74uTnypAmELjIMPDluwn3ceLF4MXbqAWdSV\niYgkVrUBZWa1gCeAk4BmwJlm1qyKQ4e7e8vY14AE15nzvvkmrJ937LFhtt6UKTBwIOyyS9SViYgk\nRzwdVGtgibt/5O7rgBeATsktS/7DHUaODJMeBg+G666DkhJo3z7qykREkiuegGoMLN/k+xWx5yo7\nzcxKzGykme1V1Q8ys+5mVmxmxWVlZVtRbm757DPo3DkM4e2xB7zzThja++//jroyEZHkS9QkiVeA\npu5+MFAIDKnqIHcvcPd8d8/Py8tL0Ftnn40bwxYYzZrBpElw//0hnA49NOrKRERSJ56A+gzYtCPa\nM/bcL9x9tbuvjX07AGiVmPJyz3vvwTHHwKWXhll6CxfCtddCbd0QICI5Jp6Amg3sZ2b7mNm2QFdg\n7KYHmFmjTb7tCCxOXIm5Yd06uPPOsKNtaSkMGhQmQvzxj1FXJiISjWr/Xe7uFWbWE5gE1AIGuXup\nmd0OFLv7WOAKM+sIVABfA+cnseasM2tWuOF24UL4+9/Dwq4NG0ZdlYhItMzdI3nj/Px8Ly4ujuS9\n08UPP8BNN8Fjj4VJEE8+CR07Rl2ViEhymdkcd8+v7jhd2YjIxIlwySXw6adw2WVwzz2w445RVyUi\nkj601FGKlZVBt25h48AddoC33gobCiqcRER+TQGVIu7w3HPhhtsRI+Dmm2HevLA1hoiI/JaG+FJg\n2bIwnDdpUtg8sH//sD2GiIhsnjqoJNqwAR55BJo3h+nT4fHHw5CewklEpHrqoJKkpCRMHZ89O1xv\n6tsXmjSJuioRkcyhDirBfv45TB1v1SoM7T3/PIwbp3ASEdlS6qASaNq0sCXGBx+EvZoeegh23TXq\nqkREMpM6qAQoLw+TII45JixZNHkyPP20wklEpCYUUDX08sth6nj//nDNNWG5ouOPj7oqEZHMpyG+\nrbRyJfTsCaNGhQVex44Nq4+LiEhiqIPaQu4wYEDomsaPD0sUzZ6tcBIRSTR1UFvggw+ge3d4441w\nvamgAP7nf6KuSkQkO6mDisP69aFTOvhgmD8/XG8qKlI4iYgkkzqoahQXhxtuFyyA004Lq0E0alT9\nfyciIjWjDmozfvwxzMpr0wZWrQqTIUaOVDiJiKSKOqgqTJ4MPXqElSB69IB774X69aOuSkQkt6iD\n2sTq1WEFiBNPhG23DZMh+vVTOEl2KygoYPfdd2fDhg2/ev6ss86io7Z4lggpoAhTx4cNC1PHn38e\n/vWvcM3p6KOjrkwk+bp06UJ5eTmFhYW/PPfDDz8wZswYunXrFmFlkutyPqA+/RROPRXOOguaNoU5\nc+DOO2H77aOuTCQ1dt55Z04++WSGDh36y3OjR4+mdu3a6qAkUjkbUBs2hBl5zZvD669D794wY0aY\nSi6Sa7p168bo0aNZs2YNAEOHDuW0005je/1LTSKUkwFVWgpHHglXXBG2XC8thSuvhFq1oq5MJBqn\nnHIKtWvXZsyYMaxatYopU6ZoeE8il1Oz+NauhbvvDjfd7rgjPPssnH02mEVdmUi0tttuO7p06cLQ\noUP56quvaNiwIe3atYu6LMlxORNQ06eHvZoWLw6h1Ls35OVFXZVI+ujWrRvt27fn448/5swzz2Sb\nbXJygEXSSNb/DfzuO7j88jCk9+OPMGECPPecwkmksqOOOorGjRuzaNEiDe9JWsjqDuqVV+DSS+Hz\nz6FXrzA7r27dqKsSSU9mxrJly6IuQ+QXWdlBffklnHEGdOwIO+8cZuc98ojCSUQkk2RVQLnD4MHh\nhtvRo+GOO8J9TW3aRF2ZiIhsqbgCysw6mNn7ZrbEzG6o4vXtzGx47PVZZtY00YVWZ+lSOO44uPBC\nOPDAsBLETTeFJYtERCTzVBtQZlYLeAI4CWgGnGlmzSoddhHwjbvvC/QG7kt0oZtTUQEPPAAHHRR2\ntu3bN9x4u//+qapARESSIZ4OqjWwxN0/cvd1wAtAp0rHdAKGxB6PBNqbJf/uorlzoXVruO46OOGE\nMIX8kktAs2NFRDJfPLP4GgPLN/l+BVD5qs4vx7h7hZmVA7sCXyWiyKqUlkJ+PtSuDc2awbffhvub\nRCR9tGwZJiiJbI2UTjM3s+5Ad4AmTZrU6Gc1bw7t2sG6dSGkREQku8Tzq/0zYK9Nvt8z9lxVx6ww\ns9rATsDqyj/I3QuAAoD8/HzfmoI3VVRU058gIiLpKp6rNbOB/cxsHzPbFugKjK10zFjgvNjj04Ei\nd69xAImISO6qtoOKXVPqCUwCagGD3L3UzG4Hit19LDAQeNbMlgBfE0JMRERkq8V19cbdJwATKj13\n8yaPfwa6JLY0ERHJZZqQLSIiaUkBJSIiaUkBJSIiaUkBJSIiaUkBJSIiaUkBJSIiacmiup/WzMqA\nTxLwoxqQxDX/0ojOM7voPLNLrpwnJOZc93b3vOoOiiygEsXMit09P+o6kk3nmV10ntklV84TUnuu\nGuITEZG0pIASEZG0lA0BVRB1ASmi88wuOs/skivnCSk814y/BiUiItkpGzooERHJQgooERFJSxkR\nUGbWwczeN7MlZnZDFa9vZ2bDY6/PMrOmqa8yMeI41/PNrMzM5se+Lo6izpows0FmtsrMFm7mdTOz\nx2L/G5SY2aGprjER4jjPdmZWvslneXNVx6U7M9vLzKaa2SIzKzWzXlUck/GfaZznmS2f6fZm9o6Z\nLYid620nsGMBAAADDklEQVRVHJP837vuntZfhE0SlwJ/ALYFFgDNKh1zGdAv9rgrMDzqupN4rucD\nfaKutYbneTRwKLBwM6+fDEwEDGgLzIq65iSdZztgXNR1JuA8GwGHxh7XAz6o4u9txn+mcZ5ntnym\nBtSNPa4DzALaVjom6b93M6GDag0scfeP3H0d8ALQqdIxnYAhsccjgfZmZimsMVHiOdeM5+7TCDsv\nb04n4BkPZgL1zaxRaqpLnDjOMyu4+0p3nxt7/D2wGGhc6bCM/0zjPM+sEPucfoh9Wyf2VXlGXdJ/\n72ZCQDUGlm/y/Qp++5fil2PcvQIoB3ZNSXWJFc+5ApwWGyYZaWZ7paa0lIr3f4dscHhsGGWimTWP\nupiaig3zHEL4F/emsuoz/Z3zhCz5TM2slpnNB1YBhe6+2c80Wb93MyGg5NdeAZq6+8FAIf/3LxjJ\nPHMJa5K1AB4HRkdcT42YWV3gJeBKd/8u6nqSpZrzzJrP1N03uHtLYE+gtZkdmOoaMiGgPgM27RL2\njD1X5TFmVhvYCVidkuoSq9pzdffV7r429u0AoFWKakuleD7zjOfu3/1nGMXdJwB1zKxBxGVtFTOr\nQ/ilPdTdR1VxSFZ8ptWdZzZ9pv/h7t8CU4EOlV5K+u/dTAio2cB+ZraPmW1LuBg3ttIxY4HzYo9P\nB4o8duUuw1R7rpXG7TsSxsGzzVjg3NjMr7ZAubuvjLqoRDOzhv8Zszez1oT/P2bcP6xi5zAQWOzu\nD2/msIz/TOM5zyz6TPPMrH7s8X8BxwPvVTos6b93ayfyhyWDu1eYWU9gEmGW2yB3LzWz24Fidx9L\n+EvzrJktIVyU7hpdxVsvznO9wsw6AhWEcz0/soK3kpkNI8x2amBmK4BbCBdhcfd+wATCrK8lwBrg\ngmgqrZk4zvN04FIzqwB+Arpm6D+sjgDOAd6NXbMAuBFoAln1mcZzntnymTYChphZLULIjnD3can+\nvauljkREJC1lwhCfiIjkIAWUiIikJQWUiIikJQWUiIikJQWUiIikJQWUiIikJQWUiIikpf8PyIoN\nVwZXXDEAAAAASUVORK5CYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plt.plot(*zip(np.zeros_like(u), u), 'b-')\n", "plt.text(1.8, 0.1, 'v', fontsize=14)\n", "plt.plot(*zip(np.zeros_like(v), v), 'b-')\n", "plt.text(2.8, 2.6, 'u', fontsize=14)\n", "plt.tight_layout()\n", "pass" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Angle between two vectors" ] }, { "cell_type": "code", "execution_count": 24, "metadata": { "collapsed": true }, "outputs": [], "source": [ "cos_angle = np.dot(u.T, v)/(np.linalg.norm(u)*np.linalg.norm(v))" ] }, { "cell_type": "code", "execution_count": 25, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[0.70710678]])" ] }, "execution_count": 25, "metadata": {}, "output_type": "execute_result" } ], "source": [ "cos_angle" ] }, { "cell_type": "code", "execution_count": 26, "metadata": { "collapsed": true }, "outputs": [], "source": [ "theta = 180/np.pi*np.arccos(cos_angle)" ] }, { "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[45.]])" ] }, "execution_count": 27, "metadata": {}, "output_type": "execute_result" } ], "source": [ "theta" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Outer product\n", "\n", "Note that the inner product is just matrix multiplication of a $1\\times n$ vector with an $n\\times 1$ vector. In fact, we may write:\n", "\n", "$$\\langle v,w \\rangle = v^Tw$$\n", "\n", "The *outer product* of two vectors is just the opposite. It is given by:\n", "\n", "$$v\\otimes w = vw^T$$\n", "\n", "Note that I am considering $v$ and $w$ as *column* vectors. The result of the inner product is a *scalar*. The result of the outer product is a *matrix*.\n", "\n", "For example, if $v$ and $w$ are both in $\\mathbb{R}^3$\n", "\n", "$$\n", "v \\otimes w = \\pmatrix{v_1\\\\v_2\\\\v_3} \\pmatrix{w_1 & w_2 & w_3} = \\pmatrix{\n", "v_1w_1 & v_1w_2 & v_1w_3\\\\\n", "v_2w_1 & v_2w_2 & v_2w_3 \\\\ \n", "v_3w_1 & v_3w_2 & v_3w_3}\n", "$$" ] }, { "cell_type": "code", "execution_count": 28, "metadata": { "collapsed": true }, "outputs": [], "source": [ "v = np.array([1,2,3]).reshape((-1,1))" ] }, { "cell_type": "code", "execution_count": 29, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[1],\n", " [2],\n", " [3]])" ] }, "execution_count": 29, "metadata": {}, "output_type": "execute_result" } ], "source": [ "v" ] }, { "cell_type": "code", "execution_count": 30, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[1, 2, 3],\n", " [2, 4, 6],\n", " [3, 6, 9]])" ] }, "execution_count": 30, "metadata": {}, "output_type": "execute_result" } ], "source": [ "v @ v.T" ] }, { "cell_type": "code", "execution_count": 31, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[1, 2, 3],\n", " [2, 4, 6],\n", " [3, 6, 9]])" ] }, "execution_count": 31, "metadata": {}, "output_type": "execute_result" } ], "source": [ "np.outer(v, v)" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.5" } }, "nbformat": 4, "nbformat_minor": 2 }