BIOS823: Biomedical Big Data

Notebooks:

• S01 Relational Databases
  • 0. Packages for working with relational databases in Python
  • 1. Motivation
  • 2. RDBMS
  • 3. Anatomy
  • 4. Concepts
  • 5. Design
  • 6. Example
  • Homework walk-through
• S02 RDBMS and SQL
  • A. More RDBMS concepts
  • B. Basic SQL queries
  • C. Window Functions
• MongoDB
  • Set up
  • Get Data
  • Insertion
  • Queries
  • Indexes
  • Aggregate Queries
  • Geospatial queries
• Redis
  • Connect to database
  • Clear database
  • Simple data types
  • Complex data types
• Neo4j
  • Graph concepts
  • Cypher
  • Queries
  • Delete all notes and relationships
• Setting up NoSQL servers on Mac
  • Installation
  • Start database services
• Scalars
  • Integers
  • Real numbers
• Vectors
  • Vector space
  • Column vectors
  • Norms and distances
  • Vector operations
  • Dot product
  • Outer product
• Matrices
  • Matrices
  • Matrix representation in Python
  • Properties of a matrix
  • Solving linear equations (Matrix-vector multiplication)
• Sparse Matrices
  • Creating a sparse matrix
  • Compressed Sparse Row and Column formats
  • COO summation convention
  • Solving large sparse linear systems
• Working with Matrices
  • Matrices and liner combinations
  • Permutation matrices
  • Matrix partitioning
• Solving Linear Equations
  • Linear Equations
  • Gaussian elimination
  • Gauss-Jordan elimination
  • Gaussian-Jordan elimination and the number of solutions
  • LU Decomposition
  • Cholesky Decomposition
• Linear least squares
  • Normal equations
  • Projection matrices
• Solving Linear Systems
  • Solve \(Ax = b\) where \(A\) is full rank square matrix
  • Solving least squares equations
• Dimension Reduction
  • PCA from scratch
  • Geometry of PCA
  • Algebra of PCA
• Nonlinear Dimension Reduction
  • Limitations of PCA
  • Multi-dimensional scaling (MDS)
  • t-distributed Stochastic Neighbor Embedding (t-SNE)
• Anomaly Detection
  • Information about data
  • Precision-recall and ROC curves
  • Case study
• Clustering
  • Toy example to illustrate concepts
  • How to cluster
  • Model selection
  • Comparing across samples
• Understanding the SVD
  • Useful reference
  • Singular value decomposition
  • (1) The matrix A
  • (2) Orthogonal matrices \(U\) and \(V^T\)
  • (3) Diagonal matrix \(S\)
  • Geometry of SVD example
  • Covariance, PCA and SVD
  • PCA
  • Using SVD for PCA
• Applications of SVD
  • Reconstruction
  • PCA
  • Data compression (Low rank approximations)
  • Denoising
  • Recommender system
• Finding Roots of Equations
  • Calculus review
  • Main Issues in Root Finding in One Dimension
  • Bisection Method
  • Secant Method
  • Newton-Raphson Method
  • Gauss-Newton
  • Inverse Quadratic Interpolation
  • Brentq Method
  • Roots of polynomials
  • Using scipy.optimize
• Numerical Optimization
  • Categorize your optimization problem
  • Sketch of lecture
• Algorithms for Optimization and Root Finding for Multivariate Problems
  • Convexity
  • Line Search Methods
  • Steepest Descent
  • Newton’s Method
  • Coordinate Descent
  • Solvers
  • GLM Estimation and IRLS
• Using optimization routines from scipy and statsmodels
  • Using scipy.optimize
  • Some applications of optimization
  • Optimization of standard statistical models
• Solving Optimization Problems Computationally
  • 1. Curve fitting
  • 2. Constrained optimization
  • 3. Gradient descent
• Line search in gradient and Newton directions
  • Demo functions
  • Gradient descent with step size found by numerical minimization
  • Gradient descent with analytic step size for quadratic function
  • Line search in Newton direction with analytic step size
• Least squares optimization
  • Working with matrices
  • Example
• Gradient Descent Optimizations
  • Smoothing with exponentially weighted averages
  • Momentum in 1D
  • Momentum and RMSprop in 2D
  • Implementing a custom optimization routine for scipy.optimize
• Constrained Optimization
  • Lagrange multipliers and constrained optimization
  • Using scipy.optimize
  • Solve unconstrained problem
  • Solve constrained problem using scipy.optimize
  • Solve constrained problem using Lagrange multipliers
• Introduction to C
  • A tutorial example - coding a Fibonacci function in C
  • C Basics
• Introduction to C++
  • Hello world
  • Type conversions
  • Command line inputs
  • Functions
  • Header, implementation and driver files
  • Using make
  • A more flexible Makefile
  • Input and output
  • Arrays
  • Loops
  • Function arguments
  • Anonymous functions
  • Function pointers
  • Generic programming with templates
  • Standard template library (STL)
  • STL algorithms
  • Random numbers
  • Numerics
• Native code compilation
  • Timing code
  • Profiling
  • Optimizing a function
  • Python
  • JIT with numba
  • Cython
  • Cython with static types
  • Wrapping C++ cdoe
• Just-in-time compilation (JIT)
  • Using numexpr
  • Using numba
  • Using numba vectorize and guvectoize
  • Parallelization with vectorize and guvectorize
• Cython
  • Resources
  • Using Cython annnotations to identify bottlenecks
  • Using Cython cdefs and directives
  • Parallel execution with Cython
• Using pybind11
  • Resources
  • A first example of using pybind11
  • Using cppimport
  • Vectorizing functions for use with numpy arrays
  • Using numpy arrays as function arguments and return values
  • More on working with numpy arrays
  • Using the C++ eigen library to calculate matrix inverse and determinant
  • Using pybind11 with openmp
• Modules and Packaging
  • References
  • Install packages we will use for packaging
  • Modules
  • Distributing your package
• Parallel Programming
  • Embarrassingly parallel programs
  • Degree of parallelism
  • Task parallelism
• Multi-Core Parallelism
  • Vanilla Python
  • Using numba to speed up computation
  • Using cython to speed up computation
  • The concurrent.futures module
  • Using processes in parallel with ProcessPoolExecutor
  • Using processes in parallel with ThreadPoolExecutor
  • Turning off the GIL in cython
  • Using processes in parallel with ThreadPoolExecutor and nogil
  • Using multiprocessing
  • Common issues with use of shared memory in parallel programs
• Parallel Programming Example
  • Python version
  • JIT with numba
  • JIT with numba and prange using parallel option
  • Parallel version with cython
• Using ipyparallel
  • Starting engines
  • Basic concepts of ipyparallel
  • Working with compiled code
  • Using parallel magic commands
• Enhancing performance
  • Python
  • Using numba
  • Using cython
  • Using pybind11
  • Using multiple cores
• Text Analysis
  • Exploratory data analysis
  • Supervised Learning
  • Maintaining context
  • Part-of-speech tagging
• Text Analysis
  • Similarity
  • Word2Vec
  • Doc2Vec
  • Latent Semantic Indexing (LSI)
  • Non-negative Matrix Factorization
  • Latent Dirichlet Allocation
• Time Series Analysis 1
  • Dates and times
  • Lag and lead with shift
  • Resampling and window functions
  • Correlation between time series
  • Visualizing space and time data
• Time Series Analysis II
  • Stationarity
  • Example
  • Decomposing a model
  • Classical models for time series
  • Moving Average models MA(q)
  • ARMA(p, q)
  • Bayesian modeling with prophet
  • Model evaluation

Homework:

• Assignment 1 Review
• Assignment 2: Data formats
• Assignment 3: Working with Relational Databases and SQL
• Assignment 4: Linear Algebra Review
• Assignment 5: Unsupervised Learning
• Assignment 6: Topic Modeling with Latent Semantic Analysis
• Assignment 7: Conjugate Gradient Descent
• Assignment 8: Supervised Learning
• Assignment 9: Improving performance
• Assignment 10: Review

Indices and tables

• Index
• Module Index
• Search Page