BIOS821: Data Science Tools

Contents:

• Introduction to bash
• Text and arithmetic in bash
• Shell Scripts
• Session contents
  • Text processing with sed
  • Construction of shell script
  • Variables
  • Arrays
  • Conditionals
  • Branching
  • Looping
  • Process control
• Introduction to Python (Part 1)
  • IPython kernel is polyglot
  • Operators
  • Strings
  • Lists
  • Mutability and immutability
  • Tuples
  • Dictionaries
  • Sets
  • Multiset, bag, counter
  • Packages and the import statement
  • Built-in functions
  • String indexing
  • String methods
  • Numpy arrays
  • Custom functions
• Introduction to Python (Part 2)
  • Getting help
  • Magics
  • Algorithmic complexity
  • More on data collections
  • Flow control
  • Iterable objects and iterators
  • Comprehensions
• Functions
  • Arguments
  • Map, filter and reduce
  • Decorators
  • Useful modules for functional programming
  • Toolz
• Streaming and functional programming
  • Understanding itertools
• Streaming and functional programming
  • Understanding toolz
  • Exercises
• Object Oriented Python
  • Guidelines
  • Object-oriented ideas
  • Class creation basics
  • Classes are factories for instances
  • Class attributes and methods
  • Instance attributes and methods
  • Special methods
  • Generic special methods
  • Special methods for containers
  • Example
  • Special methods for numbers
  • Properties
  • Inheritance
  • Chaining methods
  • Composition
  • Abstract base classes
• Using numpy
  • Basic array properties
  • Creation of arrays
  • Indexing
  • Views and copies
  • Strides
  • Vectorization and ufuncs
  • Reshaping
  • Broadcasting
  • Reductions and the axis argument
  • Splitting arrays
  • Combining arrays
• Using numpy
  • Basic array properties
  • Creation of arrays
  • Indexing
  • Strides
  • Vectorization and ufuncs
  • Reshaping
  • Broadcasting
• Using numpy
  • Reductions and the axis argument
  • Splitting arrays
  • Combining arrays
• Basic plots with matplotlib
  • Displaying matrices
  • To set display on by default
  • Images as arrays
  • Line plots
  • Scatter plots
  • Histograms
  • Multiple plots
  • Styles
• Using numpy.random
  • Random number generators
  • Discrete distributions
  • Multivariate distributions
  • Seed
  • Choice
  • Permutations
  • Exercises
• Using scipy statistics
  • Random number generators
  • Example
  • Using simulation
  • Estimating parameters
• Elementary analysis of numerical data
  • Linear algebra
  • Integration
  • Optimization
  • Interpolation
  • Exercises
• Pandas Part 1
  • Data structures
  • I/O
  • Basic data manipulation
• Pandas Part 2
  • Data cleaning
  • Special operations on strings and factors
  • Data Wrangling
• Statistical Graphics with seaborn
  • Data set
  • Look and feel
  • Types of plots
• Python is not R
• Data Archival and Management (Part 1)
  • Relational databases via sqlite3 package
  • Creating database tables
• Data Archival and Management (Part 2)
  • Using JSON
• Data Archival and Management (Part 3)
  • Using HDF5
• Data Archival and Management (Part 4)
  • Using pickle
  • Using Feather
• Using git with GitHub
  • Git cheat sheet
  • Getting help
  • Tutorial of using git with GitHub

Quizzes

Contents:

• Data Science entry test (100 points total)
• Unix Basics Quiz
  • Using the command line
  • Working with directories
  • Working with files
  • Compression, archival and checksums
  • Clean up
• Unix Text and Arithmetic Quiz
  • Text
  • Regular expressions
  • Finding stuff
  • Modifying text
  • Arithmetic
  • Clean up
• Unix Shell Scripts
• Simple Python Exercises
• Exercises with pandas 1
• Exercises with pandas 2

Homework

Contents:

• Working with the Unix Shell 1
• Working with the Unix Shell 2
• Homework: Caesar Cipher
• Homework: Finding ORFs
• Homework: Counting k-mers
• Homework: Wolfram Cellular Automata
• Homework: Numerical Algorithms
• Homework: Data Munging with pandas

Midterm exam

Contents:

• Mid-term exams
  • Start of exam
  • End of exam

Indices and tables

• Index
• Module Index
• Search Page