Phi X 174

What is it?

What are benefits of using PhiX control?

    1. Calibration Control: can be run alone and serves as a calibration control foe;
        1. Cluster generation: can be used as a positive control in the clustering process

Platform	Mode/Reagents	Optimal Raw Cluster Density
HiSeq	High Output, TruSeq v3	750-850 K/mm²
	High Output, HiSeq v4 (required upgrade)	950-1050 K/mm²
	Rapid v2	850-1,000 K/mm²
MiSeq	v2	1,000-1,200 K/mm²
	v3	1,200-1,400 K/mm²
MiniSeq	Mid and High Output	170-220 K/mm²
NextSeq	Mid and High Output, v2	170-220 K/mm²

[table 1] Cluster density guidelines for Illumina sequencing platforms

        2. Cross talk matrix generation

            1. During an illumina sequencing run, the cross-talk due to spectral overlap between the 4 fluorescently labeled nucleotides is calculated during template generation in cycle 1-5

            2. https://www.slideshare.net/idtdna/unique-dualmatched-adapters-mitigate-index-hopping-between-ngs-samples

        3. Phasing and Prephasing

            1. During sequencing by synthesis, each DNA strand in a cluster extends by 1 base per cycle

            2. A small proportion of strands may become out of phase with the current cycle, either falling a base behind(phasing) or jumping a base ahead(prephasing)

            3. For best results, use a PhiX spike-in as a control with any library that does not comprise a balanced base composition

            4. High GC samples(≧ 60%) typically show higher phasing rates, and in this case a PhiX control is required

    2. Run quality monitor: due to its small size and balanced nucleotide composition, it's an ideal in-run control (typically with >= 1% spike-in) for run quality monitoring

Platform	PhiX Aligned(%)
iSeq 100	minimum 5%
MiniSeq	10~50%
MiSeq (MCS 2.2 or higher)	minimum 5%
NextSeq	10~50%
HiSeq 2500 (HCS 2.2.38 or higher)	minimum 10%
HiSeq 3000/4000 (HCS 3.3.76 or lower)	10~50%
HiSeq 3000/4000 (HCS 3.4.0 or higher)	5~20%
NovaSeq	minimum 10%

[table 2] PhiX Control v3 library Illumina recommends spiking in when running low diversity libraries

    3. Color balancing
        1. For low diversity libraries, the PhiX Control v3 library provides balanced fluorescent signals at each cycle to improve the overall run quality
        2. You can find why the nucleotide diversity is important in here

How to remove PhiX reads from the fastq

What is nucleotide diversity and why is it important?

Why is nucleotide diversity important?

• Course: Data Analytics for Forecasting and Classification
• Professor: Chi-hyuk Jeon / POSTECH
• Goals
• Understanding data analysis methods for forecasting and classification based on statistics
• Cultivating data analysis skill and application ability by using data analytics methods
• Prerequisite
• Probability and Statistics, Linear Algebra, Optimization
• Schedule
• 1 week
• 1-1. Regression analysis, Simple regression model, Model estimation

• Regression Analysis
• In order to explain a variable, to analyze statistical causal relationships between related variables
• independent variable: causes
• dependent variable: outcomes
• Regression Model
• Simple Regression Model
• 𝑿 ⇨ 𝒀
• Observation: (𝑿₁,𝒀₁), (𝑿₂,𝒀₂), ... , (𝑿𝘯,𝒀𝘯) (𝑛 is observation number)
• Simple Regression Model: 
• 𝒀𝑖 = 𝜷₀ + 𝜷₁𝑿𝑖 + ℇ𝑖,    𝑖 = 1,2, ... , 𝑛
• ℇ𝑖: error term. 
• Assume that it follows a normal distribution with mean 0 and variance 𝛔²
• ℇ𝑖~𝙉𝙤𝙧(0,𝛔²)
• 𝑿 is not random variable, but a given value
• so, three parameters need to be estimated
• 𝜷₁: slope of the linear equation
• 𝜷₀: intercept
• 𝛔²: variance of the error term
• Estimation of intercept 𝜷₀ and slope 𝜷₁
• Using least squares method
• to minimize the objective function 𝐐
• objective function 𝐐
• sum of the square of the difference between the observed value of dependent variable 𝒀, and the fitted value provided by the model on the linear line 𝜷₀ + 𝜷₁𝑿𝑖
• 𝐐 = ∑(𝒀𝑖 - 𝜷₀ - 𝜷₁𝑿𝑖)²
• How to?
• (𝑿,𝒀) is observed value, so let 𝐐 be a function of 𝜷₀ and 𝜷₁ 
• and partially differentiate 𝐐 with respect to 𝜷₀
  = -2∑(𝒀𝑖 - 𝜷₀ - 𝜷₁𝑿𝑖) = 0
• and partially differentiate 𝐐 with respect to 𝜷₁
  = -2∑(𝒀𝑖 - 𝜷₀ - 𝜷₁𝑿𝑖)𝑿𝑖 = 0
• estimated equation: 𝒀-hat = 𝜷₀-hat + 𝜷₁-hat * 𝑿
• Estimation of variance of the error term 𝛔²
• Using sample variance of the residuals
• residual
  substract the estimated value from the observed value of 𝒀
  𝒆𝑖 = 𝒀𝑖 - 𝒀-hat = 𝒀𝑖 - 𝜷₀-hat + 𝜷₁-hat * 𝑿𝑖
• SSE
  resudual/error sum of squares
  = ∑(𝒀𝑖 - 𝒀𝑖-hat)²
• estimate 𝛔² by using MSE
  𝛔²-hat = MSE(Mean Squared Error) = SSE / 𝑛-2
  (𝑛-2) is  degree of freedom

Two reasons for that

1. To reduce gap between sample variance and population variance
   ( empirical reason )
1. "1/n" version is the maximum likelihood estimate of the population variance, however, it is also mathematically biased
2. sample variance is usually smaller than the population variance
   → estimation of the population variance is getting bigger than real
3. to reduce gap using "1/n-1" convention ( provides an unbiased estimate )
4. why not n-2 ?
1. related to degree of freedom, that is n-1
2. To match both expectation of sample variances and population variance
   ( mathematical reason )
1. let,
   : sample size
   : sample mean
   : sample variance
   : population mean
   : population variance
2. then, figure out following is true
   
   
3. first,
   
   
   
   
   
   
   
   
4. as here,
   
   
   
   
   

xargs

1. to print all vcf files in specific path
   echo `pwd` | cut -d'/' -f1-7 | xargs -I{} find {} -name '*.vcf'
2. to know the number of lines/words/characters in specific file
   ls * | xargs -IFILE grep -vc '^#' FILE
3. to count the number of all variants of vcf
   ls * | xargs -IFILE echo FILE | xargs grep -vc '^#' FILE