Skip to content

Commit

Permalink
update writing styles, fix grammar or typos, and clarify informations.
Browse files Browse the repository at this point in the history
  • Loading branch information
glennhenry committed Aug 8, 2024
1 parent 4030996 commit 8edf4ff
Show file tree
Hide file tree
Showing 3 changed files with 57 additions and 55 deletions.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
83 changes: 39 additions & 44 deletions docs/digital-signal-processing/11-sampling/sampling.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,81 +5,76 @@ title: Sampling
description: Sampling
---

**Main Source :**
**Main Source:**

- **[0612 TV w/ NERDfirst Sampling, Aliasing & Nyquist Theorem](https://youtu.be/yWqrx08UeUs)**
- **Other sources from Google and Youtube**
- **[Sampling, Aliasing & Nyquist Theorem — 0612 TV w/ NERDfirst](https://youtu.be/yWqrx08UeUs)**
- **Other sources from Google and YouTube**

**Sampling** is a technique of taking some representative data from a group of data. In digital signal processing, sampling refers to the process of converting a continuous-time signal into a discrete-time signal.
**Sampling** is the technique of taking some number of representative data from a group of data. In digital signal processing, sampling is used to convert a continuous-time signal into a discrete-time signal.

:::tip
Find out more about sampling specific related to image in [here](/computer-graphics/sampling)
:::
A continuous signal can contain an infinite amount of information within the time it's defined. In the digital world, processing an infinite amount of information is impractical. Digital signal processing overcomes this by taking a finite number of data points from the original signal to create its finite version. This process is what we call sampling, and the number of samples taken affects the accuracy of the reconstructed signal.

A continuous signal can take on an infinite number of information within the time it's defined. Processing infinite amount of information wouldn't make sense, so instead we will only take some data which should "represent" the original signal.
![Continuous signal in time sampled in specific time](./sampling.png)
Source: https://en.wikipedia.org/wiki/Sampling_%28signal_processing%29

![Continous signal in time sampled in specific time](./sampling.png)
Source : https://en.wikipedia.org/wiki/Sampling_%28signal_processing%29
Given some continuous signal, we choose the number of sample and the method to sample it. For example, we may take one data point on each second of the signal. Using this, we will know the amplitude or the value of the continuous signal every second.

These sample can now be represented in number easily. To be able to reconstruct that, we will need to connect between each samples.
### Sampling Rate

### Sampling Problem
**Sampling rate**, also known as **sampling frequency**, is the number of samples taken per unit of time (usually second (s)) during the sampling process. It is typically measured in samples per second and is denoted in hertz (Hz).

Sampling signal could introduce several challenges :
If we are taking just one data point on each second of the signal, this mean our sampling rate is 1 Hz. However, a 1 Hz sample rate is extremely low, and would result in a very poor reconstruction of the original continuous signal. We would not be able to determine what happens during the milliseconds or microseconds within that 1-second period.

- **Noise** : Noise is an unwanted signal that is added to the desired signal during the sampling process. This noise can come from a variety of sources, including the sampling device, the environment, or the signal itself.
For example, while you record your sound, you may indirectly recorded traffic, people talking, and appliances sounds.
![Noise introduce weird pattern to the original signal](./noise.png)
Source : https://www.predig.com/whitepaper/reducing-signal-noise-practice
A high sampling rate means more samples are taken per unit of time, resulting in a more detailed and accurate representation of the original signal, at the cost of more data set that needs to be stored, transmitted, and processed. For example, in audio applications, 44.1 kHz is a common sampling rate. This means that every second, we will take 44,100 data point.

- **Aliasing** : After sampling to a discrete signal, we may want to reconstruct it back to continous signal. However, this could introduce inaccuracy. **Aliasing** occurs when the sampling rate is insufficient making loss of detail and accuracy in the reconstructed signal. In other words, we don't have enough information to reconstruct it back.
![A very inaccurate reconstructed signal because of limited samples](./aliasing.png)
Source : https://www.physik.uzh.ch/local/teaching/SPI301/LV-2015-Help/lvanlsconcepts.chm/Aliasing.html
![Sampling rate comparison](./sample-rate.png)
Source: https://routenote.com/blog/what-is-sample-rate-in-audio/

- **[Quantization](/digital-signal-processing/quantization)** : Quantization is a way of simplifying a set of values by putting them into groups. It works by dividing the range of possible values into smaller intervals and assigning each value to the closest interval. This process helps to reduce the complexity of the data, but it also introduces some inaccuracy because the original values are approximated.
#### Sampling Problems

For example in the image below, we divided this continous signal into 8 groups of binary bits.
![Quantization, rounds up or down a continous signal to a square wave](./quantization.png)
Source : https://www.differencebetween.com/difference-between-uniform-and-nonuniform-quantization/
Sampling signals could introduce several challenges:

### Nyquist Theorem
- **Noise**: Noise is an unwanted signal that gets added to the desired signal during the sampling process. This noise can come from various sources, including the sampling device, the environment, or the signal itself. For example, a microphone record the analog sound wave and converts it into a discrete digital signal. While recording, we may unintentionally capture sounds like traffic, people talking, or household appliances.

#### Sampling Rate
![Noise introduce weird pattern to the original signal](./noise.png)
Source: https://www.predig.com/whitepaper/reducing-signal-noise-practice

**Sampling Rate** also referred to as the sampling frequency is the number of samples taken per unit of time during the process of analog-to-digital conversion. It is typically measured in samples per second and is denoted in hertz (Hz). A higher sampling rate means more samples are taken per unit of time, resulting in a more detailed and accurate representation of the original analog signal.
- **Aliasing**: Aliasing is a phenomenon where we fail to reconstruct the original signal accurately. This occurs when the sampling rate is insufficient, making loss of detail in the reconstructed signal.

For example in audio applications, the commonly used sampling rates are 44.1 kHz, this means every second, we will take 44.100 representation of the sound wave.
![A very inaccurate reconstructed signal because of limited samples](./aliasing.png)
Source: https://www.physik.uzh.ch/local/teaching/SPI301/LV-2015-Help/lvanlsconcepts.chm/Aliasing.html

### Nyquist Theorem

#### Nyquist-Shannon Sampling Theorem

Also known as **Nyquist Rate**, is a fundamental principle in digital signal processing to accurately reconstruct a continuous analog signal from its discrete samples. The theorem states that sampling rate should be greater than or equal to twice the bandwidth of the signal.
Also known as **Nyquist rate**, it's a fundamental principle in digital signal processing to accurately reconstruct a continuous analog signal from its discrete samples. The theorem states that sampling rate should be greater than or equal to twice the bandwidth of the signal.

Mathematically written as : $\text{Sampling rate} \ge 2B$, where $B$ is the highest frequency in the signal.
Nyquist rate is used to avoid aliasing and accurately represent signal.
Mathematically, this is written as: $\text{Sampling rate} \ge 2B$, where $B$ is the highest frequency in the signal.

![Shows how sampling rate lower than 2 leads to inaccuracy and the sampling rate above to are adequate enough to reconstruct original signal](./nyquist-theorem.gif)
Source : http://195.134.76.37/applets/AppletNyquist/Appl_Nyquist2.html
![Shows how sampling rate lower than 2 leads to inaccuracy and the sampling rate above to are adequate to reconstruct original signal](./nyquist-theorem.gif)
Source: http://195.134.76.37/applets/AppletNyquist/Appl_Nyquist2.html

#### Nyquist Frequency

Nyquist frequency, also known as folding ferquency, represents the maximum frequency that can be accurately represented in a discrete digital signal without introducing aliasing.
It is written as : $f_{\text{Nyquist}} = \frac{\text{Sampling rate}}{2}$
Also known as **folding frequency**, represents the maximum frequency that can be accurately represented in a discrete digital signal without introducing aliasing.

We can think Nyquist rate as the minimum and Nyquist frequency as the maximum sampling rate to avoid aliasing and to make sure we don't take too much sample.
It is written as: $f_{\text{Nyquist}} = \frac{\text{Sampling rate}}{2}$

For example, Audio CDs: Audio CDs have a sampling rate of 44.1 kHz. This means that the Nyquist frequency is 22.05 kHz. This is enough to represent the human hearing range, which is up to 20 kHz.
Nyquist rate and Nyquist frequency are related. Nyquist rate specifies the required sampling rate, while the Nyquist frequency represents the highest frequency that can be accurately reproduced from the sampled data, by sampling as much as the Nyquist rate. They both provide principle to avoid aliasing, while also avoiding oversampling.

### Upsampling & Downsampling
For example, Audio CDs have a sampling rate of 44.1 kHz. This means that the Nyquist frequency, or the highest frequency that can be perfectly reconstructed is 22.05 kHz. In other word, any frequency higher than that may be aliased. However, this is enough to represent the human hearing range, which is up to 20 kHz.

Upsampling and downsampling are signal processing operations that involve changing the sampling rate of a discrete signal.
### Upsampling & Downsampling

Upsampling increases the sampling rate of a signal by inserting additional samples between existing samples. A common method is to take an average of neighbour or technique like linear interpolation.
**Upsampling** and **downsampling** are signal processing operations that involve changing the sampling rate of a discrete signal.

Downsampling reduces the sampling rate of a signal by removing samples from the original signal. It applies a low-pass filter, known as an anti-aliasing filter, to remove high-frequency components above the new Nyquist frequency.
- Upsampling increases the sampling rate of a signal by inserting additional samples between existing samples. A common method to do this is by taking an average of two data sample, to get the third sample as an addition.
- Downsampling reduces the sampling rate of a signal by removing samples from the original signal. It can be done by applying a **low-pass filter**, also known as an **anti-aliasing filter**, which can remove high-frequency components above the new Nyquist frequency.

![A discrete signal is upsampled in the middle of each samples](./updown-sample.png)
Source : https://www.divilabs.com/2014/07/upsampling-interpolation-of-discrete.html
![A discrete signal is up sampled in the middle of each sample's](./updown-sample.png)
Source: https://www.divilabs.com/2014/07/upsampling-interpolation-of-discrete.html

:::tip
Find out more about anti-aliasing for image in [here](/computer-graphics/sampling)
See also [sampling in digital image](/computer-graphics/sampling).
:::
29 changes: 18 additions & 11 deletions docs/digital-signal-processing/12-quantization/quantization.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,26 +5,33 @@ title: Quantization
description: Quantization
---

**Main Source : [Quantization - (Akash Murthy - Digital Audio Fundamentals)](https://youtu.be/1KBLguIXL30)**
**Main Source:**

**Quantization** is a way of simplifying a set of values by putting them into groups. It works by dividing the range of possible values into smaller intervals and assigning each value to the closest interval. This process helps to reduce the complexity of the data, but it also introduces some inaccuracy because the original values are approximated.
- **[5. Quantization - Digital Audio Fundamentals — Akash Murthy](https://youtu.be/1KBLguIXL30)**

For example in the image below, we divided this continous signal into 8 groups of binary bits representation.
**Quantization** is the process of simplifying a set of values by grouping them into intervals. It works by dividing the range of possible values into smaller intervals and assigning each value to the nearest interval. This process helps to reduce the complexity of the data, thus reducing the amount of data that needs to be stored.

![Quantization, rounds up or down a continous signal to a square wave](./quantization.png)
Source : https://www.differencebetween.com/difference-between-uniform-and-nonuniform-quantization/
In the image below, the signal is sampled and limited to just 8 data values.

The power of two of the bits used is called **bit depth**. Bit depth determines the resolution or precision of the quantized representation.
![Quantization, rounds up or down a continuous signal to a square wave](./quantization.png)
Source: https://www.differencebetween.com/difference-between-uniform-and-nonuniform-quantization/

There is also **quantization level**, which represents the number of possible output or representation and mathematically written as : Number of quantization levels = $2^{(\text{bit depth})}$
However, quantization may introduce some inaccuracy during reconstruction. We can see that some blue lines, representing the sampled points, deviate from the original red signal. Some data samples have to be rounded up or down to the nearest defined data point, resulting in inaccuracy.

The difference between the actual analog input value and the nearest quantization level is called **Quantization Error** or **Quantization Noise**.
### Quantization Level

Typically, the number of possible values is expressed as a power of 2, as this simplifies the digital representation and processing (allows for easy representation using [binary](/computer-and-programming-fundamentals/binary-representation)). The number of bit is called the **bit depth**, which determines the resolution or precision of the quantized representation.

The number of possible output or representation is mathematically written as: $\text{quantization levels} = 2^{(\text{bit depth})}$.

The difference between the actual analog signal value with the nearest quantization level is called **quantization error** or **quantization noise**.

### Uniform & Non-uniform Quantization

**Uniform quantization** is a quantization technique where the intervals between the quantization levels are evenly spaced. In other words, the range of all possible values is divided into equal intervals.
These two are techniques to quantize signals.

**Non-uniform quantization** is a quantization technique where the intervals between quantization levels are not evenly spaced. Instead, the intervals are non-uniformly spaced to allocate more quantization levels in regions of higher signal importance or sensitivity, and fewer quantization levels in regions of lower importance.
- **Uniform quantization**: Intervals between the quantization levels are evenly spaced. In other words, the range of all possible values is divided into equal intervals.
- **Non-uniform quantization**: Intervals between quantization levels are not evenly spaced. Instead, the intervals are non-uniformly spaced to allocate more quantization levels in regions of higher signal importance or sensitivity, and fewer quantization levels in regions of lower importance.

![Uniform and non-uniform quantization comparison](./uniform-nonuniform-quantization.png)
Source : https://analogquantized.wordpress.com/tag/non-uniform-quantization/
Source: https://analogquantized.wordpress.com/tag/non-uniform-quantization/

0 comments on commit 8edf4ff

Please sign in to comment.