Introduction to Fundamentals of Signal Processing

To understand what is Digital Signal Processing (DSP) let’s examine what does each of its words mean. “Signal” is any physical quantity that carries information. “Processing” is a series of steps or operations to achieve a particular end. It is easy to see that Signal Processing is used everywhere to extract information from signals or to convert information-carrying signals from one form to another. For example, our brain and ears take input speech signals, and then process and convert them into meaningful words. Finally, the word “Digital” in Digital Signal Processing means that the process is done by computers, microprocessors, or logic circuits.

The field DSP has expanded significantly over that last few decades as a result of rapid developments in computer technology and integrated-circuit fabrication. Consequently, DSP has played an increasingly important role in a wide range of disciplines in science and technology. Research and development in DSP are driving advancements in many high-tech areas including telecommunications, multimedia, medical and scientific imaging, and human-computer interaction.

To illustrate the digital revolution and the impact of DSP, consider the development of digital cameras. Traditional film cameras mainly rely on physical properties of the optical lens, where higher quality requires bigger and larger system, to obtain good images. When digital cameras were first introduced, their quality were inferior compared to film cameras. But as microprocessors become more powerful, more sophisticated DSP algorithms have been developed for digital cameras to correct optical defects and improve the final image quality. Thanks to these developments, the quality of consumer-grade digital cameras has now surpassed the equivalence in film cameras. As further developments for digital cameras attached to cell phones (cameraphones), where due to small size requirements of the lenses, these cameras rely on DSP power to provide good images. Essentially, digital camera technology uses computational power to overcome physical limitations. We can find the similar trend happens in many other applications of DSP such as digital communications, digital imaging, digital television, and so on.

In summary, DSP has foundations on Mathematics, Physics, and Computer Science, and can provide the key enabling technology in numerous applications.

The two main characters in DSP are signals and systems. A signal is defined as any physical quantity that varies with one or more independent variables such as time (one-dimensional signal), or space (2-D or 3-D signal). Signals exist in several types. In the real-world, most of signals are continuous-time or analog signals that have values continuously at every value of time. To be processed by a computer, a continuous-time signal has to be first sampled in time into a discrete-time signal so that its values at a discrete set of time instants can be stored in computer memory locations. Furthermore, in order to be processed by logic circuits, these signal values have to be quantized in to a set of discrete values, and the final result is called a digital signal. When the quantization effect is ignored, the terms discrete-time signal and digital signal can be used interchangeability.

In signal processing, a system is defined as a process whose input and output are signals. An important class of systems is the class of linear time-invariant (or shift-invariant) systems. These systems have a remarkable property is that each of them can be completely characterized by an impulse response function (sometimes is also called as point spread function), and the system is defined by a convolution (also referred to as a filtering) operation. Thus, a linear time-invariant system is equivalent to a (linear) filter. Linear time-invariant systems are classified into two types, those that have finite-duration impulse response (FIR) and those that have an infinite-duration impulse response (IIR).

A signal can be viewed as a vector in a vector space. Thus, linear algebra provides a powerful framework to study signals and linear systems. In particular, given a vector space, each signal can be represented (or expanded) as a linear combination of elementary signals. The most important signal expansions are provided by the Fourier transforms. The Fourier transforms, as with general transforms, are often used effectively to transform a problem from one domain to another domain where it is much easier to solve or analyze. The two domains of a Fourier transform have physical meaning and are called the time domain and the frequency domain.

Sampling, or the conversion of continuous-domain real-life signals to discrete numbers that can be processed by computers, is the essential bridge between the analog and the digital worlds. It is important to understand the connections between signals and systems in the real world and inside a computer. These connections are convenient to analyze in the frequency domain. Moreover, many signals and systems are specified by their frequency characteristics.

Because any linear time-invariant system can be characterized as a filter, the design of such systems boils down to the design the associated filters. Typically, in the filter design process, we determine the coefficients of an FIR or IIR filter that closely approximates the desired frequency response specifications. Together with Fourier transforms, the z-transform provides an effective tool to analyze and design digital filters.

In many applications, signals are conveniently described via statistical models as random signals. It is remarkable that optimum linear filters (in the sense of minimum mean-square error), so called Wiener filters, can be determined using only second-order statistics (autocorrelation and crosscorrelation functions) of a stationary process. When these statistics cannot be specified beforehand or change over time, we can employ adaptive filters, where the filter coefficients are adapted to the signal statistics. The most popular algorithm to adaptively adjust the filter coefficients is the least-mean square (LMS) algorithm.