Video Signal Processing

Video compression standards are implemented in wireless data transmission technologies to provide multimedia services efficiently. These compression standards generally utilize the Discrete Cosine Transform (DCT) in conjunction with variable length codes (VLC) in order to achieve the required high compression ratios. While providing the necessary high data rates, this technique has the disadvantage of making the system more susceptible to transmission errors. The standard decoders do not manage to detect a large number of corrupted macroblocks, 40.54% not detected for H.263+, contributing to a significant reduction in the end-to-end video quality as perceived by the enduser. This paper presents three dissimilarity metrics which contain both color and texture information and that can be extracted directly from the compressed DCT coefficients. These metrics can be used to enhance the error-detection capabilities of standard DCT based codecs. Simulation results show that the proposed algorithm increases the error detection rate by 54.06% with a gain in peak signal-to-noise ratio (PSNR) of 3.21 dB. This improvement in performance is superior to other solutions found in literature.

The video compression standards commonly adopted in wireless multimedia services utilize variable length codes (VLC) in order to attain high compression ratios. While providing the high data rates required, this technique makes the system more susceptible to transmission errors. Thus the end-to-end quality of the video stream transmitted over an error-prone channel depends on the detection, and concealment of the corrupted macroblocks. The error detection capability of standard decoders is quite limited, for example, in the case of the H.263+ codec around 40.54% of the corrupted macroblocks are undetected, placing a bound on the perceived quality of the reconstructed video sequence. This paper presents a novel solution using eight pixel domain dissimilarity metrics computed in the CIE LUV color space which can be used at decode time to improve the error detection rate of the standard decoder. The spatial dissimilarity metric has been found to perform the best with an average increase in error detection rate of 60.38% when compared to the standard decoder (about 20% more than other published results) with 0% of false detection and a gain in peak signal-to-noise ratio (PSNR) of 3.94 dB.

This paper describes our recently developed system which captures pen strokes on whiteboards in real time using an off-the-shelf video camera. Unlike many existing tools, our system does not instrument the pens or the whiteboard. It analyzes the sequence of captured video images in real time, classifies the pixels into whiteboard background, pen strokes and foreground objects (e.g., people in front of the whiteboard), and extracts newly written pen strokes. This allows us to transmit whiteboard contents using very low bandwidth to remote meeting participants. Combined with other teleconferencing tools such as voice conference and application sharing, our system becomes a powerful tool to share ideas during online meetings.

This paper describes our recently developed system which captures pen strokes on whiteboards in real time using an off-the-shelf video camera. Unlike many existing tools, our system does not instrument the pens or the whiteboard. It analyzes the sequence of captured video images in real time, classifies the pixels into whiteboard background, pen strokes and foreground objects (e.g., people in front of the whiteboard), and extracts newly written pen strokes. This allows us to transmit whiteboard contents using very low bandwidth to remote meeting participants. Combined with other teleconferencing tools such as voice conference and application sharing, our system becomes a powerful tool to share ideas during online meetings.

This paper proposes a multiple-source multiple-sample fusion approach to identity verification. Fusion is performed at two levels: intramodal and intermodal. In intramodal fusion, the scores of multiple samples (e.g. utterances or video shots) obtained from the same modality are linearly combined, where the combination weights are dependent on the difference between the score values and a user-dependent reference score obtained during enrollment. This is followed by intermodal fusion in which the means of intramodal fused scores obtained from different modalities are fused. The final fused score is then used for decision making. This twolevel fusion approach was applied to audio-visual biometric authentication, and experimental results based on the XM2VTSDB corpus show that the proposed fusion approach can achieve an error rate reduction of up to 83%.

Image Inpainting is the technique of filling out a photo with missing details. The purpose of inpainting is to visualize the realistic reconstruction of lost areas in a way that looks to the human eye natural.We present a novel algorithm in this paper that strengthens, enlarges, and fasters to the previous approach algorithm.A post-processing of the inpainted image is done by enhancement to get better quality of the picture. An exemplar-based approach for image inpainting is applied where Pixel values are copied from the source region and pasted to the target region after the best matching patch for the target region in the picture has been found. The choices of various patch matching criteria, patch size, and filling sequence significantly impact the outcomes. We dynamically choose the source and destination patch sizes. To preserve the consistency of the structures and textures, we search for several source patches globally using the angle-aware rotation technique rather than only choosing one. We have easily produced high-quality results. Our solution. can be prevented somewhat and restoration of the effect of improved inpainting process

Fish-cage dysfunction in aquaculture installations can trigger significant negative consequences affecting the operational costs. Low oxygen levels, due to excessive fooling's, leads to decrease growth performance, and feed efficiency. Therefore, frequent periodic inspection of fish-cage nets is required, but this task can become quite expensive with the traditional means of employing professional divers that perform visual inspections at regular time intervals. The modern trend in aquaculture is to take advantage of IT technologies with the use of a small-sized, low-cost autonomous underwater vehicle, permanently residing within a fish cage and performing regular video inspection of the infrastructure for the entire net surface. In this study, we explore specialised image processing schemes to detect net holes of multiple area size and shape. These techniques are designed with the vision to provide robust solutions that take advantage of either global or local image structures to provide the efficient inspection of multiple net holes.

Rate control is a complicated problem in the H.264/AVC coding standard, extra computation is usually needed for the existing rate control schemes to estimate the complexity of frames or macroblocks (MBs). However, during transcoding, information from precoded video could be used to simplify the rate control. In this paper, we propose a low-complexity rate control scheme for transcoding from H.263 to H.264/AVC. The relationship between the rate of the precoded video and both the rate and distortion of the transcoded video are studied. By using only the rate information from the precoded video, we introduce a row-layer bit allocation and perform average rate shaping across a row of MBs. Estimation error diffusion is also introduced. The proposed scheme has sufficiently lower computational complexity than other methods as there is no explicit complexity measurement of MBs and complicated parameters updating. Experimental results show the effectiveness of the proposed scheme.

Rate control is a complicated problem in the H.264/AVC coding standard, extra computation is usually needed for the existing rate control schemes to estimate the complexity of frames or macroblocks (MBs). However, during transcoding, information from precoded video could be used to simplify the rate control. In this paper, we propose a low-complexity rate control scheme for transcoding from H.263 to H.264/AVC. The relationship between the rate of the precoded video and both the rate and distortion of the transcoded video are studied. By using only the rate information from the precoded video, we introduce a row-layer bit allocation and perform average rate shaping across a row of MBs. Estimation error diffusion is also introduced. The proposed scheme has sufficiently lower computational complexity than other methods as there is no explicit complexity measurement of MBs and complicated parameters updating. Experimental results show the effectiveness of the proposed scheme.

In this paper, a CMOS realization of the current differencing transconductance amplifier (CDTA) is given, which is a newly reported active building block for current-mode signal processing. Current differencing stage of the CDTA element is realized using a differential current-controlled current source and in the output stage, floating current sources are used to convert intermediate voltage of z terminal to output currents. Due to the compactness of the circuit, it is capable of high-frequency operation and suitable for video signal-processing applications. The CDTA element is used in a current-mode anti-aliasing video filter, which is designed using operational simulation of a seventh-order passive elliptic filter, so the resulting active filter has the low sensitivity feature of its passive counterpart. The filter has 0.1 dB maximum pass band ripple as imposed by ITU video anti-aliasing filter standard. SPICE-simulation results of both the CDTA element and the seventh-order elliptic filter are given. Simulation results are found in close agreement with theoretical results.

In this paper we present a novel method for gesture video decomposition based on the depicted content. From the initial content the key-frames are extracted and the neighboring frames are assigned to key-frames of similar content. The resulting frame groups are decomposed to binary trees, based on the energy of the depicted gestures. In case of reduced bandwidth we keep the original timeline but we send only a dynamically adapted video summary. The respective frames are obtained by moving appropriately across the hierarchy layers of the constructed tree. The hierarchically structured video can be used for purposes such as efficient video browsing and transmission of dynamically generated summaries over low bandwidth networks, for communication or human-computer interface applications. I

Deep learning (DL) model performance is intricately tied to the quality of
training, influenced by several parameters. Of these, the computing unit
employed significantly impacts training efficiency. Traditional setups use
central processing units (CPUs) or graphics processing units (GPUs) for DL
training. This paper proposes an alternative using field programmable gate
arrays (FPGAs) for DL training, leveraging their customizable and
parallelizable architecture. FPGA programming allows for tailored circuit
designs, optimizing DL training requirements and enabling efficient parallel
processing. The use of FPGAs in DL training has garnered attention for their
potential in achieving high computational throughput and energy efficiency,
attributed to advantages like low latency, high bandwidth, and
reconfigurability. By exploiting FPGA parallel processing capabilities, faster
training times and the potential for larger, more complex DL models are
feasible. This paper provides an overview of state-of-the-art techniques for
FPGA-based DL model training, discussing challenges such as hardware
architecture design, memory management, and algorithm optimization.
Additionally, various FPGA-based DL frameworks and libraries facilitating
DL model development and deployment on FPGAs are explored.

A saliency-based method for generating video summaries is presented, which exploits coupled audiovisual information from both media streams. Efficient and advanced speech and image processing algorithms to detect key frames that are acoustically and visually salient are used. Promising results are shown from experiments on a movie database.

In this paper, we present a comparative study of several state of the art background subtraction methods. Approaches ranging from simple background subtraction with global thresholding to more sophisticated statistical methods have been implemented and tested on different videos with ground truth. The goal of this study is to provide a solid analytic ground to underscore the strengths and weaknesses of the most widely implemented motion detection methods. The methods are compared based on their robustness to different types of video, their memory requirement, and the computational effort they require. The impact of a Markovian prior as well as some post-processing operators are also evaluated. Most of the videos used in this study

This paper proposes a multiple-source multiple-sample fusion approach to identity verification. Fusion is performed at two levels: intramodal and intermodal. In intramodal fusion, the scores of multiple samples (e.g. utterances or video shots) obtained from the same modality are linearly combined, where the combination weights are dependent on the difference between the score values and a user-dependent reference score obtained during enrollment. This is followed by intermodal fusion in which the means of intramodal fused scores obtained from different modalities are fused. The final fused score is then used for decision making. This twolevel fusion approach was applied to audiovisual biometric authentication, and experimental results based on the XM2VTSDB corpus show that the proposed fusion approach can achieve an error rate reduction of up to 83%.

This paper proposes a multiple-source multiple-sample fusion approach to identity verification. Fusion is performed at two levels: intramodal and intermodal. In intramodal fusion, the scores of multiple samples (e.g. utterances or video shots) obtained from the same modality are linearly combined, where the combination weights are dependent on the difference between the score values and a user-dependent reference score obtained during enrollment. This is followed by intermodal fusion in which the means of intramodal fused scores obtained from different modalities are fused. The final fused score is then used for decision making. This twolevel fusion approach was applied to audiovisual biometric authentication, and experimental results based on the XM2VTSDB corpus show that the proposed fusion approach can achieve an error rate reduction of up to 83%.

In mixed-resolution (MR) stereoscopic video, one view is presented with a lower resolution compared with the other one; therefore, a lower bitrate, a reduced computational complexity, and a decrease in memory access bandwidth can be expected in coding. The human visual system is known to fuse left and right views in such a way that the perceptual visual quality is closer to that of the higher-resolution view. In this paper, a subjective assessment of mixed resolution (MR) stereoscopic videos is presented and the results are analyzed and compared with previous subjective tests presented in the literature. Three downsampling ratios 1/2, 3/8, and 1/4 were used to create lower-resolution views. Hence, the lowerresolution view had different spatial resolutions in terms of pixels per degree (PPD) for each downsampling ratio. It was discovered that the subjective viewing experience tended to follow a logarithmic function of the spatial resolution of the lowerresolution view measured in PPD. A similar behavior was also found from the results of an earlier experiment. Thus, the results suggest that the presented logarithmic function characterizes the expected viewing experience of MR stereoscopic video.

This paper presents a configurable Convolutional Neural Network Accelerator (CNNA) for a System on Chip design (SoC). The goal was to accelerate inference of different deep learning networks on an embedded SoC platform. The presented CNNA has a scalable architecture which uses High Level Synthesis (HLS) and SystemC for the hardware accelerator. It is able to accelerate any Convolutional Neural Network (CNN) exported from Python and supports a combination of convolutional, max-pooling, and fully connected layers. A training method with fixed-point quantized weights is proposed and presented in the paper. The CNNA is template-based, enabling it to scale for different targets of the Xilinx Zynq platform. This approach enables design space exploration, which makes it possible to explore several configurations of the CNNA during Cand RTL-simulation, fitting it to the desired platform and model. The CNN VGG16 was used to test the solution on a Xilinx Ultra96 board using PYNQ. The result gave a high level of accuracy in training with an auto-scaled fixed-point Q2.14 format compared to a similar floating-point model. It was able to perform inference in 2.0 seconds, while having an average power consumption of 2.63 W, which corresponds to a power efficiency of 6.0 GOPS/W.

Providing real-time estimates of building occupancy to first responders during emergency events can help in search and rescue, and egress management. This paper addresses the estimation of occupancy in each zone of a building, where the building is spatially divided into nonoverlapping zones that cover all areas of the building. Each zone contains video cameras located at each portal of the zone, where each camera has a signal processing algorithm that detects number of people passing through the portal in each direction. The technical approach of this paper is to develop a non-linear stochastic state-space model of people traffic during emergency egress, and apply the extended Kalman filter which uses the video signal processing outputs and the people traffic model. The approach is demonstrated on a 16,000 square-foot building that has typical occupancy of 100 people. The estimator is tested on data from an agent-based simulation, and on data from an actual fire alarm. The results show that better estimation accuracy is achieved compared to an estimation approach that uses only the video sensors.

Providing real-time estimates of building occupancy to first responders during emergency events can help in search and rescue, and egress management. This paper addresses the estimation of occupancy in each zone of a building, where the building is spatially divided into nonoverlapping zones that cover all areas of the building. Each zone contains video cameras located at each portal of the zone, where each camera has a signal processing algorithm that detects number of people passing through the portal in each direction. The technical approach of this paper is to develop a non-linear stochastic state-space model of people traffic during emergency egress, and apply the extended Kalman filter which uses the video signal processing outputs and the people traffic model. The approach is demonstrated on a 16,000 square-foot building that has typical occupancy of 100 people. The estimator is tested on data from an agent-based simulation, and on data from an actual fire alarm. The results show that better estimation accuracy is achieved compared to an estimation approach that uses only the video sensors.

We present a computationally efficient algorithm for the eigenspace decomposition of correlated images. Our approach is motivated by the fact that for a planar rotation of a twodimensional image, analytical expressions can be given for the eigendecomposition, based on the theory of circulant matrices. These analytical expressions turn out to be good first approximations of the eigendecomposition, even for three-dimensional objects rotated about a single axis. We use this observation to automatically determine the dimension of the subspace required to represent an image with a guaranteed user-specified accuracy, as well as to quickly compute a basis for the subspace. Examples show that -the algorithm performs very well on a range of test images composed of three-dimensional objects rotated about a single axis.