Papers by Aleksandra Pizurica
Over time, crack pattern (craquelure) inevitably develops in paintings as a sign of their ageing,... more Over time, crack pattern (craquelure) inevitably develops in paintings as a sign of their ageing, sometimes accompanied by larger losses of paint (lacunas). In restoration treatments, cracks are typically not filled in, and virtual restoration is often the only option to "reverse" the ageing of paintings, simulating their original appearance. Moreover, virtual restoration can serve as an important supporting step in decision making during the physical restoration. In this research, we investigate the possibility of applying deep learning-based methods for virtual restoration. In particular, our crack detection method is based on a convolutional autoencoder (U-Net), and we employ a generative adversarial neural network (GAN) to virtually inpaint the detected cracks. We propose an original way of training the GAN model for painting restoration, which improves its practical performance. A series of experiments shows encouraging results in comparison with known methods, and indicates huge potential of deep learning for virtual painting restoratin.
Compressed Sensing (CS) methods using sparse binary measurement matrices and iterative message-pa... more Compressed Sensing (CS) methods using sparse binary measurement matrices and iterative message-passing recovery procedures have been recently investigated due to their low computational complexity and excellent performance. Drawing much of inspiration from sparse-graph codes such as Low-Density Parity-Check (LDPC) codes, these studies use analytical tools from modern coding theory to analyze CS solutions. In this paper, we consider and systematically analyze the CS setup inspired by a class of efficient, popular and flexible sparse-graph codes called rateless codes. The proposed rateless CS setup is asymptotically analyzed using tools such as Density Evolution and EXIT charts and fine-tuned using degree distribution optimization techniques.
Journal of Imaging, Feb 11, 2020
Multichannel images, i.e., images of the same object or scene taken in different spectral bands o... more Multichannel images, i.e., images of the same object or scene taken in different spectral bands or with different imaging modalities/settings, are common in many applications. For example, multispectral images contain several wavelength bands and hence, have richer information than color images. Multichannel magnetic resonance imaging and multichannel computed tomography images are common in medical imaging diagnostics, and multimodal images are also routinely used in art investigation. All the methods for grayscale images can be applied to multichannel images by processing each channel/band separately. However, it requires vast computational time, especially for the task of searching for overlapping patches similar to a given query patch. To address this problem, we propose a three-dimensional orthonormal tree-structured Haar transform (3D-OTSHT) targeting fast full search equivalent for three-dimensional block matching in multichannel images. The use of a three-dimensional integral image significantly saves time to obtain the 3D-OTSHT coefficients. We demonstrate superior performance of the proposed block matching.
Currently, modern achievements in the field of deep learning are increasingly being applied in pr... more Currently, modern achievements in the field of deep learning are increasingly being applied in practice. One of the practical uses of deep learning is to detect cracks on the surface of the roadway. The destruction of the roadway is the result of various factors: for example, the use of low-quality material, non-compliance with the standards of laying asphalt, external physical impact, etc. Detection of these damages in automatic mode with high speed and accuracy is an important and complex task. An effective solution to this problem can reduce the time of services that carry out the detection of damage and also increase the safety of road users. The main challenge for automatically detecting such damage, in most cases, is the complex structure of the roadway. To accurately detect this damage, we use U-Net. After that we improve the binary map with localized cracks from the U-Net neural network, using the morphological filtering. This solution allows localizing cracks with higher accuracy in comparison with traditional methods crack detection, as well as modern methods of deep learning. All experiments were performed using the publicly available CRACK500 dataset with examples of cracks and their binary maps.
Proceedings of SPIE, Mar 19, 2013
Vehicle classification for tunnel surveillance aims to not only retrieve vehicle class statistics... more Vehicle classification for tunnel surveillance aims to not only retrieve vehicle class statistics, but also prevent accidents by recognizing vehicles carrying dangerous goods. In this paper, we describe a method to classify vehicle images that experience different geometrical variations and challenging photometrical conditions such as those found in road tunnels. Unlike previous approaches, we propose a classification method that does not rely on the length and height estimation of the vehicles. Alternatively, we propose a novel descriptor based on trace transform signatures to extract salient and non-correlated information of the vehicle images. Also, we propose a metric that measures the complexity of the vehicles' shape based on corner point detection. As a result, these features describe the vehicle's appearance and shape complexity independently of the scale, pose, and illumination conditions. Experiments with vehicles captured from three different cameras confirm the saliency and robustness of the features proposed, achieving an overall accuracy of 97.5% for the classification of four different vehicle classes. For vehicles transporting dangerous goods, our classification scheme achieves an average recall of 97.6% at a precision of 98.6% for the combination of lorries and tankers, which is a very good result considering the scene conditions.
Compressed sensing (CS) using sparse measurement matrices and iterative messagepassing reconstruc... more Compressed sensing (CS) using sparse measurement matrices and iterative messagepassing reconstruction algorithms have been recently investigated as a low-complexity alternative to traditional CS methods. In this paper, we investigate the adaptive version of well-known Sudocodes scheme, where the sparse measurement matrix is progressively created based on the outcomes of previous measurements. Inspired by resemblance with rateless coding, we provide a detailed analysis of the adaptive Sudocodes approach in combination with the verification-based LM1 reconstruction. The results show that the adaptivity is a promising feature for reducing complexity and improving performance of CS methods based on sparse measurement matrices. * The above Lemma is exact and improves over an approximate version provided in [15](Lemma 2), which does not exhaustively cover all recovery scenarios for zero-valued nodes.
A recent trend in color image processing combines the quaternion algebra with dictionary learning... more A recent trend in color image processing combines the quaternion algebra with dictionary learning methods. This paper aims to present a generalization of the quaternion dictionary learning method by using the octonion algebra. The octonion algebra combined with dictionary learning methods is well suited for representation of multispectral images with up to 7 color channels. Opposed to the classical dictionary learning techniques that treat multispectral images by concatenating spectral bands into a large monochrome image, we treat all the spectral bands simultaneously. Our approach leads to better preservation of color fidelity in true and false color images of the reconstructed multispectral image. To show the potential of the octonion based model, experiments are conducted for image reconstruction and denoising of color images as well as of extensively used Landsat 7 images.
Magnetic Resonance in Medicine
PurposeEcho planar imaging (EPI) is commonly used to acquire the many volumes needed for high ang... more PurposeEcho planar imaging (EPI) is commonly used to acquire the many volumes needed for high angular resolution diffusion Imaging (HARDI), posing a higher risk for artifacts, such as distortion and deformation. An alternative to EPI is fast spin echo (FSE) imaging, which has fewer artifacts but is inherently slower. The aim is to accelerate FSE such that a HARDI data set can be acquired in a time comparable to EPI using compressed sensing.MethodsCompressed sensing was applied in either q‐space or simultaneously in k‐space and q‐space, by undersampling the k‐space in the phase‐encoding direction or retrospectively eliminating diffusion directions for different degrees of undersampling. To test the replicability of the acquisition and reconstruction, brain data were acquired from six mice, and a numerical phantom experiment was performed. All HARDI data were analyzed individually using constrained spherical deconvolution, and the apparent fiber density and complexity metric were eval...
Automated Visual Inspection and Machine Vision III, 2019
Quality control of welded joints is an important step before commissioning of various types of me... more Quality control of welded joints is an important step before commissioning of various types of metal structures. The main obstacles to the commissioning of such facilities are the areas where the welded joint deviates from acceptable defective standards. The defects of welded joints include non-welded, foreign inclusions, cracks, pores, etc. The article describes an approach to the detection of the main types of defects of welded joints using a combination of convolutional neural networks and support vector machine methods. Convolutional neural networks are used for primary classification. The support vector machine is used to accurately define defect boundaries. As a preprocessing in our work, we use the methods of morphological filtration. A series of experiments confirms the high efficiency of the proposed method in comparison with pure CNN method for detecting defects.
Journal of Imaging, 2020
Multichannel images, i.e., images of the same object or scene taken in different spectral bands o... more Multichannel images, i.e., images of the same object or scene taken in different spectral bands or with different imaging modalities/settings, are common in many applications. For example, multispectral images contain several wavelength bands and hence, have richer information than color images. Multichannel magnetic resonance imaging and multichannel computed tomography images are common in medical imaging diagnostics, and multimodal images are also routinely used in art investigation. All the methods for grayscale images can be applied to multichannel images by processing each channel/band separately. However, it requires vast computational time, especially for the task of searching for overlapping patches similar to a given query patch. To address this problem, we propose a three-dimensional orthonormal tree-structured Haar transform (3D-OTSHT) targeting fast full search equivalent for three-dimensional block matching in multichannel images. The use of a three-dimensional integra...
PLOS ONE, 2016
Diffusion Weighted (DW) MRI allows for the non-invasive study of water diffusion inside living ti... more Diffusion Weighted (DW) MRI allows for the non-invasive study of water diffusion inside living tissues. As such, it is useful for the investigation of human brain white matter (WM) connectivity in vivo through fiber tractography (FT) algorithms. Many DW-MRI tailored restoration techniques and FT algorithms have been developed. However, it is not clear how accurately these methods reproduce the WM bundle characteristics in real-world conditions, such as in the presence of noise, partial volume effect, and a limited spatial and angular resolution. The difficulty lies in the lack of a realistic brain phantom on the one hand, and a sufficiently accurate way of modeling the acquisition-related degradation on the other. This paper proposes a software phantom that approximates a human brain to a high degree of realism and that can incorporate complex brain-like structural features. We refer to it as a Diffusion BRAIN (D-BRAIN) phantom. Also, we propose an accurate model of a (DW) MRI acquisition protocol to allow for validation of methods in realistic conditions with data imperfections. The phantom model simulates anatomical and diffusion properties for multiple brain tissue components, and can serve as a ground-truth to evaluate FT algorithms, among others. The simulation of the acquisition process allows one to include noise, partial volume effects, and limited spatial and angular resolution in the images. In this way, the effect of image artifacts on, for instance, fiber tractography can be investigated with great detail. The proposed framework enables reliable and quantitative evaluation of DW-MR image processing and FT algorithms at the level of large-scale WM structures. The effect of noise levels and other data characteristics on cortico-cortical connectivity and tractographybased grey matter parcellation can be investigated as well.
IEEE International Conference on Acoustics Speech and Signal Processing, 2002
In this paper, a wavelet based method is proposed to estimate the blur in an image using informat... more In this paper, a wavelet based method is proposed to estimate the blur in an image using information contained in the image itself. We look at the sharpness of the sharpest edges in the blurred image, which contain information about the blurring. Specifically, a smoothness measure, the Lipschitz exponent, is computed for these sharpest edges. A relation between the variance of a gaussian point spread function and the magnitude of the Lipschitz exponent is shown, which is only dependent on the blur in the image and not on the image contents. This allows us to estimate the variance of the blur directly from the image itself.
7th International Conference on Image Processing and its Applications, 1999
This paper describes an algorithm for the noise removal in the wavelet domain, which takes into a... more This paper describes an algorithm for the noise removal in the wavelet domain, which takes into account not only the local noise measure, but also prior spatial constraints. These prior spatial or geometrical constraints express the fact that meaningful wavelet coefficients appear in spatially connected clusters, forming edges of given directions. Existing techniques that exploit this kind of prior knowledge are computationally expensive, while the proposed method exploits the spatial constraints in a different and simple manner.
2008 Computers in Cardiology, 2008
2007 IEEE International Conference on Image Processing, 2007
This paper presents a novel passive error concealment method for wavelet coded images. The propos... more This paper presents a novel passive error concealment method for wavelet coded images. The proposed method is a locally adaptive directional interpolation approach, where the interpolation weights are estimated based on the available local context. For each lost low frequency coefficient, we estimate the optimal interpolation weights based on the errors that would arise by horizontally and vertically interpolating the available neighbors of the lost coefficient. Compared to older methods of similar complexity, the proposed scheme estimates the lost coefficients much better: on average, the PSNR is increased with up to 0.6 dB. The results also indicate improvements over the best available state-of-the-art techniques. The reconstructed images also look better. As our method is fast and of low complexity, it is widely usable.
2009 International Workshop on Local and Non-Local Approximation in Image Processing, 2009
Recently, there has been a huge interest in multiresolution representations that also perform a m... more Recently, there has been a huge interest in multiresolution representations that also perform a multidirectional analysis. The Shearlet transform provides both a multiresolution analysis (such as the wavelet transform), and at the same time an optimally sparse image-independent representation for images containing edges. Existing discrete implementations of the Shearlet transform have mainly focused on specific applications, such as edge detection or denoising, and were not designed with a low redundancy in mind (the redundancy factor is typically larger than the number of orientation subbands in the finest scale). In this paper, we present a novel design of a Discrete Shearlet Transform, that can have a redundancy factor of 2.6, independent of the number of orientation subbands, and that has many interesting properties, such as shift-invariance and self-invertability. This transform can be used in a wide range of applications. Experiments are provided to show the improved characteristics of the transform.
2011 18th IEEE International Conference on Image Processing, 2011
In this paper, we present a new method for High Dynamic Range (HDR) reconstruction based on a set... more In this paper, we present a new method for High Dynamic Range (HDR) reconstruction based on a set of multiple photographs with different exposure times. While most existing techniques take a deterministic approach by assuming that the acquired low dynamic range (LDR) images are noise-free, we explicitly model the photon arrival process by assuming sensor data corrupted by Poisson noise. Taking the noise characteristics of the sensor data into account leads to a more robust way to estimate the non-parametric camera response function (CRF) compared to existing techniques. To further improve the HDR reconstruction, we adopt the split-Bregman framework and use Total Variation for regularization. Experimental results on real camera images and ground-truth data show the effectiveness of the proposed approach.
Number of samples PSNR PSNR as a function of number of samples Pseudo-random sampling Quasi-rando... more Number of samples PSNR PSNR as a function of number of samples Pseudo-random sampling Quasi-random sampling TOTAL VARIATION MINIMIZATION QUASI-RANDOM SAMPLING 2nd order SFC 3rd order SFC 1st order SFC Coordinates of the golden ratio sequence (below) are transformed from the unit interval to the unit square with the Hilbert space filling curve.
Video Surveillance and Transportation Imaging Applications, 2013
Vehicle classification for tunnel surveillance aims to not only retrieve vehicle class statistics... more Vehicle classification for tunnel surveillance aims to not only retrieve vehicle class statistics, but also prevent accidents by recognizing vehicles carrying dangerous goods. In this paper, we describe a method to classify vehicle images that experience different geometrical variations and challenging photometrical conditions such as those found in road tunnels. Unlike previous approaches, we propose a classification method that does not rely on the length and height estimation of the vehicles. Alternatively, we propose a novel descriptor based on trace transform signatures to extract salient and non-correlated information of the vehicle images. Also, we propose a metric that measures the complexity of the vehicles' shape based on corner point detection. As a result, these features describe the vehicle's appearance and shape complexity independently of the scale, pose, and illumination conditions. Experiments with vehicles captured from three different cameras confirm the saliency and robustness of the features proposed, achieving an overall accuracy of 97.5% for the classification of four different vehicle classes. For vehicles transporting dangerous goods, our classification scheme achieves an average recall of 97.6% at a precision of 98.6% for the combination of lorries and tankers, which is a very good result considering the scene conditions.
Wavelet Applications in Industrial Processing VII, 2010
We propose a real-time system for blur estimation using wavelet decomposition. The system is base... more We propose a real-time system for blur estimation using wavelet decomposition. The system is based on an emerging multi-core microprocessor architecture (Cell Broadband Engine, Cell BE) known to outperform any available general purpose or DSP processor in the domain of real-time advanced video processing solutions. We start from a recent wavelet domain blur estimation algorithm which uses histograms of a local regularity measure called average cone ratio (ACR). This approach has shown a very good potential for assessing the level of blur in the image yet some important aspects remain to be addressed in order for the method to become a practically working one. Some of these aspects are explored in our work. Furthermore, we develop an efficient real-time implementation of the novelty metric and integrate it into a system that captures live video. The proposed system estimates blur extent and renders the results to the remote user in real-time.
Uploads
Papers by Aleksandra Pizurica