Sound (Signal Processing)

In this paper, we address the tasks of audio source counting and separation for a stereo anechoic mixture of audio signals. This will be achieved in two stages. In the first stage, a novel approach is introduced for estimating the number of sources as well as the channel mixing coefficients. For this purpose, a 2-D spectrum is evaluated against both the phase and amplitude differences of the two channels. Hence, obtaining the peak locations of the spectrum yields the number of the sources and the corresponding channel coefficients. In the second stage, an extension of a single channel complex matrix factorization method to multichannel is developed to extract the individual source signals. We find primary estimates of the sources via binary masking and then apply the complex factorization to the complex spectrogram of each source. The obtained factors are then utilized as initial values in the complex multichannel factorization model. We also suggest a method for estimating the number of required components for modeling each source. The separation performance improvement over the conventional methods is investigated by calculating BSS evaluation metrics. The comparison is also carried out in terms of source counting and localization with the recently proposed DeMIX-Anechoic method.

This project explores the feasibility of detecting and tracking long-range combustion-engine UAVs using a distributed network of acoustic IoT sensors and multi-hop wireless communications. The idea is inspired by the increasing use of long-range UAVs in modern conflicts, especially lowaltitude drones powered by combustion engines. Such UAVs are difficult to detect using classical radar systems due to their low radar cross section, terrain masking, and low flight altitude. However, they generate a distinctive acoustic signature that may be exploited for passive detection. The project uses Modeling & Simulation (M&S) to prototype and evaluate: acoustic detection capabilities, probabilistic sensing, distributed wireless mesh communication, and basic UAV tracking techniques.

This paper introduces the ongoing ElectroAcoustic Resource Site Pedagogical Project, or EARS II, in some detail. EARS II is to become an online educational resource for two groups of users: children of ca. 11-14 years of age as well as people of all ages who have little to no knowledge of electroacoustic music and undergraduate students embarking upon a study of electroacoustic music. EARS II is based on a holistic approach linking appreciation to understanding and creative application. The paper’s goal is to share the project vision and provide an update concerning how far the project team has come. The project is the second phase of the original EARS site (www.ears.dmu.ac.uk) as well as the second phase of the Music, Technology and Innovation Research Centre’s Intention/Reception project.

Music genre classification is one of the sub-disciplines of music information retrieval (MIR) with growing popularity among researchers, mainly due to the already open challenges. Although research has been prolific in terms of number of published works, the topic still suffers from a problem in its foundations: there is no clear and formal definition of what genre is. Music categorizations are vague and unclear, suffering from human subjectivity and lack of agreement. In its first part, this paper offers a survey trying to cover the many different aspects of the matter. Its main goal is give the reader an overview of the history and the current state-of-the-art, exploring techniques and datasets used to the date, as well as identifying current challenges, such as this ambiguity of genre definitions or the introduction of human-centric approaches. The paper pays special attention to new trends in machine learning applied to the music annotation problem. Finally, we also include a mu...

In recent years, deep networks have led to dramatic improvements in speech enhancement by framing it as a data-driven pattern recognition problem. In many modern enhancement systems, large amounts of data are used to train a deep network to estimate masks for complex-valued short-time Fourier transforms (STFTs) to suppress noise and preserve speech. However, current masking approaches often neglect two important constraints: STFT consistency and mixture consistency. Without STFT consistency, the system’s output is not necessarily the STFT of a time-domain signal, and without mixture consistency, the sum of the estimated sources does not necessarily equal the input mixture. Furthermore, the only previous approaches that apply mixture consistency use real-valued masks; mixture consistency has been ignored for complex-valued masks.

Web-based recommendation strategy implemented in a cadastre information system is presented in the paper. This method forms the list of page profiles recommended to a given user. The idea of page recommendation uses the concept of a page profile for system pages containing forms with search criteria. The calculation of rank values for page profiles is based on the usage frequency and the significance weights of profile elements determined by users.

Introduction-Since the outbreak began in January 2020, Covid-19 has affected more than 161 million people worldwide and resulted in about 3.3 million deaths. Despite efforts to detect human infection with the virus as early as possible, the confirmatory test still requires the analysis of sputum or blood with estimated results available within approximately 30 minutes; this may potentially be followed by clinical referral if the patient shows signs of aggravated pneumonia. This work aims to implement a soft collar as a sound device dedicated to the objective evaluation of the pathophysiological state resulting from dysphonia of laryngeal origin or respiratory failure of inflammatory origin, in particular caused by Covid-19. Methods-In this study, we exploit the vibrations of waves generated by the vocal and respiratory system of 30 people. A biocompatible acoustic sensor embedded in a soft collar around the neck collects these waves. The collar is also equipped with thermal sensors and a cross-data analysis module in both the temporal and frequency domains (STFT). The optimal coupling conditions and the electrical and dimensional characteristics of the sensors were defined based on a mathematical approach using a matrix formalism. Results-The characteristics of the signals in the time domain combined with the quantities obtained from the STFT offer multidimensional information and a decision support tool for determining a pathophysiological state representative of the symptoms explored. The device, tested on 30 people, was able to differentiate patients with mild symptoms from those who had developed acute signs of respiratory failure on a severity scale of 1 to 10. Conclusion-With the health constraints imposed by the effects of Covid-19, the heavy organization to be implemented resulting from the flow of diagnostics, tests and clinical management, it was urgent to develop innovative and safe biomedical technologies. This passive listening technique will contribute to the non-invasive assessment and dynamic observation of lesions. Moreover, it merits further examina-tion to provide support for medical operators to improve clinical management.

Emotion is a complicated notion present in music that is hard to capture even with fine-tuned feature engineering. In this paper, we investigate the utility of state-of-the-art pre-trained deep audio embedding methods to be used in the Music Emotion Recognition (MER) task. Deep audio embedding methods allow us to efficiently capture the high dimensional features into a compact representation. We implement several multi-class classifiers with deep audio embeddings to predict emotion semantics in music. We investigate the effectiveness of L 3 -Net and VGGish deep audio embedding methods for music emotion inference over four music datasets. The experiments with several classifiers on the task show that the deep audio embedding solutions can improve the performances of the previous baseline MER models. We conclude that deep audio embeddings represent musical emotion semantics for the MER task without expert human engineering.

In this article, we explore the potential of using latent diffusion models, a family of powerful generative models, for the task of reconstructing naturalistic music from electroencephalogram (EEG) recordings. Unlike simpler music with limited timbres, such as MIDI-generated tunes or monophonic pieces, the focus here is on intricate music featuring a diverse array of instruments, voices, and effects, rich in harmonics and timbre. This study represents an initial foray into achieving general music reconstruction of high-quality using non-invasive EEG data, employing an end-to-end training approach directly on raw data without the need for manual pre-processing and channel selection. We train our models on the public NMED-T dataset and perform quantitative evaluation proposing neural embedding-based metrics. Our work contributes to the ongoing research in neural decoding and brain-computer interfaces, offering insights into the feasibility of using EEG data for complex auditory information reconstruction.

Vehicles generate dissimilar sound patterns under different health conditions. The sound generated by the vehicles gives a clue of some of the faults. Automotive experts diagnose the faults in vehicles based on the produced sound. This paper presents a methodology for fault source localization in motorcycles using the estimated pseudospectra of the sound signals. The pseudospectra are traced to generate a chaincode. The generated chaincode is transformed into an eigenvector, which is used as the feature vector for classification using artificial neural network (ANN). The overall classification accuracy is 88%. The proposed work finds applications in traffic census of the vehicles, traffic rule observance, machine fault detection and localization, automatic acoustic surveillance and the like.

Motorcycles generate different sound patterns under dissimilar working conditions. The generated sound pattern gives a clue of the fault. Mainly the parts of the engine that lead to change in sound are cylinder kit, crank, timing chain, and valve. The parts of the exhaust system that change sound under fault are muffler and silencer. In this study, we analyze the sound signals produced by motorcycles to locate the faults in subsystems. The work proceeds in three stages, the first stage detects the fault, the second stage identifies the faulty subsystem and finally the third stage locates the fault. The overall classification accuracy of the first stage is 0.8019. The work finds interesting applications in troubleshooting of machinery, electronic gadgets, musical instruments and the like.

In this paper we propose two generic mechanisms implemented in a cadastre internet information system. The first one is the list of last queries submitted by a given user and the second one is the list of page profiles recommended to a user. The idea of page recommendation is based on the concept of a page profile which represents a system option, type of retrieval mechanisms and search criteria. The calculation of rank values for page profiles is based on the usage frequency and the time impact coefficient. A recommended page is selected by a user from a list facilitates and accelerates his searches by moving him directly to the chosen option page with search form filled with the most expected criteria values. As an additional complementary mechanism the list of last submitted queries is available to each user.

Introduction-Since the outbreak began in January 2020, Covid-19 has affected more than 161 million people worldwide and resulted in about 3.3 million deaths. Despite efforts to detect human infection with the virus as early as possible, the confirmatory test still requires the analysis of sputum or blood with estimated results available within approximately 30 minutes; this may potentially be followed by clinical referral if the patient shows signs of aggravated pneumonia. This work aims to implement a soft collar as a sound device dedicated to the objective evaluation of the pathophysiological state resulting from dysphonia of laryngeal origin or respiratory failure of inflammatory origin, in particular caused by Covid-19. Methods-In this study, we exploit the vibrations of waves generated by the vocal and respiratory system of 30 people. A biocompatible acoustic sensor embedded in a soft collar around the neck collects these waves. The collar is also equipped with thermal sensors and a cross-data analysis module in both the temporal and frequency domains (STFT). The optimal coupling conditions and the electrical and dimensional characteristics of the sensors were defined based on a mathematical approach using a matrix formalism. Results-The characteristics of the signals in the time domain combined with the quantities obtained from the STFT offer multidimensional information and a decision support tool for determining a pathophysiological state representative of the symptoms explored. The device, tested on 30 people, was able to differentiate patients with mild symptoms from those who had developed acute signs of respiratory failure on a severity scale of 1 to 10. Conclusion-With the health constraints imposed by the effects of Covid-19, the heavy organization to be implemented resulting from the flow of diagnostics, tests and clinical management, it was urgent to develop innovative and safe biomedical technologies. This passive listening technique will contribute to the non-invasive assessment and dynamic observation of lesions. Moreover, it merits further examination to provide support for medical operators to improve clinical management.

In this work, we aim to improve the expressive capacity of waveform-based discriminative music networks by modeling both sequential (temporal) and hierarchical information in an efficient end-to-end architecture. We present MuSLCAT, or Multi-scale and Multi-level Convolutional Attention Transformer, a novel architecture for learning robust representations of complex music tags directly from raw waveform recordings. We also introduce a lightweight variant of MuSLCAT called MuSLCAN, short for Multi-scale and Multi-level Convolutional Attention Network. Both MuSLCAT and MuSLCAN model features from multiple scales and levels by integrating a frontend-backend architecture. The frontend targets different frequency ranges while modeling long-range dependencies and multi-level interactions by using two convolutional attention networks with attention-augmented convolution (AAC) blocks. The backend dynamically recalibrates multi-scale and level features extracted from the frontend by incorpor...

Hit Song Science aims to predict a songs popularity based on song structure and externalfeatures. To help provide an efficient and accurate tool for Annual Top-100 Billboard SongClassification, we apply fine-tuned BERT transformer and a joint learning approach. Wealso explore different audio descriptors and lyrics based embedding for modeling hit songclassification task on an innovative western song dataset created by us. We address classimbalance and data scarcity issues associated with traditional datasets by employing sharedlayer architecture and penalizing loss. We highlight a comparison study of three distinctlydesigned neural network architectures. All models yield high overall accuracy with relatively low training cost.

This paper proposes an effective sequential initialization for multichannel nonnegative matrix factorization to address the difficulty of initial value dependency of the conventional method. The proposed method sets initial values of parameters from those obtained for the smaller number of channels. The experimental results of music separation show that the proposed method outperforms the conventional method.

A design methodology of an all-digital phaselocked loop based on standard library cells is presented. The design route includes development of a scalable architecture to enable migration to various technology libraries. The design features of the main blocks allow automatic placement and routing in a digital route without degradation of the circuit characteristics. The proposed architecture combines design simplicity and good performance.

В ХХ веке, наряду с традиционными музыкальными складами – монодическим, полифоническим и гомофонно-гармоническим, возникли и три новых – сонористика, электроакустическая музыка и мультимедиа. Музыка, написанная в первом из них, исполняется на традиционных инструментах, но в ней уже нет опоры на аккорды; во втором, как следует из его названия, господствуют электронные звучания; в третьем появляются и визуальные элементы (жесты, фильмы и т. д.). Ни в одном из этих складов нет центрального звука или созвучия; вместе с тем их элементы и не равнозначны между собой: и здесь можно выделить из них более и менее устойчивые; но устойчивость здесь относительная. Тем не менее её можно описать математически, опираясь на общую для всей музыки константу – время. Этому и посвящено данное исследование

Anomalous sound detection (ASD) is, nowadays, one of the topical subjects in machine listening discipline. Unsupervised detection is attracting a lot of interest due to its immediate applicability in many fields. For example, related to industrial processes, the early detection of malfunctions or damage in machines can mean great savings and an improvement in the efficiency of industrial processes. This problem can be solved with an unsupervised ASD solution since industrial machines will not be damaged simply by having this audio data in the training stage. This paper proposes a novel framework based on convolutional autoencoders (both unsupervised and semisupervised) and a Gammatone-based representation of the audio. The results obtained by these architectures substantially exceed the results presented as a baseline.

Audio datasets support the training and validation of Machine Learning algorithms in audio classification problems. Such datasets include different, arbitrarily chosen audio classes. We initially investigate a unifying approach, based on the mapping of audio classes according to the Audioset ontology. Using the ESC-10 audio dataset, a tree-like representation of its classes is created. In addition, we employ an audio similarity calculation tool based on the values of extracted features (spectrum centroid, the spectrum flux and the spectral roll-off). This way the audio classes are connected both semantically and in feature-based manner. Employing the same dataset, ESC-10, we perform sound classification using CNNbased algorithms, after transforming the sound excerpts into images (based on their Mel spectrograms). The YAMNet and VGGish networks are used for audio classification and the accuracy reaches 90%. We extend the classification algorithm with segmentation logic, so that it can be applied into more complex sound excerpts, where multiple sound types are included in a sequential and/or overlapping manner. Quantitative metrics are defined on the behavior of the combined segmentation and segmentation functionality, including two key parameters for the merging operation, the minimum duration of the identified sounds and the intervals. The qualitative metrics are related to the number of sound identification events for a concatenated sound excerpt of the dataset and per each sound class. This way the segmentation logic can operate in a fine-and coarse-grained manner while the dataset and the individual sound classes are characterized in terms of clearness and distinguishability.

This article describes a computationally-efficient statistical approach to joint (semi-)blind source separation and dereverberation for multichannel noisy reverberant mixture signals. A standard approach to source separation is to formulate a generative model of a multichannel mixture spectrogram that consists of source and spatial models representing the time-frequency power spectral densities (PSDs) and spatial covariance matrices (SCMs) of source images, respectively, and find the maximum-likelihood estimates of these parameters. A state-of-the-art blind source separation method in this thread of research is fast multichannel nonnegative matrix factorization (FastMNMF) based on the lowrank PSDs and jointly-diagonalizable full-rank SCMs. To perform mutually-dependent separation and dereverberation jointly, in this paper we integrate both moving average (MA) and autoregressive (AR) models that represent the early reflections and late reverberations of sources, respectively, into the FastMNMF formalism. Using a pretrained deep generative model of speech PSDs as a source model, we realize semi-blind joint speech separation and dereverberation. We derive an iterative optimization algorithm based on iterative projection or iterative source steering for jointly and efficiently updating the AR parameters and the SCMs. Our experimental results showed the superiority of the proposed ARMA extension over its AR-or MA-ablated version in a speech separation and/or dereverberation task.

This paper describes a semi-supervised multichannel speech enhancement method that uses clean speech data for prior training. Although multichannel nonnegative matrix factorization (MNMF) and its constrained variant called independent low-rank matrix analysis (ILRMA) have successfully been used for unsupervised speech enhancement, the low-rank assumption on the power spectral densities (PSDs) of all sources (speech and noise) does not hold in reality. To solve this problem, we replace a low-rank speech model with a deep generative speech model, i.e., formulate a probabilistic model of noisy speech by integrating a deep speech model, a low-rank noise model, and a full-rank or rank-1 model of spatial characteristics of speech and noise. The deep speech model is trained from clean speech data in an unsupervised auto-encoding variational Bayesian manner. Given multichannel noisy speech spectra, the full-rank or rank-1 spatial covariance matrices and PSDs of speech and noise are estimated in an unsupervised maximum-likelihood manner. Experimental results showed that the full-rank version of the proposed method was significantly better than MNMF, ILRMA, and the rank-1 version. We confirmed that the initialization-sensitivity and local-optimum problems of MNMF with many spatial parameters can be solved by incorporating the precise speech model.

Visualizations help decipher latent patterns in music and garner a deep understanding of a song's characteristics. This paper offers a critical analysis of the effectiveness of various state-of-the-art Deep Neural Networks in visualizing music. Several implementations of auto encoders and genre classifiers have been explored for extracting meaningful features from audio tracks. Novel techniques have been devised to map these audio features to parameters that drive visualizations. These methodologies have been designed in a manner that enables the visualizations to be responsive to the music as well as provide unique visual experiences across different songs.

In many ways, all non-representational arts have distanced themselves to a greater or lesser extent from their potential public over the centuries due to the fact that art and life have been largely separated. For example, those who have supported the notion of art for art's sake for over two hundred years have been rather explicit about this separation. Nevertheless, most human beings still enjoy and find it natural to make links between the artistic and lived experience. The inclusion of the sound as potential musical material has not only led to new and radical forms of soundbased music making, but also to the opportunity for life to become part of music. This talk focuses on the impact, perhaps unintended, Pierre Schaeffer had when he coined the term, écoute réduite and considered it to be of importance in terms of the success of what is known today as acousmatic music. An opposing view is presented, namely that of the use of real-life sounds across the innovative sound-based musical spectrum, primarily those genres employing electroacoustic or related new media approaches. It will be suggested that sampling is one case where musical experimentation may actually lead towards increased appreciation and artistic participation in new forms of music making. Regardless of this suggestion, the talk's aim as evidenced in its conclusion is one of synthesis, not opposition. Preface One of the idées fixes throughout my career has been my fairly lonely attack against today's reality of many contemporary artists working their way into a corner due to a lack of connection with a public larger than that of their peers. It is almost as if artists are actively working towards various art musics' own demise. Things need not be so gloomy, however. This talk will take advantage of my view that through sampling some forms of musical experimentation may find greater access than has been achieved by a good deal of contemporary 'academic music' around the globe. The reason for this is sample-based music's ability to connect with human experience, that is, in the sense of using 'recycled' samples from the real world. The discussion will be structured as follows. It will commence with some thoughts regarding the relationship between art and life, both historically and in terms of today's art making to set the context. Then I shall investigate some perceived tensions between reduced and what I call heightened listening. The section that follows concerns music as 'organised samples' discussing issues related to sample-based work which may lead some of you to believe that I am taking sides, but this is only a step towards the talk's conclusion which is one of synthesis. The key focus of this talk, as it is in my scholarly and artistic work, is that of 'soundbased music'. This term is defined as follows: sound-based music typically designates the art form in which the sound, that is, not the musical note, is its basic unit (Landy 2007a, 17). One may query why notes are being excluded here and, also, why the more common terms electroacoustic music and sonic art are being ignored. To cut two long stories short, it was proposed in this book and its successor (Landy 2007b) that sound-based music possesses its own paradigm, as does note-based music. This paradigm is highly associated with that of other new media arts and takes into account both poietic and esthesic aspects. The reason to avoid the two more widely used terms, discussed in my two recent books at length, has to do with the term, electroacoustic music being used inconsistently and also its inclusion of certain notebased works. The issue with sonic arts is that the term can be used as an excuse for works to be considered not to be music, a view that leaves me feeling uncomfortable. The discussion will privilege the reception of sample-based works above the oftendiscussed areas of construction, tools and channels of dissemination. The reason for brought to you by CORE View metadata, citation and similar papers at core.ac.uk

A hybrid classifier obtained by hybridizing Support Vector Machines (SVM) and Artificial Neural Network (ANN) classifiers is presented here for diagnosis of gear faults. The distinctive features obtained from vibration signals of a running gearbox, which was operated in normal and fault-induced conditions, were used to feed the SVM-ANN hybrid classifier. Time-domain vibration signals were divided in segments. Features such as peaks in time domain and in spectrum, central moments, and standard deviations were obtained from signal segments. Based on the experimental results, it was shown that SVM-ANN hybrid classifier can successfully identify gear condition and that the hybrid SVM-ANN classifier performs much better than standard versions of ANNs and SVM. The effectiveness of the hybrid classifier under noise was also investigated. It was shown that if vibration signals are preprocessed by Discrete Wavelet Transform (DWT), efficacy of the SVM-ANN hybrid is significantly enhanced.

The extremely challenging nature of passive acoustic surveillance makes it a key area of research in Naval Non-Co-operative Target Recognition especially in Anti-Submarine Warfare systems. In shallow waters, the complex acoustics due to the highly varying ambient background noise as well as the multi-modal propagation in the surface-bottom bounded channel makes surveillance even difficult. In this work, an ensemble of Convolutional Neural Networks and Bidirectional Long Short Term Memory stages employing soft attention is used to effectively capture the spectro-temporal dynamics of the target signature. In order to alleviate the overall computational cost associated with the optimal model search in the extensive hyperparameter space, a recursive model elimination scheme, making frugal use of the available resources, is also proposed. Experimental analysis on acoustic target records, collected from the shallows of Arabian Sea, has yielded encouraging results in terms of model accurac...

The current work is devoted to the analysis of the sound paradigm in video games of the horror genre. The sound in computer games is an important component, since the game is a syncretic medium. The aim of the work is an attempt to establish the most relevant typology of sound elements of a given genre, as well as the correlation of the considered paradigm with various game interfaces. Understanding the functioning of sound and its relationship with various interface forms in computer games makes it possible to better comprehend their cultural significance, to better see their relationship with other media, as well as their impact on the playing subject. The article offers an analysis of the most popular and canonical elements of the video game audio paradigm. The author of the article defines the game sound, indicates its functions (semantic, immersive, gameplay), considers the concept of “diegesis” in the context of the game sound, describes the emergence and characteristics of th...

As digital music production has become mainstream, the selection of appropriate virtual instruments plays a crucial role in determining the quality of music. To search the musical instrument samples or virtual instruments that make one's desired sound, music producers use their ears to listen and compare each instrument sample in their collection, which is time-consuming and inefficient. In this paper, we call this task as Musical Instrument Retrieval and propose a method for retrieving desired musical instruments using reference mixture audio as a query. The proposed model consists of the Single-Instrument Encoder and the Multi-Instrument Encoder, both based on convolutional neural networks. The Single-Instrument Encoder is trained to classify the instruments used in single-track audio, and we take its penultimate layer's activation as the instrument embedding. The Multi-Instrument Encoder is trained to estimate multiple instrument embeddings using the instrument embeddings computed by the Single-Instrument Encoder as a set of target embeddings. For more generalized training and realistic evaluation, we also propose a new dataset called Nlakh. Experimental results showed that the Single-Instrument Encoder was able to learn the mapping from the audio signal of unseen instruments to the instrument embedding space and the Multi-Instrument Encoder was able to extract multiple embeddings from the mixture audio and retrieve the desired instruments successfully. The code used for the experiment and audio samples are

As digital music production has become mainstream, the selection of appropriate virtual instruments plays a crucial role in determining the quality of music. To search the musical instrument samples or virtual instruments that make one's desired sound, music producers use their ears to listen and compare each instrument sample in their collection, which is time-consuming and inefficient. In this paper, we call this task as Musical Instrument Retrieval and propose a method for retrieving desired musical instruments using reference mixture audio as a query. The proposed model consists of the Single-Instrument Encoder and the Multi-Instrument Encoder, both based on convolutional neural networks. The Single-Instrument Encoder is trained to classify the instruments used in single-track audio, and we take its penultimate layer's activation as the instrument embedding. The Multi-Instrument Encoder is trained to estimate multiple instrument embeddings using the instrument embeddings computed by the Single-Instrument Encoder as a set of target embeddings. For more generalized training and realistic evaluation, we also propose a new dataset called Nlakh. Experimental results showed that the Single-Instrument Encoder was able to learn the mapping from the audio signal of unseen instruments to the instrument embedding space and the Multi-Instrument Encoder was able to extract multiple embeddings from the mixture audio and retrieve the desired instruments successfully. The code used for the experiment and audio samples are

Experiment presented in this study, used vibration data obtained from a four-stroke, 295 diesel engine. Fault of the internal-combustion engine was detected by using the vibration signals of the cylinder head. The fault diagnosis system was designed and constructed for inspecting the status and fault diagnosis of a diesel engine based on discrete wavelet analysis and LabVIEW software. The cylinder-head vibration signals were captured through a piezoelectric acceleration sensor, that was attached to a surface of the cylinder head of the engine, while the engine was running at two speeds (620 and 1300 rpm) and two loads (15 and 45 N•m). Data was gathered from five different conditions, associated with the cylinder head such as single cylinder shortage, double cylinders shortage, intake manifold obstruction, exhaust manifold obstruction and normal condition. After decomposing the vibration signals into some of the details and approximations coefficients with db5 mother wavelet and decomposition level 5, the energies were extracted from each frequency sub-band of healthy and unhealthy conditions as a feature of engine fault diagnosis. By doing so, normal and abnormal conditions behavior could be effectively distinguished by comparing the energy accumulations of each sub-band. The results showed that detection of fault by discrete wavelet analysis is practicable. Finally, two techniques, Back-Propagation Neural Network (BPNN) and Support Victor Machine (SVM) were applied to the signal that was collected from the diesel engine head. The experimental results showed that BPNN was more effective in fault diagnosis of the internal-combustion engine, with various fault conditions, than SVM.

Despite there being clear evidence for top-down (e.g., attentional) effects in biological spatial hearing, relatively few machine hearing systems exploit top-down model-based knowledge in sound localisation. This paper addresses this issue by proposing a novel framework for binaural sound localisation that combines model-based information about the spectral characteristics of sound sources and deep neural networks (DNNs). A target source model and a background source model are first estimated during a training phase using spectral features extracted from sound signals in isolation. When the identity of the background source is not available, a universal background model can be used. During testing, the source models are used jointly to explain the mixed observations and improve the localisation process by selectively weighting source azimuth posteriors output by a DNN-based localisation system. To address the possible mismatch between training and testing, a model adaptation process is further employed on-the-fly during testing, which adapts the background model parameters directly from the noisy observations in an iterative manner. The proposed system therefore combines model-based and data-driven information flow within a single computational framework. The evaluation task involved localisation of a target speech source in the presence of an interfering source and room reverberation. Our experiments show that by exploiting model-based information in this way, sound localisation performance can be improved substantially under various noisy and reverberant conditions.

Essentia is a reference open-source C++/Python library for audio and music analysis. In this work, we present a set of algorithms that employ TensorFlow in Essentia, allow predictions with pre-trained deep learning models, and are designed to offer flexibility of use, easy extensibility, and real-time inference. To show the potential of this new interface with TensorFlow, we provide a number of pre-trained state-of-the-art music tagging and classification CNN models. We run an extensive evaluation of the developed models. In particular, we assess the generalization capabilities in a crosscollection evaluation utilizing both external tag datasets as well as manual annotations tailored to the taxonomies of our models.

We have built a music similarity search engine that lets video producers search by listenable music excerpts, as a complement to traditional full-text search. Our system suggests similar sounding track segments in a large music catalog by training a self-supervised convolutional neural network with triplet loss terms and musical transformations. Semi-structured user interviews demonstrate that we can successfully impress professional video producers with the quality of the search experience, and perceived similarities to query tracks averaged 7.8/10 in user testing. We believe this search tool will make for a more natural search experience that is easier to find music to soundtrack videos with.

Emotion recognition (ER) from speech signals is a robust approach since it cannot be imitated like facial expression or text based sentiment analysis. Valuable information underlying the emotions are significant for human-computer interactions enabling intelligent machines to interact with sensitivity in the real world. Previous ER studies through speech signal processing have focused exclusively on associations between different signal mode decomposition methods and hidden informative features. However, improper decomposition parameter selections lead to informative signal component losses due to mode duplicating and mixing. In contrast, the current study proposes VGG-optiVMD, an empowered variational mode decomposition algorithm, to distinguish meaningful speech features and automatically select the number of decomposed modes and optimum balancing parameter for the data fidelity constraint by assessing their effects on the VGG16 flattening output layer. Various feature vectors were employed to train the VGG16 network on different databases and assess VGG-optiVMD reproducibility and reliability. One, two, and three-dimensional feature vectors were constructed by concatenating Mel-frequency cepstral coefficients, Chromagram, Mel spectrograms, Tonnetz diagrams, and spectral centroids. Results confirmed a synergistic relationship between the fine-tuning of the signal sample rate and decomposition parameters with classification accuracy, achieving state-of-the-art 96.09% accuracy in predicting seven emotions on the Berlin EMO-DB database.

В данной работе с помощью техники регистрации событийно-связанных потенциалов (ССП), исследованы особенности нейрокогнитивных процессов при обработке расстояния тональной модуляции. В исследовании приняли участие 20 добровольцев (6 мужчин, средний возраст участников 19.7 ± 2.3 года). Все испытуемые были правшами; ни у кого из них не было профессиональной музыкальной подготовки. Испытуемым предлагалось прослушать ряд гармонических последовательностей с тональной модуляцией в субдоминанту (близкая степень модуляции, замена 1 тона по отношению к начальной гамме), в малую сексту (дальняя степень модуляции, замена 4 тонов по отношению к начальной гамме) и в тритон (дальняя модуляция, замена 6 тонов по отношению к начальной гамме). Для оценки эффекта модуляции использовалась последовательность без модуляции. Показано уменьшение амплитуды волны N200 при прослушивании гармонических последовательностей вне зависимости от степени модуляции. Было выявлено увеличение амплитуды Р600 в ответ на увеличение тонального расстояния между начальной и конечной тониками, т.е., в ответ на увеличение степени модуляции. Это позволяет предположить, что амплитуда Р600 связана с уровнем нарушения тональных ожиданий, что, в свою очередь, связано с возрастанием сложности при мысленной переориентации тональной схемы с начального тонального центра на новый тональный центр.

This paper presents a way of doing large-scale audio understanding without traditional state-of-the-art neural architectures. Ever since the introduction of deep learning for understanding audio signals in the past decade, convolutional architectures have been able to achieve state-of-the-art results surpassing traditional hand-crafted features. In the recent past, there has been a similar shift away from traditional convolutional and recurrent neural networks towards purely end-toend Transformer architectures. We, in this work, explore an approach, based on the Bag-of-Words model. Our approach does not have any convolutions, recurrence, attention, transformers, or other approaches such as BERT. We showcase an approach going against the mainstream research at the moment. We utilize micro and macro-level clustered vanilla embeddings and use an MLP head for classification. We only use feed-forward encoderdecoder models to get the bottlenecks of spectral envelops, spectral patches, and slices as well as multi-resolution spectra. A classification head (a feed-forward layer), similar to the approach in SimCLR is trained on a learned representation. Using simple codes learned on them, we show how we surpass traditional convolutional neural network architectures, and come strikingly close to outperforming powerful Transformer architectures. Our approach could have been carried out back in 2006, using the vanilla autoencoder and dimension reduction architectures, as proposed by Hinton et. al. [1]. We show that just by using this architecture, and doing simple statistics on the latent representations, we could have outperformed state of the art architectures audio understanding, like convolutional and recurrent architecture as late as 2018 [2]. This goal of this work is, hopefully to would pave way for exciting advancements in the field of representation learning without massive, end-to-end neural architectures.

In existing production plants, sensor systems and other sources provide information about the plant condition. This paper presents methods for how data can be conveniently summarized, treated, and evaluated to retain characteristic features and allocate them to certain faults respectively to use them for monitoring purposes. This work details the development of a method to be applied to selected data sets, and which then can be expanded for use in the real environment. This paper details a procedure developed for automated selection and processing to reduce the time exposure of qualified personnel. A number of possible methods of analysis were tested for their ability to point out conspicuous events, especially Wavelet Transformation for feature extraction and Support Vector Machines for classification. Data sets that are correlated to the different conditions of the system are used for training and testing. After training, the algorithm will be able to detect different faults in time. In today's practice, faults are analyzed after they have occurred. Applying the method, a major failure can be prevented by detecting contingency faults. Using real industrial data from the hot strip mill of ThyssenKrupp Steel Europe (TKSE), the developed approach will be tested offline for practical relevance.

The need for loudness compensation is a well known fact arising from the nonlinear behavior of human sound perception. Music and other sounds are mixed and mastered at a certain loudness level, usually louder than the level at which they are commonly played. This implies a change in the perceived spectral balance of the sound, which is largest in the low-frequency range. As the volume setting in music playing is decreased, a loudness compensation filter can be used to boost the bass appropriately, so that the low frequencies are still heard well and the perceived spectral balance is preserved. The present paper proposes a loudness compensation function derived from the standard equal-loudness-level contours and its implementation via a digital first-order shelving filter. Results of a formal listening test validate the accuracy of the proposed method.

Smartphones, wearables, and Internet of Things (IoT) devices produce a wealth of data that cannot be accumulated in a centralized repository for learning supervised models due to privacy, bandwidth limitations, and the prohibitive cost of annotations. Federated learning provides a compelling framework for learning models from decentralized data, but conventionally, it assumes the availability of labeled samples, whereas on-device data are generally either unlabeled or cannot be annotated readily through user interaction. To address these issues, we propose a self-supervised approach termed scalogramsignal correspondence learning based on wavelet transform to learn useful representations from unlabeled sensor inputs, such as electroencephalography, blood volume pulse, accelerometer, and WiFi channel state information. Our auxiliary task requires a deep temporal neural network to determine if a given pair of a signal and its complementary viewpoint (i.e., a scalogram generated with a wavelet transform) align with each other or not through optimizing a contrastive objective. We extensively assess the quality of learned features with our multi-view strategy on diverse public datasets, achieving strong performance in all domains. We demonstrate the effectiveness of representations learned from an unlabeled input collection on downstream tasks with training a linear classifier over pretrained network, usefulness in low-data regime, transfer learning, and cross-validation. Our methodology achieves competitive performance with fullysupervised networks, and it outperforms pre-training with autoencoders in both central and federated contexts. Notably, it improves the generalization in a semi-supervised setting as it reduces the volume of labeled data required through leveraging self-supervised learning.

The SepFormer architecture shows very good results in speech separation. Like other learned-encoder models, it uses short frames, as they have been shown to obtain better performance in these cases. This results in a large number of frames at the input, which is problematic; since the SepFormer is transformerbased, its computational complexity drastically increases with longer sequences. In this paper, we employ the SepFormer in a speech enhancement task and show that by replacing the learned-encoder features with a magnitude short-time Fourier transform (STFT) representation, we can use long frames without compromising perceptual enhancement performance. We obtained equivalent quality and intelligibility evaluation scores while reducing the number of operations by a factor of approximately 8 for a 10-second utterance.

Саундскейпы (звуковые ландшафты) города как культурно-символические пространства, опосредующие наше восприятие звуков и шумов, динамичны, изменчивы и неоднородны. Специфика повседневной жизни, культурный опыт, потребляемые аудиовизуальные тексты намечают для горожан различные траектории взаимодействия с аудиальным миром и делают границу между звуком и шумом довольно подвижной. В  данной статье представлены важные для понимания процесса формирования городских саундскейпов этнографические данные, полученные в  рамках исследования взаимодействия горожан со звуками и шумами Санкт-Петербурга. Исследование было проведено авторами в 2016–2021 гг. В нем продемонстрированы возможности использования таких методов, как «дневник звука» и «звуковая прогулка». Представлен анализ около 380 часов наблюдений, сделанных 38 горожанами, и материалы 19 интервьюпрогулок с петербуржцами, проживающими в 15 различных микрорайонах. Важным результатом стало выявление особенностей категоризации звуков и  шумов в  городских саундскейпах, а также роли аудиального опыта в формировании режимов интеракции жителей Петербурга с городской средой. Исследование дает инструменты для анализа социальной и культурной надстройки восприятия городских звуков и шумов, что может представлять интерес для социологов, урбанистов, городских антропологов, экологов и исследователей локальных сообществ.

Considers a creation task of program and hardware components of generators of accidental and psevdoaccidental numbers, built in information defense systems, describe the componental generators models of accidental and psevdoaccidental numbers, leads a realization example by model element.Рассматривается задача создания программных и аппаратных компонентов генераторов случайных и псевдослучайных чисел, встраиваемых в системы защиты информации, описываются компонентные модели генераторов случайных и псевдослучайных чисел, приводится пример реализации элементов модели

Recent general-purpose audio representations show state-of-the-art performance on various audio tasks. These representations are pre-trained by self-supervised learning methods that create training signals from the input. For example, typical audio contrastive learning uses temporal relationships among input sounds to create training signals, whereas some methods use a difference among input views created by data augmentations. However, these training signals do not provide information derived from the intact input sound, which we think is suboptimal for learning representation that describes the input as it is. In this paper, we seek to learn audio representations from the input itself as supervision using a pretext task of auto-encoding of masked spectrogram patches, Masked Spectrogram Modeling (MSM, a variant of Masked Image Modeling applied to audio spectrogram). To implement MSM, we use Masked Autoencoders (MAE), an image self-supervised learning method. MAE learns to efficiently encode the small number of visible patches into latent representations to carry essential information for reconstructing a large number of masked patches. While training, MAE minimizes the reconstruction error, which uses the input as training signal, consequently achieving our goal. We conducted experiments on our MSM using MAE (MSM-MAE) models under the evaluation benchmark of the HEAR 2021 NeurIPS Challenge. Our MSM-MAE models outperformed the HEAR 2021 Challenge results on seven out of 15 tasks (e.g., accuracies of 73.4% on CREMA-D and 85.8% on LibriCount), while showing top performance on other tasks where specialized models perform better. We also investigate how the design choices of MSM-MAE impact the performance and conduct qualitative analysis of visualization outcomes to gain an understanding of learned representations. We have made our code available online for further improvements and applications of the MSM framework.

This paper proposes a robust deep learning framework used for classifying anomaly of respiratory cycles. Initially, our framework starts with front-end feature extraction step. This step aims to transform the respiratory input sound into a two-dimensional spectrogram where both spectral and temporal features are well presented. Next, an ensemble of C-DNN and Autoencoder networks is then applied to classify into four categories of respiratory anomaly cycles. In this work, we conducted experiments over 2017 Internal Conference on Biomedical Health Informatics (ICBHI) benchmark dataset. As a result, we achieve competitive performances with ICBHI average score of 0.49, ICBHI harmonic score of 0.42.

Representations in the auditory cortex might be based on mechanisms similar to the visual ventral stream; modules for building invariance to transformations and multiple layers for compositionality and selectivity. In this paper we propose the use of such computational modules for extracting invariant and discriminative audio representations. Building on a theory of invariance in hierarchical architectures, we propose a novel, mid-level representation for acoustical signals, using the empirical distributions of projections on a set of templates and their transformations. Under the assumption that, by construction, this dictionary of templates is composed from similar classes, and samples the orbit of variance-inducing signal transformations (such as shift and scale), the resulting signature is theoretically guaranteed to be unique, invariant to transformations and stable to deformations. Modules of projection and pooling can then constitute layers of deep networks, for learning composite representations. We present the main theoretical and computational aspects of a framework for unsupervised learning of invariant audio representations, empirically evaluated on music genre classification.

In this work, we use a deep convolutional neural network (DCNN) trained with a public dataset, the Million Song Dataset, as a feature extractor. We trained the network from audio mel-spectrogram using artist labels in a discriminative manner. In particular, we used a large number of neurons in the output layer where each neuron represents an artist label. The output of the last hidden layer of the DCNN is regarded as an identity feature of the input data. The DCNN extracts feature vectors by taking 3-second audio segments as input, and summarizes them as an musical feature for the input audio. These extracted features are used for training a Support Vector Machine (SVM) classifier to perform MIREX audio classification tasks such as genre or mood classification. The results show that the proposed approach effectively captures general music audio features.

The increasing amount of online videos brings several opportunities for training self-supervised neural networks. The creation of large scale datasets of videos such as the YouTube-8M allows us to deal with this large amount of data in manageable way. In this work, we find new ways of exploiting this dataset by taking advantage of the multi-modal information it provides. By means of a neural network, we are able to create links between audio and visual documents, by projecting them into a common region of the feature space, obtaining joint audiovisual embeddings. These links are used to retrieve audio samples that fit well to a given silent video, and also to retrieve images that match a given a query audio. The results in terms of Recall@K obtained over a subset of YouTube-8M videos show the potential of this unsupervised approach for cross-modal feature learning. We train embeddings for both scales and assess their quality in a retrieval problem, formulated as using the feature extracted from one modality to retrieve the most similar videos based on the features computed in the other modality.