Proceedings of the 11th ACM Multimedia Systems Conference, 2020
The forensic investigation of a terrorist attack poses a significant challenge to the investigati... more The forensic investigation of a terrorist attack poses a significant challenge to the investigative authorities, as often several thousand hours of video footage must be viewed. Large scale Video Analytic Platforms (VAP) assist law enforcement agencies (LEA) in identifying suspects and securing evidence. Current platforms focus primarily on the integration of different computer vision methods and thus are restricted to a single modality. We present a video analytic platform that integrates visual and audio analytic modules and fuses information from surveillance cameras and video uploads from eyewitnesses. Videos are analyzed according their acoustic and visual content. Specifically, Audio Event Detection is applied to index the content according to attack-specific acoustic concepts. Audio similarity search is utilized to identify similar video sequences recorded from different perspectives. Visual object detection and tracking are used to index the content according to relevant concepts. Innovative user-interface concepts are introduced to harness the full potential of the heterogeneous results of the analytical modules, allowing investigators to more quickly follow-up on leads and eyewitness reports.
Europeana gives access to data from Galleries, Libraries, Archives and Museums across Europe. Sem... more Europeana gives access to data from Galleries, Libraries, Archives and Museums across Europe. Semantic and multilingual diversity as well as the variable quality of our metadata makes it difficult to create a digital library offering end-user services such as multilingual search. To palliate this, we build an “Entity Collection”, a knowledge graph that holds data about entities (places, people, concepts and organizations) bringing context to the cultural heritage objects. The diversity and heterogeneity of our metadata has encouraged us to re-use and combine third-party data instead of relying only on those contributed by our own providers. This raises however a number of design issues. This paper lists the most important of these and describes our choices for tackling them using Linked Data and Semantic Web approaches.
Proceedings of the 35th Annual ACM Symposium on Applied Computing
We present an approach to unsupervised audio representation learning. Based on a triplet neural n... more We present an approach to unsupervised audio representation learning. Based on a triplet neural network architecture, we harnesses semantically related cross-modal information to estimate audio track-relatedness. By applying Latent Semantic Indexing (LSI) we embed corresponding textual information into a latent vector space from which we derive track relatedness for online triplet selection. This LSI topic modelling facilitates fine-grained selection of similar and dissimilar audio-track pairs to learn the audio representation using a Convolution Recurrent Neural Network (CRNN). By this we directly project the semantic context of the unstructured text modality onto the learned representation space of the audio modality without deriving structured ground-truth annotations from it. We evaluate our approach on the Europeana Sounds collection and show how to improve search in digital audio libraries by harnessing the multilingual meta-data provided by numerous European digital libraries. We show that our approach is invariant to the variety of annotation styles as well as to the different languages of this collection. The learned representations perform comparable to the baseline of handcrafted features, respectively exceeding this baseline in similarity retrieval precision at higher cutoffs with only 15% of the baseline's feature vector length. CCS CONCEPTS • Information systems → Content analysis and feature selection; Multilingual and cross-lingual retrieval; Speech / audio search;
Libellarium: journal for the research of writing, books, and cultural heritage institutions
This paper presents a method to facilitate decision-making for the preservation of digital conten... more This paper presents a method to facilitate decision-making for the preservation of digital content in libraries and archives using institutional risk profiles that highlight endangered files formats (in danger of becoming inaccessible or unusable). The primary contribution of this work is the combined use of both machine-mined data and human-expert input to select and configure institution-specific preservation risk profiles. The machine-mined data used the developed File Format Metadata Aggregator (FFMA), and the crowdsourced expert input was collected via two surveys of digital preservation practitioners. A by-product of this endeavor is the ability to visualize risk factors for analysis. The underlying decision support system used the Cosine Similarity algorithm to provide recommendations for matching risk profiles to selected institutional risk settings. This method improves the interpretability of risk factor values and the quality of a digital preservation process. The aggregated information about the risk factors is presented as a multidimensional vector that shows a particular analysis focus and its resulting impact on selected file formats. Sample risk profile calculations and the visualization of risk factor dimensions are shared in the evaluation section.
Recommendation has a long history as a successful application area of Artificial Intelligence. Th... more Recommendation has a long history as a successful application area of Artificial Intelligence. The demand of e-commerce platforms (e.g., amazon.com) to improve the accessibility of large product-and service assortments contributed to an increased popularity of recommendation technologies. Three basic technologies supporting the personalized recommendation of products and services are presented in this paper. In order to take into account the focus of this special issue, we provide a discussion of the application of those technologies in the tourism domain (e.g., recommendation of travel destinations) with a special focus on mobile environments. Recommendation Technologies The increasing size and complexity of product assortments offered by e-commerce platforms requires appropriate technologies which alleviate the retrieval of products by online customers. Different recommendation technologies have been developed to help customers to easily find the best matching product. Those technologies have been successfully applied in different domains such as financial services, electronic goods, or movies. An overview of applications exploiting recommender technologies can be found in [16]. The most widespread technology is collaborative filtering (CF), which exploits user ratings of products in order to identify additional products that the active user may like as well [6]. User-based and item-based collaborative filtering are two basic variants of this technology. As shown in Figure 1, both variants are predicting to which extend the active user (in this case User3) would like currently unrated items. User-based approaches to collaborative filtering try to identify the k nearest neighbours of the active user (users having similar tastes), and calculate a prediction of the active user's rating for a specific item. This rating can be defined, for example, as the weighted average of the k nearest neighbours' ratings [6]. In the simplified example of Figure 1, User1 is found to be the nearest neighbour (k=1) of User3 (the active user) and his/her rating for the 4 th product ('Conspiracy Theory') will be taken as prediction for the rating of User3 (rate=2). In contrast, item-based collaborative filtering is searching for items which received similar ratings from other users and were also (positively) rated by the active user. In the example
Osteoporosis is a disease in which the density and quality of bone are reduced, leading to a weak... more Osteoporosis is a disease in which the density and quality of bone are reduced, leading to a weakness of the skeleton and increased risk of fracture, particularly of the spine, wrist, hip, pelvis and upper arm. The goal of our work is to find a reliable method to diagnose the osteoporosis starting from the examination of the X-Ray images of bones. Bi-dimensional moments describe some properties of an object from an image like the area and the center of gravity, and they give us information about the distribution of the gray levels in digital images, which can help us to classify the degree of similarity between two images. The use of thresholding and binarizing techniques for extracting information from the image is necessary. After these operations we developed two methods, one of them using the moments and threshold images which keep the most significant N pixels in the image, and the other using binary images with the pixels which are greater then a percent p% from the greatest v...
Computing Recommendations for Long Term Data Accessibility basing on Open Knowledge and Linked Data
Digital access to our cultural heritage assets was facilitated through the rapid development of t... more Digital access to our cultural heritage assets was facilitated through the rapid development of the digitization process and online publishing initiatives as Europeana or the Google books project. As Galleries, Libraries, Archiving institutions and Museums (GLAM) created digital representations of their masterpieces new concerns arise regarding the longterm accessibility of digitized and digitally born content. Repository managers of institutions need to take well-documented decisions with regard to which digital object representations to use for archiving or long term access to their valuable collectionst. The digital preservation recommender system presented within this paper aims at reducing the complexity in the process of decision making by providing support for classi�cation and the preservation riskanalysis of digital objects. Technical information which is available as linked data in open knowledge sources facilitates the construction of the DiPRec's recommender knowledg...
A Model for Format Endangerment Analysis using Fuzzy Logic
This paper presents an approach for merging information automatically aggregated from open reposi... more This paper presents an approach for merging information automatically aggregated from open repositories and expert knowledge related to digital preservation. The main contribution of this work is the employment of fuzzy models to support digital preservation experts with semi-automatic estimation of " endangerment level " for file formats. Our goal is to make use of a solid knowledge base automatically aggre-gated from linked open data repositories to detect conflicts and inaccuracies in this data in order to improve the quality of a risk analysis process. The proposed method is meant to facilitate decision making with regard to preservation of digital content in libraries and archives using knowledge of domain experts. To allow reasoning even in case of inconsistent data we employ fuzzy logic techniques for transforming information about formats in user friendly metrics. The goal is to bring conflicting and incorrect information to the surface for correction and improveme...
ECWEB2006
Image Similarity Search final
A Risk Analysis of File Formats for Preservation Planning
Aggregating a Knowledge Base of File Formats from Linked Open Data
Uploads
Papers by Sergiu Gordea