Physics

Recent research in automated learning has focused on algorithms that learn from a combination of tagged and untagged data. Such algorithms can be referred to as semi-supervised in contrast to unsupervised, which refers to algorithms requiring no tagged data whatsoever. This paper presents a Bayesian approach to semi-supervised learning. In this approach, the parameters of a probability model are estimated using Bayesian techniques and then used to perform classification. The prior probability distribution is formulated from the tagged data via a process akin to stochastic generalization. Intuitively, the generalization process starts with a small amount of tagged data and adds to it new pseudo-counts that are similar to those that would be expected in a larger data sample from the same population. The prior distribution together with the untagged data form the posterior distribution which is used to estimate the model parameters via the EM algorithm. This procedure is demonstrated by applying it to the task of word-sense disambiguation. When priors are formulated from as few as 15 randomly selected tagged instances, the resulting classifier has an accuracy that is 21% higher than the accuracy of a classifier developed using no tagged data. When 700 tagged instances are used to formulate priors, the accuracy of the classifier is greater than the accuracy of a classifier developed from 2,124 tagged instances using standard supervised learning techniques. 1 F 1 and F 2 are conditionally independent given S if P (f 1j|f 2 k , si) = P (f 1j|si).

We describe the results of performing text mining on a challenging problem in natural language processing, word sense disambiguation. We compare two methods of unsupervised learning, Ward's minimum {variance clustering and the EM algorithm, that distinguish the meaning of an ambiguous word based only on features that can be automatically identi ed in text. This is a signi cant advantage over most previous approaches which require a training sample where the meanings of ambiguous words have been ...

This paper presents an empirical comparison of a variety of model evaluation criteria used in backwards sequential search (BSS). Both information criteria (IC) and signi cance tests are compared when applied to the problem of word sense disambiguation. We present experimental results that show the evaluation criteria that selects the most accurate models for classi cation. When the degrees of freedom of the model are adjusted the Log-likelihood ratio G2 and Akaike's Information Criteria (AIC) select the most accurate models ...

The Naive Mix is a new supervised learning algorithm that is based on a sequential method for selecting probabilistic models. The usual objective of model selection is to find a single model that adequately characterizes the data in a training sample. However, during model selection a sequence of models is generated that consists of the best-fitting model at each level of model complexity. The Naive Mix utilizes this sequence of models to define a probabilistic model which is then used as a probabilistic classifier to perform word-sense disambiguation. The models in this sequence are restricted to the class of decomposable log-linear models. This class of models offers a number of computational advantages. Experiments disambiguating twelve different words show that a Naive Mix formulated with a forward sequential search and Akaike's Information Criteria rivals established supervised learning algorithms such as decision trees (C4.5), rule induction (CN2) and nearest-neighbor classification (PEBLS).

The Naive Mix is a new supervised learning algorithm that is based on a sequential method for selecting probabilistic models. The usual objective of model selection is to find a single model that adequately characterizes the data in a training sample. However, during model selection a sequence of models is generated that consists of the best-fitting model at each level of model complexity. The Naive Mix utilizes this sequence of models to define a probabilistic model which is then used as a probabilistic classifier to perform word-sense disambiguation. The models in this sequence are restricted to the class of decomposable log-linear models. This class of models offers a number of computational advantages. Experiments disambiguating twelve different words show that a Naive Mix formulated with a forward sequential search and Akaike's Information Criteria rivals established supervised learning algorithms such as decision trees (C4.5), rule induction (CN2) and nearest-neighbor classification (PEBLS).

bruce~cs, unca. edu We present a corpus-based approach to word-sense disambiguation that only requires information that can be automatically extracted from untagged text. We use unsupervised techniques to estimate the parameters of a model describing the conditional distribution of the sense group given the known contextual features. Both the EM algorithm and Gibbs Sampling are evaluated to determine which is most appropriate for our data. We compare their disambiguation accuracy in an experiment with thirteen different words and three feature sets. Gibbs Sampling results in small but consistent improvement in disambiguation accuracy over the EM algorithm.

The past three decades have seen a steady growth of interest in corpus-based techniques for speech and natural language processing. Beginning with the success of the early Markov chain-based speech recognition systems (Jelinek 1998) and continuing with the work done in information extraction as part of the Message Understanding Conferences (http://www.muc.saic.com), corpus-based speech and language processing have flourished. Such techniques are widely considered to be the primary means of developing broad-coverage speech and language systems that can be ported to different domains. As such, the range of corpus-based techniques and their applications have expanded greatly in the past decade.

Lost-PLA casting is a digital adaption of a classic sculpture technique-lost-wax casting. Mesopotamian lost-wax cast artifacts date back to approximately 3500 BC [Hunt 1980]. At that time, wax carvings were encased in clay which was fired to produce molds. The earliest casts were copper, but over time other metals were used. Our production process remains remarkably similar to the ancient casting process, except for material substitutions: 1) ceramic slurries instead of clay, 2) PLA 3D prints instead of wax, and 3) bronze or aluminum instead of copper.

This paper describes measures for evaluating the three determinants of how well a probabilistic classi er performs on a given test set. These determinants are the appropriateness, for the test set, of the results of (1) feature selection, (2) formulation of the parametric form of the model, and (3) parameter estimation. These are part of any model formulation procedure, even if not broken out as separate steps, so the tradeo s explored in this paper are relevant to a wide variety of methods. The measures are demonstrated in a large experiment, in which they are used to analyze the results of roughly 300 classi ers that perform word-sense disambiguation.