EFTA01154696.pdf

AnalogySpace: Reducing the Dimensionality of Common Sense Knowledge Robert Speer Catherine Havasi Henry Lieberman CSAIL Laboratory for Linguistics and Computation Software Agents Group Massachusetts Institute of Technology Brandeis University MIT Media Lab Ca A Abstract to reduce the dimensionality of that matrix. This results in computing principal components which represent the most We are interested in the problem of reasoning over very large common sense knowledge bases. When such a knowledge salient aspects of the knowledge, which can then be used to base contains noisy and subjective data, it is important to organize it along the most semantically meaningful dimen- have a method for making rough conclusions based on simi- sions. The key idea is that semantic similarity can be deter- larities and tendencies, rather than absolute with. We present mined using linear operations over the resulting vectors. AnalogySpace. which accomplishes this by forming the ana- logical closure of a semantic network through dimensionality What AnalogySpace Can Do reduction. It self-organizes concepts around dimensions that can be seen as making distinctions such as "good vs. bad" AnalogySpace provides a computationally efficient way to or "easy vs. hard", and generalizes its knowledge by judging calculate a wide variety of semantically meaningful opera- where concepts lie along these dimensions. An evaluation tions: demonstrates that users often agree with the predicted knowl- AnalogySpace can generalize from sparsely-collected edge, and that its accuracy is an improvement over previous knowledge. The amount of common sense knowledge techniques. needed in everyday life is extraordinary: estimates say that we have at least several million facts (Chklovski 2003) at our Introduction disposal. It has been theorized that much of semantic mean- ing is generative (Pustejovsky 1998), and thus we believe This paper introduces AnalogySpace, a new technique de- that people are likely to generalize some of their common signed to facilitate reasoning over a large knowledge base sense information by creating an analogy to existing knowl- of natural language assertions that represent common sense edge. It is important, then, for a knowledge base to be able knowledge. Reasoning about common sense poses some to be generalized in the same way, and dimensionality re- unique challenges. Unlike traditional mathematical logic, duction gives us a way to do so by inferring new features for common sense knowledge is often imprecise and inconsis- concepts in a knowledge base. tent, causing problems for traditional proof procedures. We AnalogySpace can classify information in a knowledge are often less interested in determining the absolute truth of base in a variety of ways. The reduced-dimensionality space a factual proposition, as we are in computing somewhat in- that results from principal components analysis (PCA) is the tangible qualities such as context, connotations, tendencies, one that best describes the variations in the knowledge. Dif- analogies, and patterns of similar relationships. ferent vectors in this space represent different ways of mak- AnalogySpace represents a new synthesis between sym- ing binary distinctions among the concepts in ConceptNet, bolic reasoning techniques and statistical methods. Like by projecting all of the concepts onto one line. symbolic reasoning, the source material for our reasoning Interesting distinctions come from vectors that align with is a set of assertions, which in our case are expressed in nat- a meaningful cluster of concepts, and some of them are sug- ural language or a derivative representation, rather than in gested by the principal components themselves. For exam- logic. AnalogySpace learns from this source data using a ple, we can observe that the most significant component of technique similar to the information retrieval technique of AnalogySpace distinguishes things people want from things latent semantic analysis (LSA), because of its ability to con- people do not want. Concepts that fall in the "people want" struct large-scale patterns out of a myriad small of bits of direction include love, money, and vacation, while the oppo- evidence. Unlike traditional LSA, however, AnalogySpace site direction includes lose your keys and slavery. By pro- works with semantically strong assertions, rather than the jecting concepts onto other vectors, we can make other dis- weak semantics of word co-occurrence in documents. tinctions such as "urban vs. rural", "animate vs. inanimate", AnalogySpace, using data from the Open Mind Common and "indoors vs. outdoors". Sense (OMCS) project, represents knowledge as a matrix of AnalogySpace can create "ad hoc categories" and pro- objects or concepts along one axis, and features of those ob- vide common sense justifications for why things belong to jects along another, yielding a sparse matrix of very high di- those categories. Our notion of generalized similarity makes mension. We then use singular value decomposition (SVD) it easy to extend a category using a few examples. For exam- Copyright 2008. Association for the Advancement of Artificial ple, typing "knife, fork, spoon" into our categorizer will re- Intelligence (www.aaai.org). All rights reserved. sult in a category that also includes "plate" and "chopsticks". EFTA01154696 AnalogySpace can also list the common sense features that oestmeirt aim justify the classification, such as "found in the kitchen" and "used to eat food". AnalogySpace can confirm or question existing knowl- edge. AnalogySpace can assign an inference score to any assertion, including assertions that already exist in the database. This determines which assertions are well sup- ported by other knowledge, and which appear dubious and should be checked by humans. This procedure has been used to identify bad assertions in ConceptNet such as "Books taste good". Figure I: An illustration of a small section of ConceptNet. Common Sense Computing To improve computers' understanding of the world and the way they interact with users, we must teach them about Learner and Cumulative Analogy the concepts and relationships which underlie everything we Tim Chklovski used knowledge extracted from Open Mind know and talk about. Common sense knowledge is the ba- to create the Learner system (Chklovski 2003). With sis of this network of information, expressing the relations Learner, he introduced the idea of reasoning about common between concepts that we take for granted. However, this sense by "cumulative analogy", a form of inference by in- knowledge can be difficult to capture, because much of it is duction. considered so obvious that it is rarely mentioned in corpora. First, statements of common sense knowledge are divided into objects (analogous to ConceptNet's concepts) and fea- The Open Mind Common Sense Project tures, which are descriptions of objects that complete a state- In 2000, what would become the Common Sense Computing ment about them, such as "is a part of a car". The similarity Initiative started the Open Mind Common Sense (OMCS) between two objects is defined as the number of features project, asking volunteers on the intemet to enter common they have in common. The analogy step, then, is to hypoth- sense information through a website (Singh et at 2002). esize that features that hold for one object also hold for simi- OMCS has collected over 700,000 pieces of common sense lar objects. If many of an object's nearest neighbors by sim- information in English alone from 15,000 contributors, and ilarity have a certain feature, this creates a strong inference has expanded to several other languages. that that object has the feature as well. Let us suppose, for the sake of example, that such a sys- ConceptNet tem doesn't know much about cats. To begin with, it knows that "a cat is a pet", "a cat has fur", and "a cat has a tail". Many kinds of statements that users have entered into Open It knows slightly more about dogs, including "a dog is a Mind can be expressed as relationships between two con- pet", "a dog has fur", "a dog has a tail", and "a dog has cepts, which are essentially short phrases of natural lan- four legs". Based on the three existing similarities in its guage. This fact was used to create ConceptNet, the seman- knowledge about cats and dogs, it could transfer the "has tic network representation of the OMCS knowledge base four legs" feature from the dog concept to the cat concept, (Havasi, Speer, & Alonso 2007; Liu & Singh 2004). Con- correctly inferring that "a cat has four legs". ceptNet is made up of a network of concepts connected by Cumulative analogy is the ideological predencsor to one of about twenty relations such as "IsN', "PartOf", or AnalogySpace, which uses a vector space of potential analo- "UsedFor", which are labeled as expressing positive or neg- gies to add more power, efficiency and resistance to noise to ative information using a polarity flag. The relations are the process. based on the most common types of knowledge entered into the OMCS database, both through free text entry and semi- AnalogySpace structured entry. For the assertion "A trunk is part of a car", for instance, the two concepts are "trunk" and "car", the re- The Motivation for Dimensionality Reduction lation is "PartOr, and the polarity is positive. For the asser- In order to draw new conclusions from analogies, we need tion "People don't want to be hurt", the concepts are "per- to be able to identify similarities between concepts in Con- son" and "hurt", the relation is "Desires", and the polarity is ceptNet, with a technique that retains its effectiveness as the negative. number of concepts and features increases. Concepts that The current version of ConceptNet contains over 250,000 share features that are themselves similar, but not identical, assertions. 3.4% of these assertions have negative polarity. are missed by the basic process of cumulative analogy. This ConceptNet is useful because it provides applications motivates us to use truncated singular value decomposition with a connection between natural language text and an un- to discover similarities, which simultaneously reduces the derstanding of the world (Lieberman et al. 2004). A spe- dimensionality of our data and generalizes the notion of sim- cific example of its use is to improve the accuracy of speech ilarity to one that is less brittle. recognition (Lieberman et at 2005) by selecting words that The crucial observation to make is that similarity is a lin- make sense in context. ear operation over vectors. This makes it possible to gen- EFTA01154697  o.l ' pcfsontopatiearhman'CapstleCe sionality (as many dimensions as there are features) and are l'... " 58lIce wk. difficult to work with. 'am cica nom* The process of truncated singular value decomposition thecIFoolunic° osich a okttool 0..06 lets us approximate these dot products in a useful way. A ns truncated SVD projects all of the concepts from the space use a ithigrtlSepaaa, of features into a space with many fewer dimensions. It ha?t, h nand %CS get 14.411 also performs the dual operation, projecting features from a nemaDevis2 o 01000y space of concepts into the same reduced-dimensional space. ed.. Non 10100d We can now run the cumulative analogy process in this new, Yale '00(katente ' more compact space of concepts and features, which we call pl AnalogySpace. 'wen, on Alto 'mar. 'mak on woo/ breathe wok. Building AnalogySpace The principle of singular value decomposition (Wall, Becht- stainer, & Rocha 2003) is that any matrix A can be factored Detintbaty into an orthonormal matrix U, a diagonal matrix £, and an orthonormal matrix VT. so that A = U£VT. The singular Figure 2: AnalogySpace projected onto its first two compo- values in £ are ordered from largest to smallest, where the nents, with some concepts and features labeled. We inter- larger values correspond to the vectors in U and V that are pret these components as representing the notions of "desir- more significant components of the initial A matrix. ability" and - feasibility". The features "Person/Desires" and We discard all but the first k components — the princi- "Person/CapableOr have very large values on these respec- pal components of A — resulting in the smaller matrices tive axes, indicated by arrows pointing off the graph. Uk, £k, and VI. The components that are discarded repre- sent relatively small variations in the data, and the principal components form a good approximation to the original data. This is called a truncated SVD, representing the approxima- eralize and optimize the cumulative analogy process using tion A Ak = UkEklc. singular value decomposition, the linear algebra technique As AnalogySpace is an orthogonal transformation of used in principal component analysis and latent semantic in- the original concept and feature spaces, dot products in ference. AnalogySpace approximate dot products in the original As in Learner, we define features as the complement of spaces. This fact can be used to compute similarity between concepts. Because assertions relate two concepts, each as- concepts or between features in AnalogySpace. We call the sertion can be decomposed into a concept and a feature in result generalized similarity. two ways, by focusing on the concept from each side. For Concepts that ought to be similar, but share no exact fea- example, the assertion which states that "a trunk is part tures, get an ordinary similarity value of 0, but their gener- of a car" applies the feature (PartOf, "car") to the concept alized similarity can be a positive number. So generalized "trunk", and also applies the feature ("trunk", PartOf) to the similarity allows a useful similarity value to be calculated concept "car". for any two concepts, not just concepts with exact features Each concept can then be associated with a vector in the in common. space of possible features. The values of this vector are pos- The truncated SVD can be seen as a way of finding a space itive for features that produce an assertion of positive polar- of "eigenconcepts" — k independent linear combinations of ity when combined with that concept, negative for features concepts that span the k dimensions of AnalogySpace — and that produce an assertion of negative polarity, and zero when representing both concepts and features as linear combina- nothing is known about the assertion formed by combining tions of eigenconcepts. Specifically, if there are in concepts that concept with that assertion. As an example, the feature and n features, the in x n matrix representing ConceptNet vector for "steering wheel" could have +1in the position for is factored into: "is part of a car". +1 for "is round", and —1 for "is alive". • Uk, a in x k orthogonal matrix that relates concepts and (We will address more specifically what positive and nega- eigenconcepts tive values we assign later, in the Scores and Normalization section). These vectors together form a matrix whose rows • £k, ak x k diagonal matrix of singular values, which assign a weight to each eigenconcept are concepts, whose columns are features, and whose values indicate truth values of assertions. • 14, an x k orthogonal matrix that relates features and The degree of similarity between two concepts, then, is eigenconcepts. the dot product between their rows in the concept/feature A plot of the first two components of AnalogySpace, rep- matrix. The value of such a dot product increases when- resenting both U2 and V2. is shown in Figure 2. ever two concepts are described with the same feature, and decreases when they are described by features that are nega- Making Inferences using AnalogySpace tions of each other. When performed on the concept/feature In the AnalogySpace representation, concepts and features matrix, however, these dot products have very high dimen- can be directly compared, now that they are both represented EFTA01154698 in terms of eigenconcepts. In fact, if we take the dot prod- values for AnalogySpace. However, we find that too large of uct of a concept and a feature, while weighting each eigen. a value of k makes inference run slower while generalizing concept by the singular value that represents how significant less, so we typically use k = 50 or k = 100. A value of it is, we simply get the defining equation for the truncated k = 50 was used for the evaluation in this paper. SVD: Ak = UkEkVkr - Each row of Ak corresponds to a concept, and contains a Open Mind Commons vector that assigns numerical values to its features. These The inferences produced by AnalogySpace have an im- values become larger than they were in the original matrix mediate purpose, completing a cycle of feedback in the A when they belong to multiple similar concepts. This is way that Open Mind learns. Open Mind Commons (Speer exactly what cumulative analogy is meant to do. In short, Ak 2007) is a knowledge collection interface that runs on top is not just an approximation, it is the closure of cumulative of ConceptNet 3, bridging the gap between the computer- analogy over ConceptNet. understandable knowledge and contributors. When contrib- Another way to view the truncated SVD is that it utors enter new statements through this interface, they im- "smooths" the somewhat noisy data in ConceptNet. Asser- mediately become part of ConceptNet. They are then used tions which do not correlate well with other assertions are to infer new possible assertions, which are presented to the discarded, while large-scale similarities between many con- user for approval. This helps make the knowledge entry pro- cepts are emphasized. Individual concepts are no longer as- cess more interactive, and aids us by encouraging users to sociated with a complete list of their features; instead, they enter assertions whose answers will fill gaps in ConceptNet. are blurred together somewhat with similar concepts. We take advantage of this smoothing effect to make inferences Related Work in AnalogySpace. Other Methods for Acquiring Common Sense Scores and Normalization Some projects aim to extract general knowledge from exist- Every existing assertion in ConceptNet has an integer con- ing corpora. Suh, Halpin, & Klein (2006) use named en- fidence score, which is initially I. This score is automati- tity recognition and noun and verb group detection to ex- cally increased when multiple users enter the same assertion tract common sense statements from Wikipedia, and (Eslick and decreased when users enter contradictory assertions, and 2006) has used data mining techniques to extract common users of the Web site can increment or decrement the score sense from websites on the Internet. The KNEXT project of an assertion by I. Assertions with zero or negative confi- (Schubert 2002) uses patterns to extract semantic relation- dence are not used in the matrix. ships from the Penn Treebank. The knowledge collected by We combine the confidence score with the polarity when these projects is of a different nature than the common sense assigning numerical values to assertions in the matrix. As- OMCS aim to collect, because it does not include the set sertions with positive polarity are assigned their confidence of fundamental knowledge that is so "obvious" that people score as a value, while negative assertions are assigned the leave it unstated. negation of their confidence score. The Cyc project (Lenat 1995) is another attempt to collect Before computing the SVD, we normalize the rows of the common sense knowledge. Started by Doug Lenat in 1984, matrix so that a few high-information concepts such as "per- this project utilizes knowledge engineers who handcraft as- son" do not dominate all the others. We scale the rows down sertions and place them in Cyc's logical frameworks, us- by their Euclidean norm, plus a small additional term; this ing a logical representation called CycL. Cyc has tradition- makes all concepts have nearly the same magnitude, with the ally relied on trained knowledge engineers to acquire their magnitude decreasing for very low-information concepts. knowledge but has recently made efforts to acquire knowl- edge from other sources. For example, the goal of the WAC Parameters project (Coursey 2007) is to let Cyc discover patterns in its Because it is parsed from free text, ConceptNet contains own data using the Weka (Witten & Frank 2005) classifier. some spurious concepts that appear, for example, in only The project then uses this classifier to allow Cyc to classify one assertion. Very rare concepts make the SVD take longer whether email is spam based only on its subject line. to compute for little benefit, so we set a minimum on how many assertions a concept needs to be involved in before PCA for Semantic Relationships we represent it in AnalogySpace. We currently use a mini- The most well-known use of principal component analysis, mum of 4 assertions. (In other research, we are working on or singular value decomposition, in language is in latent se- processes that encourage Open Mind users to "bootstrap" mantic analysis (LSA), which attempts to find similarities new concepts, answering questions about them that provide in the domains of words and documents (Deerwester et at enough information to represent them in AnalogySpace.) 1990). LSA is often used in information retrieval, where it As a parameter to the SVD process, we need to specify k, is also known as latent semantic indexing. In the representa- the size of the truncated matrices — in other words, the num- tion used by LSA, a document is seen as an unordered col- ber of singular values or eigenconcepts to find. The sparse lection of words, and the matrix of words versus documents SVD implementation we use, Doug Rohde's svdlibc, can is analyzed with SVD, so that documents are sorted into im- be asked to produce all singular values it considers statisti- plicit categories according to the words that are contained in cally significant, in which case it produces over 400 singular them. EFTA01154699  Various research projects have expanded the use of SVD number of principal components as AnalogySpace (k = 50) in semantic contexts to examine broader contexts than words for each SVD. We refer to this implementation below as within documents. An approach used by Patwardhan & Ped- the "within-relations SVD". The final 25% of the asser- ersen (2006) is to replace the documents in LSA with sen- tions were nonsense, generated from random combinations tences. This reduced connections made between words or of concepts and features. concepts that are far from each other in a document. Participants rated each assertion with one of the choices Tumey (2005) creates a matrix using the co-occuring "Generally true", "Sometimes / Somewhat true", "Don't words as the rows and the context in which the words ap- know / Opinion", "Generally false", "Doesn't make sense", pear within a sentence as the columns. This project aims at and "Not true but amusing" (an option suggested by partic- creating more literal analogies: its goal is to answer the kind ipants in a previous study). Figure 3 shows the breakdown of analogy questions that would appear on a standardized of ratings that participants gave to assertions from all four test, such as "quart:volume :: mile:distance". sources. The performance of both methods of inference was The field of applying SVD to lexical resources themselves much closer to that of existing assertions than to randomly- is small so far. Banerjee & Pedersen (2003) use the co- generated assertions. The most prominent difference be- occurrence of words in WordNet glosses as a measure of tween AnalogySpace and the within-relations SVD is that semantic similarity. This measure of similarity is based only AnalogySpace produced considerably fewer assertions that on dictionary-like glosses within WordNet, rather than the were rated as untrue or nonsensical. connections in WordNet itself. Perhaps the closest thing to our application of SVD is another project descended from Eiteakdoon of ra:ings In user evaluation Open Mind Common Sense, using Honda's domain-specific 100 Open Mind Indoor Common Sense corpus. The constructed second-order SVD mapping concept to concept within an in- 80 dividual relation type, such as PartOf or AlLocation (Gupta & Kochenderfer 2004). However, this technique cannot learn by synthesizing information expressed with different 3 60 relations. Because this technique is the closest to the meth- ods we have used, we choose to evaluate our system in com- 40 parison to this algorithm. 20 Evaluation We use AnalogySpace to fill gaps in ConceptNet's common sense knowledge and to ask relevant questions to contribu- Random Wein AnabgySpace People relations tors, so our goal is for it to produce inferences that make Not Rue/ Nonsense O Don't know Opinion =1 sense to users, and which they frequently consider true. Sometimes / Sone...tat true The truth of inferences is important in unsupervised uses of Generally Due AnalogySpace, and in the context of user interaction, it as- sures users that the system is learning from their input. To evaluate the success of inference using AnalogySpace, we Figure 3: The ratings given by participants in the study to assertions from four different sources. asked users to evaluate the truth of assertions produced by AnalogySpace, as compared with assertions produced with other sources. In order to quantitatively compare these ratings, we Our subjects were 40 college students and recent college mapped the rating choices to scores, where "Generally true" graduates, most of whom had never interacted with Open was worth 2, "Sometimes / Somewhat" worth I, "Don't Mind before, taking this evaluation on a special page of the know / Opinion" worth 0, and all other options were worth Open Mind Commons web site. No compensation was of- -I. On this scale, the existing assertions in AnalogySpace fered. We presented each subject with a list of 60 assertions, received an average score of 1.315. The new assertions in- converted from assertions to English sentences using a nat- ferred by AnalogySpace scored 1.025, outperforming the ural language procedure in Open Mind Commons. within-relations SVD which scored 0.882. The random as- The assertions were produced from four sources and shuf- sertions, as expected, scored much lower at -0.644. fled together. 25% of them were existing assertions in Con- Next, we recalibrated the average ratings for each sub- ceptNet, entered by human contributors, sampled randomly ject, to take into account the fact that they would have differ- from all those with a confidence score of at least 2. An- ing interpretations of the options. We linearly mapped each other 25% were produced by AnalogySpace but did not al- user's scores to a scale where 0 represented that user's av- ready exist in ConceptNet, and were sampled with a proba- erage rating of random assertions, and 100 represented that bility function that increases with their inferred score. 25% user's average rating of assertions contributed by people. more were sampled in the same way from a modification On this scale, the within-relations SVD received a mean of AnalogySpace to emulate the previous work by Gupta & score of 78.1 and a standard deviation of 18.5, while Kochenderfer (2004), by using a separate SVD for each re- AnalogySpace received a mean score of 87.5 and a stan- lation instead of a single combined SVD, and using the same dard deviation of 22.2. These large standard deviations come EFTA01154700 from the fact that different users used the scale very differ- Deerwester. S. C.: Dumais, S. T.; Landauer. T. K.; Fumas, G. W.: ently, which the reason why we have calibrated the ratings and Harshman. R. A. 1990. Indexing by latent semantic anal- per user. Comparing the calibrated scores for each user us- ysis. Journal of the American Society of Information Science ing a paired t-test showed that the average 9A point differ- 41(6):391107. ence was statistically significant at the p < 0.05 level, with Eslick. I. 2006. Searching for Commonsense. Ph.D. Dissertation. a 95% confidence interval of (1.0, 17.8). We conclude from MIT Media Lab. this that AnalogySpace's use of features that involve many Gupta. R.. and Kochenderfer. M. J. 2004. Using statistical tech- different relations gives it additional inferential power over niques and WordNet to reason with noisy data. In Workshop on the previous state of the art. Adaptive Text Extraction and Mining. Nineteenth National Con- ference on Artificial Intelligence (AAAI-04). Discussion Havasi. C.: Speer, R.: and Alonso. J. 2007. ConceptNet 3: a flexi- ble, multilingual semantic network for common sense knowledge. By reducing the dimensionality of the sparse knowledge In Recent Advances in Natural Language Processing. in ConceptNet, AnalogySpace generalizes that knowledge Lenat D. 1995. Cyc: A large-scale investment in knowledge and makes it more broadly useful to applications. Analogy- infrastructure. Communications of the ACM 11:33-38. Space's ability to predict new assertions has already helped Lieberman. H.: Liu. H.: Singh. P.: and Barry. B. 2004. Beat- to improve the way that OMCS collects its knowledge, by ing common sense into interactive applications. Al Magazine giving its users feedback in the form of relevant inferences. 25(4):63-76. The fact that these inferences are frequently true or at least Lieberman. H.: Faaborg, A.: Daher. W.: and Espinosa, J. 2005. sensible, as shown by our user evaluation, helps to give our How to wreck a nice beach you sing calm incense. In Proceed- users confidence that the system is learning from their input. ings of the 10th International Conference on Intelligent User In- AnalogySpace can be used to accomplish other tasks tetfaces. besides filling in the gaps of ConceptNet. As men- Liu. H.. and Singh. P. 2004. ConceptNet: A practical common- tioned in the introduction, one of our example applica- sense reasoning toolkit. BT Technology Journal 22(4):211-226. tions uses the notion of generalized similarity to find Patwardhan, S., and Pedersen. T. 2006. Using WordNet-based members of ad-hoc categories. Given several phrases, context vectors to estimate the semantic relatedness of concepts. it averages together their corresponding vectors in Con- In EACL 2006 Workshop Making Sense ofSense—Bringing Com- ceptNet and returns other concepts that are sufficiently putational Linguistics and Psycholinguistics Together.I—8. similar. This technique can fill in the missing mem- Pustejovsky, J. 1998. The Generative Lexicon. Cambridge. MA: bers of a category, much like the proprietary Google Sets MIT Press. (http://labsl.google.com/sets). Schubert. L. K. 2002. Can we derive general world knowledge The combination of ConceptNet and AnalogySpace pro- from texts? In Proceedings of the Human Language Technology vides a broad base of domain-general knowledge that can Conference. benefit Al systems, particularly those that work with natural Singh. P: Lin. T.: Mueller, E. T.: Lim. G.: Perkins. T.: and Thu. language. A large corpus alone, or an inference technique W. L 2002. Open Mind Common Sense: Knowledge acqui- alone, would not accomplish this, but in combination they sition from the genera I public. In On the Move to Meaningful give access to a wealth of knowledge. Internet Systems. 2002 - DOA/Coop 1S/ODBASE 2002 Confeder- ated International Conferences DOA, CooplS and ODBASE 200 2. 1223-1237. London. UK: Springer-Verlag. Acknowledgments Speer. R. 2007. Open Mind Commons: An inquisitive approach We would like to thank James Pustejovsky for all of his help. to learning common sense. Proceedings ofthe Workshop on Com- Additional thanks go to Jason Alonso and Kenneth Arnold mon Sense andInteractive Applications. for their work on Divisi, the software package which can Suh, S.: Halpin, H.: and Klein, E. 2006. Extracting common sense be downloaded for use in working with AnalogySpace. The knowledge from Wikipedia. In Proc. of the ISIVC2006 Workshop Common Sense Computing Initiative would like to thank the on Web Content Mining with Human Language technology. tens of thousands of users whose contributions have made Tumey. P. D. 2005. Measuring semantic similarity by latent rela- the Open Mind project possible, as well as the many people tional analysis. who volunteered to take our evaluation. Wall. M. E: Rechtsteiner. A.: and Rocha. L. M. 2003. A Practical Approach to Microarray Data Analysis. Nonvell, MA: Kluwel. References chapter 5,91-109. Banerjee. S.. and Pedersen. T. 2003. Extended gloss overlaps as Witten. I. H.. and Frank. E. 2005. Data Mining: Practical Ma- a measure of semantic relatedness. In "Eighteenth International chine Learning Tools and Techniques. Morgan Kaufmann Series Joint Conference on Artificial Intelligence". in Data Management Systems. Morgan Kaufmann. second edi- Chklovski, T. 2003. Learner: a system for acquiring common- tion. sense knowledge by analogy. In K-CAP Proceedings of the 2ndInternational Conference on Knowledge Capture, 4-12. New York. NY. USA: ACM Press. Coursey, K. 2007. WAC: Weka and Cyc: Teaching Cyc to learn through self-recursive data mining. In Proceedings of the Work- shop on Common Sense and Intelligent User Interfaces. EFTA01154701