EFTA02489347.pdf

From: jeffrey E. <[email protected]> Sent: Monday, September 7, 2015 12:43 AM To: Ben Goertzel Subject: Re: hilbert did questions ? im open for ideas =AO looking to encourage work based on the concept of coherence. =AO or sense making modules.. babys learn la=guage. many of them . figure out how the baby does it. On Sun, Sep 6, =015 at 8:16 PM, Ben Goertzel > wrote: Hi, I agree w/ Joscha's caution about discrimination tasks: They can be often be solved rather well, but in devious ways, by statistical supervised learning algorithms. Suppose you pose a linguistic discrimination task of some sort -- and a supervised learning algorithm, trained on a mass of data, can solve it with 97% accuracy. The algorithm's pattern of errors may indicate to YOU, intuitively, that it doesn't really understand what's going on. But then, =t may be that the average person solves the task with only 95% accuracy, though with a different pattern of errors that indicates intuitively they have a different kind of understanding... I like the idea of a language learning challenge, but posing it properly seems tricky. As soon as something becomes a "cha=lenge", one has to worry about protecting against various subterfuges (deception, once again!). Suppose one poses a challenge to learn a language from an un-annotated corpus of texts. OK, but then som= nefarious clever person can try to solve this using an algorithm whose parameters were all carefully tuned via analysis of an annotated corpus in that same language. And these parameters may be quite=br> complex structures. The winning approach would then not be able=to work on another language for which there was no large annotated corpus (no Penn Treebank analogue, etc.). It seems that challenges are easier to formulate for engineering breakthroughs than science breakthroughs... Here is one idea, off the top of my head.... Perhaps at least i= can stimulate thoughts .... This is not about language learning, th=ugh, it's about recognizing and generating coherent, meaningful language.. 1) Show human subjects some videos of game characters carrying out certain sequences of behaviors in a video-game environment 2) For each behavior-sequence B, ask the human subjects to generate some textual instructions, that would enable the reader to emulate behavior-sequence B (even if the reader had not seen the videos) 3a) Ask the Al to figure out which textual instructions would actually work, for each behavior-sequence B 3b) EFTA_R1_01609758 EFTA02489347 Ask the Al to actually generate textual instructions, based on behavior-sequences (then the judgment is whether people, when following, the Al's instructions, actually carry out the appropriate Note that 3a and 3b both measure "coherence" in a concrete and I remember seeing some NL generation challenge vaguely like this a few years ago, but don't have the link handy. Ruiting will probably b= able to find the reference if it's of interest... For language learning, the only good way I can think of to make a challenge would be to use languages for which there are no annotated corpora. So, the challenge would be to take some unannotated t=xt (or speech) from an arbitrary human language (could be an Australian aboriginal language, or an African language, etc.), and then figure out how to generate grammatical and coherent utterances in that language. This is pretty hard obviously. If someone=chose to "cheat" by building annotated corpora or rule-bases for every obs=ure language in the world, at least they would be doing the world a big service along the way ;-D Interesting thought-direction, anyhow... ! -- Ben On Mon, Sep 7, 2015 at 4:23 AM, Jeffrey E. <[email protected] <mailto:[email protected]> wrote: > I dont want statistical modeling you and ben for years hav= stated you > wanted to put an avatar , and hope it can do things a 2 year old can d=. > the challenge is learning a language. different that=moving blocks in a > video game. > On Sun, Sep 6, 2015 at 2:38 PM, Joscha Bach > wrote: >> » This challenge idea is excellent; I really love it! >> >» first draft. of the Chomsky Challenge. =AO . Produce a non- living >» system that can be put into an environment for a while=C2 and --- 1. be >» able to discriminate language from noise.. = prize . a 1dollar bill >» signed by Noam and 100k. >> » What is the system allowed to have when it starts? We would need t= » define the environment, for instance text based or audio, or » movies/youtu=e. Once the contestants know the environment, they can » use standard machin= learning methods to discern entropy in the 2 EFTA_R1_01609759 EFTA02489348 » signal, and separate language-li=e noise from non language-like » noise. Google does this pretty well, and » I imagine you want to go beyond that? >» . 2. be able to discriminate co=erent sentences from non ( we >» provide 10 test sentences ). » I suspect that this is harder, Noam might point out that a lot of<=r> » » grammatically well-formed sentences used in politics are not » coher=nt ;-) » > prize a 10 dollar signed Chomsky bill , =AO and 500k. 3. a >» language learning module. » Build a system that is able to learn a new language without » hand-c=ding, and translate sentences from this language into English and back? =xcellent! >» 20 dollar bill signed and 1 million, 4. a sense m=king module that can >» understand meaning inference.. etc. the no= recommendation >» recommendation. . ie the student has = nice family. etc. a 100 dollar >» signed bill and 10 million dollars. ? » > >» lets also do a minsky challenge and if you want martin = NOVAK >» challenge. » Yes! Let us ask Marvin and Martin about the biggest unsolved probl=ms » in their field. > -- > please note > The information contained in this communication is confidential, may > be attorney-client privileged, may constitute inside information, and > is intended only for the use of the addressee. It is the property of > JEE Unauthorized use, disclosure or copying of this communication or > any part thereof is strictly prohibited and may be unlawful. If you > have received this communication in error, please notify us > immediately by return e-mail or by e-mail to > <mailto:[email protected]> , and destroy this communication and > all copies thereof, including all attachments. copyright -all rights > reserved Ben Goertzel, PhD http:/=goertzel.org <http://goertzel.org> 3 EFTA_R1_01609760 EFTA02489349 "The reasonable man adapts himself to the world: the unreasonable one<=r> persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man." -- George Bernard Shaw =C2 please note The information contained in this communication =s confidential, may be attorney-client privileged, may constitute in=ide information, and is intended only for the use of the addressee. It =s the property of JEE Unauthorized use, disclosure or copying of thi= communication or any part thereof is strictly prohibited and may be=unlawful. If you have received this communication in error, please noti=y us immediately by return e-mail or by e-mail to [email protected] <mailto:[email protected]> , and des=roy this communication and all copies thereof, including all attachment=. copyright -all rights reserved 4 EFTA_R1_01609761 EFTA02489350