EFTA02585482.pdf

From: Joscha Bach Sent: Wednesday, October 11, 2017 5:56 PM To: Jeffrey Epstein Subject: Re: After skimming their paper, the idea seemed unexciting to me at first: =asically, if we have enough feature dimensions we can almost always =ind a linear separation. This is also related to how Support Vector =achines work: they project the data into an extremely high-dimensional =pace, find a separating hyperplane with linear regression, and then =roject that plane back into the original space as the separator. A =imilar idea is behind Echo State networks, which use a randomly wired =ecurrent neural network and then only train the output layer with a =ingle linear regression. The authors take an existing trained neural network, and whenever it =akes a mistake, they train a linear classifier on the network state and =ata, i.e. they try to find out when the network goes wrong. Instead of =mproving the network (which is also likely to make it worse in other =ases), they add an additional layer to it. For engineering, this makes = lot of sense, because large neural networks are cheap to use and =eploy but expensive to train. On a more philosophical level, it is tempting to ask if that might be a =eneral learning principle for brains: when you don't perform well, add =ore control structure on top. It probably makes sense whenever you are =onfident that training the existing structure won't improve it that =uch, but unless training the weights in an existing network, it also =dds quite a few milliseconds to the processing time. There is probably =n optimal tradeoff for this. The other thing is that the new layer is a =inear classifier only (at least in this paper), and it is creating a =ocal override on the system's results, instead of integrating with it, =omewhat similar to how reasoning might override our subconscious =ehavior. One of the drawbacks is that this won't allow us to use the =ew layer for simulating/understanding the structure of the domain =odeled by the rest of the network. — Joscha > On Oct 10, 2017, at 09:43, jeffrey E. <[email protected]> wrote: > https://www.sciencedaily.com/releases/2017/08/170821102725.htm > -- > please note > The information contained in this communication is confidential, may > be attorney-client privileged, may constitute inside information, and > is intended only for the use of the addressee. It is the property of > JEE Unauthorized use, disclosure or copying of this communication or > any part thereof is strictly prohibited and may be unlawful. If you > have received this communication in error, please notify us > immediately by return e-mail or by e-mail to [email protected], > and destroy this communication and all copies thereof, including all > attachments. copyright -all rights reserved <?xml version=.0" encoding=TF-8"?> <IDOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/Propertylist-1.0.dtd"> <plist version=.0"> alict> <key>conversation-idgkey> EFTA_R1_01765067 EFTA02585482  <Integer>28259</integer> <key>date-last-viewed</key> <integer>0</integer> <key>date-received</key> <integer>1507744549</integer> <key>flags</key> <integer>8590195717</integer> <key>gmail-label-ids</key> <array> <integer>6</integer> <integer>2</integer> </array> <key>remote-id</key> <string>757802</string> </diet> </plist> 2 EFTA_R1_01765068 EFTA02585483