> 2021年07月29日信息消化 ### Week 6 Diagnosing Bias vs. Variance ##### Diagnosing Bias vs. Variance **High bias (underfitting)**: both $J_{train}(\Theta)$ and $J_{CV}(\Theta)$ will be high. Also, $J_{CV}(\Theta)\approx J_{train}(\Theta)$. **High variance (overfitting)**: $J_{train}(\Theta)$ will be low and $J_{CV}(\Theta)$ will be much greater than .$J_{train}(\Theta)$ The is summarized in the figure below:  ###### Follow-up test  ##### Regularization and Bias/Variance  In the figure above, we see that as \lambda*λ* increases, our fit becomes more rigid. On the other hand, as \lambda*λ* approaches 0, we tend to over overfit the data. So how do we choose our parameter \lambda*λ* to get it 'just right' ? In order to choose the model and the regularization term λ, we need to: 1. Create a list of lambdas (i.e. λ∈{0,0.01,0.02,0.04,0.08,0.16,0.32,0.64,1.28,2.56,5.12,10.24}); 2. Create a set of models with different degrees or any other variants. 3. Iterate through the *λ*s and for each *λ* go through all the models to learn some Θ. 4. Compute the cross validation error using the learned Θ (computed with λ) on the$ J_{CV}(\Theta)$ **without** regularization or λ = 0. 5. Select the best combo that produces the lowest error on the cross validation set. 6. Using the best combo Θ and λ, apply it on $J_{test}(\Theta)$ to see if it has a good generalization of the problem.  ##### Learning Curves **Experiencing high bias:** **Low training set size**: causes $J_{train}(\Theta)$ to be low and $J_{CV}(\Theta)$ to be high. **Large training set size**: causes both $J_{train}(\Theta)$ and $J_{CV}(\Theta)$ to be high with $J_{train}(\Theta)≈J_{CV}(\Theta)$.  **Experiencing high variance:** **Low training set size**: $J_{train}(\Theta)$ will be low and $J_{CV}(\Theta)$ will be high. **Large training set size**: $J_{train}(\Theta)$ increases with training set size and $J_{CV}(\Theta)$ continues to decrease without leveling off. Also, b$J_{train}(\Theta)<J_{CV}(\Theta)$ but the difference between them remains significant.  ###### Follow-up Test  ##### Deciding What to Do Next Revisited Our decision process can be broken down as follows: - **Getting more training examples:** Fixes high variance - **Trying smaller sets of features:** Fixes high variance - **Adding features:** Fixes high bias - **Adding polynomial features:** Fixes high bias - **Decreasing λ:** Fixes high bias - **Increasing λ:** Fixes high variance. **Model Complexity Effects:** - Lower-order polynomials (low model complexity) have high bias and low variance. In this case, the model fits poorly consistently. - Higher-order polynomials (high model complexity) fit the training data extremely well and the test data extremely poorly. These have low bias on the training data, but very high variance. - In reality, we would want to choose a model somewhere in between, that can generalize well but also fits the data reasonably well. ###### Follow-up Test  ### 记忆 Memories | 中国人民大学哲学与认知科学明德讲坛 origin: [记忆 Memories | 中国人民大学哲学与认知科学明德讲坛](https://mp.weixin.qq.com/s/Py0J7eaRM4v3YPfEm_n1_g) 刚才主持人提到一个问题，作为一个作家，记忆和虚构之间的关系是怎样的？的确，对作家来讲，我觉得**记忆和虚构有时候难以区分**。而且这种难以区分带有主动性，而不是一种被动状态。有时候人的记忆可能是虚构的，但他自己不自知。好比事情可能并没有发生，但是由于洗脑、灌输、暗示等各种原因，他认为事情发生了，他把没有发生的事也当成自己的记忆。对于一个作家而言，记忆和虚构既有被动的、普通人的这一面，也有主动的一面。因为他从事虚构，就是要把记忆重构。**没有重构就没有小说，甚至有时候重构的、虚构的记忆比记忆本身还要真实，这也恰恰是记忆主动性虚构存在的理由**。 As we know that there is a lot of reconstruction in everybody's memory, part of the reconstruction is passive. But as a writer that the construction is both passive and also most importantly active reconstruction. So when we reconstruct the memory, actively, which means we fictionalize memory, but in that fictionalization that there is actually is also a deep mining of truth and a greater truth for that matter. 首先，因为在座的很多朋友还不是专业人士，我给大家解释一下，阿尔茨海默病是什么样的疾病？它是100多年前由一位德国精神科大夫阿尔茨海默发现的。他当时收治了一个51岁的女病人，名字叫奥格斯特，当时这个病人是以精神行为症状发病的。这个病人住院四年零六个月后去世。上个世纪初在德国等欧美国家，病理检查（尸体解剖）是做的比较好的。她去世了以后，阿尔茨海默获取她的脑组织，在显微镜下面看。发现这个病人的大脑组织结构与其他病人不一样，因为一般的精神科病人，脑子里是没有什么太多病理变化的，但是这个病人的脑子里有**老年斑沉积**，还有**神经元纤维的缠结**，这么两个特殊的变化出现。 那么从临床医学来看，我们怎么区分正常老化和阿尔茨海默病人呢？实际上就是靠临床经验。我总结出三点： 第一，如果是正常老化的病人找你来看门诊，他往往说“大夫，我这记性已经很差了”，就比如刚才主持人说“我现在老了记不住了”，这种情况反而是正常老化引起的；而**阿尔茨海默病人往往是乐呵呵地说我没病**，我没事，但家属或与其亲近的人发现他有问题了。这是最重要的一点区别。 第二，如果是正常老化，记忆力减退往往是经过别人的提示、提醒还可以回忆起来；而**阿尔茨海默病人他回忆不起来，怎么着想不起来了**，这是第二条。 第三，**阿尔茨海默病引起的记忆力减退，往往还有其它伴随症状出现，就是身体上还有别的表现**。但是正常老化引起的记忆力减退，往往只有记忆力减退，没有别的伴随症状或者体征。所以从我们的（临床）经验上来看，有这三条就能鉴别开来。 Even though there are many clinical tools to tell the difference between Alzheimer's and the normal aging process, according to Prof. Jia, there are three very simple rubrics to tell the difference between the two. First, when the patient comes in and complains about memory problem, normally that patient does not have Alzheimer’s, because a true Alzheimer’s patient is not aware of his or her own memory problem. The second is that the Alzheimer’s they will have a very true difficulty in cute recollections, whereas the normal people are relatively having less problem with that. And the third, Alzheimer's is not just a memory problem, but also is accompanied by other physical impairment and physical symptoms. 关于技术和记忆，我还想指出一个重要的点。首先我同意如今的社交媒体上有很多以照片和推文的形式出现的回忆线索，而他们确实可以使我们想起许多事以及避免我们遗忘许多事情。然而还有很多心理研究提到过一个有趣的现象，叫做**提取诱发遗忘**。意思是**我们回忆一个事件时，虽然会加强我们对于这一事件的记忆，但也会同时弱化与这个事件相关但却没有被回忆的经历**。 I have one follow up comment on that important point about technology and memory. That's something I've been writing about recently. I agree that there are a lot of retrieval cues out there in the form of photographs and posts on social media that are potentially memories that can remind us of lots of things and possibly prevent forgetting. But we also know from lots of research in psychology that there is an interesting phenomenon known as retrieval induced forgetting. What that means is that when we remember an experience, we will strengthen our memory for that experience, but we may also weaken memory for related experiences that we don't retrieve it at the moment. 我最近还在读美国作家卡佛的短篇小说，阅读他的小说能唤醒我在日常生活中的一些体验。我这样的作家平常很留心生活的一些小是小非，但是如果我没有阅读的话，我觉得就回忆不起来昨天干了什么，前天干了什么，一个星期以及一个月前干了什么。更不用说一年前了，这根本想不起来。 但是通过阅读卡佛的一些非常生活化的小说，我就能回忆起我某天是怎么回事，干了什么。而且他小说中的细节一下就唤醒了那些我跟他类似的感觉。我觉得我的生活很丰富，**我觉得这种唤醒的功能恐怕只有文学能做到。还有一些好的、比较生活化的电影也能做到**。而不是那种纯娱乐的恐怖片，悬疑片，这些都只是满足人们的娱乐需要。但是你要想能够感觉得到自己的存在，那种具体而微的存在的感觉，还是**需要纯粹的，去描述人的心理、描述人的情感、梦想、烦恼、渴望、焦虑等一系列和心理有关的日常生活**。 这样的文学作品最能够唤醒记忆，而记忆的本质，我认为就是生命。没有了记忆可以说就没有了生命。比如说像刚才贾教授说的阿尔茨海默病，得了阿尔茨海默病就等于一个人丧失了记忆世界，也就丧失了生命。他自己没有任何感觉，但他给别人带来了很大的负担。 Sorry, I apologize again, sorry that I don't know what happened tonight. The question is that how to reawaken the lost memory from the writer's point of view. Professor Lin Ken's answer is that the first and foremost, we have to rely on emotional reenactments through fictional reconstruction. And the emotional reenactments are the only way to reawaken the lost memory. This is true both individually and also collectively. That's the gist of professor Lin Ken's point. I'm sorry that I cannot translate every word of it because I didn't catch much of what he was saying. 我现在提一个给Vogt教授的问题。这个问题其实跟去年出版的一本书有关，是她主编的与皮浪学说的怀疑论有关的一本著作。我想问，记忆本身其实是一种纪律的机制，包括个人记忆、集体记忆和文化记忆，福柯把这样的一种记忆称之为**权力的仪式**。记忆和知识、权力都具有同样的性别、种族和阶级的特征，也是社会秩序合理化的方式。所以在这个意义上，对记忆的怀疑其实就是对现实的一种怀疑和抵抗。Vogt在这本著作当中谈到，作为一个怀疑论者，并不是持有某种肯定或者否定的观点，而是需要进行持续的探究。**对记忆概念的无知和对记忆概念的教条主义，都会使得这种探究变得不可能**。我想请Vogt教授从怀疑论跟当代认识论的关系，谈一谈如何处理记忆矛盾的表象和分歧？有请Vogt教授。 I recently read book by professor Vogt. The book’s name is “**Epistemology After Sextus Empiricus**”. In the book, professor Vogt emphasized that being a skeptic is not a matter of holding an opinion, but to engage in some kind of a continuous inquiry where's both ignorant of and also documents about memory will make that inquiry impossible. And also that as a focal emphasize that memory itself is a disciplinary mechanism, which he called the ritual of power. And it always chooses that what matters. And it sort out history full of fractures and convictions and turn them into a continuous narrative. So memory as a form of knowledge has a profound gender, race, and class characteristics and also indicate rationalization of social order and power structure. Therefore that I’d like to solicit comments from professor Vogt about, from a histological point of view, that's how to really deal with these all kinds of contradictions and distortions in memory as an active skeptic in the sense of continuous inquiry and of history and ourselves and also ritual power and things like that. 我认为记忆是否是具有事实性的动词的问题很难回答。从动词的语法上讲它是事实性的，但这一点可能会误导我们。**似乎我们越去思考记忆，就越会意识到认知者在其中扮演的主动成分**。我对于Schacter教授对上一个问题的回答中提到的一点比较感兴趣，就是说如果我们记得一件事情，比如有记忆被某个事件的照片所触发，那可能同时意味着我们就不太记得其他一些事情了。 所以**即使我们的记忆不被有意的误导或者操纵，这种选择性不可避免地会创造自己的叙事**。选择性意味着可以有不同的故事，比如不同的照片触发了不同的记忆等等，之后总体上你就有了对于自己过去的不同叙事。所以我认为这些问题很难回答。有趣的是你联系到了我在怀疑论方面的兴趣，因为我认为我们越去思考记忆的主动性维度，我们就越对于，记忆的事实性与否、记忆是怎样的、包括了多少主动重构，保持开放的心态，即使是在记忆力很好、不编造事情的最佳情景下。 不过选择性不可避免地包括了主动性维度。我觉得很有意思的另外一种区分来自柏拉图。他说，我们对一件事的记忆，可能是由相似或者不相似的线索所触发的。比如，相似性体现在，你看到朋友的照片想到这个朋友，而不相似性在柏拉图的例子中体现在，你看到一套裙子想到经常穿这类裙子的人。似乎不相似的线索触发记忆的情况有着不同的机制，它们已经涉及了联想链条，也就是说头脑中的一些事情通过几步联想被联系起来。而联想链条带有随机性的特征，比如，对我来说小提琴会让我想起巴黎，对你来说小提琴会让你想起葡萄酒。这进一步强调了即使没有对记忆的人为操纵，记忆仍然具有选择性和随机性。这很有趣。 But nevertheless the mere selectivity does involve I think inevitably kind of active dimensions. And maybe one more kind of distinction that I find very interesting here is and that is something because when I was invited to this, you said that I should also mention some Plato and I'll do that. Now one distinction that interests Plato is whether a memory is prompted by something that is similar or by something that is dissimilar. He finds it very interesting that both happens. And I also find that very interesting like memory can be promoted or it could be a cue for a memory that you see a photograph with your friends. So you see the photo of your friend, it prompts the memory of the friend. But it could also be in Plato’s example, you could see a dress and it reminds you of person because that person used to wear that kind of dress and the dress is very dissimilar from the person.So it seems that those cases where something dissimilar prompts a memory function differently, they already kind of involve like chains of association where certain things are kind of connected in the mind and through sort of several steps of association. And that if we factor in that dimension that there is not only the kind of being remembered by something similar, but also by these dissimilar things. And then by these chains of association and then if these chains of association also involve as an inevitably some contingency like you know what are the sorts of things that I happen to associate violin music with Paris and you happen to associate it with having a glass of wine. And those kinds of things seem highly contingent, but then that I think strengthens this concern that even if there's kind of no intentional manipulation of memory, there is selectivity. And there is as it were, some chance, even with it, there's some kind of happenstance and those are things I find very interesting. 我觉得你的问题一个方面是比较难的一个问题。但是另外一方面又让我想起，也许“记忆”类似于哲学家所说的，比如说“理解”，“知道”他们都有事实性，“factive”，就是说你知道某件事情，本身那件事情就带有事实性，不然你是不会知道它的。**“记忆”是否也有这种事实性在里面？**就是说你记得某个东西，或者回忆起某件事，尽管回忆中不可避免地包含着重构或者是选择性，但是回忆事实本身也许带有它事实性的一面。 从另外一方面来讲，这种重构一方面可能带有一定的虚构成分，但另一方面它也是一种不可避免的记忆选择性的结果，而在选择性的层面，它带有一种随机性。这种随机性以及选择性，不一定是一般意义上的歪曲或者操纵。 我想朱教授可能已经读过很多关于海马体和空间记忆相关研究的文献。因为得过诺贝尔奖的缘故，这些研究最近非常火热。但时间也是一个很有趣的问题。我认为记忆的原理之一就是：**能被感知的东西都能被记忆——都会被储存在记忆中**。我们都知道动物能够感知时间。很多神经科学家（的研究）显示，**猴子、啮齿类以及鸟类等动物都能够判断一个持续的时间段，能够理解事件的时间顺序**。它们有时间顺序的概念，可以将顺序存储在记忆中，甚至可以在事后重现这个顺序。所以，如果我们只讨论时间长度的记忆或者时间序列的记忆，动物们显然有。 有意思的是，海马体不仅有助于空间记忆（众所周知），它也与时间记忆高度相关。因为正如我们所知，当我们或动物从空间中的A点移动到B点，发生空间位置的变化，总有一个先后顺序会伴随着位置的变化产生。当你在运动时，你会接收到不同种类的视觉刺激，以及其他与运动相关的感官刺激。所有这些刺激一起形成一个时间序列。在某种程度上，空间可以由这些感官输入的时序来定义。因此有研究者认为，**海马体对空间信息和时间信息的处理可能具有通用的机制**。 仅仅基于这种行为，至少许多动物能够储存关于“什么，何时，何地”的记忆，能够把这些要素联系在一起来指导自身的行为。所以我个人认为，动物，至少哺乳动物包括非人灵长类动物，都具有与我们非常相似的记忆系统。动物没有语言，这是动物和人类之间的一个重要区别。所以言语记忆在我们人类和动物之间肯定是不同的。 Interestingly, the hippocampus does not only contribute to the spatial memory, as we all know, it’s also highly relevant to the memory of time. As we know, when we or animal travels through space from point A to point B, in addition to the change of spatial locations, there is always a a temporal sequence accompanying that change of location. When you’re moving, you will be receiving different kinds of visual stimuli and other sensory stimuli related to your movement. All these stimuli join together and form a temporal sequence. To some degree, the space itself can be defined by this temporal sequence of sensory inputs. So, there are some researchers believe that the hippocampus may be processing spatial information and temporal information with a universal mechanism. So I think that’s about the question about the memory of time. But maybe Prof. Zhu’s question also includes another aspect, which is, do animals have episodic memory? To people who are not familiar with the concept, episodic memory is simply a memory of specific events, like what you had for breakfast this morning or the memory about a party in last week. There are three important elements of episodic memory- what, when and where. “When” is of course a very important component. Some people believe that animals don’t have real episodic memory, partly because they think that episodic memory requires mental time travel, which could be a unique capability of person. Also, because animals don’t have languages, we cannot just ask them if they remember something or not. 有趣的问题。首先我会快速讲讲关于创伤的问题。我认为总体上，**创伤的记忆是最容易记住的**，这在现在的记忆研究中被反复说明。经历过创伤的人大多数尽管努力但却很难忘记创伤的经历。而对于是否存在被压抑的创伤相关机制的问题则更有争议，很长时间以来这都是文献中有争议的部分。不过更清楚的一点是，创伤是更容易被记住的，这涉及到一个特定的神经环路。很多神经科学领域的研究都关注于杏仁核在情绪和创伤记忆中的地位，它和我们之前提到的海马体离得很近。我们对于情绪记忆的神经基础了解很多，尽管对于重度创伤相对少些。但是我们所知道的都支持这些记忆被更鲜活真实地记住，而不是被压抑到心灵的黑暗角落。 第一个问题很重要，就是说我们之间该如何怎样找到记忆的共同基础。我认为我们需要记得**尽管记忆易于被扭曲，充满主观性，我们都对过去有自己的呈现，但是在每个人之间仍然共享记忆的核心**。如果十个人经历了同样的事件，你可能在每个人中会得到记忆的不同版本，这被它们的关注点所影响。但是在大多数情况下存在记忆的共同的核心，我们可以根据它协作。 更普遍的说，在心理学上以及神经科学上有大量的证据说明，**人在想象未来的时候，跟人在回忆过去的时候，神经过程、心理过程是几乎是一样的**。 我们实验室针对这个问题研究了15年，其他实验室也在研究，我们以及其他人做了脑扫描研究，把被试放到FMRI扫描仪中，让他们思考一个过去的事件或者想象未来可能发生的事件，会看到非常相似的大脑活动。所以我近十年或者更久所持的一个观点是，**我们会经历这些记忆的扭曲和个体差异的原因之一，是记忆的一个重要作用不仅是帮助我们回忆过去，也是使用我们过去的经验投影到未来，为将来发生的事情做规划**。我们需要一个非常灵活的记忆系统，所以我们才能把不同记忆的成分重新组合，基于之前的经历为未来可能发生的事情做语言，而这种灵活性也许就是记忆为什么充满重构的原因之一。我们最近也为这个观点提供了新的证据。我想强调的是我们倾向于认为记忆就是关于过去的，但实际上它对于根据过去经验预测未来的作用至少和回顾过去一样重要。 在我看来**回忆是信息提取的一个小的子类**。我们可能出于多种原因进行信息提取。当这个原因不是回忆过去的时候，我们会叫它决策、思考未来、想象或创造力。只有当我们有意地回顾过去想到一个过去的事件，或者突然头脑中出现一个事件我主观感受到这是我过去的一部分的时候，我才为这个信息提取的子类打上“回忆”的标签。信息提取是我们实验室最近在研究的东西之一，而回忆是信息提取中的一部分。 I'll just give a very quick response to that. It's a very important and interesting question. What I would think about is that remembering is a subset of retrieval. In other words, we can retrieve information for lots of different purposes. And often when we're retrieving information for a purpose, other than recalling event in the past where we call that decision making or we call that future thinking，We call that imagination or we call that creativity and it's only a subset of experiences where we apply the label “Remembering” when we either are intentionally trying to think back and come up with a past event or something comes to mind, and we have the subjective sense that this is a particular part of my past. And that's one of the things we've been studying in my lab recently. I think remembering is a small subset of retrieval. 那么我接下来把问题抛给贾建军老师，这个问题同样关于记忆的衰退和丧失。我想问，**这种记忆障碍，无论是衰退还是丧失，是否会影响情绪的感受和疏离？**还是说这个事件的信息消失了，但是经历事件时的情绪感受并不会受影响。再或是正因为无法去归因到底是什么样的事件导致这样的情感，反而会导致我们在这种情感状态、甚至是负面的情感状态当中持续更长时间。好，有请贾建军老师。 谢谢主持人。我想这个问题可能在座的各位，特别是像杨老师这样搞神经机制研究的专家，在这方面理解更深一点。我想**记忆和情绪都对维持我们人体正常的生理活动发挥作用**。我们体内血液、体液里，有很多神经递质，比如记忆与乙酰胆碱有关系，情绪更多的是与五羟色胺、多巴胺有关系，这些神经递质在体内都是相互作用的，情绪可能更多的和有关，但这些神经递质在体内都是有相互作用的，这是第一点。 第二点，我们现在通过发展迅速的神经影像学和脑网络的研究发现，**很多的神经环路都相互联系**，所以说情绪和记忆是密切相关的。但并不是说一个人没有了记忆就没有了情绪，它并不是很同步。 另外我们发现，**情绪低落是抑郁症最主要的特征之一，而抑郁本身就是阿尔茨海默症发病的一个高危因素**。对于它们两个关系的研究，包括基础研究、临床研究目前来讲有很多。就其发生发展的机制目前有很多的假设，比如说我刚才提到的神经递质的关系，脑网络的关系等等，但是确切的机制目前也不是太清楚。 因为我是做纪录片和当代艺术摄影的，我理解记**忆本质上就是视觉的图像，同时也是想象力的媒介**。在这个意义上，所有的艺术作品在某种意义上都是一种记忆装置，而所有的记忆装置都具有塑造未来的潜力。也正是因为记忆的这种冲突和张力，激发出特别丰富多样的当代艺术实践。我想讲一个观点，**作为当代艺术而言，要主动地、积极地、激进地遗忘**。主动地遗忘历史，就像宁老师主动地虚构记忆一样。因为历史是权力建构的，它把断裂的、矛盾的历史整理为一个连续的叙事，人们会因为这样重构的历史丧失思考和生活的能力。所以当代艺术要主动激进的去遗忘，跟这样的历史相对立。借用尼采的话说，遗忘对于任何行动都是必不可少的。 最后我想用下面的话来做结语。我们今天从文学、哲学、心理学、神经科学等角度来探讨记忆，只有深刻地理解记忆的这种复杂性、矛盾性、脆弱性和灵活性，我们才可能真正去理解和改变我们所身处的这个被记忆影响和决定的世界。 As a documentary film director and also an artist, from my personal point of view, I regard memory as essentially the image and all the images, visual images and auditory images, are memory apparati, and they actively shape and reconstruct our past our future and also our world all the time. It’s running out of time. So I don't have time to share my view, but you can see me as a kind of activist in a certain sense, a contemporary artist. I advocate for this active radical forgetting as a movement that we should try to forget it as we can because as Nietzsche said very well that forgetting is divine and forgetting is very important to everything. And tonight that we have this gathering of these prominent scholars and also artist, doctors. And we are very, very honored to have all of you here and the time is running out. Thank you so much and much appreciate.