> 2021年12月31日信息消化 ### Hopes for 2022 from Alexei Efros, Abeba Birhane, Yoav Shoham, Chip Huyen, Matt Zeiler, Wolfram Burgard, Yale Song > MEMO > > 2022趋势 -- AI数据集审计，robotics实战，ecological / bottom-up AI, batch system → continual learning, models that reason, models for vision > > “AI is not when a computer can write poetry. AI is when a computer will want to write poetry.” origin: [Hopes for 2022 from Alexei Efros, Abeba Birhane, Yoav Shoham, Chip Huyen, Matt Zeiler, Wolfram Burgard, Yale Song](https://info.deeplearning.ai/the-batch-hopes-for-2022-from-alexei-efros-abeba-birhane-yoav-shoham-chip-huyen-matt-zeiler-wolfram-burgard-yale-song-2?ecid=ACsprvu0szcvij6LYN08ydARgLmcy4Ca8bkdijjVNttIOPNCh2gsg0Ay1bZ136BHgOD6uUt8cxhC&_hsmi=199575832&_hsenc=p2ANqtz-8PcbWV8VGUbCTHC0-7lPC4VU717gtkD7FbRGZ15nTeogZXZ2q0pGb_-v5JpliC7aJB8nugXoku7G3Y5jUHYgIVtzSbkw) #### Abeba Birhane: Clean Up Web Datasets *Abeba Birhane is a cognitive science PhD researcher at the Complex Software Lab in the school of computer science at University College Dublin.* The field does not lack optimism. In fact, everywhere you look, you find overenthusiasm, overpromising, overselling, and exaggeration of what AI models are capable of doing. Mainstream media outlets aren’t the only parties guilty of making unsustainable claims, overselling capabilities, and using misleading language; AI researchers themselves do it, too. 该领域并不缺乏乐观情绪。事实上，无论你看哪里，你都会发现对 AI 模型的能力过度热情、过度承诺、过度推销和夸大其词。主流媒体并不是唯一做出不可持续的声明、过度宣传能力和使用误导性语言的一方；人工智能研究人员自己也这样做。 Language models, for example, are given human-like attributes such as “awareness” and “understanding” of language, when in fact models that generate text simply predict the next word in a sequence based on the previous words, with no understanding of underlying meaning. We won't be able to foresee the impact our models have on the lives of real people if we don't see the models themselves clearly. Acknowledging their limitations is the first step toward addressing the potential harms they are likely to cause. The nascent work in this area so far — and the acknowledgement, however limited, that it has received — fills me with hope in the coming year. Instead of blind faith in models and overoptimism about AI, let’s pause and appreciate the people who are doing the dirty background work to make datasets, and therefore models, more accurate, just, and equitable. Then, let's move forward — with due caution — toward a future in which the technology we build serves the people who suffer disproportionately negative impacts; in the word of [Pratyusha Kalluri](https://info.deeplearning.ai/e3t/Btc/LX+113/cJhC404/VWNSbV3Pd4TBW81c4rq4Khj92W3Z50n14CHK3FN8hWs4J3q3nJV1-WJV7CgLR_W6kwmZk2pJSQsW2py_wx1HpHYQW5T1fM4986NpdW4nMc274CTqxrW6GMfLH6_lnLcVJYb3c2b3111N2XHFrW8rX1QW4R-41S7wCTm5W8M7H9J6KbdNfW7k7mkq8RdLzWW1rTF7X1GMVlfW3W49MC6_vV1WW4jqYd_8vL3J9W6yYvyz7YkSMPW2Vx43v17hHjBW6vw4NF8sHpY-N7N_pTDcJxG3W4ml-M-4PZLswW2rwRl33-0RmFW99sG378SMf_tW6LzW0R40wMLMW8vm86B1FZClTN4HwwBblH52LW315X0y1pk_T72W01), towards technology that shifts power from the most to the least powerful. #### Wolfram Burgard: Train Robots in the Real World *Wolfram Burgard is a professor at the University of Freiburg, where he heads the Autonomous Intelligent Systems research lab.* After decades of programming robots to act according to rules tailored for specific situations, roboticists now embrace machine learning as the most promising path to building machines that achieve human performance in tasks like the warehouse robot’s pick-and-place. Deep learning provides excellent visual perception including object recognition and semantic segmentation. Meanwhile, reinforcement learning offers a way to learn virtually any task. Together, these techniques offer the most promising path to harnessing robots everywhere they would be useful in the real world. In the coming year, I hope that roboticists will shut down their sims, buy physical robots, take them out of the lab, and start training them on practical tasks in real-world settings. Let’s try this for a year and see how far we get! #### Alexei Efros:Learning From the Ground Up Things are really starting to get going in the field of AI. After many years (decades?!) of focusing on algorithms, the AI community is finally ready to accept the central role of data and the high-capacity models that are capable of taking advantage of this data. But when people talk about “AI,” they often mean very different things, from practical applications (such as self-driving cars, medical image analysis, robo-lawyers, image/video editing) to models of human cognition and consciousness. Therefore, it might be useful to distinguish two broad types of AI efforts: **semantic or top-down AI versus [ecological](https://info.deeplearning.ai/e3t/Btc/LX+113/cJhC404/VWNSbV3Pd4TBW81c4rq4Khj92W3Z50n14CHK3FN8hWs4J3q3nJV1-WJV7CgF_JW7KrL1l6wrf3yV1hgwd4xNdM3W1dkCHt4kGWqxW5dQg_F5ZJvqhW3Qt0LV1ZlP8zW5fXvgK8n1NWxW21qd8h7fFMDPW6plt9y858kGSW89DBKP8f0D9xVNVvZm3TfNpSW5BZp5G4wyL58W3hZtM08wkPJBW38MK_v49wJnsW4T_6pN74ZfcFW3bmcNw3gdCyvN8TxHBLqTZYBW1-2NXd2xFTj4W3wfQld21gq5PW5bwJRB1cYPnZW2mv-HZ6CTTfFW4Mr1v766g-8CN7dxPkGHMhWhW6LhQz31FqR85W4hjCQl8MM74V3qqm1) or bottom-up AI.** The goal of top-down AI is to match or exceed human performance on a specific human task like image labeling, driving, or text generation. The tasks are defined either by explicit labels (supervised learning), a set of rules (e.g., rules of the road), or a corpus of human-produced artifacts (for instance, GPT3 is trained on human-written texts using human-invented words). Thus, top-down AI is necessarily subjective and anthropocentric. It is the type of AI where we have seen the most advances to date. Bottom-up AI, on the other hand, aims to ignore humans, their tasks and their labels. Its only goal is to predict the surrounding world given sensory inputs (passive and active). Because the world is continuously changing, this goal will never be reached. But the hope is that, along the way, a general, task-agnostic model of the world will emerge. [Self-supervised learning](https://info.deeplearning.ai/e3t/Btc/LX+113/cJhC404/VWNSbV3Pd4TBW81c4rq4Khj92W3Z50n14CHK3FN8hWs4_3q3n_V1-WJV7Cg_45W2jc7SM4clgjrW31RFxh3B_v6zN8NR0kdcNS5LVm_5zl7WK7SzW3cVbn92Tj6fgW3BNW2P49HRGQW6DF3RS5dNGYMW2G79Nc9fmPKQW8n0t2q31SJtSW1C-0J48P5mY-W7_h-D-5gXCLkW6Jp4Kk2ScRvnW8QpkFG5M08WRW5Hy_P268nkd5W4ZFDnB6YvQyxW6dxCQz4-F0XmW8qjfYC832VbcW2XgmZC7PrSmtW3KljDq7c9zGDW5w7HDp4PwRKQN3zFT-M1Mm82Vrb6cL33Zy2fW8q4tY46k77PkW6ks_dS9cYVJ-W3ffgBP5Rnl_wW8N9Z8S4KQwdQ3pXY1) on raw sensory data, various generative models such as GANs, and intrinsic motivation approaches (e.g., [curiosity](https://info.deeplearning.ai/e3t/Btc/LX+113/cJhC404/VWNSbV3Pd4TBW81c4rq4Khj92W3Z50n14CHK3FN8hWs4p3q3npV1-WJV7CgWLYW7gfjnQ99G5T4W7WH02d2R5T_5VRKjCz4gmS0fW4DXbB12PPC16W5zJ3ZM35DdlLW5QN3sQ7DfGkZW5BgYWj1pcH9xW4KnDFX8nFms0W9hn88W4g_R5CW944wxp8pkYz9W7Bdzy09bgVtYW2DcN-p8blkL8W2DQNBH6f9CNMW4N1xbh2zLK1gW5l7-vs68p4C8W5JbdsR5JHLC4W4Cbttf67ByGXW6nTz0V96mQL3W6DnbzF68bJD0W3pPNFJ3C270YV-8-4J9fWlPwW1FlBZJ5952T834py1)) are all attempts at bottom-up AI. While top-down AI is currently king in industry as well as academia, its focus on imitating humans (via labels and tasks) points to its main limitation. It is like an undergraduate student who didn’t attend lectures all semester but still gets an A by cramming for the final exam — its knowledge is of a superficial nature. Real understanding must be built up slowly and patiently, from the raw sensory inputs upward. This is already starting to [happen](https://info.deeplearning.ai/e3t/Btc/LX+113/cJhC404/VWNSbV3Pd4TBW81c4rq4Khj92W3Z50n14CHK3FN8hWs453q3n5V1-WJV7CgMnzN1KK5TvP2N11W777ntY5gKVgBW5RQcld8qg2FrW3hVs8f2Qg3ddW3LmsmX6KflmzN1lN83lQd4HtW9l0scw3n61y1W4mZDHW3GKQSXW666yMj2z81hjN8X3zy7H90GFW2M_PDc14gRxXV26gnB1Cwt2VVkKNpD47gvCZW5cw82J27SB5QW97T8-t94lFWpW7HPNN244CRQmW4s8yMS2mYkTTW3g-Zc01r82bkN49ChQ159QjYW4ZC7Mc8PR4PZ3f1N1), and I hope the progress of bottom-up AI will continue in 2022. As a teenager in the 1980s USSR, I spent a lot of time hanging out with young physicists (as one does) talking about computers. One of them gave a definition of artificial intelligence that I still find the most compelling: “AI is not when a computer can write poetry. AI is when a computer will want to write poetry.” By this definition, AI may be a tall order, but if we want to bring it closer, I suspect we will need to start from the bottom up. Happy 2022! Bottoms up! #### Chip Huyen: AI That Adapts to Changing Conditions *Chip Huyen works on a startup that helps companies move toward real-time machine learning. She teaches Machine Learning Systems Design at Stanford University.* Until recently, **big data** processing has been dominated by **batch systems** like **MapReduce** and **Spark**, which allow us to periodically process a large amount of data very efficiently. As a result, most of today’s machine learning workload is done in batches. For example, a model might generate predictions once a day and be updated with new training data once a month. While batch-first machine learning still works for many companies, this paradigm often leads to suboptimal model performance and lost business opportunities. In the coming year, I hope that more companies will deploy models that can generate predictions in real time and update more frequently to adapt to changing environments. Consider an ecommerce website where half the visitors are new users or existing users who aren’t logged in. Because these visitors are new, there are no recommendations personalized to them until the next batch of predictions is computed. By then, it’s likely that many of these visitors will have left without making a purchase because they didn’t find anything relevant to them. In the last couple of years, technically progressive companies have moved toward real-time machine learning. The first level is online prediction. These companies use streaming technologies like Kafka and Kinesis to capture and process a visitor’s activities on their sites — often called behavioral data — in real-time. This enables them to extract online features and combine them with batch features to generate predictions tailored to a specific visitor based on their activities. Companies that have switched to online prediction, which include Coveo, eBay, Faire, Stripe, and Netflix, have seen more accurate predictions. This leads to higher conversion rates, retention rates, and eventually higher revenue. Online inference also enables sophisticated evaluation techniques like contextual bandits that can determine the best-performing model using much less data than traditional A/B testing. The next level of real-time machine learning is **continual learning**. While machine learning practitioners understand that data distributions shift continually and models go stale in production, the vast majority of models used in production today can’t adapt to shifting data distributions. The more the distribution shifts, the worse the model’s performance. Frequent retraining can help combat this, but the holy grail is to automatically and continually update the model with new data whenever it shows signs of going stale. Continual learning not only helps improve performance, but it can also reduce training costs. When you retrain your model once a month, you may need to train it from scratch on a lot of data. However, with continual learning, you may only need to fine-tune it with a much smaller amount of new data. A handful of companies have used continual learning successfully including Alibaba, ByteDance, and Tencent. However, it requires heavy infrastructure investment and a mental shift. Therefore, it still meets with a lot of resistance, and I don’t expect many companies to embrace it for at least a few years. In 2022, I expect a lot more companies to move toward online prediction, thanks to increasingly mature streaming technologies and a growing number of success stories. And the same underlying streaming infrastructure can be leveraged for real-time model analytics. #### Yoav Shoham: Language Models That Reason *Yoav Shoham is a co-founder of AI21 Labs and professor emeritus of computer science at Stanford University.* I believe that natural language processing in 2022 will re-embrace symbolic reasoning, harmonizing it with the statistical operation of modern neural networks. Let me explain what I mean by this. AI has been undergoing a natural language revolution for the past half decade, and this will continue into 2022 and well beyond. Fueling the revolution are so-called large language models (sometimes called foundation models), huge neural networks pretrained on gigantic corpora that encode rich information about not only language but also the world as described by language. Models such as GPT-3 (OpenAI), Jurassic-1 (AI21 Labs), Megatron-Turing NLG (Microsoft-Nvidia), and Wu-Dao 2.0 (Beijing Academy of Artificial Intelligence), to name some of the largest ones, perform impressively well on a variety of natural language tasks from translation to paraphrasing. These models dominate academic leaderboards and are finding their way into compelling commercial applications. For all the justified excitement around large language models, they have significant shortcomings. Perhaps most notably, they don’t exhibit true understanding of any kind. They are, at heart, statistical behemoths that can guess sentence completions or missing words surprisingly well, but they don’t understand (nor can they explain) these guesses, and when the guesses are wrong — which is often — they can be downright ridiculous. Take arithmetic. GPT-3 and Jurassic-1 can perform one- and two-digit addition well. This is impressive, as these general-purpose models were not trained with this task in mind. But ask them to add 1,123 to 5,813 and they spit out nonsense. And why would they not? None of us learned addition merely by observing examples; we were taught the underlying principles. What’s missing is reasoning, and math is just an example. We reason about time, space, causality, knowledge and belief, and so on via symbols that carry meaning and inference on those symbols. These abstract symbols and reasoning don’t emerge from the statistics encoded in the weights of a trained neural network. The new holy grail is to inject this sort of semantic, symbolic reasoning into the statistical operation of the neural machinery. My co-founders and I started AI21 Labs with this mission, and we’re not alone. So-called neuro-symbolic models are the focus of much recent (and some less-recent) research. I expect that 2022 will see significant advances in this area. The result will be models that can perform tasks such as mathematical, relational, and temporal reasoning reliably. No less important, since the models will have access to symbolic reasoning, they will be able to explain their answers in a way that we can understand. This robustness and explainability will help move natural language processing from the current era of statistical pattern recognition into an era of trustworthy, understandable AI. This is not only intellectually exciting, but it also unlocks practical applications in domains in which trustworthiness is essential such as finance, law, and medicine. The year 2022 likely will not mark the end of the quest for such models, but I believe that it may be recognized as a pivotal year in this quest. #### Yale Song: Foundation Models for Vision *Yale Song is a researcher at Microsoft Research in Redmond, where he works on large-scale problems in computer vision and artificial intelligence.* Large models pretrained on immense quantities of text have been proven to provide strong foundations for solving specialized language tasks. My biggest hope for AI in 2022 is to see the same thing happen in computer vision: foundation models pretrained on exabytes of unlabeled video. Such models, after fine-tuning, are likely to achieve strong performance and provide label efficiency and robustness for a wide range of vision problems. Foundation models like GPT-3 by OpenAI and Gopher by DeepMind have shown a powerful ability to generalize in numerous natural language processing tasks, and vision models pretrained jointly on images and text, such as CLIP by OpenAI, Florence by Microsoft, and FLAVA by Facebook have achieved state-of-the-art results on several vision-and-language understanding tasks. Given the large amount of video readily available, I think the most promising next step is to investigate how to take advantage of unlabeled video to train large-scale vision models that generalize well to challenging real-world scenarios. Why video? Unlike static images, videos capture dynamic visual scenes with temporal and audio signals. Neighboring frames serve as a form of natural data augmentation, providing various object (pose, appearance), camera (geometry), and scene (illumination, object placements) configurations. They also capture the chronological order of actions and events critical for temporal reasoning. In these ways, the time dimension provides critical information that can improve the robustness of computer vision systems. Furthermore, the audio track in video can contain both natural sounds and spoken language that can be transcribed into text. These multimodal (sound and text) signals provide complementary information that can aid learning visual representations. Learning from large amounts of unlabeled video poses unique challenges that must be addressed by both fundamental AI research and strong engineering efforts: - What architectures are most appropriate to process multimodal signals from video? Can they be handcrafted, or should we search out optimal architectures that capture the inductive biases of multimodal data more effectively? - What are the most effective ways to use temporal and multimodal information in an unsupervised or self-supervised manner? - How should we deal with noise such as compression artifacts, visual effects added after recording, abrupt scene changes, and misalignment between imagery, soundtrack, and transcribed audio? - How can we design challenging video tasks that measure progress in a conclusive manner? Existing video benchmarks contain human actions that are short-term (e.g., run, push, pull) and some are easily recognized from a single frame (e.g., playing a guitar). This makes it difficult to draw conclusive insights. What kinds of tasks would be compelling and comprehensive for video understanding? - Video processing is notoriously resource heavy. How can we develop compute- and memory-efficient video models to speed up large-scale distributed training? These are exciting research and engineering challenges for which I hope to see significant advances in 2022. ### **How to Respond to Negative Feedback** origin: [#277: How to Respond to an Angry Client](https://b2blauncher.com/episode277) ### **Step one: Empathize and apologize** Brandon points out that in these kinds of situations, it doesn’t really matter who’s right and who’s wrong. The customer feels the way they feel. You have to meet them where they are. When you empathize with them (“I’m so sorry about your experience with ____! I would feel the same way if I were in your shoes”), you lower their defenses. They’re ready for a fight. When you react in a relaxed, empathetic way, it disarms them. #### **Step two: Communicate your values** Next, you need to paint a picture of your values and how they align with what they hoped to experience. For example, you could tell the client that this is not what you’re about. This mistake or oversight is [below the standards](https://b2blauncher.com/episode155/) to which you hold yourself. #### **Step three: Ask for forgiveness** Asking for forgiveness is pretty straightforward. Simply say something to the effect that you hope they can forgive [your mistake](https://b2blauncher.com/prospecting-mistake/) or oversight. #### **Step four: Offer resolution** Explain how you’re going to make it right. Be clear about what they can expect and when they can expect the fix. #### **Step five: Show authenticity** You need to show humility and humanity—even if you know you’ll never work with this client again. Show that you truly want to make this right and do whatever it takes (within acceptable limits) to make that happen. Regardless of what happens from there, you’ll know that you’ve done everything you could to resolve the issue. #### **A Real-Life Example** Let’s take a real-life example. A couple of months ago, a coaching client of mine was having a difficult time with a client. She had a scheduled phone interview with the company’s CEO. But just three hours before the interview, her mother was in a serious, life-threatening accident—and she had to rush her mom to the hospital. Fortunately, her mom started to improve after a few days, but it was a scary and stressful time. The following week she tried to reschedule the call. The CEO reminded her that she’d cancelled the last meeting unexpectedly and asked her to suggest a time that was 100% guaranteed because he didn’t want to have to move the meeting again. Rude! When I heard this story, my gut reaction was to send the CEO the nastiest email possible. But that’s an emotion-driven reaction—and probably not the best course of action. Instead, I asked my client to consider that the CEO’s frustration may have little to do with her. Maybe he was having a bad day and took his frustrations out on her. It’s not fair. But it happens. Next, I suggested that she apologize again, explain why she had to cancel the call, and assure him she has every intention of making it this time. Something like this: *Hi Joe,* *I’m very sorry about having to move that call last week. I know how frustrating it is to have last-minute cancellations. [apologize and empathize]* *I take client appointments very seriously. But my mother had a major medical emergency three hours before our call, and I had to rush her to the hospital. Totally unexpected situation. [communicate your values]* *I have X day/time blocked in my calendar, if that still works for you. [offer resolution]* *I have every intention of being there. And again, my apologies for canceling last time. I look forward to reconnecting. [ask for forgiveness + show authenticity]* One last tip: once you’ve tried to make amends in this way, move on. Try not to dwell on the situation. Remind yourself that you did your best to fix it, and part of being human is making mistakes. And mistakes are the way we all learn and get better.