Ch1：古今的manipulation与仿真、ROS和Drake介绍

不同的机器人研究与仿真

以前（15年左右）只能用仿真环境训练行走机器人，对于manipulation任务，有两个问题：1）相机不真实；2）接触行为太复杂。
I remember just a few years ago (~2015) talking to my PhD students, who were all quite adept at using simulation for developing control systems for walking robots, about using simulation for manipulation. “You can’t work on manipulation in simulation” was their refrain, and for good reason. The complexity of the contact mechanics in manipulation has traditionally been much harder to simulate than a walking robot that only interacts with the ground and through a minimal set of contact points. Looming even larger, though, was the centrality of perception for manipulation; it was generally accepted that one could not simulate a camera well enough to be meaningful.

但时代在发展，现在的游戏引擎已经可以很好地建模相机，使其不仅能在仿真环境中测试感知系统，还可以在仿真中训练感知系统并sim2real到现实世界中。
How quickly things can change! The last few years has seen a rapid adoption of video-game quality rendering by the robotics and computer vision communities. The growing consensus now is that game-engine renderers can model cameras well enough not only to test a perception system in simulation, but even to train perception systems in simulation and expect them to work in the real world! This is fairly amazing, as we were all very concerned before that training a deep learning perception system in simulation would allow it to exploit any quirks of the simulated images that could make the problem easier.

且模拟接触行为的质量和性能也大大提高（这方面的模拟主要是靠复杂的geometry queries和stiff (measure-)微分方程），虽然目前在控制方程的数值求解方面仍有改进空间，但现今的求解器已经足够好用了
We have also seen dramatic improvements in the quality and performance of contact simulation. Making robust and performant simulations of multi-body contact involves dealing with complex geometry queries and stiff (measure-) differential equations. There is still room for fundamental improvements in the mathematical formulations of the numerical solutions for the governing equations, but today’s solvers are good enough to be extremely useful.

上面是MIT里比较概括性的介绍，下面是Drake的一篇post里更具体一点的关于行走机器人和manipulation与仿真的介绍：

我在深耕行走机器人多年后意识到manipulation任务远难于行走任务。因为几何、接触等都太多样了，成功率低、期望却高（如行走机器人脚偏了一点无所谓，但机器人拿咖啡时手偏了一点就洒了）
After working on legged robots for many years, I came to appreciate that simulating robots manipulating diverse objects (with their hands) is more difficult than simulating a legged robot. Both systems make and break contact with the environment, but the diversity of contact geometries and configurations in manipulation is much greater and the numerics are much worse. Expectations are high, too. If a simulated humanoid robot’s foot slides a little when coming into contact with the ground, nobody will notice. But if a coffee mug slowly slides out of the robot’s hand, that’s bad news. Achieving accuracy and efficiency requires a nuanced relationship between the multibody equations and the numerical integration codes.

ROS (the Robot Operating System)

ROS的重要性：ROS模块化组件的设计（即ROS package）使得各分支学科专家可以轻松共享各自的专业知识。不同的ROS package甚至可以使用不同语言编写、在不同操作系统上编写。
I believe that ROS was one of the best things to happen to robotics in the last decades. It meant that experts from different subdisciplines could easily share their expertise in the form of modular components. Components (as ROS packages) simply agree on the messages that they will send and receive on the network; packages can inter-operate even if they are written in different programming languages or even on different operating systems.

但本课并不采用ROS中基于消息传递的模块交互方式。因为这种模块化方法并不会对模块的内部状态、参数和时间行为做统一的规定，并不适合教学中对复杂系统的理解和调试。在DRAKE中，每个模块以一致的方式声明状态、参数、时间语义，有助于我们理解和调试manipulation全栈。且支持可重复的确定性仿真。
Although ROS makes it relatively easy to get started with manipulation, it doesn’t serve my pedagogical goal of thinking clearly about manipulation. The modular approach to authoring the compunonets is extremely good, and we will adopt it here. But in DRAKE we ask for a little bit more from each of the components – essentially that they declare their states, parameters, and timing semantics in a consistent way – so that we have a much better change of understanding the complex relationships between systems. This has great practical value as well; the ability to debug a full manipulation stack with repeatable deterministic simulations (even if they include randomness) is surprisingly rare in the field but hugely valuable.

DRAKE就是我们的key building block，在Drake中可将所有模块组合成系统图Diagrams，并在notebook中可视化这些系统图
The key building block in our work will be DRAKE Systems, and systems can be combined in complex combinations into Diagrams. System diagrams have long been the modeling paradigm used in controls, and the software aspect of it will be very familiar to you if you’ve used tools like Simulink, LabView, or Modelica. Theses software tools refer to the block-diagram design paradigm as “model-based design”.

Drake作者对自己的介绍：一个开源软件工具包（C++、Python），功能包括：动力学引擎（即研究物体之间的碰撞、接触等物理行为，旨在解决复杂的动力学问题）、系统图整理、数学优化
Drake is TRI’s (Toyota Research Institute) attempt to provide mature, open-source tools for another large portion of the roboticist’s toolkit, complementary to the deep learning toolboxes, to further buttress advanced research in the field. It is a library (in C++ with Python bindings) with three major components: 1) the multi body dynamics engine, 2) the “system framework” for organizing and combining system models from a library into a block diagram, and the 3) optimization framework for mathematical programming. Each of these components is important for robotics research; the vision for Drake is to facilitate making powerful combination of all three components within a single application.

本课安排

各章节按照整个manipulation任务中不同的building blocks组件进行讲述，涵盖计算机视觉、动力学和控制学等的领域知识，本课仅介绍各领域与manipulation最相关的idea。
The remaining chapters of these notes are organized around the component-level building blocks of manipulation. Many of these components each individually build on a wealth of literature (e.g. from computer vision, or dynamics and control). Rather than be overwhelmed, I’ve chosen to focus on delivering a consistent coherent presentation of the most relevant ideas from each field as they relate to manipulation, and pointers to more literature. Even finding a single notation across all of the fields can be a challenge!

会讲相关的机器人硬件、仿真这些硬件的细节，以及几何和动力学基础
The next few chapters will give you a minimal background on the relevant robot hardware that we are simulating, on (some of) the details about simulating them, and on the geometry and kinematics basics that we will use heavily through the notes.