This is a dissertation about simple things. Walking, moving stuff around, performing tasks in a certain order: these are easy for most people. But human competence in locomotion, manipulation, and task planning masks their fundamental complexity. Nowhere is this more obvious than in robotics. Robots that can move around, affect their environment, and plan to achieve multi-step goals have tremendous potential to impact human flourishing, from performing dull or dangerous tasks on Earth to pushing the boundaries of science on distant planets. Despite amazing recent progress in computing and artificial intelligence, however, robots with even a fraction of human abilities remain firmly in the realm of science fiction. Why are locomotion, manipulation, and task planning so deceptively difficult? One key reason is scalability. To move through the world and move objects around, robots must make and break contact with the environment. As the number of possible contact points increases, the number of dynamics modes grows exponentially. Similarly, the computational cost of planning to achieve a high-level task grows exponentially as the task becomes longer and more complex. This dissertation addresses scalability challenges in locomotion, manipulation, and task planning. For locomotion and manipulation, we focus on high-degree-of-freedom robots composed of rigid links, a category that includes manipulator arms, quadrupeds, and humanoids. For task planning, we focus on specifications expressed in temporal logic. A unifying theme is optimization: by carefully formulating optimization problems with desirable characteristics, we arrive at solutions that scale to larger systems, more complex contact configurations, and more difficult tasks. We first consider temporal logic task planning and propose planners based on nonlinear programming, mixed-integer programming, and convex optimization. These methods outperform the state-of-the-art in terms of scalability to high-dimensional systems and complex specifications, but require some simplification of the robot's dynamics. To apply the resulting plans to manipulation and locomotion, we need a bridge between simplified models and the robot's full nonlinear dynamics.  We then present two ways of building such a bridge. The first method, approximate simulation, provides an error bound between a simplified model used for planning and the robot's nonlinear dynamics. We use passivity principles to certify approximate simulation for large nonlinear systems, and present a new notion of robust approximate simulation that accounts for disturbances and modeling errors. Approximate simulation provides strong formal guarantees, but is also conservative. It cannot use the full dynamics of the robot, including the ability to spontaneously make and break contact, to achieve the task.  This motivates an alternative bridge between high-level plans and the actual robot: trajectory optimization through contact. Optimization through contact offers fewer guarantees than approximate simulation, but requires fewer assumptions. This approach takes a high-level motion plan as a reference and uses the full system dynamics, including making and breaking contact, to achieve the task. We present several new methods for trajectory optimization through contact, including a custom solver that enables real-time model predictive control. We demonstrate the speed and scalability of these methods in locomotion and manipulation tasks.