Writing Robot Controllers with Python Generators

A typical form for a robot controller that produces some behavior has
at its core a loop that repeatedly sends control signals to the robot,
often with a some code before and after the loop for set-up and
clean-up. This is how, e.g., the example code in Sutton and
Barto’s Reinforcement Learning
is written.

The problem with this structure is that often the control algorithm
doesn’t have top level control. For example in Pyro,
the Pyro engine handles top level loop, and the main routine of the
controller is called periodically. This kind of time-sliced
architectures simplifies some aspects
of writing controllers, but breaks the nice single-program structure
above. Now the set-up and clean-up code need to be broken out into
separate functions or methods, and the body of the loop encapsulated
in a step() or next() function. Things get
even more complicated if the controller consists of a sequence of
sub-behaviors, requiring a sequence of control loops. Then the
step() function must embody a finite state
, and branch to the correct behavior for the current state
on each call. The more complicated the behavior, the more of a pain
it becomes to program this way.

For my research I need extended, compound actions that are treated as
a single action. My solution? Write the controllers as Python generators.

Generators are a new standard language feature in Python 2.3, though
they’re available in Python 2.2 using

from __future__ import generators

Generators are syntactically the same as functions, but use the yield statement to return values. When the function is called, it returns an iterator that can be looped over. Each iteration returns to the point after the last yield. Here’s an example generator that generates the sequence of integers from zero to N, then back to zero again:

def updown(n):
i = 0
# first ascending order
while i = 0
yield i
i -= 1

Running it looks like this:

>>> for x in updown(3): print x,
0 1 2 3 2 1 0

Using generators, robot controllers in time-sliced systems can be
written without having to build in a finite-state machine, or even
separate set-up/clean-up methods. The main trick is to
yield each time the controller performs a primitive
control action. In my implementation, the controller actually yields
the action, either as a function to be called, or as an index to an
action in a table, and a generic higher-level step function iterating
over the controller receives that and acts upon it.

Here is an example of a controller that follows a hallway centerline
until it reaches an obstacle, when it turns around, and follows the
hallway in the other direction. It does this N times. The control
dynamics are deliberately oversimplified, and the code is totally
untested, but it illustrates the idea:

def pace(N):
for i in range(N):
# wall following phase
while not obstacle_in_front():
if right_distance() > left_distance():
yield turn_right
elif right_distance() < left_distance():
yield turn_left
yield go_straight
# turning phase
while obstacle_in_front():
yield turn_left

My implementation supports hierarchical action, so for example,
turn_right could itself be a generator function, that
performs another extended action.

The top-level .step() method of the
Pyro Brain contains a controller stack. Each step calls
.next() on the iterator on the top of the stack. If that
call yields another generator function, the resulting iterator is
pushed on the stack and called. When the top item on the stack throws
a StopIteration exception, it is popped off and iteration resumes with
the iterator below


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: