Yestersol's drive ended about 1m short of the third waypoint, so we didn't execute the stutter-step at the end. But we did turn for comm, so we got lots of downlink and are ready to drive again.
Spirit's total odometry now stands at 3950m. I probably still won't get to take her over the 4km mark, but I'm going to get one more chance at it than I thought: Ashitey can't work tomorrow, so he asks me to work for him. I have a standing policy about this: whenever you get a chance to drive the rover, take it. Much as it pains me to think of it, theoretically they won't be around until the end of time, and I don't want to be kicking myself later for any missed opportunities (or spirits, har har). So of course I say yes.
This introduces a small problem: it means that on the day after that (Thursday), both RPs will be coming in cold. (Normally, we like to have at least one of the RPs be one who was on shift the previous day.) So we do some complicated schedule juggling; I switch with Ashitey on Thursday, so I'll be on Spirit yet another day. What's more, I'll be Spirit's RP-1 (with John as RP-2 -- that brings back memories). Maybe I'll take her over the 4km mark after all.
Spirit and Opportunity have some cultural differences, one of them being the schedule of the rover planners. On Opportunity, both RPs arrive around the same time, but Spirit retained some of the nominal-mission behavior, where the RP-2 comes in later than the RP-1. Normally, when I'm RP-2 on Spirit, I arrive about one hour ahead of the activity plan approval meeting. This usually gives me enough time to come up to speed before the meeting, then the RP-1 hands over to me.
But today I'm startled when, just a couple of minutes after I arrive, Saina asks Chris (RP-1) if he has an animation to show. I look up and realize that the APAM is already happening. I should have known: when Saina's the TUL, she tends to push the process along as fast as it will go. (But, to her credit, she's never given me a moment of hassle when I've needed extra time. When I ask for something, she gives it.) Justin says she's got a hot date (every time she works?), but today it's something different: she's going to Europe for a couple of weeks, starting tomorrow, and she's eager to go home and pack.
We spot a problem at the CAM: in one of the paths through the sequence, the rover won't turn for comm. So, even though it's a pain to do this so late in the process, we fix the problem and redeliver. Then almost everyone goes home, and I stick around to do other work ... when I think of another problem.
As our drives have gotten more complex, we've been working out ways to simplify them. One method is to use "helper sequences" -- take groups of repeating commands, make a subsequence for those, and then insert multiple calls to the helper sequences. This makes the sequences shorter, as well as being easier to read, write, and reason about. We've been getting more and more aggressive about this; we now even have helpers for the helpers.
But we do have to keep something in mind. MER lets you call sequences in one of two modes -- "abort" mode or "no-abort" mode -- depending on what you want the caller to do when the helper sequence isn't there. (Since we're sending the commands across hundreds of millions of kilometers of space, sometimes interference prevents part of the command load from reaching the spacecraft.) In "abort" mode, the calling sequence immediately stops its own execution. In "no-abort" mode, the most common mode, the calling sequence continues. Either of these can be what you want (which is why they're both there). The helper sequence might be doing something relatively innocuous, like taking a picture you'd be willing not to get, in which case you use the no-abort mode. Or the sequence might set up conditions that the caller is relying on, which is often the case in IDD sequences and drive sequences.
And the thing I just realized is that we're calling some of the drive helper sequences in no-abort mode.
OK, no need to panic. Maybe. I take a few minutes to think through the various cases, all combinations: if this sequence doesn't make it but that one does, are we still OK? Most combinations wouldn't hurt us -- but one of them would.
Saina'a still here. I explain the situation to her and, as eager as she is to go home, she says, "Well, it's the right thing to do. Let's fix the problem." It'll be even more painful to fix the problem now than it was to fix the other one at the CAM, but by pure luck we happen to still have the necessary people to do it. Saina prepares to call Leo to advise him of the situation, and I flip over to RSVP to start implementing the changes.
At which point I see that the sequences in question are called in abort mode, after all. There isn't a problem; never was.
I am so embarrassed. Saina just laughs -- and, wisely, leaves.[1]
Footnotes:
[1] Saina recently started a charity called Geeti, which gives people around the world printed photographs of themselves and their loved ones. People in many parts of the world have never seen a photograph of themselves, have no way to show their children a picture of their parents' wedding day -- no way to visually preserve their memories. Surrounded as I am by pictures of my family, of myself, of friends -- in some cases, pictures so important to me that I'd risk my life to save them -- I was touched by the idea of a way to help provide something so priceless to so many. Check out her site, and if you like the idea, as I did, consider donating there.
"Geeti," by the way, is Saina's mom's name -- it means "world" in Persian.
Hello. Thanks for writing this blog, it has been fun to read.
ReplyDeleteI'm a programmer and I started to wonder how do you create those IDD sequences? Are they some sort of script which is compiled into some bytecode which is sent to the rovers or something entirely different? Are they something what you can show in this blog to give a feeling what's it all about?
Thanks and greetings from sunny but snow full Finland :)
@Garo Glad you're enjoying it!
ReplyDeleteThe way I'd describe writing sequences to a fellow programmer is that it's very much like writing code. The rover has a simple command language that feels similar to a dumbed-down Tcl or BASIC.
Here's an example that gets the flavor right, although for legal reasons the command names are all made up:
DRIVE_FORWARD 1.5m 10deg
IF HEADING LESS_THAN 0deg
STOP DRIVING
END_IF
(Sorry about the formatting; Blogger doesn't allow much.) In my made-up example, the rover would execute an arc that (in a no-slip environment) would move her forward 1.5 meters and change her heading 10 degrees clockwise. If she then found that her heading was west of north (0 is north), she'd stop driving for the day.
Commanding the IDD (arm) is similar, though a lot harder to visualize because you're commanding the five joints of a very complex arm. For example, we have a command whose real name is similar to SET_IDD_JOINTS; the arguments to the command are the joint angles you want it to reach, plus some other things. It is very, very hard to do this correctly in your head, especially when you want the tip of the arm to place an instrument within millimeters of a science target.
Fortunately, we have RSVP, the software I helped write, which lets us simulate and visualize the results of the commands we're going to send to Spirit and Opportunity. So we can send these commands to the simulator, which interprets them and shows us the results -- including any errors. Much as in a debugger, you can step through one command at a time, or just run the animation and watch what your commands will do. In addition, RSVP helps you build the commands in the first place, so that you don't have to guess at the joint angles; for instance, you can have it snap the arm onto a target, then ask it to generate commands to move the arm there.
We'd be sunk without RSVP. Part of the art of the job, however, is doing the things RSVP can't do for you -- modeling slip on a new terrain type, or showing the effect of a low-level command on a flaky piece of hardware, and so on.
When we're done, the commands we've written are translated into a more compact binary form, then sent to the rovers in that form; part of the job of the rovers' flight software is to interpret and execute the commands. This is exactly analogous to -- indeed, it is -- a virtual machine executing bytecode.
I've written a paper about the rovers' command language and sequence execution environment, covering this stuff in much more detail than I can go into here; that paper has been accepted for IEEE Aerospace 2010 (Big Sky, Montana, USA, March 6-13). Which is one reason I know it's lawful to talk about it here. :-) I'd point you to that paper, which will be published in a few months, for more information.
Just so you know what to expect, while that paper does describe our command language and the sequence execution environment at a high level, it focuses on our command language's tradeoffs between simplicity and power and how those tradeoffs have played out in practice.
@Scott thank you for a great response. I'm hoping to read that paper after it has been released, thanks in advance. Have a nice week.
ReplyDelete