Spirit Sol 266

I'm just sitting down to catch up on the latest pictures when I hear Henry Stone stride by in the hallway outside, speaking tensely into his cell phone. I only hear a few words, but it's enough to make me worried.

Screw the pictures. I go to the SMSA and sit down with Rich Petras and Jeng Yen, our Mob/IDD folks.

"How's Spirit?" I ask.

"It's broken," Rich says. "We did the half-meter and 45cm arcs but failed at the turn-in-place." He says it's something to do with the dynamic brakes, which I've never heard of before. Whatever the dynamic brakes are, they were off when they were supposed to be on (or maybe the reverse). The flight software detected that the brakes were in the wrong state and correctly refused to drive on.

"So it's a hardware problem?" I ask. "Not a sequencing problem?"

"Yeah, it's a hardware problem," he confirms. This makes me feel better and worse. Better, because it's not my fault. Worse, because I'd rather have screwed up than have this be a hardware problem. Much rather. When you're two hundred million miles away from the hardware, you don't like hardware problems.

Dynamic brakes turn out to be this. They're brakes (duh) on the wheels that are intended to prevent unintended steering or rolling. So there are brakes for the steering actuators (arranged in pairs) and drive actuators (likewise, I think). The brakes that failed are steering brakes, in particular the pair that controls the right front and left rear steering actuators. The idea is that the brakes should be released when the steering actuators are being run, and then reapplied while you're driving, so that the wheels won't wobble (inadvertent steering) while you're moving. Roughly, then, the way the rover drives is: release the steering brakes, steer the wheels to the desired orientation, apply the steering brakes and release the drive brakes, drive, release the drive brakes. And somewhere in there, the brakes were apparently in the wrong state.

When I showed up, Rich and Jeng were digging around in the flight software, trying to better understand the error message. What they're hoping for is that it's a flight software problem that just looks like a hardware problem. It might be, for instance, that the software just happened to check the state of the dynamic brakes at the wrong time, one of those one-in-a-thousand bugs that you'll nevertheless run into if you do it enough.

Even if it is a hardware problem, it might be something temporary. For instance, the relay that controls the dynamic brakes might have been slow to respond due to the cold temperatures. It might work perfectly the next time we try it, and might never fail again.

And even if the this set of dynamic brakes is permanently stuck, we can still drive. We can deliberately blow the fuse from Earth, permanently releasing the brakes. Then we tell the flight software to ignore the state of those brakes, and we drive normally. Well, not normally, unfortunately -- we'll have to change the way we drive to account for the fact that the wheels will wobble during the drive, possibly taking the rover off course. But at least we'll be able to do it.

Naturally, this changes our plans. The scientists are going ahead on the assumption that we can safely use the IDD, though this is not guaranteed -- some failure modes could involve electronic problems that using the IDD would exacerbate. But this is considered unlikely, and anyway the science team has nothing else to do.

With the engineering teams, it's a different story, as you might expect. The mechanical team has already started working on the problem, and a meeting is scheduled for 13:30 to discuss the results. Rick Welch schedules an 11:00 meeting to work out what they're all going to say at the 13:30 meeting -- which, since the two meetings have different audiences, is less useless than it might sound. The 11:00 meeting is the actual working meeting; the 13:30, since it's the formally declared meeting, will attract more high-level types who want to know what's going on.

I go to the 11:00.

In addition to the various graphs and charts people are already coming up with, the participants work out a list of possible causes. There might have been some failure in the command's attempt to send a signal to the relay, the relay itself might have failed (or might be in the early stages of degrading -- someone's assigned to find out the part's design lifetime and known failure modes), the relay might have worked but incorrectly communicated its status. There might also be a bug in the flight software, including a race condition, but the responsible developer -- Jeff Biesiadecki -- calls this unlikely. The relay is supposed to respond within 40msec under all circumstances, and his code gives it 65. Still, he volunteers to look into it.

With the mobility subsystem off-limits for now, we're left with IDD sequences only. The scientists choose to spend most of the weekend calibrating the APXS. We're going to take several APXS readings of a rock that happened to end up in front of us when the rover stopped driving -- first with the APXS directly on the rock, then at 5mm, 10mm, and 15mm away. Last of all, we change from the APXS to the MB.

I could do that in my sleep. If I could sleep.

[Next post: sol 270, October 6. Due in part to a mobility stand-down to allow time to investigate the dynamic brake issue, we didn't resume driving for a few sols.]