Spirit Sol 739

The weekend went flawlessly. The MIs completed without a hitch, and they look absolutely gorgeous. Oded was right about this being a vesicular basalt (though it turns out to be wind-eroded as well, which confuses the issue), so it turns out to have the scientific value they were hoping for. ("You sure got lucky on this one, Oded," one of Friday's objectors graciously admits at the SOWG.) But as I told him in email I sent him over the weekend, if this IDD deployment wasn't worth it for the science, it was worth it for the art. The thing is weirdly twisted, with little fingers sticking up and out every which way, reminding me of one of those Japanese woodcuts with the waves. By the famous Japanese artist, uh, What's-His-Name-San.[1]

It's a bit of a shame we only had time for MBing it, but as it turns out, Oded says, "I'm not sure we would have gotten a good APXS measurement on this thing anyway. It's so eroded, and has soil blown into all the eroded spaces; I'm not sure the instrument would have seen anything."

Too late now, anyway: that rock's 30m behind us, thanks to John's excellent driving. (During which time, the dynamic brakes appear to have been no problem at all. Nice to have that behind us as well.) That's pretty good for recent drives, but it won't be good enough. Our plan was to be at Home Plate by sol 740 so that we could do 20 sols of science and be zooming toward McCool Hill by sol 760. Since tomorrow's sol 739, and our radial distance to Home Plate hasn't changed much -- the weekend's drive was mostly moving laterally to Home Plate, skirting Mitcheltree Ridge -- we have essentially no hope of making it there on schedule.

Moreover, our power situation's bad. The sun keeps heading north, and we're on south-facing slopes. One of the ways this is starting to affect us is in our downlink: we can't afford overnight passes until our power improves, so they have to cut science to the bone. And even then, we can afford only 2.5 hours of driving.

Sigh. Winter. I haven't missed it.

Thisol's blind drive is limited by the ridgeline, still about 20m away. The drive sequence is a fairly simple one: skirting around a couple of nearby rocks (that aren't really big enough to hurt us; we're avoiding them mostly on principle), scoot to the ridgeline, turn on autonav, and hope it will find a path downhill. If it doesn't, we'll probably still be OK: perched up on the ridge like that, we'll have a lovely view of the scene before us and should be able to plan a wicked blind drive on the following sol. Although that honor will have to go to Paolo; I'm off tomorrow, darn it.

After the walkthrough, John decides to change the sequence slightly to give us a little more margin around a sandy hollow. None of us notices that he introduced an error. The first waypoint, instead of being executed blind and taking just a few minutes, will be executed in autonav mode. If it completes at all, it'll take about 45 minutes.

Oops.

We don't notice this at the CAM, and almost everybody leaves. John's building the animation for the uplink report when he notices something funny -- the rover moving much more slowly than it's supposed to through that bit. We end up calling everybody back in to fix it.

I'm glad he caught it, and I'm glad, upon reflection, that it's been so long since we've had to do something like that.

I'm not so glad that we didn't catch it earlier. We come up with a fr_check rule that would catch it, but the underlying problem -- human error -- will be less easy to eliminate.

[Next post: sol 743 (Opportunity sol 723), February 4.]

---

Courtesy NASA/JPL-Caltech. One of the MI images we took of this rock. Man, this thing is cool. More here.

---

Footnotes:

[1] Hokusai. This, courtesy of Wikipedia, is his woodcut Great Wave Off Kanagawa:

Spirit Sol 736

Once again, I woke up in the middle of the night with an urgent desire to check on the drive ... and looked at the clock, decided not to make this a habit, and went back to sleep.

By the time I get in, Ashley and John have already roughed out the next drive. Yestersol's drive made about 26m of progress, stopping when autonav failed to find a path between Scylla and Charybdis (as we thought might happen). Still, this is a decent showing -- I think we might have actually caught up on our drive metric a bit (we're now 90m behind on that, maybe 170m from Home Plate) -- and we're ready for the next one.

And boy, are we ever ready for the next one. We got all the diagnostic data down, and Jeff Biesiadecki's call is that this is merely a recurrence of the previous Spirit steering anomaly. So we're good to ignore the dynamic brakes and resume normal driving. Triumph!

The original plan for the weekend had been to resume the autonav-only continuation-drive sols. But there's this rock smack dab in the middle of the IDD work volume .... It's not a big rock, only about 5cm tall and 12cm wide. But it's possibly a vesicular basalt, a rock type of particular interest to at least some of the science team. In particular, to Oded Aharonson. And since he's the SOWG chair today, there'll be at least one voice calling for a change of plans.

I'd like to be able to argue that we should blow it off and drive. But when Oded asks, I have to admit that the autonav-only sols haven't been performing well. Let's be generous and figure that we might make as much as 40m from a two-hour autonav-only drive. But if the first sol stops because autonav can't find a safe path, the second won't even try to move. Given the increasingly rocky terrain we're in (it looks like we're seeing another ejecta blanket), I figure there's about a 70% chance of no progress at all on the second sol. Which really means we're giving up only about 30% of 40m, or about 12m -- and that's using generous assumptions.

With those numbers in front of him, Oded can't resist the rock. Though the matter is unusually contentious at the SOWG meeting, with a number of the science folks saying we should skip the thing. (Justin is particularly vocal, and characteristically stubborn. At one point I suggest we let him name the rock; that idea gets a big laugh.) We have time only for MB or APXS, not both, and the results from only one of these instruments are a lot less scientifically valuable than the results from both together. Oded, however, makes the point that we mostly want this thing for the MI images -- morphology is the key to a vesicular basalt, more than chemistry -- and he gets his way in the end.

It's a more than usually tough sol. John and Ashley worry about the drive, while Terry and I take care of the IDD sequencing. IDDing this rock is just plain tricky: we want to try to get four MI stacks on it, but its small size makes that awkward. The least dusty and therefore most desirable face is toward us -- not on top -- so as the IDD gets close enough to plant an instrument on the rock, the hardware slides uncomfortably close to the ground. Worse, the rock is more or less rounded on top -- kind of like a half-buried football -- so that as we raise the IDD to increase its clearance from the ground, we get much less rock in the instrument's field of view. If only it were a LEGO brick.

The other tricky part is that the face we're poking at is fairly irregular, so that a simple linear mosaic won't do; we need to customize each stack, and have a MB touch for each (each of which also needs to be carefully analyzed to ensure we won't smack into the ground). Oh, and the MB will contact the rock at its outer surface, but because it's a vesicular basalt, we want to see into the vesicles -- the holes -- so we want to take MI images as close as 11mm. Which means we get that much closer to the rock, and to the ground ....

But, what the heck, we make it. And by aggressively parallelizing the work, we finish the whole thing -- from start of SOWG to end of CAM -- in only about seven hours.

Maybe we could have thrown in that autonav-only drive as well ....

[Next post: sol 739, January 31.]

---

Courtesy NASA/JPL-Caltech. Stupid vesicular basalt. Grumble, grumble.

Spirit Sol 735

Things being in the state they're in, we start the day with an oddly subdued anomaly meeting. Yestersol's drive went OK, with occasional warnings about the dynamic brakes, but since we had disallowed steering on those wheels, there were no resulting faults. The drive terminated after 9m (actually 8.99m, bleah) due to the hazard-avoidance code's decision that there was no safe path behind us. This was really the result of a hair-triggered autonav tilt setting; we're actually quite safe to drive on. Indeed, as one person points out, we're far enough behind the drive metric that driving is the conservative option.

And that's the plan: drive. We might not be able to make that much progress, but at least we'll fall behind our drive metric more slowly this way. (We're about 110m behind now, and roughly 170m from Home Plate.) Due partly to some impassioned pleading on my part, we're going to pack pretty much all of the diagnostics into this sol, so that tomorrow we can plan a normal -- and, I hope, long -- drive for the weekend. Maybe even a pair of drives: we're far enough behind the drive metric that every meter counts, so we might revert to the autonav continuation thing.

John Wright's raring to drive, to the point that I actually have to rein in his enthusiasm a bit. He originally sequences it as a 40m all-blind drive, but I'm uncomfortable with that. The drive sequence is mainly made up of little helpers that check our heading, then take a 60cm step. If we're aimed too far to the left, the step veers a little to the right; if too far to the right, it veers a little to the left; and if we're close enough, it just goes straight.

The problem with this is what happens in the marginal case: you're aimed off course, but not so far that you veer back on course. This keeps you inside a cone centered on the desired heading, but then you have to be sure that the cone doesn't include anything scary. I put together a version of John's sequence that shows the worst case -- what happens if we skate along the extreme left edge of the cone -- and display it on the big projection screen. All kinds of bad stuff is in there -- a sand trap, some big rocks, you name it. He takes one look, hems and haws for a minute, and does the right thing. We scale back to a 23m blind + 15m autonav drive, with tighter limits on the helpers.

Coming back from lunch, I bump into Mark Adler in the elevator. "How'd it go with the Congressman?" he asks.

"Oh, I thought it went pretty well. People seemed to think so."

"Good," he says. "I was talking to Gail Robinson about you." He sees my blank look. "She's Elachi's assistant," he explains. "She was there that day when he brought his tour group through. Yeah, I was talking to her about the Congressman's visit, and she asked me, 'Who was that guy we talked to?' I told her you should do it."

That explains a lot, and is actually pretty close to what I figured. "I wondered what happened with that," I say. "I was like, how would Charles Elachi even know who the hell I am?"

"He knows now," Mark says.

Spirit Sol 734

I woke up at 3AM with a bad feeling about the drive. I logged in from home, checked out the data, and sure enough, something bad happened. This would sound impressive, as if I'm psychic or something, were it not for the dozens of times that I've done exactly the same thing and the rover's been perfectly fine.

But she's not fine this time. Most of the automated post-pass processing isn't done yet, so I have to fumble around in the low-level data for a while. But I work out that Spirit had executed the 40m blind waypoint just fine, but when she turned to set up for her first autonav waypoint, she experienced some kind of motor fault.

Oh, this is bad. This is genuinely and truly bad. The only thing I can remember that's caused something similar to happen is when we picked up a potato while climbing Husband Hill, and the pain and the time it took to recover from that -- it doesn't bear thinking of. And we don't have time for an anomaly recovery now, considering how far behind our drive metric we are.

To get a clue as to exactly what might be wrong, I grep the past data for the warnings I'm seeing here. They turn up twice: once on sol 265, and once on sol 277. I check my notes to find out what was happening on those sols, and I'm almost relieved to find that it wasn't potato time, it was the dynamic brake anomaly. What happened then was that Spirit lost the ability to sense the state of some of the brakes that keep the wheels from wobbling (uncommanded steering) during driving. And since her FSW couldn't get responses from those brakes, she freaked out. Eventually, after exhaustive analysis and testing, we decided that the brakes were working fine -- she just couldn't tell they were fine after commanding them -- and sent commands to ignore their state. And then I just forgot about it, because everything has been working fine since then.

The brakes work in connected pairs: one relay controls the RF and LR brakes, another controls the LF and RR brakes. Last time, the first of those pairs failed. This time, the symptoms are the same except that the motor IDs are different, indicating that the other pair has failed.

If we were going to have a problem like this, this might be the best problem we could have had. We'll have to do some analysis, but the odds are that we'll just do the same thing we did last time: tell Spirit to continue commanding the brakes but ignore their status, and life returns to normal.

Oh, please, oh, please, oh, please.

At least it's not my fault. This doesn't mean I don't get ribbed for it. As Chris Leger walks into the SOWG meeting room, he rolls his eyes at me. "You just had to break the rover, didn't ya?"

"I know," I say mournfully. "You give it to me for one day, and look what I do to it."

(He also points out a 25cm rock that we juuust missed. The tracks skate right by it. We could have gone over it, but it's alarming to have come this close to something that's right at our danger threshold. I look back at yesterday's PCAMs, pick out that rock, and measure it. I feel vindicated when I see the same number I saw yesterday: 19cm. "Yeah, I don't trust those rock heights," Chris says, looking over my shoulder. "I'm starting to see why not," I reply shamefacedly. It's possible the rock isn't actually that big; all we have are the HAZCAMs at this point, and their range data is sketchy. But still. As bad as things are, this is a vivid reminder that they could have been much, much worse.)

Chris is RP-1 today, and Ashley's RP-2. I'm neither -- I'm off shift today -- but I can't bring myself to ignore what's going on, so I decide to stick around for the SOWG meeting and the subsequent, hastily assembled anomaly meeting.

Options on the table at the SOWG meeting are to do mobility diagnostics and remote sensing, or mobility diagnostics followed by a drive. The latter option might sound surprising, but it's not terribly so: if Spirit fails the diagnostics, she'll raise an error and refuse to drive; if she passes, why not let her drive as far as she can get? We are, after all, 100m behind our drive metric, and 200m from Home Plate; we need to make all the progress we can. However, this isn't the place to make that decision.

That comes at the anomaly meeting. It's clear what the problem is, and we've been down this road before. (Or so we think, but one of the assignments coming out of this meeting is to ensure that this really is the same problem as before, as opposed to something else that's manifesting in a similar way.) We won't be able to do all of the diagnostics we'd like today, as we need to do some testing in the testbed first. (Luckily, Chris had the testbed reserved today anyhow.) But, somewhat to my surprise, we get the go-ahead to drive. We need to make sure we won't hit a sandy patch and dig in, but "with the appropriate bells and whistles," as John Callas puts it, we're cleared to drive 20m or so. Anything to get us closer to Home Plate!

Steve Squyres, dialed in on the telecon as usual, asks perhaps the most important question of the meeting. A scarred veteran of the ongoing Opportunity IDD anomaly, he asks, "Are we confident we can keep the program office and NASA headquarters out of our faces on this one?"

Callas cups his hand behind his ear. "Sorry, Steve, you're breaking up," he jokes.

I do my little part by looking into the timeline of events the last time we went through this -- what responses did we have, and in what order? Just as I'm finishing that, I get pulled into helping with the drive sequence.

Did I mention that I'm not on shift? I had other stuff I meant to do, but oh, well, who am I kidding? I love doing this.

Most of the rest of my day is taken up in another meeting, the Opportunity IDD science meeting. Two years and one day, it's been, and we're discussing how to drive with our gradually degrading arm. This meeting is to get the science team's buy-in on what we should do. The slides make the point, in large red letters, that we're time-limited, not usage-limited -- the IDD is going to fail at a certain date in the future whether we use it or not. The four options Ashitey presents are these:

1. When driving, stow the IDD, drive, then unstow again. This avoids leaving the IDD stowed during a thermal cycle (i.e., overnight), and has a narrow but nonzero window of vulnerability where the IDD could cease to work during the stow and be permanently stuck somewhere under the vehicle. This could block the FHAZ's view of the terrain, and worse, as the arm bounced around during drives, the elbow would gradually drop farther and farther until we were dragging the IDD on the ground like an anchor.

2. Drive in the thinker-stow configuration, as we did the last time. This will badly limit mobility -- best case, we could drive over 7cm obstacles, and our poor ability to reliably detect obstacles of that size would limit drives to 5-7m per sol.

3. Combine options 1 and 2: use option 1 for long drives, 2 for short drives.

4. Stow the IDD and never use it again.

"I don't think option 4 will be the favorite of the scientists," Ashitey jokes.

There's hardly any argument; everyone on the science team favors option 3. (As does the engineering team, but it's not our meeting.) Ray Arvidson's response is the most poetic: "Think fifty years into the future and think about our legacy. Our legacy is going to be the major stratigraphic sections we've done -- at Eagle and Endurance, and now this one in front of us. We have to protect mobility, and not at the rate of five meters per sol."

"Even if the IDD fails, we still have a bang-up mission in terms of remote sensing," he adds shortly afterward.

Squyres sums it up: "We all hate option 4, and option 3 combines the best of 1 and 2. As Jim Erickson said in the anniversary celebration the other day, our job now has become to use these rovers up. That doesn't mean we can be cavalier about using these vehicles, but we don't get any points for ending this mission with a healthy vehicle on the surface of Mars."

This is all correct, but depressing. All good things must come to an end, but is it so wrong to want them to live forever? Or at least until I get sick of them?

Not that there's a difference.

Spirit Sol 733

The thing is, I'm not even supposed to be working today. I originally had a doctor's appointment, which I rescheduled to next week so that I wouldn't miss the big Two-Earth-Years-On-Mars celebration over at von Karman Auditorium. It was supposed to be a big deal; the Lab's making a fuss over it, inviting press and congressmen, and so on. And there's usually free cake at these things. So, you know. Cake.

Then, last week, I got a call from John Callas. "Are you working next Tuesday?" he asked.

"Nope," I told him.

"Okaaay," he said slowly. "Well, you know that Congressman Drier is coming next week for the anniversary celebration? We wanted someone to help him pick out a science target, and Dr. Elachi requested you specifically."

Well, if the Director of the Lab wants me specifically .... Wait a minute, I think as I hang up with John. How would Charles Elachi even know who the heck I am? What I figure is, he's remembering a few weeks back, when he and Pete Theisinger brought some tour group through and I gave them a spiel. He probably said, "That guy did a pretty good job -- whoever he was, use him."

However it happened, I'm now on shift on Spirit. (And I'm getting teased a lot about being famous. Worse, the story has now mutated -- Charles Elachi asked for me by name, according to the new version, which is not how I heard it, anyway. But let's leave all that aside.) The plan for Drier's visit crystallizes into having him choose which of a pair of science targets we're going to take an image of before we drive today. (It was originally to choose between a couple of IDD targets, but we're not IDDing today, so that's out.) Taking these images is, of course, not my job, but as Sharon Laubach says, "What we're going to do is: fake it."

I can do that.

I talk to the science team and get a sense of the two targets we're going to have the guy choose between. The first is a patch of light soil -- indicating maybe salts, which of course are mobilized by water, which is what we're all about. The other choice is a pair of nearby rocks, which Larry Crumpler thinks might represent the leading edge of an ancient lava flow; studying them might give us clues about the properties of the lava and of the paleoclimate on Mars at that time.

The sequencing itself is pretty straightforward -- a relatively simple drive. And that's a good thing, because one way or another, the Congressman's impending visit keeps interrupting.

First, Jim Erickson comes in, followed quickly by John Callas, Pete Theisinger, and Fouk Li. They ask me to go over what I plan to show Drier.

"Well, I'm gonna do the whole spiel about where we are, how we work, and so forth. Then I'm gonna show him the first picture, with the salts, and then the second picture, with the two rocks, and then invite our guest to choose which one we should take the picture of."

Jim cocks an eyebrow at me, playing the Congressman's part. "So you're saying, no matter which one I choose, the scientists will be unhappy?"

"I'm saying, either way, the scientists will be equally happy," I reply smoothly. "They're both excellent choices."

That seems to work. They flutter a bit more, make a couple of good suggestions, and then disappear again.

I get back to work, and the next thing I know, it's coming up on time for the anniversary celebration and Congressman Drier's still not here. I'm not fretting; I've got plenty to do. Besides, he's one of the main celebrants -- a star of the show. They'll wait for him.

But they are trying to keep from getting too far behind, as I discover when Callas pops in. "Uh, we're running kind of late," he says. "Try to keep it to about thirty seconds." He pops out again.

I can't help but laugh about this. Whatever they want, I figure.

At last, Congressman Drier and his entourage show up. One guy's from the LA Times, another's taking pictures (I assume for the press, though I don't know). One is Sally Ride.

I do the usual speech, explaining what they're looking at, how the commanding process works, and so on -- trying to keep it short, although of course I have no hope of keeping it down to thirty seconds. Still, they don't seem terribly anxious to leave, which I take to be a good sign.

I get to the point about choosing between the two images, and I start this way. "Just as in Congress, you can't spend every dollar you want, so it is that we can't take every picture we want."

"Ah, we just spend all of 'em!" Congressman Drier says.

"Uh, you do remember that the LA Times is here, right?" I tease him.

He doesn't want to choose either picture, showing his political instincts, so we agree to flip a coin. The rocks win. I talk to them a little more about where we'll go from here, answering questions from the congressman and his group, and they move on.

As they're leaving, I take my chance to say, "The Sally Ride?"

She grins. "The Sally Ride," she confirms.

I don't meet a lot of people with cooler jobs than mine, but the first American woman in space is on the short list. "I remember watching your shuttle go up," I tell her. "That was awesome."

She grins again. "It was cool for me, too," she says.

Overall, I thought the visit went quite well, and I'm not the only one. As soon as our guests are out of earshot, people start congratulating me. Including Squyres, who I didn't realize was listening in on the telecon: "Good job, man," he says. "That was really well done!" Coming from someone like Steve Squyres, that's high praise indeed.

So then we all troop down to von Karman for the festivities and cake. They turn out to be filming a TV show, with multiple cameras, including one on a boom that gets those swooping shots of the audience. We are of course late -- although the Congressman and his crew are still not there, and don't show up for another 20 minutes or so -- so I end up standing at the back, with Mark and Ashitey and Ashley and Craig Leff. Waiting is not so bad, though; on the big video screens at the front, they're flashing congratulatory messages emailed from people around the world. One says that she gets so lost in looking at the images sometimes that she automatically starts planning hikes in them. "I have to remind myself that these images are on Mars!"

"I and my ten-year-old son have been following the progress of the two rovers every day since they landed," writes another woman. "Thank you."

When the thing gets started at last, it's pretty much what I would have expected, with Elachi and others saying that it's not the machines, it's the people, and so on. And the mayor of La Canada Flintridge presents us with a proclamation from the city, as well as taking a few good-natured jabs at the mayor of Pasadena (who sent a similar proclamation in absentia). In between, they show video segments that I suppose will form the main part of whatever this TV show ends up being. I'm even in some of them -- in the foreground, yet; they used some of the video footage they shot when they were putting together B-roll for the CBS Evening News. Every time I show up on the screen, the people around me rib me about it. It's cool, though.

One of the speakers is none other than Steve Squyres, who couldn't be here in person, so they put up a picture of him and pipe in his voice via phone. "Most of the team's so used to me being just a voice on the phone that this is really the most fitting way to do it anyhow," he jokes.

His little spiel is about teamwork. "What's really made these rovers succeed is teamwork. In fact, I was just listening in as one of our rover drivers, Scott Maxwell, was talking about a couple of science targets to Congressman Drier. Now, he's an engineer. But his explanation of the science was as good as I could have imagined. And that's the kind of teamwork I'm talking about, between the scientists and engineers."

Aw, shucks.[1]

Today has got to rank among the best days of my life. It's so great, I don't even care that in the end, there's no cake.

---

Footnotes:

[1] I might as well admit it. I was ... there's a Yiddish word for this, kvelling. It's when you're so happy there's a light in your tummy and it's shining out of you no matter what you do about it. I was kvelling.

Opportunity Sol 712 (Spirit Sol 732)

"I think I owe you a beer," says Larry.

"You got lucky," I tell him, "I don't drink."

He's referring to the fact that our drive wasn't quite perfect. While we ended up almost exactly where we wanted to go -- and we still have an IDD -- Upper Overgaard was just slightly out of the IDD work volume. However, the next-priority targets, located on Lower Overgaard, were reachable, and Opportunity did some IDD work on them last week.

And she was supposed to do more over the weekend, but she spazzed out again. Just at the end of the first IDD mosaic, when she was all of one milliradian away from returning to the ready position, she experienced another joint stall. To make matters worse, it was a joint-2 stall -- not a joint-1 stall, as we have been seeing -- and nobody seemed entirely sure what to make of that.

So they did some diagnostics over the weekend, and the good news is that Opportunity passed those with flying colors. The general feeling is that there's nothing to panic about and we should just proceed with IDD work today. We obviously want some diagnostic info, but in this case there aren't any diagnostics that would be materially better than just continuing the IDD work, and obviously we get science as a bonus with that.

That is, that's the general feeling among the engineers in the meeting we're having about it. And it's echoed by the scientists. As Matt Golombek puts it, "Upper and Lower Overgaard are one of the highest-priority science targets we've had for 200 sols. Even if we knew we had only five uses of the IDD left, this would be one of 'em."

So with science and engineering in complete harmony, we should be all good to go. But as usual, things are more complicated than they have any right to be. For one thing, it's not immediately clear that we're going to be doing any planning on the rover today. Because of the unusually high winds and accompanying wind-blown debris here at JPL, people are now being turned away at the gate. That apparently has included data management people, and Opportunity's tight on flash, so without them, we might not be able to do anything useful thisol.

That problem gets solved when Jake tracks down a data management guy who arrived early -- he's not technically on shift but agrees to help out. But there are other problems as well. Chief among these is that we need approval from NASA HQ before we can proceed, and we haven't gotten it yet. They spend some time rehearsing their arguments, to maximize the chances that HQ will accede. Joe Melko makes the point that what's killing the IDD is simple thermal cycling, not usage. Whether we use the IDD or just sit here and fret, its lifetime's the same. "You shouldn't be afraid of using it, you should use it as much as you can while you still have it."

"Use it or lose it," Squyres paraphrases.

"No," I point out. "Just 'lose it.' Whether you use it or not."

On the assumption that HQ will show good sense, we go ahead and plan for the resumption of the IDD campaign. The sequence has already been built, although it wrongly used 26mm (not 23mm) for the MI best-focus position after the MB touch that precedes each stack; that's something I'll have to change, but it's trivial to do. However, before I can really sink my teeth into it, Squyres is on the telecon.

"No IDD," he says flatly.

Everyone groans. We've just finished the APAM, and we now need to replan everything, as well as losing another sol to this problem for no good reason.

He seems to feel he owes us an explanation. "Basically what happened over the weekend was, people at JPL who didn't know what was going on made promises to people at NASA HQ who didn't know what was going on, and promised we wouldn't use the IDD at all until HQ was brought up to speed." And for some reason, the woman at HQ we need to speak to is unavailable today.

"This is ridiculous!" Matt Golombek fumes. And he's not the only one.

"Hey, feel free to vent, guys," Steve says. "Callas and I just yelled at each other -- I yelled at him and then I felt better, and then he yelled at me and then he felt better. But at this point, promises have been made that can't be unmade."

We can all see the justice of this, as frustrating as it might be. I (and others) thank Steve and John for trying anyway. And then they replan the rover's day, and, I'm sorry to say, it won't include me.

Opportunity Sol 707 (Spirit Sol 728)

Ah, that's a relief! Our second try at stowing the IDD in the "thinker" pose worked! The images look just as predicted, and the telemetry shows that the IDD's final position is spot on.

Now we just have to drive it. We've already put in a couple of days' work on the drive sequence, which is a good thing, because this one's going to the testbed for sure, later today.

Even with that testing, Ashitey expects me to be pretty worried about this -- "You won't sleep well tonight!" he laughs.

Won't I? I'm not so sure. "I'm not really that worried about it," I say, realizing it's true even as I say it. "I already told everyone it might take us more than one sol to do this, so it's okay if the target's not reached at the end of this drive. I mean, at this point, we just want to demonstrate that we can do a successful drive without tearing the IDD off." Maybe I shouldn't have phrased it that way, but it's essentially accurate.

Then again, Opportunity's been sitting still for almost 60 sols. Purgatory Ripple was only 40. As long as we get the damn rover in motion, I think some people wouldn't mind if we did tear the IDD off.

But that would disappoint Ashitey and Frank, who are hoping to use it tomorrow to start the 7x7x5+stereo MI mosaic of Upper and Lower Overgaard. (Well, maybe it would disappoint some of the scientists, too.) If we end up doing the whole thing, it'll be the largest-ever MI mosaic of a single object on Mars (obliterating one of my records -- the 6x4x5+stereo on Keystone). "I'm happy I'll be doing the first drive with the arm in this weird pose, but I was hoping to be in on that mosaic," I tell Ashitey.

He commiserates. "If you want it, you won't get it. That's how it works." He feels my pain, since he's been missing out on the cool Spirit drives we've done while he's been struggling with this IDD problem. "They promised me the last half-meter to Home Plate, though," he jokes.

Eventually, our testbed time rolls around, and Ashitey heads down there with Jeng to test our sequence. I stick around the sequencing room for a bit to handle the walkthrough, but then I can't stand it any more -- I have to go down there, too. Before I leave, Rich, the TUL, asks when I think they'll be done in the (notoriously time-eating) testbed. Who the hell knows? I shrug, look at the clock. It's a little after 3. "Uh, 4:27," I tell him.

By the time I get to the testbed, they're just starting to run the sequence. Amazingly, the thing runs all the way to completion without a hitch -- even the tight suspension limits work, at least in the testbed. As we're finishing up and preparing to head back up to the sequencing room, I check the time. 4:25.

I wait two minutes before calling them to tell them we're done.

Despite our success in the testbed, I'm actually somewhat pessimistic about the drive. Larry Soderblom asks me what I think are the odds it will succeed, with Upper Overgaard in the IDD work volume. "Fifty-fifty," I tell him. Then I can't resist completing the old joke: "Either it'll happen or it won't."

"I bet you a beer it'll work," he says.

I have a policy: never bet against the rovers. Besides that, I don't drink. So I'd have nothing to look forward to: either I "win," and get a beer I can't drink, or I lose, and have to buy him a beer. Nevertheless, in the spirit of camaraderie -- what the heck, just this once, I accept the bet.

[Next post: sol 732 (Opportunity sol 712), January 23.]

---

Courtesy NASA/JPL-Caltech. The IDD in the "thinker-stow" position.

Opportunity Sol 706 (Spirit Sol 727)

Opportunity's start time is sliding later, thanks to the Earth-Mars day-length difference, but it's not late enough yet that I can actually come in to work any earlier -- between 9:30 and 11:30, there's no parking to be had.

As a result, I'm in a couple of hours before we actually start, and this gives me time to check out Ashitey's meeting with Erickson, Callas, Eric Baumgartner, Joe Melko, and so on, about the current state of the IDD investigation. The first question is whether we should finish the mini-Martian-tai-chi (our cute name for the recalibration activity we were doing) before stowing. Nobody wants to bother; they just want to get the damn arm stowed so we can drive. So much for that.

Next question: is it OK to increase the rotor resistance for the duration of the stow? This resistance value is the thing that lets us work around the broken motor winding, and which had been preventing stalls until this weekend. Eric's take on this, which mirrors email he wrote over the weekend, is that this will be fine. They'd worked out the present value partly empirically, and everyone's willing to believe that we simply needed a little more margin in that number.

What else do we need to do to get ready to drive? We won't be driving today -- we're just going to stow the IDD in its new stow configuration -- but it's what everyone's gearing up for. First, we have to make sure we're not driving over anything too big. Ashitey negotiates a slight relaxation of the definition of "too big," but we're still left with this rule: the front and middle wheels can't go over anything bigger than 3cm; the rear wheels, 4cm. This is about what I'd already planned for, so no problem there.

Next, we have to set up the new stow parameters. The FSW's Fault Protection code has a feature where you can tell it what joint angles represent a stowed IDD and have it check constantly whether we're in or out of bounds; if we're out of bounds, it will stop the drive. We never used this on Mars, because when the IDD is stowed the normal way, it hangs securely onto a hook. But we'll be setting it up today.

Finally, Ashitey suggests reducing the wheels' max speed by 66%, on the theory that that should make us come off of an abrupt drop (if we screw up and have one of those) a little more gently. To my surprise, this is rejected: the rovers already move so slowly that there's little to be gained by this.

Oh, and we have to clear the errors left over from the most recent stall fault. And Jim Erickson asks me to add a rule to our flight rule checker that warns if you send the "stow IDD" command on Opportunity, since what that command does is no longer what we want to do.

That's all for the meeting. As was the case Friday, the sequencing itself is simple. Today it's even simpler, since we're basically just repeating Friday's sequence (after clearing the errors and increasing the rotor resistance). But, once again, it's worrying about the upcoming drive that consumes the greater part of the day.

First, Ashitey relays a concern from the science side about closing the APXS doors. You close the APXS doors by rotating the turret to one of the extremes, and in the new stow configuration, we're commanding it pretty close to that value. In addition, the wrist and turret have more "play" than the other joints -- they'll jiggle around while we're driving -- and we might therefore end up crossing over the line we're starting so close to. This is worrisome because the doors are so hard to get open once they're closed. That wasn't always the case, but we started having trouble with it on the surface, so we decided just to leave the doors open permanently. What happens if we inadvertently close the doors and can't get them open again? We effectively lose that instrument, and with it the ability to sense the chemical composition of rocks and soil.

But we're dealing with a failing mechanism here, and we need to make some hard choices. And that's the risk we decide to take.

We also spend a considerable time working out the suspension limits. We frequently set these limits, especially on Opportunity (come to think of it, I don't know if we've ever set them on Spirit), to help us decide whether we're crawling over something bigger than expected. So in theory, we should be able to just work out how much a 3cm or 4cm rock would articulate the suspension, poke in those limits, and proceed with a bit of extra insurance.

The problem is that, as with so many things we do, this is beyond the limits of what the hardware is rated for. The potentiometers that sense the suspension aren't accurate to more than 2 degrees or so -- they're not intended for this level of precision -- and a 3cm rock changes the rocker differential by a little less than that. Ashitey wants to be fairly generous with the limits, but I don't. "We've already sold everyone on the concept that this might take us a couple of sols here," I point out. "We might as well be conservative." We add a little margin, but it's still possible we'll stop for no good reason. So be it. I'm getting paranoid.

By the end of the day, we've got a pretty decent drive sequence. It's not terribly complex; it's another one of those sequences with a high discussion-to-command ratio. But that's okay. We're doing something new and weird, and we can screw things up that we could never fix again. So the more talk, the better.

But first let's just hope we can stow the damn arm, so that we can find out whether this even works.

Opportunity Sol 702 (Spirit Sol 723)

Because Monday is MLK Day, we're doing a 3-sol plan this weekend. We expect it to be fairly simple, if a little scary: we're putting Opportunity's IDD in its new stowed configuration today. This is the "Rodin's Thinker" pose: the IDD will be swung up over the solar panels, elbow-out, with the turret resting about 5cm over the left front solar array hinge. Ashitey's worked everything out in the testbed, where they've even tried driving in this configuration. But it's scary all the same.

Actually, just putting the IDD in that pose won't be scary -- driving like that will be.

Anyway, our day will be relatively simple. Unlike Spirit's. Spirit has turned around to IDD the chewed-up Paso-Robles-like stuff she turned up while climbing Husband Hill. This stuff appears to be salts, and since salts are mobilized by water -- you know, the stuff we're here to find evidence for -- most of the science team is hopping-up-and-down excited about it. (Though not Larry Soderblom, who dismissed it in email as "bright dirt." He thinks we should spend this time zooming toward Home Plate, as we're increasingly behind schedule. I can't say I think he's wrong, but I'm no scientist.) And they're doing one hell of an IDD campaign, some twelve targets or so over the 3-day weekend.

John Wright's the RP-1, so I have no doubt it'll get done, and get done well, too. But people keep asking me, "Aren't you glad you're not on Spirit?" And I have to tell them no. Truth is, I wish I were John. The harder the task, the greater the glory.

I guess I'll have to let him have a little glory.

Since we have a lot of time, I go ahead and get to work on next week's drive. I expect a lot of scrutiny, so the earlier I get a handle on this, the better. Happily, Ken Herkenhoff is out here, so I discuss the bump target with him and Wendy Calvin. They want us to get to a pair of rocks -- "Upper Overgaard" and "Lower Overgaard" -- that are slightly ahead of us and to the right. Before going there, we need to back up 1m and image the outcrop we've been parked in front of since the IDD anomaly.

Normally, this would be trivial: turn on visodom, back up, take our pictures, turn, and bump forward to the new target. But, oh, the restrictions we labor under now! In addition to the limitations we already have when driving Opportunity -- the requirements for periodic slip checks and the problems induced by the failed RF steering actuator -- we have new restrictions caused by the IDD's new stow configuration. In particular, we've decided we can't tolerate a drop of more than 3cm, as this might cause the arm to jounce too much and whack the turret against the solar panel. So we can't drive over anything bigger than two or three centimeters -- in common units, about an inch.

You think there will be anything bigger than that between us and Victoria Crater, still some kilometers away across Meridiani Planum? Naaahh ....

And, of course, since we're also restricting the IDD's azimuth -- to ensure that we can still drive even if it fails altogether -- we have to really nail the drive.

And there are a couple of obstacles of just the right size, right where we need to go. Of course. (And there is no position that gets both Upper Overgaard and Lower Overgaard into the new, sharply narrowed work volume, without doing something like driving way around and approaching from a different direction. Which we won't.)

So I have to plan the drive very, very carefully, and one crazy thing I come up with is not to use visodom. This is something of a gamble, but I think it's a sound one. Our slip is likely to be low, and it will carry us mostly away from the many 3cm-or-larger "obstacles" scattered all around us. It might mean we need an additional sol for fine positioning, but I can sell that. Everyone's so happy to be moving at all, they'll buy it. The alternative is to risk a bad visodom position update that would cause the rover to drive farther than it's supposed to, climbing the obstacles and possibly forever damaging the arm.

Did I say I wanted this to be hard, to increase the consequent glory? I take it back. Less glory, please.

Just as we're wrapping up for the sol, about 7 PM, we get the downlink from yestersol. I take one look and sigh. "Bad news, everyone!" I announce. "The last sequence faulted out near the end, during the return to the ready position." I hope this is another joint-1 stall, as opposed to a completely different problem, but that data hasn't come down yet. "So the stow sequence we built today won't execute over the weekend. Worse, the MB is aimed directly into one of the HAZCAMs." This is a condition we're supposed to avoid; the instrument's radiation source can damage the cameras.

So should we try to recover? Luckily for us, Jake Matijevic is still here. Jake gives us a pass on the MB situation, pointing out that the MB's radiation source has degraded significantly since we got to Mars, and at its current distance from the cameras, it should no longer be a threat to leave it like this for a few days. Emily decides not to reassemble the team at this point to try to fix the problem, especially because, due to missing data in the pass, we don't even know exactly what the problem is yet.

So we go home. I can't help thinking that one consequence of this fault is that it'll be at least one more sol before we can stow, so at the earliest we'll drive Wednesday instead of Tuesday (the day after the holiday). Depending on the analysis, we might have to push the drive later than that. And since I'm on Opportunity only Tuesday and Wednesday of next week, that means I might not get to do the first Mars drive in the new stowed configuration.

Hey, I want a little glory. I'm only human.

[Next post: sol 727 (Opportunity sol 706), January 18.]

Opportunity Sol 701 (Spirit Sol 722)

This was not a big sol. We were planning to do some RAT bit imaging, but that got scrapped at the SOWG meeting.

"Just so you know, we need to do the RAT bit imaging before the next time Opportunity uses her RAT," Emily Eelkema pointed out.

But, somebody replied, Steve's position was that "the next time the RAT comes out will be at Victoria Crater."

"I guess we've got some time to do the imaging, then."

And that was about that for that.

Still, we at least had something to do -- namely, we did more tai chi. The latest results showed about a 7mm discrepancy between the predicted position and the position computed from the images -- within our error budget, sure, but there's room for improvement all the same. So thisol's big IDD activity will be to move to the image-computed positions, take images there, and run them through the stereo correlation again to help with the (potential) recalibration.

It's not much to do, and delays elsewhere in the process make it a fairly slow day. This is a good thing, as it gives Matt Heverly, who's shadowing Ashitey thisol, a chance to really dig into the sequence. Indeed, Ashitey has Matt pretty much build the sequence from scratch, then disappears. Matt and I take the time to go over everything in the sequence in great detail, a good learning experience for him (I hope).

Life in the Land of Opportunity will stay slow until we start driving again, which could be this weekend but likely won't be until next week. Our next drive target is Overgaard, part of the same outcrop we're studying now -- the drive will of course be a very short one. But it's still important and interesting because PCAM imaging revealed "festoons," little smile-shaped ledges that are strong indicators of past liquid water. So compelling is this evidence that Emily refers to Overgaard as "the Holy Grail of Meridiani Planum."

"I thought that was Victoria Crater?" says Wendy Calvin.

"That's too far away," Emily says simply. "Let's focus on the present."

Victoria Crater is indeed far away. But so were the Columbia Hills when we started driving to them. Behold the triumph of persistence.

Opportunity Sol 698 (Spirit Sol 719)

Ashitey's already seen the IMAX movie -- he saw it this weekend. "It was okay," is his review, though he's more enthusiastic later when discussing it with Squyres. They both got to see it because they helped -- Steve, by being in it; Ashitey, by running the testbed's RAT and PMA actuators so that the movie could use realistic sounds.

Opportunity (I'm back to the other side of the planet) doesn't have much to do today. We're not driving yet; that gets pushed back about two weeks every two weeks. No real science-related IDD work, either, though we're doing a miniature version of the "Martian tai-chi" IDD recalibration. Not because we really think we're out of calibration, but because people keep asking Ashitey about it at the IDD anomaly meetings, and he wants to have something to tell them.

So it's a pretty easy day on Mars.

Before leaving, I get into a discussion with Chris Leger about the Spirit driving plan. We've been trying out these two-sol drives with an autonav-only drive on the second sol, but they haven't been working out too well, and he's worried that the time we spend on them gains us little and costs us a lot of attention that would be better spent on the first sol's drive. I don't quite disagree with this, but I think he's a little too pessimistic. All of our techniques start off with large investments and little initial return, but we rarely regret developing them. I see this as being another instance of the same deal. He doesn't quite disagree with that, but we both have the "I don't disagree, but let's both agree that I'm right" disease, so the conversation limps along a lot longer than it should.

Fortunately, it's interrupted by a phone call from my parents, telling me excitedly that they just saw me on the CBS Evening News. (It still hasn't aired out here, so they also want to let me know so that I can catch it.) I filmed an interview with a news crew last week, and I was just going to record it on the TiVo and send DVDs to the family as an FYI. But it turns out they caught it when it aired. Funny thing: when I'm on the phone with Dad and Trish, Roxanne[1] calls to tell me the same thing. Who knew that many people watched the CBS Evening News? Apparently, it's some kind of big deal or something.

When I get a chance to see the interview myself, I'm relieved. I didn't really feel that it went all that well, but the piece came out great, and I even look like I know what I'm talking about. Good editing, I assure you. I expected the two-and-a-half minute piece to be two minutes and fifteen seconds of Squyres, then ten seconds of Nicole (whom they interviewed about naming science targets) and one shot of me saying "It's great!" It turns out to have less of Squyres, more of me, and -- erk -- no Nicole. Well, that sucks, but it's America's loss.

[Next post: sol 722 (Opportunity sol 701), January 13.]

---

Footnotes:

[1] My then-mother-in-law.

Spirit Sol 714

"Our objectives today," Steve announces, "are to drive like the wind and then get the uplink team home in time to watch Texas beat USC." It's Rose Bowl Day, and Steve, in case you hadn't guessed, is a Texas fan.

Spirit made a respectable showing for her second Earth-birthday: 65m, most of it in the direction of Home Plate. It's not as terrific as I was hoping for, but it's sure not bad. Slowly but surely, the drive-metric graph is starting to look less scary.

She made it a good chunk of the way through the dune field that yesterday lay between us and Lorre Ridge, but a slip check measured 70% slip -- apparently real, not a visodom artifact -- and stopped the drive a little earlier than it would have otherwise.

Looking at our tracks, we can see that we really are slipping a lot. Rather than leaving nice, widely spaced cleat marks, the wheel tracks are all bunched up. However, we're not digging in, so Chris and I agree that it's safe to drive away.

Which will be fun. The path out of the dune field is relatively obvious, and firmer terra (marsa?) is only about 12m away, but on the way is a 30cm rock pile we definitely don't want to climb. With that, and the fact that we're slipping so much, there's only one way out: visodom. It'll be slow, but we've got a whopping four hours to drive, so we've got plenty of time for it. And after that we have a short blind segment followed by autonav until the time-of-day limit. (I'd rather spend the four hours on an 80m blind drive followed by a three-hour autonav segment, but we aren't in that kind of terrain. Mars wins.)

And there's something we need to do before we even get on the road. For the last few sols, we've been getting intermittent stall warnings from the left front wheel. We got another batch yesterday, and Matt Heverly and Terry Huntsberger work out that it happens whenever autonav decides to take a sharp left, meaning that that wheel is steered almost to the extreme counterclockwise position.

This probably isn't a failing actuator, but it could be. More likely, it's either a miscalibrated actuator or some junk physically jammed into the wheel. Hoping for the former, we start the drive with some commands to recalibrate the wheel, steer it to the troublesome position (to see if we get more stalls), then straighten the wheels and install new soft-stops that will prevent Spirit from steering that wheel back to the position where we get the warnings (for the rest of the day, giving us a chance to look at the data before she tries to go there again). If we see no warnings tomorrow, then the recalibration likely fixed it. If we do see warnings, well, we'll make the new soft-stops permanent. The new limit is only about one degree away from the old limit, so that won't compromise Spirit's driving appreciably anyway.

All the same, I'm hoping it'll go away and stay gone. One rover with a bad steering actuator is plenty.

[Next post: sol 719 (Opportunity sol 698), January 10.]

---

Courtesy NASA/JPL-Caltech. Spirit's tracks look a little closely spaced -- this is different terrain, probably with somewhat looser soil, than we normally drive her in. Still and all, though a little worrisome, it's traversable.

Spirit Sol 713

I couldn't resist checking in on the drive from home last night, and I was somewhat disappointed. Shortly after starting the autonav segment of her drive, Spirit found herself in a cul-de-sac. Her existing path planner isn't smart enough to find its way back out when this happens, so she just struggles with it for a while and gives up in frustration. Total progress: 50m or so -- respectable, but much less than we hoped for.

"What are the chances we'll get outta the cul-de-sac and make progress on sol 712?" Steve asks me just before the SOWG meeting starts. That's the final sol of the weekend plan, whose data should start flowing in in half an hour or so.

"Slim," I answer. "Less than ten percent."

"Well, that's Mars. The plan for today and tomorrow is gonna be drive, drive, drive."

"Well, if you insist," I shrug as he laughs.

When the data arrives at last, it confirms my prediction. I'd have been happier to be wrong, but so be it. I don't have much time to worry about it, not only because we've got a sol to plan, but also because today's Spirit's "Earth-birthday," as Brenda calls it -- the second Earth-year anniversary of her landing on Mars. I still have trouble believing it -- not just believing that they've lasted so long, but also that I've been on mission operations for this long.

So we interrupt sequencing long enough for a small celebration in the SOWG room. Jim Erickson and Charles Elachi and Pete Theisinger and Steve Squyres all say nice things, and thank and congratulate the rest of the team, and then we're back to work.

An occasion like this one is a time for personal reflection, too. It's so corny to say it, but I've grown and changed a lot in those two years. I can see it in looking over some of my earliest notes on ops. I never thought of myself as lacking self-confidence, but back then I was ... well ... I'm different now. These rovers have changed me, and they've changed me for the better. They're old friends to me now. We've been through a lot together.

What better way to celebrate than by dodging some rocks, putting the pedal to the metal, and lighting out for Home Plate? So I do that -- or, rather, I ask Spirit to do it. As far as I'm concerned, there's really no difference any more, if there ever was.[1]

---

[1] Happy birthday, Spirit, my love.

Spirit Sol 710

Today we're planning three sols. For a long time now, when we've had three sols to plan, we've usually done it so that the rover drivers have only one sol to plan -- at most, two. Today we have stuff to do for all three sols. The first sol, we're continuing IDD work on the El Dorado scuff; on the second, we stow and drive; on the third, we do an autonav-only continuation drive.

Which is kind of insane. But, as Larry Crumpler points out at the SOWG meeting, we're 300-400m behind our drive metric. You know, half of our original mission-success drive metric, more or less. Nothing major. So we need to squeeze in as much driving as possible.

I hate to admit it, but I don't handle it very well. Not in terms of my attitude, which is fine, but in terms of delegating. I somehow get the idea in my head that I should first knock out the IDD stuff and then work with Chris on the drive. But I take longer to finish the IDD work than intended, leaving Chris with little to do. And the drives are both pretty complex. What I ought to have done is push the IDD work onto his plate immediately and take care of the drives -- or vice versa. Another lesson learned, or so I hope.

Not to mention that I keep getting interrupted. The biggest interruption is a tour group behind shepherded around by Charles Elachi and Pete Theisinger. So I figure that's important enough to interrupt my work for a little while. (Turns out they're all related to some high muckety-muck whose name I don't recognize but who's always on our review boards or something. Good call.) I've done a million of these; I give them the usual spiel about why we can't joystick the rovers, what we're up to now, here's a bit about our software, and so forth. They love it, and I love doing it, so what the hell.

Nevertheless, all this delays getting the drive done. Even for a relatively simple drive, this takes longer than it used to, or it feels that way. What makes the game so hard (and so much fun) is that there are a million little things to worry about, many of them interact, and you have to get them all just right. "I almost long for the old days, when we didn't know what could go wrong with a drive," Chris says, and I know what he means. Ignorance wasn't quite bliss, but it can seem that way in hindsight.

[Next post: sol 713, January 4.]