Opportunity Sol 1074 (Spirit Sol 1094)

No driving is planned today. But yesterday's drive stopped a bit earlier than we'd hoped, after covering 2.19m of the planned 2.5m, due to increasing tilt. We knew the tilt limit was tight, but we allow only 8deg of tilt this close to the rim, and our limit of 7.8deg was about as high as we'll go.

But, looking at the data ... well, I'd be comfortable commanding another 30cm at a higher tilt if it came to that. Our slip wasn't increasing, which is what the tilt rules are really about, and if anything, we'd probably actually flatten out a bit more over that distance.

The possibility is alive for a while, until Larry Soderblom comes by. "Two point one nine meters is close enough," he announces, and that's that. No driving today.

I do have to come back later in the afternoon to give a demo for the college-aged son of a friend of Charles Elachi, the Lab's director. I don't know what's happened to college kids since I was in college myself. They seem to be opening the entrance gates to younger and younger students. Pretty soon, they'll be letting kindergarten kids in.

But he's a nice guy, and appropriately enthusiastic and impressed. So I won't hold his youth against him. He's clearly wise beyond his years.

[Next post: sol 1096 (Opportunity sol 1075), February 1.]


Opportunity Sol 1071 (Spirit Sol 1091)

I have good news and bad news. And good news. And bad news. And good news.

The good news is, we got satisfactory results from our VO test. Wiggle-and-scuff produced good results, so we're going to use that for our drives in this area for a while. (And indeed, it was used for yestesol's drive, which I wasn't on shift for. And it kept working.)

The bad news is, the drunken-sailor test didn't work.

But the good news is, that's due to a simple flight software bug in the command that tells the rover which arcs are allowed. And it's a bug that we can probably compensate for, so drunken-sailor isn't out of the running (or staggering) yet.

The bad news is, we can't take advantage of our success by driving thisol. We need to take PCAM images from this spot before we can move again -- they're needed for long-baseline stereo -- and we need NCAMs to target those PCAMs. And the NCAMs aren't down yet.

The good news is, the NCAMs eventually come down after all -- stuck in a pipe somewhere in the intarweb, or something. Whatever. We're driving!

Well, maybe it's a bit of a stretch to call it a drive. It's just a little 2.5m bump further along this cape. But the wheels are going to turn and they're going to take the vehicle with them; call it what you will.

The simplicity of the sequence leaves me time for chatting about an upcoming radio interview I'm doing. They've asked for the top five moments in the mission. "I've got Spirit's landing, Opportunity's landing, Spirit at the top of Husband Hill, Opportunity at Victoria ... and I'm taking suggestions for the last one," I announce.

There's no shortage of suggestions. Jake Matijevic proposes Opportunity at Endurance. "It was the first time we got to a really big crater," he says. "And it was both a scientific and an engineering accomplishment."

Ashitey has plenty of suggestions, too, and he disparages one of my choices. "Personally, I think getting to the top of Husband Hill was kind of an anti-climax," he says. "A lot of the drives we did to get there were more interesting."

"Fair enough," I counter. "But to me, getting to the top of Husband Hill wasn't the point; it's that it represents the stuff we had to do to get there." He looks half-convinced, so I press on. "It's like running a marathon. Crossing the finish line is great, but that doesn't happen without all the stuff that led up to it -- all the training, the 26 miles you had to run to get there ...." That does it. He's on board.

Julie Townsend plumps for the sol-18 anomaly. I wasn't going to include that one, mostly because I'm tired of it, but she makes a good case. "It had the whole team working together to solve a problem -- and actually, we had two rovers in crisis, with Opportunity on her way to landing right when that happened." Hmmm ... it's a good point. That one goes on the maybe list.

Then that leads us into the top five scariest moments of the mission. The sol-18 anomaly is one, of course; then there's catching a potato in Spirit's wheel, Purgatory, losing Spirit's right-front wheel with winter coming, ....

For me, one of the scariest moments of the mission is something Jake says during that discussion. I've heard it before, but not from him. "We're going to have to think about another hard decision," he says. "When to give up on Spirit."

Ashitey's response captures my feelings perfectly. "Never!" he exclaims. "We'll drive her on one wheel!"

But Jake points out that, well, budgets are being cut, and Spirit's limited mobility makes simply turning her off a possible choice.

Later in the day, John Callas (MER's project manager) stops by to see what's up, and Ashitey mentions that possibility to him.

"We're not going to sacrifice a vehicle," John says flatly. "That's not gonna happen."

And that's the best news I've had all day.

[Next post: sol 1094 (Opportunity sol 1074), January 30.]


Opportunity Sol 1066 (Spirit Sol 1087)

We're in what is probably the flattest, most featureless terrain this vehicle has been in, ever. You'd think that would be good news: no obstacles, nothing to hit. The problem is that we have this big honking hole in the ground called Victoria Crater, and we need to stay the heck away from it.

Our basic tools for making sure we stay away from something are visodom and autonav. Visodom helps you by telling you where you are, by comparing "before" and "after" pictures to figure out how far the rover actually went -- as compared to how far it thinks it went because of how many times it spun its wheels. The problem with that is that in featureless terrain, the "before" and "after" pictures look the same. We try to fix that by looking at our own tracks, but those have a nice repeating pattern -- a "picket fence effect" -- that can confuse the software. Looking into the crater at the nifty featureful stuff in there doesn't work either, mostly because it's too far away.

Autonav has its own problems in this terrain. With no features, it can't get good 3-D data about the world, so it treats that as a scary lack of data. Essentially, the flat featureless stuff gets treated like a cliff. And since there is a real cliff that we want it to be scared of, we wouldn't turn that off even if we could.

Normally, for historical reasons as much as anything else, Opportunity relies on visodom. Yesterday's drive stopped due to a VO failure caused in turn by the "picket fence effect" when we looked at our highly repetitive tracks. So what do we do to fix visodom's problems here? We have to make our own tracks more interesting.

So that's the plan. We pick a direction -- a direction more or less straight away from the crater -- and plan a test. Mark has a particular idea he wants to try for the first 5m, "but after that, it's up to you," he says. Gracious SOWG chair Larry Soderblom is on board with it; I don't even need to point out that spending a sol or two to do this checkout could save us a whole bunch of sols down the line.

So here's today's drive. We plan a 25m drive, broken into five segments, each of which tries a different test:

  1. Scuff with both front wheels on each step. As we drive backward, we periodically stop to rotate the right front wheel, then the left front wheel, away from us, in order to push up a chunk of dirt.

  2. Same as above, but we wiggle the left wheel instead of scuffing with it. This creates a different pattern on each side.

  3. "Drunken sailor" -- my favorite, of course, but not only for the name. This test tells the rover that it's not allowed to drive straight toward its goal; it must veer a bit to one side or the other. Since the real-world performance of the vehicle is never perfectly symmetric, this should create an interesting pattern out of little more than randomness. Plus, this version is the easiest to sequence.

  4. Every step, do a small turn in place.

  5. Finally, the "mother of all tests": run a step of test 1, a step of test 2, and so on down the line, until each has been done twice.

One of those oughta work. If they don't, we're screwed.

[Next post: sol 1091 (Opportunity sol 1071), January 27.]

Courtesy NASA/JPL-Caltech. Oh, my gods, is it ever flat here. Flat. Flat. Flat flat flat. Except, you know, for the giant scary hole in the ground. But otherwise, flat.


Spirit Sol 1085

Ah, brilliant. We've been working on releasing a new RSVP version to go with the new Linux version we're moving to. And they decided that today would be a good day to upgrade one of the RP workstations. (Not, say, yesterday, when there was no planning on Spirit and we would have had the whole day to work out any problems.)

And, of course, we're doing a whole lot of crap today. We're doing the first RAT daisy we've done for hundreds of sols, with the usual extensive MI followup.

Yay ....

It turns out right away that something about the new installation has major problems. Applications are crashing more or less randomly, including RSVP, sometimes within seconds of being launched. It looks like some kind of weird problem with the video driver, though why this didn't show up on our development machine is a mystery for now. But the bottom line is that the newly upgraded machine is useless for real work, and we have to switch to a backup. This isn't too bad, just irritating and time-consuming.

But we get through the day, as we always do. We discover late in the game -- fortunately not too late -- that due to a silly editing mistake, I'd screwed up the calls to some of our helper sequences. So I fix that before delivering, but I feel the need to go back and obsessively check everything before we can go on. "Sorry," I say, "it's just my OCD."

"That's OK," Cindy Oda says, "I like it better when the RPs have a little bit of OCD."

"In that respect, Scott's one of the best," says Ashley.

I get no respect around here.

[Next post: sol 1087 (Opportunity sol 1066), January 23.]


Spirit Sol 1082

"According to Brenda," Khaled tells me, "we're doing a tool change from MB to APXS, and that's it for the three sols."

Khaled and Brenda turn out to be right. You could say it's an easy sol.

But in our wonted way, we put the spare time to good use. At my request, Paolo has started to lay out an approach back to Tyrone, the spot where we bogged down on the way to McCool Hill. We won't be going into the scary soft stuff, but we're going to try to get close enough for the MTES to get good pictures of the unusually high-sulfur sand we churned up there. Then we'll turn around and head for Home Plate.

So we stand in front of this enormous printout of the winter panorama, and start drawing imaginary lines on it. Yeah, we could follow our old tracks here, but that would take us too close to these big rocks with a not-terribly-maneuverable rover. So we have to go this way, but we'll have to watch out so we don't anchor the wheel on this cluster of pebbles over here. We should drive to that mound, no farther, and halfway there we should assess whether to do a wide U-turn that intersects the mound or just go all the way and do a sharp turn once we're there.

That kind of thing.

Now here's a joke you really have to be on MER to get. "Hey, Rich," I say to Rich Morris. "You know how I told you you should write a song and call it the Rat Brush Jam?"

"Yeah," he laughs.

"Well, you should make up a dance to go with it, and call it the Solar Array Stomp."

The whole room cracks up.

"My joy is tainted only by my depression that I didn't think of that first," Rich says.

It would be a terrific day if it weren't for some bad news I get, confidentially, at the end. Sharon and I are talking about some rover-driving issues, and she tells me quietly something I need to know about the importance of getting things right on the Spirit drives.

"NASA HQ thinks Spirit is useless," she says bluntly. "They might shut her off in October. So it's that much more important that we have some visible successes."

Well, that's good to know.

But shut her off? Over my dead body.

[Next post: sol 1085, January 21.]


Spirit Sol 1079

I haven't worked with Khaled for a while. He's in fine form, still struggling a little but well on his way to being a full RP again. Today is all IDD work, and it should give him a decent workout.

The first sol's nothing: just pick the APXS up and put it down about 1cm away. It's the second sol that causes all the trouble.

We've got three IDD targets to choose from, spread across the face of the ridge that forms part of the Troll feature. Target 1, off to the right, is the science team's preferred target, and we can reach it pretty well, though we have to disable the wheel-volume collision checks to do it. Target 2, off to the left, is at the very limit of reachability, and is in fact unreachable by the APXS. Target 3, in the middle, is easy to reach but least interesting to the scientists. So we go with Target 1.

Only we discover, partway through sequencing, something we really should have noticed at the outset. There's a projecting knob of rock to the right of Target 1, so when we put the MB on that target, the APXS nearly collides with the knob. Bloody hell. Target 3, here we come.

With that under control, I go up to the Land of Opportunity to help resolve a question raised by one of the mission managers, Rich Morris. The planned Opportunity drive uses a waydisc that's mostly inside of Victoria -- in particular, the center of the waydisc, the waypoint, is inside Victoria. Now, the rover is aimed straight at the waydisc, and it'll stop when it reaches the waydisc's edge, which happens well before it enters Victoria. But theoretically, the rover's mobility code could decide to wander far off to the side, head into Victoria, and enter the waydisc from the side.

Now, we have various guards against that happening. They've already given the rover so-called keepout zones, which are like stringing up "Police Line -- Do Not Cross" tape around the terrain. If the rover crosses the line, it'll stop, and there's no way for it to get into Victoria without crossing one of these lines. In addition, there's a set of reactive checks, where the rover will look at its tilt and suspension and freak out if it appears to be going over a cliff. And, of course, there's our years of experience driving the vehicle, which tell us that it'll do the nominal thing.

But ... a while back, as a defensive measure, I wrote up "Rules of the Rim," which are similar to the "Rules of the Road" in which we documented our best practices for driving through the etched terrain. These rules are meant to document our best practices for driving around Victoria, and one of them says not to sequence drives such that the drive could nominally go into the crater. Rich's question is, does today's planned drive violate that rule?

It's a tough call. Realistically, the answer is no. But technically, we're violating both the letter and the spirit of the rule. And, grimly, because I know this means more work for the RPs on shift -- and, in the real world, pointless work -- I have to tell the truth. Yeah, it's against the rules. And moreover, it's against the rules for a good reason. We really shouldn't do this.

Happily, Paolo Bellutta, one of the RPs on shift, comes up with an elegant fix. There's a knob you can twiddle to tell the rover that it's only allowed to drive so far toward a waypoint. This means we can turn the normal, unbounded-distance command into a command whose nominal behavior constrains the rover to a circle around its starting point. By correctly choosing this setting, we can be that much more sure the rover won't enter the crater; it puts us within the letter and spirit of the rules, and it's a low-effort change. Paolo's brilliant, and everyone's happy.

So I'm back downstairs, back to my own rover. And just in time to hear John Callas point out something I hadn't noticed, and I used to keep fairly close track of this. Today we're doing a three-sol plan, for sols 1079 through 1081. Sol 1080 will be 12 times our prime mission of 90 sols.

Every month a year long, if you see what I mean. And we just keep going.

[Next post: sol 1082, January 18.]


Spirit Sol 1073

Well, let's see if we can handle this. We've got to do a tool change, swapping the APXS for the MB.

No, wait -- there's more. Before plunking down the MB, we need to swing the IDD off to one side so we can take a PCAM image. Oh, how will we ever finish in time?

Yeah. It's an easy day.

[Next post: sol 1079, January 15.]


Spirit Sol 1070

"Way to go, RPs!" Oded Aharonson exclaims over the telecon. "You guys are awesome!"

We can't believe it ourselves. There's Montalva, right in the middle of the IDD workspace. Spirit ended up less than 8mm from her goal position, and she kept trying to make it until the clock ran out.

Good girl. I love this rover. Way to kick off that fourth Earth year, baby!

So, since she made it, we're going to take full advantage of her new position: whip out the IDD and start working this outcrop over. We RAT-brush it, MI it, and plant the APXS.

[Next post: sol 1073, January 9.]

Courtesy NASA/JPL-Caltech. Way to start that fourth Earth year, girl. I love you.


Spirit Sol 1069

As of this sol, we're starting our fourth Earth year on Mars. For Spirit, at least; Opportunity's anniversary doesn't happen for another few weeks.

But we're celebrating it in a big way, with an aggressive drive to the Troll outcrop. Yestersol's IDD autoplace checkout got killed because it was taking too much time, but it looks like it was running OK. It just took it a little longer than expected to find targets, and when it got killed, the sequence was in the final cleanup phase anyway. So we're good to go.

But go where? The scientists would like us to go to the part of Troll nicknamed "Riquelme," but there's a slight problem with that. When you look carefully at our current position, it appears that Riquelme isn't just under the left solar panel, it's under the left middle wheel. That means we drove over it with at least that wheel, probably also with the left front wheel, and we'll drive over it even more getting away. When we do that, we contaminate the target with tracked-in dirt and also with small amounts of metal from the wheel itself, and in a case like this, that makes the target less attractive to the scientists.

So we're off to another part of the Troll outcrop, a section called Montalva. This is a low slab of exposed rock visible off to our left. As with any Spirit drive these days, this is going to be a challenge. We've got to back up while turning 100 degrees counterclockwise, then bump forward until Montalva is close enough to reach it with the arm.

Not only does this call for some fairly precise driving, it also means trying to climb a slope of maybe 12 to 14 degrees -- forward, yet, pushing one wheel. Not only that, but time is tight; if just one or two steps don't do everything they should, we'll probably fall short. It's about at the limit of what Spirit can do now, maybe over that limit, and we're careful to say so to the science team.

But I hope she makes it. That'll be a nice, quiet, but glorious way to start our fourth year here.

As another milestone, thisol we uplink r1000 -- the 1000th drive sequence sent to the vehicle. That's not the same as 1000 drives, because it's common for a single drive to use multiple sequences. But it's a cool moment to see those numbers wrap.


Spirit Sol 1068

Happy New Year, everybody! It's January 3, which means that tonight at 8:35PM Pacific Time is the 3-year anniversary of Spirit's landing.

We always take care in sequencing, but never more than when we have an anniversary like this one. You don't want the story of your three-year anniversary to be that you hurt the rover -- not that you'd want that any other day, of course.

So we're particularly careful about planning the drive to Troll. Troll is an outcrop just a few meters from our Winter Haven. Currently, it's under our left solar panel, but we think we can get there -- although, with the high tau we've been experiencing lately, we're concerned that the obvious approach to Troll leaves us with an unfavorable tilt.

At least we'll be able to gather lots of data about the drive -- Spirit's got only 10.3Mbits of data on the file system, perhaps an historic low!

Except that we won't be able to do the drive after all. Not today, anyway. It turns out tau is even higher than we thought, which means we have less energy to play with than we thought. So the drive will have to wait for another day.

Instead, we're going to repeat the third step of the IDD autoplace checkout -- just another capability added in R9.2 of the rover's FSW that aims to make us rover drivers damn near obsolete.[1]

While Chris Leger -- who wrote the code for that new capability -- works on that, Ashley and Terry and I go ahead and work on the drive anyway. So what if we can't send it up? That's no reason to deprive ourselves of the fun of planning it!

The drive to Troll will be challenging for our little five-wheeled bot. We'll face a decline in our northerly tilt, from about 7.5 degrees to about 5, and that can be a lot of change for a power-starved vehicle. Moreover, as we climb the outcrop, our westerly tilt increases, which means the solar-array wakeup would happen later in the day. Furthermore, we may simply be unable to climb the slope: it's about 12 degrees, and that seems to be around the limit of what we can do now.

But Jake Matijevic OKs the power issues, so our plan will be to do what we always do: try our damndest, and find a way to make it work if humans -- and plucky robots exploring other planets -- can.


[1] IDD autoplace enabled the rover to safely unstow the arm and place tools on targets that were outside the IDD's reach at the start of the drive. That is, you could drive to a target and use the IDD on it, all in one planning cycle. It was slow, but it worked. Just like us, the rovers got smarter as they got older -- but I'm pretty sure they didn't lose their glasses all the time. Damn glasses. Where did I put them?