Spirit Sol 156

Yestersol's drive tilted out. I think that's a first for us. The rovers have tilt sensors to tell them when they're on dangerous slopes -- sort of like a glorified pinball machine. Yesterday the rover climbed onto a slope that was just a little bit too steep, and, well, game over.

We knew yestersol that there was some risk of this. So when we were working on yestersol's drive sequence, I wrote a script that turned RSVP's simulation results into a graphical plot of tilt versus time. The graph showed a maximum tilt of about 17 degrees as the simulated rover climbed a ridge toward the end of the blind drive -- which was less than the current on-board limit of 20 degrees, so it should have been fine. Unfortunately, as it turns out, the simulation was just a little bit wrong, causing us to exceed the tilt limit. Well, these things happen.

Recovering is simple enough. We're going to increase the tilt limit, which was set lower than it should have been anyway, to finish climbing the ridge. Then we'll drive another 30m or so down the back of the ridge and across a little swale, and we're at the base of the Columbia Hills at last. (Or so it seems from here. But the slope at the base of the hills is gentle enough that there's no clear dividing line. Some say we're already there, but -- as in a nightmare -- the closer we get, the farther away it seems to be. It's become something of a running joke.)

Of course, it's never that simple. Continuing our effort to figure out what's starting to go wrong with the right front wheel, Joe Melko has devised a sort of exercise program for the rover. We're supposed to drive the rover for varying distances at varying speeds -- two meters at full speed, followed by 25cm very slowly, etc. -- with one series of these motions at the beginning (when the actuators are relatively cold) and a similar but different series at the end (when they'll be warmer).

The first problem with that is, none of us knows how to tell the rover to drive at anything less than full speed. There are no rover drivers upstairs -- they're not needed on Opportunity thisol -- and the best possible person to ask, Jeff Biesiadecki, is at home today. I poke around in the command dictionary and find what I think is the right command for this, but it's not very clearly documented. Then I ask around a little, and everybody I talk to ends up saying, "But you should call Jeff Biesiadecki to be sure." I hate doing that -- but they're right. So I call him.

At least I had pinned down the right command, and I had the right idea about how to use it. The basic idea is simple enough: there are six drive motors, one for each of the wheels, and you just use this command six times to set the maximum turn rate for each. There's a little more to it than that, and Jeff researches the matter and emails me the details later, but we already know enough to get started.

Which is a good thing, because these extra commands, along with associated data prioritization commands we also need to uplink so Joe can get the results quickly, complicate the drive considerably. Not to mention that we actually have to ensure that the rover is safe while it's performing the exercises Joe thought up for it, and that it ends up reaching thisol's destination. It turns into what I'm pretty sure is our most complex drive sequence ever. I quickly write a couple of new RoSE macros to help, but it's going to be a lot of work anyway.

Naturally, to make things worse, they had a traversability meeting this morning that effectively pushed back the start of the day. While Chris is working hard on the drive sequence, Julie asks him, "Will you be ready for the walkthrough in ten to fifteen minutes?" Chris turns around and says, "How much do you want me to laugh at that right now?" Which I interpret as a no.

Chris finishes his part and hands the rough-cut drive sequence off to me. I start fixing it up and making sure the details are right while he straps on the LCD shutter goggles and revisits his high-level decisions. Every once in a while he comes over and says, "You know, I don't think I like that waypoint after all," and I have a bunch of things to change. Under other circumstances, this is the sort of behavior that makes me want to strangle people. But he's not doing this to be an asshole or because he was sloppy, he's doing it because we're solving an unusually hard problem on an unusually tight schedule. And, as always, the rover's safety is paramount. If we need to do extra work, then fine, we need to do extra work. The rover matters more than our time.

With all this going on, it falls to Justin Maki, as usual, to ask: "Did you guys know that today is the one-year anniversary?" Holy smoke, I can't believe I didn't notice! Spirit launched June 10, 2003 -- one year ago today. And appropriately enough, this is the day we'll reach the Columbia Hills. Naturally, Justin's observation triggers reminiscences (from those who have time for it).

"Seems like a lot longer and a lot shorter time than that," Julie observes.

"We flew those damn things here --" Scott Doudrick says.

"That took a long time."

"-- the sun blew up on us --"[1]

"We lived two years in that six months," Julie says. She starts to say something else -- and is interrupted by the exigencies of the moment. Spirit needs her attention, again.

As Chris changes the waypoints, Jeff emails me other things we need to change in the sequence. The first is a potential buffer-size problem. The flight software allocates only a 300KB memory buffer for gathering motor data. The way it works is, at the start of each low-level drive command, the rover starts writing data into the buffer; it flushes the buffer (moving the data to another area so that it can be downlinked later) when the command completes. But since some of the steps in Joe's exercise program are taken at a very slow speed, some of the commands will take several minutes to execute -- much longer than normal. Also, we're gathering data at a higher-than-usual rate, to give Joe the maximum information possible. But this combination of long commands and high data rates will produce more than 300KB of data, causing the buffer to overflow. I don't think the software will crash, but we'll lose some of the data we're trying to gather. So we have to go back and break up the arc commands into multiple shorter arcs, short enough that each one will come in under the limit.

Then Jeff realizes there's another problem. Joe wants to run the motors at a much lower speed than anybody has ever tried before, and Jeff doesn't know for sure that that will work. When we're in a situation like that, we're supposed to try it in the testbed first.

Ah, wonderful.

Well, maybe we don't need to do that, and maybe we do. It's not my call to make; and fortunately, we have a meeting coming up that will include the right people to make this decision. It also occurs to me that Mark Maimone has probably run these rovers in the testbed more than anyone, and even if he's never run them so slowly, he may have a valuable opinion. So I go find him -- he's eating lunch while hacking away at some code in the SMSA -- and ask him to join us.

"You didn't think you were going to do anything useful today, did you?" I ask him.

"I guess not," he sighs.

The options as I see them, I tell him, are three: (1) try this in the testbed before sending the sequence up today, (2) punt and do it tomorrow, giving us more time in the testbed, or (3) ignore Jeff and try this for the first time on the flight vehicles. (I am no fan of option 3; I include it mainly to make Mark laugh.) Waiting wouldn't normally be a problem ... except we know we can fit the exercise into today's drive, but once we start climbing the hills, our opportunities to try it may be curtailed.

Mark looks at the Mars clock. "How long do we have until the sequence goes up?"

"About nine hours."

"So theoretically, if someone went to the testbed for an hour or two ...." he says. He knows he'll be the chosen someone, and he clearly isn't in the mood to sacrifice a productive day to this. But he's a good guy, and he's willing.

"Yeah, theoretically," I say. "But we're going to have a meeting to see whether we even want to do that, so don't head over there just yet."

At the meeting, we need to show the somewhat tricky drive in 3-D. Which means we need to scare up a bunch of 3-D glasses. I steal some from the science room, and snag my own five-dollar glasses as well (determined, as I am, to wring some value from them). As I hand a pair to Julie, I tease her, "Remember, men don't make passes at girls who wear 3-D glasses."

"My boyfriend will make me wear them all the time, then," she replies.

While people are still filtering in for the meeting, Mark Adler mentions that he's bringing a Senator around on Saturday. I think he says it's Senator Stevens, the head of the Senate Appropriations Committee and, consequently, one of the most powerful Senators. It would be fun to be there, but I'll be out house-hunting, so I'll have to give it a miss. Besides, after the President and Vice President, mere Senators are yawnsville.

The meeting starts. We've got three mission managers ("which guarantees we'll have one eye open among us," Andy quips), among others -- more than enough firepower to decide whether to run Joe's experiment tomorrow (though that matter has to wait until the goal of the meeting, planning our near-term drive strategy, is met). The decision is to split the difference: do as much of Joe's experiment as we can, but don't drive the wheels any slower than Jeff feels comfortable with. Joe had us driving the wheels as little as 1/34th of normal speed; Jeff feels comfortable going down only as far as 1/3rd. So I delve into the sequence once again, raising the minimum speed into Jeff's comfort zone.

One other matter that comes up in the meeting is the name of an upcoming destination -- Tacopa. As soon as I hear this, I start singing, "At Tacopa, Copacabana." Mark Maimone chimes in: "Her name was MOLA ...." Now that's something: a joke you can get only if you know old Barry Manilow songs and if you're enough of a space geek to know about the Mars Orbiter Laser Altimeter.

I think Mark is glad he came to the meeting after all. Just so he could make that pun.

As I'm finishing up for the day, I happen to look at the rover in its current state as determined from telemetry, frozen climbing the hill. I've hidden the terrain and am just looking at the rover itself from a front three-quarter view, when I'm struck by something. Tilted back to an unusual degree and with the front wheels extended in front of it, Spirit looks uncommonly like a rearing horse.

Steady on, girl, I think. And take care of that right front hoof.

[Next post: sol 161, June 16.]

---

Footnotes:

[1] While the rovers were on their months-long flight to Mars, they experienced a burst of unusually high solar radiation -- caused by sunspots, if I recall correctly. There was nothing we could do about it, and the rovers survived unharmed, but it was a scary incident.