The good news is, Ashitey was right: we did make contact on the first retry with the tweaked surface normal -- none of the fancy new stuff was even needed. The bad news is, the resulting RAT brush was fairly poor, so they had to redo it last week. The good news is, the second one worked, and they drove away already.
And a nice job they did of it, too: 71m on the first sol, 47m on the second (autonav-only) sol. (Incidentally, this was Spirit's first such back-to-back drive.) The bad news is, the drive was sort of perpendicular to Home Plate, so our radial distance to our long-term goal remains more or less unchanged, even as time wasteth. Not that I'm worrying about it.
Anyway, as a result, we're only about 40m from El Dorado, and John and I are looking to get the rest of the drive today. Happily, it's just about as simple as it could be -- there are practically no obstacles here, just a nice flat zone. Just before El Dorado begins, there's a slight rise, so we'll have to be on autonav for the last part, but it shouldn't be any problem at all.
The big question is whether we should enter El Dorado or not. Our goal here is to find out what this thing is made of, which we intend to do by digging a trench. If we enter it thisol, then tomorrow we'll know a lot more about what it's made of and what we can expect from the trench. The problem is, maybe we'll know too much -- maybe we'll be bogged down in it, just as Opportunity was at Purgatory. And with winter coming on, and us already behind the curve ....
On the other hand, if we're going to trench the thing, we've got to enter it sometime, at least with the front wheels. Ideally, we'd stop with just the front wheels in it, but it's 40m away (or so -- our range data is poor, compounding the problem) and if we want to plant the front wheels only, we'd have an error budget of only 50cm or thereabouts. We just can't drive these vehicles that precisely. (They were designed for two-meter approaches. We do a lot better than that, but 40m really is out of the question.)
Fortunately, of course, we've got world-class experts to help us answer these questions. I ask both Brenda Franklin and Rob Sullivan about what we can expect from El Dorado. They agree that the material is coarse-grained (you can tell because it's dark -- dust is light). Brenda thinks the stuff is likely relatively compact, though maybe not; Rob's not even that sure. At least it's not likely to be quicksand -- so-called "foo-foo dust," which we'd just sink right into.
John and I decide we're going to proceed with caution. We'll just zoom right over the stuff we can see, but when we're about 10m from El Dorado, we turn on autonav and start doing slip checks every 2-3m (as opposed to the usual 10-15m between checks on this vehicle). That way, we can't get more than about a vehicle length into a trouble zone. And we make the slip checks pretty paranoid; if they fail to converge, we bail on the whole drive -- a common practice on Opportunity, but not something we usually bother with on Spirit. By the time we're done messing with it, we've got a drive that's likely to give up early for one reason or another. But better that, than be lost forever in El Dorado.
Mark Adler keeps discovering more about how things have changed since he was here last. "You're not doing anything during the 11 minutes of drive heating?" he asks incredulously. We used to pack the sol as full as possible; we'd have found something to do during those 11 minutes.
"It's not like it used to be," I shrug. "It grated on me for a while, but I got used to it. We have to make compromises to get out of here in shorter amounts of time, and to keep people from burning out." Sad, but true. I'd been thinking about just this when walking in this morning. When we changed the mission from a sprint to a marathon, we adjusted our pace accordingly. It was the right choice, and yet I regret being unable to use the vehicles to their fullest capacity -- it seems such a waste of a priceless scientific resource that the limiting factor in their operation would be us humans. I suppose that's a lesson to take forward to MSL: engineer them (and the ground tools, and whatever) so that they can be used to their fullest capacity in an eight-hour planning cycle.
But Mark remains stubbornly old-school in another way. "I'm gonna play a wakeup song," he announces. "It's 'El Dorado,' by ELO." He plays it for us. I'm sorry to say it's a pretty lousy song. But it's nice to have the old days back, even in such a small way, all the same.
[Next post: sol 710, January 1.]
2010-12-28
2010-12-21
Spirit Sol 699
Yestersol's IDD sequence had mixed results. The RAT failed to establish solid contact with the bumpy surface of this rock, so we didn't do the brush. Instead, the sequence simply proceeded to take MIs of the unbrushed surface and place the MB there.
This isn't the worst thing that could have happened. The MB is pretty good at seeing through the dust anyway, so we'll still get good science results from it even though the brushing never took place. However, the APXS can't see through the dust, so we really need to brush the darn thing thisol. We're falling farther and farther behind our drive metric all the time, and with Christmas coming this weekend, a failure to brush now could end up blowing most of a week. And that really wouldn't be good.
What happened yesterday was complex: almost exactly at the nominal contact position, the RAT's contact switches tripped briefly, but then rapidly gained and lost contact several times in less than a second. The initial firing of the contact switches stopped any further motion toward the rock face, but the sequence's subsequent check for contact failed because the switches had changed state by the time that check happened.
Clearly, we need some approach that checks whether we've made contact and autonomously tries again if it hasn't. The solution Frank and Matt Heverly and I come up with is something we've never tried before; we call it the "phased approach." First we'll try to place the RAT the same as we did yesterday, only without attempting any overdrive. (This will probably work, since we contacted 2mm early yestersol.) Then we'll have the sequence check for contact, and if it doesn't see contact, it'll push forward 5mm and check again. If still not in contact, it'll push forward 5mm and try again, then yet again, for a total of three tries (and 1.5cm total overdrive, about the most we dare command). And if we're still not in contact after all that, we'll try the whole routine again on an alternate target -- the same position, but with a slight tweak to the surface normal, so that we'll come in at a different angle.
If we're still not in contact, well ... then we give up. But at least we will have tried, and tried hard.
Though this approach sounds pretty complex, it's reasonably simple to sequence. I'm a good chunk of the way through it when Ashitey arrives to object. At first he's not comfortable brushing at all, since this switch-bouncing behavior isn't something we've seen before. He ultimately relents there, but still has problems with what we're up to.
"I think what you're doing is too complicated," he says. "You're doing things we've never done before." He doesn't say so outright, but this is clearly a sort of threat -- if we want to do what we're doing, he's going to make enough of a stink that we'll have to run it by Project management. "You should keep it simple -- just try again one time, with the tweaked surface normal."
"Besides," he continues, "you know it'll make contact if you just try again with the tweaked target."
But I don't know that. And I don't really think what we're doing is unreasonably complex, especially considering the downside: we have to stay here until we get this done, and since Home Plate is slipping farther and farther away every day, we should make every effort we can to get this to work (consistent with the safety of the vehicle, as always). And while it's true that we're doing something we've never done before, all of the parts of it are things that we have done before. "We invent techniques all the time without taking them to the Project level," I point out. "You think we should involve Jim Erickson every time we add something to a slip check?"
Ashitey backs off on the "first-time activity" line, but still pushes to make the thing simpler. We argue about it for a while, and perversely, our eventual compromise is even more complex than what we had when he came in. Now, we're first going to try the usual-style (non-phased) approach on the tweaked target. If that fails to make contact, we go to the original plan: try a phased approach on the original target, and if that fails to work, we try a phased approach on the tweaked target -- and only then, only if all that fails, we give up.
I'm not sure how that happened, but that's where we wind up.
All of this arguing pushes us pretty late -- by the time we're done, it's approaching 08:00, rover time, only an hour or so before we uplink. Someone remarks on this, but it fails to impress Mark Adler. "Ah, that's nothing," he says. "We used to CAM during the uplink window."
Well, I know they did that once. But it's not something anyone wants to repeat, you may be sure.
[Next post: sol 706, December 28.]
This isn't the worst thing that could have happened. The MB is pretty good at seeing through the dust anyway, so we'll still get good science results from it even though the brushing never took place. However, the APXS can't see through the dust, so we really need to brush the darn thing thisol. We're falling farther and farther behind our drive metric all the time, and with Christmas coming this weekend, a failure to brush now could end up blowing most of a week. And that really wouldn't be good.
What happened yesterday was complex: almost exactly at the nominal contact position, the RAT's contact switches tripped briefly, but then rapidly gained and lost contact several times in less than a second. The initial firing of the contact switches stopped any further motion toward the rock face, but the sequence's subsequent check for contact failed because the switches had changed state by the time that check happened.
Clearly, we need some approach that checks whether we've made contact and autonomously tries again if it hasn't. The solution Frank and Matt Heverly and I come up with is something we've never tried before; we call it the "phased approach." First we'll try to place the RAT the same as we did yesterday, only without attempting any overdrive. (This will probably work, since we contacted 2mm early yestersol.) Then we'll have the sequence check for contact, and if it doesn't see contact, it'll push forward 5mm and check again. If still not in contact, it'll push forward 5mm and try again, then yet again, for a total of three tries (and 1.5cm total overdrive, about the most we dare command). And if we're still not in contact after all that, we'll try the whole routine again on an alternate target -- the same position, but with a slight tweak to the surface normal, so that we'll come in at a different angle.
If we're still not in contact, well ... then we give up. But at least we will have tried, and tried hard.
Though this approach sounds pretty complex, it's reasonably simple to sequence. I'm a good chunk of the way through it when Ashitey arrives to object. At first he's not comfortable brushing at all, since this switch-bouncing behavior isn't something we've seen before. He ultimately relents there, but still has problems with what we're up to.
"I think what you're doing is too complicated," he says. "You're doing things we've never done before." He doesn't say so outright, but this is clearly a sort of threat -- if we want to do what we're doing, he's going to make enough of a stink that we'll have to run it by Project management. "You should keep it simple -- just try again one time, with the tweaked surface normal."
"Besides," he continues, "you know it'll make contact if you just try again with the tweaked target."
But I don't know that. And I don't really think what we're doing is unreasonably complex, especially considering the downside: we have to stay here until we get this done, and since Home Plate is slipping farther and farther away every day, we should make every effort we can to get this to work (consistent with the safety of the vehicle, as always). And while it's true that we're doing something we've never done before, all of the parts of it are things that we have done before. "We invent techniques all the time without taking them to the Project level," I point out. "You think we should involve Jim Erickson every time we add something to a slip check?"
Ashitey backs off on the "first-time activity" line, but still pushes to make the thing simpler. We argue about it for a while, and perversely, our eventual compromise is even more complex than what we had when he came in. Now, we're first going to try the usual-style (non-phased) approach on the tweaked target. If that fails to make contact, we go to the original plan: try a phased approach on the original target, and if that fails to work, we try a phased approach on the tweaked target -- and only then, only if all that fails, we give up.
I'm not sure how that happened, but that's where we wind up.
All of this arguing pushes us pretty late -- by the time we're done, it's approaching 08:00, rover time, only an hour or so before we uplink. Someone remarks on this, but it fails to impress Mark Adler. "Ah, that's nothing," he says. "We used to CAM during the uplink window."
Well, I know they did that once. But it's not something anyone wants to repeat, you may be sure.
[Next post: sol 706, December 28.]
2010-12-20
Spirit Sol 698
I love this rover. I really, really do.
I'm up at 05:45, and the first thing I do is log in and check the data from home. No images. That's just a touch worrisome. Maybe it's another OSS crash? But I see no evidence of that.
Hmmm ... I keep poking, and find that we did get some data -- so Spirit's alive -- but I find nothing to show that we've moved at all. It's as if the drive sequence never executed.
I shake it off, take a shower, and get to work as fast as I can. By the time I get there, the processing glitch -- if that's what it was -- has been resolved. We've got at least some drive data, and, most crucially, the post-drive images.
They show Comanche Spur dead in the middle of the IDD work volume. It's beautiful. We couldn't have gotten it more perfect. Or let me say, she couldn't have gotten it more perfect. All credit to our awesome rover.[1]
Visodom failed to converge at the end, and as a result, we didn't do our normal wheel wiggle, which we normally strive to do in order to settle the rover before deploying the IDD. When we look carefully at the images, though, we're able to convince ourselves that all wheels are in a stable configuration. This, coupled with past data and recent testbed work that has helped demonstrate that this rover design is highly stable when deploying the IDD even on apparently unstable surfaces, makes us willing to proceed.
What we proceed with is our pretty much standard IDD campaign -- MI, RAT-brush, MI, MB. Depending on the MB results we see tomorrow, we might stay here an extra sol, or cut things short and light out for our next destination, a large dark blob nicknamed "El Dorado."
Chris is RP-1 thisol, back on the rover after a FSW-induced absence of a couple of weeks, and he hasn't lost his touch. He cranks out the IDD sequence, I patch up a couple of minor errors, and the day's over before you can say "Jack Robinson."
Well, maybe not that fast. But pretty fast.
Courtesy NASA/JPL-Caltech. Comanche Spur, dead in our final front HAZCAM. I am more proud of this image than of almost any other in our nearly seven years on the surface.
Footnotes:
[1] I am unable to express just how proud I am of this drive to this very day. It was an amazing piece of work under extraordinary circumstances; if not my very best ever, it has to be damn close. Also, trying it was plain crazy, and hot damn, did it ever go perfectly. Big. Fat. Win.
I'm up at 05:45, and the first thing I do is log in and check the data from home. No images. That's just a touch worrisome. Maybe it's another OSS crash? But I see no evidence of that.
Hmmm ... I keep poking, and find that we did get some data -- so Spirit's alive -- but I find nothing to show that we've moved at all. It's as if the drive sequence never executed.
I shake it off, take a shower, and get to work as fast as I can. By the time I get there, the processing glitch -- if that's what it was -- has been resolved. We've got at least some drive data, and, most crucially, the post-drive images.
They show Comanche Spur dead in the middle of the IDD work volume. It's beautiful. We couldn't have gotten it more perfect. Or let me say, she couldn't have gotten it more perfect. All credit to our awesome rover.[1]
Visodom failed to converge at the end, and as a result, we didn't do our normal wheel wiggle, which we normally strive to do in order to settle the rover before deploying the IDD. When we look carefully at the images, though, we're able to convince ourselves that all wheels are in a stable configuration. This, coupled with past data and recent testbed work that has helped demonstrate that this rover design is highly stable when deploying the IDD even on apparently unstable surfaces, makes us willing to proceed.
What we proceed with is our pretty much standard IDD campaign -- MI, RAT-brush, MI, MB. Depending on the MB results we see tomorrow, we might stay here an extra sol, or cut things short and light out for our next destination, a large dark blob nicknamed "El Dorado."
Chris is RP-1 thisol, back on the rover after a FSW-induced absence of a couple of weeks, and he hasn't lost his touch. He cranks out the IDD sequence, I patch up a couple of minor errors, and the day's over before you can say "Jack Robinson."
Well, maybe not that fast. But pretty fast.
Courtesy NASA/JPL-Caltech. Comanche Spur, dead in our final front HAZCAM. I am more proud of this image than of almost any other in our nearly seven years on the surface.
Footnotes:
[1] I am unable to express just how proud I am of this drive to this very day. It was an amazing piece of work under extraordinary circumstances; if not my very best ever, it has to be damn close. Also, trying it was plain crazy, and hot damn, did it ever go perfectly. Big. Fat. Win.
2010-12-19
Spirit Sol 697
Amazingly, the drive went absolutely perfectly. During sequencing, I'd realized that if we changed the direction we turned when scuffing, we should be able to get a perfect view of Comanche in the FHAZ -- and there it is, large as life. The NCAM images are spectacular, showing just what we expected -- the larger, main part of Comanche on the left, a smaller spur on the right, and a kind of saddle in between.
The scuff, too, is right where it's supposed to be, although the soil here was hard enough that the scuff (Squyres derides it as a "smudge") isn't scientifically interesting. As a result, they decide after some discussion to bag IDDing the scuff and just do a couple of sols of remote sensing followed by an approach to Comanche.
So that's all good. But the OSS -- our shared filesystem -- crashed over the weekend, which means that all the automated processing of downlinked data is screwed up and backlogged, which in turn delays our start. And, of course, we're still in tight sols, and we're trying to do a three-sol plan today. All in all, it's a perfect day for a critical piece of infrastructure to have hiccups.
Fortunately, our drive is going to be perfectly simple. The most obvious approach, a Hillary-style drive partway up onto Comanche, won't work; the exposed rock face is too far up, and we'd have to cross too much loose sand to get there. But when we cast around, we find another solution, a couple of low, flat rocks lying on the ground on the northeast part of Comanche. They're only about 10m away, and we've got a nice, smooth path straight to 'em. Admittedly, they're dusty-looking, except for one small, possibly more-or-less vertical face, but them's the breaks. So that's nice and easy.
Only one fly in that ointment: when we show Steve where we're thinking of heading, he sums up his reaction in these words: "I hate it."
Oooo ... kaaaaayyyyy ....
Well, there is one other option we'd previously dismissed. We could just climb up into the saddle between the two chunks of rock; plenty of good stuff up there. This is more difficult, and somewhat more dangerous. Not to mention that thisol's RP-2 is Frank -- normally an Opportunist, not a Spirit -- which makes the whole thing a shade more risky.
We do it anyway. If all goes well, Spirit will climb more than a meter up a rocky 14-degree slope, hang a right, and drive across a relatively narrow channel to the spur. To her right will be a sharp sandy slope, which we might or might not get stuck on; to the left, a wide sandy patch, which we could bog down in. Monday, we'll be about 8m from here, with Comanche Spur dead in the center of our IDD work volume.
That, or we'll never hear from Spirit again, and have to live with ourselves forever.
Oddly enough, even though part of me says this is insane, I have so much history with this rover, and so much trust in her, that I'm not as worried about it as I'd expect. Am I right to think so? Monday morning, I'll know for sure.
Courtesy NASA/JPL-Caltech. Our final front HAZCAM, showing Comanche right in front of us.
Courtesy NASA/JPL-Caltech. A NAVCAM view of the same scene, aimed at the "saddle."
The scuff, too, is right where it's supposed to be, although the soil here was hard enough that the scuff (Squyres derides it as a "smudge") isn't scientifically interesting. As a result, they decide after some discussion to bag IDDing the scuff and just do a couple of sols of remote sensing followed by an approach to Comanche.
So that's all good. But the OSS -- our shared filesystem -- crashed over the weekend, which means that all the automated processing of downlinked data is screwed up and backlogged, which in turn delays our start. And, of course, we're still in tight sols, and we're trying to do a three-sol plan today. All in all, it's a perfect day for a critical piece of infrastructure to have hiccups.
Fortunately, our drive is going to be perfectly simple. The most obvious approach, a Hillary-style drive partway up onto Comanche, won't work; the exposed rock face is too far up, and we'd have to cross too much loose sand to get there. But when we cast around, we find another solution, a couple of low, flat rocks lying on the ground on the northeast part of Comanche. They're only about 10m away, and we've got a nice, smooth path straight to 'em. Admittedly, they're dusty-looking, except for one small, possibly more-or-less vertical face, but them's the breaks. So that's nice and easy.
Only one fly in that ointment: when we show Steve where we're thinking of heading, he sums up his reaction in these words: "I hate it."
Oooo ... kaaaaayyyyy ....
Well, there is one other option we'd previously dismissed. We could just climb up into the saddle between the two chunks of rock; plenty of good stuff up there. This is more difficult, and somewhat more dangerous. Not to mention that thisol's RP-2 is Frank -- normally an Opportunist, not a Spirit -- which makes the whole thing a shade more risky.
We do it anyway. If all goes well, Spirit will climb more than a meter up a rocky 14-degree slope, hang a right, and drive across a relatively narrow channel to the spur. To her right will be a sharp sandy slope, which we might or might not get stuck on; to the left, a wide sandy patch, which we could bog down in. Monday, we'll be about 8m from here, with Comanche Spur dead in the center of our IDD work volume.
That, or we'll never hear from Spirit again, and have to live with ourselves forever.
Oddly enough, even though part of me says this is insane, I have so much history with this rover, and so much trust in her, that I'm not as worried about it as I'd expect. Am I right to think so? Monday morning, I'll know for sure.
Courtesy NASA/JPL-Caltech. Our final front HAZCAM, showing Comanche right in front of us.
Courtesy NASA/JPL-Caltech. A NAVCAM view of the same scene, aimed at the "saddle."
2010-12-15
Spirit Sol 694
In summary: a good drive, but not great. We made 30m of progress, carrying us all the way to the ridge. But Murphy's Law is interplanetary, and visodom failed to converge during Spirit's top-of-ridge slip check. Which, thanks to our increased paranoia, meant she stopped driving.
As it happens, she probably would have stopped driving shortly after that anyway, since there is a rough semicircle of autonav-unfriendly tilts just a couple of meters below us. However, it's something we feel comfortable blind-driving through.
In fact, as we look down the ridge and across the terrain separating us from Comanche, it looks like we could blind-drive across the whole thing. It's a total of about 40m to the base of Comanche, and there are hardly any obstacles at all. They've allocated over three hours for driving again thisol, and I end up giving most of it back.
(The scientists have no problem figuring out what to do with the time, though. Turns out we're parked on the largest expanse of outcrop they've yet seen at Gusev, so they pack in plenty of PCAM and MTES imaging of it before we leave it behind forever. We'll also be taking a couple of pictures and relaying them through MEX as a friendly bit of international cooperation. Since our European friends plan to show them off, as one person notes, "Let's make 'em pretty." We settle on a nice color PCAM of Comanche and an NCAM of the intervening terrain. Should look pretty darn cool.)
The nice straight path makes the sequencing simple as well, which leaves me time to get lunch. As I'm walking over there with our prodigal son, Mark Adler, I ask him what impression he has of how MER's changed since he left.
"There's a lot of automation," he says immediately. "It all runs really smoothly. 'Course, if something goes wrong, well, that's a runout sol."
He's right about the increased automation, but as to the rest, I don't remember the last time an automation failure led to dropping the plan and executing the runout. I don't know if it's ever happened. We don't have the kind of team that likes to let it happen.
Another kind of automation failure gets us worried, though -- at one point, the file server freezes for a few minutes. We can't do anything without it, so this brings the process to a grinding halt. Fortunately, it comes back after a short delay, and the ball resumes rolling.
While all this is going on in the Spirit World, Opportunity's been making progress of her own. Slowly but surely, they're performing a limited IDD campaign on the juicy bit of outcrop we parked in front of, lo, these many sols ago. In the process, they're learning about the fault, and one of the things they learned yesterday is that the amount of resistance caused by the broken wire isn't a constant. Instead, it seems to vary as the arm articulates.
But the way they found out was by having another joint stall, so they're in the middle of a tool change -- with the MI pointing straight up and its dust cover open, to make matters worse. It was supposed to be a skip sol, but they need someone to help them recover and get things back on track. And who just became available, but little old me?
So I go upstairs and help Ashitey sequence the recovery tool change. It's not quite the first time a rover driver has worked on both rovers in one sol -- if nothing else, Cooper did it when the rest of us were in Hawaii -- but it's the first time I've done it. ("A personal best," as Emily Eelkema puts it.)
Two Mars rovers in one day, baby. It's every geek's dream.
[Next post: sol 697, December 19.]
As it happens, she probably would have stopped driving shortly after that anyway, since there is a rough semicircle of autonav-unfriendly tilts just a couple of meters below us. However, it's something we feel comfortable blind-driving through.
In fact, as we look down the ridge and across the terrain separating us from Comanche, it looks like we could blind-drive across the whole thing. It's a total of about 40m to the base of Comanche, and there are hardly any obstacles at all. They've allocated over three hours for driving again thisol, and I end up giving most of it back.
(The scientists have no problem figuring out what to do with the time, though. Turns out we're parked on the largest expanse of outcrop they've yet seen at Gusev, so they pack in plenty of PCAM and MTES imaging of it before we leave it behind forever. We'll also be taking a couple of pictures and relaying them through MEX as a friendly bit of international cooperation. Since our European friends plan to show them off, as one person notes, "Let's make 'em pretty." We settle on a nice color PCAM of Comanche and an NCAM of the intervening terrain. Should look pretty darn cool.)
The nice straight path makes the sequencing simple as well, which leaves me time to get lunch. As I'm walking over there with our prodigal son, Mark Adler, I ask him what impression he has of how MER's changed since he left.
"There's a lot of automation," he says immediately. "It all runs really smoothly. 'Course, if something goes wrong, well, that's a runout sol."
He's right about the increased automation, but as to the rest, I don't remember the last time an automation failure led to dropping the plan and executing the runout. I don't know if it's ever happened. We don't have the kind of team that likes to let it happen.
Another kind of automation failure gets us worried, though -- at one point, the file server freezes for a few minutes. We can't do anything without it, so this brings the process to a grinding halt. Fortunately, it comes back after a short delay, and the ball resumes rolling.
While all this is going on in the Spirit World, Opportunity's been making progress of her own. Slowly but surely, they're performing a limited IDD campaign on the juicy bit of outcrop we parked in front of, lo, these many sols ago. In the process, they're learning about the fault, and one of the things they learned yesterday is that the amount of resistance caused by the broken wire isn't a constant. Instead, it seems to vary as the arm articulates.
But the way they found out was by having another joint stall, so they're in the middle of a tool change -- with the MI pointing straight up and its dust cover open, to make matters worse. It was supposed to be a skip sol, but they need someone to help them recover and get things back on track. And who just became available, but little old me?
So I go upstairs and help Ashitey sequence the recovery tool change. It's not quite the first time a rover driver has worked on both rovers in one sol -- if nothing else, Cooper did it when the rest of us were in Hawaii -- but it's the first time I've done it. ("A personal best," as Emily Eelkema puts it.)
Two Mars rovers in one day, baby. It's every geek's dream.
[Next post: sol 697, December 19.]
2010-12-14
Spirit Sol 693
We made a respectable 25m of progress -- good, but not great. The visodom and blind segments went fine, but autonav failed shortly afterward, after finding no safe path. Oddly, we're not sure why autonav didn't want to proceed; what we can see in the images looks good.
Anyway, we're slowly but surely gaining on Comanche. We've still got about 35m to the ridge, and we've got a good chance of getting there and beyond because they've scared up over three hours of drive time today.
The path from here will be an odd one. After a bit of maneuvering to get onto the mesh, we've got to make our way through a trio of rocks arranged as the vertices of a roughly equilateral triangle, whose closest side lies transverse to our path about 10m from here. Our plan is to drive between the rocks to the center of the triangle, then jog left to dodge the rock at its apex before getting back on course and heading to the ridge.
About 10m after that -- still before we get to the ridge -- there's enough undulation in the terrain that our mesh starts to drop out. So we'll switch on autonav for the remaining 12m to the ridge, then turn and head to Comanche.
We're still worried about that ridge, though. I think there's a better-than-even chance that the far side will be too steep or too rocky for autonav to make any progress, in which case I'll feel guilty tomorrow about blowing a lot of drive time. But there's not too much I can do about that and still keep the rover safe.
One other modification we make to our usual Spirit-driving practices is to make the slip check more paranoid. Before starting autonav, and periodically during autonav drives, we've been having Spirit perform slip checks, looking to see whether she's slipping too much and stopping the drive early if so. Normally, if visodom fails to converge -- that is, if the slip-check calculations don't work, so we don't know what our slip is -- we let her drive on anyway. After all, that's a fairly unusual case, and even if that slip check fails, there will be another one shortly.
But in this case, we're just nervous about that. Once she gets over that ridge, she could be in really bad territory, and if she can't prove she's safe to go on, we want her to stop. So we tell her to do that. Will it cut our day short? We'll find out tomorrow ... same Mars time, same Mars channel.
Courtesy NASA/JPL-Caltech. Comanche, dead ahead.
Courtesy NASA/JPL-Caltech. In the other direction, a look back at our tracks. My girl's such a trouper.
Anyway, we're slowly but surely gaining on Comanche. We've still got about 35m to the ridge, and we've got a good chance of getting there and beyond because they've scared up over three hours of drive time today.
The path from here will be an odd one. After a bit of maneuvering to get onto the mesh, we've got to make our way through a trio of rocks arranged as the vertices of a roughly equilateral triangle, whose closest side lies transverse to our path about 10m from here. Our plan is to drive between the rocks to the center of the triangle, then jog left to dodge the rock at its apex before getting back on course and heading to the ridge.
About 10m after that -- still before we get to the ridge -- there's enough undulation in the terrain that our mesh starts to drop out. So we'll switch on autonav for the remaining 12m to the ridge, then turn and head to Comanche.
We're still worried about that ridge, though. I think there's a better-than-even chance that the far side will be too steep or too rocky for autonav to make any progress, in which case I'll feel guilty tomorrow about blowing a lot of drive time. But there's not too much I can do about that and still keep the rover safe.
One other modification we make to our usual Spirit-driving practices is to make the slip check more paranoid. Before starting autonav, and periodically during autonav drives, we've been having Spirit perform slip checks, looking to see whether she's slipping too much and stopping the drive early if so. Normally, if visodom fails to converge -- that is, if the slip-check calculations don't work, so we don't know what our slip is -- we let her drive on anyway. After all, that's a fairly unusual case, and even if that slip check fails, there will be another one shortly.
But in this case, we're just nervous about that. Once she gets over that ridge, she could be in really bad territory, and if she can't prove she's safe to go on, we want her to stop. So we tell her to do that. Will it cut our day short? We'll find out tomorrow ... same Mars time, same Mars channel.
Courtesy NASA/JPL-Caltech. Comanche, dead ahead.
Courtesy NASA/JPL-Caltech. In the other direction, a look back at our tracks. My girl's such a trouper.
2010-12-13
Spirit Sol 692
The drive went well. Spirit hit a limit cycle check -- she wasn't making progress fast enough, so she stopped driving after the first autonav waypoint -- but she still made a respectable 30m of progress. We're far enough around Miami (which looks even more impressive from here, as we've descended about 3m) to see Comanche from the other side.
And the news from here is not so bad. Rather than a shallow ramp all the way to the valley floor, it looks like we've still got a ridge to cross, but we've got a decent path to it, at least. And no time pressure -- today's Spirit's skip day, so Frank and Jeng and Ashley and I are planning the drive a day ahead. Frank and Jeng are as experienced at rover driving as they come, but they spend all their time on Opportunity, so we're avoiding making them plan a drive on an unfamiliar rover on a tight schedule, which is what would happen if they waited until tomorrow.
It's a good opportunity for me to revisit what's different between the vehicles. Frank actually has to ask what's the height of obstacles we're worried about -- after all, they never see anything that big on the other side of the world. "When we see a rock over there, they want us to drive onto it," he points out. And he's quite right: for months, the only rocks Opportunity's seen have been either tiny cobbles or, more commonly, flat outcrop patches that make for good driving.
Well, they showed up for an interesting drive. We've got several actual obstacles to avoid, so we'll have a visodom segment at the beginning to help navigate around them, then a short blind drive through a friendly patch, then autonav from there. I don't think we'll quite make it, but if we get really lucky, we'll make it to the ridge and have a spectacular view of the valley beyond. (And an equally good view of whatever we're going to have to cross to get down there.) If that doesn't happen this drive, it'll happen on the next one. And since I'm working on Spirit for the rest of the year, while they continue to investigate Opportunity's IDD problem, I'll be sure to see it.
I can't wait!
And the news from here is not so bad. Rather than a shallow ramp all the way to the valley floor, it looks like we've still got a ridge to cross, but we've got a decent path to it, at least. And no time pressure -- today's Spirit's skip day, so Frank and Jeng and Ashley and I are planning the drive a day ahead. Frank and Jeng are as experienced at rover driving as they come, but they spend all their time on Opportunity, so we're avoiding making them plan a drive on an unfamiliar rover on a tight schedule, which is what would happen if they waited until tomorrow.
It's a good opportunity for me to revisit what's different between the vehicles. Frank actually has to ask what's the height of obstacles we're worried about -- after all, they never see anything that big on the other side of the world. "When we see a rock over there, they want us to drive onto it," he points out. And he's quite right: for months, the only rocks Opportunity's seen have been either tiny cobbles or, more commonly, flat outcrop patches that make for good driving.
Well, they showed up for an interesting drive. We've got several actual obstacles to avoid, so we'll have a visodom segment at the beginning to help navigate around them, then a short blind drive through a friendly patch, then autonav from there. I don't think we'll quite make it, but if we get really lucky, we'll make it to the ridge and have a spectacular view of the valley beyond. (And an equally good view of whatever we're going to have to cross to get down there.) If that doesn't happen this drive, it'll happen on the next one. And since I'm working on Spirit for the rest of the year, while they continue to investigate Opportunity's IDD problem, I'll be sure to see it.
I can't wait!
2010-12-11
Spirit Sol 690
We didn't get all the data we were supposed to get -- it's stuck someplace on Earth, but hasn't made it to the flight ops LAN yet.
The question is whether we can still do the drive with what we've got. (Or rather, with what we had yesterday: we came in then and planned most of the drive already, since there was no time pressure that way.)
We could do a better drive if we had all the data, but it looks like we've got enough -- at least we have NAVCAM coverage of the areas to the right and left of Miami, which towers over us, like, uh, a tower or something. What we can see of the direct path to Comanche is unpromising; most paths are blocked or worrisomely bumpy, and from what we can see, the one path we find that leads straight to the ridge looks like we'd more or less fall right over it. Visions of Wile E. Coyote dance in my head. Of course, we'd have the rover on autonav, so she'd keep herself safe, but our best estimate, backed up by what we can see of this area from images taken higher on the hill, is that the path down from there is steep and bumpy.
This contrasts with the longer path that leads to the left (east) around Miami -- which I take to calling the Miami Bypass. There the route is relatively smooth along the area we can see, and the earlier images suggest a smooth and gentle path. A longer one, but an easier one; all things considered, the long way might be faster. So we go that way.
It's starting to feel like Old Timer's Day around here. Mark Adler's been here recently, though not today. But Julie Townsend is back, as is Matt Keuneke, and Rob Manning stops by for a visit. As does Steve Squyres, which is always a pleasant surprise. He's on a layover between flights or something, and couldn't stay away.
He also bugs us about the drive we're planning -- why are we going to the left of Miami, not to the right? He's only seen the 2-D versions of the images, as it turns out. I bring up the true-stereo 3-D images in RSVP, and Steve slaps on the goggles. It just takes one glance. "I'm convinced!"
[Next post: sol 692, December 13.]
The question is whether we can still do the drive with what we've got. (Or rather, with what we had yesterday: we came in then and planned most of the drive already, since there was no time pressure that way.)
We could do a better drive if we had all the data, but it looks like we've got enough -- at least we have NAVCAM coverage of the areas to the right and left of Miami, which towers over us, like, uh, a tower or something. What we can see of the direct path to Comanche is unpromising; most paths are blocked or worrisomely bumpy, and from what we can see, the one path we find that leads straight to the ridge looks like we'd more or less fall right over it. Visions of Wile E. Coyote dance in my head. Of course, we'd have the rover on autonav, so she'd keep herself safe, but our best estimate, backed up by what we can see of this area from images taken higher on the hill, is that the path down from there is steep and bumpy.
This contrasts with the longer path that leads to the left (east) around Miami -- which I take to calling the Miami Bypass. There the route is relatively smooth along the area we can see, and the earlier images suggest a smooth and gentle path. A longer one, but an easier one; all things considered, the long way might be faster. So we go that way.
It's starting to feel like Old Timer's Day around here. Mark Adler's been here recently, though not today. But Julie Townsend is back, as is Matt Keuneke, and Rob Manning stops by for a visit. As does Steve Squyres, which is always a pleasant surprise. He's on a layover between flights or something, and couldn't stay away.
He also bugs us about the drive we're planning -- why are we going to the left of Miami, not to the right? He's only seen the 2-D versions of the images, as it turns out. I bring up the true-stereo 3-D images in RSVP, and Steve slaps on the goggles. It just takes one glance. "I'm convinced!"
[Next post: sol 692, December 13.]
2010-12-08
Spirit Sol 687
Second time was the charm. "As usual, a perfect drive!" exclaims the SOWG chair. "As usual," John mutters sardonically, "not as always."
Well, be that as it may. They have indeed revised the drive metric, making us a whopping 100m behind schedule. So we're driving.
Ha, ha! Only kidding. Of course we're not driving! Instead, we're spending the next couple of sols IDDing the patch of outcrop this drive took us to. And, of course, imaging nearby rocks and such. The outcrop is named Algonquin, and the target names have been selected from a list of tongue-twisting Indian names. After listening to the team stumbling over the names in the SOWG room, I thumb the mike and say, "I just wanted to remind everybody that the reason we name features is to make them easier to refer to." This wins laughter and applause. The SOWG chair proposes that our next target namespace be limited to three-letter names.
The IDD sequencing is our usual abbreviated campaign: on the first sol, an MI mosaic followed by an APXS placement; on the second, a RAT brush followed by a repeat of the MI mosaic and APXS placement. John has this well in hand -- true to form, he mostly copies and pastes from a previous campaign -- which leaves me time to look ahead to Friday's drive.
Which doesn't look pretty. About 15m away is a ridge, beyond which we can see terrain, but we have no range data on it -- that is, we have no idea how far away it is. PCAMs usually can give us range data out to 100m or so, so the fact that we're getting nothing suggests that what we see is at least that much farther away. So our strategy must be to drive to the ridge, then switch on autonav and hope for the best beyond that.
If we can even get to the ridge. The terrain here is just rugged enough to make that really, really difficult -- every apparent solution is thwarted by some rock or other.
Moreover, the target beyond, a big red mound called Comanche, is part of the stuff we have no range data for. So we don't know how far away it is, and thus we don't know how long it'll take us to drive there.
Because Monday's such a tight sol, we don't want to plan a drive that sol. Instead, we'll take the unusual step of planning two drives Friday. The first will take us to the ridge, then autonav past it. The second will be an all-autonav drive. Since we're fairly sure Comanche is more than 100m away, and the two drives combined aren't likely to make that much distance, we can just pick a point ~ 100m away and have both drives aim for it.
We've never done this on Spirit, but it's similar to what we did on Opportunity over President's Day weekend, so we're confident it'll work. We end up scrapping the idea, though, less for technical than for staffing reasons. Next week, John will be absent because of his fiancee Helen's surgery, Ashitey is still tied up with debugging Opportunity's IDD, and Chris Leger is focused on the FSW release. This leaves me as the only usual Spirit RP, and I have an unavoidable doctor's appointment Tuesday, a follow-up with my surgeon.
As a result, Sharon's had to schedule two Opportunity RPs, Frank and Jeng, for Tuesday. And we really don't want to make them drive. They're both good RPs, and everything would probably go fine, but the combination of a tight uplink, two RPs somewhat unfamiliar with Spirit, and unknown and likely hazardous terrain makes us nervous enough to call it off.
Instead, Friday we'll plan just one drive, and Monday we'll look at the data from that drive and plan the next one. We won't send that sequence Monday, though -- Monday's schedule is just too tight for that -- we'll hang onto it and have Frank and Jeng send it Tuesday.
This means I get to work on the rovers Monday, instead of spending Monday pushing on the RSVP software release for Phoenix. I can't say I'm disappointed.
[Next post: sol 690, December 11.]
Well, be that as it may. They have indeed revised the drive metric, making us a whopping 100m behind schedule. So we're driving.
Ha, ha! Only kidding. Of course we're not driving! Instead, we're spending the next couple of sols IDDing the patch of outcrop this drive took us to. And, of course, imaging nearby rocks and such. The outcrop is named Algonquin, and the target names have been selected from a list of tongue-twisting Indian names. After listening to the team stumbling over the names in the SOWG room, I thumb the mike and say, "I just wanted to remind everybody that the reason we name features is to make them easier to refer to." This wins laughter and applause. The SOWG chair proposes that our next target namespace be limited to three-letter names.
The IDD sequencing is our usual abbreviated campaign: on the first sol, an MI mosaic followed by an APXS placement; on the second, a RAT brush followed by a repeat of the MI mosaic and APXS placement. John has this well in hand -- true to form, he mostly copies and pastes from a previous campaign -- which leaves me time to look ahead to Friday's drive.
Which doesn't look pretty. About 15m away is a ridge, beyond which we can see terrain, but we have no range data on it -- that is, we have no idea how far away it is. PCAMs usually can give us range data out to 100m or so, so the fact that we're getting nothing suggests that what we see is at least that much farther away. So our strategy must be to drive to the ridge, then switch on autonav and hope for the best beyond that.
If we can even get to the ridge. The terrain here is just rugged enough to make that really, really difficult -- every apparent solution is thwarted by some rock or other.
Moreover, the target beyond, a big red mound called Comanche, is part of the stuff we have no range data for. So we don't know how far away it is, and thus we don't know how long it'll take us to drive there.
Because Monday's such a tight sol, we don't want to plan a drive that sol. Instead, we'll take the unusual step of planning two drives Friday. The first will take us to the ridge, then autonav past it. The second will be an all-autonav drive. Since we're fairly sure Comanche is more than 100m away, and the two drives combined aren't likely to make that much distance, we can just pick a point ~ 100m away and have both drives aim for it.
We've never done this on Spirit, but it's similar to what we did on Opportunity over President's Day weekend, so we're confident it'll work. We end up scrapping the idea, though, less for technical than for staffing reasons. Next week, John will be absent because of his fiancee Helen's surgery, Ashitey is still tied up with debugging Opportunity's IDD, and Chris Leger is focused on the FSW release. This leaves me as the only usual Spirit RP, and I have an unavoidable doctor's appointment Tuesday, a follow-up with my surgeon.
As a result, Sharon's had to schedule two Opportunity RPs, Frank and Jeng, for Tuesday. And we really don't want to make them drive. They're both good RPs, and everything would probably go fine, but the combination of a tight uplink, two RPs somewhat unfamiliar with Spirit, and unknown and likely hazardous terrain makes us nervous enough to call it off.
Instead, Friday we'll plan just one drive, and Monday we'll look at the data from that drive and plan the next one. We won't send that sequence Monday, though -- Monday's schedule is just too tight for that -- we'll hang onto it and have Frank and Jeng send it Tuesday.
This means I get to work on the rovers Monday, instead of spending Monday pushing on the RSVP software release for Phoenix. I can't say I'm disappointed.
[Next post: sol 690, December 11.]
2010-12-06
Spirit Sol 685
The weekend drive faulted out right at the start, before even going anywhere. Painfully, it was really a rookie mistake. They fired off a helper sequence, and, as we usually do, put in a check to see if the helper was present, precluding further driving if not. The problem was, they accidentally reversed the sense of the test, precluding driving if the helper was present. Which of course it was, so the drive kicked off the helper sequence and then refused to go anywhere.
I'm just glad it wasn't me.[1]
Partly as a result of this mistake, we're now well below our drive metric. Also contributing to the problem is that they're going to raise the bar. Recognizing that our power has dropped and will continue to drop, we're revising our estimate of when we need to make it to Home Plate. The result will be more driving -- and more pressure on us to get it right.
Not that they seem to be in a terrible hurry. This drive will be all of 15m or so, to a nearby rock target called Algonquin. Where we'll stop to IDD. "But let's have the discipline," SOWG chair Albert Yen cautions, "to spend only one drive sol there." Since we're in restricted sols, that really means spending two drive sols there -- we'll drive only once more this week.
This mistake is the third time in about a week that we've blown all or part of a drive. The first case was when we hit the tilt limit right at the start of an autonav drive, which cut that sol from something like 40m to something like 20m. The second was when we screwed up the time-of-day limit logic, blowing our turn for comm. (At least that one made all of its planned distance, but the mistake is still troubling.) And now this is the third one.
I talk about these mistakes with John and Ashley, to see if we can come up with fundamental underlying reasons. "Complacency" is John's answer, which might be as true as anything. Still, we come up with a couple of new flight rules, which I write automated checks for. Maybe they'll help.
Later, I talk to Frank about how Opportunity is doing. There's good news and bad news, he tells me. The bad news is that we're still not seeing any motion from Opportunity's IDD. The good news is that Joe Melko has identified a failure mode in which a wire cracks, doubling the resistance. And there seems to be some support for this theory in the data. If this were true, we'd be able to resume normal work by just doubling the current through that joint.
If it's true. I can't help getting my hopes up, but I have to remind myself that we still don't know what's wrong, and the IDD could still be lost forever.
I'm hoping not.
[Next post: sol 687, December 8.]
Footnotes:
[1] Technically, I'm glad it wasn't I.
I'm just glad it wasn't me.[1]
Partly as a result of this mistake, we're now well below our drive metric. Also contributing to the problem is that they're going to raise the bar. Recognizing that our power has dropped and will continue to drop, we're revising our estimate of when we need to make it to Home Plate. The result will be more driving -- and more pressure on us to get it right.
Not that they seem to be in a terrible hurry. This drive will be all of 15m or so, to a nearby rock target called Algonquin. Where we'll stop to IDD. "But let's have the discipline," SOWG chair Albert Yen cautions, "to spend only one drive sol there." Since we're in restricted sols, that really means spending two drive sols there -- we'll drive only once more this week.
This mistake is the third time in about a week that we've blown all or part of a drive. The first case was when we hit the tilt limit right at the start of an autonav drive, which cut that sol from something like 40m to something like 20m. The second was when we screwed up the time-of-day limit logic, blowing our turn for comm. (At least that one made all of its planned distance, but the mistake is still troubling.) And now this is the third one.
I talk about these mistakes with John and Ashley, to see if we can come up with fundamental underlying reasons. "Complacency" is John's answer, which might be as true as anything. Still, we come up with a couple of new flight rules, which I write automated checks for. Maybe they'll help.
Later, I talk to Frank about how Opportunity is doing. There's good news and bad news, he tells me. The bad news is that we're still not seeing any motion from Opportunity's IDD. The good news is that Joe Melko has identified a failure mode in which a wire cracks, doubling the resistance. And there seems to be some support for this theory in the data. If this were true, we'd be able to resume normal work by just doubling the current through that joint.
If it's true. I can't help getting my hopes up, but I have to remind myself that we still don't know what's wrong, and the IDD could still be lost forever.
I'm hoping not.
[Next post: sol 687, December 8.]
Footnotes:
[1] Technically, I'm glad it wasn't I.
2010-12-03
Opportunity Sol 660 (Spirit Sol 682)
I'm not needed to cover Ashitey's shift thisol, so I'm back on Opportunity, where things are not looking up. The day starts with a meeting in which the usual suspects are gathered around the table (or on the phone line) for a review of the spacecraft's state, and to try to figure out where we go from here.
In short: no joy. The IDD is showing small or no motion. To make matters worse, since it's in the stowed configuration, where there's very little room for it to move, we can't command large enough motions to get a lot of data. So there are few new facts to argue about, which is good or bad depending on how you look at it. Mostly bad.
Thisol will be more of the same: a couple of short motions designed to rotate the arm slightly -- just a couple of degrees -- forward, the usual next step in the unstow process. We're going to be a little more liberal in what we allow, permitting the startup current to be high for a little longer, in the hope that this will get it over whatever hump it needs to get over in order to move. And we're taking a bunch of pictures. And keeping our fingers crossed.
[Next post: sol 685, December 6.]
In short: no joy. The IDD is showing small or no motion. To make matters worse, since it's in the stowed configuration, where there's very little room for it to move, we can't command large enough motions to get a lot of data. So there are few new facts to argue about, which is good or bad depending on how you look at it. Mostly bad.
Thisol will be more of the same: a couple of short motions designed to rotate the arm slightly -- just a couple of degrees -- forward, the usual next step in the unstow process. We're going to be a little more liberal in what we allow, permitting the startup current to be high for a little longer, in the hope that this will get it over whatever hump it needs to get over in order to move. And we're taking a bunch of pictures. And keeping our fingers crossed.
[Next post: sol 685, December 6.]
2010-12-01
Spirit Sol 680
Our drive failed. Well, that might be too strong a word. We got through the blind portion, the initial 25m or so, but when we switched to autonav, we lowered the tilt limit (as usual), almost immediately drove over a relatively sharp little ramp, and tilt-faulted.
What makes this more annoying is that, unbeknownst to me, we had terrain meshes covering that entire area -- from an earlier sol, sol 667, when they'd driven up to that point and looked out over it. (I was on Opportunity then and missed that part of the story. When I came in, they'd driven back to examine some outcrop they'd passed by previously.) So we could easily have done a blind drive right through this area -- with a higher tilt limit, so we wouldn't have faulted.
Foo.
Just to make things more awkward, there was a problem at the DSN station, and they somehow lost a bunch of data from our downlink. (The data somehow made it all the way from Mars to Spain, but couldn't quite make it the last few thousand km to California. This is like swimming the Atlantic and drowning a meter from shore.) As a result, we have very little imagery from our current position -- HAZCAMs are all we had last night, though the PCAMs showed up in the morning pass. We're still missing the creamy middle -- I mean, crucial middle -- that's ordinarily supplied by the NCAMs.
We'll be able to do a drive, but it will be mostly autonav and therefore rather short -- but, hey. We still have that imagery from sol 667! John loads up the terrain mesh, and though it shows we've accumulated significant position error, we've still got good enough data to extend the blind drive to 12m or so. That gets us to the start of the PCAM coverage, which would ordinarily mean we'd be able to extend the blind drive even further, but there's some scary-looking stuff at the start of the PCAM coverage that we don't want to blind-drive through. Oh, well.
Anyway, there's a growing sentiment for bypassing Miami in favor of Comanche, a slightly more distant target that's redder (as opposed to Miami's black, which might suggest it's just regular old basalt) and has a more unusual morphology. So we aim this drive for a spot to the right of Miami -- somewhat splitting the difference between it and Comanche. And off we go.
Just after I deliver, I check the data -- and sure enough, the NCAMs have just come down. And they show what we suspected even without them: we'd have been perfectly fine sequencing this entire drive as a blind drive, which would have more than doubled our distance. Too late to change it now. Double-foo.
While I've been covering Ashitey's shifts on this side of the planet, he's been on the other side of the planet, investigating Opportunity's IDD anomaly. And the news isn't good, or at least, that's what Frank tells me. They haven't done anything with Opportunity's IDD since the fault, but they've been busy in the testbed, and their testing there has tended to disconfirm the hypothesis that the IDD is simply temporarily having trouble getting off its hook. Rather, it's looking more and more like a real actuator problem, a hardware problem we won't be able to solve and may not be able to work around.
I'll be back on Opportunity tomorrow morning, starting with an early meeting to explore the results of the tiny move they'll send up tonight. And then we'll know more.
[Next post: sol 682 (Opportunity sol 660), December 3.]
What makes this more annoying is that, unbeknownst to me, we had terrain meshes covering that entire area -- from an earlier sol, sol 667, when they'd driven up to that point and looked out over it. (I was on Opportunity then and missed that part of the story. When I came in, they'd driven back to examine some outcrop they'd passed by previously.) So we could easily have done a blind drive right through this area -- with a higher tilt limit, so we wouldn't have faulted.
Foo.
Just to make things more awkward, there was a problem at the DSN station, and they somehow lost a bunch of data from our downlink. (The data somehow made it all the way from Mars to Spain, but couldn't quite make it the last few thousand km to California. This is like swimming the Atlantic and drowning a meter from shore.) As a result, we have very little imagery from our current position -- HAZCAMs are all we had last night, though the PCAMs showed up in the morning pass. We're still missing the creamy middle -- I mean, crucial middle -- that's ordinarily supplied by the NCAMs.
We'll be able to do a drive, but it will be mostly autonav and therefore rather short -- but, hey. We still have that imagery from sol 667! John loads up the terrain mesh, and though it shows we've accumulated significant position error, we've still got good enough data to extend the blind drive to 12m or so. That gets us to the start of the PCAM coverage, which would ordinarily mean we'd be able to extend the blind drive even further, but there's some scary-looking stuff at the start of the PCAM coverage that we don't want to blind-drive through. Oh, well.
Anyway, there's a growing sentiment for bypassing Miami in favor of Comanche, a slightly more distant target that's redder (as opposed to Miami's black, which might suggest it's just regular old basalt) and has a more unusual morphology. So we aim this drive for a spot to the right of Miami -- somewhat splitting the difference between it and Comanche. And off we go.
Just after I deliver, I check the data -- and sure enough, the NCAMs have just come down. And they show what we suspected even without them: we'd have been perfectly fine sequencing this entire drive as a blind drive, which would have more than doubled our distance. Too late to change it now. Double-foo.
While I've been covering Ashitey's shifts on this side of the planet, he's been on the other side of the planet, investigating Opportunity's IDD anomaly. And the news isn't good, or at least, that's what Frank tells me. They haven't done anything with Opportunity's IDD since the fault, but they've been busy in the testbed, and their testing there has tended to disconfirm the hypothesis that the IDD is simply temporarily having trouble getting off its hook. Rather, it's looking more and more like a real actuator problem, a hardware problem we won't be able to solve and may not be able to work around.
I'll be back on Opportunity tomorrow morning, starting with an early meeting to explore the results of the tiny move they'll send up tonight. And then we'll know more.
[Next post: sol 682 (Opportunity sol 660), December 3.]
2010-11-29
Spirit Sol 678
Opportunity experienced a disturbing failure this weekend: her IDD failed to deploy. We talked about it this weekend long enough to rule out a sequencing error, which meant we shouldn't try to bring in the normal uplink team to try to fix it and salvage the IDD work. It's possible we would have tried to get the right people together, but it was Thanksgiving, and many people were out of town, and so on. And these days we don't engage in heroics. So we let it go until Monday.
Now it's Monday, and it's not immediately clear what we should do about this, because it's not immediately clear what the failure is. We know we got a tiny amount of motion followed by a motor stall, but why? Eric Baumgartner's (well-educated) guess is that we've simply lost a small amount of knowledge as to the position of the IDD. As a result, when the IDD tried to go to the commanded position as part of its unstow routine, it actually failed to get the IDD off of the little hook it stows itself on, and then it couldn't rotate itself away from the hook. This hypothesis has some confirmation in some image analysis Paolo performed, which may show a tiny discrepancy in the arm's position after the last time we stowed it.
This would be relatively straightforward to solve -- not easy, but not dreadfully bad. We'd have to figure out where the arm actually is, then manually guide it through the unstow routine, at which point we'd be able to recalibrate the IDD by driving each of the joints against its hardstop. We might also need to repeat the "Martian Tai Chi" routine that recalibrates the camera models, so that the IDD and the cameras would be in agreement. It would be a long, slow process -- maybe a week or two, all put together -- but in the end we'd be as good as new.
But Eric's explanation isn't the only one that fits the observed facts. Ashitey fears there's something worse going on. One of the actuators in the testbed rover has started to fail, and it stalls out in pretty much the same way as we've just observed Opportunity's joint motor stall. If this is what's going on, it means the IDD has become unreliable -- every once in a while, our IDD work might randomly fail in this way. That would introduce a severe, somewhat random penalty into all IDD work.
With the testbed rover's IDD, the failures are merely annoying; repeat the failed command, and it usually works the second try. If Opportunity's IDD is indeed failing in this way, that means we have an occasional one-sol penalty to look forward to; probably nothing worse than that (though that would be unpleasant). But it could be worse still: the actuator might simply have failed permanently. And if that's what happened, a large fraction of Opportunity's scientific potential just disappeared forever.
There are other possible explanations, too, with varying degrees of benignity. Erickson wants the full-up anomaly investigation, and the right guy to do that is Ashitey. But Ashitey's RP-1 on Spirit today, which would leave him precious little time for something like that. So he and I switch places -- he heads up the investigation, and I go back to Spirit.
Which means I'm driving today -- driving to Miami. (I try to remember not to leave the left turn signal on.) This drive is reasonably simple, snaking around a few rocks before heading more or less straight toward Miami. Well, not quite Miami -- Coral Gables. See, I didn't have an exact target for Miami, just picked a spot in the neighborhood, and Coral Gables is a Miami suburb ....
Maybe you had to be there.
If all goes well, we'll cover a third to a half of the distance to Miami in this drive. (Maybe we'll stop for the night at a HoJo's. Sorry, I can't help it.)
I'd seen a birthday card in the Spirit sequencing room last week, but it hadn't registered on me until today. I pick it up and read it. I figured it was for someone on the team. Turns out I was right; it's for the most important member of our team. It's a birthday card for a one-year-old -- our one-year-old, Spirit. (Or maybe that should be a "birthsol" card, but I'm not picky.) From the nice (and clever) folks at unmannedspaceflight.com.
I love it. We should make them Spirit's godparents or something.
[Next post: sol 680, December 1.]
Now it's Monday, and it's not immediately clear what we should do about this, because it's not immediately clear what the failure is. We know we got a tiny amount of motion followed by a motor stall, but why? Eric Baumgartner's (well-educated) guess is that we've simply lost a small amount of knowledge as to the position of the IDD. As a result, when the IDD tried to go to the commanded position as part of its unstow routine, it actually failed to get the IDD off of the little hook it stows itself on, and then it couldn't rotate itself away from the hook. This hypothesis has some confirmation in some image analysis Paolo performed, which may show a tiny discrepancy in the arm's position after the last time we stowed it.
This would be relatively straightforward to solve -- not easy, but not dreadfully bad. We'd have to figure out where the arm actually is, then manually guide it through the unstow routine, at which point we'd be able to recalibrate the IDD by driving each of the joints against its hardstop. We might also need to repeat the "Martian Tai Chi" routine that recalibrates the camera models, so that the IDD and the cameras would be in agreement. It would be a long, slow process -- maybe a week or two, all put together -- but in the end we'd be as good as new.
But Eric's explanation isn't the only one that fits the observed facts. Ashitey fears there's something worse going on. One of the actuators in the testbed rover has started to fail, and it stalls out in pretty much the same way as we've just observed Opportunity's joint motor stall. If this is what's going on, it means the IDD has become unreliable -- every once in a while, our IDD work might randomly fail in this way. That would introduce a severe, somewhat random penalty into all IDD work.
With the testbed rover's IDD, the failures are merely annoying; repeat the failed command, and it usually works the second try. If Opportunity's IDD is indeed failing in this way, that means we have an occasional one-sol penalty to look forward to; probably nothing worse than that (though that would be unpleasant). But it could be worse still: the actuator might simply have failed permanently. And if that's what happened, a large fraction of Opportunity's scientific potential just disappeared forever.
There are other possible explanations, too, with varying degrees of benignity. Erickson wants the full-up anomaly investigation, and the right guy to do that is Ashitey. But Ashitey's RP-1 on Spirit today, which would leave him precious little time for something like that. So he and I switch places -- he heads up the investigation, and I go back to Spirit.
Which means I'm driving today -- driving to Miami. (I try to remember not to leave the left turn signal on.) This drive is reasonably simple, snaking around a few rocks before heading more or less straight toward Miami. Well, not quite Miami -- Coral Gables. See, I didn't have an exact target for Miami, just picked a spot in the neighborhood, and Coral Gables is a Miami suburb ....
Maybe you had to be there.
If all goes well, we'll cover a third to a half of the distance to Miami in this drive. (Maybe we'll stop for the night at a HoJo's. Sorry, I can't help it.)
I'd seen a birthday card in the Spirit sequencing room last week, but it hadn't registered on me until today. I pick it up and read it. I figured it was for someone on the team. Turns out I was right; it's for the most important member of our team. It's a birthday card for a one-year-old -- our one-year-old, Spirit. (Or maybe that should be a "birthsol" card, but I'm not picky.) From the nice (and clever) folks at unmannedspaceflight.com.
I love it. We should make them Spirit's godparents or something.
[Next post: sol 680, December 1.]
2010-11-27
Spirit Sol 676
Thisol should be easy. Spirit's been IDDing a patch of outcrop named "Seminole," on the target "Osceola." (It's coming up on Thanksgiving, and since we did Pilgrim names last year, we're doing American Indian names this year.) And the big science waypoint we've been heading toward has been renamed from "Waypoint" to "Miami." I'm getting a kick out of this, having gone to Seminole High School and all. But I digress.
So all we're doing is repeating the same stuff they did yesterday on Osceola, only we're changing the target to Abiaka (another Seminole chief). Load up those sequences, change the targets and a few sequence IDs, and you're done.
As is so often the case, it's not that simple. Abiaka unfortunately turns out to be a little too vertical (strictly speaking, Abiaka is a too-vertical normal to a too-horizontal rock surface) -- a problem because the IDD gets confused when you ask it to move straight up and down. That has to do with the target being too close to a wrist-flip condition, where the IDD could choose almost equally well between two different wrist configurations, and if conditions on Mars vary only slightly from the predicts, it could change its mind in the middle and decide it can't do what we want.
The remedy for this is known but tedious: tweak the angle of the target, so that it's tilted slightly away from vertical, until the simulation says you've gone far enough. What complicates this is that we can't tilt the normal too far, or the RAT brushing won't work. Also making it more complex is that John's having his shadow, Ashley, handle the IDD sequence, and while she generally knows what she's doing, she's still new to this and somewhat slow at it.
Oy. Well, it takes forever, but it gets done, and that's what matters.
[Next post: sol 678, November 29.]
So all we're doing is repeating the same stuff they did yesterday on Osceola, only we're changing the target to Abiaka (another Seminole chief). Load up those sequences, change the targets and a few sequence IDs, and you're done.
As is so often the case, it's not that simple. Abiaka unfortunately turns out to be a little too vertical (strictly speaking, Abiaka is a too-vertical normal to a too-horizontal rock surface) -- a problem because the IDD gets confused when you ask it to move straight up and down. That has to do with the target being too close to a wrist-flip condition, where the IDD could choose almost equally well between two different wrist configurations, and if conditions on Mars vary only slightly from the predicts, it could change its mind in the middle and decide it can't do what we want.
The remedy for this is known but tedious: tweak the angle of the target, so that it's tilted slightly away from vertical, until the simulation says you've gone far enough. What complicates this is that we can't tilt the normal too far, or the RAT brushing won't work. Also making it more complex is that John's having his shadow, Ashley, handle the IDD sequence, and while she generally knows what she's doing, she's still new to this and somewhat slow at it.
Oy. Well, it takes forever, but it gets done, and that's what matters.
[Next post: sol 678, November 29.]
2010-11-19
Opportunity Sol 649 (Spirit Sol 668)
Meanwhile, on the other side of the planet ... Opportunity is still skirting the rim of Erebus Crater, making her way to the potential entry point we can see from orbital data. Thisol's plan is to finish up a little IDD work, then drive about 50m -- mostly along outcrop -- toward a flat patch of outcrop we can see in the distance.
This plan is complicated by the fact that the flat patch of outcrop isn't so flat. Looking at it in stereo, we realize that it's actually a somewhat steep face, rising about 50cm over less than 2m. (The fact that the outcrop seems to have some delta-Z makes me think the science team might be interested in it, as it might provide some layered rock to examine. Cooper, ever eager to get moving, is comically reluctant to point this out to them, but I do it anyway.) The ends of the patch are still relatively flat, though, so we decide to aim toward those instead.
Our drive sequence is further complicated by the fact that we're heading at about 180, but need to turn for comm to about 115 -- a large counterclockwise turn, problematic on this vehicle because of the stuck right-front steering actuator. If we knew in advance what our heading would be for the comm turn, that would make it a lot easier to sequence. But we don't, because toward the end there's a slight change in direction: we'll be headed either about 175 or 185 when the time-of-day limit hits. And the only tool we have to make the sequence conditional on the vehicle's yaw gives awkward results around 180, because it gives its answers in the range of -180 to +180 (actually -pi to +pi, since it's in radians) -- nothing we can't sequence around, it's just one more damn thing.
All this still wouldn't be too bad, if Cooper or I were driving. But we decided to let Paolo do it, since he needs the experience. He actually handles it really well, making good decisions and so on. The only problem is that he's still somewhat new, and therefore somewhat slow, and this is a complicated sol, and, well ... it takes a long time. I sit with him pretty much the whole time and strive to stay cheerful and positive. Add that to the list of challenges.
Oh, but we're not through yet. Because I'd been sitting with Paolo, I hadn't had a chance to closely examine Brian's IDD sequence. During the CAM, I discover that the MI poker gets awfully close to a rock during a lateral move. There's less than 1cm of clearance in the prediction, which is within our error budget. Luckily for me, Ken Herkenhoff -- the MI PI, basically the designer and builder and owner of the instrument -- is sitting in the room. (He's normally one of the remote scientists, but is out here to participate in training the new crop.) So I put the CAM on hold, show Ken the situation, and ask his advice.
Now, the MI poker has a bit of spring steel near its base, allowing it to flex safely in just such circumstances as these. And the simulation does show that we have clearance, albeit not much clearance. What's more, we're first touching the surface near the potential contact point, so any error about the location of that point will likely be accounted for -- if our imagery is misleading about that, it should be misleading in the same way about the nearby spot we'll touch, and the sequencing is relative to the touched location, so the error will effectively magically disappear. All this taken into account, Ken's comfortable with proceeding as planned, so we go ahead. (What occurs to me later, as well, is that the simulation we're looking at is actually relative to the 1cm overdrive position -- which aleady accounts for our uncertainty about the terrain -- which means we'll likely have 1cm more clearance than shown. Even better.)
We also realize another place where there's a potential problem. We're in that phase where the Earth-Mars time difference requires us to plan several sols ahead, so we don't yet know the true state of the IDD at the beginning of this sol's sequences -- for the very good reason that that stuff simply hasn't happened yet. And since we're starting (we hope) with the MB in contact with the CCT, we have to disable self-collision checking at the beginning of the sequence in order to allow ourselves to retract the IDD into free space. The problem is, if the planned IDD sequences don't go as predicted, the IDD could start off sol 649 someplace unexpected, and disabling self-collision checking is not what you want to do under those circumstances.
Our solution is to remove the IDD sequence from the bundle -- we'll call in Sunday morning, after we've looked at the data, and we'll send this IDD sequence to the spacecraft only after we know it's done what we expected it to do. This potentially means we'll have to drop the IDD sequence as well as the subsequent drive, since the rover won't drive when the IDD is unstowed -- and it also means we've all got some work to do Saturday night and Sunday morning -- but that's a lot better than risking the IDD.
At long last, we complete the CAM -- and not a moment too soon, as it's been a long and grueling day and we're all getting kind of punchy -- and I realize we've made yet another mistake.
"We're not going to change anything," I say carefully, "but we did screw something up." Normally, our drive sequences detect absent helper sequences, precluding further driving if a helper is missing. (This would mean, for instance, that we were unable to perform a slip check; without that, we don't know whether the vehicle is bogged down, so we don't want to keep trying to drive for tens of meters, making the situation worse.) Our predicted downlink is so poor thisol, however, that we wanted to be sure we'd turn for comm even if one of the helper sequences didn't make it on board. That's rare, but it happens, whether due to a human screw-up somewhere in the process, equipment problems at the DSN stations, or simply because of interference in the long journey of the radio waves from Earth to Mars.
So we pulled a clever trick: during the sequence, we abuse the OK-to-IDD flag -- if a helper is absent and we preclude driving for that reason, we also preclude IDDing. Later, the sequence checks whether we're OK to IDD, and if not, that means a helper must have been missing, so we re-enable driving (and IDDing) so that we can turn for comm.
"What we forgot," I conclude, "is that if you try to drive while driving is precluded -- which we would -- you also get a goal error. And we don't clear that goal error, so the turn for comm still wouldn't happen."
But this bites us only if a helper doesn't make it onto the vehicle -- a rare event -- and it doesn't truly hurt us, it just fails to help us. And at this point on this sol, nobody wants to go back and fix it. So we don't.
[Next post: sol 676, November 27.]
This plan is complicated by the fact that the flat patch of outcrop isn't so flat. Looking at it in stereo, we realize that it's actually a somewhat steep face, rising about 50cm over less than 2m. (The fact that the outcrop seems to have some delta-Z makes me think the science team might be interested in it, as it might provide some layered rock to examine. Cooper, ever eager to get moving, is comically reluctant to point this out to them, but I do it anyway.) The ends of the patch are still relatively flat, though, so we decide to aim toward those instead.
Our drive sequence is further complicated by the fact that we're heading at about 180, but need to turn for comm to about 115 -- a large counterclockwise turn, problematic on this vehicle because of the stuck right-front steering actuator. If we knew in advance what our heading would be for the comm turn, that would make it a lot easier to sequence. But we don't, because toward the end there's a slight change in direction: we'll be headed either about 175 or 185 when the time-of-day limit hits. And the only tool we have to make the sequence conditional on the vehicle's yaw gives awkward results around 180, because it gives its answers in the range of -180 to +180 (actually -pi to +pi, since it's in radians) -- nothing we can't sequence around, it's just one more damn thing.
All this still wouldn't be too bad, if Cooper or I were driving. But we decided to let Paolo do it, since he needs the experience. He actually handles it really well, making good decisions and so on. The only problem is that he's still somewhat new, and therefore somewhat slow, and this is a complicated sol, and, well ... it takes a long time. I sit with him pretty much the whole time and strive to stay cheerful and positive. Add that to the list of challenges.
Oh, but we're not through yet. Because I'd been sitting with Paolo, I hadn't had a chance to closely examine Brian's IDD sequence. During the CAM, I discover that the MI poker gets awfully close to a rock during a lateral move. There's less than 1cm of clearance in the prediction, which is within our error budget. Luckily for me, Ken Herkenhoff -- the MI PI, basically the designer and builder and owner of the instrument -- is sitting in the room. (He's normally one of the remote scientists, but is out here to participate in training the new crop.) So I put the CAM on hold, show Ken the situation, and ask his advice.
Now, the MI poker has a bit of spring steel near its base, allowing it to flex safely in just such circumstances as these. And the simulation does show that we have clearance, albeit not much clearance. What's more, we're first touching the surface near the potential contact point, so any error about the location of that point will likely be accounted for -- if our imagery is misleading about that, it should be misleading in the same way about the nearby spot we'll touch, and the sequencing is relative to the touched location, so the error will effectively magically disappear. All this taken into account, Ken's comfortable with proceeding as planned, so we go ahead. (What occurs to me later, as well, is that the simulation we're looking at is actually relative to the 1cm overdrive position -- which aleady accounts for our uncertainty about the terrain -- which means we'll likely have 1cm more clearance than shown. Even better.)
We also realize another place where there's a potential problem. We're in that phase where the Earth-Mars time difference requires us to plan several sols ahead, so we don't yet know the true state of the IDD at the beginning of this sol's sequences -- for the very good reason that that stuff simply hasn't happened yet. And since we're starting (we hope) with the MB in contact with the CCT, we have to disable self-collision checking at the beginning of the sequence in order to allow ourselves to retract the IDD into free space. The problem is, if the planned IDD sequences don't go as predicted, the IDD could start off sol 649 someplace unexpected, and disabling self-collision checking is not what you want to do under those circumstances.
Our solution is to remove the IDD sequence from the bundle -- we'll call in Sunday morning, after we've looked at the data, and we'll send this IDD sequence to the spacecraft only after we know it's done what we expected it to do. This potentially means we'll have to drop the IDD sequence as well as the subsequent drive, since the rover won't drive when the IDD is unstowed -- and it also means we've all got some work to do Saturday night and Sunday morning -- but that's a lot better than risking the IDD.
At long last, we complete the CAM -- and not a moment too soon, as it's been a long and grueling day and we're all getting kind of punchy -- and I realize we've made yet another mistake.
"We're not going to change anything," I say carefully, "but we did screw something up." Normally, our drive sequences detect absent helper sequences, precluding further driving if a helper is missing. (This would mean, for instance, that we were unable to perform a slip check; without that, we don't know whether the vehicle is bogged down, so we don't want to keep trying to drive for tens of meters, making the situation worse.) Our predicted downlink is so poor thisol, however, that we wanted to be sure we'd turn for comm even if one of the helper sequences didn't make it on board. That's rare, but it happens, whether due to a human screw-up somewhere in the process, equipment problems at the DSN stations, or simply because of interference in the long journey of the radio waves from Earth to Mars.
So we pulled a clever trick: during the sequence, we abuse the OK-to-IDD flag -- if a helper is absent and we preclude driving for that reason, we also preclude IDDing. Later, the sequence checks whether we're OK to IDD, and if not, that means a helper must have been missing, so we re-enable driving (and IDDing) so that we can turn for comm.
"What we forgot," I conclude, "is that if you try to drive while driving is precluded -- which we would -- you also get a goal error. And we don't clear that goal error, so the turn for comm still wouldn't happen."
But this bites us only if a helper doesn't make it onto the vehicle -- a rare event -- and it doesn't truly hurt us, it just fails to help us. And at this point on this sol, nobody wants to go back and fix it. So we don't.
[Next post: sol 676, November 27.]
2010-11-17
Spirit Sol 666
Sol 666: the sol of the Beast. It's a real shame this didn't happen to be Halloween. Maybe thisol should be Martian Halloween.
If there's a Beast involved, he's being awfully good to us. Yestersol's drive went just as expected, a 42m drive -- not much farther than the previous sol, but in a shorter time, thanks to (ahem) remembering to turn on step-skipping during the autonav segment.
Other things are looking up, as well. Our flash volume and data-product count are both still high, but dropping, like the late stages of a Flash Flu. We still need to exercise some restraint; SOWG chair Wendy Calvin proposes alternating atmospheric science sols and geological science sols as a way to reduce flash pressure. Driving is the thing we won't cut -- at least, not entirely, though we do end up shortening the duration once again thisol -- since we have a definite goal and need to make continual progress. The sun is tracking northward in the sky once more, and it isn't going to wait for our little rover, no matter how much she needs and deserves it to.
Yestersol's drive left us about 40m from a small ridge, over which we can just barely see the current science goal (a pile of jumbly black stuff creatively named "Waypoint"). It's 150m away -- we had been thinking it was somewhat closer -- and with a clear, blind-drive-safe 35m to the ridge, we could have made 80m of progress if given three hours to drive (autonav willing). As it is, they cut the drive back to include only the blind part, and maybe a few meters of autonav. We'll make only 40 to 45m.
There's a lot of difference between those two numbers. But if that's all we can afford, that's what we'll do.
[Next post: sol 668 (Opportunity sol 649), November 19.]
Courtesy NASA/JPL-Caltech. The creatively named "Waypoint."
If there's a Beast involved, he's being awfully good to us. Yestersol's drive went just as expected, a 42m drive -- not much farther than the previous sol, but in a shorter time, thanks to (ahem) remembering to turn on step-skipping during the autonav segment.
Other things are looking up, as well. Our flash volume and data-product count are both still high, but dropping, like the late stages of a Flash Flu. We still need to exercise some restraint; SOWG chair Wendy Calvin proposes alternating atmospheric science sols and geological science sols as a way to reduce flash pressure. Driving is the thing we won't cut -- at least, not entirely, though we do end up shortening the duration once again thisol -- since we have a definite goal and need to make continual progress. The sun is tracking northward in the sky once more, and it isn't going to wait for our little rover, no matter how much she needs and deserves it to.
Yestersol's drive left us about 40m from a small ridge, over which we can just barely see the current science goal (a pile of jumbly black stuff creatively named "Waypoint"). It's 150m away -- we had been thinking it was somewhat closer -- and with a clear, blind-drive-safe 35m to the ridge, we could have made 80m of progress if given three hours to drive (autonav willing). As it is, they cut the drive back to include only the blind part, and maybe a few meters of autonav. We'll make only 40 to 45m.
There's a lot of difference between those two numbers. But if that's all we can afford, that's what we'll do.
[Next post: sol 668 (Opportunity sol 649), November 19.]
Courtesy NASA/JPL-Caltech. The creatively named "Waypoint."
2010-11-16
Spirit Sol 665
Ah, heck. We meant to make 60m of progress, and came up with only 37m.
How did this happen? Let's see, autonav imaging takes roughly 3 minutes per 50cm step, which means we should expect to take 20 such steps per hour, or -- oops -- 10m per hour. And we spent two hours on autonav, as predicted, and thus made only about 20m of progress in that time.
Both Ashitey and I were remembering that autonav makes more like 35m per hour. But this is true only when we turn on a feature called "step-skipping." Instead of "image, step, image, step," it does "image, step, step, image, step, step," or even "image, step, step, step, image, step, step, step." Since the "image" part is the slow part, this makes a dramatic improvement in how fast we can drive.
But since this is less safe, since the rover moves farther before taking images of the world around it, it's not the default.
Duh.
Thisol we decide we'll turn on step-skipping.
To make matters worse, we had a poor downlink, and we have more data products on board than we'd like. (We have to worry not only about the total amount of data on the rover but also the number of separate data products.) SOWG chair John Grant proposes cutting the drive short as a way of reducing the generation of new data products until some old ones can be downlinked.
"Nah," I say, "we're not gonna do that."
"But if we don't, the rover will start autodeleting old science products," Emily Eelkema chimes in.
That's what I was waiting for. Since it's been on my mind, I say: "Forget those old MI images! They're history! Let it go! Embrace change!"
They cut the drive short.
Well, it won't be too bad. With step-skipping on, we'll make the same amount of distance as today (about 40m) in the shortened time -- still a decent amount to show for ourselves.
As long as we're changing things in the sequence ... a significant contributor to the drive time is the periodic slip-check sequence. Since it has to invoke visodom, and all the complex 3-D processing that goes along with it, this tends to chew up a lot of duration. What's more, since we're now fairly confident we can make it all the way downhill, most of the original rationale for the slip checks (ensuring we'd be able to make it back to the top of Husband Hill if we got into a bad spot along the way) has disappeared. So I ask Ashitey what he thinks about taking them out.
"Maybe after the Martian-year anniversary," he says. It's next week. "We want both rovers healthy for that."
This is wisdom. I let it go.
Late in the day, Justin Maki points out an article on the Aviation Week Web site. It's something Craig Covault wrote about his recent visit. I didn't realize it at the time, but he was still here for our big discussion about IDDing Hillary.
Emily seems a little worried about the article, but I'm not. "I'm glad he was here for that," I say. "I think it shows us at our best. We had a real concern about a genuinely difficult and novel problem, we took it seriously, we did the analysis we had to do, we proceeded with the right degree of caution -- and in the end, we got the science."
Covault's article is hampered somewhat by the fact that he was on the other end of a teleconference, listening to unfamiliar voices. As a result, some of the details get smudged. But taking that into account, I think he got the big things right: our dedication, our talent, our -- well, damn it, I'm not ashamed to say our brilliance.
Because we're damn good at this. Even if we do forget to turn on step-skipping once in a while.
How did this happen? Let's see, autonav imaging takes roughly 3 minutes per 50cm step, which means we should expect to take 20 such steps per hour, or -- oops -- 10m per hour. And we spent two hours on autonav, as predicted, and thus made only about 20m of progress in that time.
Both Ashitey and I were remembering that autonav makes more like 35m per hour. But this is true only when we turn on a feature called "step-skipping." Instead of "image, step, image, step," it does "image, step, step, image, step, step," or even "image, step, step, step, image, step, step, step." Since the "image" part is the slow part, this makes a dramatic improvement in how fast we can drive.
But since this is less safe, since the rover moves farther before taking images of the world around it, it's not the default.
Duh.
Thisol we decide we'll turn on step-skipping.
To make matters worse, we had a poor downlink, and we have more data products on board than we'd like. (We have to worry not only about the total amount of data on the rover but also the number of separate data products.) SOWG chair John Grant proposes cutting the drive short as a way of reducing the generation of new data products until some old ones can be downlinked.
"Nah," I say, "we're not gonna do that."
"But if we don't, the rover will start autodeleting old science products," Emily Eelkema chimes in.
That's what I was waiting for. Since it's been on my mind, I say: "Forget those old MI images! They're history! Let it go! Embrace change!"
They cut the drive short.
Well, it won't be too bad. With step-skipping on, we'll make the same amount of distance as today (about 40m) in the shortened time -- still a decent amount to show for ourselves.
As long as we're changing things in the sequence ... a significant contributor to the drive time is the periodic slip-check sequence. Since it has to invoke visodom, and all the complex 3-D processing that goes along with it, this tends to chew up a lot of duration. What's more, since we're now fairly confident we can make it all the way downhill, most of the original rationale for the slip checks (ensuring we'd be able to make it back to the top of Husband Hill if we got into a bad spot along the way) has disappeared. So I ask Ashitey what he thinks about taking them out.
"Maybe after the Martian-year anniversary," he says. It's next week. "We want both rovers healthy for that."
This is wisdom. I let it go.
Late in the day, Justin Maki points out an article on the Aviation Week Web site. It's something Craig Covault wrote about his recent visit. I didn't realize it at the time, but he was still here for our big discussion about IDDing Hillary.
Emily seems a little worried about the article, but I'm not. "I'm glad he was here for that," I say. "I think it shows us at our best. We had a real concern about a genuinely difficult and novel problem, we took it seriously, we did the analysis we had to do, we proceeded with the right degree of caution -- and in the end, we got the science."
Covault's article is hampered somewhat by the fact that he was on the other end of a teleconference, listening to unfamiliar voices. As a result, some of the details get smudged. But taking that into account, I think he got the big things right: our dedication, our talent, our -- well, damn it, I'm not ashamed to say our brilliance.
Because we're damn good at this. Even if we do forget to turn on step-skipping once in a while.
2010-11-15
Spirit Sol 664
Many writers and philosophers have spoken of the same idea: life is change, the Eternal Now, you can't step in the same river twice, etc. This concept is logically sound, and I should embrace it. But I have a really hard time with change, especially when the team's changing.
The team's changing.
In a good way; at least nobody I like is leaving. We're bringing in a raft of new scientists, and they're sitting there monitoring the SOWG meeting. It feels weird to be, increasingly, an old-timer on this project that I still think of as the Fresh New Thing. But more, I'm worried that the new scientists are harbingers of unhappy days ahead, days when some of the scientists I've known and liked will be leaving.
I'm sure I'll like the new people, too. But it won't be the same. The moving finger writes, and, having writ, moves on.
We're doing okay on our drive template. Unlike the primary mission, the scientists are actually trying to keep ahead of the line on the graph that shows how far the rover needs to progress each sol. We'll take a hit at Thanksgiving and maybe another at Christmas, but for now we're ahead of the line, even after turning around and going back to spend a week or so IDDing Larry's Bench. Today we're going to make the situation even better, by plowing ahead another sixty meters.
The gaggle of new scientists has brought back a familiar face, Steve Squyres, who's still kvelling about our recent 100m drive (okay, 94m) and looking forward to this one. But, he points out, "You can't slide anything past this uplink team -- even if you try to blow past something interesting, they'll make you turn around and go back!"
Maybe I should figure out a way to cow the new scientists now, before they can develop a troublesome attitude. Change can also be seen as an opportunity, of course.
The team's changing.
In a good way; at least nobody I like is leaving. We're bringing in a raft of new scientists, and they're sitting there monitoring the SOWG meeting. It feels weird to be, increasingly, an old-timer on this project that I still think of as the Fresh New Thing. But more, I'm worried that the new scientists are harbingers of unhappy days ahead, days when some of the scientists I've known and liked will be leaving.
I'm sure I'll like the new people, too. But it won't be the same. The moving finger writes, and, having writ, moves on.
We're doing okay on our drive template. Unlike the primary mission, the scientists are actually trying to keep ahead of the line on the graph that shows how far the rover needs to progress each sol. We'll take a hit at Thanksgiving and maybe another at Christmas, but for now we're ahead of the line, even after turning around and going back to spend a week or so IDDing Larry's Bench. Today we're going to make the situation even better, by plowing ahead another sixty meters.
The gaggle of new scientists has brought back a familiar face, Steve Squyres, who's still kvelling about our recent 100m drive (okay, 94m) and looking forward to this one. But, he points out, "You can't slide anything past this uplink team -- even if you try to blow past something interesting, they'll make you turn around and go back!"
Maybe I should figure out a way to cow the new scientists now, before they can develop a troublesome attitude. Change can also be seen as an opportunity, of course.
2010-11-13
Spirit Sol 662
This was supposed to be an easy sol -- just an IDD tool change, a sequence I could crank out in a couple of minutes. It turns out to be even easier than that -- no RP sequences at all.
I spend the morning working with Ashley to build a candidate drive sequence for Monday. We don't get many opportunities to let an RP-in-training do something like this when there's no time pressure, so it seems like the perfect time.
When we're all done, we've got a sequence that Ashitey will build on for our first drive next week. And I go off to start clearing away the crushing backlog of other work.
[Next post: sol 664, November 15.]
I spend the morning working with Ashley to build a candidate drive sequence for Monday. We don't get many opportunities to let an RP-in-training do something like this when there's no time pressure, so it seems like the perfect time.
When we're all done, we've got a sequence that Ashitey will build on for our first drive next week. And I go off to start clearing away the crushing backlog of other work.
[Next post: sol 664, November 15.]
2010-11-10
Spirit Sol 660
Today we're planning two sols, 660 and 661. Spirit finds herself poised before an enticing chunk of outcrop called "Larry's Bench," and the question on the science team's mind is whether we want to IDD the part we can reach, or reposition the rover to IDD another part, just to the right.
At first, this seems like a silly question to me. How can this bit of outcrop be so much different from a chunk of the same stuff, a few centimeters away, that it's worth spending even one sol to reach the other bit? (Indeed, it would end up being two sols; VEX needs at least one of our uplinks.)
The answer, it turns out, is dust. The stuff in front of us is pretty dusty, and there's no very good surface to RAT-brush it. But the competing section is already less dusty and has a broader and flatter surface that may be more amenable to brushing.
But maybe we can brush the part in front of us ... and so the discussion goes on, and on, and on. The SOWG meeting, which we normally try to finish in under an hour, goes to an hour and a half. And a good chunk of it is spent going in circles, at that. Alicia Falacarro -- now Alicia Vaughan -- can't stand it when the meetings go this way. She looks at me, rolls her eyes, and groans, "Definitely the J.V. team today."
But the answer in the end is that we're going to IDD, not drive, and the IDD work is reasonably straightforward. (Ed Guinness names the IDD target "Thrasher" -- "since that's what we did in this meeting.") Indeed, Ashitey does most of it. It's your standard unstow, MB touch, MI-2x2x5+stereo, MB re-touch, RAT-brush, post-brush MI-2x2x5+stereo, MB re-touch, APXS kind of sol. Oh, and then another change-tool back to MB on the following sol.
Matt Heverly is shadowing me today, so I take the time to walk him carefully through the whole sequence and talk over his questions in detail. The price of this attention, of course, is that I put him on the hook to do the walkthrough at the CAM. And a good job he does of it, too.
Sol 661. Sol six hundred and sixty-one. Unfathomable. It's almost a Martian year -- a Martian year being 669.7 sols. They're going to celebrate it in von Karman next week.
Six hundred. And sixty-one.
And counting.
[Next post: sol 662, November 13.]
Courtesy NASA/JPL-Caltech. Larry's Bench, right between our wheels.
At first, this seems like a silly question to me. How can this bit of outcrop be so much different from a chunk of the same stuff, a few centimeters away, that it's worth spending even one sol to reach the other bit? (Indeed, it would end up being two sols; VEX needs at least one of our uplinks.)
The answer, it turns out, is dust. The stuff in front of us is pretty dusty, and there's no very good surface to RAT-brush it. But the competing section is already less dusty and has a broader and flatter surface that may be more amenable to brushing.
But maybe we can brush the part in front of us ... and so the discussion goes on, and on, and on. The SOWG meeting, which we normally try to finish in under an hour, goes to an hour and a half. And a good chunk of it is spent going in circles, at that. Alicia Falacarro -- now Alicia Vaughan -- can't stand it when the meetings go this way. She looks at me, rolls her eyes, and groans, "Definitely the J.V. team today."
But the answer in the end is that we're going to IDD, not drive, and the IDD work is reasonably straightforward. (Ed Guinness names the IDD target "Thrasher" -- "since that's what we did in this meeting.") Indeed, Ashitey does most of it. It's your standard unstow, MB touch, MI-2x2x5+stereo, MB re-touch, RAT-brush, post-brush MI-2x2x5+stereo, MB re-touch, APXS kind of sol. Oh, and then another change-tool back to MB on the following sol.
Matt Heverly is shadowing me today, so I take the time to walk him carefully through the whole sequence and talk over his questions in detail. The price of this attention, of course, is that I put him on the hook to do the walkthrough at the CAM. And a good job he does of it, too.
Sol 661. Sol six hundred and sixty-one. Unfathomable. It's almost a Martian year -- a Martian year being 669.7 sols. They're going to celebrate it in von Karman next week.
Six hundred. And sixty-one.
And counting.
[Next post: sol 662, November 13.]
Courtesy NASA/JPL-Caltech. Larry's Bench, right between our wheels.
2010-11-08
Opportunity Sol 637 (Spirit Sol 658)
Thisol I'm RP-1 on Opportunity, and Paolo's soloing as RP-2. But it's a straightforward IDD sol, and he hasn't had many chances to practice IDDing, so he asks if he can carry the bulk of it and I accept.
This is convenient for me, since I'm putting together a training presentation. At Sharon's prodding, I'm doing something Saina asked for, oh, I've lost track of how many months ago: trying to educate the rest of the uplink team about what rover drivers do, what all the mobility commands mean, and so on. This presentation has turned into something of a monster -- about 75 slides -- and I'm not done with it yet. So being able to turn Paolo loose, even part of the time, and work on this presentation, turns out to be an enormous help.
Of course, when he needs me, I watch over Paolo's shoulder and guide him through the process. But he's doing pretty well already.
When we catch a break, he asks me wonderingly, "How did you guys train originally?"
I tell him about the thread tests, the limited training sessions we had, and so on. But it wasn't really much. "If a new candidate rover driver came along who knew only as much as I knew when we landed, I wouldn't let him anywhere near the rovers," I say truthfully. "But we made it through somehow."[1]
How, I'll never know. Mars must have been in a good mood.
[Next post: sol 660, November 10.]
Footnotes:
[1] Sadly, five years on, this has only become even more true: it's gotten harder and harder to drive the rovers, because the list of things you need to know has expanded so much. Also, our training opportunities are more limited right now: Spirit's not driving, and Opportunity ops tends to be very routine, with limited IDD work and drives that all look much the same as we cruise across the plains. So it's become harder and harder to train new people, no matter how sharp and talented they might be (as both of our current trainees are).
This is convenient for me, since I'm putting together a training presentation. At Sharon's prodding, I'm doing something Saina asked for, oh, I've lost track of how many months ago: trying to educate the rest of the uplink team about what rover drivers do, what all the mobility commands mean, and so on. This presentation has turned into something of a monster -- about 75 slides -- and I'm not done with it yet. So being able to turn Paolo loose, even part of the time, and work on this presentation, turns out to be an enormous help.
Of course, when he needs me, I watch over Paolo's shoulder and guide him through the process. But he's doing pretty well already.
When we catch a break, he asks me wonderingly, "How did you guys train originally?"
I tell him about the thread tests, the limited training sessions we had, and so on. But it wasn't really much. "If a new candidate rover driver came along who knew only as much as I knew when we landed, I wouldn't let him anywhere near the rovers," I say truthfully. "But we made it through somehow."[1]
How, I'll never know. Mars must have been in a good mood.
[Next post: sol 660, November 10.]
Footnotes:
[1] Sadly, five years on, this has only become even more true: it's gotten harder and harder to drive the rovers, because the list of things you need to know has expanded so much. Also, our training opportunities are more limited right now: Spirit's not driving, and Opportunity ops tends to be very routine, with limited IDD work and drives that all look much the same as we cruise across the plains. So it's become harder and harder to train new people, no matter how sharp and talented they might be (as both of our current trainees are).
2010-11-05
Spirit Sol 655
The plan is to drive 17m at an azimuth of 130deg. But I have a better idea. We're perched at the top of a local hillock, and spread out below us is Haskin Ridge. We've got a beautiful view of it, a superb terrain mesh -- and, best of all, three hours of driving in the plan.
I think we can do more like 100m.
And it should be fairly simple. The hillock itself is pretty steep -- we might see 22deg tilt on the way down it -- but that's only for the first 15m or so. After that, the terrain quickly shallows out, so that we're in a far more forgiving 8-10deg of tilt all the way down. We can do 50m blind, and the terrain beyond looks decent for autonav. We don't get opportunities like this so often that we can just throw them away.
The day has its ups and downs. It turns out they forgot about a coordinated observation with HST, so they have to completely swap the plan around, but we still end up with about three hours of driving time. And Squyres decides they'd rather we drove at an azimuth of more like 100deg than 130deg, so that they can do a long-baseline stereo observation along the ridge. But at the end of the day, we've got a 110m drive planned, and a real shot at making it to the end -- or at least breaking 100m.
I expect to have real problems selling this to management, but everyone's pretty enthusiastic. When I tell Jim how far we're driving, he says, "I'd kill to get that on Opportunity right now." Then he reconsiders. "Well ... I'd wound," he amends.
Not surprisingly, John Wright -- one of the more aggressive drivers, and RP-2 today -- is fully up for this. "Throw caution to the wind!" he says.
"It's thrown!" I answer.
"Damn the torpedoes!"
"Damned!"
A LATER NOTE:
That was Thursday. We were planning well ahead, for a drive to execute Saturday. And Friday, Chris and Ashitey look at the drive and decide to raise some objections.
Their basic argument is reasonable enough: we're driving into terrain that looks benign enough, but in the hills we've found that we can get bogged down even at relatively low tilts and in terrain that looks benign from a distance. So we should add slip checks to the sequence.
John and I don't really agree, and I don't want to call in the uplink team on a day they'd been promised off, but I can't say they're being unreasonable. And our general approach is that if an RP has a reasonable concern, we take it seriously. So I call in Jake Matijevic and Beth Dewell, and they agree that we should act on Chris's concerns.
The required sequencing is easy enough. I simply break up the final waypoint into 15m segments, adding a slip check between segments. Voila.
John complains bitterly about this: "They're just trying to see to it that we don't get our hundred meters."
"Look," I tell him, "they've got a reasonable concern and we've agreed to act on it, so there's no use arguing about it now."
"I agree," he nods. "But I never said I wouldn't bitch about it."
Good point.
A LATER NOTE [2]:
94 meters. It doesn't quite have the cachet of 100m, but who cares? It feels gooood.
[Next post: sol 658 (Opportunity sol 637), November 8.]
I think we can do more like 100m.
And it should be fairly simple. The hillock itself is pretty steep -- we might see 22deg tilt on the way down it -- but that's only for the first 15m or so. After that, the terrain quickly shallows out, so that we're in a far more forgiving 8-10deg of tilt all the way down. We can do 50m blind, and the terrain beyond looks decent for autonav. We don't get opportunities like this so often that we can just throw them away.
The day has its ups and downs. It turns out they forgot about a coordinated observation with HST, so they have to completely swap the plan around, but we still end up with about three hours of driving time. And Squyres decides they'd rather we drove at an azimuth of more like 100deg than 130deg, so that they can do a long-baseline stereo observation along the ridge. But at the end of the day, we've got a 110m drive planned, and a real shot at making it to the end -- or at least breaking 100m.
I expect to have real problems selling this to management, but everyone's pretty enthusiastic. When I tell Jim how far we're driving, he says, "I'd kill to get that on Opportunity right now." Then he reconsiders. "Well ... I'd wound," he amends.
Not surprisingly, John Wright -- one of the more aggressive drivers, and RP-2 today -- is fully up for this. "Throw caution to the wind!" he says.
"It's thrown!" I answer.
"Damn the torpedoes!"
"Damned!"
A LATER NOTE:
That was Thursday. We were planning well ahead, for a drive to execute Saturday. And Friday, Chris and Ashitey look at the drive and decide to raise some objections.
Their basic argument is reasonable enough: we're driving into terrain that looks benign enough, but in the hills we've found that we can get bogged down even at relatively low tilts and in terrain that looks benign from a distance. So we should add slip checks to the sequence.
John and I don't really agree, and I don't want to call in the uplink team on a day they'd been promised off, but I can't say they're being unreasonable. And our general approach is that if an RP has a reasonable concern, we take it seriously. So I call in Jake Matijevic and Beth Dewell, and they agree that we should act on Chris's concerns.
The required sequencing is easy enough. I simply break up the final waypoint into 15m segments, adding a slip check between segments. Voila.
John complains bitterly about this: "They're just trying to see to it that we don't get our hundred meters."
"Look," I tell him, "they've got a reasonable concern and we've agreed to act on it, so there's no use arguing about it now."
"I agree," he nods. "But I never said I wouldn't bitch about it."
Good point.
A LATER NOTE [2]:
94 meters. It doesn't quite have the cachet of 100m, but who cares? It feels gooood.
[Next post: sol 658 (Opportunity sol 637), November 8.]
2010-10-29
Opportunity Sol 627 (Spirit Sol 648)
This was one of the busiest days I've had lately. RP-1 shifts are usually busier than RP-2 shifts, but this one reminds me of the super-stressful days of the nominal mission. It's not as bad as that, but it's close.
Naturally, I loved every minute of it.
Yestersol's drive was another good one. It went only 35m, including 15m of autonav, mainly because we were limited by the lack of PCAM coverage. But at the end, we turned for comm, so we got a good downlink and are rarin' to go again.
Of course, our comm direction is basically north, and we have to drive basically south. So we have to do a nearly 180-degree turn, on a vehicle where large point turns are prohibited, and in a relatively confined space (due to ripples before and behind).
Our first approach is to drive backward down the current trough a bit past the point where we plan to hop the ripples, then do a series of sharp forward arcs that change direction and put us in the next trough over. (We have to end up heading forward at some point because we can't autonav backward as well when using the NCAMs -- their view of the nearby terrain is blocked by the rover deck.)
But this involves a bit too much uncertainty, so I bag it for a different approach. Counterclockwise arcs work well even with the stuck RF steering actuator and aren't subject to any rules-of-the-road limits, so I decide to use a bunch of those. First, we arc 75cm forward with a 22.5-degree counterclockwise heading change. Then we back straight up 50cm. We repeat this maneuver until we have the desired heading, then drive on -- straight -- for the rest of the sol.
Why not make the forward and backward arcs the same length? Because that would take us back and forth over the soil in the same spot too many times, and we're a bit worried about whether that will interfere with our traversability here. We had an experience like that in Endurance, and it left its mark. Of course, maybe it will just compact the soil and make driving better, not worse -- but if we always knew stuff like that, it wouldn't be exploration. So we try the asymmetric approach, and document it copiously with HAZCAMs as we go. And somesol soon, when those images get downlinked, we'll know whether this is a good or bad technique to use next time.
Other than that, this drive is not too remarkable -- about 30m blind, then switch to autonav. All through the sol, our TUL, Rich Morris works to free up more time, and when he's done, we have a whopping two hours and 47 minutes. This is a little embarrassing for me, actually, since if the slip checks work a little faster than they have been, we might actually complete the drive. That wouldn't be such a bad thing, although it would be a shame to waste any time.
Emily Eelkema's picture of the day is an image emailed to her by Ray Arvidson, a picture of the Rio Tinto. This is a naturally highly acidic river in Spain, so acidic that it's actually red. Some shallow pools of the stuff, the color of a nice Cabernet, take on a jelly-like consistency if you stir them. "That's Meridiani Planum, four billion years ago," says Ray. How appropriate. Even Mars's water was red.
And the Red Planet -- and with it, of course, our girls Spirit and Opportunity -- are about as close as they're going to get for a while. The one-way light-time is three minutes 52 seconds. Monday it will start rising again. Mars makes its closest approach at 8:52 PM Saturday -- tomorrow. Maybe I'll take Zenobia[1] outside with me and have a look at it.
[Next post: sol 655, November 5.]
[1] My cat, then. She was ornery and bossy and I loved her to pieces.
Naturally, I loved every minute of it.
Yestersol's drive was another good one. It went only 35m, including 15m of autonav, mainly because we were limited by the lack of PCAM coverage. But at the end, we turned for comm, so we got a good downlink and are rarin' to go again.
Of course, our comm direction is basically north, and we have to drive basically south. So we have to do a nearly 180-degree turn, on a vehicle where large point turns are prohibited, and in a relatively confined space (due to ripples before and behind).
Our first approach is to drive backward down the current trough a bit past the point where we plan to hop the ripples, then do a series of sharp forward arcs that change direction and put us in the next trough over. (We have to end up heading forward at some point because we can't autonav backward as well when using the NCAMs -- their view of the nearby terrain is blocked by the rover deck.)
But this involves a bit too much uncertainty, so I bag it for a different approach. Counterclockwise arcs work well even with the stuck RF steering actuator and aren't subject to any rules-of-the-road limits, so I decide to use a bunch of those. First, we arc 75cm forward with a 22.5-degree counterclockwise heading change. Then we back straight up 50cm. We repeat this maneuver until we have the desired heading, then drive on -- straight -- for the rest of the sol.
Why not make the forward and backward arcs the same length? Because that would take us back and forth over the soil in the same spot too many times, and we're a bit worried about whether that will interfere with our traversability here. We had an experience like that in Endurance, and it left its mark. Of course, maybe it will just compact the soil and make driving better, not worse -- but if we always knew stuff like that, it wouldn't be exploration. So we try the asymmetric approach, and document it copiously with HAZCAMs as we go. And somesol soon, when those images get downlinked, we'll know whether this is a good or bad technique to use next time.
Other than that, this drive is not too remarkable -- about 30m blind, then switch to autonav. All through the sol, our TUL, Rich Morris works to free up more time, and when he's done, we have a whopping two hours and 47 minutes. This is a little embarrassing for me, actually, since if the slip checks work a little faster than they have been, we might actually complete the drive. That wouldn't be such a bad thing, although it would be a shame to waste any time.
Emily Eelkema's picture of the day is an image emailed to her by Ray Arvidson, a picture of the Rio Tinto. This is a naturally highly acidic river in Spain, so acidic that it's actually red. Some shallow pools of the stuff, the color of a nice Cabernet, take on a jelly-like consistency if you stir them. "That's Meridiani Planum, four billion years ago," says Ray. How appropriate. Even Mars's water was red.
And the Red Planet -- and with it, of course, our girls Spirit and Opportunity -- are about as close as they're going to get for a while. The one-way light-time is three minutes 52 seconds. Monday it will start rising again. Mars makes its closest approach at 8:52 PM Saturday -- tomorrow. Maybe I'll take Zenobia[1] outside with me and have a look at it.
[Next post: sol 655, November 5.]
[1] My cat, then. She was ornery and bossy and I loved her to pieces.
2010-10-28
Opportunity Sol 626 (Spirit Sol 647)
Our autonav drive actually went great -- a total of 45 meters, quite respectable by current standards. That means we got about 15m out of the autonav, which is good news.
The only fly in the ointment is that we failed to do our turn for comm, and for a reason we should have thought of. When we planned this drive, we planned for it to extend beyond the time-of-day limit. This is common -- indeed, something we used to do all the time when using autonav; we just have it drive until the time-of-day limit kicks in, then we have the sequence detect that we were stopped by the time-of-day limit, clear the error, and reset the limit to allow time for end-of-drive activities such as turning to comm. We did all that logic on this drive, so we thought we were in the clear.
But here's what we forgot: both the blind and autonav portions of the drive are punctuated by slip checks, helper subsequences that try to drive a short distance and preclude further driving if we didn't make a decent fraction of the commanded distance. After the time-of-day limit hit, the sequence dutifully tried a slip check, noticed that it hadn't actually gone anywhere, and precluded driving. This is something our end-of-sol error-clearing logic didn't check for (indeed, has no good way to check for). So we still had an error -- the rover thought it was stuck in another dune -- and we didn't turn for comm.
Oops. We're still happy, but oops.
Well, our downlink was still pretty good. We didn't get the PCAMs, so that limits our blind driving thisol.
Hey, no problem. Jeng and I plan a sequence that drives blind as far as possible, then uses autonav until stopped by the time-of-day limit.
And this time, we do it correctly.
The only fly in the ointment is that we failed to do our turn for comm, and for a reason we should have thought of. When we planned this drive, we planned for it to extend beyond the time-of-day limit. This is common -- indeed, something we used to do all the time when using autonav; we just have it drive until the time-of-day limit kicks in, then we have the sequence detect that we were stopped by the time-of-day limit, clear the error, and reset the limit to allow time for end-of-drive activities such as turning to comm. We did all that logic on this drive, so we thought we were in the clear.
But here's what we forgot: both the blind and autonav portions of the drive are punctuated by slip checks, helper subsequences that try to drive a short distance and preclude further driving if we didn't make a decent fraction of the commanded distance. After the time-of-day limit hit, the sequence dutifully tried a slip check, noticed that it hadn't actually gone anywhere, and precluded driving. This is something our end-of-sol error-clearing logic didn't check for (indeed, has no good way to check for). So we still had an error -- the rover thought it was stuck in another dune -- and we didn't turn for comm.
Oops. We're still happy, but oops.
Well, our downlink was still pretty good. We didn't get the PCAMs, so that limits our blind driving thisol.
Hey, no problem. Jeng and I plan a sequence that drives blind as far as possible, then uses autonav until stopped by the time-of-day limit.
And this time, we do it correctly.
2010-10-27
Opportunity Sol 625 (Spirit Sol 646)
I've switched rovers once more. Back in the Land of Opportunity, they've backed away from the northern part of Erebus Crater -- pretty far away, as in 100m or so -- and now we're planning a drive to the west side of Erebus Crater. They had to back off so far because of the enormous ripples they encountered in the crater's immediate vicinity, and we're going to keep our distance as we head west.
Or that's what Frank and I think. The scientists would like to complicate things by adding a stop to the tour: in the orbital images, we can see a large expanse of outcrop at the northwest part of the crater. Can we stop by on the way?
Frank puts it to them bluntly: we can drive south to that outcrop, but if the ripples there are like they are elsewhere near the crater, we won't be able to go directly from there to the point where they're thinking of entering Erebus. Instead, we'll have to drive north from the outcrop before we can drive west to the point where we can drive south to get to the west entrance. Just thinking about it is enough to make your head spin, and it'll definitely make our wheels spin for an extra week or more.
This leads to the usual lengthy back-and-forth, and eventually to a compromise solution: we'll drive a little south of west, at an azimuth of 240 or so, hoping it will help us reconnoiter without driving all the way to the outcrop. If what we see tells us that the outcrop would be an expensive journey, we'll skip it; else we'll go there.
No matter how we do it, this will be a lot of driving. So I suggest to Frank that Opportunity revive an old trick, one Spirit uses often but Opportunity hasn't used for quite a while: autonav. We're in such a benign zone right now -- no ripples over 15cm, and nearly all under 10cm or so -- that we ought to take advantage of it to get as much extra distance per sol as we can.
Frank readily agrees. In fact, as it turns out, he and Jeng tried to do just this a few weeks ago, but that drive stopped for other reasons before it got to the autonav section. Nevertheless, it's a straightforward matter to add an autonav section to thisol's drive sequence.
When we actually do the math ... even with about two and a half hours to drive, the visodom-based slip checks are just so time-consuming that autonav is going to end up giving us only 5 or 10 meters. That's discouraging, but, well, it's something. Five meters here, 10 meters there, pretty soon you're talking about ... well ... it's something.
Or that's what Frank and I think. The scientists would like to complicate things by adding a stop to the tour: in the orbital images, we can see a large expanse of outcrop at the northwest part of the crater. Can we stop by on the way?
Frank puts it to them bluntly: we can drive south to that outcrop, but if the ripples there are like they are elsewhere near the crater, we won't be able to go directly from there to the point where they're thinking of entering Erebus. Instead, we'll have to drive north from the outcrop before we can drive west to the point where we can drive south to get to the west entrance. Just thinking about it is enough to make your head spin, and it'll definitely make our wheels spin for an extra week or more.
This leads to the usual lengthy back-and-forth, and eventually to a compromise solution: we'll drive a little south of west, at an azimuth of 240 or so, hoping it will help us reconnoiter without driving all the way to the outcrop. If what we see tells us that the outcrop would be an expensive journey, we'll skip it; else we'll go there.
No matter how we do it, this will be a lot of driving. So I suggest to Frank that Opportunity revive an old trick, one Spirit uses often but Opportunity hasn't used for quite a while: autonav. We're in such a benign zone right now -- no ripples over 15cm, and nearly all under 10cm or so -- that we ought to take advantage of it to get as much extra distance per sol as we can.
Frank readily agrees. In fact, as it turns out, he and Jeng tried to do just this a few weeks ago, but that drive stopped for other reasons before it got to the autonav section. Nevertheless, it's a straightforward matter to add an autonav section to thisol's drive sequence.
When we actually do the math ... even with about two and a half hours to drive, the visodom-based slip checks are just so time-consuming that autonav is going to end up giving us only 5 or 10 meters. That's discouraging, but, well, it's something. Five meters here, 10 meters there, pretty soon you're talking about ... well ... it's something.
2010-10-26
Spirit Sol 645
I'm Spirit's RP-1 today, but Spirit doesn't really need an RP. That's because they planned two sols yesterday, leaving nothing for an RP to do today.
Well, not quite nothing: I do have to weigh in on whether we should take PCAMs to support a possible southward, as opposed to the originally planned eastward, drive. My reaction: meh. We still ought to drive (eastward) to the lip and see if the slope there is as bad as we suspect before we give up on that path and head southward, so a south-looking PCAM from this position isn't likely to help much. But they're looking to spend the bits and taking it does keep our options open, so they take it anyway.
See? I'm not completely useless today.
Well, not quite nothing: I do have to weigh in on whether we should take PCAMs to support a possible southward, as opposed to the originally planned eastward, drive. My reaction: meh. We still ought to drive (eastward) to the lip and see if the slope there is as bad as we suspect before we give up on that path and head southward, so a south-looking PCAM from this position isn't likely to help much. But they're looking to spend the bits and taking it does keep our options open, so they take it anyway.
See? I'm not completely useless today.
2010-10-23
Spirit Sol 642
Spirit saw no safe path, so she gave up after only about 14m. Happily, this is mainly due to the conservative autonav settings; the new images reveal a perfectly safe path straight to the lip, now about 20m away.
But between us and the lip is an enormous patch of outcrop, and it might just be too shiny for the science team to ignore. They're going into restricted sols next week, so if we stop to IDD this outcrop, we're going to end up spending the whole week at it. That's a big time investment, considering that the sun's gradual southward motion means we now have a deadline to get to south-facing slopes.
"RPs, whaddaya think?" Steve asks.
"It's a science mission," I answer. "You know the constraints as well as anyone. If this is important for science, then let's delay going to the lip."
And that's what we do. So the drive itself turns out to be pretty easy, just a gentle 6.7m descent to a big outcrop patch. I have a bad feeling about it, just the same. Something about it reminds me of the hell that was Mazatzal: a big time investment in a highly important science target, a low rock just 7m away, and I think it's easy ... and I came in after that drive to find out we didn't really hit our target, and I felt miserable for a week. (And I still feel that way about it, in case you can't tell.) So I check and recheck, and if I made a mistake thisol, it won't be because I didn't try to learn the lesson.
The rock itself is named "Kansas." This is continuing the practice of naming stuff in this area in honor of Larry Haskin, who apparently was a well-liked friend and well-respected colleague to many on the science team (and who is, sadly, now deceased). Apparently, he had a farm in Kansas, and targets on the rock will be named after towns near his farm.
Me, I'm just excited because when we drive away, we'll get to say that we don't think we're in Kansas any more.
[Next post: sol 645, October 26.]
But between us and the lip is an enormous patch of outcrop, and it might just be too shiny for the science team to ignore. They're going into restricted sols next week, so if we stop to IDD this outcrop, we're going to end up spending the whole week at it. That's a big time investment, considering that the sun's gradual southward motion means we now have a deadline to get to south-facing slopes.
"RPs, whaddaya think?" Steve asks.
"It's a science mission," I answer. "You know the constraints as well as anyone. If this is important for science, then let's delay going to the lip."
And that's what we do. So the drive itself turns out to be pretty easy, just a gentle 6.7m descent to a big outcrop patch. I have a bad feeling about it, just the same. Something about it reminds me of the hell that was Mazatzal: a big time investment in a highly important science target, a low rock just 7m away, and I think it's easy ... and I came in after that drive to find out we didn't really hit our target, and I felt miserable for a week. (And I still feel that way about it, in case you can't tell.) So I check and recheck, and if I made a mistake thisol, it won't be because I didn't try to learn the lesson.
The rock itself is named "Kansas." This is continuing the practice of naming stuff in this area in honor of Larry Haskin, who apparently was a well-liked friend and well-respected colleague to many on the science team (and who is, sadly, now deceased). Apparently, he had a farm in Kansas, and targets on the rock will be named after towns near his farm.
Me, I'm just excited because when we drive away, we'll get to say that we don't think we're in Kansas any more.
[Next post: sol 645, October 26.]
2010-10-20
Spirit Sol 639
Ahh, it's one of those sols where we don't have to start until noon. Oh, my, this feels wonderful.
"I got to sleep until ten today," I say blissfully to Saina.
"I can't sleep until ten," she says. "I get bored."
Which makes no sense to me. But there's not much time to dwell on it; we've got a rover to drive.
About 25-30m away from us, the terrain slopes down more sharply. We can't see over this lip, so we don't know if there's a safe path to Haskin Ridge from here. So our plan is, naturally, to drive to the lip and peek over. If we like what we see, we keep going; if not, we'll have to head most of the way back up to the summit and find a different path down.
Between us and the lip, there's another problem. About 15m from here, there's a series of obstacles that cuts across our path. To the left is a ridge; to the right is a knoll -- which is, perversely, both too tall and too narrow for us to climb.
And straight ahead is a rock. The rock isn't that tall, only about 10cm on the face we can see. But because of the shape of the terrain around it, we can't tell what the terrain looks like on the far side, meaning we can't tell what slopes or dropoffs the rover will experience as it climbs over. The first point we can see beyond the rock is 30cm below the rock's top, meaning the rover could potentially experience a dropoff of that size, and that's too big for this vehicle.
The solution: let the rover handle it. Our plan is to drive Spirit about 11m straight toward the danger area, then turn on autonav and let her find her own path through. (Starting autonav a few meters back from the hazardous area means she can see more of it, giving her a better chance to find a passage.) If she finds a way, great. If she doesn't, then at least tomorrow will bring us better images, which we can maybe use to blind-drive past this troublesome area.
[Next post: sol 642, October 23.]
"I got to sleep until ten today," I say blissfully to Saina.
"I can't sleep until ten," she says. "I get bored."
Which makes no sense to me. But there's not much time to dwell on it; we've got a rover to drive.
About 25-30m away from us, the terrain slopes down more sharply. We can't see over this lip, so we don't know if there's a safe path to Haskin Ridge from here. So our plan is, naturally, to drive to the lip and peek over. If we like what we see, we keep going; if not, we'll have to head most of the way back up to the summit and find a different path down.
Between us and the lip, there's another problem. About 15m from here, there's a series of obstacles that cuts across our path. To the left is a ridge; to the right is a knoll -- which is, perversely, both too tall and too narrow for us to climb.
And straight ahead is a rock. The rock isn't that tall, only about 10cm on the face we can see. But because of the shape of the terrain around it, we can't tell what the terrain looks like on the far side, meaning we can't tell what slopes or dropoffs the rover will experience as it climbs over. The first point we can see beyond the rock is 30cm below the rock's top, meaning the rover could potentially experience a dropoff of that size, and that's too big for this vehicle.
The solution: let the rover handle it. Our plan is to drive Spirit about 11m straight toward the danger area, then turn on autonav and let her find her own path through. (Starting autonav a few meters back from the hazardous area means she can see more of it, giving her a better chance to find a passage.) If she finds a way, great. If she doesn't, then at least tomorrow will bring us better images, which we can maybe use to blind-drive past this troublesome area.
[Next post: sol 642, October 23.]
2010-10-18
Opportunity Sol 616 (Spirit Sol 637)
It's 06:30 at the start of a tight sol, and Beth Dewell is as bleary-eyed as I am.
"I think the rovers were missing you guys," she yawns, reciting the litany of things that went wrong while most of us rover drivers were away at the conference. Opportunity underwent another mystery reset during the AM science block, then missed a crucial part of the uplink, so that she ended up doing nothing but runout science for a couple of days. And we had ground system problems, with a crashing file server among other things. ("I think you can say that Friday was legally a nightmare," as Brenda Franklin puts it.) There's more, but I'm too tired to remember it.
It's not a terribly difficult day, just a Sunday -- er, Monday -- drive, hunting decent ripple-crossing spots as we trend west around Erebus Crater. The only hard bit is that we're on such a tight schedule: we need to be done by about 13:00.
And, what the hell, we make it. Man, I need a nap.
[Next post: sol 639, October 20.]
"I think the rovers were missing you guys," she yawns, reciting the litany of things that went wrong while most of us rover drivers were away at the conference. Opportunity underwent another mystery reset during the AM science block, then missed a crucial part of the uplink, so that she ended up doing nothing but runout science for a couple of days. And we had ground system problems, with a crashing file server among other things. ("I think you can say that Friday was legally a nightmare," as Brenda Franklin puts it.) There's more, but I'm too tired to remember it.
It's not a terribly difficult day, just a Sunday -- er, Monday -- drive, hunting decent ripple-crossing spots as we trend west around Erebus Crater. The only hard bit is that we're on such a tight schedule: we need to be done by about 13:00.
And, what the hell, we make it. Man, I need a nap.
[Next post: sol 639, October 20.]
2010-10-08
Spirit Sol 627
"We can't say we're stable," Ashitey says, "and we can't say we're unstable." What we know is, we wiggled the wheels, and we slipped. But we slipped more the first and second time, losing about two degrees of overall tilt in the process; the change on the third wiggle was definitely less. But then, at the end, when we cocked the wheels against the direction of slip, we slipped more during that motion.
My take on this is that everything is in fact just fine. We're on a slope; we're tilted; we're going to move when we wiggle the wheels. But we shouldn't be freaking out about that, because it's not what we care about. What we care about is whether we're going to move (or tilt) when we deploy the IDD, which is a different question. And when you look at how little we slip when we wiggle the wheels, I think you'd have a hard time saying we're going to slip when we deploy the IDD.
We're freaking out more because we took the wiggle-cam images -- hazcams before and after wiggling the wheels -- when we first got here, and we saw a change in those images. But the thing is, we've never taken such images before, so we don't really know whether that change is in family with what's happened previously.
At the SOWG meeting, I advocate a staged approach. Take it slowly. On the first sol, just deploy the IDD -- nothing more. Take images before and after this deployment. If we see any vehicle motion, or an unacceptable change in tilt, don't uplink the next IDD sequence. On the second sol, we do just one more thing -- place the MB. Again, accompany this with lots of imaging, and pull the plug on the process if we see any unexpected changes. We'll go on in this way for each of the days until the RPs are back in town (nearly everyone's going out of town for a convention).
This is generally accepted, in principle, although Emily is still reluctant to deploy at all. ("I reserve the right to change my mind," she is careful to say.)
So Ashitey and I move ahead on working out these sequences. They're not hard, but it turns out that this is one of those days with a high debate-to-command ratio. We're only about halfway done when Justin Maki puts together a movie showing each image of the rover sliding as we had it wiggle its wheels several times over the last few days, and then the concerns blow up again.
That isn't the only thing that blows up. Ashitey and Justin get into an argument about whether it's safe to deploy, and Ashitey just loses his cool. Ashitey's trying to make the point that our decision should be based on analysis, not on anyone's gut feeling, but Justin keeps cutting him off. Suddenly, Ashitey's had enough. "Stop interrupting!" he snaps. Next thing I know, Justin's grabbed his coffee and stalked out the door.
This is not a normal sol.
By this time I've already realized two things. One is that I need to stop advocating a position and go back to basics: regardless of any questions about winning or losing, what is objectively the best thing to do?[1] And second, we need to know whether what we're seeing is normal behavior in similar conditions.
We haven't taken wiggle-cams before, which is part of the problem. But we do have very good data on the tilt changes. We always collect this data at 8Hz -- the maximum rate the rover's capable of -- during wheel wiggles. So I've been trolling through past sols, looking for times when our tilt's been high. I find a couple of these, plot the tilt data, and -- yep, looks to me like what we saw is normal. I still have to convince the doubters, but it's good evidence.
But first I have another concern. Justin's a smart guy who's done a lot of good on this mission, and we ought to make sure we've heard him out on this topic. So I track him down -- he's upstairs, in the Opportunity room -- and talk it out with him. In the end, it's just gut feeling on his part (he says so himself), and while I don't want to dismiss gut feelings, evidence counts for more. (While we're having this conversation, Ashitey comes to find me. And apologizes to Justin. So that's back on track.)
Also in the Opportunity sequencing room is Jeff Biesiadecki. He knows the rover's mobility system better than almost anybody, and we don't call him "Paranoid Jeff" for nothing, so we drag him downstairs to have a look through it. Somewhat to our surprise, not that we're complaining, Paranoid Jeff gives our careful, phased plan a thumbs-up. So far, so good.
But Khaled gets wind of this, and objects. Strongly. "I wouldn't deploy in this situation," he says firmly, "no way." He doesn't directly take issue with the historical data showing that what we're seeing seems to be normal, but he raises the Shuttle Fallacy: just because we've gotten away with something dangerous in the past, that doesn't mean we're safe to do it again in the future.
"The problem with that," I point out, "is that in situations like this you don't know whether what you're doing is dangerous, and you're getting away with it; or it's safe, and you're worrying for nothing. You can't tell the difference between the two cases without data, and the data we have shows that the rover doesn't tilt excessively when we deploy the IDD in circumstances as close to this as we've gotten."
The debate rages. Some people are trying to figure out exactly where the left front wheel is, and in particular whether it's solidly planted. Jim Erickson and John Callas get involved. They're arguing with Steve Squyres, who pushes the scientific importance of this particular observation. (Squyres nearly scores an own-goal: when discussing what damage could occur to the IDD in the pose where the MB is deployed, he says, "You know, the steel noseplate on the MB is one of the most robust parts of the arm." You don't say it'll survive the scraping to people who want to hear that it's not going to scrape against anything at all.)
This goes on for, I'm not kidding you, three hours. Emily is stressed. She doesn't want to (seem to) be the one to destroy the IDD, with its irreplaceable science value, but I think that in the end she recognizes she doesn't have a fact-based argument. So she makes the right decision: "We're going to proceed with the deployment," she says at last.
But with careful, objective limits on the tilt changes we'll use to decide whether to pull the plug. (We already have decided that no change in position would be acceptable: even a shift of a millimeter would put a stop to this little excursion. But tilt's another matter: we always see a small change in tilt when we deploy the IDD.) Ashitey and Terry Huntsberger do some additional data mining, to come up with numeric bounds. The final number: 0.3 degrees.
We see more tilt than that, and we switch to an IDD-less, image-heavy backup plan. Less tilt, and we proceed.
Surprisingly, even Khaled eventually agrees to this. And with Paranoid Jeff and Even-More-Paranoid Khaled on board, we must be doing something right.
[Next post: sol 637 (Opportunity sol 616), October 18.]
Courtesy NASA/JPL-Caltech. Slipping slowly down the face of Hillary.
Footnotes:
[1] Always, always, always the right approach. Whenever I've strayed from it, I've regretted it, and whenever I've gone back to that path, I've been glad of it. Always.
My take on this is that everything is in fact just fine. We're on a slope; we're tilted; we're going to move when we wiggle the wheels. But we shouldn't be freaking out about that, because it's not what we care about. What we care about is whether we're going to move (or tilt) when we deploy the IDD, which is a different question. And when you look at how little we slip when we wiggle the wheels, I think you'd have a hard time saying we're going to slip when we deploy the IDD.
We're freaking out more because we took the wiggle-cam images -- hazcams before and after wiggling the wheels -- when we first got here, and we saw a change in those images. But the thing is, we've never taken such images before, so we don't really know whether that change is in family with what's happened previously.
At the SOWG meeting, I advocate a staged approach. Take it slowly. On the first sol, just deploy the IDD -- nothing more. Take images before and after this deployment. If we see any vehicle motion, or an unacceptable change in tilt, don't uplink the next IDD sequence. On the second sol, we do just one more thing -- place the MB. Again, accompany this with lots of imaging, and pull the plug on the process if we see any unexpected changes. We'll go on in this way for each of the days until the RPs are back in town (nearly everyone's going out of town for a convention).
This is generally accepted, in principle, although Emily is still reluctant to deploy at all. ("I reserve the right to change my mind," she is careful to say.)
So Ashitey and I move ahead on working out these sequences. They're not hard, but it turns out that this is one of those days with a high debate-to-command ratio. We're only about halfway done when Justin Maki puts together a movie showing each image of the rover sliding as we had it wiggle its wheels several times over the last few days, and then the concerns blow up again.
That isn't the only thing that blows up. Ashitey and Justin get into an argument about whether it's safe to deploy, and Ashitey just loses his cool. Ashitey's trying to make the point that our decision should be based on analysis, not on anyone's gut feeling, but Justin keeps cutting him off. Suddenly, Ashitey's had enough. "Stop interrupting!" he snaps. Next thing I know, Justin's grabbed his coffee and stalked out the door.
This is not a normal sol.
By this time I've already realized two things. One is that I need to stop advocating a position and go back to basics: regardless of any questions about winning or losing, what is objectively the best thing to do?[1] And second, we need to know whether what we're seeing is normal behavior in similar conditions.
We haven't taken wiggle-cams before, which is part of the problem. But we do have very good data on the tilt changes. We always collect this data at 8Hz -- the maximum rate the rover's capable of -- during wheel wiggles. So I've been trolling through past sols, looking for times when our tilt's been high. I find a couple of these, plot the tilt data, and -- yep, looks to me like what we saw is normal. I still have to convince the doubters, but it's good evidence.
But first I have another concern. Justin's a smart guy who's done a lot of good on this mission, and we ought to make sure we've heard him out on this topic. So I track him down -- he's upstairs, in the Opportunity room -- and talk it out with him. In the end, it's just gut feeling on his part (he says so himself), and while I don't want to dismiss gut feelings, evidence counts for more. (While we're having this conversation, Ashitey comes to find me. And apologizes to Justin. So that's back on track.)
Also in the Opportunity sequencing room is Jeff Biesiadecki. He knows the rover's mobility system better than almost anybody, and we don't call him "Paranoid Jeff" for nothing, so we drag him downstairs to have a look through it. Somewhat to our surprise, not that we're complaining, Paranoid Jeff gives our careful, phased plan a thumbs-up. So far, so good.
But Khaled gets wind of this, and objects. Strongly. "I wouldn't deploy in this situation," he says firmly, "no way." He doesn't directly take issue with the historical data showing that what we're seeing seems to be normal, but he raises the Shuttle Fallacy: just because we've gotten away with something dangerous in the past, that doesn't mean we're safe to do it again in the future.
"The problem with that," I point out, "is that in situations like this you don't know whether what you're doing is dangerous, and you're getting away with it; or it's safe, and you're worrying for nothing. You can't tell the difference between the two cases without data, and the data we have shows that the rover doesn't tilt excessively when we deploy the IDD in circumstances as close to this as we've gotten."
The debate rages. Some people are trying to figure out exactly where the left front wheel is, and in particular whether it's solidly planted. Jim Erickson and John Callas get involved. They're arguing with Steve Squyres, who pushes the scientific importance of this particular observation. (Squyres nearly scores an own-goal: when discussing what damage could occur to the IDD in the pose where the MB is deployed, he says, "You know, the steel noseplate on the MB is one of the most robust parts of the arm." You don't say it'll survive the scraping to people who want to hear that it's not going to scrape against anything at all.)
This goes on for, I'm not kidding you, three hours. Emily is stressed. She doesn't want to (seem to) be the one to destroy the IDD, with its irreplaceable science value, but I think that in the end she recognizes she doesn't have a fact-based argument. So she makes the right decision: "We're going to proceed with the deployment," she says at last.
But with careful, objective limits on the tilt changes we'll use to decide whether to pull the plug. (We already have decided that no change in position would be acceptable: even a shift of a millimeter would put a stop to this little excursion. But tilt's another matter: we always see a small change in tilt when we deploy the IDD.) Ashitey and Terry Huntsberger do some additional data mining, to come up with numeric bounds. The final number: 0.3 degrees.
We see more tilt than that, and we switch to an IDD-less, image-heavy backup plan. Less tilt, and we proceed.
Surprisingly, even Khaled eventually agrees to this. And with Paranoid Jeff and Even-More-Paranoid Khaled on board, we must be doing something right.
[Next post: sol 637 (Opportunity sol 616), October 18.]
Courtesy NASA/JPL-Caltech. Slipping slowly down the face of Hillary.
Footnotes:
[1] Always, always, always the right approach. Whenever I've strayed from it, I've regretted it, and whenever I've gone back to that path, I've been glad of it. Always.
2010-10-07
Spirit Sol 626
This morning brings good news and bad news. The good news is, we made it. This is truly spectacular and receives its due of praise from all concerned: it was one of the most difficult approaches we've ever done, and we nailed it.
The bad news is, it's not safe to IDD. At the end of the drive, we carefully wiggled the wheels, and the data and images show that the vehicle moved during the wiggle. This implies that it might well be unsafe to deploy the IDD. So we won't be able to start IDDing thisol; we're going to have to spend the sol figuring out how to either settle the rover a bit better or bumping to a new position. (We also displaced a rock we drove over on the way in, tilting it up such that there was probably only 2-3cm clearance between it and the bottom of the WEB. Oops.)
The basic decision is: wiggle only, or drive and then wiggle? Ashitey wants to drive first, but I eventually talk him out of it. In this area, treacherous as it is, driving is merely a way of trading the devil we know for the devil we don't. We don't have any good way to raise the odds that we'll be stable at the end of the next little bump, so we might as well try to make Spirit more stable right here.
This argument would probably have been a loser, given that thisol is absolutely our last margin sol -- we can't just try for stability thisol, then drive and try again nextersol if this doesn't work, because we're going to be on travel then. (Or, to put it another way, we could drive nextersol, but there's not much point because nobody will be around to develop the IDD sequences that are the real point of this endeavor.)
But Squyres declares that Hillary is such a high-priority science target that they're willing to wait. If we simply can't get the IDD into position before leaving town, they'll wait until RPs start returning to town Wednesday. That means we have sols to spare, so we can try one thing thisol and something else nextersol if it doesn't work. That's the least risky approach, so we go with that.
The settling tactic itself is a fairly straightforward one. We wiggle each wheel, one at a time, taking plenty of documentation images and visodom updates as we do. Then we cock each wheel to the left, so that the paddles are transverse to the direction of slip. If all goes well, we'll come in tomorrow and see a tiny amount of slip on the first wiggle and no slip after that, and we'll be stable and ready to deploy the IDD. If not, well ... the scientists will have a lander for a few days.
"Alternatively," Steve says, "if this doesn't work, we could give up and drive to another rock. There are other rocks around here -- much less interesting from a science perspective, but they'd probably tell us what we need to know. It's the RPs' call."
"I'm not going to want to give up, Steve," I tell him. "I'm still pissed off about Burns Cliff."
And I am, too.
Courtesy NASA/JPL-Caltech. Some days I rock. Other days I rock hard. This was one of the second kind.
The bad news is, it's not safe to IDD. At the end of the drive, we carefully wiggled the wheels, and the data and images show that the vehicle moved during the wiggle. This implies that it might well be unsafe to deploy the IDD. So we won't be able to start IDDing thisol; we're going to have to spend the sol figuring out how to either settle the rover a bit better or bumping to a new position. (We also displaced a rock we drove over on the way in, tilting it up such that there was probably only 2-3cm clearance between it and the bottom of the WEB. Oops.)
The basic decision is: wiggle only, or drive and then wiggle? Ashitey wants to drive first, but I eventually talk him out of it. In this area, treacherous as it is, driving is merely a way of trading the devil we know for the devil we don't. We don't have any good way to raise the odds that we'll be stable at the end of the next little bump, so we might as well try to make Spirit more stable right here.
This argument would probably have been a loser, given that thisol is absolutely our last margin sol -- we can't just try for stability thisol, then drive and try again nextersol if this doesn't work, because we're going to be on travel then. (Or, to put it another way, we could drive nextersol, but there's not much point because nobody will be around to develop the IDD sequences that are the real point of this endeavor.)
But Squyres declares that Hillary is such a high-priority science target that they're willing to wait. If we simply can't get the IDD into position before leaving town, they'll wait until RPs start returning to town Wednesday. That means we have sols to spare, so we can try one thing thisol and something else nextersol if it doesn't work. That's the least risky approach, so we go with that.
The settling tactic itself is a fairly straightforward one. We wiggle each wheel, one at a time, taking plenty of documentation images and visodom updates as we do. Then we cock each wheel to the left, so that the paddles are transverse to the direction of slip. If all goes well, we'll come in tomorrow and see a tiny amount of slip on the first wiggle and no slip after that, and we'll be stable and ready to deploy the IDD. If not, well ... the scientists will have a lander for a few days.
"Alternatively," Steve says, "if this doesn't work, we could give up and drive to another rock. There are other rocks around here -- much less interesting from a science perspective, but they'd probably tell us what we need to know. It's the RPs' call."
"I'm not going to want to give up, Steve," I tell him. "I'm still pissed off about Burns Cliff."
And I am, too.
Courtesy NASA/JPL-Caltech. Some days I rock. Other days I rock hard. This was one of the second kind.
2010-10-06
Spirit Sol 625
I must be bad luck or something. This is the third consecutive sol I've worked where we were missing important drive imagery.
Thisol we're missing most of the NCAMs that were to support our final approach to Hillary, a juicy rock target here at the true summit of Husband Hill. Fortunately, the one NCAM we do have covers the straightforward approach to Hillary. Unfortunately, the straightforward approach is a no-go for comm reasons. So instead of tackling it head-on in a northward drive, we have to skirt around so that we're heading at it more northeasterly. And this means taking on much higher tilt -- nearly 30 degrees, above our comfort level.
(Hillary is, of course, named in honor of Everest conqueror Sir Edmund Hillary. To my surprise, the man is still alive, in his late eighties. Squyres is planning to email him some images of the rock we've named for him. I find this somehow bizarre, as if we were going to send a fax to Erik the Red.)
We're trying to get Spirit into position to IDD, including some very long-term MB and APXS integrations, for the few days that there's no RP coverage because everyone's in Hawaii. We could possibly recover if we need to use tomorrow (Thursday) for a final bump, but that will make Friday hellacious. And if we don't make it on Thursday either, well, somebody's not going to Hawaii, I guess.
The high tilt on this drive is making everyone nervous. After much discussion, Ashitey and I come up with a set of safeguards that should keep Spirit safe while still giving us a good chance of making it to the goal. An even lengthier discussion sells it to the rest of the uplink team. "Just give me time to update my résumé," deadpans Saina.
Courtesy NASA/JPL-Caltech. Our NCAM of Hillary.
Thisol we're missing most of the NCAMs that were to support our final approach to Hillary, a juicy rock target here at the true summit of Husband Hill. Fortunately, the one NCAM we do have covers the straightforward approach to Hillary. Unfortunately, the straightforward approach is a no-go for comm reasons. So instead of tackling it head-on in a northward drive, we have to skirt around so that we're heading at it more northeasterly. And this means taking on much higher tilt -- nearly 30 degrees, above our comfort level.
(Hillary is, of course, named in honor of Everest conqueror Sir Edmund Hillary. To my surprise, the man is still alive, in his late eighties. Squyres is planning to email him some images of the rock we've named for him. I find this somehow bizarre, as if we were going to send a fax to Erik the Red.)
We're trying to get Spirit into position to IDD, including some very long-term MB and APXS integrations, for the few days that there's no RP coverage because everyone's in Hawaii. We could possibly recover if we need to use tomorrow (Thursday) for a final bump, but that will make Friday hellacious. And if we don't make it on Thursday either, well, somebody's not going to Hawaii, I guess.
The high tilt on this drive is making everyone nervous. After much discussion, Ashitey and I come up with a set of safeguards that should keep Spirit safe while still giving us a good chance of making it to the goal. An even lengthier discussion sells it to the rest of the uplink team. "Just give me time to update my résumé," deadpans Saina.
Courtesy NASA/JPL-Caltech. Our NCAM of Hillary.
2010-09-30
Captain's Log, Supplemental
And so it was just about this time, five years ago, that Spirit took the Husband Hill panorama -- a beautiful color panorama taken from the very summit of Husband Hill. (It's also called the "Everest panorama," and sharp-eyed readers might notice it's the image at the very top of this blog page.)
The rovers have returned more than a quarter of a million images, but this one would be on my list of the top five or so. There's so much in this image. First, and maybe most importantly, are Spirit's own tracks leading to the summit, a record of the struggle and hardship we faced on the way up -- and the relentless determination we showed in its face. In the distance, there's a dust devil on the plains, caught in the act of moving between frames. (So it looks like two dust devils, but it's really one, moving as Spirit took her images using a series of three different-colored filters.)
And, of course, there's Home Plate, to whose immediate west Spirit sits today, trying -- I hope, I fervently hope -- to phone home.
The Husband Hill panorama. (The link takes you to the full -- very large -- image.)
The rovers have returned more than a quarter of a million images, but this one would be on my list of the top five or so. There's so much in this image. First, and maybe most importantly, are Spirit's own tracks leading to the summit, a record of the struggle and hardship we faced on the way up -- and the relentless determination we showed in its face. In the distance, there's a dust devil on the plains, caught in the act of moving between frames. (So it looks like two dust devils, but it's really one, moving as Spirit took her images using a series of three different-colored filters.)
And, of course, there's Home Plate, to whose immediate west Spirit sits today, trying -- I hope, I fervently hope -- to phone home.
The Husband Hill panorama. (The link takes you to the full -- very large -- image.)
2010-09-29
Spirit Sol 619
Coincidentally, my last drive was performed without the assistance of the ultimate FHAZ, and so must this one be. Spirit's just a few meters away from the true summit of Husband Hill -- a slightly higher point than the spot we last thought was the summit -- and we're going to drive up onto it.
That's a given. The big discussion at the SOWG meeting is what we're going to do next -- I mean, over the weekend. Should we pick a juicy rock and IDD it, and so use that rock as thisol's drive target? Or should we just get into a good position to do imaging, and do self-IDDing (such as by putting the MB on the CCT) over the weekend if we IDD at all?
Hap's take: "We're too experienced to be wasting our time exploring dusty rocks. We should wait and do clean rocks somewhere else."
Squyres agrees: "We may have to trade this strike-and-dip imaging against the IDD position." But they can't get the pictures of rocks they want anywhere else, while the rocks they have pictures of are not particularly interesting IDD targets, so that's a bad trade. "So let's do structural imaging on some face of Hillary" -- named for Sir Edmund Hillary, of course; another rock up here is named for his Sherpa guide, Tenzing Norgay -- "and a panorama."
Alian suggests that we could get into position for imaging and IDD whatever's there, but Squyres says no. "I'm reluctant to use the IDD for low-value targets. Maybe we could get into a good position for imaging and recapture the MB-on-the-CCT measurement we recently lost." And that's pretty much it.
John handles most of the drive, with me looking over his shoulder and offering feedback as he goes. This is a practice we sometimes call "backseat rover driving," and it's a really effective one. I never feel like I'm contributing as much when we plan drives that way, but I'm actually being more useful than if I make the RP-1 do everything himself and then get involved only later.
As we review the drive at the walkthrough, Steve chimes in to remind us of what a big deal this is. "This is the final drive to the summit of a mountain we've been climbing for more than a year," he observes. "And it's our last real uphill drive for a while."
This should feel like a more momentous occasion than it does. I think we're all just kind of fatigued. Morale isn't what it once was. And that's a shame, because we put a lot of hard work into getting here, and we ought to feel damn proud of that. I'm sure we do, really. It's in there somewhere, behind all the exhaustion.
[Next post: sol 625, October 6.]
That's a given. The big discussion at the SOWG meeting is what we're going to do next -- I mean, over the weekend. Should we pick a juicy rock and IDD it, and so use that rock as thisol's drive target? Or should we just get into a good position to do imaging, and do self-IDDing (such as by putting the MB on the CCT) over the weekend if we IDD at all?
Hap's take: "We're too experienced to be wasting our time exploring dusty rocks. We should wait and do clean rocks somewhere else."
Squyres agrees: "We may have to trade this strike-and-dip imaging against the IDD position." But they can't get the pictures of rocks they want anywhere else, while the rocks they have pictures of are not particularly interesting IDD targets, so that's a bad trade. "So let's do structural imaging on some face of Hillary" -- named for Sir Edmund Hillary, of course; another rock up here is named for his Sherpa guide, Tenzing Norgay -- "and a panorama."
Alian suggests that we could get into position for imaging and IDD whatever's there, but Squyres says no. "I'm reluctant to use the IDD for low-value targets. Maybe we could get into a good position for imaging and recapture the MB-on-the-CCT measurement we recently lost." And that's pretty much it.
John handles most of the drive, with me looking over his shoulder and offering feedback as he goes. This is a practice we sometimes call "backseat rover driving," and it's a really effective one. I never feel like I'm contributing as much when we plan drives that way, but I'm actually being more useful than if I make the RP-1 do everything himself and then get involved only later.
As we review the drive at the walkthrough, Steve chimes in to remind us of what a big deal this is. "This is the final drive to the summit of a mountain we've been climbing for more than a year," he observes. "And it's our last real uphill drive for a while."
This should feel like a more momentous occasion than it does. I think we're all just kind of fatigued. Morale isn't what it once was. And that's a shame, because we put a lot of hard work into getting here, and we ought to feel damn proud of that. I'm sure we do, really. It's in there somewhere, behind all the exhaustion.
[Next post: sol 625, October 6.]
Subscribe to:
Posts (Atom)
Posts
