Opportunity experienced a disturbing failure this weekend: her IDD failed to deploy. We talked about it this weekend long enough to rule out a sequencing error, which meant we shouldn't try to bring in the normal uplink team to try to fix it and salvage the IDD work. It's possible we would have tried to get the right people together, but it was Thanksgiving, and many people were out of town, and so on. And these days we don't engage in heroics. So we let it go until Monday.
Now it's Monday, and it's not immediately clear what we should do about this, because it's not immediately clear what the failure is. We know we got a tiny amount of motion followed by a motor stall, but why? Eric Baumgartner's (well-educated) guess is that we've simply lost a small amount of knowledge as to the position of the IDD. As a result, when the IDD tried to go to the commanded position as part of its unstow routine, it actually failed to get the IDD off of the little hook it stows itself on, and then it couldn't rotate itself away from the hook. This hypothesis has some confirmation in some image analysis Paolo performed, which may show a tiny discrepancy in the arm's position after the last time we stowed it.
This would be relatively straightforward to solve -- not easy, but not dreadfully bad. We'd have to figure out where the arm actually is, then manually guide it through the unstow routine, at which point we'd be able to recalibrate the IDD by driving each of the joints against its hardstop. We might also need to repeat the "Martian Tai Chi" routine that recalibrates the camera models, so that the IDD and the cameras would be in agreement. It would be a long, slow process -- maybe a week or two, all put together -- but in the end we'd be as good as new.
But Eric's explanation isn't the only one that fits the observed facts. Ashitey fears there's something worse going on. One of the actuators in the testbed rover has started to fail, and it stalls out in pretty much the same way as we've just observed Opportunity's joint motor stall. If this is what's going on, it means the IDD has become unreliable -- every once in a while, our IDD work might randomly fail in this way. That would introduce a severe, somewhat random penalty into all IDD work.
With the testbed rover's IDD, the failures are merely annoying; repeat the failed command, and it usually works the second try. If Opportunity's IDD is indeed failing in this way, that means we have an occasional one-sol penalty to look forward to; probably nothing worse than that (though that would be unpleasant). But it could be worse still: the actuator might simply have failed permanently. And if that's what happened, a large fraction of Opportunity's scientific potential just disappeared forever.
There are other possible explanations, too, with varying degrees of benignity. Erickson wants the full-up anomaly investigation, and the right guy to do that is Ashitey. But Ashitey's RP-1 on Spirit today, which would leave him precious little time for something like that. So he and I switch places -- he heads up the investigation, and I go back to Spirit.
Which means I'm driving today -- driving to Miami. (I try to remember not to leave the left turn signal on.) This drive is reasonably simple, snaking around a few rocks before heading more or less straight toward Miami. Well, not quite Miami -- Coral Gables. See, I didn't have an exact target for Miami, just picked a spot in the neighborhood, and Coral Gables is a Miami suburb ....
Maybe you had to be there.
If all goes well, we'll cover a third to a half of the distance to Miami in this drive. (Maybe we'll stop for the night at a HoJo's. Sorry, I can't help it.)
I'd seen a birthday card in the Spirit sequencing room last week, but it hadn't registered on me until today. I pick it up and read it. I figured it was for someone on the team. Turns out I was right; it's for the most important member of our team. It's a birthday card for a one-year-old -- our one-year-old, Spirit. (Or maybe that should be a "birthsol" card, but I'm not picky.) From the nice (and clever) folks at unmannedspaceflight.com.
I love it. We should make them Spirit's godparents or something.
[Next post: sol 680, December 1.]
2010-11-29
2010-11-27
Spirit Sol 676
Thisol should be easy. Spirit's been IDDing a patch of outcrop named "Seminole," on the target "Osceola." (It's coming up on Thanksgiving, and since we did Pilgrim names last year, we're doing American Indian names this year.) And the big science waypoint we've been heading toward has been renamed from "Waypoint" to "Miami." I'm getting a kick out of this, having gone to Seminole High School and all. But I digress.
So all we're doing is repeating the same stuff they did yesterday on Osceola, only we're changing the target to Abiaka (another Seminole chief). Load up those sequences, change the targets and a few sequence IDs, and you're done.
As is so often the case, it's not that simple. Abiaka unfortunately turns out to be a little too vertical (strictly speaking, Abiaka is a too-vertical normal to a too-horizontal rock surface) -- a problem because the IDD gets confused when you ask it to move straight up and down. That has to do with the target being too close to a wrist-flip condition, where the IDD could choose almost equally well between two different wrist configurations, and if conditions on Mars vary only slightly from the predicts, it could change its mind in the middle and decide it can't do what we want.
The remedy for this is known but tedious: tweak the angle of the target, so that it's tilted slightly away from vertical, until the simulation says you've gone far enough. What complicates this is that we can't tilt the normal too far, or the RAT brushing won't work. Also making it more complex is that John's having his shadow, Ashley, handle the IDD sequence, and while she generally knows what she's doing, she's still new to this and somewhat slow at it.
Oy. Well, it takes forever, but it gets done, and that's what matters.
[Next post: sol 678, November 29.]
So all we're doing is repeating the same stuff they did yesterday on Osceola, only we're changing the target to Abiaka (another Seminole chief). Load up those sequences, change the targets and a few sequence IDs, and you're done.
As is so often the case, it's not that simple. Abiaka unfortunately turns out to be a little too vertical (strictly speaking, Abiaka is a too-vertical normal to a too-horizontal rock surface) -- a problem because the IDD gets confused when you ask it to move straight up and down. That has to do with the target being too close to a wrist-flip condition, where the IDD could choose almost equally well between two different wrist configurations, and if conditions on Mars vary only slightly from the predicts, it could change its mind in the middle and decide it can't do what we want.
The remedy for this is known but tedious: tweak the angle of the target, so that it's tilted slightly away from vertical, until the simulation says you've gone far enough. What complicates this is that we can't tilt the normal too far, or the RAT brushing won't work. Also making it more complex is that John's having his shadow, Ashley, handle the IDD sequence, and while she generally knows what she's doing, she's still new to this and somewhat slow at it.
Oy. Well, it takes forever, but it gets done, and that's what matters.
[Next post: sol 678, November 29.]
2010-11-19
Opportunity Sol 649 (Spirit Sol 668)
Meanwhile, on the other side of the planet ... Opportunity is still skirting the rim of Erebus Crater, making her way to the potential entry point we can see from orbital data. Thisol's plan is to finish up a little IDD work, then drive about 50m -- mostly along outcrop -- toward a flat patch of outcrop we can see in the distance.
This plan is complicated by the fact that the flat patch of outcrop isn't so flat. Looking at it in stereo, we realize that it's actually a somewhat steep face, rising about 50cm over less than 2m. (The fact that the outcrop seems to have some delta-Z makes me think the science team might be interested in it, as it might provide some layered rock to examine. Cooper, ever eager to get moving, is comically reluctant to point this out to them, but I do it anyway.) The ends of the patch are still relatively flat, though, so we decide to aim toward those instead.
Our drive sequence is further complicated by the fact that we're heading at about 180, but need to turn for comm to about 115 -- a large counterclockwise turn, problematic on this vehicle because of the stuck right-front steering actuator. If we knew in advance what our heading would be for the comm turn, that would make it a lot easier to sequence. But we don't, because toward the end there's a slight change in direction: we'll be headed either about 175 or 185 when the time-of-day limit hits. And the only tool we have to make the sequence conditional on the vehicle's yaw gives awkward results around 180, because it gives its answers in the range of -180 to +180 (actually -pi to +pi, since it's in radians) -- nothing we can't sequence around, it's just one more damn thing.
All this still wouldn't be too bad, if Cooper or I were driving. But we decided to let Paolo do it, since he needs the experience. He actually handles it really well, making good decisions and so on. The only problem is that he's still somewhat new, and therefore somewhat slow, and this is a complicated sol, and, well ... it takes a long time. I sit with him pretty much the whole time and strive to stay cheerful and positive. Add that to the list of challenges.
Oh, but we're not through yet. Because I'd been sitting with Paolo, I hadn't had a chance to closely examine Brian's IDD sequence. During the CAM, I discover that the MI poker gets awfully close to a rock during a lateral move. There's less than 1cm of clearance in the prediction, which is within our error budget. Luckily for me, Ken Herkenhoff -- the MI PI, basically the designer and builder and owner of the instrument -- is sitting in the room. (He's normally one of the remote scientists, but is out here to participate in training the new crop.) So I put the CAM on hold, show Ken the situation, and ask his advice.
Now, the MI poker has a bit of spring steel near its base, allowing it to flex safely in just such circumstances as these. And the simulation does show that we have clearance, albeit not much clearance. What's more, we're first touching the surface near the potential contact point, so any error about the location of that point will likely be accounted for -- if our imagery is misleading about that, it should be misleading in the same way about the nearby spot we'll touch, and the sequencing is relative to the touched location, so the error will effectively magically disappear. All this taken into account, Ken's comfortable with proceeding as planned, so we go ahead. (What occurs to me later, as well, is that the simulation we're looking at is actually relative to the 1cm overdrive position -- which aleady accounts for our uncertainty about the terrain -- which means we'll likely have 1cm more clearance than shown. Even better.)
We also realize another place where there's a potential problem. We're in that phase where the Earth-Mars time difference requires us to plan several sols ahead, so we don't yet know the true state of the IDD at the beginning of this sol's sequences -- for the very good reason that that stuff simply hasn't happened yet. And since we're starting (we hope) with the MB in contact with the CCT, we have to disable self-collision checking at the beginning of the sequence in order to allow ourselves to retract the IDD into free space. The problem is, if the planned IDD sequences don't go as predicted, the IDD could start off sol 649 someplace unexpected, and disabling self-collision checking is not what you want to do under those circumstances.
Our solution is to remove the IDD sequence from the bundle -- we'll call in Sunday morning, after we've looked at the data, and we'll send this IDD sequence to the spacecraft only after we know it's done what we expected it to do. This potentially means we'll have to drop the IDD sequence as well as the subsequent drive, since the rover won't drive when the IDD is unstowed -- and it also means we've all got some work to do Saturday night and Sunday morning -- but that's a lot better than risking the IDD.
At long last, we complete the CAM -- and not a moment too soon, as it's been a long and grueling day and we're all getting kind of punchy -- and I realize we've made yet another mistake.
"We're not going to change anything," I say carefully, "but we did screw something up." Normally, our drive sequences detect absent helper sequences, precluding further driving if a helper is missing. (This would mean, for instance, that we were unable to perform a slip check; without that, we don't know whether the vehicle is bogged down, so we don't want to keep trying to drive for tens of meters, making the situation worse.) Our predicted downlink is so poor thisol, however, that we wanted to be sure we'd turn for comm even if one of the helper sequences didn't make it on board. That's rare, but it happens, whether due to a human screw-up somewhere in the process, equipment problems at the DSN stations, or simply because of interference in the long journey of the radio waves from Earth to Mars.
So we pulled a clever trick: during the sequence, we abuse the OK-to-IDD flag -- if a helper is absent and we preclude driving for that reason, we also preclude IDDing. Later, the sequence checks whether we're OK to IDD, and if not, that means a helper must have been missing, so we re-enable driving (and IDDing) so that we can turn for comm.
"What we forgot," I conclude, "is that if you try to drive while driving is precluded -- which we would -- you also get a goal error. And we don't clear that goal error, so the turn for comm still wouldn't happen."
But this bites us only if a helper doesn't make it onto the vehicle -- a rare event -- and it doesn't truly hurt us, it just fails to help us. And at this point on this sol, nobody wants to go back and fix it. So we don't.
[Next post: sol 676, November 27.]
This plan is complicated by the fact that the flat patch of outcrop isn't so flat. Looking at it in stereo, we realize that it's actually a somewhat steep face, rising about 50cm over less than 2m. (The fact that the outcrop seems to have some delta-Z makes me think the science team might be interested in it, as it might provide some layered rock to examine. Cooper, ever eager to get moving, is comically reluctant to point this out to them, but I do it anyway.) The ends of the patch are still relatively flat, though, so we decide to aim toward those instead.
Our drive sequence is further complicated by the fact that we're heading at about 180, but need to turn for comm to about 115 -- a large counterclockwise turn, problematic on this vehicle because of the stuck right-front steering actuator. If we knew in advance what our heading would be for the comm turn, that would make it a lot easier to sequence. But we don't, because toward the end there's a slight change in direction: we'll be headed either about 175 or 185 when the time-of-day limit hits. And the only tool we have to make the sequence conditional on the vehicle's yaw gives awkward results around 180, because it gives its answers in the range of -180 to +180 (actually -pi to +pi, since it's in radians) -- nothing we can't sequence around, it's just one more damn thing.
All this still wouldn't be too bad, if Cooper or I were driving. But we decided to let Paolo do it, since he needs the experience. He actually handles it really well, making good decisions and so on. The only problem is that he's still somewhat new, and therefore somewhat slow, and this is a complicated sol, and, well ... it takes a long time. I sit with him pretty much the whole time and strive to stay cheerful and positive. Add that to the list of challenges.
Oh, but we're not through yet. Because I'd been sitting with Paolo, I hadn't had a chance to closely examine Brian's IDD sequence. During the CAM, I discover that the MI poker gets awfully close to a rock during a lateral move. There's less than 1cm of clearance in the prediction, which is within our error budget. Luckily for me, Ken Herkenhoff -- the MI PI, basically the designer and builder and owner of the instrument -- is sitting in the room. (He's normally one of the remote scientists, but is out here to participate in training the new crop.) So I put the CAM on hold, show Ken the situation, and ask his advice.
Now, the MI poker has a bit of spring steel near its base, allowing it to flex safely in just such circumstances as these. And the simulation does show that we have clearance, albeit not much clearance. What's more, we're first touching the surface near the potential contact point, so any error about the location of that point will likely be accounted for -- if our imagery is misleading about that, it should be misleading in the same way about the nearby spot we'll touch, and the sequencing is relative to the touched location, so the error will effectively magically disappear. All this taken into account, Ken's comfortable with proceeding as planned, so we go ahead. (What occurs to me later, as well, is that the simulation we're looking at is actually relative to the 1cm overdrive position -- which aleady accounts for our uncertainty about the terrain -- which means we'll likely have 1cm more clearance than shown. Even better.)
We also realize another place where there's a potential problem. We're in that phase where the Earth-Mars time difference requires us to plan several sols ahead, so we don't yet know the true state of the IDD at the beginning of this sol's sequences -- for the very good reason that that stuff simply hasn't happened yet. And since we're starting (we hope) with the MB in contact with the CCT, we have to disable self-collision checking at the beginning of the sequence in order to allow ourselves to retract the IDD into free space. The problem is, if the planned IDD sequences don't go as predicted, the IDD could start off sol 649 someplace unexpected, and disabling self-collision checking is not what you want to do under those circumstances.
Our solution is to remove the IDD sequence from the bundle -- we'll call in Sunday morning, after we've looked at the data, and we'll send this IDD sequence to the spacecraft only after we know it's done what we expected it to do. This potentially means we'll have to drop the IDD sequence as well as the subsequent drive, since the rover won't drive when the IDD is unstowed -- and it also means we've all got some work to do Saturday night and Sunday morning -- but that's a lot better than risking the IDD.
At long last, we complete the CAM -- and not a moment too soon, as it's been a long and grueling day and we're all getting kind of punchy -- and I realize we've made yet another mistake.
"We're not going to change anything," I say carefully, "but we did screw something up." Normally, our drive sequences detect absent helper sequences, precluding further driving if a helper is missing. (This would mean, for instance, that we were unable to perform a slip check; without that, we don't know whether the vehicle is bogged down, so we don't want to keep trying to drive for tens of meters, making the situation worse.) Our predicted downlink is so poor thisol, however, that we wanted to be sure we'd turn for comm even if one of the helper sequences didn't make it on board. That's rare, but it happens, whether due to a human screw-up somewhere in the process, equipment problems at the DSN stations, or simply because of interference in the long journey of the radio waves from Earth to Mars.
So we pulled a clever trick: during the sequence, we abuse the OK-to-IDD flag -- if a helper is absent and we preclude driving for that reason, we also preclude IDDing. Later, the sequence checks whether we're OK to IDD, and if not, that means a helper must have been missing, so we re-enable driving (and IDDing) so that we can turn for comm.
"What we forgot," I conclude, "is that if you try to drive while driving is precluded -- which we would -- you also get a goal error. And we don't clear that goal error, so the turn for comm still wouldn't happen."
But this bites us only if a helper doesn't make it onto the vehicle -- a rare event -- and it doesn't truly hurt us, it just fails to help us. And at this point on this sol, nobody wants to go back and fix it. So we don't.
[Next post: sol 676, November 27.]
2010-11-17
Spirit Sol 666
Sol 666: the sol of the Beast. It's a real shame this didn't happen to be Halloween. Maybe thisol should be Martian Halloween.
If there's a Beast involved, he's being awfully good to us. Yestersol's drive went just as expected, a 42m drive -- not much farther than the previous sol, but in a shorter time, thanks to (ahem) remembering to turn on step-skipping during the autonav segment.
Other things are looking up, as well. Our flash volume and data-product count are both still high, but dropping, like the late stages of a Flash Flu. We still need to exercise some restraint; SOWG chair Wendy Calvin proposes alternating atmospheric science sols and geological science sols as a way to reduce flash pressure. Driving is the thing we won't cut -- at least, not entirely, though we do end up shortening the duration once again thisol -- since we have a definite goal and need to make continual progress. The sun is tracking northward in the sky once more, and it isn't going to wait for our little rover, no matter how much she needs and deserves it to.
Yestersol's drive left us about 40m from a small ridge, over which we can just barely see the current science goal (a pile of jumbly black stuff creatively named "Waypoint"). It's 150m away -- we had been thinking it was somewhat closer -- and with a clear, blind-drive-safe 35m to the ridge, we could have made 80m of progress if given three hours to drive (autonav willing). As it is, they cut the drive back to include only the blind part, and maybe a few meters of autonav. We'll make only 40 to 45m.
There's a lot of difference between those two numbers. But if that's all we can afford, that's what we'll do.
[Next post: sol 668 (Opportunity sol 649), November 19.]
Courtesy NASA/JPL-Caltech. The creatively named "Waypoint."
If there's a Beast involved, he's being awfully good to us. Yestersol's drive went just as expected, a 42m drive -- not much farther than the previous sol, but in a shorter time, thanks to (ahem) remembering to turn on step-skipping during the autonav segment.
Other things are looking up, as well. Our flash volume and data-product count are both still high, but dropping, like the late stages of a Flash Flu. We still need to exercise some restraint; SOWG chair Wendy Calvin proposes alternating atmospheric science sols and geological science sols as a way to reduce flash pressure. Driving is the thing we won't cut -- at least, not entirely, though we do end up shortening the duration once again thisol -- since we have a definite goal and need to make continual progress. The sun is tracking northward in the sky once more, and it isn't going to wait for our little rover, no matter how much she needs and deserves it to.
Yestersol's drive left us about 40m from a small ridge, over which we can just barely see the current science goal (a pile of jumbly black stuff creatively named "Waypoint"). It's 150m away -- we had been thinking it was somewhat closer -- and with a clear, blind-drive-safe 35m to the ridge, we could have made 80m of progress if given three hours to drive (autonav willing). As it is, they cut the drive back to include only the blind part, and maybe a few meters of autonav. We'll make only 40 to 45m.
There's a lot of difference between those two numbers. But if that's all we can afford, that's what we'll do.
[Next post: sol 668 (Opportunity sol 649), November 19.]
Courtesy NASA/JPL-Caltech. The creatively named "Waypoint."
2010-11-16
Spirit Sol 665
Ah, heck. We meant to make 60m of progress, and came up with only 37m.
How did this happen? Let's see, autonav imaging takes roughly 3 minutes per 50cm step, which means we should expect to take 20 such steps per hour, or -- oops -- 10m per hour. And we spent two hours on autonav, as predicted, and thus made only about 20m of progress in that time.
Both Ashitey and I were remembering that autonav makes more like 35m per hour. But this is true only when we turn on a feature called "step-skipping." Instead of "image, step, image, step," it does "image, step, step, image, step, step," or even "image, step, step, step, image, step, step, step." Since the "image" part is the slow part, this makes a dramatic improvement in how fast we can drive.
But since this is less safe, since the rover moves farther before taking images of the world around it, it's not the default.
Duh.
Thisol we decide we'll turn on step-skipping.
To make matters worse, we had a poor downlink, and we have more data products on board than we'd like. (We have to worry not only about the total amount of data on the rover but also the number of separate data products.) SOWG chair John Grant proposes cutting the drive short as a way of reducing the generation of new data products until some old ones can be downlinked.
"Nah," I say, "we're not gonna do that."
"But if we don't, the rover will start autodeleting old science products," Emily Eelkema chimes in.
That's what I was waiting for. Since it's been on my mind, I say: "Forget those old MI images! They're history! Let it go! Embrace change!"
They cut the drive short.
Well, it won't be too bad. With step-skipping on, we'll make the same amount of distance as today (about 40m) in the shortened time -- still a decent amount to show for ourselves.
As long as we're changing things in the sequence ... a significant contributor to the drive time is the periodic slip-check sequence. Since it has to invoke visodom, and all the complex 3-D processing that goes along with it, this tends to chew up a lot of duration. What's more, since we're now fairly confident we can make it all the way downhill, most of the original rationale for the slip checks (ensuring we'd be able to make it back to the top of Husband Hill if we got into a bad spot along the way) has disappeared. So I ask Ashitey what he thinks about taking them out.
"Maybe after the Martian-year anniversary," he says. It's next week. "We want both rovers healthy for that."
This is wisdom. I let it go.
Late in the day, Justin Maki points out an article on the Aviation Week Web site. It's something Craig Covault wrote about his recent visit. I didn't realize it at the time, but he was still here for our big discussion about IDDing Hillary.
Emily seems a little worried about the article, but I'm not. "I'm glad he was here for that," I say. "I think it shows us at our best. We had a real concern about a genuinely difficult and novel problem, we took it seriously, we did the analysis we had to do, we proceeded with the right degree of caution -- and in the end, we got the science."
Covault's article is hampered somewhat by the fact that he was on the other end of a teleconference, listening to unfamiliar voices. As a result, some of the details get smudged. But taking that into account, I think he got the big things right: our dedication, our talent, our -- well, damn it, I'm not ashamed to say our brilliance.
Because we're damn good at this. Even if we do forget to turn on step-skipping once in a while.
How did this happen? Let's see, autonav imaging takes roughly 3 minutes per 50cm step, which means we should expect to take 20 such steps per hour, or -- oops -- 10m per hour. And we spent two hours on autonav, as predicted, and thus made only about 20m of progress in that time.
Both Ashitey and I were remembering that autonav makes more like 35m per hour. But this is true only when we turn on a feature called "step-skipping." Instead of "image, step, image, step," it does "image, step, step, image, step, step," or even "image, step, step, step, image, step, step, step." Since the "image" part is the slow part, this makes a dramatic improvement in how fast we can drive.
But since this is less safe, since the rover moves farther before taking images of the world around it, it's not the default.
Duh.
Thisol we decide we'll turn on step-skipping.
To make matters worse, we had a poor downlink, and we have more data products on board than we'd like. (We have to worry not only about the total amount of data on the rover but also the number of separate data products.) SOWG chair John Grant proposes cutting the drive short as a way of reducing the generation of new data products until some old ones can be downlinked.
"Nah," I say, "we're not gonna do that."
"But if we don't, the rover will start autodeleting old science products," Emily Eelkema chimes in.
That's what I was waiting for. Since it's been on my mind, I say: "Forget those old MI images! They're history! Let it go! Embrace change!"
They cut the drive short.
Well, it won't be too bad. With step-skipping on, we'll make the same amount of distance as today (about 40m) in the shortened time -- still a decent amount to show for ourselves.
As long as we're changing things in the sequence ... a significant contributor to the drive time is the periodic slip-check sequence. Since it has to invoke visodom, and all the complex 3-D processing that goes along with it, this tends to chew up a lot of duration. What's more, since we're now fairly confident we can make it all the way downhill, most of the original rationale for the slip checks (ensuring we'd be able to make it back to the top of Husband Hill if we got into a bad spot along the way) has disappeared. So I ask Ashitey what he thinks about taking them out.
"Maybe after the Martian-year anniversary," he says. It's next week. "We want both rovers healthy for that."
This is wisdom. I let it go.
Late in the day, Justin Maki points out an article on the Aviation Week Web site. It's something Craig Covault wrote about his recent visit. I didn't realize it at the time, but he was still here for our big discussion about IDDing Hillary.
Emily seems a little worried about the article, but I'm not. "I'm glad he was here for that," I say. "I think it shows us at our best. We had a real concern about a genuinely difficult and novel problem, we took it seriously, we did the analysis we had to do, we proceeded with the right degree of caution -- and in the end, we got the science."
Covault's article is hampered somewhat by the fact that he was on the other end of a teleconference, listening to unfamiliar voices. As a result, some of the details get smudged. But taking that into account, I think he got the big things right: our dedication, our talent, our -- well, damn it, I'm not ashamed to say our brilliance.
Because we're damn good at this. Even if we do forget to turn on step-skipping once in a while.
2010-11-15
Spirit Sol 664
Many writers and philosophers have spoken of the same idea: life is change, the Eternal Now, you can't step in the same river twice, etc. This concept is logically sound, and I should embrace it. But I have a really hard time with change, especially when the team's changing.
The team's changing.
In a good way; at least nobody I like is leaving. We're bringing in a raft of new scientists, and they're sitting there monitoring the SOWG meeting. It feels weird to be, increasingly, an old-timer on this project that I still think of as the Fresh New Thing. But more, I'm worried that the new scientists are harbingers of unhappy days ahead, days when some of the scientists I've known and liked will be leaving.
I'm sure I'll like the new people, too. But it won't be the same. The moving finger writes, and, having writ, moves on.
We're doing okay on our drive template. Unlike the primary mission, the scientists are actually trying to keep ahead of the line on the graph that shows how far the rover needs to progress each sol. We'll take a hit at Thanksgiving and maybe another at Christmas, but for now we're ahead of the line, even after turning around and going back to spend a week or so IDDing Larry's Bench. Today we're going to make the situation even better, by plowing ahead another sixty meters.
The gaggle of new scientists has brought back a familiar face, Steve Squyres, who's still kvelling about our recent 100m drive (okay, 94m) and looking forward to this one. But, he points out, "You can't slide anything past this uplink team -- even if you try to blow past something interesting, they'll make you turn around and go back!"
Maybe I should figure out a way to cow the new scientists now, before they can develop a troublesome attitude. Change can also be seen as an opportunity, of course.
The team's changing.
In a good way; at least nobody I like is leaving. We're bringing in a raft of new scientists, and they're sitting there monitoring the SOWG meeting. It feels weird to be, increasingly, an old-timer on this project that I still think of as the Fresh New Thing. But more, I'm worried that the new scientists are harbingers of unhappy days ahead, days when some of the scientists I've known and liked will be leaving.
I'm sure I'll like the new people, too. But it won't be the same. The moving finger writes, and, having writ, moves on.
We're doing okay on our drive template. Unlike the primary mission, the scientists are actually trying to keep ahead of the line on the graph that shows how far the rover needs to progress each sol. We'll take a hit at Thanksgiving and maybe another at Christmas, but for now we're ahead of the line, even after turning around and going back to spend a week or so IDDing Larry's Bench. Today we're going to make the situation even better, by plowing ahead another sixty meters.
The gaggle of new scientists has brought back a familiar face, Steve Squyres, who's still kvelling about our recent 100m drive (okay, 94m) and looking forward to this one. But, he points out, "You can't slide anything past this uplink team -- even if you try to blow past something interesting, they'll make you turn around and go back!"
Maybe I should figure out a way to cow the new scientists now, before they can develop a troublesome attitude. Change can also be seen as an opportunity, of course.
2010-11-13
Spirit Sol 662
This was supposed to be an easy sol -- just an IDD tool change, a sequence I could crank out in a couple of minutes. It turns out to be even easier than that -- no RP sequences at all.
I spend the morning working with Ashley to build a candidate drive sequence for Monday. We don't get many opportunities to let an RP-in-training do something like this when there's no time pressure, so it seems like the perfect time.
When we're all done, we've got a sequence that Ashitey will build on for our first drive next week. And I go off to start clearing away the crushing backlog of other work.
[Next post: sol 664, November 15.]
I spend the morning working with Ashley to build a candidate drive sequence for Monday. We don't get many opportunities to let an RP-in-training do something like this when there's no time pressure, so it seems like the perfect time.
When we're all done, we've got a sequence that Ashitey will build on for our first drive next week. And I go off to start clearing away the crushing backlog of other work.
[Next post: sol 664, November 15.]
2010-11-10
Spirit Sol 660
Today we're planning two sols, 660 and 661. Spirit finds herself poised before an enticing chunk of outcrop called "Larry's Bench," and the question on the science team's mind is whether we want to IDD the part we can reach, or reposition the rover to IDD another part, just to the right.
At first, this seems like a silly question to me. How can this bit of outcrop be so much different from a chunk of the same stuff, a few centimeters away, that it's worth spending even one sol to reach the other bit? (Indeed, it would end up being two sols; VEX needs at least one of our uplinks.)
The answer, it turns out, is dust. The stuff in front of us is pretty dusty, and there's no very good surface to RAT-brush it. But the competing section is already less dusty and has a broader and flatter surface that may be more amenable to brushing.
But maybe we can brush the part in front of us ... and so the discussion goes on, and on, and on. The SOWG meeting, which we normally try to finish in under an hour, goes to an hour and a half. And a good chunk of it is spent going in circles, at that. Alicia Falacarro -- now Alicia Vaughan -- can't stand it when the meetings go this way. She looks at me, rolls her eyes, and groans, "Definitely the J.V. team today."
But the answer in the end is that we're going to IDD, not drive, and the IDD work is reasonably straightforward. (Ed Guinness names the IDD target "Thrasher" -- "since that's what we did in this meeting.") Indeed, Ashitey does most of it. It's your standard unstow, MB touch, MI-2x2x5+stereo, MB re-touch, RAT-brush, post-brush MI-2x2x5+stereo, MB re-touch, APXS kind of sol. Oh, and then another change-tool back to MB on the following sol.
Matt Heverly is shadowing me today, so I take the time to walk him carefully through the whole sequence and talk over his questions in detail. The price of this attention, of course, is that I put him on the hook to do the walkthrough at the CAM. And a good job he does of it, too.
Sol 661. Sol six hundred and sixty-one. Unfathomable. It's almost a Martian year -- a Martian year being 669.7 sols. They're going to celebrate it in von Karman next week.
Six hundred. And sixty-one.
And counting.
[Next post: sol 662, November 13.]
Courtesy NASA/JPL-Caltech. Larry's Bench, right between our wheels.
At first, this seems like a silly question to me. How can this bit of outcrop be so much different from a chunk of the same stuff, a few centimeters away, that it's worth spending even one sol to reach the other bit? (Indeed, it would end up being two sols; VEX needs at least one of our uplinks.)
The answer, it turns out, is dust. The stuff in front of us is pretty dusty, and there's no very good surface to RAT-brush it. But the competing section is already less dusty and has a broader and flatter surface that may be more amenable to brushing.
But maybe we can brush the part in front of us ... and so the discussion goes on, and on, and on. The SOWG meeting, which we normally try to finish in under an hour, goes to an hour and a half. And a good chunk of it is spent going in circles, at that. Alicia Falacarro -- now Alicia Vaughan -- can't stand it when the meetings go this way. She looks at me, rolls her eyes, and groans, "Definitely the J.V. team today."
But the answer in the end is that we're going to IDD, not drive, and the IDD work is reasonably straightforward. (Ed Guinness names the IDD target "Thrasher" -- "since that's what we did in this meeting.") Indeed, Ashitey does most of it. It's your standard unstow, MB touch, MI-2x2x5+stereo, MB re-touch, RAT-brush, post-brush MI-2x2x5+stereo, MB re-touch, APXS kind of sol. Oh, and then another change-tool back to MB on the following sol.
Matt Heverly is shadowing me today, so I take the time to walk him carefully through the whole sequence and talk over his questions in detail. The price of this attention, of course, is that I put him on the hook to do the walkthrough at the CAM. And a good job he does of it, too.
Sol 661. Sol six hundred and sixty-one. Unfathomable. It's almost a Martian year -- a Martian year being 669.7 sols. They're going to celebrate it in von Karman next week.
Six hundred. And sixty-one.
And counting.
[Next post: sol 662, November 13.]
Courtesy NASA/JPL-Caltech. Larry's Bench, right between our wheels.
2010-11-08
Opportunity Sol 637 (Spirit Sol 658)
Thisol I'm RP-1 on Opportunity, and Paolo's soloing as RP-2. But it's a straightforward IDD sol, and he hasn't had many chances to practice IDDing, so he asks if he can carry the bulk of it and I accept.
This is convenient for me, since I'm putting together a training presentation. At Sharon's prodding, I'm doing something Saina asked for, oh, I've lost track of how many months ago: trying to educate the rest of the uplink team about what rover drivers do, what all the mobility commands mean, and so on. This presentation has turned into something of a monster -- about 75 slides -- and I'm not done with it yet. So being able to turn Paolo loose, even part of the time, and work on this presentation, turns out to be an enormous help.
Of course, when he needs me, I watch over Paolo's shoulder and guide him through the process. But he's doing pretty well already.
When we catch a break, he asks me wonderingly, "How did you guys train originally?"
I tell him about the thread tests, the limited training sessions we had, and so on. But it wasn't really much. "If a new candidate rover driver came along who knew only as much as I knew when we landed, I wouldn't let him anywhere near the rovers," I say truthfully. "But we made it through somehow."[1]
How, I'll never know. Mars must have been in a good mood.
[Next post: sol 660, November 10.]
Footnotes:
[1] Sadly, five years on, this has only become even more true: it's gotten harder and harder to drive the rovers, because the list of things you need to know has expanded so much. Also, our training opportunities are more limited right now: Spirit's not driving, and Opportunity ops tends to be very routine, with limited IDD work and drives that all look much the same as we cruise across the plains. So it's become harder and harder to train new people, no matter how sharp and talented they might be (as both of our current trainees are).
This is convenient for me, since I'm putting together a training presentation. At Sharon's prodding, I'm doing something Saina asked for, oh, I've lost track of how many months ago: trying to educate the rest of the uplink team about what rover drivers do, what all the mobility commands mean, and so on. This presentation has turned into something of a monster -- about 75 slides -- and I'm not done with it yet. So being able to turn Paolo loose, even part of the time, and work on this presentation, turns out to be an enormous help.
Of course, when he needs me, I watch over Paolo's shoulder and guide him through the process. But he's doing pretty well already.
When we catch a break, he asks me wonderingly, "How did you guys train originally?"
I tell him about the thread tests, the limited training sessions we had, and so on. But it wasn't really much. "If a new candidate rover driver came along who knew only as much as I knew when we landed, I wouldn't let him anywhere near the rovers," I say truthfully. "But we made it through somehow."[1]
How, I'll never know. Mars must have been in a good mood.
[Next post: sol 660, November 10.]
Footnotes:
[1] Sadly, five years on, this has only become even more true: it's gotten harder and harder to drive the rovers, because the list of things you need to know has expanded so much. Also, our training opportunities are more limited right now: Spirit's not driving, and Opportunity ops tends to be very routine, with limited IDD work and drives that all look much the same as we cruise across the plains. So it's become harder and harder to train new people, no matter how sharp and talented they might be (as both of our current trainees are).
2010-11-05
Spirit Sol 655
The plan is to drive 17m at an azimuth of 130deg. But I have a better idea. We're perched at the top of a local hillock, and spread out below us is Haskin Ridge. We've got a beautiful view of it, a superb terrain mesh -- and, best of all, three hours of driving in the plan.
I think we can do more like 100m.
And it should be fairly simple. The hillock itself is pretty steep -- we might see 22deg tilt on the way down it -- but that's only for the first 15m or so. After that, the terrain quickly shallows out, so that we're in a far more forgiving 8-10deg of tilt all the way down. We can do 50m blind, and the terrain beyond looks decent for autonav. We don't get opportunities like this so often that we can just throw them away.
The day has its ups and downs. It turns out they forgot about a coordinated observation with HST, so they have to completely swap the plan around, but we still end up with about three hours of driving time. And Squyres decides they'd rather we drove at an azimuth of more like 100deg than 130deg, so that they can do a long-baseline stereo observation along the ridge. But at the end of the day, we've got a 110m drive planned, and a real shot at making it to the end -- or at least breaking 100m.
I expect to have real problems selling this to management, but everyone's pretty enthusiastic. When I tell Jim how far we're driving, he says, "I'd kill to get that on Opportunity right now." Then he reconsiders. "Well ... I'd wound," he amends.
Not surprisingly, John Wright -- one of the more aggressive drivers, and RP-2 today -- is fully up for this. "Throw caution to the wind!" he says.
"It's thrown!" I answer.
"Damn the torpedoes!"
"Damned!"
A LATER NOTE:
That was Thursday. We were planning well ahead, for a drive to execute Saturday. And Friday, Chris and Ashitey look at the drive and decide to raise some objections.
Their basic argument is reasonable enough: we're driving into terrain that looks benign enough, but in the hills we've found that we can get bogged down even at relatively low tilts and in terrain that looks benign from a distance. So we should add slip checks to the sequence.
John and I don't really agree, and I don't want to call in the uplink team on a day they'd been promised off, but I can't say they're being unreasonable. And our general approach is that if an RP has a reasonable concern, we take it seriously. So I call in Jake Matijevic and Beth Dewell, and they agree that we should act on Chris's concerns.
The required sequencing is easy enough. I simply break up the final waypoint into 15m segments, adding a slip check between segments. Voila.
John complains bitterly about this: "They're just trying to see to it that we don't get our hundred meters."
"Look," I tell him, "they've got a reasonable concern and we've agreed to act on it, so there's no use arguing about it now."
"I agree," he nods. "But I never said I wouldn't bitch about it."
Good point.
A LATER NOTE [2]:
94 meters. It doesn't quite have the cachet of 100m, but who cares? It feels gooood.
[Next post: sol 658 (Opportunity sol 637), November 8.]
I think we can do more like 100m.
And it should be fairly simple. The hillock itself is pretty steep -- we might see 22deg tilt on the way down it -- but that's only for the first 15m or so. After that, the terrain quickly shallows out, so that we're in a far more forgiving 8-10deg of tilt all the way down. We can do 50m blind, and the terrain beyond looks decent for autonav. We don't get opportunities like this so often that we can just throw them away.
The day has its ups and downs. It turns out they forgot about a coordinated observation with HST, so they have to completely swap the plan around, but we still end up with about three hours of driving time. And Squyres decides they'd rather we drove at an azimuth of more like 100deg than 130deg, so that they can do a long-baseline stereo observation along the ridge. But at the end of the day, we've got a 110m drive planned, and a real shot at making it to the end -- or at least breaking 100m.
I expect to have real problems selling this to management, but everyone's pretty enthusiastic. When I tell Jim how far we're driving, he says, "I'd kill to get that on Opportunity right now." Then he reconsiders. "Well ... I'd wound," he amends.
Not surprisingly, John Wright -- one of the more aggressive drivers, and RP-2 today -- is fully up for this. "Throw caution to the wind!" he says.
"It's thrown!" I answer.
"Damn the torpedoes!"
"Damned!"
A LATER NOTE:
That was Thursday. We were planning well ahead, for a drive to execute Saturday. And Friday, Chris and Ashitey look at the drive and decide to raise some objections.
Their basic argument is reasonable enough: we're driving into terrain that looks benign enough, but in the hills we've found that we can get bogged down even at relatively low tilts and in terrain that looks benign from a distance. So we should add slip checks to the sequence.
John and I don't really agree, and I don't want to call in the uplink team on a day they'd been promised off, but I can't say they're being unreasonable. And our general approach is that if an RP has a reasonable concern, we take it seriously. So I call in Jake Matijevic and Beth Dewell, and they agree that we should act on Chris's concerns.
The required sequencing is easy enough. I simply break up the final waypoint into 15m segments, adding a slip check between segments. Voila.
John complains bitterly about this: "They're just trying to see to it that we don't get our hundred meters."
"Look," I tell him, "they've got a reasonable concern and we've agreed to act on it, so there's no use arguing about it now."
"I agree," he nods. "But I never said I wouldn't bitch about it."
Good point.
A LATER NOTE [2]:
94 meters. It doesn't quite have the cachet of 100m, but who cares? It feels gooood.
[Next post: sol 658 (Opportunity sol 637), November 8.]
Subscribe to:
Posts (Atom)
Posts
