We're planning two sols today, and my part could hardly be simpler: just a single tool change from APXS to MB. Less than twenty-five minutes after I arrive on Lab, I'm done for the day.
Or I should be. But I have to stick around and play both my part and Ashitey's, since he's been asked to look into Spirit's RAT anomaly. This happened some weeks ago: the teeth on Spirit's RAT finally wore away (after 15 grindings, five times as many as its design required). While this was disappointing, in itself it was neither surprising nor worrisome. But what was worrisome was what happened afterward: as the bare metal below the worn-away grind head contacted the rock, the IDD lost contact with the rock face and started skipping around, vibrating the entire arm. This went on for an hour and is extremely dangerous -- it could have broken the arm. Luckily, Spirit survived this event, but we're worried about what could happen if Opportunity goes through the same thing. So Ashitey's looking into ways to keep that from happening.
He's got to be down in the testbed thisol, but he promises to come back for the meetings and for this afternoon's end-of-sol science discussion. (That's no problem. There are sols when we need 1.5 RPs, and sols when we need 2.5. We get 2.0 each sol, so I guess we average right. Thisol is more like 0.5, so Ashitey could go hang out at the beach today for all it really mattered.) "I think we're not going to the summit," he says. "At least, that's my feeling."
"We've got to go to the summit," I exclaim.
"I agree," he nods. "We shouldn't let this thing conquer us." He turns to go. "We should convince the scientists we can get to the top in 25 sols." This is a joke: it'll take considerably longer than that, and we both know it. The only problem is, so do the scientists.
Well, since I've got some spare time, I put it to good use. I know approximately what they're going to want tomorrow, so I go ahead and put together a candidate sequence to do it. That should help my fellow RPs tomorrow. Part of this work involves planning for sol 500. Sol 500! In a few more days, we'll have a combined total of 1000 sols of operations on both rovers.
Between this and other catch-up work, I manage to find ways to keep myself busy until the end-of-sol meeting. The first part of this meeting is an hour of pent-up discussion of science results and what they mean. I haven't been able to attend one in a long time, and usually find these fascinating, but last night I barely slept, and I just can't focus. I snap to when the discussion changes to a strategic one.
Squyres puts the central question to the group: "Where do we go next? Home Plate? The summit? Into the Tennessee Valley?" Basically, this means: do we go around Husband Hill, to its top, or turn our back on it and descend to the valley we can see from here?
I have my strongly held opinion, but I keep it to myself a while. The valley gets more or less ruled out, mainly because there's nothing obviously compelling about it and we might not be able to climb its far side if we went there.
To focus the discussion, Steve asks another question: "The dust storm season is coming. Suppose we had only two to three months to use these vehicles; how would we use them?" The science team's answer to this question revolves around characterizing the local structure and stratigraphy -- basically, wandering around more or less where we are. I can't hold my tongue any longer. If we have only two or three months of life left in this rover, it deserves to go to the top of this hill, and I say so. In his heart of heart, that's what Steve wants, too -- this I'm sure of -- but he's got to be able to make a purely scientific case for it. So he evinces a reserved enthusiasm, and shortly the discussion moves on.
I mainly keep quiet, but I can't resist speaking up when someone asks about our chances of surviving the dust storms. "If we're at the summit," I point out, "there's less dust between us and the sun." This gets a big laugh.
It also gets a serious answer from Steve: "The dust storm season is variable. We could be dead in two weeks from a tau of 5, or we could sail right on through with a tau of 1.2. Both things are possible. Our strategy from the start on this vehicle has been to do as much science as we can, as fast as we can."
I admit I've gotten out of the habit of thinking of our babies that way. They've survived so much, they've come so far, I've been thinking nothing would ever stop them.
But I know something will. I don't know what, but something will.
And, damn it, I want to be on the top of that hill before it happens!
[Next post: sol 502, June 1.]
2010-05-27
2010-05-25
Spirit Sol 495
We're just about to enter a period of restricted sols; consequently, the downlink's late -- 12:40 is the nominal start time. When it shows up, everyone's congratulating me on the success of the drive. But they're more sure it was a success than I am. I really wanted this target rock right in front of the front wheels, and it's definitely farther away than that.
As more data shows up, it becomes clear that we're close enough. The rock is a flattish slab, and the lower part -- the side facing us -- is reachable, though the top is not. But as Albert Yen points out, the top is the dustiest part anyway. They're more interested in the relatively dust-free side part, which we can reach. So we won't need to bump.
The whole situation reminds me, oddly enough, of Mazatzal: a relatively short approach to a flattish rock, and we came up just a little bit short of the goal, but close enough to get the job done. This time, unlike Mazatzal, I don't feel lousy about the result. I've changed.
We plan two sols' worth of activities on this as-yet-unnamed rock. Following a now-familiar pattern, on the first sol, we'll do an MI mosaic, then switch to APXS. And on the second sol, we RAT-brush the spot and redo the MI mosaic and APXS. Back when we did this on Mazatzal, I'd have been frantic. Now I'm hardly breaking a sweat.
As the SOWG meeting wraps up, Albert points out that we need target names. "Are we still sticking with the nautical theme?" I ask. "Because I was going to suggest that we call the rock 'Pequod' and do a Moby Dick theme." This suggestion goes over well. When it comes time to pick some target names, I key the mike again: "We should call one of them 'Ishmael,'" I point out to appreciative laughter. The preliminary IDD target gets that name, though later, based on advice from the RAT team, we switch to a different target, Ahab. Well, that's OK: if you're going to be obsessed with a target for a few days, it might as well be a target named for a character who's practically a synonym for obsession.
I finish both sols' IDD sequences and hand them over to Chris. Then I check out the images from yestersol, which include some very cool MIs. The ones we have are slightly out of focus -- not surprising, since we forewent the MB touch in order to preserve the soil -- but we can see that they're going to be awesome when we get the in-focus versions. One looks like a cliff, the others like a miniature city or something.
Just before the CAM I step out to stretch and run into Mark Maimone. He fills me in on Opportunity's progress. They're seeing 99.5% slip: so far they've commanded 48m and have made 27cm of progress. This in itself is more or less what we'd expect, based on the testbed data. But what worries him is that we don't seem to have risen any yet -- it's as if we're just pushing this stuff in front of us all the way down the side of the ripple. He's concerned that we'll end up at the bottom of the ripple and still be buried, in which case we might never get back above the stuff.
On a lighter note, he points out that our expectations have dropped four orders of magnitude. "We used to do 200m sols, now we do 2cm sols," he says. And we're just as happy to get them.
[Next post: sol 497, May 27.]
Courtesy NASA/JPL-Caltech. The view from the end of the drive, with Pequod just in reach.
As more data shows up, it becomes clear that we're close enough. The rock is a flattish slab, and the lower part -- the side facing us -- is reachable, though the top is not. But as Albert Yen points out, the top is the dustiest part anyway. They're more interested in the relatively dust-free side part, which we can reach. So we won't need to bump.
The whole situation reminds me, oddly enough, of Mazatzal: a relatively short approach to a flattish rock, and we came up just a little bit short of the goal, but close enough to get the job done. This time, unlike Mazatzal, I don't feel lousy about the result. I've changed.
We plan two sols' worth of activities on this as-yet-unnamed rock. Following a now-familiar pattern, on the first sol, we'll do an MI mosaic, then switch to APXS. And on the second sol, we RAT-brush the spot and redo the MI mosaic and APXS. Back when we did this on Mazatzal, I'd have been frantic. Now I'm hardly breaking a sweat.
As the SOWG meeting wraps up, Albert points out that we need target names. "Are we still sticking with the nautical theme?" I ask. "Because I was going to suggest that we call the rock 'Pequod' and do a Moby Dick theme." This suggestion goes over well. When it comes time to pick some target names, I key the mike again: "We should call one of them 'Ishmael,'" I point out to appreciative laughter. The preliminary IDD target gets that name, though later, based on advice from the RAT team, we switch to a different target, Ahab. Well, that's OK: if you're going to be obsessed with a target for a few days, it might as well be a target named for a character who's practically a synonym for obsession.
I finish both sols' IDD sequences and hand them over to Chris. Then I check out the images from yestersol, which include some very cool MIs. The ones we have are slightly out of focus -- not surprising, since we forewent the MB touch in order to preserve the soil -- but we can see that they're going to be awesome when we get the in-focus versions. One looks like a cliff, the others like a miniature city or something.
Just before the CAM I step out to stretch and run into Mark Maimone. He fills me in on Opportunity's progress. They're seeing 99.5% slip: so far they've commanded 48m and have made 27cm of progress. This in itself is more or less what we'd expect, based on the testbed data. But what worries him is that we don't seem to have risen any yet -- it's as if we're just pushing this stuff in front of us all the way down the side of the ripple. He's concerned that we'll end up at the bottom of the ripple and still be buried, in which case we might never get back above the stuff.
On a lighter note, he points out that our expectations have dropped four orders of magnitude. "We used to do 200m sols, now we do 2cm sols," he says. And we're just as happy to get them.
[Next post: sol 497, May 27.]
Courtesy NASA/JPL-Caltech. The view from the end of the drive, with Pequod just in reach.
2010-05-24
Spirit Sol 494
Even though we're not getting the downlink until late -- so we could be starting at a decent hour -- I've had to come in early to support a 10:00 planning meeting. (For some reason, we have this pre-planning meeting at 10:00, even though the uplink process doesn't start until 13:00.) I arrive more than an hour early, because if you show up on Lab past 09:00 all the parking is gone. And I get everything nicely set up, and start some work of my own, and my damn machine reboots.
Turns out they're having some problem with the file servers and they've decided to try to fix it by rebooting all the clients. Without warning.
Well, it's not entirely true that they gave no warning. Apparently, they sent some people email (not me). So if you happened to be one of those people, and you happened to check your email at the right time -- you know, because everybody checks their email every five minutes, just in case the SAs are going to reboot the machines for some incomprehensible reason -- then you had some warning. I guess. The rest of us are supposed to just deal with it, apparently.
Perhaps my attitude is slipping. I need John Wright to come by and grin and say, "This is so cool."
So I get everything back into some semblance of decent shape before the meeting starts. Squyres, despite a badly stuffed-up nose, takes a few minutes to say, "We're shortly going to be facing a big decision on Spirit: how to continue exploring Husband Hill, in light of our recent discoveries here and our improved power situation." He's been mentioning the need for this discussion for a while now, planting the seed. I think this means: "I want to meander up the hillside more slowly than we had been planning to," which is really fine with me. As long as we go to the top.
The pre-planning meeting itself explores a number of options. Are we going to stay here and IDD? Do a long drive back uphill? Do a short drive and explore another part of the outcrop? It's the last choice that wins. We pick a spot at one edge of the outcrop, about 2m from our present position. They'd like to go farther away, but that's where the outcrop ends.
So we'll do one last bit of IDD work on the current chunk of outcrop, then drive on to set ourselves up to continue at another part. I have both sequences roughed out before the SOWG meeting ends, and Chris and I split them: he takes the IDD sequence, I take the drive.
It seems like a long time since I've been RP-1 for a meaningful drive in the Spirit World. It's a short one, but I'm still worried about it. Lots of places where things could go wrong, unknown terrain under our wheels, a long approach we're trying to nail in one sol ... heck, come to think of it, it's just like old times.
Turns out they're having some problem with the file servers and they've decided to try to fix it by rebooting all the clients. Without warning.
Well, it's not entirely true that they gave no warning. Apparently, they sent some people email (not me). So if you happened to be one of those people, and you happened to check your email at the right time -- you know, because everybody checks their email every five minutes, just in case the SAs are going to reboot the machines for some incomprehensible reason -- then you had some warning. I guess. The rest of us are supposed to just deal with it, apparently.
Perhaps my attitude is slipping. I need John Wright to come by and grin and say, "This is so cool."
So I get everything back into some semblance of decent shape before the meeting starts. Squyres, despite a badly stuffed-up nose, takes a few minutes to say, "We're shortly going to be facing a big decision on Spirit: how to continue exploring Husband Hill, in light of our recent discoveries here and our improved power situation." He's been mentioning the need for this discussion for a while now, planting the seed. I think this means: "I want to meander up the hillside more slowly than we had been planning to," which is really fine with me. As long as we go to the top.
The pre-planning meeting itself explores a number of options. Are we going to stay here and IDD? Do a long drive back uphill? Do a short drive and explore another part of the outcrop? It's the last choice that wins. We pick a spot at one edge of the outcrop, about 2m from our present position. They'd like to go farther away, but that's where the outcrop ends.
So we'll do one last bit of IDD work on the current chunk of outcrop, then drive on to set ourselves up to continue at another part. I have both sequences roughed out before the SOWG meeting ends, and Chris and I split them: he takes the IDD sequence, I take the drive.
It seems like a long time since I've been RP-1 for a meaningful drive in the Spirit World. It's a short one, but I'm still worried about it. Lots of places where things could go wrong, unknown terrain under our wheels, a long approach we're trying to nail in one sol ... heck, come to think of it, it's just like old times.
2010-05-18
Spirit Sol 488
"Good, but not perfect." That's Chris's assessment of yestersol's drive. The first guarded arc, meant to take us safely down a smooth slide to our final position, refused to move. This later turns out to be a combination of dangers from nearby hazards -- a couple of rocks beside the top of the slide that are near, or barely above, the hazard-avoidance code's safety threshold -- and an unknown patch on the slide itself, a result of the slide being too smooth for the stereo correlator to determine distance.
So we're about 7m away from our goal. It should be an easy drive thisol, and luckily I prepped the science team in advance, warning them that it might take us three sols to get there. I'm learning.
Today I find out why the science team is so eager to return to Larry's Lookout. From this side, they can see that it's volcanic rock that was altered by water. Not only is water what we came to find, but this alteration is specifically interesting because this rock is tens of meters above the surrounding terrain. That means the water here at Gusev Crater wasn't shallow, and probably wasn't just embedded in a light but long-term sulfuric acid mist (one of the theories I've heard before). This stuff was deep -- tens of meters deep at least. This could be the evidence we were hoping for all along, the evidence of an actual lake here at Gusev.
Tomorrow we should be at the base of the lookout, maybe another sol to climb to the rocks. Then we'll know.
[Next post: sol 494, May 24.]
Courtesy NASA/JPL-Caltech. The view out our front windshield as we planned thisol ....
Courtesy NASA/JPL-Caltech. ... and the corresponding view out the back.
So we're about 7m away from our goal. It should be an easy drive thisol, and luckily I prepped the science team in advance, warning them that it might take us three sols to get there. I'm learning.
Today I find out why the science team is so eager to return to Larry's Lookout. From this side, they can see that it's volcanic rock that was altered by water. Not only is water what we came to find, but this alteration is specifically interesting because this rock is tens of meters above the surrounding terrain. That means the water here at Gusev Crater wasn't shallow, and probably wasn't just embedded in a light but long-term sulfuric acid mist (one of the theories I've heard before). This stuff was deep -- tens of meters deep at least. This could be the evidence we were hoping for all along, the evidence of an actual lake here at Gusev.
Tomorrow we should be at the base of the lookout, maybe another sol to climb to the rocks. Then we'll know.
[Next post: sol 494, May 24.]
Courtesy NASA/JPL-Caltech. The view out our front windshield as we planned thisol ....
Courtesy NASA/JPL-Caltech. ... and the corresponding view out the back.
2010-05-17
Spirit Sol 487
As I'm driving in I see the asshole who got thrown out of this year's JPL Open House for shouting abuse at Mark Adler and Nagin Cox. This happened day before yesterday. Mark and Nagin were standing with the testbed rover outside the 303 cafeteria (where most of the Mars stuff was), volunteering their personal time to answer questions asked by visitors who were in the line to get into the building. I watched this for a while (indeed, late in the day, I did it myself for a while) and most of the visitors were polite, friendly, and supportive. Just nice folks, spending the day there with their kids, learning about space and science.
But not this guy. This jackass was shouting at the top of his lungs, personally insulting Mark and Nagin, calling them liars. Some of the visitors, perhaps worried for their kids, called the guards, and although Nagin pleaded with them not to throw the guy out -- she didn't want anyone to think they weren't free to ask any question they wanted -- I saw them talking to him later on, and shortly after that I heard he had been asked to leave. Now he's standing out by the main road into JPL, where he's put up posters directly and specifically accusing Tim Parker -- by name -- of being a liar, of doctoring images.
I ought to be amused by this. I want to run him over. But if aikido has taught me anything, it's that you don't confront anger with anger, attack with attack. You smile, welcome your attacker's momentum as a gift ... and then spin like a Martian dust devil and throw that motherfucker headfirst into a wall.
If he gets up and comes back, you smile and bow a little and do it again.
See you next year, asshole.
The weekend went well for the rovers. Friday's attempt to drive Opportunity made more progress than expected -- we're still talking about only a couple of centimeters, but we'll take it. And Spirit's IDD work went nominally, too.
So thisol we're finishing up the IDD work, then stowing the IDD and driving the remaining 30m or so to Larry's Lookout. It's a complex sol, so we break up the work: Chris does the drive, I do the IDD stuff. The IDD stuff is actually not too hard, although it does have a twist: the MI images we got back over the weekend suggest that our best-focus position on this surface might not be where we thought, so I work with Craig Leff to come up with a custom strategy.
Chris has the hard part, and he turns out to be the tall pole for the day. I think John Grant regrets that thisol is the sol we chose to drive, because he's got tickets for the Washington Nationals game. When we're done, the game's already in the third inning.
But he makes it.
But not this guy. This jackass was shouting at the top of his lungs, personally insulting Mark and Nagin, calling them liars. Some of the visitors, perhaps worried for their kids, called the guards, and although Nagin pleaded with them not to throw the guy out -- she didn't want anyone to think they weren't free to ask any question they wanted -- I saw them talking to him later on, and shortly after that I heard he had been asked to leave. Now he's standing out by the main road into JPL, where he's put up posters directly and specifically accusing Tim Parker -- by name -- of being a liar, of doctoring images.
I ought to be amused by this. I want to run him over. But if aikido has taught me anything, it's that you don't confront anger with anger, attack with attack. You smile, welcome your attacker's momentum as a gift ... and then spin like a Martian dust devil and throw that motherfucker headfirst into a wall.
If he gets up and comes back, you smile and bow a little and do it again.
See you next year, asshole.
The weekend went well for the rovers. Friday's attempt to drive Opportunity made more progress than expected -- we're still talking about only a couple of centimeters, but we'll take it. And Spirit's IDD work went nominally, too.
So thisol we're finishing up the IDD work, then stowing the IDD and driving the remaining 30m or so to Larry's Lookout. It's a complex sol, so we break up the work: Chris does the drive, I do the IDD stuff. The IDD stuff is actually not too hard, although it does have a twist: the MI images we got back over the weekend suggest that our best-focus position on this surface might not be where we thought, so I work with Craig Leff to come up with a custom strategy.
Chris has the hard part, and he turns out to be the tall pole for the day. I think John Grant regrets that thisol is the sol we chose to drive, because he's got tickets for the Washington Nationals game. When we're done, the game's already in the third inning.
But he makes it.
2010-05-14
Spirit Sol 484
Thisol we're planning a little light IDD work for the weekend -- an MI mosaic, a RAT brush, a RAT-to-APXS tool change. The next sol, we just switch from the APXS to the MB.
Since I'm coming back from the other side of the planet, I have no context at all. The SOWG meeting keeps me hopping, evaluating terrain and imaging requests as well as target reachability. This is the sort of thing that used to drive me nuts back in the nominal mission, but now I have more experience -- and confidence -- so I handle it with aplomb. Maybe even two plombs. Whatever those are.
Later on I find that the target we're using isn't quite as good as I thought earlier. At the end of the first sol, our APXS placement brings the turret right up to the hardstop. The maximum angle for the turret is 3.30 radians, and RSVP shows this placement goes to 3.297. If you allow for the possibility of early contact (a possibility we must always consider), the simulation gives an even more worrisome result: 3.299. Theoretically we should be fine, but that's not exactly a whole lot of margin. So I end up tweaking the target normal slightly, just for the APXS placement.
It's Jascha's last sol before he goes off to grad school, and it's Heather Arneson's last sol as a PANCAM PUL; in two weeks, she'll also be gone. I never knew Jascha well, but I knew Heather very well. She was not only damn good at her job, she also had a fantastic attitude. I missed her when she left JPL to go back for Ithaca -- for months she's been just a voice on the phone -- and I'll miss her when she leaves us for good. As it happens, Heather (and Miles, her boyfriend and also a PANCAM team member) is heading to my own alma mater, the UIUC. "It's a great school, if you can handle the weather," I tell her. Then again, she's used to Ithaca, NY. Heck, Illinois weather might be better.
Shoot, Mars might be better.
[Next post: sol 487, May 17.]
Since I'm coming back from the other side of the planet, I have no context at all. The SOWG meeting keeps me hopping, evaluating terrain and imaging requests as well as target reachability. This is the sort of thing that used to drive me nuts back in the nominal mission, but now I have more experience -- and confidence -- so I handle it with aplomb. Maybe even two plombs. Whatever those are.
Later on I find that the target we're using isn't quite as good as I thought earlier. At the end of the first sol, our APXS placement brings the turret right up to the hardstop. The maximum angle for the turret is 3.30 radians, and RSVP shows this placement goes to 3.297. If you allow for the possibility of early contact (a possibility we must always consider), the simulation gives an even more worrisome result: 3.299. Theoretically we should be fine, but that's not exactly a whole lot of margin. So I end up tweaking the target normal slightly, just for the APXS placement.
It's Jascha's last sol before he goes off to grad school, and it's Heather Arneson's last sol as a PANCAM PUL; in two weeks, she'll also be gone. I never knew Jascha well, but I knew Heather very well. She was not only damn good at her job, she also had a fantastic attitude. I missed her when she left JPL to go back for Ithaca -- for months she's been just a voice on the phone -- and I'll miss her when she leaves us for good. As it happens, Heather (and Miles, her boyfriend and also a PANCAM team member) is heading to my own alma mater, the UIUC. "It's a great school, if you can handle the weather," I tell her. Then again, she's used to Ithaca, NY. Heck, Illinois weather might be better.
Shoot, Mars might be better.
[Next post: sol 487, May 17.]
2010-05-13
Opportunity Sol 463 (Spirit Sol 483)
I'm back on Spirit as of today, but yesterday they started a two-sol MB integration. So I basically have nothing to do. (But I do have one nice surprise at the SOWG meeting: I overhear Brenda say, "Even Squyres admitted the other day that Spirit is now his favorite.")
So I do some other work and then go hang out with the Opportunity drivers. The good news is, the wheels steered, exactly the way we wanted. Jeff and Frank are mucking around with the candidate sequence Khaled and I wrote yesterday (they almost seem apologetic about it, but I tell them, "Mess with it to your heart's content"). And just after I poke my head in, there's a meeting about the Opportunity mobility plan.
I tag along for the meeting. Khaled's there, along with Jeng and Mark Maimone. When Jeff, Frank, and I show up, I count heads. Counting Mark as one, that makes six rover drivers -- one for each wheel. Not that we'd ever overdo things around here.
The meeting consists of getting everybody (RPs, managers, scientists, mechanical people, press people, and more) together in the SOWG room to tell them that the wheels steered and that we're going to try to drive.
Well, all right, there's a little more to it than that. The real point of the meeting is to go over the details. Mark is the main one doing the analysis, and is, as usual, amazingly thorough. Plot after plot, image after image, he methodically lays out the entire sequence of events. Which can be summarized this way: we turned the wheels, and sank a tiny amount (as expected) when we did that. And we're ready to try driving.
So I do some other work and then go hang out with the Opportunity drivers. The good news is, the wheels steered, exactly the way we wanted. Jeff and Frank are mucking around with the candidate sequence Khaled and I wrote yesterday (they almost seem apologetic about it, but I tell them, "Mess with it to your heart's content"). And just after I poke my head in, there's a meeting about the Opportunity mobility plan.
I tag along for the meeting. Khaled's there, along with Jeng and Mark Maimone. When Jeff, Frank, and I show up, I count heads. Counting Mark as one, that makes six rover drivers -- one for each wheel. Not that we'd ever overdo things around here.
The meeting consists of getting everybody (RPs, managers, scientists, mechanical people, press people, and more) together in the SOWG room to tell them that the wheels steered and that we're going to try to drive.
Well, all right, there's a little more to it than that. The real point of the meeting is to go over the details. Mark is the main one doing the analysis, and is, as usual, amazingly thorough. Plot after plot, image after image, he methodically lays out the entire sequence of events. Which can be summarized this way: we turned the wheels, and sank a tiny amount (as expected) when we did that. And we're ready to try driving.
2010-05-12
Opportunity Sol 462 (Spirit Sol 482)
It's sol 462 here in the Land of Opportunity -- or, rather, we're planning sol 462. But we still haven't uplinked sol 461, because we're in that weird phase.
It's a restricted sol, which means we RPs have no sequences to build. The SOWG meeting finishes in record time (literally).
We have no idea whether yestersol's wheel-steering sequence worked, but Khaled and I plan a candidate drive sequence anyway. We know the general outlines of what Jeff wants to try, and we build that: a 2-meter arc back the way we came, with a heading change to the left, broken up into 10 steps of 20cm each. And lots and lots and lots of pictures.
Will we send it? We'll have to wait until we see what happened when we tried to steer.
It's a restricted sol, which means we RPs have no sequences to build. The SOWG meeting finishes in record time (literally).
We have no idea whether yestersol's wheel-steering sequence worked, but Khaled and I plan a candidate drive sequence anyway. We know the general outlines of what Jeff wants to try, and we build that: a 2-meter arc back the way we came, with a heading change to the left, broken up into 10 steps of 20cm each. And lots and lots and lots of pictures.
Will we send it? We'll have to wait until we see what happened when we tried to steer.
2010-05-11
Opportunity Sol 461 (Spirit Sol 481)
Frank Hartman, Jeng Yen, and Khaled Ali are already at the SOWG meeting when I show up, making me the fourth RP in the room. All this for a drive sequence that just steers the wheels to a different position. "We're multiplying," Frank says. "We can each take one milliradian of the turn."
Well, with so many of us around, and the bulk of the work already done yesterday, I suppose I won't have much to do. I do get involved in a discussion about some drive-supporting PANCAM imaging. Blue-stereo images usually give us the sharpest picture, but Elaina McCartney points out that the images will be taken at a particularly warm time of day, and consequently the blue-stereo images will have speckling that might interfere with the analysis we're trying to do. So we switch to the next-best thing, red-stereo. But she also generously volunteers to take a blue-stereo pair as well, just so we can see what the images would look like, in case they might prove useful after all. Despite the large data volume required for this, the SOWG chair, Ed Guinness, shrugs acceptance. "Damn," I say, "it's almost like we're a team here or something."
My easy, laid-back day turns out to be a lot busier than I anticipated. Frank and Jeng go down to the testbed to start testing out the next sequence, and Khaled and I have more work to do on thisol's sequence than we'd thought. Nothing major, just lots of picky little things. Working out the final image pointing. Mucking with the visodom image parameters so we'll get more data. Going over the timing with the TULs. That sort of thing.
Indeed, when lunchtime approaches, we're both too busy to leave. "We need a gofer," I say, and Khaled agrees wholeheartedly, or rather wholestomachedly.
But we're ready in time for all the usual meetings. (I even show an animation at the APAM: the rover sits still for five seconds, then turns the wheels. Woo-hoo. I attach the animation to the uplink report.)
When it's all said and done, not only all four RPs who were there at the SOWG but also Jeff Biesiadecki and Mark Maimone have reviewed it, along with many others, such as Rick Welch. If it takes all this just to steer the wheels now, I don't want to see what happens when we try to drive. It's a far cry from the days when sequencing Opportunity meant: well, we expanded the über-macro, there's your 200m drive -- see ya!
Well, with so many of us around, and the bulk of the work already done yesterday, I suppose I won't have much to do. I do get involved in a discussion about some drive-supporting PANCAM imaging. Blue-stereo images usually give us the sharpest picture, but Elaina McCartney points out that the images will be taken at a particularly warm time of day, and consequently the blue-stereo images will have speckling that might interfere with the analysis we're trying to do. So we switch to the next-best thing, red-stereo. But she also generously volunteers to take a blue-stereo pair as well, just so we can see what the images would look like, in case they might prove useful after all. Despite the large data volume required for this, the SOWG chair, Ed Guinness, shrugs acceptance. "Damn," I say, "it's almost like we're a team here or something."
My easy, laid-back day turns out to be a lot busier than I anticipated. Frank and Jeng go down to the testbed to start testing out the next sequence, and Khaled and I have more work to do on thisol's sequence than we'd thought. Nothing major, just lots of picky little things. Working out the final image pointing. Mucking with the visodom image parameters so we'll get more data. Going over the timing with the TULs. That sort of thing.
Indeed, when lunchtime approaches, we're both too busy to leave. "We need a gofer," I say, and Khaled agrees wholeheartedly, or rather wholestomachedly.
But we're ready in time for all the usual meetings. (I even show an animation at the APAM: the rover sits still for five seconds, then turns the wheels. Woo-hoo. I attach the animation to the uplink report.)
When it's all said and done, not only all four RPs who were there at the SOWG but also Jeff Biesiadecki and Mark Maimone have reviewed it, along with many others, such as Rick Welch. If it takes all this just to steer the wheels now, I don't want to see what happens when we try to drive. It's a far cry from the days when sequencing Opportunity meant: well, we expanded the über-macro, there's your 200m drive -- see ya!
2010-05-10
Opportunity Sol 460 (Spirit Sol 480)
Yikes. Cooper had an emergency appendectomy this weekend. He's consequently unable to come to work, so I volunteer to fill in for him. This puts me back on Opportunity for the first time in quite a while -- today, and for the next couple of days.
The Opportunity team has just about decided to drive out of Purgatory. They haven't even tried moving yet, but testbed testing has been encouraging. They've been able to simulate most of the important conditions of the real vehicle, and they keep managing to drive it out.
Indeed, we're back in the testbed again today. Jeng and I whip up a candidate sequence, whose only mobility command is to steer the wheels in preparation for the first stage of the drive. This single command is surrounded (fore and aft) by a zillion imaging commands. We're taking this very, very slowly.
Then we walk down to the testbed, where we meet up with Frank and Mark. We won't be driving the testbed rover in the special soil they cooked up to simulate Opportunity's current site, due to medical concerns: the fine particulates are dangerous to breathe for long periods, and we haven't been formally trained on the respirators. (I don't honestly see what's to know. It's a face mask. You strap it on. But their intent is to keep us healthy, and I can't find too much fault with that. They're insane, but they mean well.[1])
As usual, it just takes forever to get anything done. A couple of hours go by, and we haven't run the sequence yet. Both Frank's and my ESD certifications have lapsed, so we can't man the kill switch anyway, even once the rover gets going. We mostly stand there chit-chatting -- about the rovers, about LISP, about camera phones, about whatever. An employee I've never met brings a couple of guests by, and I go over and talk with them for a while about Opportunity's current status and what we're working on. "That's the second time I've seen you do that," Frank says after they depart. "You're like a rover ambassador."[2]
Then, just before things really get going, I have to leave. I'm working the Open House this weekend, and they have an orientation meeting this afternoon. Which is mostly useless, but one of the speakers makes a good point: "Not everyone has the privilege of working somewhere where everyone wants to go there and see what they do." True, and that's why I'm doing the Open House: for the sake of all those poor slobs who want to be me.
But seriously, I am impressed with the scale of the effort. As another speaker puts it, "This place was not set up to be Disneyland." But we turn it into Disneyland for a couple of days a year, and that's pretty darn cool, really.[3] I'm only too happy to be a part of it again. (Also, she mentions offhandedly that they used to have an open house on the last Sunday of every month! I can't imagine it was anything like the scale of the present undertaking, but even so, that must have been a very different time.)
I make it back to the testbed just after the sequence finishes running, which pretty much officially means I wasted my entire day. We then have a long conference call with the rest of the uplink team, to set up for tomorrow. They'll be doing additional testing on this sequence tomorrow in parallel with the uplink process, but it's clear that at least some people think we're jumping through a ridiculous number of hoops: as Jeff Biesiadecki puts it, "The rover would have to catch on fire for us not to send this sequence."[4]
So I guess we're going ahead with that, then.
I go back to my office and review the sequence once again, and I do catch a couple of minor errors -- one of which would have prevented Opportunity from moving at all. So I guess it wasn't a complete waste after all.
Footnotes:
[1] I eventually had the face-mask training, as part of the Spirit extrication effort. Turns out it's a face mask, and you strap it on.
OK, so there's a little more to it than that. But not much.
[2] To this day, that was the nicest way anyone's ever called me an attention whore.
[3] This is probably why I started calling it "Deep Space Disneyland."
[4] What sequence do we send if that happens?!
The Opportunity team has just about decided to drive out of Purgatory. They haven't even tried moving yet, but testbed testing has been encouraging. They've been able to simulate most of the important conditions of the real vehicle, and they keep managing to drive it out.
Indeed, we're back in the testbed again today. Jeng and I whip up a candidate sequence, whose only mobility command is to steer the wheels in preparation for the first stage of the drive. This single command is surrounded (fore and aft) by a zillion imaging commands. We're taking this very, very slowly.
Then we walk down to the testbed, where we meet up with Frank and Mark. We won't be driving the testbed rover in the special soil they cooked up to simulate Opportunity's current site, due to medical concerns: the fine particulates are dangerous to breathe for long periods, and we haven't been formally trained on the respirators. (I don't honestly see what's to know. It's a face mask. You strap it on. But their intent is to keep us healthy, and I can't find too much fault with that. They're insane, but they mean well.[1])
As usual, it just takes forever to get anything done. A couple of hours go by, and we haven't run the sequence yet. Both Frank's and my ESD certifications have lapsed, so we can't man the kill switch anyway, even once the rover gets going. We mostly stand there chit-chatting -- about the rovers, about LISP, about camera phones, about whatever. An employee I've never met brings a couple of guests by, and I go over and talk with them for a while about Opportunity's current status and what we're working on. "That's the second time I've seen you do that," Frank says after they depart. "You're like a rover ambassador."[2]
Then, just before things really get going, I have to leave. I'm working the Open House this weekend, and they have an orientation meeting this afternoon. Which is mostly useless, but one of the speakers makes a good point: "Not everyone has the privilege of working somewhere where everyone wants to go there and see what they do." True, and that's why I'm doing the Open House: for the sake of all those poor slobs who want to be me.
But seriously, I am impressed with the scale of the effort. As another speaker puts it, "This place was not set up to be Disneyland." But we turn it into Disneyland for a couple of days a year, and that's pretty darn cool, really.[3] I'm only too happy to be a part of it again. (Also, she mentions offhandedly that they used to have an open house on the last Sunday of every month! I can't imagine it was anything like the scale of the present undertaking, but even so, that must have been a very different time.)
I make it back to the testbed just after the sequence finishes running, which pretty much officially means I wasted my entire day. We then have a long conference call with the rest of the uplink team, to set up for tomorrow. They'll be doing additional testing on this sequence tomorrow in parallel with the uplink process, but it's clear that at least some people think we're jumping through a ridiculous number of hoops: as Jeff Biesiadecki puts it, "The rover would have to catch on fire for us not to send this sequence."[4]
So I guess we're going ahead with that, then.
I go back to my office and review the sequence once again, and I do catch a couple of minor errors -- one of which would have prevented Opportunity from moving at all. So I guess it wasn't a complete waste after all.
Footnotes:
[1] I eventually had the face-mask training, as part of the Spirit extrication effort. Turns out it's a face mask, and you strap it on.
OK, so there's a little more to it than that. But not much.
[2] To this day, that was the nicest way anyone's ever called me an attention whore.
[3] This is probably why I started calling it "Deep Space Disneyland."
[4] What sequence do we send if that happens?!
2010-05-06
Spirit Sol 477
Yestersol's drive was too complex. I was thinking about it last night, and my hopes sank lower and lower. We should have broken it into multiple sols. We shouldn't have tried to thread the needle at the end. I should have looked more carefully for obstacles.
Of course, it went splendidly. (If I learned anything from experience, I'd have been expecting this. But I never do.) Dan Moyers tells me the good news as I walk into the SOWG meeting. "Excellent!" I reply. "I was all ready to blame Chris if it went wrong."
As has become our Friday custom, we're planning the rover's activities for the whole weekend. There's lots of IDD work, and lots of remote sensing. (In keeping with the nautical theme of Jib Sheet, most of the remote-sensing targets get names such as "gunwale," "keel," etc. They're taking some pictures of the tracks as well, and Charles Budney suggests the name "wake" for that target.) We have a limited time -- not only are we supposed to stick to a 10-hour day anyway, but uplink's at 19:00, which pretty much enforces that limit thisol. As a result of this tight schedule, Dan rejects a request from the atmospheres team -- it'll just be too hard to fit it in in the time we've got, given the complexity of the overall plan.
As you can imagine, the atmospheres team isn't too happy about that, and they push back hard. But Dan sticks to his guns, and Emily backs him up. "The resource we need to optimize here is team planning time," she says. One of the scientists turns to me off mike and says, "So the engineering team is no longer competent to --" but he's interrupted, and he never finishes the thought. Just as well.
In the room behind us, the Opportunity team is having its own debate, joined by some higher management. Opportunity's been stuck on a dune for the last few weeks, buried up to the tops of the wheels -- almost up to the bottom of the WEB -- and management's been paranoid about allowing us to drive it out. I notice that Jay Torres was sitting by a window that afforded a view into that room, so I ask him if he saw any of the presentation. He shakes his head no, as Chris chimes in, "The content is, we can be a lander or we can drive backward a couple of meters."
"Which we knew three weeks ago," I say.
"Yeah," he replies, "but now our ass is covered."
Yeah, that probably sums it up.
Back downstairs, Emily asks our new TDL what he thought of the acrimony in the SOWG meeting. "I can see why they keep the scientists and engineers in physically separate locations," he says diplomatically.
"Well, it used to be better," Emily says. "But the vehicles have gotten harder to operate. We used to have no power, so we had no concerns about thermal issues or planning time -- there was no power to do anything. But now we're power-rich again, and we'd forgotten there were other constraints."
This is true, but I think it's incomplete. Another part of the problem is that the scientists just aren't here any more. They're voices on a phone, and that makes it easier for both sides to degenerate into an us-versus-them mentality. Worse, they simply don't get to see the whole process any more -- most of them sign off when they're done with their part of the planning, but we keep working for hours. If they can't see what effect their decisions have on the sequencing team, how can they hope to make wise decisions about what can and can't get done? I point this out to Emily, and semi-jokingly suggest a solution: "We should get Squyres to require all of the science team to stay online until we're done. You want us to do some observation that's going to take us two extra hours to fit into the plan? Fine, but you stay here until we leave."
I think Emily likes the idea, but I doubt she'll pitch it to Steve. I might. If the problem gets worse, I might have to.
But at least the rover planners have an easy sol. It's just some light IDD work. Chris is done with the sequences maybe half an hour after the SOWG meeting ends. We hand off and I make some minor changes, but nothing particularly urgent.
Since we're so far ahead of the game, I consider going ahead with the time-consuming process of building an MPEG movie of the planned IDD work for the weekend, which I customarily attach to the sol's uplink report. But when we're this far ahead, something always happens to change the plan, and today's no exception. When we show the animation at the activity plan approval meeting, we find out that the science team changed its mind about which IDD targets they wanted us to use, and somehow the word didn't make it to us. No big deal; that's why we do that. I have it fixed within 15 minutes or so.
But I was right not to go ahead and build the movie early. Huh. Seems like sometimes I do learn from experience, after all.
[Next post: sol 480 (Opportunity sol 460), May 10.]
Of course, it went splendidly. (If I learned anything from experience, I'd have been expecting this. But I never do.) Dan Moyers tells me the good news as I walk into the SOWG meeting. "Excellent!" I reply. "I was all ready to blame Chris if it went wrong."
As has become our Friday custom, we're planning the rover's activities for the whole weekend. There's lots of IDD work, and lots of remote sensing. (In keeping with the nautical theme of Jib Sheet, most of the remote-sensing targets get names such as "gunwale," "keel," etc. They're taking some pictures of the tracks as well, and Charles Budney suggests the name "wake" for that target.) We have a limited time -- not only are we supposed to stick to a 10-hour day anyway, but uplink's at 19:00, which pretty much enforces that limit thisol. As a result of this tight schedule, Dan rejects a request from the atmospheres team -- it'll just be too hard to fit it in in the time we've got, given the complexity of the overall plan.
As you can imagine, the atmospheres team isn't too happy about that, and they push back hard. But Dan sticks to his guns, and Emily backs him up. "The resource we need to optimize here is team planning time," she says. One of the scientists turns to me off mike and says, "So the engineering team is no longer competent to --" but he's interrupted, and he never finishes the thought. Just as well.
In the room behind us, the Opportunity team is having its own debate, joined by some higher management. Opportunity's been stuck on a dune for the last few weeks, buried up to the tops of the wheels -- almost up to the bottom of the WEB -- and management's been paranoid about allowing us to drive it out. I notice that Jay Torres was sitting by a window that afforded a view into that room, so I ask him if he saw any of the presentation. He shakes his head no, as Chris chimes in, "The content is, we can be a lander or we can drive backward a couple of meters."
"Which we knew three weeks ago," I say.
"Yeah," he replies, "but now our ass is covered."
Yeah, that probably sums it up.
Back downstairs, Emily asks our new TDL what he thought of the acrimony in the SOWG meeting. "I can see why they keep the scientists and engineers in physically separate locations," he says diplomatically.
"Well, it used to be better," Emily says. "But the vehicles have gotten harder to operate. We used to have no power, so we had no concerns about thermal issues or planning time -- there was no power to do anything. But now we're power-rich again, and we'd forgotten there were other constraints."
This is true, but I think it's incomplete. Another part of the problem is that the scientists just aren't here any more. They're voices on a phone, and that makes it easier for both sides to degenerate into an us-versus-them mentality. Worse, they simply don't get to see the whole process any more -- most of them sign off when they're done with their part of the planning, but we keep working for hours. If they can't see what effect their decisions have on the sequencing team, how can they hope to make wise decisions about what can and can't get done? I point this out to Emily, and semi-jokingly suggest a solution: "We should get Squyres to require all of the science team to stay online until we're done. You want us to do some observation that's going to take us two extra hours to fit into the plan? Fine, but you stay here until we leave."
I think Emily likes the idea, but I doubt she'll pitch it to Steve. I might. If the problem gets worse, I might have to.
But at least the rover planners have an easy sol. It's just some light IDD work. Chris is done with the sequences maybe half an hour after the SOWG meeting ends. We hand off and I make some minor changes, but nothing particularly urgent.
Since we're so far ahead of the game, I consider going ahead with the time-consuming process of building an MPEG movie of the planned IDD work for the weekend, which I customarily attach to the sol's uplink report. But when we're this far ahead, something always happens to change the plan, and today's no exception. When we show the animation at the activity plan approval meeting, we find out that the science team changed its mind about which IDD targets they wanted us to use, and somehow the word didn't make it to us. No big deal; that's why we do that. I have it fixed within 15 minutes or so.
But I was right not to go ahead and build the movie early. Huh. Seems like sometimes I do learn from experience, after all.
[Next post: sol 480 (Opportunity sol 460), May 10.]
2010-05-05
Spirit Sol 476
As we work our way back down the ridge to Larry's Lookout, our next stop is Jib Sheet. Thisol we're sequencing a drive to take us most of the way there; wherever we end up, we'll IDD over the weekend, and then complete the Jib Sheet approach early next week.
It's a complex drive. To get there, we'll need to drive in a big "U." It's a total distance of about 25m, a much longer drive than we're normally able to accomplish in this challenging terrain. In our favor, the terrain does flatten out considerably as we go, which will make the vehicle's performance more predictable. But the length of the drive and the nature of the overall terrain will make this a long and difficult plan.
We're not sure for a while exactly where we want the rover to end up, or how it will get there, or even whether we'll drive forward or backward. Dan Moyers sees this and decides to have a little fun with TUL-in-training Rich. "We need the final heading so we can do comm predicts," he says. "Ask the rover planners what our final heading will be."
But Rich is on the ball. "Dan wants to know our final heading."
"Sixty," Chris says without turning around. "Plus or minus 180."
It's mostly Chris's show today. I do the stow and work out the best plan for the first couple of meters of the drive, just enough to get us on the road. Chris does most of the work of planning out the route, though I throw in my opinion now and then. For instance, he's hoping to take a shortcut between two larger rocks, letting autonav find a safe path. I do some analysis to work out that the rover won't be able to find a safe path without loosening the autonav settings, and neither one of us wants to do that. So we take the long path.
Our accidental IDD animation keeps turning up new surprises. I composed and posted the full two-sol version of the animation the other day, and Brenda Franklin's been looking over it carefully. Her scrutiny was rewarded with a neat surprise: in the second-to-last frame on the second sol, a huge gust of wind -- maybe a dust devil -- kicks up a huge cloud of dust on a far hill. "Definitely lemonade with whiskey," she says.
(I've had trouble with animated GIFs and Blogger. You should be able to see the animation at twitpic.)
Courtesy NASA/JPL-Caltech. The dust devil is in the upper-left corner, near the horizon. Subtle, but it's there.
It's a complex drive. To get there, we'll need to drive in a big "U." It's a total distance of about 25m, a much longer drive than we're normally able to accomplish in this challenging terrain. In our favor, the terrain does flatten out considerably as we go, which will make the vehicle's performance more predictable. But the length of the drive and the nature of the overall terrain will make this a long and difficult plan.
We're not sure for a while exactly where we want the rover to end up, or how it will get there, or even whether we'll drive forward or backward. Dan Moyers sees this and decides to have a little fun with TUL-in-training Rich. "We need the final heading so we can do comm predicts," he says. "Ask the rover planners what our final heading will be."
But Rich is on the ball. "Dan wants to know our final heading."
"Sixty," Chris says without turning around. "Plus or minus 180."
It's mostly Chris's show today. I do the stow and work out the best plan for the first couple of meters of the drive, just enough to get us on the road. Chris does most of the work of planning out the route, though I throw in my opinion now and then. For instance, he's hoping to take a shortcut between two larger rocks, letting autonav find a safe path. I do some analysis to work out that the rover won't be able to find a safe path without loosening the autonav settings, and neither one of us wants to do that. So we take the long path.
Our accidental IDD animation keeps turning up new surprises. I composed and posted the full two-sol version of the animation the other day, and Brenda Franklin's been looking over it carefully. Her scrutiny was rewarded with a neat surprise: in the second-to-last frame on the second sol, a huge gust of wind -- maybe a dust devil -- kicks up a huge cloud of dust on a far hill. "Definitely lemonade with whiskey," she says.
(I've had trouble with animated GIFs and Blogger. You should be able to see the animation at twitpic.)
Courtesy NASA/JPL-Caltech. The dust devil is in the upper-left corner, near the horizon. Subtle, but it's there.
2010-04-30
Spirit Sol 471
It's a commonplace among engineers that you do your best work in the shower. Your mind and body are relaxed, and you see solutions that you just don't see under other circumstances.
That's not to say you always like the answers you get. While I'm getting ready in the morning, I realize we completely screwed up the huge MI mosaic of Keystone. It's the largest MI mosaic of a single object ever; it took about three hours for the IDD to do the work; and I single-handedly managed to get nearly all of the images out of focus. I've got my eyes closed, I'm putting on the shampoo, and suddenly I visualize the relationship between the rock face and the motion of the IDD as it moves the MI across the surface, and it becomes blindingly clear that the angle is completely wrong for the majority of the surface.
See, the MI has a very shallow field of view. You can think of it as being like a small square sheet of paper, held about 4.6cm (call it two inches) in front of the lens. The "piece of paper" is maybe 3cm on a side and 3mm thick -- maybe instead of a single sheet of paper, you should think of it as a pad of just a few Post-It notes. Anything not within that volume will be either invisible or unfocused. If you look at a surface at too oblique an angle, most of the "notepad" will be either above or below that surface. A small amount of the image will be in focus, but most won't.
We held the paper at the wrong angle.
Some small fraction of the MIs will be in focus, but most won't. We basically just blew two or three sols, four or five hours of IDD actuator life, and hundreds of megabits of downlink. And it's all my fault.
I get to JPL a little early, anxious to know how bad the situation is. Nobody says anything in particular to me, which is usually a bad sign. I fire up mphoto (the image browser application I wrote during solar conjunction), bring up the MI thumbnails, and click on one of them. It happens to be one of the in-focus images, and looks pretty good. Then I click on another one. It's in focus, too. I click on a few more. They're all in focus.
Um. Evidently, I was mistaken. Happily, it would seem that I do better IDD work in the sequencing room than in the shower after all.
And I realize I'm now the holder of another record (not that any of us holds these individually, of course; it's always a team effort). Not only am I the single-sol distance-record holder on both sides of Mars, I've now also done the most MIs of a single object ever.
And they're even in focus!
Sadly, the MI sky flats don't work out again -- there's no time to do them in the weekend plan. When they officially sink the idea, I hit the mike and say, "Screw you guys, I'm doing it anyway!" Which provokes a laugh, and Ken says, "Go, Scott!" But he's philosophical about the lost chance: "I waited 400 sols, I can wait a little longer."
I did make one mistake in the MI sequence. We were performing the MI mosaic in six columns of four stacks apiece, covering the broad rectangular face of Keystone. Now, normally, we take an image of the IDD each time we place an instrument, taking higher-quality images when we place the MI to give us data points for calibrating the HAZCAMs. (When the MI is down, the MB is up, right where the cameras can see it, and its shiny contact ring is easy to pick out in the images.) But we didn't want to spend time or downlink volume taking those images for every stack, since there were so many. Instead, we want to image the IDD only once per column.
So I came up with a clever way to do that. When we're driving, we sometimes use the OK-to-IDD flag as a simple marker -- for instance, if we get to a certain point in the drive, we drop the OK-to-IDD flag. Then later on, the sequence checks if it's OK to IDD; if not, we know we got that far in the sequence and can do something special for that case. So I figured, why not turn that around? In the backbone sequence, we'll drop the OK-to-drive flag as we start each column, then make four calls to the helper sequence (each call to the helper sequence performs one stack and moves to set up for the next stack). The helper sequence will check whether it's OK to drive and, if not, will take the IDD image and then re-raise the flag so that the next call to the helper sequence won't take the image. That way, we'll take the image for only the first stack in each quartet.
Like so many solutions, this one is simple, elegant, and wrong. What I stupidly forgot was that when the IDD is unstowed, we're never OK to drive. So instead of imaging the IDD placement on every fourth call to the helper, we imaged it on every call -- sixteen IDD placement images, not four. And we reused that code when we completed the mosaic, so we're going to repeat the mistake. Oops.
So we got a whole bunch of images of the IDD moving across the surface. High-quality images. Hey ... on a whim, I make a simple animation and project it on the shared screen in the sequencing room. It's awesome -- not only do you see the IDD doing its thing, you even see the rover's own shadow marching away as the sun passes overhead. Steve sees the movie and loves it. "Maybe we should do more things like this," he remarks.
"Yeah, if only all our mistakes turned out this well," I reply.
Brenda Franklin also pipes up. "When life gives you lemons, make lemonade. This is lemonade. This is lemonade with whiskey!"
"Hey, you know what you should do?" Steve says. "You should send this to Guy Webster!" (That's JPL's main press dude for MER.) "When we release the MI mosaic, we can release this movie to go along with it."
Which is a great idea, and displays Steve's flair for outreach. So I put together a version of the movie for release -- actually, two versions, one from each eye -- and email the press people about it. We'll see what happens.[1]
I finish up the sequencing and hand over to Chris, then hang out to work on some other stuff. A little later, Steve comes on the teleconference line again. This time he's raving about the MIs. You can practically hear him jumping up and down with excitement. "Those MIs just look great!" he exclaims. "This is gonna be one of the premier data sets of the mission. I just talked to Ken, and they're going crazy in Flagstaff!" (This is not some strange expression; Flagstaff -- the University of Arizona -- is where Ken and his team are.)
I couldn't be happier, especially since this is the exact diametrical polar opposite of what I expected to hear when I came in today. I tell Steve I'll pass his message along to Chris and Ashitey, and I do.
Steve's got one more thing to mention. He was unable to cover the SOWG meeting this morning because he was teaching a class -- "Astrophysics 102 -- 300 students. At the start of every class, I give them a five-minute update on what's going on on Mars. It's the only part they pay attention for."
[Next post: sol 476, May 5.]
Courtesy NASA/JPL-Caltech. An accidental animation of the first sol's work on Keystone: best mistake ever. Follow the link for a full-sized version.
Courtesy NASA/JPL-Caltech. The mosaic of MI images on Keystone. Again, follow the link to get a full-sized version.
[1] As always, Steve's instincts were spot on. They loved it. They released the animation at the same time as the Keystone press release, and it's shown up more places than I can count since then. Best mistake ever.
That's not to say you always like the answers you get. While I'm getting ready in the morning, I realize we completely screwed up the huge MI mosaic of Keystone. It's the largest MI mosaic of a single object ever; it took about three hours for the IDD to do the work; and I single-handedly managed to get nearly all of the images out of focus. I've got my eyes closed, I'm putting on the shampoo, and suddenly I visualize the relationship between the rock face and the motion of the IDD as it moves the MI across the surface, and it becomes blindingly clear that the angle is completely wrong for the majority of the surface.
See, the MI has a very shallow field of view. You can think of it as being like a small square sheet of paper, held about 4.6cm (call it two inches) in front of the lens. The "piece of paper" is maybe 3cm on a side and 3mm thick -- maybe instead of a single sheet of paper, you should think of it as a pad of just a few Post-It notes. Anything not within that volume will be either invisible or unfocused. If you look at a surface at too oblique an angle, most of the "notepad" will be either above or below that surface. A small amount of the image will be in focus, but most won't.
We held the paper at the wrong angle.
Some small fraction of the MIs will be in focus, but most won't. We basically just blew two or three sols, four or five hours of IDD actuator life, and hundreds of megabits of downlink. And it's all my fault.
I get to JPL a little early, anxious to know how bad the situation is. Nobody says anything in particular to me, which is usually a bad sign. I fire up mphoto (the image browser application I wrote during solar conjunction), bring up the MI thumbnails, and click on one of them. It happens to be one of the in-focus images, and looks pretty good. Then I click on another one. It's in focus, too. I click on a few more. They're all in focus.
Um. Evidently, I was mistaken. Happily, it would seem that I do better IDD work in the sequencing room than in the shower after all.
And I realize I'm now the holder of another record (not that any of us holds these individually, of course; it's always a team effort). Not only am I the single-sol distance-record holder on both sides of Mars, I've now also done the most MIs of a single object ever.
And they're even in focus!
Sadly, the MI sky flats don't work out again -- there's no time to do them in the weekend plan. When they officially sink the idea, I hit the mike and say, "Screw you guys, I'm doing it anyway!" Which provokes a laugh, and Ken says, "Go, Scott!" But he's philosophical about the lost chance: "I waited 400 sols, I can wait a little longer."
I did make one mistake in the MI sequence. We were performing the MI mosaic in six columns of four stacks apiece, covering the broad rectangular face of Keystone. Now, normally, we take an image of the IDD each time we place an instrument, taking higher-quality images when we place the MI to give us data points for calibrating the HAZCAMs. (When the MI is down, the MB is up, right where the cameras can see it, and its shiny contact ring is easy to pick out in the images.) But we didn't want to spend time or downlink volume taking those images for every stack, since there were so many. Instead, we want to image the IDD only once per column.
So I came up with a clever way to do that. When we're driving, we sometimes use the OK-to-IDD flag as a simple marker -- for instance, if we get to a certain point in the drive, we drop the OK-to-IDD flag. Then later on, the sequence checks if it's OK to IDD; if not, we know we got that far in the sequence and can do something special for that case. So I figured, why not turn that around? In the backbone sequence, we'll drop the OK-to-drive flag as we start each column, then make four calls to the helper sequence (each call to the helper sequence performs one stack and moves to set up for the next stack). The helper sequence will check whether it's OK to drive and, if not, will take the IDD image and then re-raise the flag so that the next call to the helper sequence won't take the image. That way, we'll take the image for only the first stack in each quartet.
Like so many solutions, this one is simple, elegant, and wrong. What I stupidly forgot was that when the IDD is unstowed, we're never OK to drive. So instead of imaging the IDD placement on every fourth call to the helper, we imaged it on every call -- sixteen IDD placement images, not four. And we reused that code when we completed the mosaic, so we're going to repeat the mistake. Oops.
So we got a whole bunch of images of the IDD moving across the surface. High-quality images. Hey ... on a whim, I make a simple animation and project it on the shared screen in the sequencing room. It's awesome -- not only do you see the IDD doing its thing, you even see the rover's own shadow marching away as the sun passes overhead. Steve sees the movie and loves it. "Maybe we should do more things like this," he remarks.
"Yeah, if only all our mistakes turned out this well," I reply.
Brenda Franklin also pipes up. "When life gives you lemons, make lemonade. This is lemonade. This is lemonade with whiskey!"
"Hey, you know what you should do?" Steve says. "You should send this to Guy Webster!" (That's JPL's main press dude for MER.) "When we release the MI mosaic, we can release this movie to go along with it."
Which is a great idea, and displays Steve's flair for outreach. So I put together a version of the movie for release -- actually, two versions, one from each eye -- and email the press people about it. We'll see what happens.[1]
I finish up the sequencing and hand over to Chris, then hang out to work on some other stuff. A little later, Steve comes on the teleconference line again. This time he's raving about the MIs. You can practically hear him jumping up and down with excitement. "Those MIs just look great!" he exclaims. "This is gonna be one of the premier data sets of the mission. I just talked to Ken, and they're going crazy in Flagstaff!" (This is not some strange expression; Flagstaff -- the University of Arizona -- is where Ken and his team are.)
I couldn't be happier, especially since this is the exact diametrical polar opposite of what I expected to hear when I came in today. I tell Steve I'll pass his message along to Chris and Ashitey, and I do.
Steve's got one more thing to mention. He was unable to cover the SOWG meeting this morning because he was teaching a class -- "Astrophysics 102 -- 300 students. At the start of every class, I give them a five-minute update on what's going on on Mars. It's the only part they pay attention for."
[Next post: sol 476, May 5.]
Courtesy NASA/JPL-Caltech. An accidental animation of the first sol's work on Keystone: best mistake ever. Follow the link for a full-sized version.
Courtesy NASA/JPL-Caltech. The mosaic of MI images on Keystone. Again, follow the link to get a full-sized version.
[1] As always, Steve's instincts were spot on. They loved it. They released the animation at the same time as the Keystone press release, and it's shown up more places than I can count since then. Best mistake ever.
2010-04-29
Spirit Sol 470
We're in that weird phase where we're planning in the blind. The sequences we wrote yestersol haven't even been uplinked to the rover yet, much less executed, so clearly we haven't gotten the downlink yet. And today's are going to look similar to yesterday's, so we could blow two whole days if we did something wrong yesterday. So I ask Chris -- RP-2 today -- to look them over. In theory, if he finds a problem, we could fix it and uplink the result instead of what we planned yesterday. Not that our mission manager would let us get away with that. But Chris doesn't find anything anyway, so it's moot. At least I'll be able to sleep a little better.
In theory.
Yesterday we did a 4x4 subset of the total 4x6 mosaic. Today we pick up where we left off, doing the remaining 2x6 subset of the mosaic. Then we brush off a spot with the RAT and place the APXS there. Overall, not such a bad day -- yesterday was harder.
The hard part is to fit in the MI sky flats. This is where we take microscopic images of the sky. Yes, you read that right. It's a calibration thing. The problem is that there are restrictions on how we take the sky flats, and it's awkward to satisfy them all at once. First, the MI must be aimed at least 90 degrees away from the sun. Second, the sky-flat images must be taken late in the day, after about 14:00. Third, the MI needs an unobstructed view of the sky, so no part of the rover body or the terrain can be in the way. Fourth, it must be aimed about 40 degrees above the horizon.
The first and second constraints combine to force the MI to aim to the rover's right. So now the problem is to find a solution where the MI is aimed to the right, 40 degrees above the horizon, without any terrain or rover body in view. There doesn't seem to be any way to do this. In one wrist configuration, the rover deck blocks our view when the MI is close to the WEB and the IDD self-collides when the MI is pushed farther from the WEB. In the other wrist configuration, we can choose between having the rover deck in the way or running into a hardstop -- with the arm otherwise in the needed pose, the 40-degree elevation is in a "blind spot" where the turret can't aim the MI.
This means we can't do the sky flats, which makes me feel very bad. The MI PUL, Ken, is such a nice guy, and it's been literally 400 sols since he's had this calibration done. Then at last they make room for it in the plan, and I can't find a solution for him. It's not my fault that the solution's not there to be found, but I still feel bad about it.
But otherwise, the sol goes well. We're practically drunk on power -- 937 Watt-hours thisol, more than we had when we landed. Thanks, dust devils! And the sequencing proceeds without any major hiccups.
At the end of the day, Steve fills us in on the science team's current thinking about where we want to go from here. First, images taken from the other side -- that is, the current side -- of Larry's Lookout (a landmark partway up Husband Hill, which we passed on the way up) have apparently made the scientists even more interested in it, so we're going to head back downhill to it. Downhill is the wrong direction, I want to say (damn, I want to see the view from that summit!) -- but I'm just the chauffeur, so I wisely keep my mouth shut. The right direction is wherever the science is, and if that's downhill from here, then so be it.
On the way back down to Larry's Lookout, we'll check out another rock near this one, then this area called Jib Sheet. We probably won't get quite close enough to Larry's Lookout to touch it, since there's a steep pile of soft sand in the way, but we'll take highly detailed pictures from as close as we can. (Like Burns's Cliff, Steve points out.) Then we turn back around and start slogging up the hill again.
"Sorry about that," Steve says. "This is the best exposed geology we'll see this side of Husband Hill, so we gotta do it."
That's okay, it really is. The big disappointment from today comes from an unexpected direction, when Chris finds a way we could have done the MI sky flats after all. It's a configuration I thought I'd tried, but obviously not. So now I feel twice as bad about not finding a way to do it, because there was a way to be found. I just didn't find it.
(And writing this, it occurs to me there was always a way anyhow. The IDD's elbow joint is fully double-jointed; we could simply have swung the turret up so that it was above the elbow rather than below it. Then aiming the MI for the sky flats would have been trivial -- the rover deck is never in the way in that pose. Oh, I am dumb sometimes, I truly am.)
I swallow my pride and send Ken and Steve email about this. We couldn't do the sky flats today, but we'll still be here over the weekend. Maybe we'll squeeze it in tomorrow.
In theory.
Yesterday we did a 4x4 subset of the total 4x6 mosaic. Today we pick up where we left off, doing the remaining 2x6 subset of the mosaic. Then we brush off a spot with the RAT and place the APXS there. Overall, not such a bad day -- yesterday was harder.
The hard part is to fit in the MI sky flats. This is where we take microscopic images of the sky. Yes, you read that right. It's a calibration thing. The problem is that there are restrictions on how we take the sky flats, and it's awkward to satisfy them all at once. First, the MI must be aimed at least 90 degrees away from the sun. Second, the sky-flat images must be taken late in the day, after about 14:00. Third, the MI needs an unobstructed view of the sky, so no part of the rover body or the terrain can be in the way. Fourth, it must be aimed about 40 degrees above the horizon.
The first and second constraints combine to force the MI to aim to the rover's right. So now the problem is to find a solution where the MI is aimed to the right, 40 degrees above the horizon, without any terrain or rover body in view. There doesn't seem to be any way to do this. In one wrist configuration, the rover deck blocks our view when the MI is close to the WEB and the IDD self-collides when the MI is pushed farther from the WEB. In the other wrist configuration, we can choose between having the rover deck in the way or running into a hardstop -- with the arm otherwise in the needed pose, the 40-degree elevation is in a "blind spot" where the turret can't aim the MI.
This means we can't do the sky flats, which makes me feel very bad. The MI PUL, Ken, is such a nice guy, and it's been literally 400 sols since he's had this calibration done. Then at last they make room for it in the plan, and I can't find a solution for him. It's not my fault that the solution's not there to be found, but I still feel bad about it.
But otherwise, the sol goes well. We're practically drunk on power -- 937 Watt-hours thisol, more than we had when we landed. Thanks, dust devils! And the sequencing proceeds without any major hiccups.
At the end of the day, Steve fills us in on the science team's current thinking about where we want to go from here. First, images taken from the other side -- that is, the current side -- of Larry's Lookout (a landmark partway up Husband Hill, which we passed on the way up) have apparently made the scientists even more interested in it, so we're going to head back downhill to it. Downhill is the wrong direction, I want to say (damn, I want to see the view from that summit!) -- but I'm just the chauffeur, so I wisely keep my mouth shut. The right direction is wherever the science is, and if that's downhill from here, then so be it.
On the way back down to Larry's Lookout, we'll check out another rock near this one, then this area called Jib Sheet. We probably won't get quite close enough to Larry's Lookout to touch it, since there's a steep pile of soft sand in the way, but we'll take highly detailed pictures from as close as we can. (Like Burns's Cliff, Steve points out.) Then we turn back around and start slogging up the hill again.
"Sorry about that," Steve says. "This is the best exposed geology we'll see this side of Husband Hill, so we gotta do it."
That's okay, it really is. The big disappointment from today comes from an unexpected direction, when Chris finds a way we could have done the MI sky flats after all. It's a configuration I thought I'd tried, but obviously not. So now I feel twice as bad about not finding a way to do it, because there was a way to be found. I just didn't find it.
(And writing this, it occurs to me there was always a way anyhow. The IDD's elbow joint is fully double-jointed; we could simply have swung the turret up so that it was above the elbow rather than below it. Then aiming the MI for the sky flats would have been trivial -- the rover deck is never in the way in that pose. Oh, I am dumb sometimes, I truly am.)
I swallow my pride and send Ken and Steve email about this. We couldn't do the sky flats today, but we'll still be here over the weekend. Maybe we'll squeeze it in tomorrow.
2010-04-28
Spirit Sol 469
It's been nearly two weeks since I was last on shift. The drive we planned last time I was on shift went splendidly, putting us just where we wanted. Once there, as we started climbing up the long way around, the scientists started finding more stuff they just had to take a look at. So Spirit spent a while investigating some crumbly stuff called Methuselah, and now we've just driven up to a rock called Keystone, which they're practically salivating over. (And I'm not the only one who thinks so: "Steve's like a kid in a candy store," Dan Moyers remarks to me during the SOWG meeting.)
Thisol we're starting to IDD Keystone. Just to make sure I don't get bored, this is going to be the biggest MI mosaic of a single object ever, a total of 6x4x5+stereo = 144 images total. Our approach is to sequence as many of them today as the rover will have time to execute, and finish the rest tomorrow.
Before we can even really get started, we have to figure out what we want to image. This is troublesome, because we first have to know the angle of the MI with respect to the roughly rectangular rock surface -- that affects how the image footprints are oriented, which affects how you lay out the mosaic -- and none of our tools does that very well. So we basically just eyeball it, and it turns out it's reasonably close to the same orientation as the overall rectangle of rock itself, which simplifies things.
For the next step in the process, Ashitey and I (mostly Ashitey) pick a candidate center for the mosaic and use an image processing program to sketch out what the mosaic will cover with that as its center. Drawing the rectangle is easy, but figuring out where it lies is hard. No tools show you what the MI image covers (RSVP shows you a predicted MI image, but it's hard to figure out exactly where this lies in the original images), so you have to do a lot of eyeballing and detective work, and fit the results back into the original image. Oy. We email the resulting sketch to Steve, who prints it out, plays around with a ruler, and asks us to shift the whole thing 4.5cm to the left. So we do.
I think I have some ideas for improvements for future missions.
Anyway, with that done, the rest of the work isn't so hard. There are a lot of images in the mosaic, but they fall into groups, a total of 24 stacks of six images (including the stereo image) apiece. I further break down the mosaic into six columns of four stacks each, and then there's only a little overhead work to set up each column. It takes time, but I've had harder sequences to build. In the end, it turns out the rover won't have time to do more than four of the columns thisol, but that's still two-thirds of the total; we'll fill in the rest nextersol.
The only serious issue that arises is a question of using the IMU during the IDD sequence. This is something we do when there's a chance the vehicle's overall attitude would change during the sequence, which is something that could potentially endanger the IDD: if the vehicle tips forward when the arm's extended, suddenly it's that much closer to the ground, and could run into something we thought it would be clear of.
We're supposed to need to turn the IMU on when IDDing only when the rover's preloading the RAT or when we're at a significant pitch -- or anyway, that's what I thought the rules were. Ashitey thinks we need to do it when we're tilted, not just when we're pitched. (That is, he thinks we should do it even when we have zero pitch but lots of roll.) Thisol's pitch is only about 6.5 degrees but the overall tilt is a much higher 14.5 degrees, so it's going to matter which of us is right. John Wright is around, so I bring him in as a tie-breaker, and he votes with Ashitey, darn it. ("That was the wrong answer," I tell him.[1])
Why not just turn on the IMU anyway? Spirit's got power to burn, so it's not that. The problem is that there's ten seconds of overhead in every IDD move when the IMU is on, and what with all the crazy stuff we're doing thisol, the ten seconds per move add up to a whopping 38 minutes of overhead! And we don't have that to spare.
We could make back the 38 minutes by cutting another column, but John comes up with a better idea than that. It's a compromise: turn on the IMU during the unstow and first placement, and thereafter only when we actually contact the surface. Those are the times when there's the greatest risk (not that there's any real risk, but we're paid to be paranoid and we earn every penny) that the attitude would change. And that compromise adds only about six or eight minutes to the run-time, not 38. So that's what we do.
It's another fairly long day, but still less than ten hours. And if these images look as cool as I hope they will, it will have been well worth it.
Courtesy NASA/JPL-Caltech. Poised to examine Keystone.
Footnotes:
[1] Turns out I was right: pitch matters, roll doesn't. Fortunately, with the maturity and distance of five years, I have the class not to say nyaah nyaah.
Thisol we're starting to IDD Keystone. Just to make sure I don't get bored, this is going to be the biggest MI mosaic of a single object ever, a total of 6x4x5+stereo = 144 images total. Our approach is to sequence as many of them today as the rover will have time to execute, and finish the rest tomorrow.
Before we can even really get started, we have to figure out what we want to image. This is troublesome, because we first have to know the angle of the MI with respect to the roughly rectangular rock surface -- that affects how the image footprints are oriented, which affects how you lay out the mosaic -- and none of our tools does that very well. So we basically just eyeball it, and it turns out it's reasonably close to the same orientation as the overall rectangle of rock itself, which simplifies things.
For the next step in the process, Ashitey and I (mostly Ashitey) pick a candidate center for the mosaic and use an image processing program to sketch out what the mosaic will cover with that as its center. Drawing the rectangle is easy, but figuring out where it lies is hard. No tools show you what the MI image covers (RSVP shows you a predicted MI image, but it's hard to figure out exactly where this lies in the original images), so you have to do a lot of eyeballing and detective work, and fit the results back into the original image. Oy. We email the resulting sketch to Steve, who prints it out, plays around with a ruler, and asks us to shift the whole thing 4.5cm to the left. So we do.
I think I have some ideas for improvements for future missions.
Anyway, with that done, the rest of the work isn't so hard. There are a lot of images in the mosaic, but they fall into groups, a total of 24 stacks of six images (including the stereo image) apiece. I further break down the mosaic into six columns of four stacks each, and then there's only a little overhead work to set up each column. It takes time, but I've had harder sequences to build. In the end, it turns out the rover won't have time to do more than four of the columns thisol, but that's still two-thirds of the total; we'll fill in the rest nextersol.
The only serious issue that arises is a question of using the IMU during the IDD sequence. This is something we do when there's a chance the vehicle's overall attitude would change during the sequence, which is something that could potentially endanger the IDD: if the vehicle tips forward when the arm's extended, suddenly it's that much closer to the ground, and could run into something we thought it would be clear of.
We're supposed to need to turn the IMU on when IDDing only when the rover's preloading the RAT or when we're at a significant pitch -- or anyway, that's what I thought the rules were. Ashitey thinks we need to do it when we're tilted, not just when we're pitched. (That is, he thinks we should do it even when we have zero pitch but lots of roll.) Thisol's pitch is only about 6.5 degrees but the overall tilt is a much higher 14.5 degrees, so it's going to matter which of us is right. John Wright is around, so I bring him in as a tie-breaker, and he votes with Ashitey, darn it. ("That was the wrong answer," I tell him.[1])
Why not just turn on the IMU anyway? Spirit's got power to burn, so it's not that. The problem is that there's ten seconds of overhead in every IDD move when the IMU is on, and what with all the crazy stuff we're doing thisol, the ten seconds per move add up to a whopping 38 minutes of overhead! And we don't have that to spare.
We could make back the 38 minutes by cutting another column, but John comes up with a better idea than that. It's a compromise: turn on the IMU during the unstow and first placement, and thereafter only when we actually contact the surface. Those are the times when there's the greatest risk (not that there's any real risk, but we're paid to be paranoid and we earn every penny) that the attitude would change. And that compromise adds only about six or eight minutes to the run-time, not 38. So that's what we do.
It's another fairly long day, but still less than ten hours. And if these images look as cool as I hope they will, it will have been well worth it.
Courtesy NASA/JPL-Caltech. Poised to examine Keystone.
Footnotes:
[1] Turns out I was right: pitch matters, roll doesn't. Fortunately, with the maturity and distance of five years, I have the class not to say nyaah nyaah.
2010-04-14
Spirit Sol 455
The good news is that we're not dug in. The bad news is that we're not uphill.
Instead, we experienced alarmingly high levels of side-slip, gravity pulling us downhill as we tried to drive across the slope. The backoff worked, but then, only a couple of meters or so into the uphill segment, the sequence detected that we came too close to an obstacle and shut us down.
Ashitey tells me something I don't want to hear. "We can't make it," he says. "We should go back downhill and try to go around."
I hate it, but he's right. Straight uphill doesn't work. Crossing at a 45-degree angle doesn't work. Shallower angles -- either east or west -- take us into impassable slopes on one side, or rocks and an uncrossable berm on the other. The only way out is down.
Ashitey sets up a meeting among all the concerned parties. We gather in the SOWG room and, in short, agree that Ashitey's right. We'll have to go down.
Twelve meters or so downhill, there's what looks like a graded path around the side. My big concern is that we can't see all the way along the grade, so for all we know, we'll follow it to a certain point and then find that that way is blocked as well. But there's no way up from here, so it's the right next thing to try.
It's a complex drive. Ashitey and I are dumb enough to make it even harder on ourselves by adding a cross-slope leg after the downhill leg. But at least if it works, we'll have made real progress. If you can call a retreat progress. I guess we'll have made real regress, or something like that.
We also manage to pack in a couple of visits. The first is by science
fiction writer Bruce Sterling, whom Frank invited to come see us drive
a Mars rover. This is so wild I can't describe it.
Sterling's not what I expected. I don't know what I expected, but he's not it. He's short, dumpy, long-haired, laconic -- and creative, that part's no surprise. He takes one look at the six-wheeled rover with solar-panel wings and calls it a bug, which is a comparison that somehow never occurred to me, and I'm practically an avocational entomologist.
Our other visitor -- or group of visitors -- is brought around by Eric Baumgartner. They're a quartet of science fair winners from his kid's school, all around eleven or twelve years old. Plus two teachers. Ashitey and I do a good job with them, I think. He makes sure they look through the 3-D glasses, which appeals to everyone but especially kids. I don't think I'm generally very good with kids, but I remember being their age and being excited about science, and I'd have died for a trip like this. They're just like that -- a little intimidated at first, but grooving on it. I manage to get them to laugh and loosen up a little, and then they're just a bunch of smart kids having a blast. I love my job.
The long day goes on. We work apace. Jim Erickson shows up, with his usual grim look, and subtly chides us for adding the cross-slope leg. He's not the type to second-guess our decisions, but he's got a point to make and he makes it. We're putting in a lot of late nights, which exhausts the team. We've got to keep it simple. The extended mission is a marathon, not a sprint.
At this point, it's too late to gain anything by cutting the cross-slope leg. Since it's a sunk cost, we decide to leave it in for now, but we'll take it out if it ends up sucking up any more of our time. (It doesn't: the cross-slope leg makes it into the final sequence.)
We reach that point in the day where we're just waiting for the TAP/SIE to put everything in its final form. Today we put the time to good use. I've been worried about the fact that nobody really pays attention in the walkthroughs any more, which is partly because the complexity of the sequences, and the amount of stuff you have to know about the rover in order to follow along, is really overwhelming. So I break out a few pieces of the sequence and start going over them in detail -- an impromptu rover-driving course. When I see that people's brains are getting full, I declare success and move on to other things. It's only a little progress, but it's progress.
You can tell it's late, by the way. Jeff Favretto, following up on Erickson's earlier comments, says something about the value of getting out by 8:00 so we can spend time with our wives. (It's now nearly 10:00, and we'll be there another hour. We started twelve hours ago.) "I'm single, but if I don't leave by 8, I'll never get a girlfriend," Dan Moyers chimes in. And Favretto quips: "Dan, you'll have to leave a lot earlier than 8 if you want to get a girlfriend."
Ouch!
On a lighter note, Steve Squyres's book (Roving Mars) is coming out soon, and he's asked people on the project to help him make sure that the name of everyone involved in the project is listed in an appendix. This includes everyone from the project managers down to off-site contractors who sewed the parachutes. He had a good start already, but for the past week or so he's been beefing up the list, adding names as he's sent them.
So far he's got four thousand.
[Next post: sol 469, April 28.]
Instead, we experienced alarmingly high levels of side-slip, gravity pulling us downhill as we tried to drive across the slope. The backoff worked, but then, only a couple of meters or so into the uphill segment, the sequence detected that we came too close to an obstacle and shut us down.
Ashitey tells me something I don't want to hear. "We can't make it," he says. "We should go back downhill and try to go around."
I hate it, but he's right. Straight uphill doesn't work. Crossing at a 45-degree angle doesn't work. Shallower angles -- either east or west -- take us into impassable slopes on one side, or rocks and an uncrossable berm on the other. The only way out is down.
Ashitey sets up a meeting among all the concerned parties. We gather in the SOWG room and, in short, agree that Ashitey's right. We'll have to go down.
Twelve meters or so downhill, there's what looks like a graded path around the side. My big concern is that we can't see all the way along the grade, so for all we know, we'll follow it to a certain point and then find that that way is blocked as well. But there's no way up from here, so it's the right next thing to try.
It's a complex drive. Ashitey and I are dumb enough to make it even harder on ourselves by adding a cross-slope leg after the downhill leg. But at least if it works, we'll have made real progress. If you can call a retreat progress. I guess we'll have made real regress, or something like that.
We also manage to pack in a couple of visits. The first is by science
fiction writer Bruce Sterling, whom Frank invited to come see us drive
a Mars rover. This is so wild I can't describe it.
Sterling's not what I expected. I don't know what I expected, but he's not it. He's short, dumpy, long-haired, laconic -- and creative, that part's no surprise. He takes one look at the six-wheeled rover with solar-panel wings and calls it a bug, which is a comparison that somehow never occurred to me, and I'm practically an avocational entomologist.
Our other visitor -- or group of visitors -- is brought around by Eric Baumgartner. They're a quartet of science fair winners from his kid's school, all around eleven or twelve years old. Plus two teachers. Ashitey and I do a good job with them, I think. He makes sure they look through the 3-D glasses, which appeals to everyone but especially kids. I don't think I'm generally very good with kids, but I remember being their age and being excited about science, and I'd have died for a trip like this. They're just like that -- a little intimidated at first, but grooving on it. I manage to get them to laugh and loosen up a little, and then they're just a bunch of smart kids having a blast. I love my job.
The long day goes on. We work apace. Jim Erickson shows up, with his usual grim look, and subtly chides us for adding the cross-slope leg. He's not the type to second-guess our decisions, but he's got a point to make and he makes it. We're putting in a lot of late nights, which exhausts the team. We've got to keep it simple. The extended mission is a marathon, not a sprint.
At this point, it's too late to gain anything by cutting the cross-slope leg. Since it's a sunk cost, we decide to leave it in for now, but we'll take it out if it ends up sucking up any more of our time. (It doesn't: the cross-slope leg makes it into the final sequence.)
We reach that point in the day where we're just waiting for the TAP/SIE to put everything in its final form. Today we put the time to good use. I've been worried about the fact that nobody really pays attention in the walkthroughs any more, which is partly because the complexity of the sequences, and the amount of stuff you have to know about the rover in order to follow along, is really overwhelming. So I break out a few pieces of the sequence and start going over them in detail -- an impromptu rover-driving course. When I see that people's brains are getting full, I declare success and move on to other things. It's only a little progress, but it's progress.
You can tell it's late, by the way. Jeff Favretto, following up on Erickson's earlier comments, says something about the value of getting out by 8:00 so we can spend time with our wives. (It's now nearly 10:00, and we'll be there another hour. We started twelve hours ago.) "I'm single, but if I don't leave by 8, I'll never get a girlfriend," Dan Moyers chimes in. And Favretto quips: "Dan, you'll have to leave a lot earlier than 8 if you want to get a girlfriend."
Ouch!
On a lighter note, Steve Squyres's book (Roving Mars) is coming out soon, and he's asked people on the project to help him make sure that the name of everyone involved in the project is listed in an appendix. This includes everyone from the project managers down to off-site contractors who sewed the parachutes. He had a good start already, but for the past week or so he's been beefing up the list, adding names as he's sent them.
So far he's got four thousand.
[Next post: sol 469, April 28.]
2010-04-13
Spirit Sol 454
We've recovered from the weekend's fault, and the downlink shows we're in pretty much the state we expected: somewhat dug in. A small potato hovers by the left front wheel, and another possible potato -- the scientists bet strongly it's just a dirt clod, not a rock -- sits in the left rear wheel. So it'll be tricky just to get away.
Fortunately, we have a little more time than expected. On the large scale, that is: "There's nothing magic about sol 500 any more," Steve declares at the SOWG meeting. This had previously been the date by which we had to make it to the summit, before the sun angle started to favor south-facing slopes. The resulting lack of energy on our current north-facing slope would mean we'd face serious consequences. In the benign scenario, we'd have less and less time to drive each day, making the top farther and farther away. In the less benign scenario, we'd freeze to death.
But the dust devil that cleaned the solar panels bought us more time than I'd thought: even with the reduced energy level, we can last well past sol 500 now. The pace is now to be driven by the Martian terrain and scientific discoveries, not a mad scramble for the top.
Which is a good thing, because we haven't exactly been making a lot of progress lately. The next thing we're going to try is cross-slope driving. The last drive failed because it was fighting directly uphill -- the same mistake we made around Wopmay, back on the other side of the planet. This time we're going to try to drive at about a 45-degree angle to the slope instead. Before we can even do that, we have to carefully back away downhill, to extricate ourselves from our dug-in position. If you thought the going had been slow recently, just think what it would be like if we picked up another potato ....
Just a few meters away, the slope breaks, and should be easier going. Once the rover gets that far, the drive aims a little more uphill, to avoid a gap in the terrain mesh (representing an area we can't quite see from our current position, and therefore can't evaluate).
This is the first drive I've written for Spirit in quite some time. I have this fantasy of coming downstairs and magically putting everything back into shape, coming up with some brilliant scheme that puts the rover on the top of this hill in no time. Awing the crowd. But no such thing will happen: the guys who've been doing this are plenty smart already, and if there were any such clever trick, they'd have thought of it. Getting to the top is just going to be a long, tough slog, and it's really possible that we just won't make it.
I have a talk coming up in a few days, down in Anaheim at the USENIX 2005 conference. I'll be talking about how to drive a Mars rover. I spend some time thinking about how to put the audience in the right mindset. This is what I come up with:
A little too dramatic, I think; I decide not to use it. But it sounds cool.
Fortunately, we have a little more time than expected. On the large scale, that is: "There's nothing magic about sol 500 any more," Steve declares at the SOWG meeting. This had previously been the date by which we had to make it to the summit, before the sun angle started to favor south-facing slopes. The resulting lack of energy on our current north-facing slope would mean we'd face serious consequences. In the benign scenario, we'd have less and less time to drive each day, making the top farther and farther away. In the less benign scenario, we'd freeze to death.
But the dust devil that cleaned the solar panels bought us more time than I'd thought: even with the reduced energy level, we can last well past sol 500 now. The pace is now to be driven by the Martian terrain and scientific discoveries, not a mad scramble for the top.
Which is a good thing, because we haven't exactly been making a lot of progress lately. The next thing we're going to try is cross-slope driving. The last drive failed because it was fighting directly uphill -- the same mistake we made around Wopmay, back on the other side of the planet. This time we're going to try to drive at about a 45-degree angle to the slope instead. Before we can even do that, we have to carefully back away downhill, to extricate ourselves from our dug-in position. If you thought the going had been slow recently, just think what it would be like if we picked up another potato ....
Just a few meters away, the slope breaks, and should be easier going. Once the rover gets that far, the drive aims a little more uphill, to avoid a gap in the terrain mesh (representing an area we can't quite see from our current position, and therefore can't evaluate).
This is the first drive I've written for Spirit in quite some time. I have this fantasy of coming downstairs and magically putting everything back into shape, coming up with some brilliant scheme that puts the rover on the top of this hill in no time. Awing the crowd. But no such thing will happen: the guys who've been doing this are plenty smart already, and if there were any such clever trick, they'd have thought of it. Getting to the top is just going to be a long, tough slog, and it's really possible that we just won't make it.
I have a talk coming up in a few days, down in Anaheim at the USENIX 2005 conference. I'll be talking about how to drive a Mars rover. I spend some time thinking about how to put the audience in the right mindset. This is what I come up with:
You're 450 sols into a 90-sol mission. Threats surround you; every path is blocked. You're chasing the sun to the top of a hill and if you don't beat it you'll die. Welcome to Mars.
A little too dramatic, I think; I decide not to use it. But it sounds cool.
2010-04-12
Spirit Sol 453
On my way in I notice we're on the front page. Our funding was extended for 18 more months, making the front page of the Pasadena Star-News. I remember when I'd come in and we were front-page news every day. I buy a copy for nostalgia's sake.
I'm in an hour early, to get a jump on the drive sequence, but Jim Erickson needs a minute of my time. "Did you hear about the problems over the weekend?" he asks. Oh, dear.
Both rovers had bad weekends. Opportunity's drive faulted out before it even drove anywhere. Last week the suspension limits tripped during a drive, and tripped again during the auto-recovery -- an off-nominal circumstance, but one we plan for. For the weekend, they sequenced a drive where they backed off the ripple they tripped on, then continued, ignoring the suspension limits during the backoff. But they didn't explicitly disable the limits, they just assumed the limits were i the "ignored" state for the first part of the drive. However, it turns out that the flag that tells the rover to check the suspension limits (not ignore them) was accidentally saved to EEPROM as the default weeks ago, so Opportunity actually applied the most recently set suspension limits, and since they'd tripped before, and Opportunity hadn't moved, it went right ahead and tripped them again. This was a problem that was just waiting to bite us. And then, possibly because of that error, they mis-timed the solar-array stomp, which meant the rover didn't nap when it was supposed to, and thus is power-poor.
Spirit had problems, too. They didn't properly clean up after last Thursday's fatal. (Which fatal, you will recall, was my damn fault.) The fatal caused Spirit to decide it didn't know where its HGA was pointed, so it didn't listen for the uplink from Earth. As a result, none of our cleanup sequences made it up over the weekend. Which means we can't drive today; we still need to do our imaging and update the rover's position. Lucky for us, Beth Dewell noticed the problem and already got Spirit back into decent shape over the weekend, which is a good thing -- else we'd be down for another day.
So, for those of you keeping score at home: last time I was on Spirit, I jammed a potato in the wheel, which blew about ten or twenty sols. This time I fataled it and blew Thursday through today (Monday) at least. The Opportunity problem is not my fault, but I couldn't feel worse if it were. I am a bad rover driver. I'm glum. I want to kill myself.
But if we're not driving today, I have time to answer Jim's questions, at least. He ticks them off. "These problems we've been having with the vehicles lately: are they problems due to unfamiliarity with the new version of the flight software? Are the RPs overworked? Are they complacent?"
I answer these as best I can. I know some of the problems were due to unfamiliarity with the new flight software; it's a partial explanation. I don't think the RPs are generally overworked. (I personally have a lot more work than I have time for, but this afternoon I'm going to talk to someone about coming in to the RSVP development team to pick some of it up, which will mitigate the problem.)
As for complacency ... well, I want to say no, but the honest answer is probably yes. We're not as paranoid about these vehicles as we used to be. Some of that's understandable and even proper. Some of it, maybe not.
Jim nods thoughtfully and goes away. As if on cue, Frank enters, fresh from the Land of Opportunity. He tells me he wouldn't have caught the problem that led to a fatal, either, but: "I was gonna give you a ribbing about it anyway, but then we had our problem over the weekend, and now I can't give you a hard time."
"The universe smiles on Scott," Al Herrera remarks.
If he's right, it means the problem on Opportunity was my fault. You know when I said I couldn't feel worse? I might have been wrong.
Upstairs, while we're waiting for the SOWG meeting to start, we try to look on the bright side of this extended anomaly. "The batteries will be topped off," Chris points out.
"And now the bar is lowered," I add. "If we can just manage to drive without fataling, we'll be so happy." It's black humor, but it's humor.
Squyres is -- what's the equivalent of "visibly unhappy" for a disembodied voice, "audibly unhappy?" Well, he's unhappy. We've got basically nothing to show for last week, with the ODY problem followed by the fatal, and today will be basically spent on recovery. He pokes at this to see if we can drive anyway, but we really can't do it without more data. He manages to talk Callas into allowing a MTES checkout, so I suppose that's something, from his point of view.
But we're 120 sols into the climb and only halfway up the hill, and we're 50 sols away from our deadline. We're power-rich right now, but in about 50 more sols, the sun's wandering position in the sky starts to favor south-facing slopes over north-facing slopes such as the one we're on. Our friendly, solar-panel-cleaning dust devils might have extended the deadline, but we'll eventually have to cross to the far slope or die. Steve wants to go, go, go, and we've got to just sit here. At least for today.
"We're gonna have a good week," Steve says emphatically at the close of the meeting. "Power of positive thinking."
I'm in an hour early, to get a jump on the drive sequence, but Jim Erickson needs a minute of my time. "Did you hear about the problems over the weekend?" he asks. Oh, dear.
Both rovers had bad weekends. Opportunity's drive faulted out before it even drove anywhere. Last week the suspension limits tripped during a drive, and tripped again during the auto-recovery -- an off-nominal circumstance, but one we plan for. For the weekend, they sequenced a drive where they backed off the ripple they tripped on, then continued, ignoring the suspension limits during the backoff. But they didn't explicitly disable the limits, they just assumed the limits were i the "ignored" state for the first part of the drive. However, it turns out that the flag that tells the rover to check the suspension limits (not ignore them) was accidentally saved to EEPROM as the default weeks ago, so Opportunity actually applied the most recently set suspension limits, and since they'd tripped before, and Opportunity hadn't moved, it went right ahead and tripped them again. This was a problem that was just waiting to bite us. And then, possibly because of that error, they mis-timed the solar-array stomp, which meant the rover didn't nap when it was supposed to, and thus is power-poor.
Spirit had problems, too. They didn't properly clean up after last Thursday's fatal. (Which fatal, you will recall, was my damn fault.) The fatal caused Spirit to decide it didn't know where its HGA was pointed, so it didn't listen for the uplink from Earth. As a result, none of our cleanup sequences made it up over the weekend. Which means we can't drive today; we still need to do our imaging and update the rover's position. Lucky for us, Beth Dewell noticed the problem and already got Spirit back into decent shape over the weekend, which is a good thing -- else we'd be down for another day.
So, for those of you keeping score at home: last time I was on Spirit, I jammed a potato in the wheel, which blew about ten or twenty sols. This time I fataled it and blew Thursday through today (Monday) at least. The Opportunity problem is not my fault, but I couldn't feel worse if it were. I am a bad rover driver. I'm glum. I want to kill myself.
But if we're not driving today, I have time to answer Jim's questions, at least. He ticks them off. "These problems we've been having with the vehicles lately: are they problems due to unfamiliarity with the new version of the flight software? Are the RPs overworked? Are they complacent?"
I answer these as best I can. I know some of the problems were due to unfamiliarity with the new flight software; it's a partial explanation. I don't think the RPs are generally overworked. (I personally have a lot more work than I have time for, but this afternoon I'm going to talk to someone about coming in to the RSVP development team to pick some of it up, which will mitigate the problem.)
As for complacency ... well, I want to say no, but the honest answer is probably yes. We're not as paranoid about these vehicles as we used to be. Some of that's understandable and even proper. Some of it, maybe not.
Jim nods thoughtfully and goes away. As if on cue, Frank enters, fresh from the Land of Opportunity. He tells me he wouldn't have caught the problem that led to a fatal, either, but: "I was gonna give you a ribbing about it anyway, but then we had our problem over the weekend, and now I can't give you a hard time."
"The universe smiles on Scott," Al Herrera remarks.
If he's right, it means the problem on Opportunity was my fault. You know when I said I couldn't feel worse? I might have been wrong.
Upstairs, while we're waiting for the SOWG meeting to start, we try to look on the bright side of this extended anomaly. "The batteries will be topped off," Chris points out.
"And now the bar is lowered," I add. "If we can just manage to drive without fataling, we'll be so happy." It's black humor, but it's humor.
Squyres is -- what's the equivalent of "visibly unhappy" for a disembodied voice, "audibly unhappy?" Well, he's unhappy. We've got basically nothing to show for last week, with the ODY problem followed by the fatal, and today will be basically spent on recovery. He pokes at this to see if we can drive anyway, but we really can't do it without more data. He manages to talk Callas into allowing a MTES checkout, so I suppose that's something, from his point of view.
But we're 120 sols into the climb and only halfway up the hill, and we're 50 sols away from our deadline. We're power-rich right now, but in about 50 more sols, the sun's wandering position in the sky starts to favor south-facing slopes over north-facing slopes such as the one we're on. Our friendly, solar-panel-cleaning dust devils might have extended the deadline, but we'll eventually have to cross to the far slope or die. Steve wants to go, go, go, and we've got to just sit here. At least for today.
"We're gonna have a good week," Steve says emphatically at the close of the meeting. "Power of positive thinking."
2010-04-09
Spirit Sol 450
I have to go give an RSVP demo, but first I have to know how Spirit did on my first attempt to drive it for months. I borrow a Web browser window from Colette Wilklow and quickly bring up the automatically generated Web page that summarizes the results.
The first line says more than eight meters -- not too shabby, for this terrain. Then I notice the next line, the usually more accurate line -- the one that tells you not just how far the wheels spun but how far the rover actually moved. About 87cm. That's all.
Fuck!
Just then my cell phone rings.
"Hello?"
It's Mark Maimone. "Well, I ruined my record," he says. "I fataled a spacecraft."
"Fataled" doesn't mean the rover is dead, but it is a bad thing. It means the rover rebooted. And it's our fault.
After I get back from my demo, I get the story. What happened here, it transpires, is that we screwed up something really stupid. It has to do with a feature in the new release of the flight software. The original flight software let you write sequences that made decisions based on a limited set of parameters -- so that the rover could decide to do one thing or another based on its distance to a defined target, whether the previous command had succeeded or failed, and about a dozen more. The new flight software added a bunch more of these parameters, one of them being the rover's current heading. (So now we can write a sequence that says, for instance, "if you're not already heading more or less east, turn to the east.")
Now, there was only one way to add these new parameters in a way that was backward-compatible with the old command dictionary (a requirement imposed from above), and that was sort of a kludge. The trick was to take one of the old parameters -- one we picked because we never actually used it in flight -- and use a command to redefine that one so that it's an alias for one of the new ones, and then tell the rover to test that. The "if" command itself resets the alias to an undefined state afterward. And when you test this parameter when it's in an undefined state, the flight software fatals.
Now, we did everything right. In the main sequence, that is. What we forgot is that we also had a background sequence running, one that checks periodically to see if the rover is too close to an obstacle, so that it can abort driving. And this sequence is basically a long string of "if you're too close to obstacle #1, stop driving; if you're too close to obstacle #2, stop driving ...." One of these commands just happened to undefine the special parameter set in the main sequence, triggering the fatal.
Ashitey and I didn't realize that any "if" reset the special parameter -- we thought only the ones that used that parameter reset it. Mark Maimone knew better, but he was awfully tired and stressed out yesterday. I fear he's overloaded.
John Callas declares by fiat that the team is too tired to work this weekend. We're just going to clean up after our boneheaded error and do a minimal weekend plan. No driving, no IDD. It's a hint that we should go home and rest up. But do I? Nope: too much other work to do.[1]
I'm not surprised the team is tired. This whole hill-climbing campaign with Spirit looks like it could become frustrating. For the rover, it's a long climb. We've been at it three months, and we're only halfway up. The hell of it is, two decently strong men could pick the damn rover up and carry it to the summit in half an hour.
When Andy's new schedule comes out, it shows I'm not downstairs just temporarily. Looks like I'm back on Spirit at least through the end of May, which probably means for good. I'm surprised that I'm disappointed by this. I'm almost ashamed to admit it, but there was a lot to be said for working on Opportunity. The team's more laid back. It's easier. You drive really far. It's a cheap kind of glory but I'll take it.
But in the end, I'm not too disappointed. This rover and I have been through a lot together already, and this just means there's more adventure ahead.
I can't wait to get started.
[Next post: sol 453, April 12.]
Footnotes:
[1] Easy for me to say now, but that was a bad decision: incompatible with rover safety. Shoulda gone home and slept. Shame on me.
The first line says more than eight meters -- not too shabby, for this terrain. Then I notice the next line, the usually more accurate line -- the one that tells you not just how far the wheels spun but how far the rover actually moved. About 87cm. That's all.
Fuck!
Just then my cell phone rings.
"Hello?"
It's Mark Maimone. "Well, I ruined my record," he says. "I fataled a spacecraft."
"Fataled" doesn't mean the rover is dead, but it is a bad thing. It means the rover rebooted. And it's our fault.
After I get back from my demo, I get the story. What happened here, it transpires, is that we screwed up something really stupid. It has to do with a feature in the new release of the flight software. The original flight software let you write sequences that made decisions based on a limited set of parameters -- so that the rover could decide to do one thing or another based on its distance to a defined target, whether the previous command had succeeded or failed, and about a dozen more. The new flight software added a bunch more of these parameters, one of them being the rover's current heading. (So now we can write a sequence that says, for instance, "if you're not already heading more or less east, turn to the east.")
Now, there was only one way to add these new parameters in a way that was backward-compatible with the old command dictionary (a requirement imposed from above), and that was sort of a kludge. The trick was to take one of the old parameters -- one we picked because we never actually used it in flight -- and use a command to redefine that one so that it's an alias for one of the new ones, and then tell the rover to test that. The "if" command itself resets the alias to an undefined state afterward. And when you test this parameter when it's in an undefined state, the flight software fatals.
Now, we did everything right. In the main sequence, that is. What we forgot is that we also had a background sequence running, one that checks periodically to see if the rover is too close to an obstacle, so that it can abort driving. And this sequence is basically a long string of "if you're too close to obstacle #1, stop driving; if you're too close to obstacle #2, stop driving ...." One of these commands just happened to undefine the special parameter set in the main sequence, triggering the fatal.
Ashitey and I didn't realize that any "if" reset the special parameter -- we thought only the ones that used that parameter reset it. Mark Maimone knew better, but he was awfully tired and stressed out yesterday. I fear he's overloaded.
John Callas declares by fiat that the team is too tired to work this weekend. We're just going to clean up after our boneheaded error and do a minimal weekend plan. No driving, no IDD. It's a hint that we should go home and rest up. But do I? Nope: too much other work to do.[1]
I'm not surprised the team is tired. This whole hill-climbing campaign with Spirit looks like it could become frustrating. For the rover, it's a long climb. We've been at it three months, and we're only halfway up. The hell of it is, two decently strong men could pick the damn rover up and carry it to the summit in half an hour.
When Andy's new schedule comes out, it shows I'm not downstairs just temporarily. Looks like I'm back on Spirit at least through the end of May, which probably means for good. I'm surprised that I'm disappointed by this. I'm almost ashamed to admit it, but there was a lot to be said for working on Opportunity. The team's more laid back. It's easier. You drive really far. It's a cheap kind of glory but I'll take it.
But in the end, I'm not too disappointed. This rover and I have been through a lot together already, and this just means there's more adventure ahead.
I can't wait to get started.
[Next post: sol 453, April 12.]
Footnotes:
[1] Easy for me to say now, but that was a bad decision: incompatible with rover safety. Shoulda gone home and slept. Shame on me.
2010-04-08
Spirit Sol 449
ODY is back online; our broadband connection to Mars has been restored. I still don't know what the heck was wrong with it, but I guess I'll find out.
So we're driving. Our plan is to continue the drive that faulted out this past weekend; we've been sitting still since then because of the relay problem. I'm glad we're going to do some driving; else I'd come in next week and have to be RP-1, without any way to get properly up to speed. Sure, I've read through the drive sequences they've been building, but it's never the same as when the pressure's on.
My brain still hasn't caught up with this side of the world. I go over to MIPL to ask them to build us a new terrain mesh with the extra NAVCAM images that just came down, and unconsciously I go over to talk to the Opportunity support guy. Who's as puzzled as I am until I realize I'm supposed to be talking to the Spirit support guy. Sheesh.
But it's fun sequencing this rover again. I'd almost forgotten what challenging drives were like. Opportunity is just a slow car in a parking lot full of speed bumps. Driving Spirit up a rocky hill is delicate, exacting work. Obstacles to the left of you, obstacles to the right of you -- and a slope that's only too happy to help you slide into them.
One of the innovative techniques the Spirits have developed is a parallel obstacle-check sequence. The way we used to drive through hazardous zones was to first construct a list of hazards visible in the imagery. Then we'd write a sequence that would drive a little way, check if we were too close to the obstacles, drive a little farther, check again, and so on. At each stage, you'd tend to check only for obstacles you thought you'd be likely to be near at that point, because it was just a nightmare to try to check for all of them all of the time. The resulting drive sequences had a lot of repetition and were hard to modify: if you wanted to change your driving path, you'd have to revisit a lot of the places where you stopped and checked for hazards to make sure you were checking for the right ones.
So the Spirits worked out a way to separate those concerns. They now build one sequence that checks whether the rover is near any of the known hazards; this is padded out with wait commands so that it takes one minute to run. Then they have a second sequence that calls the first one ten times, and a third sequence that calls the ten-minute version 24 times (so it runs for 240 minutes -- or four hours, longer than their drives). Finally, this third sequence is activated in parallel with the drive. So the drive can just contain the needed commands to take the rover along the desired path, and the parallel obstacle-check sequence will stop it if it gets too close to any of the hazards. The parallel sequence checks all of the hazards, and checks them every minute, and doesn't interfere with your development of the drive sequences; in all three ways, it's better than the old approach.[1]
There's a lot more to come up to speed on as well -- a lot of fiddling with parameters that we don't need to do on Opportunity, but which are needed on Spirit, mainly to avoid catching another potato. Finally, as gravy, thisol we're doing two things for the first time on Mars: at the end of the drive, we're using autonav with visual odometry (so the rover will autonomously avoid obstacles as well as keeping careful track of its location), and we're using a new flight software feature that lets us make run-time decisions based on the vehicle's heading. If all goes well, we'll come in tomorrow to find we've climbed Husband Hill another 15 meters or so.
There's another difference between the two rovers: Opportunity's drive distances have more digits.
[1] A later flight software update added a notion of "keepout zones," which replaced this elaborate tree of obstacle-checking sequences with just a few commands, one command per obstacle. Incomparably better.
So we're driving. Our plan is to continue the drive that faulted out this past weekend; we've been sitting still since then because of the relay problem. I'm glad we're going to do some driving; else I'd come in next week and have to be RP-1, without any way to get properly up to speed. Sure, I've read through the drive sequences they've been building, but it's never the same as when the pressure's on.
My brain still hasn't caught up with this side of the world. I go over to MIPL to ask them to build us a new terrain mesh with the extra NAVCAM images that just came down, and unconsciously I go over to talk to the Opportunity support guy. Who's as puzzled as I am until I realize I'm supposed to be talking to the Spirit support guy. Sheesh.
But it's fun sequencing this rover again. I'd almost forgotten what challenging drives were like. Opportunity is just a slow car in a parking lot full of speed bumps. Driving Spirit up a rocky hill is delicate, exacting work. Obstacles to the left of you, obstacles to the right of you -- and a slope that's only too happy to help you slide into them.
One of the innovative techniques the Spirits have developed is a parallel obstacle-check sequence. The way we used to drive through hazardous zones was to first construct a list of hazards visible in the imagery. Then we'd write a sequence that would drive a little way, check if we were too close to the obstacles, drive a little farther, check again, and so on. At each stage, you'd tend to check only for obstacles you thought you'd be likely to be near at that point, because it was just a nightmare to try to check for all of them all of the time. The resulting drive sequences had a lot of repetition and were hard to modify: if you wanted to change your driving path, you'd have to revisit a lot of the places where you stopped and checked for hazards to make sure you were checking for the right ones.
So the Spirits worked out a way to separate those concerns. They now build one sequence that checks whether the rover is near any of the known hazards; this is padded out with wait commands so that it takes one minute to run. Then they have a second sequence that calls the first one ten times, and a third sequence that calls the ten-minute version 24 times (so it runs for 240 minutes -- or four hours, longer than their drives). Finally, this third sequence is activated in parallel with the drive. So the drive can just contain the needed commands to take the rover along the desired path, and the parallel obstacle-check sequence will stop it if it gets too close to any of the hazards. The parallel sequence checks all of the hazards, and checks them every minute, and doesn't interfere with your development of the drive sequences; in all three ways, it's better than the old approach.[1]
There's a lot more to come up to speed on as well -- a lot of fiddling with parameters that we don't need to do on Opportunity, but which are needed on Spirit, mainly to avoid catching another potato. Finally, as gravy, thisol we're doing two things for the first time on Mars: at the end of the drive, we're using autonav with visual odometry (so the rover will autonomously avoid obstacles as well as keeping careful track of its location), and we're using a new flight software feature that lets us make run-time decisions based on the vehicle's heading. If all goes well, we'll come in tomorrow to find we've climbed Husband Hill another 15 meters or so.
There's another difference between the two rovers: Opportunity's drive distances have more digits.
[1] A later flight software update added a notion of "keepout zones," which replaced this elaborate tree of obstacle-checking sequences with just a few commands, one command per obstacle. Incomparably better.
Subscribe to:
Posts (Atom)
Posts
