We've had some blind students working with the project to make the Web pages more accessible. They've also been observing ops for a while, and Callas picks one of them to shadow the RPs today. Her name's Grace King. She's got a fine, self-deprecating sense of humor and is deeply sarcastic. I like her already.
Unfortunately, despite having been forewarned about this by Callas, we don't really have much for her to do. The job is inherently mostly visual -- at least, as we currently conceive it. I kind of hoped there would be a choice we could throw to her, a fork in the road she could help us decide about or something. But I fail to spot any such opportunity. She seems pretty upbeat about the whole experience, but I feel as though we've failed her.
The science team is having a big meeting at Cornell, which has an interesting effect on the dynamics of the SOWG meeting. Instead of people at different sites trying to share the telecom line, it sounds as though Steve is running the game much as the SOWG chairs used to do when all the scientists were gathered here, passing a mike around. "Sounds like Donahue," I mutter to Cooper. "Just as long as it doesn't turn into Jerry Springer," he quips.
Cooper handles most of the driving thisol, while I try to engage our shadow. Neither she nor I really ends up with much to do. What a letdown.
[Next post: sol 562 (Opportunity sol 541), August 1.]
2010-07-27
2010-07-26
Opportunity Sol 535 (Spirit Sol 555)
I don't remember the last time this happened. We appear to have entirely lost the first part of the pass, leaving us nearly blind. We got the side-looking NAVCAMs and all of the PANCAMs, but we don't have any front HAZCAMs, the final rear HAZCAMs, or the center NAVCAMs -- not even thumbnails.
So the question is, can we drive anyway? The last drive aimed for the entrance to a particularly broad and flat zone, so if we can prove that that's where we ended up, we're fine. In an ideal world, this would be mostly a problem of triangulation -- first, find several landmarks in the images we do have, and estimate azimuth and range to those. Then do the same for the same landmarks in the previous sol's images, and compare the two results. That will tell us for certain where we are with respect to where we started.
There's only one small problem: the landmarks. In large measure, every part of Meridiani looks like every other part. How do you pick out the same point on the top of a ripple in two images taken 30m apart? We're able to find a couple of Scotty-like cobblestones in both the old and new images, but they're very distant in the older images and won't provide solid enough data for a convincing solution.
Things aren't looking good. Even if we get a good fix on our relative location, we won't be able to drive farther than about 20m with the data we have. But they managed to get an urgent retransmit request up to the spacecraft, so if we wait until tomorrow, we'll have all the data we normally have for a drive. It's less risky that way. And the way things work out with the restricted sols, we'll actually make more distance for the week with that approach.
So we reluctantly bag it. Beth Dewell breaks the bad news to Steve, who shrugs it off. "If that's the consensus at your end, then that's OK with me," he says. They replan the sol to do nothing but remote sensing, and we're released for the day; we'll come back tomorrow to plan the drive when we have the data. Well, I'd been planning to take a vacation day tomorrow, but what the heck. I figure I'll just take the rest of the day today instead.
A couple of hours later, Beth Dewell calls me at home. It turns out we had the needed data on the ground all along. Due to some kind of ground processing error, it was just stuck at the DSN. It's too late to change our minds today, so we're just continuing with the current plan.
What can you do? I just laugh.
So the question is, can we drive anyway? The last drive aimed for the entrance to a particularly broad and flat zone, so if we can prove that that's where we ended up, we're fine. In an ideal world, this would be mostly a problem of triangulation -- first, find several landmarks in the images we do have, and estimate azimuth and range to those. Then do the same for the same landmarks in the previous sol's images, and compare the two results. That will tell us for certain where we are with respect to where we started.
There's only one small problem: the landmarks. In large measure, every part of Meridiani looks like every other part. How do you pick out the same point on the top of a ripple in two images taken 30m apart? We're able to find a couple of Scotty-like cobblestones in both the old and new images, but they're very distant in the older images and won't provide solid enough data for a convincing solution.
Things aren't looking good. Even if we get a good fix on our relative location, we won't be able to drive farther than about 20m with the data we have. But they managed to get an urgent retransmit request up to the spacecraft, so if we wait until tomorrow, we'll have all the data we normally have for a drive. It's less risky that way. And the way things work out with the restricted sols, we'll actually make more distance for the week with that approach.
So we reluctantly bag it. Beth Dewell breaks the bad news to Steve, who shrugs it off. "If that's the consensus at your end, then that's OK with me," he says. They replan the sol to do nothing but remote sensing, and we're released for the day; we'll come back tomorrow to plan the drive when we have the data. Well, I'd been planning to take a vacation day tomorrow, but what the heck. I figure I'll just take the rest of the day today instead.
A couple of hours later, Beth Dewell calls me at home. It turns out we had the needed data on the ground all along. Due to some kind of ground processing error, it was just stuck at the DSN. It's too late to change our minds today, so we're just continuing with the current plan.
What can you do? I just laugh.
2010-07-23
Opportunity Sol 532 (Spirit Sol 552)
Unfortunately, the "Scotty" target we picked out last time doesn't seem to be worthy of more detailed study. So we won't be hanging around here; we'll be moving on. Or, as the LTP lead puts it: "It's time to say a fond goodbye to 'Scotty' and turn and go where no rover has gone before." We plan another 30m drive south -- another step toward Erebus.
However, we're not done with the Scotty theme yet. The target names for thisol follow a Scottish theme (e.g., "Glasgow"). I go a bit nuts with it, attaching a picture of Scotty to our uplink report (as "today's guest rover driver") and adding a bunch of Trek-related comments to the sequence. The get-fine-attitude command becomes "remodulating the shield harmonics." One of my comments on a drive command speaks of going to warp 5. A little ripple we cross along the way is grandly named "DenebianSlimeDevil." And, of course, our destination target for the drive is named "FinalFrontier."
I think this puzzles Paolo Bellutta, who grew up in Italy and apparently doesn't know much about Star Trek. As if he didn't have enough to deal with -- he's shadowing Frank (whom I haven't worked with since forever) as RP-2, and we're kicking that up a notch.
"You want to handle the walkthrough?" I ask him.
"No, no," he says.
I clap him on the shoulder and grin. "That's exactly why you're going to do it."
And he does. He's kind of shy, I think, so I feel for him. But it's the only way he'll grow. And you know what? He does a fine job with the difficult circumstances, proving himself a true engineer. Paolo doesn't know it, but Scotty would be proud.
[Next post: sol 555 (Opportunity sol 535), July 26.]
However, we're not done with the Scotty theme yet. The target names for thisol follow a Scottish theme (e.g., "Glasgow"). I go a bit nuts with it, attaching a picture of Scotty to our uplink report (as "today's guest rover driver") and adding a bunch of Trek-related comments to the sequence. The get-fine-attitude command becomes "remodulating the shield harmonics." One of my comments on a drive command speaks of going to warp 5. A little ripple we cross along the way is grandly named "DenebianSlimeDevil." And, of course, our destination target for the drive is named "FinalFrontier."
I think this puzzles Paolo Bellutta, who grew up in Italy and apparently doesn't know much about Star Trek. As if he didn't have enough to deal with -- he's shadowing Frank (whom I haven't worked with since forever) as RP-2, and we're kicking that up a notch.
"You want to handle the walkthrough?" I ask him.
"No, no," he says.
I clap him on the shoulder and grin. "That's exactly why you're going to do it."
And he does. He's kind of shy, I think, so I feel for him. But it's the only way he'll grow. And you know what? He does a fine job with the difficult circumstances, proving himself a true engineer. Paolo doesn't know it, but Scotty would be proud.
[Next post: sol 555 (Opportunity sol 535), July 26.]
2010-07-22
Opportunity Sol 530 (Spirit Sol 551)
After all that ... after all that work, we barely even moved. The first step of the drive was a 15-degree turn-in-place -- from 150 degrees to 165 degrees. Applying conservative limits in the spirit of our current rules of the road, we allowed 15 seconds for this turn. This should have been plenty of time, since the rover nominally turns at a rate of 2 degrees per second.
But there was something we didn't know: since the steering actuator stuck on Opportunity's right front wheel, she's been turning more slowly. More like 1 degree per second. On top of that, there's some fixed setup time (for example, to steer the other wheels to the right position). What with one thing and another, the turn timed out. So nothing else happened -- we just changed our heading a bit.
Just to pour salt in the wound, it timed out at a heading of 164.949 -- just 0.051 of a degree from the commanded heading. Another tenth of a second would have been plenty.
We discovered all this when the data came down yesterday afternoon. Since then I've been brooding about one of the decisions we made with this drive, specifically the decision to continue driving 20m after we cross Ripple B. The trick we're relying on to stop us when we get bogged down hasn't been tested -- what if there's some flaw in it that we hadn't thought of? We'd be pretty embarrassed -- and the subject of intense media and management scrutiny -- if we found that we'd actually dug in rather than stopping. Making this mistake at Purgatory, when we couldn't have predicted it, was one thing. But repeating the mistake would be very bad.
Based on this, I sent out email explaining what happened yesterday ("Executive summary: we are dumb, but Opportunity is fine and can proceed") and also suggesting that we shorten the drive when we redo it, and it's this email Jim Erickson is referring to when he stops me in the hallway this morning. "Why did you change your mind?"
I tell him, and he grins, nods, and then looks sharply at me. "What I don't want to see is that we lose our courage," he says. "We still have to do our job, which is to explore Mars."[1]
Ashitey has his own take on my email. "You're so hard on yourself! Back it off!" he scolds me. "How many sols have you saved? We all make mistakes on a daily basis. You're one of the best! Don't worry about it!"
The funny thing is, I rewrote the email to take out the self-flagellating tone. I thought.
Others are philosophical. "I don't turn as quickly as I used to, either," Squyres deadpans.
Instead of cutting the drive short because the slip-detector isn't tested, we could ... wait for it ... test the slip detector. Paolo and Jeng go down to the testbed to do that while I update the sequence based on what we've learned. It turns out that the slip-detector works fine -- which we all expected, but it's nice to have demonstrated the fact -- and we can go the distance.
Along the way, we're going to pass some interesting-looking flat rocky patches surrounded by pebbles -- the scientists refer to these as "cobblestones" -- which might be the signs that we're on the fringes of Erebus Highway territory. We won't stop and take pictures of them in the middle of the drive, we'll just take a low-quality image from the end position that will help us aim the higher-quality images we'll take before our next drive.
There are two of these patches, and they need names. As it happens, just before the issue comes up, I notice something in the news. Sadly, James Doohan -- better known to the world as Star Trek's "Scotty" -- died. Scotty was an icon for all engineers, and especially those of us in the space program. (One engineering school asked its students what got them interested in engineering. Half -- half -- said, "Scotty." They gave him an honorary engineering degree.) I suggest naming one of the little rocks after him, and the suggestion is received enthusiastically.
The other is named "Apollo," as today (20 July) is the anniversary of the first Moon landing. So we get to honor both real and fictional space exploration in one drive.
I hope it works.
Footnotes:
[1] Now that I'm the rover driver team lead, I have stolen that sentiment -- sometimes the exact line -- more times than I care to admit.
But there was something we didn't know: since the steering actuator stuck on Opportunity's right front wheel, she's been turning more slowly. More like 1 degree per second. On top of that, there's some fixed setup time (for example, to steer the other wheels to the right position). What with one thing and another, the turn timed out. So nothing else happened -- we just changed our heading a bit.
Just to pour salt in the wound, it timed out at a heading of 164.949 -- just 0.051 of a degree from the commanded heading. Another tenth of a second would have been plenty.
We discovered all this when the data came down yesterday afternoon. Since then I've been brooding about one of the decisions we made with this drive, specifically the decision to continue driving 20m after we cross Ripple B. The trick we're relying on to stop us when we get bogged down hasn't been tested -- what if there's some flaw in it that we hadn't thought of? We'd be pretty embarrassed -- and the subject of intense media and management scrutiny -- if we found that we'd actually dug in rather than stopping. Making this mistake at Purgatory, when we couldn't have predicted it, was one thing. But repeating the mistake would be very bad.
Based on this, I sent out email explaining what happened yesterday ("Executive summary: we are dumb, but Opportunity is fine and can proceed") and also suggesting that we shorten the drive when we redo it, and it's this email Jim Erickson is referring to when he stops me in the hallway this morning. "Why did you change your mind?"
I tell him, and he grins, nods, and then looks sharply at me. "What I don't want to see is that we lose our courage," he says. "We still have to do our job, which is to explore Mars."[1]
Ashitey has his own take on my email. "You're so hard on yourself! Back it off!" he scolds me. "How many sols have you saved? We all make mistakes on a daily basis. You're one of the best! Don't worry about it!"
The funny thing is, I rewrote the email to take out the self-flagellating tone. I thought.
Others are philosophical. "I don't turn as quickly as I used to, either," Squyres deadpans.
Instead of cutting the drive short because the slip-detector isn't tested, we could ... wait for it ... test the slip detector. Paolo and Jeng go down to the testbed to do that while I update the sequence based on what we've learned. It turns out that the slip-detector works fine -- which we all expected, but it's nice to have demonstrated the fact -- and we can go the distance.
Along the way, we're going to pass some interesting-looking flat rocky patches surrounded by pebbles -- the scientists refer to these as "cobblestones" -- which might be the signs that we're on the fringes of Erebus Highway territory. We won't stop and take pictures of them in the middle of the drive, we'll just take a low-quality image from the end position that will help us aim the higher-quality images we'll take before our next drive.
There are two of these patches, and they need names. As it happens, just before the issue comes up, I notice something in the news. Sadly, James Doohan -- better known to the world as Star Trek's "Scotty" -- died. Scotty was an icon for all engineers, and especially those of us in the space program. (One engineering school asked its students what got them interested in engineering. Half -- half -- said, "Scotty." They gave him an honorary engineering degree.) I suggest naming one of the little rocks after him, and the suggestion is received enthusiastically.
The other is named "Apollo," as today (20 July) is the anniversary of the first Moon landing. So we get to honor both real and fictional space exploration in one drive.
I hope it works.
Footnotes:
[1] Now that I'm the rover driver team lead, I have stolen that sentiment -- sometimes the exact line -- more times than I care to admit.
2010-07-19
Opportunity Sol 528 (Spirit Sol 549)
The weekend's drive went splendidly, racking up 32m. And we're poised to do it again, with a broad lane continuing before us.
Except for the speed bump. About 12m away from us is a decision point, where the trough sort of splits into two other troughs. Each of the branches (Left Fork and Right Fork) has a ripple in front of it, a ripple with geometry that's ominously similar to Purgatory's. Alternatively, we're sitting more or less at another fork in the road -- but the third choice that provides (Right-of-Right Fork) appears to end in a cul-de-sac with bad-looking stuff at the end. But we don't have to take any of those three paths. We could start ripple-hopping to the east, hoping to find a better trough.
Thus begins the Big Debate. Right-of-Right Fork gets thrown out pretty quickly, as just a way to store up trouble. Of Left Fork vs. Right Fork, Rob Sullivan's analysis shows that Left Fork's ripple is shorter but steeper, and angle is more worrisome than height, so that's a loser.
Which leaves us with Right Fork and ripple-hopping. The eventual decision is to prefer Right Fork: ripple-hopping affords no immediate progress toward our goal, is not guaranteed to provide a better alternative soon, and has dangers of its own. Besides, we're going to come across ripples like these sooner or later on the way to Erebus and, beyond it, Victoria. We might as well start learning to deal with them.
But still, that ripple ....
At least Jeng tells me something that makes me feel better about tackling it. The problem at Purgatory, he says, wasn't just the geometry of Purgatory itself. It was exacerbated by the fact that when we hit Purgatory, we were just descending another ripple (later called "North Ripple"). This meant the rover was initially angled down and thus "wanted" to bury itself in Purgatory. For this ripple, we'd be pretty much flat as we encountered it, which is a much more favorable geometry for climbing it.
So Jeng and I plan a route along the trough, while Rob Sullivan goes off and does more analysis and Beth Dewell calls Jim Erickson and John Callas. Jim and John are nervous about this, and also nervous about the fact that our planned path extends 20m beyond the ripple. This would mean that if our new slip-detection approach doesn't work, we could spin our wheels for 20m worth of driving in the ripple -- about half the damage we did at Purgatory, but much more than we'd be comfortable with. "You'd have a new Project Manager if you did that," Jim says grimly. "You say that like it's a bad thing," jokes John -- his deputy -- but Jim is serious. Though it didn't cost us a rover, Purgatory was a costly mistake, yet a forgivable one because it had never happened before and really was not foreseeable. But if we did it twice -- "Even I would wonder about my fitness for the job if it happened again," Jim says.
In the end they accept the drive as it stands, trusting our argument about the robustness of the slip check. Really, it's hard to get the slip checks to pass even when we're genuinely not slipping, depending as they do on the finicky mechanism of visodom used in a very difficult environment. If anything, the checks are conservative. At their suggestion, we mitigate the risk further by splitting the post-drive segment so that it effectively checks for stuckage an extra time.
Still, I'd feel a lot more comfortable if we were risking only the rover, and not also risking Jim's career.[1]
So we're done. Or not. While all this has been going on, Rob Sullivan's continued analysis turns up two other possible dangers past the initial ripple. A second ripple lies behind the first one and to its left, while a third lies a bit farther on and to the right. So even once we pass the first ripple, we've got some zig-zagging to do. I sigh, roll up my sleeves, and start hacking the sequence some more.
It's an awfully good thing Rob found those ripples -- it would really have sucked to sail past the first one, only to get stuck another couple of meters in. Our terrain mesh thisol is pretty poor for some reason, so the farther ripples didn't show up well; only his experienced geologist's eye gave us warning of the danger.
As if all that weren't enough, Rob saves the day once more. We decided to take "cleat movies" of our progress over the ripple -- small HAZCAM images subframed to the wheels, taken after each 50cm of commanded motion. Normally these are at a relatively low priority. But then we realize that taking these at a low priority is a mistake: if we do get bogged down, we won't get the images for another day, and that will be an awfully tense day. So we'd really like to get them sooner, but by the time we realize this, it's so late on the East Coast that the person who's normally responsible for this has had to go home. But Rob knows the job, so with his usual good humor and great attitude, he stays late and fixes them for us. He even sticks around through the CAM to make sure everything is ready for the spacecraft.
You know how there was a Fifth Beatle? Rob is, like, the (n+1)th Rover Planner. I love that guy.[2]
There were many obstacles thisol, figuratively and literally, but we got some good news, too. We had another cleaning event -- the dust devils worked their magic -- and we now have another 100 Watt-hours to play with each sol. Thanks, Mars!
[Next post: sol 551 (Opportunity sol 530), July 22.]
Footnotes:
[1] But then, this is why Jim was such an awesome guy: even feeling that his career was on the line, he trusted us unless he had an articulable, rational reason not to.
[2] Much later, I even gave him an official Rover Planner T-shirt. Yes, there really are official Rover Planner T-shirts. And no, you can't buy one.
Except for the speed bump. About 12m away from us is a decision point, where the trough sort of splits into two other troughs. Each of the branches (Left Fork and Right Fork) has a ripple in front of it, a ripple with geometry that's ominously similar to Purgatory's. Alternatively, we're sitting more or less at another fork in the road -- but the third choice that provides (Right-of-Right Fork) appears to end in a cul-de-sac with bad-looking stuff at the end. But we don't have to take any of those three paths. We could start ripple-hopping to the east, hoping to find a better trough.
Thus begins the Big Debate. Right-of-Right Fork gets thrown out pretty quickly, as just a way to store up trouble. Of Left Fork vs. Right Fork, Rob Sullivan's analysis shows that Left Fork's ripple is shorter but steeper, and angle is more worrisome than height, so that's a loser.
Which leaves us with Right Fork and ripple-hopping. The eventual decision is to prefer Right Fork: ripple-hopping affords no immediate progress toward our goal, is not guaranteed to provide a better alternative soon, and has dangers of its own. Besides, we're going to come across ripples like these sooner or later on the way to Erebus and, beyond it, Victoria. We might as well start learning to deal with them.
But still, that ripple ....
At least Jeng tells me something that makes me feel better about tackling it. The problem at Purgatory, he says, wasn't just the geometry of Purgatory itself. It was exacerbated by the fact that when we hit Purgatory, we were just descending another ripple (later called "North Ripple"). This meant the rover was initially angled down and thus "wanted" to bury itself in Purgatory. For this ripple, we'd be pretty much flat as we encountered it, which is a much more favorable geometry for climbing it.
So Jeng and I plan a route along the trough, while Rob Sullivan goes off and does more analysis and Beth Dewell calls Jim Erickson and John Callas. Jim and John are nervous about this, and also nervous about the fact that our planned path extends 20m beyond the ripple. This would mean that if our new slip-detection approach doesn't work, we could spin our wheels for 20m worth of driving in the ripple -- about half the damage we did at Purgatory, but much more than we'd be comfortable with. "You'd have a new Project Manager if you did that," Jim says grimly. "You say that like it's a bad thing," jokes John -- his deputy -- but Jim is serious. Though it didn't cost us a rover, Purgatory was a costly mistake, yet a forgivable one because it had never happened before and really was not foreseeable. But if we did it twice -- "Even I would wonder about my fitness for the job if it happened again," Jim says.
In the end they accept the drive as it stands, trusting our argument about the robustness of the slip check. Really, it's hard to get the slip checks to pass even when we're genuinely not slipping, depending as they do on the finicky mechanism of visodom used in a very difficult environment. If anything, the checks are conservative. At their suggestion, we mitigate the risk further by splitting the post-drive segment so that it effectively checks for stuckage an extra time.
Still, I'd feel a lot more comfortable if we were risking only the rover, and not also risking Jim's career.[1]
So we're done. Or not. While all this has been going on, Rob Sullivan's continued analysis turns up two other possible dangers past the initial ripple. A second ripple lies behind the first one and to its left, while a third lies a bit farther on and to the right. So even once we pass the first ripple, we've got some zig-zagging to do. I sigh, roll up my sleeves, and start hacking the sequence some more.
It's an awfully good thing Rob found those ripples -- it would really have sucked to sail past the first one, only to get stuck another couple of meters in. Our terrain mesh thisol is pretty poor for some reason, so the farther ripples didn't show up well; only his experienced geologist's eye gave us warning of the danger.
As if all that weren't enough, Rob saves the day once more. We decided to take "cleat movies" of our progress over the ripple -- small HAZCAM images subframed to the wheels, taken after each 50cm of commanded motion. Normally these are at a relatively low priority. But then we realize that taking these at a low priority is a mistake: if we do get bogged down, we won't get the images for another day, and that will be an awfully tense day. So we'd really like to get them sooner, but by the time we realize this, it's so late on the East Coast that the person who's normally responsible for this has had to go home. But Rob knows the job, so with his usual good humor and great attitude, he stays late and fixes them for us. He even sticks around through the CAM to make sure everything is ready for the spacecraft.
You know how there was a Fifth Beatle? Rob is, like, the (n+1)th Rover Planner. I love that guy.[2]
There were many obstacles thisol, figuratively and literally, but we got some good news, too. We had another cleaning event -- the dust devils worked their magic -- and we now have another 100 Watt-hours to play with each sol. Thanks, Mars!
[Next post: sol 551 (Opportunity sol 530), July 22.]
Footnotes:
[1] But then, this is why Jim was such an awesome guy: even feeling that his career was on the line, he trusted us unless he had an articulable, rational reason not to.
[2] Much later, I even gave him an official Rover Planner T-shirt. Yes, there really are official Rover Planner T-shirts. And no, you can't buy one.
2010-07-16
Opportunity Sol 525 (Spirit Sol 546)
I envy Cooper and Jeng. Yestersol they were pushed pretty hard to improve our rate of travel while remaining compliant with the Rules of the Road, and they came up with a good one (along with Mark Maimone). Essentially, instead of using visodom at 40cm intervals throughout the drive, they drive 5m at a normal rate and then did a clever slip check using visodom over the next 40cm. This does two or three visodom updates every 5.4m of travel, rather than the 13 or so we were doing over the same distance. This let them greatly increase our traverse distance; now we can attain the full 30m/sol we're currently allowed by the Rules, and do it in only an hour and a half or so.[1]
I hate myself for not thinking of this. And I came in in such a good mood. Well, Jeng and I are going to pull the same trick thisol, and any sol where we have a nice long trough to drive in.
But maybe we won't drive south along the trough thisol. According to the orbital mesh, we're only 25m or so from the Erebus Highway. Driving there would mean driving at an azimuth of about 140deg -- across the ripples, not along the troughs, which would significantly limit our distance each sol.
I start looking into whether this is a sensible approach. What I find is that the images from our present location don't seem to agree with the orbital mesh. In our current images, we can't see anything that looks like Erebus Highway closer than 100m or so, putting it at least 6 driving sols away. (More actual sols than that, since we can't drive every sol over weekends and we're coming into restricted sols soon.) And even that's a gamble: it might be farther away, and it might not even speed up our traverse once we do get there. Meanwhile, we believe we can drive 30m per sol straight south toward Erebus itself. We talk it over as a team and decide on the direct route. But we're going to keep our eyes open, and if we see nearby Erebus Highway material, we'll consider going for it. If nothing else, we eventually cross something that looks like Erebus Highway -- Rob Sullivan calls it the Erebus Parking Lot -- even if we just head straight south.
So that's the plan.
Somehow we missed a major milestone. That is, a kilometerstone. Recently, the rovers' combined odometry crossed the 10km mark. And climbing.
[Next post: sol 549 (Opportunity sol 528), July 19.]
Footnotes:
[1] A close descendant of this technique is used to check Opportunity's slip to this day. We also check slip only every 20m or so now, rather than every 5m, so the expense is even further reduced.
I hate myself for not thinking of this. And I came in in such a good mood. Well, Jeng and I are going to pull the same trick thisol, and any sol where we have a nice long trough to drive in.
But maybe we won't drive south along the trough thisol. According to the orbital mesh, we're only 25m or so from the Erebus Highway. Driving there would mean driving at an azimuth of about 140deg -- across the ripples, not along the troughs, which would significantly limit our distance each sol.
I start looking into whether this is a sensible approach. What I find is that the images from our present location don't seem to agree with the orbital mesh. In our current images, we can't see anything that looks like Erebus Highway closer than 100m or so, putting it at least 6 driving sols away. (More actual sols than that, since we can't drive every sol over weekends and we're coming into restricted sols soon.) And even that's a gamble: it might be farther away, and it might not even speed up our traverse once we do get there. Meanwhile, we believe we can drive 30m per sol straight south toward Erebus itself. We talk it over as a team and decide on the direct route. But we're going to keep our eyes open, and if we see nearby Erebus Highway material, we'll consider going for it. If nothing else, we eventually cross something that looks like Erebus Highway -- Rob Sullivan calls it the Erebus Parking Lot -- even if we just head straight south.
So that's the plan.
Somehow we missed a major milestone. That is, a kilometerstone. Recently, the rovers' combined odometry crossed the 10km mark. And climbing.
[Next post: sol 549 (Opportunity sol 528), July 19.]
Footnotes:
[1] A close descendant of this technique is used to check Opportunity's slip to this day. We also check slip only every 20m or so now, rather than every 5m, so the expense is even further reduced.
2010-07-14
Opportunity Sol 523 (Spirit Sol 544)
I was worried about yestersol's drive -- needlessly, as it turns out. (I can't believe I'm still doing that.) It went damn near perfectly, putting us about 10cm from our goal.
Now that we're here, it looks like the way ahead of us is not scary after all. Quite the contrary: with only the FHAZ and a single NCAM wedge to go by, we can drive about 20m down a nice smooth lane, with only a bit of leftward curve to follow the trough.
John Callas has brought a couple of blind students in to observe. I have some time before the SOWG starts, so I talk with them a little. They're eager and enthusiastic (who can blame them?). John told me about them the other day, floating the idea of having one or more of them shadow a rover driver for a while. I told him I'd be happy to help with this, but wasn't sure exactly how we'd achieve that -- it's a very visual job. But we'll figure something out. In the meanwhile, they're going to help make our Web pages more accessible to blind people, or something.
Since the way ahead of us is clear, the SOWG grants us even more time to drive -- three and a half hours thisol. We'll go about 20m. The only thing I dislike about this decision is that it makes it even worse if the drive fails, because so much science gets pushed out to make way for it. But I don't think they really care that much; there's very little good science they can do before we get to the Erebus Highway anyhow, so they're actually doing themselves a favor by letting us drive as much as we can each sol.
When we look at the planned drive in an overhead map, it looks as if we could potentially make it to the Erebus Highway in just two or three more sols. We won't -- if only because we've run out of tricks for working around the data-products limit and might need to shorten the next drive to get some crap downlinked -- but we're getting close. For several sols, we've glimpsed it -- maybe -- in the distance, in the NCAMs. Very, very close.
[Next post: sol 546 (Opportunity sol 525), July 16.]
Now that we're here, it looks like the way ahead of us is not scary after all. Quite the contrary: with only the FHAZ and a single NCAM wedge to go by, we can drive about 20m down a nice smooth lane, with only a bit of leftward curve to follow the trough.
John Callas has brought a couple of blind students in to observe. I have some time before the SOWG starts, so I talk with them a little. They're eager and enthusiastic (who can blame them?). John told me about them the other day, floating the idea of having one or more of them shadow a rover driver for a while. I told him I'd be happy to help with this, but wasn't sure exactly how we'd achieve that -- it's a very visual job. But we'll figure something out. In the meanwhile, they're going to help make our Web pages more accessible to blind people, or something.
Since the way ahead of us is clear, the SOWG grants us even more time to drive -- three and a half hours thisol. We'll go about 20m. The only thing I dislike about this decision is that it makes it even worse if the drive fails, because so much science gets pushed out to make way for it. But I don't think they really care that much; there's very little good science they can do before we get to the Erebus Highway anyhow, so they're actually doing themselves a favor by letting us drive as much as we can each sol.
When we look at the planned drive in an overhead map, it looks as if we could potentially make it to the Erebus Highway in just two or three more sols. We won't -- if only because we've run out of tricks for working around the data-products limit and might need to shorten the next drive to get some crap downlinked -- but we're getting close. For several sols, we've glimpsed it -- maybe -- in the distance, in the NCAMs. Very, very close.
[Next post: sol 546 (Opportunity sol 525), July 16.]
2010-07-13
Opportunity Sol 522 (Spirit Sol 543)
Surprisingly, visodom performed well through the entire drive yestersol. We'd been worried visodom wouldn't work reliably with nothing but our own tracks for it to look at, but so far, so good.
The drive itself went where it was supposed to. We're at the end of one trough, and thisol our plan is to hop into the next one (about 5m worth of driving, in a lazy "S") and drive about 10m along it, to a point where the terrain seems to fall off slightly. If we could see it better in the NCAM coverage, we might be able to drive farther -- or so I think. Later the PCAMs arrive, and they show that, yep, the terrain appears to fall off a bit there. Oh, well; we're using up all the time we've got anyway. (Visodom works, but it's slow! This 15m of driving will take us about 2.5 hours. We'll actually spend maybe 10 minutes turning the wheels; all the rest of that is visodom overhead.)
It's Rob Sullivan's first sol as SOWG chair. He does a great job, but at one point, when he's having trouble reaching consensus on what science should be slotted into the tiny amount of time left over after driving, he passes on a bit of wisdom: "One of the things Steve told me was that you've failed as SOWG chair if you have to decide rather than reach consensus." They reach consensus.
We're settling on a drive template, which makes things easier. Between the enormous setup block and the smaller teardown block, we cut the drive into a series of segments each about 5m long. Between segments, we reseed visodom -- slow, but necessary to work around a flight software bug that causes visodom's estimated roll and pitch errors to accumulate, which can abort the drive. And at the start of each segment, we zero the failure count and choose a new max failure count for that segment. This helps us play by the Rules of the Road, which demand that we catch slip -- if we've bogged down entirely, visodom will fail to converge and the error limit will stop us. (We can't just choose a single count large enough for the entire drive, because then we can dig in a lot more before stopping. The approach we've come up with uses several smaller counts instead, making the trigger more reliable.)
I also remember something we forgot on yestersol's drive: at the end of the drive, we have to straighten the wheels. The end of yestersol's drive included a last-minute comm turn, which left the wheels cocked in. There's nothing inherently wrong with this, but if another steering actuator should fail on this vehicle, we want it to "fail straight," since that's the least bad configuration for continued driving. I've made quite a point of this in my "Driving a Mars Rover for Dummies" talk -- which I've given twice publicly now -- so I'd feel like twice as much of a dumbass if I forgot this on some sol when it really mattered!
The drive itself went where it was supposed to. We're at the end of one trough, and thisol our plan is to hop into the next one (about 5m worth of driving, in a lazy "S") and drive about 10m along it, to a point where the terrain seems to fall off slightly. If we could see it better in the NCAM coverage, we might be able to drive farther -- or so I think. Later the PCAMs arrive, and they show that, yep, the terrain appears to fall off a bit there. Oh, well; we're using up all the time we've got anyway. (Visodom works, but it's slow! This 15m of driving will take us about 2.5 hours. We'll actually spend maybe 10 minutes turning the wheels; all the rest of that is visodom overhead.)
It's Rob Sullivan's first sol as SOWG chair. He does a great job, but at one point, when he's having trouble reaching consensus on what science should be slotted into the tiny amount of time left over after driving, he passes on a bit of wisdom: "One of the things Steve told me was that you've failed as SOWG chair if you have to decide rather than reach consensus." They reach consensus.
We're settling on a drive template, which makes things easier. Between the enormous setup block and the smaller teardown block, we cut the drive into a series of segments each about 5m long. Between segments, we reseed visodom -- slow, but necessary to work around a flight software bug that causes visodom's estimated roll and pitch errors to accumulate, which can abort the drive. And at the start of each segment, we zero the failure count and choose a new max failure count for that segment. This helps us play by the Rules of the Road, which demand that we catch slip -- if we've bogged down entirely, visodom will fail to converge and the error limit will stop us. (We can't just choose a single count large enough for the entire drive, because then we can dig in a lot more before stopping. The approach we've come up with uses several smaller counts instead, making the trigger more reliable.)
I also remember something we forgot on yestersol's drive: at the end of the drive, we have to straighten the wheels. The end of yestersol's drive included a last-minute comm turn, which left the wheels cocked in. There's nothing inherently wrong with this, but if another steering actuator should fail on this vehicle, we want it to "fail straight," since that's the least bad configuration for continued driving. I've made quite a point of this in my "Driving a Mars Rover for Dummies" talk -- which I've given twice publicly now -- so I'd feel like twice as much of a dumbass if I forgot this on some sol when it really mattered!
2010-07-12
Opportunity Sol 521 (Spirit Sol 542)
I come in early to get a jump on things. This was a mistake -- the downlink's been delayed (Spirit's data was queued ahead of ours on ODY), so there's nothing for me to get a jump on. At least it's being delayed for a good reason. Spirit has an earlier start and a tighter deadline, and they're also downlinking part of a huge panorama they took over the Independence Day weekend. I see part of this as it's coming down, and it's clear the full version will be spectacular.
Two hours later, our data starts flowing. The news isn't great, but it's okay. We'd planned about a 16m traverse, and we actually made about 10m progress. We'd have made more, but visual odometry -- which we rely upon for our slip checks -- produced a couple of erroneous results, making the rover think it was slipping a lot more than it actually was. As it happens, the rover thought it was slipping 32.5%, just barely over the 30% limit then in effect. So it stopped short.
Thisol we're just carrying on. The rover hasn't dug in and isn't in any apparent danger, so there's no reason not to just shoot for another 15m or so. The project issued new Rules of the Road that allow us to accept up to 40% slip, but this makes me a little nervous. The good slip estimates -- the ones produced during the larger portion of the drive, where visodom was working normally -- were in the neighborhood of 5% or less. So wouldn't it be better to actually lower the allowed slip percentage? We could set it to, say, 20% -- well above what we actually expect a good update to produce -- so that if visodom suddenly produces a bogus update that exceeds that threshold, the rover will stop moving instead of thinking it's somewhere it's not and driving into a ripple instead of along the trough.
But Jeng talks me out of this. The rover can actually climb these ripples, he points out. As long as it keeps heading more or less south -- there's essentially no chance of anything else, bogus update or no bogus update -- it'll be doing the right thing. And if it should get bogged down, visodom will fail to converge (because it doesn't expect the terrain not to change, an interesting result I'd never thought of). If that happens, the visodom failure-count limit will be exceeded, and the rover will stop shortly after.
The other thing that worries me is that we don't really know why the bogus updates occurred. We didn't get much data down; in particular, the data that would let us debug this problem is still aboard Opportunity. So if it was related to the unusual stuff we're doing with visodom -- looking over our right shoulder at the tracks we're leaving behind ourselves -- we'll just get more bogus updates at the start of this drive and fail again, probably without making much real progress. But with no debugging data available until tomorrow and no other real options anyway, there's little else we can do but cross our fingers and continue along the Blue Line toward Erebus.
Two hours later, our data starts flowing. The news isn't great, but it's okay. We'd planned about a 16m traverse, and we actually made about 10m progress. We'd have made more, but visual odometry -- which we rely upon for our slip checks -- produced a couple of erroneous results, making the rover think it was slipping a lot more than it actually was. As it happens, the rover thought it was slipping 32.5%, just barely over the 30% limit then in effect. So it stopped short.
Thisol we're just carrying on. The rover hasn't dug in and isn't in any apparent danger, so there's no reason not to just shoot for another 15m or so. The project issued new Rules of the Road that allow us to accept up to 40% slip, but this makes me a little nervous. The good slip estimates -- the ones produced during the larger portion of the drive, where visodom was working normally -- were in the neighborhood of 5% or less. So wouldn't it be better to actually lower the allowed slip percentage? We could set it to, say, 20% -- well above what we actually expect a good update to produce -- so that if visodom suddenly produces a bogus update that exceeds that threshold, the rover will stop moving instead of thinking it's somewhere it's not and driving into a ripple instead of along the trough.
But Jeng talks me out of this. The rover can actually climb these ripples, he points out. As long as it keeps heading more or less south -- there's essentially no chance of anything else, bogus update or no bogus update -- it'll be doing the right thing. And if it should get bogged down, visodom will fail to converge (because it doesn't expect the terrain not to change, an interesting result I'd never thought of). If that happens, the visodom failure-count limit will be exceeded, and the rover will stop shortly after.
The other thing that worries me is that we don't really know why the bogus updates occurred. We didn't get much data down; in particular, the data that would let us debug this problem is still aboard Opportunity. So if it was related to the unusual stuff we're doing with visodom -- looking over our right shoulder at the tracks we're leaving behind ourselves -- we'll just get more bogus updates at the start of this drive and fail again, probably without making much real progress. But with no debugging data available until tomorrow and no other real options anyway, there's little else we can do but cross our fingers and continue along the Blue Line toward Erebus.
2010-07-09
Opportunity Sol 518 (Spirit Sol 539)
Somewhat to my surprise, the drive executed quite well. We made the whole 15m, right along the intended path, so we're just continuing along the Blue Line today.
The drive even went faster than we'd thought, executing in only an hour and a half, as opposed to the two and a half we expected. But it turns out this is due to unintentional cheating. When we used to charge along the plains, we used step-skipping -- if autonav saw a sufficiently benign path, the rover could take a couple of extra steps without updating its world map. We don't want to be doing that now, but we forgot to turn it off, so the rover didn't pay as close attention to the drive as it was supposed to. That's why it went faster. Unfortunately, we'll have to turn off step-skipping as of thisol.
We also continue to face a data-products crunch. The problem right now is not the amount of data on the rover, but the number of files, a separate limit. The kind of driving we've been doing generates huge numbers of data products, and our downlink has been fairly poor, so today we have a choice to make. We can either cut back the verbosity of the drive data -- undesirable, because then if something goes wrong, we may lack the data to diagnose it; or delete yestersol's output without downlinking it -- but then we won't know why visodom didn't converge in some cases where we expected it to; or not drive -- but nobody wants that; or find a bunch of other stuff we can delete. They go with the last one.
Thisol brings a revision to the Rules of the Road. In short, they're relaxing. We can now accept 40% slip, rather than the 25% that had been the previous rule (or the 30% we'd actually been targeting). And tilt limits and max drive distances are no longer explicitly specified in most cases; we're now supposed to use "appropriate" figures. This is good and bad -- on the one hand, it means the RPs will take more of the heat if we make a poor choice. But on the upside, we get to make the choices, which is how I think it should be. And it's a reflection of our overall excellent performance since egressing Purgatory. So, hell, you look at it the right way, it's a compliment. Can't complain about that.
[Next post: sol 542 (Opportunity sol 521), July 12.]
The drive even went faster than we'd thought, executing in only an hour and a half, as opposed to the two and a half we expected. But it turns out this is due to unintentional cheating. When we used to charge along the plains, we used step-skipping -- if autonav saw a sufficiently benign path, the rover could take a couple of extra steps without updating its world map. We don't want to be doing that now, but we forgot to turn it off, so the rover didn't pay as close attention to the drive as it was supposed to. That's why it went faster. Unfortunately, we'll have to turn off step-skipping as of thisol.
We also continue to face a data-products crunch. The problem right now is not the amount of data on the rover, but the number of files, a separate limit. The kind of driving we've been doing generates huge numbers of data products, and our downlink has been fairly poor, so today we have a choice to make. We can either cut back the verbosity of the drive data -- undesirable, because then if something goes wrong, we may lack the data to diagnose it; or delete yestersol's output without downlinking it -- but then we won't know why visodom didn't converge in some cases where we expected it to; or not drive -- but nobody wants that; or find a bunch of other stuff we can delete. They go with the last one.
Thisol brings a revision to the Rules of the Road. In short, they're relaxing. We can now accept 40% slip, rather than the 25% that had been the previous rule (or the 30% we'd actually been targeting). And tilt limits and max drive distances are no longer explicitly specified in most cases; we're now supposed to use "appropriate" figures. This is good and bad -- on the one hand, it means the RPs will take more of the heat if we make a poor choice. But on the upside, we get to make the choices, which is how I think it should be. And it's a reflection of our overall excellent performance since egressing Purgatory. So, hell, you look at it the right way, it's a compliment. Can't complain about that.
[Next post: sol 542 (Opportunity sol 521), July 12.]
2010-07-08
Opportunity Sol 517 (Spirit Sol 538)
I've got my clock/radio tuned to the news station, so I'm literally awakened by the news that my favorite city in the world, London, was viciously attacked by barbarous cowards today. Bombs on the subway, and on one of those iconic red double-decker buses. Dozens of people are known dead, hundreds more probably dead. Maybe a thousand injured. Nobody really knows just how bad it is yet, but it's bad.
Not for the first time, I think to myself that I'm making a mistake. As much as I love it, I should leave Mars and go work in counter-terrorism in some capacity. I have a moral responsibility to act. I can't solve it all myself, but I should do my tiny little part rather than fuck around with an interplanetary Erector Set.
I usually tell myself that my job actually is important, that people need something to live for and we're part of providing that. But on a day like today, that justification -- I should call it a rationalization -- sounds awfully damn hollow.
I don't know what to do. I love you, London. I go to work.
Our drive onto the on-ramp, sol before yestersol, went fine. Yesterday Cooper and Jeff took us to the other end of the on-ramp; today we drive onto the freeway -- or East Path, or the Blue Line, as I've started calling it (from the blue lines in Rob Sullivan's images, designating the path we eventually chose).
Today's plan is to make about 15m total distance, about 10m of which will actually be along the Blue Line. Over the weekend, we might do some IDD work on the magnets or the solar panels -- or we might just do one drive followed by light remote sensing, to let us clear flash a bit.
Flash is a big worry, because our downlinks haven't been that great. Thisol we're looking at a meager 46 Mbits, for example. And since the drive and associated post-drive imaging consume about 35 Mbits of that -- maybe more -- there's not a lot for the science team to squabble over. The plan that comes out of the SOWG is heavy on the drive, light on science. And that's even after we help out by cutting back the motor/IMU data-collection rate from 8 Hz to to 4 Hz, nice fellas that we are.
While Khaled works on the drive, I scramble around, looking into a number of issues we need to resolve for thisol's plan. First priority is to meet with the Spirit rover planners to understand why they considered the limit-cycle check (which stops drives if the vehicle detects excessive slip) a hair-trigger check on their drives. I had formed the impression that the limit-cycle check wasn't very reliable, but it turns out I was wrong; it's reliable, they just didn't want to use it to detect slip much under 100%. This is good news; it means we can use it on Opportunity to detect slip in the neighborhood of 30%, which is all we're allowed.[1]
Next thing is to look into the suspension and tilt limits. I graph the data for the last several drives to try to understand how we're articulating when driving in this terrain. The idea is to set limits that are a bit higher than we expect, and it turns out that we're already setting them pretty close to the right numbers.
Finally, I have to scare up a Mobility/IDD person to do some longer-term analysis. I've generally been wanting them to do more of this. We ought to know more about how the vehicle has been performing -- how close IDD placements have been to predictions, how well visual odometry has been performing, and so on. During the nominal mission, those guys spent a lot of time at such tasks as these, but not any more. So I figure I should set them tasks as they occur to me; there's plenty they could be doing, but they don't always know what we want. But it turns out that the Mobility/IDD guy on shift today is Marcel -- who's from London, and has family there. I let it go.
Khaled and I spend some time working on the drive together, and then he works out the tricky visodom pointing with Mark while I take care of a visitor -- another friend of Susan Kurtik's, someone's foster child or something.[2] Like most kids, she gets into playing around with the rover, zooming the camera around and dragging it all over the terrain.
It's been a hectic day, and it doesn't stop at the CAM. Turns out they messed up an ODY comm window request or something, and instead of the meager 46 Mbits we thought we were getting, we're going to have only 27 Mbits. Oops. That's not even enough for the critical drive data. People start scrambling, and luckily there's an ODY ACE on console who's able to command the spacecraft to accept more data. So we're back up to 46. Lucky us.
A hundred million miles away, a long way from all this pain and death and worry, a plucky little robot trundles slowly toward its next destination. And so it goes.
Footnotes:
[1] Briefly put, the limit-cycle check periodically looks to see how far the rover has actually progressed versus the distance commanded. When you're driving with visodom, this is an easy-to-sequence slip check -- though expensive because visodom driving itself is expensive. Later, as our paranoia levels subsided slightly, we switched to a much less expensive slip-check technique.
[2] Susan Kurtik, as you'll recall, is the woman who hired me at JPL in the first place, recruiting me from the University of Illinois at Urbana-Champaign. Whatever Susan asks me for, Susan gets.
Not for the first time, I think to myself that I'm making a mistake. As much as I love it, I should leave Mars and go work in counter-terrorism in some capacity. I have a moral responsibility to act. I can't solve it all myself, but I should do my tiny little part rather than fuck around with an interplanetary Erector Set.
I usually tell myself that my job actually is important, that people need something to live for and we're part of providing that. But on a day like today, that justification -- I should call it a rationalization -- sounds awfully damn hollow.
I don't know what to do. I love you, London. I go to work.
Our drive onto the on-ramp, sol before yestersol, went fine. Yesterday Cooper and Jeff took us to the other end of the on-ramp; today we drive onto the freeway -- or East Path, or the Blue Line, as I've started calling it (from the blue lines in Rob Sullivan's images, designating the path we eventually chose).
Today's plan is to make about 15m total distance, about 10m of which will actually be along the Blue Line. Over the weekend, we might do some IDD work on the magnets or the solar panels -- or we might just do one drive followed by light remote sensing, to let us clear flash a bit.
Flash is a big worry, because our downlinks haven't been that great. Thisol we're looking at a meager 46 Mbits, for example. And since the drive and associated post-drive imaging consume about 35 Mbits of that -- maybe more -- there's not a lot for the science team to squabble over. The plan that comes out of the SOWG is heavy on the drive, light on science. And that's even after we help out by cutting back the motor/IMU data-collection rate from 8 Hz to to 4 Hz, nice fellas that we are.
While Khaled works on the drive, I scramble around, looking into a number of issues we need to resolve for thisol's plan. First priority is to meet with the Spirit rover planners to understand why they considered the limit-cycle check (which stops drives if the vehicle detects excessive slip) a hair-trigger check on their drives. I had formed the impression that the limit-cycle check wasn't very reliable, but it turns out I was wrong; it's reliable, they just didn't want to use it to detect slip much under 100%. This is good news; it means we can use it on Opportunity to detect slip in the neighborhood of 30%, which is all we're allowed.[1]
Next thing is to look into the suspension and tilt limits. I graph the data for the last several drives to try to understand how we're articulating when driving in this terrain. The idea is to set limits that are a bit higher than we expect, and it turns out that we're already setting them pretty close to the right numbers.
Finally, I have to scare up a Mobility/IDD person to do some longer-term analysis. I've generally been wanting them to do more of this. We ought to know more about how the vehicle has been performing -- how close IDD placements have been to predictions, how well visual odometry has been performing, and so on. During the nominal mission, those guys spent a lot of time at such tasks as these, but not any more. So I figure I should set them tasks as they occur to me; there's plenty they could be doing, but they don't always know what we want. But it turns out that the Mobility/IDD guy on shift today is Marcel -- who's from London, and has family there. I let it go.
Khaled and I spend some time working on the drive together, and then he works out the tricky visodom pointing with Mark while I take care of a visitor -- another friend of Susan Kurtik's, someone's foster child or something.[2] Like most kids, she gets into playing around with the rover, zooming the camera around and dragging it all over the terrain.
It's been a hectic day, and it doesn't stop at the CAM. Turns out they messed up an ODY comm window request or something, and instead of the meager 46 Mbits we thought we were getting, we're going to have only 27 Mbits. Oops. That's not even enough for the critical drive data. People start scrambling, and luckily there's an ODY ACE on console who's able to command the spacecraft to accept more data. So we're back up to 46. Lucky us.
A hundred million miles away, a long way from all this pain and death and worry, a plucky little robot trundles slowly toward its next destination. And so it goes.
Footnotes:
[1] Briefly put, the limit-cycle check periodically looks to see how far the rover has actually progressed versus the distance commanded. When you're driving with visodom, this is an easy-to-sequence slip check -- though expensive because visodom driving itself is expensive. Later, as our paranoia levels subsided slightly, we switched to a much less expensive slip-check technique.
[2] Susan Kurtik, as you'll recall, is the woman who hired me at JPL in the first place, recruiting me from the University of Illinois at Urbana-Champaign. Whatever Susan asks me for, Susan gets.
2010-07-06
Opportunity Sol 515 (Spirit Sol 536)
It's the fifth of July. Jim Erickson had asked me to check out the drive results yesterday so that we'd have a coherent story going into the planning meeting today. I did that, and I come in half an hour early today, to boot, but somehow we still don't have a coherent story.
When Steve Squyres polls the RPs (and we all know how painful that can be), we're not ready. "We're still arguing," is all I say on mike. He laughs and tells us we can have a few more minutes.
Our overall plan had been to drive north to positions where we could evaluate two south-leading paths. One of these is West Path, which tries to take us west around Purgatory Ripple; the other is East Path. The first half of that drive, which took us to where we could evaluate West Path, occurred over the weekend. We were going to do the second half thisol, but -- we eventually agree -- we seem to have good enough imaging for both paths from our present position. (Steve is visibly overjoyed to hear this -- he really didn't want to spend another sol just figuring out where we'd go.)
So much for the easy part. But the real kicker is, do we go east or west? Since the part of Erebus we're heading for is southeast of us, it would seem like a no-brainer to choose East Path -- if we choose West Path, we're gambling that we'll eventually find a way to ripple-hop east, and we don't like to gamble. But the ripples to the east of our current position look more fearsome than their westward counterparts, making the gamble look more attractive.
Just as Steve gets back to us, though, we find a pretty good eastward solution. So we have an answer, albeit just in the nick of time. East Path wins.
As hard as it was to reach that answer, that was cake compared to the next part. The East Path on-ramp is fairly narrow, not much wider than the rover's wheel base, and it's "guarded" by a low ripple. The most appealing way to enter the on-ramp (the way that takes us directly across the lowest part of the ripple) puts us at an extremely poor comm heading -- possibly in the PMA exclusion zone, which we're supposed to try to avoid because it means the rover's trying to radiate a signal through the mast. (Which works, but it further reduces the quality of an already poor comm pass.)
We solve it by doing some stuff we're not supposed to do: we drive straight north until we're just west of the on-ramp, then turn in place and scoot directly backward (east) across the ripple. The bad things we're doing include crossing the ripple at an oblique angle, turning in place (Opportunity has a bad steering actuator in one wheel, which hinders turning in place), and driving backward instead of forward across the ripple (so if we should get stuck, we'll have a little less strength if we have to drive out forward rather than backward). But this puts us at a much better comm heading, which we need; and anyway, this is a relatively low ripple, so we shouldn't bog down in it. I just hope those aren't famous last words.
We still have one more problem. According to our new Rules of the Road, we're not supposed to drive more than 5m in one sol unless we're monitoring slip to ensure that we're not slipping more than 25%. With the way we've written the sequence, we can't check our slip precisely enough to meet this requirement. Our choices are to rewrite the sequence in a way that will make it less likely to correctly execute the drive, or cut the drive short, so that we won't make it onto the on-ramp -- or waive the rule.
John Callas, who's the acting project manager today, is extremely reluctant to waive the rule. "It's not how we said we were going to drive here," he points out. "If we're not going to do what we said we were going to do, we should be willing to change what we said we were going to do."
He's got a real point. Cindy Oda tries to get him to let us slide: "Can you just let us do it this way today, and tomorrow we'll do it whatever way you want to?" But I don't want to put John in that position, and he doesn't like it either: "So you've already got your hand in the cookie jar, so why not just let you have this cookie?" he grins.
Still, it looks to me as if John is feeling the pressure from the team, but he needs a face-saving way out. In the end, Jake Matijevic rides to the rescue. "This isn't the place to apply the Rules of the Road," he points out gently. "They were meant for terrain we haven't been over before. Here, we're just following along our old tracks, except at the very end, which is only a couple of meters of driving -- a distance we'd be willing to tolerate anyway." This gets John off the hook, and it has the extra added benefit of being true. So we're heading for the on-ramp at last.
[Next post: sol 538 (Opportunity sol 517), July 8.]
When Steve Squyres polls the RPs (and we all know how painful that can be), we're not ready. "We're still arguing," is all I say on mike. He laughs and tells us we can have a few more minutes.
Our overall plan had been to drive north to positions where we could evaluate two south-leading paths. One of these is West Path, which tries to take us west around Purgatory Ripple; the other is East Path. The first half of that drive, which took us to where we could evaluate West Path, occurred over the weekend. We were going to do the second half thisol, but -- we eventually agree -- we seem to have good enough imaging for both paths from our present position. (Steve is visibly overjoyed to hear this -- he really didn't want to spend another sol just figuring out where we'd go.)
So much for the easy part. But the real kicker is, do we go east or west? Since the part of Erebus we're heading for is southeast of us, it would seem like a no-brainer to choose East Path -- if we choose West Path, we're gambling that we'll eventually find a way to ripple-hop east, and we don't like to gamble. But the ripples to the east of our current position look more fearsome than their westward counterparts, making the gamble look more attractive.
Just as Steve gets back to us, though, we find a pretty good eastward solution. So we have an answer, albeit just in the nick of time. East Path wins.
As hard as it was to reach that answer, that was cake compared to the next part. The East Path on-ramp is fairly narrow, not much wider than the rover's wheel base, and it's "guarded" by a low ripple. The most appealing way to enter the on-ramp (the way that takes us directly across the lowest part of the ripple) puts us at an extremely poor comm heading -- possibly in the PMA exclusion zone, which we're supposed to try to avoid because it means the rover's trying to radiate a signal through the mast. (Which works, but it further reduces the quality of an already poor comm pass.)
We solve it by doing some stuff we're not supposed to do: we drive straight north until we're just west of the on-ramp, then turn in place and scoot directly backward (east) across the ripple. The bad things we're doing include crossing the ripple at an oblique angle, turning in place (Opportunity has a bad steering actuator in one wheel, which hinders turning in place), and driving backward instead of forward across the ripple (so if we should get stuck, we'll have a little less strength if we have to drive out forward rather than backward). But this puts us at a much better comm heading, which we need; and anyway, this is a relatively low ripple, so we shouldn't bog down in it. I just hope those aren't famous last words.
We still have one more problem. According to our new Rules of the Road, we're not supposed to drive more than 5m in one sol unless we're monitoring slip to ensure that we're not slipping more than 25%. With the way we've written the sequence, we can't check our slip precisely enough to meet this requirement. Our choices are to rewrite the sequence in a way that will make it less likely to correctly execute the drive, or cut the drive short, so that we won't make it onto the on-ramp -- or waive the rule.
John Callas, who's the acting project manager today, is extremely reluctant to waive the rule. "It's not how we said we were going to drive here," he points out. "If we're not going to do what we said we were going to do, we should be willing to change what we said we were going to do."
He's got a real point. Cindy Oda tries to get him to let us slide: "Can you just let us do it this way today, and tomorrow we'll do it whatever way you want to?" But I don't want to put John in that position, and he doesn't like it either: "So you've already got your hand in the cookie jar, so why not just let you have this cookie?" he grins.
Still, it looks to me as if John is feeling the pressure from the team, but he needs a face-saving way out. In the end, Jake Matijevic rides to the rescue. "This isn't the place to apply the Rules of the Road," he points out gently. "They were meant for terrain we haven't been over before. Here, we're just following along our old tracks, except at the very end, which is only a couple of meters of driving -- a distance we'd be willing to tolerate anyway." This gets John off the hook, and it has the extra added benefit of being true. So we're heading for the on-ramp at last.
[Next post: sol 538 (Opportunity sol 517), July 8.]
2010-07-01
Opportunity Sol 510 (Spirit Sol 531)
Cooper had a car accident. I figured it was only a matter of time before this happened to one of us, and I'm glad it wasn't I. The car is a wreck, but his daughter, who was in the car with him, was fine, and so is he. Except that he thinks he has a cracked rib. So he's leaving early.
So anyway.
Thisol is the last sol we'll spend around Purgatory Ripple. We've got two MI stacks to do, then we take off. One of the MI targets, "Cleat Tab," is pretty easy. The other, "Torment," lives up to its name. (I wanted to call it "Cleat Fanta," but I was overruled.) As we exited Purgatory Ripple, the front and rear wheels left separate but overlapping tracks, so that the tracks have a sort of inner wall and an outer wall. Torment is on the interior of the outer wall and we're trying to shoot a stack normal to it, which means at about a 45-degree angle to the surrounding terrain. This in itself is not hard; the problem is that by the time the IDD is close enough to get a focused image, the MI poker will contact the inner wall. This is something we want to avoid.
If we were closer to the target, we could solve it by flipping the wrist into the other configuration, which would put the MI poker on the high side of the terrain. But to reach the target, we have to stretch the arm out so far that that wrist configuration would cause a self-collision.
In the end, we just have to come straight down on it -- not normal to the wall -- and hope for the best.
While I'm doing that, Cooper and Paolo are solving a problem with the drive. The drive itself is quite simple: we just back up 2m across North Ripple, turning as we go to set ourselves up for a northward drive along a trough between two ripples. (Our ultimate goal is southeast of here; we're taking a few steps north so that we can get into position to evaluate candidate routes that would take us around Purgatory Ripple without crossing too many other ripples along the way.)
The problem has to do with the turn. We need to track our position with visual odometry so that we can tell whether we're slipping too much -- it's one of the Rules of the Road for this vehicle in the post-Purgatory era -- but in this sandy, rock-free terrain, the only features visodom can converge on are our own tracks. And we're turning so much during the backup that our tracks won't be in the cameras' field of view the whole time.
Unless we change the camera pointing in midstream. Visual odometry works by comparing successive pairs of images -- you take an image, step, take another image, and compare the two. Since it only needs two pictures at any given time, the current one and the one from the previous step, you can "reseed" it at any point. It's time-consuming, but it works. And we overestimated the drive time, so we'll have enough time for one such reseed. So Cooper and Paolo figure out a pair of pointings -- one for the first half of the drive, and one for the second -- and that's what we go with.
That's all. Except for one more problem that comes up at the CAM. "What if," Emily asks -- "what if the poker does contact the terrain after all?"
I have to admit it's a good question. "We don't expect any contact, but all the moves are in free-space mode. So if we contact the terrain, it shouldn't damage the arm, but it'll fault out the IDD sequence and keep us from stowing, which will also mean we don't drive."
We generally sequence this way. For a time, there was a war between what Jeff Biesiadecki termed the "Baumgartian" and "Bonitzian" views of this problem.[1] In the Baumgartian view, unexpected contact with the terrain is something that should immediately stop motion: the terrain isn't where you think it is, so you should stop and reassess everything from the ground. So you use free-space mode unless you're actually expecting contact. In the Bonitzian view, it's acceptable for a sequence to tolerate early contact -- as long as you've written it carefully, so that it will proceed safely whether you have early contact or not.
For a long time I was a Bonitzian, but somewhere in there I converted to the Baumgartian camp, as I think most of us did. Even leaving the safety arguments aside, the Baumgartian approach makes sequencing faster and simpler, since you don't have to validate that the sequence always works correctly in the face of any unexpected contact.
So this sequence was Baumgartian. But in this case, the Baumgartian downside -- faulting out the sequence, and blowing the drive with it -- would be worse than usual. Not as bad as harming the vehicle, but still pretty bad. The Mars Program office has been breathing down our necks to get us moving again (I'm told), so nobody really wants to risk the drive. Luckily, the fix -- converting to Bonitzian style -- is easy in this case: I simply change the MI stack moves so that they use guarded mode rather than free-space mode. This way, if the MI poker senses contact with the terrain, it won't approach the terrain any more closely but will generally go on with the sequence. We can't always make that change so easily, but in this case it happens to work out. Score one for the Bonitzians.[2]
[Next post: sol 536 (Opportunity sol 515), July 6.]
Footnotes:
[1] This is after Eric Baumgartner and Bob Bonitz, two of the original rover drivers -- both of them deeply involved in the development of the arm. Though long gone from the project, they remain important influences to this day.
[2] But this was one of the last scores for the Bonitzians. Today, we're all Baumgartians. Mainly, this is due to a project-wide change in attitude about sols: they're clearly more plentiful than we once thought, so we don't go as far out of our way to save them as we used to. In that world, the main advantage of Bonitzianism -- it can enable you to autonomously recover and go on to do other activities -- disappears, and the main advantage of Baumgartianism -- it's more paranoid about unexpected behavior -- wins out.
So anyway.
Thisol is the last sol we'll spend around Purgatory Ripple. We've got two MI stacks to do, then we take off. One of the MI targets, "Cleat Tab," is pretty easy. The other, "Torment," lives up to its name. (I wanted to call it "Cleat Fanta," but I was overruled.) As we exited Purgatory Ripple, the front and rear wheels left separate but overlapping tracks, so that the tracks have a sort of inner wall and an outer wall. Torment is on the interior of the outer wall and we're trying to shoot a stack normal to it, which means at about a 45-degree angle to the surrounding terrain. This in itself is not hard; the problem is that by the time the IDD is close enough to get a focused image, the MI poker will contact the inner wall. This is something we want to avoid.
If we were closer to the target, we could solve it by flipping the wrist into the other configuration, which would put the MI poker on the high side of the terrain. But to reach the target, we have to stretch the arm out so far that that wrist configuration would cause a self-collision.
In the end, we just have to come straight down on it -- not normal to the wall -- and hope for the best.
While I'm doing that, Cooper and Paolo are solving a problem with the drive. The drive itself is quite simple: we just back up 2m across North Ripple, turning as we go to set ourselves up for a northward drive along a trough between two ripples. (Our ultimate goal is southeast of here; we're taking a few steps north so that we can get into position to evaluate candidate routes that would take us around Purgatory Ripple without crossing too many other ripples along the way.)
The problem has to do with the turn. We need to track our position with visual odometry so that we can tell whether we're slipping too much -- it's one of the Rules of the Road for this vehicle in the post-Purgatory era -- but in this sandy, rock-free terrain, the only features visodom can converge on are our own tracks. And we're turning so much during the backup that our tracks won't be in the cameras' field of view the whole time.
Unless we change the camera pointing in midstream. Visual odometry works by comparing successive pairs of images -- you take an image, step, take another image, and compare the two. Since it only needs two pictures at any given time, the current one and the one from the previous step, you can "reseed" it at any point. It's time-consuming, but it works. And we overestimated the drive time, so we'll have enough time for one such reseed. So Cooper and Paolo figure out a pair of pointings -- one for the first half of the drive, and one for the second -- and that's what we go with.
That's all. Except for one more problem that comes up at the CAM. "What if," Emily asks -- "what if the poker does contact the terrain after all?"
I have to admit it's a good question. "We don't expect any contact, but all the moves are in free-space mode. So if we contact the terrain, it shouldn't damage the arm, but it'll fault out the IDD sequence and keep us from stowing, which will also mean we don't drive."
We generally sequence this way. For a time, there was a war between what Jeff Biesiadecki termed the "Baumgartian" and "Bonitzian" views of this problem.[1] In the Baumgartian view, unexpected contact with the terrain is something that should immediately stop motion: the terrain isn't where you think it is, so you should stop and reassess everything from the ground. So you use free-space mode unless you're actually expecting contact. In the Bonitzian view, it's acceptable for a sequence to tolerate early contact -- as long as you've written it carefully, so that it will proceed safely whether you have early contact or not.
For a long time I was a Bonitzian, but somewhere in there I converted to the Baumgartian camp, as I think most of us did. Even leaving the safety arguments aside, the Baumgartian approach makes sequencing faster and simpler, since you don't have to validate that the sequence always works correctly in the face of any unexpected contact.
So this sequence was Baumgartian. But in this case, the Baumgartian downside -- faulting out the sequence, and blowing the drive with it -- would be worse than usual. Not as bad as harming the vehicle, but still pretty bad. The Mars Program office has been breathing down our necks to get us moving again (I'm told), so nobody really wants to risk the drive. Luckily, the fix -- converting to Bonitzian style -- is easy in this case: I simply change the MI stack moves so that they use guarded mode rather than free-space mode. This way, if the MI poker senses contact with the terrain, it won't approach the terrain any more closely but will generally go on with the sequence. We can't always make that change so easily, but in this case it happens to work out. Score one for the Bonitzians.[2]
[Next post: sol 536 (Opportunity sol 515), July 6.]
Footnotes:
[1] This is after Eric Baumgartner and Bob Bonitz, two of the original rover drivers -- both of them deeply involved in the development of the arm. Though long gone from the project, they remain important influences to this day.
[2] But this was one of the last scores for the Bonitzians. Today, we're all Baumgartians. Mainly, this is due to a project-wide change in attitude about sols: they're clearly more plentiful than we once thought, so we don't go as far out of our way to save them as we used to. In that world, the main advantage of Bonitzianism -- it can enable you to autonomously recover and go on to do other activities -- disappears, and the main advantage of Baumgartianism -- it's more paranoid about unexpected behavior -- wins out.
2010-06-30
Opportunity Sol 509 (Spirit Sol 530)
Bad news: in about ten days, Khaled will be a Rover Planner no more. He's leaving, taking over the role of coordinating the flight software uploads.
As politely as I can, I ask him the only question that comes to mind: "Are you crazy?"
"I've been asking myself that," he admits with a mournful tone. "It's certainly not going to be as much fun as this job. But this one doesn't meet those ECAP goals I keep hearing, so ...."[1]
Well, the guy's gotta do what he's gotta do. Andy isn't going to be happy to hear this -- more headaches for him. I'm not too happy about it myself, honestly. Khaled's a fine RP; he came up to speed quickly, knows the rovers well, and is thorough, reliable, and conscientious. He'll be hard to replace.
One thing I won't miss is the smell of his wife's cooking, when he brings in his leftovers for lunch nearly every day. Not that her cooking smells bad. Quite the contrary, it always smells delicious. The problem with this is that it makes my own hunger impossible to ignore -- whenever I get a whiff of Khaled's lunch, I have to go buy lunch from the cafeteria. And that's never as good as his. Darn him!
Thisol's sequencing is not much more complex than yestersol's. Khaled and I put the extra time to good use by preparing the drive sequences. He'd already put together a zeroth draft for the first couple of upcoming drive sols (510 and 511), and we go over them and make some changes together.
Gee, he's not only thorough, reliable, and conscientious, he also works ahead, making things simpler for the rest of us. And now he's leaving.
Bastard.[2]
Footnotes:
[1] An ECAP -- Employee Contribution Assessment and Planning -- is JPL's annual employee performance review.
[2] Khaled has come and gone a couple of times now. Right now, he's off working on Moonrise, but I hope he'll be back again soon. He's damn good.
As politely as I can, I ask him the only question that comes to mind: "Are you crazy?"
"I've been asking myself that," he admits with a mournful tone. "It's certainly not going to be as much fun as this job. But this one doesn't meet those ECAP goals I keep hearing, so ...."[1]
Well, the guy's gotta do what he's gotta do. Andy isn't going to be happy to hear this -- more headaches for him. I'm not too happy about it myself, honestly. Khaled's a fine RP; he came up to speed quickly, knows the rovers well, and is thorough, reliable, and conscientious. He'll be hard to replace.
One thing I won't miss is the smell of his wife's cooking, when he brings in his leftovers for lunch nearly every day. Not that her cooking smells bad. Quite the contrary, it always smells delicious. The problem with this is that it makes my own hunger impossible to ignore -- whenever I get a whiff of Khaled's lunch, I have to go buy lunch from the cafeteria. And that's never as good as his. Darn him!
Thisol's sequencing is not much more complex than yestersol's. Khaled and I put the extra time to good use by preparing the drive sequences. He'd already put together a zeroth draft for the first couple of upcoming drive sols (510 and 511), and we go over them and make some changes together.
Gee, he's not only thorough, reliable, and conscientious, he also works ahead, making things simpler for the rest of us. And now he's leaving.
Bastard.[2]
Footnotes:
[1] An ECAP -- Employee Contribution Assessment and Planning -- is JPL's annual employee performance review.
[2] Khaled has come and gone a couple of times now. Right now, he's off working on Moonrise, but I hope he'll be back again soon. He's damn good.
2010-06-29
Opportunity Sol 508 (Spirit Sol 529)
This is one of the simplest days I've had in a long time: just a tool change from APXS to MB, continuing our exploration of the soil near Purgatory Ripple. And I'm not even RP-1 (Jeng is), so I just have to review it.
Yawn.
One-way light-time to Mars has been falling. Right now it's eight minutes, twenty-seven seconds -- we're just about equidistant from Mars and the Sun. A big Isosceles triangle in space. 8:27 -- that's 507 seconds, one less than the sol number we're planning. That's the first time those numbers have crossed. "The rovers haven't been this close to Earth since December 2003," Steve remarks.
The CAM goes smoothly; another one down. "One more day and we hit the gas!" Steve exclaims. "I can't wait to get outta here!"
Yawn.
One-way light-time to Mars has been falling. Right now it's eight minutes, twenty-seven seconds -- we're just about equidistant from Mars and the Sun. A big Isosceles triangle in space. 8:27 -- that's 507 seconds, one less than the sol number we're planning. That's the first time those numbers have crossed. "The rovers haven't been this close to Earth since December 2003," Steve remarks.
The CAM goes smoothly; another one down. "One more day and we hit the gas!" Steve exclaims. "I can't wait to get outta here!"
2010-06-28
Opportunity Sol 507 (Spirit Sol 528)
Ah, that feels better. The drive went just exactly as planned; the old track, tailings, and wide swaths of undisturbed soil are within the IDD work volume. I've never been so excited about an 80cm drive.
Nor has most of the rest of the team. I forget, they actually started their approach to this spot well before I got here -- it took something like a week to do the drive away, turn-in-place, and drive back -- and today was the last chance to get it. But we got it. Brenda says Steve is "happy as a clam," and he is. "Take your time getting SAP set up, Brenda," he says. "I'm just gonna sit here and gaze in wonder at this reachability map -- nice job, Rover Planners!"
I feel a little silly about taking a bow (figuratively!). It's just 80cm; I'm supposed to be able to do this with my eyes closed. Well, whatever.
I take a moment to look at the drive telemetry. After all that time we spent worrying about what to set them to, I'm curious about what the bogie angles were during this drive. The left bogie started at 14 degrees -- the value that aborted our previous drive -- and we knew it was going to increase as we finished climbing North Ripple, but how much did it increase?
It went to 15 degrees. That's it. All that time we spent quibbling about the drive, arguing over whether 20 degrees was enough headroom, and so on ... and we needed about one extra degree.
Well, at least it worked. It did execute nicely, and now we're ready to swing the IDD out and attack the soil. We'll start with a 2x2 MI mosaic around a patch of undisturbed soil near the RF wheel, then do another such mosaic, followed by an APXS placement, in the track.
This is fairly straightforward IDD work, though it's making Jeng nervous. Maybe it's that he doesn't yet trust his ability to sequence the IDD, or maybe it's the early deadline -- this is what we call a "tight" sol, with uplink in just a few hours, leaving little room for error. But it's an undramatic day; with me looking over his shoulder, Jeng cranks out the IDD sequence in plenty of time.
At the Activity Plan Approval Meeting, Paolo points out that the near side of the track, the spot we've picked as our representative undisturbed-soil location, is darker than the far side. This means that the far side has lighter-colored dust, which generally means smaller, finer dust particles. This is the stuff we sometimes call "foo-foo dust," and there's a going hypothesis that that's what Purgatory Ripple is made of. So, Paolo suggests, maybe we should be IDDing there instead?[1]
Rob Sullivan answers the question. "What we think we're seeing there is the result of a recent wind event -- in the last few sols, a gust of wind came through here and blew away a lot of the material we dredged up when we exited Purgatory Ripple. So a lot of that dust probably blew in from that event. The nearby patch is more likely to be the native material of Purgatory Ripple."
"So your interpretation is right," Steve Squyres chimes in. "That lighter-colored stuff probably is foo-foo dust -- but in this case, that's a bad thing. But keep makin' suggestions!"
The rest of the day goes ahead pretty nominally. At the CAM, Dan resurrects the Picture-of-the-Day tradition to show a true-color PANCAM image of the trench we (inadvertently) made in Purgatory Ripple. It's enormous, as wide as a wheel and at least that deep.
"How did we ever get outta that?" he wonders aloud.
"We have really good rover drivers," Wendy Calvin says.
"And a whole lotta patience," adds Steve.
Courtesy NASA/JPL-Caltech. At last. That's what we've been trying to do.
Footnotes:
[1] A characteristically brilliant observation. At this point, Paolo had been on the project for only a few weeks -- a couple of months, at most -- but he was already showing the keen insights and attention to detail that would later make him a top-notch rover driver.
Nor has most of the rest of the team. I forget, they actually started their approach to this spot well before I got here -- it took something like a week to do the drive away, turn-in-place, and drive back -- and today was the last chance to get it. But we got it. Brenda says Steve is "happy as a clam," and he is. "Take your time getting SAP set up, Brenda," he says. "I'm just gonna sit here and gaze in wonder at this reachability map -- nice job, Rover Planners!"
I feel a little silly about taking a bow (figuratively!). It's just 80cm; I'm supposed to be able to do this with my eyes closed. Well, whatever.
I take a moment to look at the drive telemetry. After all that time we spent worrying about what to set them to, I'm curious about what the bogie angles were during this drive. The left bogie started at 14 degrees -- the value that aborted our previous drive -- and we knew it was going to increase as we finished climbing North Ripple, but how much did it increase?
It went to 15 degrees. That's it. All that time we spent quibbling about the drive, arguing over whether 20 degrees was enough headroom, and so on ... and we needed about one extra degree.
Well, at least it worked. It did execute nicely, and now we're ready to swing the IDD out and attack the soil. We'll start with a 2x2 MI mosaic around a patch of undisturbed soil near the RF wheel, then do another such mosaic, followed by an APXS placement, in the track.
This is fairly straightforward IDD work, though it's making Jeng nervous. Maybe it's that he doesn't yet trust his ability to sequence the IDD, or maybe it's the early deadline -- this is what we call a "tight" sol, with uplink in just a few hours, leaving little room for error. But it's an undramatic day; with me looking over his shoulder, Jeng cranks out the IDD sequence in plenty of time.
At the Activity Plan Approval Meeting, Paolo points out that the near side of the track, the spot we've picked as our representative undisturbed-soil location, is darker than the far side. This means that the far side has lighter-colored dust, which generally means smaller, finer dust particles. This is the stuff we sometimes call "foo-foo dust," and there's a going hypothesis that that's what Purgatory Ripple is made of. So, Paolo suggests, maybe we should be IDDing there instead?[1]
Rob Sullivan answers the question. "What we think we're seeing there is the result of a recent wind event -- in the last few sols, a gust of wind came through here and blew away a lot of the material we dredged up when we exited Purgatory Ripple. So a lot of that dust probably blew in from that event. The nearby patch is more likely to be the native material of Purgatory Ripple."
"So your interpretation is right," Steve Squyres chimes in. "That lighter-colored stuff probably is foo-foo dust -- but in this case, that's a bad thing. But keep makin' suggestions!"
The rest of the day goes ahead pretty nominally. At the CAM, Dan resurrects the Picture-of-the-Day tradition to show a true-color PANCAM image of the trench we (inadvertently) made in Purgatory Ripple. It's enormous, as wide as a wheel and at least that deep.
"How did we ever get outta that?" he wonders aloud.
"We have really good rover drivers," Wendy Calvin says.
"And a whole lotta patience," adds Steve.
Courtesy NASA/JPL-Caltech. At last. That's what we've been trying to do.
Footnotes:
[1] A characteristically brilliant observation. At this point, Paolo had been on the project for only a few weeks -- a couple of months, at most -- but he was already showing the keen insights and attention to detail that would later make him a top-notch rover driver.
2010-06-25
Opportunity Sol 505 (Spirit Sol 525)
Six fucking centimeters. That's it. Six fucking centimeters is all the distance we made yestersol.
It's my fault. In the drive sequence, we retained most of the limits that had been set for the previous drive. We were just passing over the ground they had meant to pass over, so we figured the same limits would be fine.
Frankly, this was just lazy. And it's what stopped us. As it happened, Opportunity's left bogie articulated just a little more than we expected it to as it climbed the ripple that's under us now (North Ripple), and that terminated the drive. After six centimeters.
We really, really should have caught this in advance. The bogie limit was 14 degrees, and if we'd checked the telemetry, we'd have noticed that it was already just over 11 degrees -- not a lot of headroom there. The really painful part is that I thought of that while we were doing the sequencing, and then got distracted by something and never got back to checking. Oy vey.[1]
And this changes our plans for the weekend. We'd originally intended to IDD the far track -- the one we were driving to -- but since we didn't make it there, that's out. So it'll be a touch-and-go sol, where we IDD whatever happens to be close to us and then try to drive the remaining 69cm or so to the far track.
Assuming we're capable of that much.
Both Jeff Favretto and Steve Squyres are philosophical about the drive result. "This is just a consequence of using conservative limits," Jeff says. And conservative limits are exactly what we promised the Mars Program Office we'd use, adds Steve. But as usual, things aren't that simple. Apparently, the Program management has been putting some pressure on John Callas (acting project manager while Jim Erickson is out of town). He's getting it from both sides: some people are complaining that we're being reckless cowboys; others, that we're too conservative. They want us to be successful without making any mistakes, which is fine to a point, but they've pushed it far enough that it amounts to demanding that we be lucky.[2] I'm glad I don't have his job.[3]
Squyres might have been getting some of that "too-conservative" flack as well. "I want you to drive safely," he tells me, "but I should say that if we don't make it to the target after this attempt, we're just going to IDD whatever we've got and then get out of here." "So, no pressure?" I ask. "Yeah," he grins, "no pressure."
The IDD work is fairly straightforward, and I get it more or less knocked off in a couple of hours. It initially looks like one of the targets the science team chose was too close to our right front wheel, but the simulation says it's OK, so I move on.
The bulk of the sol's work is in arguing over the drive. Not in implementing it, just in arguing about it. I go through every single parameter limit and check it against both the current state and the RSVP-simulated maximum, trying to pick sensible numbers for everything. We're left with a small set of numbers to argue about, the suspension limits -- most crucially, what should be the proper settings for the bogie limits that stopped us on 503? This turns into, I kid you not, a couple of hours' discussion at least.[4]
Maybe this is all ridiculous. The distance of this drive is about equal to a single wheel rotation -- how much trouble could we get into, even if we abandoned the limits altogether? But at this point we're so damn paranoid that we're ready to entertain the ridiculous.
The number we eventually pick for the max bogie limit is 25 degrees. That's about twice what we're at now, and just 5 degrees shy of the hardstop. It's almost impossible for us to actually hit it, and all things considered, I can feel OK about that. If we actually do reach that limit, something really strange will be happening, and we'll want the drive to stop. Which is what the limit is for.
Our paranoia doesn't end there. Steve asked me earlier in the day for a list of everything that could possibly stop the drive, and one thing that could stop it is for the IDD work to fail -- that would prevent the drive from even starting. So I look back at that, and the IDD target that concerned me before starts to look even more worrisome.
RSVP can show something called the "collision volumes," which RSVP visualizes as a set of purple boxes that are slightly bigger than the rover parts. They're like the rover is wearing bulky translucent purple padding all over. The on-board flight software uses these collision volumes to detect a pending self-collision; the idea is to be conservative about the possibility of collisions during (especially) IDD work -- if the padded version of the rover doesn't collide with itself, then the real thing won't. RSVP also tells us whether these collisions will occur in the sequence we've planned, and since the simulation is happy, we should expect that the real thing will be okay.
And yet ... when we're inspecting the nearby IDD target, the collision volumes come so close to intersecting that we have to really work to show that they're not. What if the flight vehicle is just a tiny bit different from this simulation? It shouldn't be, but as I said, we're pretty paranoid now, and the difference would have to be only a tiny one -- maybe small enough that it would have gone undetected until now -- to screw us up once again.
I'm just being paranoid. I'm sure I'm just being paranoid. But I'm not the only one, and we end up deciding to turn off those collision checks during the close approach. We're not coming so close that there's a risk of an actual collision, just a risk that the collision volumes -- the padding -- could collide, and turning them off means we can be sure it won't happen.
I won't feel good about any of this until Monday.
[Next post: sol 528 (Opportunity sol 507), June 28.]
Footnotes:
[1] This exact incident, if I recall aright, is why checking the suspension history from the previous drive is something I now do every sol. It's bad enough to make a dumb mistake, much worse to repeat it.
[2] Fortunately, this was the last time that Mars Program ever made such a stupid demand. cough
[3] Still true. And how.
[4] In a way, this is one of the things I really love about my job: the obsessive focus on getting these things right. We might spend an hour arguing about a degree, or a millimeter. Sometimes those discussions are horrendously painful when I'm in the middle of them, but when I take a step back, I love being in an environment where people are that passionate about even the little things.
It's my fault. In the drive sequence, we retained most of the limits that had been set for the previous drive. We were just passing over the ground they had meant to pass over, so we figured the same limits would be fine.
Frankly, this was just lazy. And it's what stopped us. As it happened, Opportunity's left bogie articulated just a little more than we expected it to as it climbed the ripple that's under us now (North Ripple), and that terminated the drive. After six centimeters.
We really, really should have caught this in advance. The bogie limit was 14 degrees, and if we'd checked the telemetry, we'd have noticed that it was already just over 11 degrees -- not a lot of headroom there. The really painful part is that I thought of that while we were doing the sequencing, and then got distracted by something and never got back to checking. Oy vey.[1]
And this changes our plans for the weekend. We'd originally intended to IDD the far track -- the one we were driving to -- but since we didn't make it there, that's out. So it'll be a touch-and-go sol, where we IDD whatever happens to be close to us and then try to drive the remaining 69cm or so to the far track.
Assuming we're capable of that much.
Both Jeff Favretto and Steve Squyres are philosophical about the drive result. "This is just a consequence of using conservative limits," Jeff says. And conservative limits are exactly what we promised the Mars Program Office we'd use, adds Steve. But as usual, things aren't that simple. Apparently, the Program management has been putting some pressure on John Callas (acting project manager while Jim Erickson is out of town). He's getting it from both sides: some people are complaining that we're being reckless cowboys; others, that we're too conservative. They want us to be successful without making any mistakes, which is fine to a point, but they've pushed it far enough that it amounts to demanding that we be lucky.[2] I'm glad I don't have his job.[3]
Squyres might have been getting some of that "too-conservative" flack as well. "I want you to drive safely," he tells me, "but I should say that if we don't make it to the target after this attempt, we're just going to IDD whatever we've got and then get out of here." "So, no pressure?" I ask. "Yeah," he grins, "no pressure."
The IDD work is fairly straightforward, and I get it more or less knocked off in a couple of hours. It initially looks like one of the targets the science team chose was too close to our right front wheel, but the simulation says it's OK, so I move on.
The bulk of the sol's work is in arguing over the drive. Not in implementing it, just in arguing about it. I go through every single parameter limit and check it against both the current state and the RSVP-simulated maximum, trying to pick sensible numbers for everything. We're left with a small set of numbers to argue about, the suspension limits -- most crucially, what should be the proper settings for the bogie limits that stopped us on 503? This turns into, I kid you not, a couple of hours' discussion at least.[4]
Maybe this is all ridiculous. The distance of this drive is about equal to a single wheel rotation -- how much trouble could we get into, even if we abandoned the limits altogether? But at this point we're so damn paranoid that we're ready to entertain the ridiculous.
The number we eventually pick for the max bogie limit is 25 degrees. That's about twice what we're at now, and just 5 degrees shy of the hardstop. It's almost impossible for us to actually hit it, and all things considered, I can feel OK about that. If we actually do reach that limit, something really strange will be happening, and we'll want the drive to stop. Which is what the limit is for.
Our paranoia doesn't end there. Steve asked me earlier in the day for a list of everything that could possibly stop the drive, and one thing that could stop it is for the IDD work to fail -- that would prevent the drive from even starting. So I look back at that, and the IDD target that concerned me before starts to look even more worrisome.
RSVP can show something called the "collision volumes," which RSVP visualizes as a set of purple boxes that are slightly bigger than the rover parts. They're like the rover is wearing bulky translucent purple padding all over. The on-board flight software uses these collision volumes to detect a pending self-collision; the idea is to be conservative about the possibility of collisions during (especially) IDD work -- if the padded version of the rover doesn't collide with itself, then the real thing won't. RSVP also tells us whether these collisions will occur in the sequence we've planned, and since the simulation is happy, we should expect that the real thing will be okay.
And yet ... when we're inspecting the nearby IDD target, the collision volumes come so close to intersecting that we have to really work to show that they're not. What if the flight vehicle is just a tiny bit different from this simulation? It shouldn't be, but as I said, we're pretty paranoid now, and the difference would have to be only a tiny one -- maybe small enough that it would have gone undetected until now -- to screw us up once again.
I'm just being paranoid. I'm sure I'm just being paranoid. But I'm not the only one, and we end up deciding to turn off those collision checks during the close approach. We're not coming so close that there's a risk of an actual collision, just a risk that the collision volumes -- the padding -- could collide, and turning them off means we can be sure it won't happen.
I won't feel good about any of this until Monday.
[Next post: sol 528 (Opportunity sol 507), June 28.]
Footnotes:
[1] This exact incident, if I recall aright, is why checking the suspension history from the previous drive is something I now do every sol. It's bad enough to make a dumb mistake, much worse to repeat it.
[2] Fortunately, this was the last time that Mars Program ever made such a stupid demand. cough
[3] Still true. And how.
[4] In a way, this is one of the things I really love about my job: the obsessive focus on getting these things right. We might spend an hour arguing about a degree, or a millimeter. Sometimes those discussions are horrendously painful when I'm in the middle of them, but when I take a step back, I love being in an environment where people are that passionate about even the little things.
2010-06-24
Opportunity Sol 504 (Spirit Sol 524)
Yestersol was a two-sol plan and tomorrow's a three-sol weekend plan; we just have a pre-planning meeting today. Assuming none of our paranoid limit checks triggers and we actually make it to our goal, what's on tap for the weekend is simply some light IDD work: a little MI, a little APXS, a little MB. Nothing out of the ordinary.
I hope that will mean we have time to start planning the drive away from Purgatory. I haven't given any thought to it, and I just noticed that I'm scheduled all next week. I've been off this rover for a while, but I want to look like I know what I'm doing.
I hope that will mean we have time to start planning the drive away from Purgatory. I haven't given any thought to it, and I just noticed that I'm scheduled all next week. I've been off this rover for a while, but I want to look like I know what I'm doing.
2010-06-23
Opportunity Sol 503 (Spirit Sol 523)
It's my first sol driving the rovers since I came back from jury duty, and I've switched to Opportunity -- my first sol on this rover since, well, since before they encountered Purgatory Ripple.
Yestersol's drive was a 20-degree turn in place followed by a whopping 50cm drive forward, in two 25cm arcs. But only one of the two arcs completed. Opportunity sensed that she'd slipped 32% on that arc, and we'd set a limit of 30%, so she stopped. Opportunity is scared of bogging down again.
But Khaled and Paolo analyze the drive and conclude that it's safe for us to proceed. What happened was that as we did the turn in place, the right-front wheel -- whose steering actuator is broken -- was mostly just dragged along, digging itself a small trench in the process. Then when we tried to step forward, we slipped a touch more than expected as we climbed out of that trench. But the rover's safe, on top of the soil rather then underneath it, and the way ahead is clear.
Our goal here is simply to drive back to a part of the tracks we left as we exited Purgatory. Yestersol's drive was supposed to leave us about 50cm from them, but, well ... so today we'll drive a whopping 75cm; if it goes well, we'll be right where we need to be to IDD the tracks over the weekend. Then next week we'll pack up and leave.
Khaled and I do spend a fair amount of time working on the drive -- everyone's paranoid about re-approaching this ripple -- but we're ready in plenty of time. Jeff Favretto notices the level of effort we're putting in, and during one of the meetings, he just grins and holds his hands a little more than shoulder-width apart. "Seventy-five centimeters," he grins, shaking his head. "This is how far we're going. It takes all that ...."
Meanwhile, Spirit's exhibiting a little bit of unusual behavior. During her last drive, the left bogie angles changed in unusual ways, indicating that the middle wheel kept lifting well into the air and then dropping back down. One hypothesis is that something's stuck on the outside of the wheel. If this is true, we'll never be able to see it: they're not visible from any of the cameras.[1] Jeff thinks he knows what's stuck in the wheel: "It's that dinosaur bone we've been looking for all this time!"
Courtesy NASA/JPL-Caltech. Post-drive front HAZCAM image showing the track created by the right front wheel when we turned in place on the previous drive.
Footnotes:
[1] As you can see, we hadn't then thought of using the MI to view the middle wheels. We didn't think of that trick until much later, when Spirit was embedded in Troy.
Yestersol's drive was a 20-degree turn in place followed by a whopping 50cm drive forward, in two 25cm arcs. But only one of the two arcs completed. Opportunity sensed that she'd slipped 32% on that arc, and we'd set a limit of 30%, so she stopped. Opportunity is scared of bogging down again.
But Khaled and Paolo analyze the drive and conclude that it's safe for us to proceed. What happened was that as we did the turn in place, the right-front wheel -- whose steering actuator is broken -- was mostly just dragged along, digging itself a small trench in the process. Then when we tried to step forward, we slipped a touch more than expected as we climbed out of that trench. But the rover's safe, on top of the soil rather then underneath it, and the way ahead is clear.
Our goal here is simply to drive back to a part of the tracks we left as we exited Purgatory. Yestersol's drive was supposed to leave us about 50cm from them, but, well ... so today we'll drive a whopping 75cm; if it goes well, we'll be right where we need to be to IDD the tracks over the weekend. Then next week we'll pack up and leave.
Khaled and I do spend a fair amount of time working on the drive -- everyone's paranoid about re-approaching this ripple -- but we're ready in plenty of time. Jeff Favretto notices the level of effort we're putting in, and during one of the meetings, he just grins and holds his hands a little more than shoulder-width apart. "Seventy-five centimeters," he grins, shaking his head. "This is how far we're going. It takes all that ...."
Meanwhile, Spirit's exhibiting a little bit of unusual behavior. During her last drive, the left bogie angles changed in unusual ways, indicating that the middle wheel kept lifting well into the air and then dropping back down. One hypothesis is that something's stuck on the outside of the wheel. If this is true, we'll never be able to see it: they're not visible from any of the cameras.[1] Jeff thinks he knows what's stuck in the wheel: "It's that dinosaur bone we've been looking for all this time!"
Courtesy NASA/JPL-Caltech. Post-drive front HAZCAM image showing the track created by the right front wheel when we turned in place on the previous drive.
Footnotes:
[1] As you can see, we hadn't then thought of using the MI to view the middle wheels. We didn't think of that trick until much later, when Spirit was embedded in Troy.
2010-06-10
Spirit Sol 511
"My Ph.D. thesis was on writing a program that would design robots for you," Chris says. "You'd tell it what you want, and it would design a robot to do it. That way, I could spend more time climbing." (He's an avid rock climber.)
"See, I'd have done it the other way around. I'd have designed a robot that would go climbing for me, so I could spend my time designing robots," I say.
"That's the difference between you and me," he deadpans.
It's Friday, and that means it's another three-sol plan. I did essentially all of the IDD work for all three sols yesterday so that I could focus on the drive today, which should make today easy.
Ha.
Since we're on tight sols, our uplink's at 17:40, and for a while there it really doesn't look like we're going to make it. In the end we finish with less than half an hour to spare. We haven't cut things that close in a while. I am awfully glad I did the IDD sequences up front, though; it was only two or three hours' work, but we really didn't have those two or three hours today.
This might be my last sol on Spirit, at least for the next month or so. I'm potentially on jury duty next week, and then I head upstairs to the Land of Opportunity. Which will be different from the last time I was up there. I came downstairs not long before they ran into what they're now calling "Purgatory Ripple," and they extracted themselves already. But the effects of Purgatory Ripple will outlast the actual experience of our being embedded in it. We're going to spend some time investigating the ripple itself, and we're going to drive differently from now on -- probably much less aggressively, but certainly differently.
How differently? That's still an open question. Khaled's working out some of the details, and he comes by to talk to me and Chris about it. They've got a list of proposals going, including my two sneaky tricks. (First trick: periodically stop and try to turn in place. If you can't turn, you're stuck; preclude further driving. Second trick: periodically try to run visodom. If it converges, you must not have moved significantly between the two images, meaning that you're stuck; preclude further driving.) As we talk, I come up with a couple of other creative ideas. One of them is to abuse the IMU: if it doesn't show that your local "down vector" is changing with respect to time, you're probably not climbing ripples and hence may be stuck; preclude driving. Similarly, we could check the suspension limits: if the suspension isn't flexing over time, stop driving.[1]
Nearly all of our ideas are reactive measures: try to detect when you're bogging down and keep from making a bad situation worse. What we'd really like are predictive measures, where we avoid getting stuck in the first place. We're short on those.
But we'd better do at least some driving. Recent email traffic prompted by the 1000-sol news has pointed out that we're close to a couple more milestones: 95,303 images so far (nearly 100,000) and a total distance of 9752.26m driven so far (nearly 10,000). I'm so proud of these rovers.
Footnotes:
[1] The end result was to use VO in a cleverer way: take a picture, move 20cm, and update VO. If VO reports that you've slipped more than 40%, preclude further driving. If it doesn't converge, try again, and preclude driving if you slip more than 40% or if it still doesn't converge.
But that's expensive: it takes at least six minutes, as many as 10, and we have to stop and do it every 20m or so on Opportunity. So, as we speak, there's an MIT grad student working to make an IMU-based classifier do the work for us -- an essentially free check that will notice relatively quickly if we're embedded and stop driving.
[Next post: sol 523 (Opportunity sol 503), June 23. Damn jury service.]
"See, I'd have done it the other way around. I'd have designed a robot that would go climbing for me, so I could spend my time designing robots," I say.
"That's the difference between you and me," he deadpans.
It's Friday, and that means it's another three-sol plan. I did essentially all of the IDD work for all three sols yesterday so that I could focus on the drive today, which should make today easy.
Ha.
Since we're on tight sols, our uplink's at 17:40, and for a while there it really doesn't look like we're going to make it. In the end we finish with less than half an hour to spare. We haven't cut things that close in a while. I am awfully glad I did the IDD sequences up front, though; it was only two or three hours' work, but we really didn't have those two or three hours today.
This might be my last sol on Spirit, at least for the next month or so. I'm potentially on jury duty next week, and then I head upstairs to the Land of Opportunity. Which will be different from the last time I was up there. I came downstairs not long before they ran into what they're now calling "Purgatory Ripple," and they extracted themselves already. But the effects of Purgatory Ripple will outlast the actual experience of our being embedded in it. We're going to spend some time investigating the ripple itself, and we're going to drive differently from now on -- probably much less aggressively, but certainly differently.
How differently? That's still an open question. Khaled's working out some of the details, and he comes by to talk to me and Chris about it. They've got a list of proposals going, including my two sneaky tricks. (First trick: periodically stop and try to turn in place. If you can't turn, you're stuck; preclude further driving. Second trick: periodically try to run visodom. If it converges, you must not have moved significantly between the two images, meaning that you're stuck; preclude further driving.) As we talk, I come up with a couple of other creative ideas. One of them is to abuse the IMU: if it doesn't show that your local "down vector" is changing with respect to time, you're probably not climbing ripples and hence may be stuck; preclude driving. Similarly, we could check the suspension limits: if the suspension isn't flexing over time, stop driving.[1]
Nearly all of our ideas are reactive measures: try to detect when you're bogging down and keep from making a bad situation worse. What we'd really like are predictive measures, where we avoid getting stuck in the first place. We're short on those.
But we'd better do at least some driving. Recent email traffic prompted by the 1000-sol news has pointed out that we're close to a couple more milestones: 95,303 images so far (nearly 100,000) and a total distance of 9752.26m driven so far (nearly 10,000). I'm so proud of these rovers.
Footnotes:
[1] The end result was to use VO in a cleverer way: take a picture, move 20cm, and update VO. If VO reports that you've slipped more than 40%, preclude further driving. If it doesn't converge, try again, and preclude driving if you slip more than 40% or if it still doesn't converge.
But that's expensive: it takes at least six minutes, as many as 10, and we have to stop and do it every 20m or so on Opportunity. So, as we speak, there's an MIT grad student working to make an IMU-based classifier do the work for us -- an essentially free check that will notice relatively quickly if we're embedded and stop driving.
[Next post: sol 523 (Opportunity sol 503), June 23. Damn jury service.]
Subscribe to:
Posts (Atom)
Posts
