Surprise, surprise, the drive went exactly where it was supposed to go. Most of the participants in the SOWG meeting couldn't be happier about it. The TPS [Thermal Protection Subsystem] folks don't seem too happy, though; SAP's reachability maps are showing that the areas they're most interested in aren't reachable by the IDD.
But they're having trouble getting this idea across, because they keep using the word "targets" to try to express themselves. This confuses the other science and engineering teams, because they think the TPS folks are trying to tell them that the SAP targets we picked yestersol are no longer valid, and they keep responding that yes, that's how the tool works, you just need to reselect them thisol. (When you pick out imaging or IDD targets in SAP on one sol, they're not automatically propagated to the next sol. This has been a common source of problems and confusion, so it's what the science team is geared to think the TPS guys are having a problem with.)
After a few minutes of this, I bring my translation skills to bear. I walk over to the TPS team and say, off mike, "You're just trying to say that the reachability maps show that the IDD can't go where you want it to, right?"
"Right!"
"You need to say that without using the magic word 'targets.'"
They look at me skeptically.
I give them my best Harrison Ford grin. "Trust me."
With one eye on me, one of them keys the mike and says what I told her to. It works instantly, breaking the confusion loop. I go back to my seat.
The meeting also stumbles over the lack of terrain meshes. SAP's reachability maps are just an approximation, and might be confused in this case by the weird (non-rock-or-soil-like) shape of the flankshield; Frank and I suspect that the areas the TPS team wants to see, though marked unreachable in SAP, are in fact reachable. RSVP can tell us that quickly, but we need a terrain mesh. The meshes are produced by another team, the MIPL team, and there seems to be all kinds of variability in the process.
Frank's getting annoyed, both by the delay on the meshes and by what he sees as the TPS team's negativity about the job we're doing for them. After answering the third or fourth question about reachability with "Well, we could tell you that if we had a terrain mesh, but ...," he's had it. "There seems to be a correlation," he mutters grimly, "between the trickiness of reaching targets and the latency of the fucking meshes."
There's also a tension developing between the science team and the TPS team, if I'm reading the room right. I think the scientists are ready to get out of here -- going to the heat shield was fun, but they're ready to get back to science. They want us to start charging across the plains to the etched terrain to the south of us. But the TPS folks would spend the rest of the mission here if they could; they want to explore the heat shield to death. Ray looks like he's as tired of these guys as Frank is, though he disguises it better, at least until the end of the meeting, when he snaps at last: "Okay, I'm fed up with this. Just pick some targets we can reach and let's get this thing done already."[1]
Later, after the TPS folks work with Alicia Fallacaro (one of the scientists) to pick targets, they sit down with me and Frank to ensure we're up to speed on their goals. At one point, they're showing us an image of the section they want an image of. "For scale," one of them says, "the width of this section is point six five."
"Point six five ... centimeters?" Frank asks.
She looks at him blankly. "No, inches."
"Inches?!" Frank and I cry in unison. We can't believe it. It's as if she'd given us a measurement in furlongs, or cubits. Or as if she'd specified her dates using the Mayan calendar.
But she doesn't know when to stop digging the hole. In fact, from the blank look on her face, I don't think she realizes she's in one. "I can't think in centimeters," she says innocently.
Frank and I are nearly livid. "How'd we ever get to Mars with you guys?!" he asks, incredulous.
"We don't always," I remind him.
This is a reference to the destruction of Mars Climate Orbiter ... you know, where Lockheed Martin Aerospace computed thrust in imperial units instead of metric, and just gave JPL the number without the unit, in violation of LMA's contract?[2]
And five years later, the problem's not solved. THAT'S EXACTLY WHAT HAPPENED JUST NOW. And -- wait, a NASA engineer who can't think in metric? It's as if she'd said she can't do arithmetic, or tie her own shoes. The guy with her isn't much better; he can convert to metric, but it didn't occur to him to do it until Frank and I blew up.
Holy. Fucking. Shit.
So they leave, and when Frank and I calm down, we return to working on the IDD sequence. By this time, the terrain meshes have mystically appeared. The areas the TPS team was interested in are indeed reachable, at least mostly, which is the good news. The bad news is that to reach one of the regions, we're going to have to move the IDD close to the right front wheel -- closer than we've ever done, and closer than the IDD flight software will allow. By default, the IDD flight software masks out certain regions around the rover's body, effectively making "no-go zones" where we're not allowed to place the arm. This is, obviously, a safety measure, but as with many safety measures, it's computed conservatively.
We dig up the commands to turn off wheel-volume collision testing and use RSVP to simulate putting the IDD where it needs to go. It's close. There's air, but it's close. And just because it's safe in RSVP doesn't mean it's necessarily safe in the real vehicle, which has cabling we don't model. Frank and I look at each other. "Testbed."
Frank and I don't know how to use the testbed rover -- it's supported by lots of complex test equipment, and we have no idea even how to boot the stuff -- so he manages to find Ashitey and Mark Maimone, and we all head down to the testbed to try it out there. (Ashitey's such a great guy. He's on shift driving Spirit and could easily have gotten out of this if he'd wanted to, but he comes down to help us out anyway.) Before leaving, I split out the dangerous part of the sequence, so that we can do the rest even if we can't prove that the scary bits are safe.
The testbed is -- well, it's the testbed. It takes hours to do anything there; it's all false starts. (In the end, it takes us three hours to run a ten-minute sequence, and this is pretty good by testbed standards.) I spend the time getting to know Mark's intern, who's in school but hopes to be a third-generation JPLer. His mom worked on Voyager, and his grandfather also worked here. Weirdly, his grandfather hired Jim Erickson, who is now MER's project manager. I try to get him to tell me some embarrassing stories about Jim, but he won't. ("Jim got me this job," he laughs, "I can't squeal on him." Damn engineers, with their honesty and consciences!) He does mention something about "Little Jimmy," though ....
And one of the other engineers is bringing a group of friends through. He asks someone to talk to them about what the rovers are doing now, and being an attention whore, I tell them about the heat shield exploration and why we're in the testbed. The cute Asian girl has her friend take her picture with me. I like being an attention whore.
When we get the sequence running at last, it's clear it's safe. I take a movie of the rover with my kick-ass Palm Zire 72, so I can show it to Leo in case there's any question later.
We get done just barely in time for the CAM, which now goes smoothly since we've validated the parts we're most worried about.
Frank and I are done, but neither of us goes home for a while. I've found a HyperDrive bug -- a potentially serious one; the code doesn't save and restore the state of the vehicle suspension. Thisol was exactly the sort of rare sol where that bug could have bitten us: we're moving the IDD close to the wheels, telling the vehicle to ignore collisions because we're going to do the checking manually, and the tools are lying to us about where the wheels are ... potentially very bad. It happens not to bite us this time, since the bug makes the rover look like it's on flat ground and Opportunity really is on flat ground, but Frank can't let it go. ("We can't go into the new year like this!") So he fixes the bug on the fly, and installs an updated version for testing.
Well, we'd been running the officially delivered version for about two weeks now. It was time we moved on. Ahem.
It's time to leave. I shake Frank's hand on the way out. "Pleasure working with you, dude. Happy New Year."
"Same to you, my friend."
[Next post: sol 358 (Opportunity sol 337), January 4.]
Courtesy NASA/JPL-Caltech. Flankshield, dead ahead, cap'n.
Footnotes:
[1] The TPS folks were connected to the heat shield manufacturer, Lockheed Martin Aerospace -- I think they were JPLers who'd worked with LMA on the heat shield design -- so they'd been part of MER's development. However, they were not part of the MER ops team, and it showed. This is not their fault, but ops has a certain pace, a certain rhythm, that outsiders just don't appreciate -- and, to be fair, we must somehow fail to communicate it to them. But this wasn't the only time we've had this sort of problem; it generally happens whenever we have any kind of outside expert participating in a tactical shift.
[2] Not to be pedantic (who, me?) or anything, but ... this is why it's doubly irritating when halfwits sneer at NASA's supposed inability to convert metric units. The mistake was a lot more complicated than that, a communication error (or perhaps a system engineering error) rather than an arithmetic error, and it was LMA rather than NASA (or JPL) that fundamentally erred. It's fine to be snide, when it's deserved -- as long as you are also right.
Also, strictly speaking, I suppose that LMA didn't compute using imperial units but rather US customary units, although for the unit in question -- foot-pounds, god help us -- the two systems are the same.
2009-12-31
2009-12-30
Opportunity Sol 332 (Spirit Sol 353)
The drive went perfectly. We're now on the other side of the flankshield, with the section of interest directly behind us. There's no nasty debris between us and our goal, so thisol's drive will be a simple one: we'll turn in place and bump a little over a meter to the flankshield.
Or maybe there won't be any point. The flankshield started off as kind of a big cardboard O; the impact trauma cut the O and twisted it into an S, with a vertical piece in the middle. This vertical piece is part of what the thermal protection folks want to MI, but the gap surrounding it might not be wide enough for us to fit the IDD's turret into. Frank checks it out, and while it looks like a tight squeeze, we can probably get to some of the bits in the center of the upright piece if we hold our breath.
The dust storm continues to be on everyone's mind. Frank thinks the images we're getting are cool, at least. The light's more diffuse; the shadows not as black.
As for the power effects, are we down the expected 20%? "No, just five percent," says Beth Dewell. "Hey, great!" I exclaim. Then I think about what I just said. "Yeah, our power is down five percent. Great ...."
Incidentally, we're not the only ones seeing increased tau -- Spirit saw a rise as well, though not as dramatic as ours and it's already falling off.
The drive's easy, so it should be a short sol. Should. But it doesn't work out that way: thanks to sloppy (or pessimistic) coders who assumed our rovers wouldn't last the year, one of the scripts our uplink process depends on breaks (it looks ahead a couple of days, sees a new year, and freaks out); it takes a couple of hours to find and fix this bug. So Frank and I have lots of time to read and chat.
One of the things we talk about is that the Opportunity Way is better -- specifically, the practice of having the RP-2 come in about the same time as the RP-1. "Back when Spirit was just charging across the plains, and every sol was pretty much the same as every other sol, we could get away with that," I say. "But when things are more interesting than that, that approach just doesn't work."
Frank agrees. "And it's great to have you here to bounce ideas off of or take the pressure off or whatever."
"Not to mention that if you come in later in the process and you think the RP-1's approach is flawed, it's too late to do anything about it." Oh, if only I could count the number of times I've regretted that ....
Courtesy NASA/JPL-Caltech. The heat shield, from our new perspective.
Courtesy NASA/JPL-Caltech. The "flankshield," pretty much straight behind us.
Courtesy NASA/JPL-Caltech. And last but not least, the impact crater.
Or maybe there won't be any point. The flankshield started off as kind of a big cardboard O; the impact trauma cut the O and twisted it into an S, with a vertical piece in the middle. This vertical piece is part of what the thermal protection folks want to MI, but the gap surrounding it might not be wide enough for us to fit the IDD's turret into. Frank checks it out, and while it looks like a tight squeeze, we can probably get to some of the bits in the center of the upright piece if we hold our breath.
The dust storm continues to be on everyone's mind. Frank thinks the images we're getting are cool, at least. The light's more diffuse; the shadows not as black.
As for the power effects, are we down the expected 20%? "No, just five percent," says Beth Dewell. "Hey, great!" I exclaim. Then I think about what I just said. "Yeah, our power is down five percent. Great ...."
Incidentally, we're not the only ones seeing increased tau -- Spirit saw a rise as well, though not as dramatic as ours and it's already falling off.
The drive's easy, so it should be a short sol. Should. But it doesn't work out that way: thanks to sloppy (or pessimistic) coders who assumed our rovers wouldn't last the year, one of the scripts our uplink process depends on breaks (it looks ahead a couple of days, sees a new year, and freaks out); it takes a couple of hours to find and fix this bug. So Frank and I have lots of time to read and chat.
One of the things we talk about is that the Opportunity Way is better -- specifically, the practice of having the RP-2 come in about the same time as the RP-1. "Back when Spirit was just charging across the plains, and every sol was pretty much the same as every other sol, we could get away with that," I say. "But when things are more interesting than that, that approach just doesn't work."
Frank agrees. "And it's great to have you here to bounce ideas off of or take the pressure off or whatever."
"Not to mention that if you come in later in the process and you think the RP-1's approach is flawed, it's too late to do anything about it." Oh, if only I could count the number of times I've regretted that ....
Courtesy NASA/JPL-Caltech. The heat shield, from our new perspective.
Courtesy NASA/JPL-Caltech. The "flankshield," pretty much straight behind us.
Courtesy NASA/JPL-Caltech. And last but not least, the impact crater.
2009-12-29
Opportunity Sol 331 (Spirit Sol 352)
Everything went well over Xmas, and yestersol Jeff and Frank drove to the next spot, maybe ten meters from the broken-off fragment of the heat shield. (We're calling this the "flank" or "flankshield.") The thermal protection team is interested in this piece, so we're going to drive to it over the next couple of sols and MI it.
Because of the surrounding debris, driving in this area continues to have its dangers. We can't drive where we can't see. Generally, this means we're limited to driving no farther than we can see in the NAVCAMs (maybe 15m or 20m). Another constraint is that we can't drive to a point where we don't have imaging, such as the region we currently can't see on the other side of the flankshield. When there are only rocks around, we can make reasonable assumptions about what might exist in areas we can't see, but with artificial hazards, we can't safely assume much of anything. (For example, I'm personally kind of worried about the six springs that were attached to the heat shield. We've found three, maybe four, so far, leaving two or three more for us to discover. You know how it's easy to find the tack you dropped on the carpet? Just walk barefoot across the floor ....) As a result of these constraints, even though the drive to the far side of the flankshield is a fairly short one -- maybe 12m or so total -- it'll take us at least two sols, one to drive to the far side of the flankshield, and another to bump from there into the zone we can't see from here.
Assuming the rover lives that long. Well, maybe that's pessimistic of me, but there is a developing concern about our power situation. Alarmingly, our power went down 13% yestersol and another 17% thisol -- that translates to about 100 Watt-hours each sol, a considerable fraction of the 800 Watt-hours or so that we started with.
At least the cause is known: PANCAM tau (atmospheric opacity) measurements indicate a big dust storm. We have a big meeting about this, which I cover so Frank can work on the drive. Frank and I both hate meetings, so I decide to get some mileage out of this: "You owe me one!" Here's what comes out of this meeting:
As the meeting is breaking up, Jim Erickson walks in and tries to discreetly take a seat in the back. (Jim's on vacation this week, so naturally he's here at work.) Art turns to him and says, "We drove your spacecraft into a storm."
Jim shrugs. "It's our spacecraft," he corrects Art, "and storms happen."
When I return to the sequencing room, I realize I have something to say to Frank. "Okay, that meeting was really cool. You don't owe me one."
Courtesy NASA/JPL-Caltech. The heat shield ...
Courtesy NASA/JPL-Caltech. ... and the large fragment that broke off it.
Courtesy NASA/JPL/Cornell. The heat shield, in color. (Follow the link for a full-sized image.)
Courtesy NASA/JPL/Cornell. The whole scene, from the sol before, in beautiful color. This is just magnificent, truly one of the signature images of the entire mission. It's a piece of a crashed spaceship on another world. OK, so it's our spaceship -- you think that makes it less impressive? (Follow the link for a full-sized image.)
Because of the surrounding debris, driving in this area continues to have its dangers. We can't drive where we can't see. Generally, this means we're limited to driving no farther than we can see in the NAVCAMs (maybe 15m or 20m). Another constraint is that we can't drive to a point where we don't have imaging, such as the region we currently can't see on the other side of the flankshield. When there are only rocks around, we can make reasonable assumptions about what might exist in areas we can't see, but with artificial hazards, we can't safely assume much of anything. (For example, I'm personally kind of worried about the six springs that were attached to the heat shield. We've found three, maybe four, so far, leaving two or three more for us to discover. You know how it's easy to find the tack you dropped on the carpet? Just walk barefoot across the floor ....) As a result of these constraints, even though the drive to the far side of the flankshield is a fairly short one -- maybe 12m or so total -- it'll take us at least two sols, one to drive to the far side of the flankshield, and another to bump from there into the zone we can't see from here.
Assuming the rover lives that long. Well, maybe that's pessimistic of me, but there is a developing concern about our power situation. Alarmingly, our power went down 13% yestersol and another 17% thisol -- that translates to about 100 Watt-hours each sol, a considerable fraction of the 800 Watt-hours or so that we started with.
At least the cause is known: PANCAM tau (atmospheric opacity) measurements indicate a big dust storm. We have a big meeting about this, which I cover so Frank can work on the drive. Frank and I both hate meetings, so I decide to get some mileage out of this: "You owe me one!" Here's what comes out of this meeting:
- On sol 328, we had 740 W-hr and a tau about 0.6; on sol 329, 670 W-hr and a tau about 0.8; on sol 330, 563 W-hr and a tau about 1.2. For reference, there was a dust storm just before we landed, and since then the tau has been less than 0.45 or so. Conservatively, we're going to plan on seeing 20% degradation per sol until we bottom out at ~350 W-hr (which will take only another 2 or 3 sols at that rate). That would correspond to a tau of about 3. The safe assumption is that we have not yet seen the worst.
Art skates a tau plot over to me. It's the whole story in one picture. There's this nice, steady decline for more than three hundred sols, and then, just at the right edge of the graph, bam!, it shoots up off the scale. - We're going to ask MGS, ODY, and maybe MEX if they have data that can add anything to our understanding of the situation. If not, maybe they'd like to gather some -- we have ground truth to offer them, making it potentially win-win. But one of the participants online says, "It would have to be a hell of a storm for the orbiters to see it." So in a sense, if we're lucky, that won't work out.
- I remember something I read, pre-landing, about how Martian dust storms work. The dust traps heat like a blanket, making the local area warmer; this causes wind to rush in from colder areas, bringing more dust, which thickens the blanket and makes the area still warmer, and so on. In other words, once the winds get started, they can start a feedback loop that causes them to accelerate. So, lots of wind, and scattered debris nearby ....
"Hey," I ask, "will the wind blow this debris into the cameras or the wheels? Should we get the heck out of here -- maybe go back and hide in the crater? Maybe just find a way to get upwind?"
This idea causes some consternation, but the consensus is that there's no likely danger. Mars's air is so thin that the wind force is about the square root of what we see on Earth -- a 100km/hr wind there would be like a 10km/hr wind here. (Another interesting tidbit that comes out of this: Art mentions that the crater has its own weather system, a fascinating fact I'll have to remember to follow up on.)
One of the online experts also seems to think the local winds aren't significant anyway. "This dust could be just drifting in from a storm kilometers away," he says. "In fact, I'd put my money on it."
Still, Art wants someone to keep an eye on this. "Can we tell the wind direction?" he asks. People start chewing on the problem. "We could watch the debris move." "Watch a pile of dust?" "Make a pile of dust with the wheels and watch that."
I love working with smart people. - "Statistically," the disembodied voice offers, "there are two things working in your favor. First, MGS has been here, what, three Martian years or so, and has seen no global dust events in this season. That's consistent with what Viking saw. And second, tau will fall off faster than you saw from the post-landing storm. At this season, water ice will nucleate around the airborne dust and help scrub it."
- Summary consensus from the meeting: we should act cautiously, but this is not a spacecraft-threatening event.
As the meeting is breaking up, Jim Erickson walks in and tries to discreetly take a seat in the back. (Jim's on vacation this week, so naturally he's here at work.) Art turns to him and says, "We drove your spacecraft into a storm."
Jim shrugs. "It's our spacecraft," he corrects Art, "and storms happen."
When I return to the sequencing room, I realize I have something to say to Frank. "Okay, that meeting was really cool. You don't owe me one."
Courtesy NASA/JPL-Caltech. The heat shield ...
Courtesy NASA/JPL-Caltech. ... and the large fragment that broke off it.
Courtesy NASA/JPL/Cornell. The heat shield, in color. (Follow the link for a full-sized image.)
Courtesy NASA/JPL/Cornell. The whole scene, from the sol before, in beautiful color. This is just magnificent, truly one of the signature images of the entire mission. It's a piece of a crashed spaceship on another world. OK, so it's our spaceship -- you think that makes it less impressive? (Follow the link for a full-sized image.)
2009-12-24
Opportunity Sol 326 (Spirit Sol 347)
"Nice job!" Ray says as I walk into the SOWG meeting. It looks like our drive went just about perfectly, though some of the post-drive imaging missed its target. (Naturally, I initially suspect that this is somehow my fault. Instead, it turns out to be simply a result of poor range data from our pre-drive position -- our targets were on the hairy edge of the NAVCAMs' capabilities, and we got unlucky.)
Supposedly, the weekend is already planned out -- we did it yesterday. But, as is so often the case, it's not that simple. I initially see something in one of the PANCAM images -- a spring that dropped off the heat shield -- that makes me nervous because I can't see it where it should be in the NAVCAMs, and I'm worried we'll drive over it or some other piece of debris we can't see. (We'd probably be okay anyway, but artificial objects don't always behave like rocks; it's possible for us to get a spring wound around our axle where a comparably sized rock would be no hazard at all.) So I ask for some more drive-supporting imaging in the plan, and waste some time trying to figure out where the hell the spring is. Then I figure out it's not even close to where I thought it was; I was thrown off by the mis-pointed PANCAM images. It's nowhere close to the rover. Once I realize that, it's a simple matter to work out that we already have all the imaging we need, and we don't need to change the plan at all.
While I'm doing that, Justin Maki and Jim Bell are having an unusually, and increasingly, acrimonious argument on the telecon. Jim really wants to spend part of the weekend taking sky images to help calibrate the PANCAM (his instrument). Justin thinks this is a bad idea; as valuable as the calibration images would be, now's not the time to take them, he says. (As Justin puts it: "We're at the heat shield right now. The sky will still be there next week.") The discussion reaches such a pitch of bitterness that Ray must step in to resolve it. (Justin wins, but Jim doesn't give up. "This is your call, Ray," he says, "but I disagree with it.")
Frank mostly built thisol's sequences yesterday, making today a light day. I have time to entertain a couple of guests Art brings through, and to show off RSVP's 3-D image browser to Jay Torres. Jay's one of the TAP/SIEs -- he works with the scientists to create the detailed plan for the sol, then does most of the hard work involved in integrating the sequences and preparing them for the spacecraft. Since he's expressed an interest, I load up some of the coolest 3-D images of Wopmay and hand him a pair of the LCD shutter goggles to view them.
He looks through the goggles, and his jaw drops. "Woah, you have a cool job," he says. "Our job sucks."
"No, your job is cool," I reassure him. "You're miscalibrated. Most people, like, sell insurance and stuff.[1] They would kill to do what you do. Your job is cool!"
He laughs. "Well, okay," he says, "but your job is cooler."
No argument. This is something I've been thinking about lately: JPL is already one of the greatest places in the world to work, and even other JPLers envy me. Is it shallow or selfish of me to absolutely love this?
So be it.
Though the day's a generally simple one, I do spot a potential problem at the CAM, the very last review of the day. The first IDD sequence ends with a tricky maneuver that places the MB on the capture magnet; the second sequence retracts the MB to start some other work. Since it expects the IDD to be in contact with the rover, the second sequence temporarily turns off the IDD flight software's normal self-collision detection, which is what keeps the IDD from whacking into the rover body. The thought that occurs to me at the CAM is, What if the first sequence doesn't make it up to the spacecraft? Then the second sequence will start off stowed, and we'll turn off collision detection and tell the arm to move .... That could be bad, very bad.
Fortunately, we have a sort of stock workaround for this type of problem, so the solution is well understood and we just have to take a few minutes to implement it. "Good catch, Scott," Frank says. "And I'm glad I was here for the last time the rover drivers derail the CAM in 2004."[2]
So we finish, and then we head downstairs to check out a happy rumor: we've heard Spirit is depotatoized! It's a rumor Chris and Ashitey are glad to confirm. Indeed, the news is even better than we'd thought: Art told us they weren't sure where the rock had gone, so it could still be in a part of the wheel we can't see. But Chris and Ashitey found that they could indeed see a sliver of it in one of the images, and it is definitely ejected.
We have a new rule to live by: when driving in this type of terrain (loose sand, steep slopes, and rocks), don't try to overcommand the drive -- that is, don't tell the rover to drive much farther than it really needs to go as a way of accounting for large amounts of slip. When you do, you make it more likely the rover will sink in and pick up a rock like this one -- that was the essence of how we got into this problem. Better to turn a one-day drive into a two-day drive, than turn it into a week-long nightmare. When climbing the rest of this hill, it's slow and steady for Spirit.
But for now, all's well. Spirit is potato-free, all six wheels once again in working order, and ready to resume its uphill charge. Opportunity is performing historic work, imaging its own heat shield on Mars. The team couldn't be happier, and neither could I.
So, from Mars to Earth, Merry Christmas. And Happy Festivus, for the rest of us.
[Next post: sol 352 (Opportunity sol 331), December 29.]
Footnotes:
[1] No offense.
[2] The year wasn't over yet. In retrospect, that seems unusually optimistic for Frank. ;-)
Supposedly, the weekend is already planned out -- we did it yesterday. But, as is so often the case, it's not that simple. I initially see something in one of the PANCAM images -- a spring that dropped off the heat shield -- that makes me nervous because I can't see it where it should be in the NAVCAMs, and I'm worried we'll drive over it or some other piece of debris we can't see. (We'd probably be okay anyway, but artificial objects don't always behave like rocks; it's possible for us to get a spring wound around our axle where a comparably sized rock would be no hazard at all.) So I ask for some more drive-supporting imaging in the plan, and waste some time trying to figure out where the hell the spring is. Then I figure out it's not even close to where I thought it was; I was thrown off by the mis-pointed PANCAM images. It's nowhere close to the rover. Once I realize that, it's a simple matter to work out that we already have all the imaging we need, and we don't need to change the plan at all.
While I'm doing that, Justin Maki and Jim Bell are having an unusually, and increasingly, acrimonious argument on the telecon. Jim really wants to spend part of the weekend taking sky images to help calibrate the PANCAM (his instrument). Justin thinks this is a bad idea; as valuable as the calibration images would be, now's not the time to take them, he says. (As Justin puts it: "We're at the heat shield right now. The sky will still be there next week.") The discussion reaches such a pitch of bitterness that Ray must step in to resolve it. (Justin wins, but Jim doesn't give up. "This is your call, Ray," he says, "but I disagree with it.")
Frank mostly built thisol's sequences yesterday, making today a light day. I have time to entertain a couple of guests Art brings through, and to show off RSVP's 3-D image browser to Jay Torres. Jay's one of the TAP/SIEs -- he works with the scientists to create the detailed plan for the sol, then does most of the hard work involved in integrating the sequences and preparing them for the spacecraft. Since he's expressed an interest, I load up some of the coolest 3-D images of Wopmay and hand him a pair of the LCD shutter goggles to view them.
He looks through the goggles, and his jaw drops. "Woah, you have a cool job," he says. "Our job sucks."
"No, your job is cool," I reassure him. "You're miscalibrated. Most people, like, sell insurance and stuff.[1] They would kill to do what you do. Your job is cool!"
He laughs. "Well, okay," he says, "but your job is cooler."
No argument. This is something I've been thinking about lately: JPL is already one of the greatest places in the world to work, and even other JPLers envy me. Is it shallow or selfish of me to absolutely love this?
So be it.
Though the day's a generally simple one, I do spot a potential problem at the CAM, the very last review of the day. The first IDD sequence ends with a tricky maneuver that places the MB on the capture magnet; the second sequence retracts the MB to start some other work. Since it expects the IDD to be in contact with the rover, the second sequence temporarily turns off the IDD flight software's normal self-collision detection, which is what keeps the IDD from whacking into the rover body. The thought that occurs to me at the CAM is, What if the first sequence doesn't make it up to the spacecraft? Then the second sequence will start off stowed, and we'll turn off collision detection and tell the arm to move .... That could be bad, very bad.
Fortunately, we have a sort of stock workaround for this type of problem, so the solution is well understood and we just have to take a few minutes to implement it. "Good catch, Scott," Frank says. "And I'm glad I was here for the last time the rover drivers derail the CAM in 2004."[2]
So we finish, and then we head downstairs to check out a happy rumor: we've heard Spirit is depotatoized! It's a rumor Chris and Ashitey are glad to confirm. Indeed, the news is even better than we'd thought: Art told us they weren't sure where the rock had gone, so it could still be in a part of the wheel we can't see. But Chris and Ashitey found that they could indeed see a sliver of it in one of the images, and it is definitely ejected.
We have a new rule to live by: when driving in this type of terrain (loose sand, steep slopes, and rocks), don't try to overcommand the drive -- that is, don't tell the rover to drive much farther than it really needs to go as a way of accounting for large amounts of slip. When you do, you make it more likely the rover will sink in and pick up a rock like this one -- that was the essence of how we got into this problem. Better to turn a one-day drive into a two-day drive, than turn it into a week-long nightmare. When climbing the rest of this hill, it's slow and steady for Spirit.
But for now, all's well. Spirit is potato-free, all six wheels once again in working order, and ready to resume its uphill charge. Opportunity is performing historic work, imaging its own heat shield on Mars. The team couldn't be happier, and neither could I.
So, from Mars to Earth, Merry Christmas. And Happy Festivus, for the rest of us.
[Next post: sol 352 (Opportunity sol 331), December 29.]
Footnotes:
[1] No offense.
[2] The year wasn't over yet. In retrospect, that seems unusually optimistic for Frank. ;-)
2009-12-23
Opportunity Sol 325 (Spirit Sol 346)
[Yes, this makes two sols in a row labeled "Opportunity Sol 325." That's because we spent two Earth days working on this sol, as a way of taking advantage of the on-the-cusp time phasing between Earth days and Martian sols.]
Since it's a tight sol and we have more than one sol to plan, we start early -- 7 AM.
"Just like old times," I say as I walk into the SOWG meeting. "Getting up before dark ...."
"Scott loves this time of day," Rick Welch says.
"No, no, I just love working on this mission. I hate this time of day."
Yesterday's drive could hardly have gone better -- a "spectacular drive," as Ray calls it. Truthfully, this has practically nothing to do with my skill, talent, or intelligence, formidable as they may be; the hazard-free terrain is just making me look good. Who couldn't drive this thing in a parking lot? Nevertheless, I (and Jeff) do now hold the record for the longest single blind drive on Mars: 91m.
Anyway, we're now about 30m from West Point, the spot where the scientists want Opportunity to spend a long Christmas weekend taking pictures of the heat shield, its impact site and debris trail. The drive is once again a snap: pick an azimuth (Ray estimates 172deg, but I bias it to avoid debris and it ends up being more like 179deg), then drive a long leg, turn to face the heat shield, and take a half-meter step in its direction.
"Damn, this is easier than driving in the Spirit World," I say to Frank.
"Yeah, this plains driving is easy," he agrees. "'Eh, I got a thirty-meter error bar on this drive ....' Talk about low stress."
Speaking of the Spirit World -- Ashitey stops by to congratulate us on our new record, so of course we ask him how the depotatoization is going.
"You don't want to know," he says. He seems to mean it, but we cajole him into telling us anyway. They did some testing in the testbed over the last couple of days and had a hard time dislodging potatoes. (Must have been an intense session -- Frank and I keep throwing out ideas, and Ashitey keeps saying, "Well, we tried that ....") The next thing they're trying is a short, straight downhill drive, which they hope will lift the stuck wheel out of its hole; even if this doesn't remove the rock, it'll put them in a better position for continued attempts.
Spirit isn't the only JPL spacecraft having problems. MGS went into "mode C" -- i.e., contingency mode, which is what we call it when a spacecraft has a panic attack. As a result, they're looking for additional DSN coverage; this may cause them to take station time away from Odyssey. And since we normally uplink via Odyssey, this would mean we'd be unable to command the rovers as planned "for a while." In particular, we'd likely miss out on the weekend uplink. Both of our rovers will be fine; they'll just execute run-out sequences. We'll have to delay our drive and West Point imaging, and the Spirits will have to delay their potato ejection; as a result, both rovers will have a relatively lazy Christmas weekend. Hmm, maybe I should be hoping for this.
Courtesy NASA/JPL/Cornell. Behold, the heat shield approacheth! One giant step closer, and even more beautiful. (Follow the link for the full-sized image.)
Since it's a tight sol and we have more than one sol to plan, we start early -- 7 AM.
"Just like old times," I say as I walk into the SOWG meeting. "Getting up before dark ...."
"Scott loves this time of day," Rick Welch says.
"No, no, I just love working on this mission. I hate this time of day."
Yesterday's drive could hardly have gone better -- a "spectacular drive," as Ray calls it. Truthfully, this has practically nothing to do with my skill, talent, or intelligence, formidable as they may be; the hazard-free terrain is just making me look good. Who couldn't drive this thing in a parking lot? Nevertheless, I (and Jeff) do now hold the record for the longest single blind drive on Mars: 91m.
Anyway, we're now about 30m from West Point, the spot where the scientists want Opportunity to spend a long Christmas weekend taking pictures of the heat shield, its impact site and debris trail. The drive is once again a snap: pick an azimuth (Ray estimates 172deg, but I bias it to avoid debris and it ends up being more like 179deg), then drive a long leg, turn to face the heat shield, and take a half-meter step in its direction.
"Damn, this is easier than driving in the Spirit World," I say to Frank.
"Yeah, this plains driving is easy," he agrees. "'Eh, I got a thirty-meter error bar on this drive ....' Talk about low stress."
Speaking of the Spirit World -- Ashitey stops by to congratulate us on our new record, so of course we ask him how the depotatoization is going.
"You don't want to know," he says. He seems to mean it, but we cajole him into telling us anyway. They did some testing in the testbed over the last couple of days and had a hard time dislodging potatoes. (Must have been an intense session -- Frank and I keep throwing out ideas, and Ashitey keeps saying, "Well, we tried that ....") The next thing they're trying is a short, straight downhill drive, which they hope will lift the stuck wheel out of its hole; even if this doesn't remove the rock, it'll put them in a better position for continued attempts.
Spirit isn't the only JPL spacecraft having problems. MGS went into "mode C" -- i.e., contingency mode, which is what we call it when a spacecraft has a panic attack. As a result, they're looking for additional DSN coverage; this may cause them to take station time away from Odyssey. And since we normally uplink via Odyssey, this would mean we'd be unable to command the rovers as planned "for a while." In particular, we'd likely miss out on the weekend uplink. Both of our rovers will be fine; they'll just execute run-out sequences. We'll have to delay our drive and West Point imaging, and the Spirits will have to delay their potato ejection; as a result, both rovers will have a relatively lazy Christmas weekend. Hmm, maybe I should be hoping for this.
Courtesy NASA/JPL/Cornell. Behold, the heat shield approacheth! One giant step closer, and even more beautiful. (Follow the link for the full-sized image.)
2009-12-22
Opportunity Sol 325 (Spirit Sol 345)
It's a skip sol -- the drive we planned yesterday hasn't even started executing yet, so we can't sequence the next one. But because of the upcoming Christmas break (that is, we hope we get a break), we're getting a head start on tomorrow's two-sol plan.
Ray lays out the goal for the sol-325 drive, the one we'll plan tomorrow morning: we want to get to a spot we're calling "North Point," 20m north of the heat shield. We work out an estimate of the needed drive resources, but until we know how this drive went, we can't do too much more. (That is, we can't do too much more with respect to the driving; the scientists have a lot to talk about.)
I do manage to enshrine the half-bit-per-pixel penultimate FHAZ in the plan, presumably making it the default for many sols to come. They've got 180Mbits of imaging coming up, and I'm determined to save them every megabit I can -- whether they want it or not, damn it!
Ray lays out the goal for the sol-325 drive, the one we'll plan tomorrow morning: we want to get to a spot we're calling "North Point," 20m north of the heat shield. We work out an estimate of the needed drive resources, but until we know how this drive went, we can't do too much more. (That is, we can't do too much more with respect to the driving; the scientists have a lot to talk about.)
I do manage to enshrine the half-bit-per-pixel penultimate FHAZ in the plan, presumably making it the default for many sols to come. They've got 180Mbits of imaging coming up, and I'm determined to save them every megabit I can -- whether they want it or not, damn it!
2009-12-21
Opportunity Sol 324 (Spirit Sol 344)
Despite my misgivings about it, the drive appears to have gone splendidly. The heat shield looks significantly closer, though range data indicates it's still about 130m away. There are a lot of jokes about this being like a dream: the closer we get to it, the farther it recedes. Some of us -- those who are hoping this isn't our heat shield but some alien artifact[1] -- prefer the explanation that it's moving away from us to stay at the edge of our sensor range. Next thing you know, it will be modulating its shield harmonics.
However, we're not sure exactly how far away it is. Estimates range from 100m to 150m, my personal favorite being the 132m estimate the PANCAM data suggests. Since we're trying to get 40m away as of this next drive, and we also want to keep at least 20m away in order to avoid a possible debris field, this uncertainty is a problem. I decide to drive 90m in its direction, but aimed slightly to the right. That way, if the range to the heat shield is on the low side, we'll still be safe; if I'm right, we'll be smack where we want to be.
It's a gamble, but it's one I feel really good about. Opportunity is eating distance like it was born to (which, come to think of it, is true), and the PANCAMs have been pretty reliable in these circumstances before.
What I don't know, until after I've already planned the drive, is that 90m is a record-setting drive. Not in terms of one-sol distance, but it's the longest blind drive ever -- all previous drives longer than that have used autonav for anything past about 70m. And I thought the days (sols) of record-setting drives were over!
The lengthy blind drive makes some people nervous. To my own surprise, I manage to talk Jeff Biesiadecki (RP-2 today) into it, but Mark Maimone is a tougher sell. Jeff has to leave to support the ongoing depotatoization efforts on the other side of the planet, but Mark and I stay upstairs and argue -- or, as I try to think of it, negotiate.
I look at the terrain and see flat, flat, flat all the way to the horizon. Mark's more conservative: he agrees that what we can see looks flat, but he points out that our imaging resolution diminishes rapidly after 60m or so, and it's theoretically possible there are hazard-sized dips along our path that would be invisible from our starting location.
Unlikely? Sure, but I have to admit it's possible, for sufficiently small values of "possible." We've already configured the rover to stop if it finds itself tilting more than 12 degrees, or if the suspension articulates significantly, which should catch problems like this. But Mark doesn't feel that the limit is set tight enough. So we look back at the actuals from the past few plain drives and choose newer, even tighter limits that we tell the rover to install after it's gone 65m. (Six degrees of tilt or a 6.5-degree bogie angle, and the rover will stop cold. With those limits, it won't take much more than a pebble to halt us.) So it'll be "normally conservative" for the first two-thirds of the drive and ultra-conservative after that. Neither of us is really happy with the result, which is always the sign of a good compromise, so I let it go.
When Jeff returns from the far side of the world, I bring him up to speed on the changes. He notices an extra command I added when making these modifications -- harmless, but unnecessary -- and since it's so late in the process, we just leave it in. When I shake my head in mock-disgust at my mistake, he tries to console me.
"Well, it's harmless," he points out.
"It'll take one second to execute, though." (All commands take at least that long.)
"I did the math once," he says. "I think I figured the rover cost about ten thousand dollars per second during the nominal mission. Of course, it's cheaper than that now."
"Hey, that gives me an idea! We could have each command sponsored by a corporation. Put it in the comments field: 'This command brought to you by Intel.' We'd make a mint!"
If only.
(For the record: The rovers cost about $45 per second during the nominal mission -- that's $4 million per sol, divided by the number of seconds in a sol. The extended mission has reduced the cost by about a factor of three so far, so they're maybe $15 per second.[2] So I guess I owe the project $15 for my mistake. Let 'em take it out of my paycheck.)
Jeff also shows me a video of the rover driving in the testbed with limits similar to the ones Mark and I agreed on. This is disheartening: it seems to stop the instant one wheel begins to dip into a hole. Indeed, the limits aren't far off what the rover can achieve, even on flat terrain, by just steering the wheels to the turn-in-place configuration. Partly, I'm worried that this will ruin my chance to set another record, but that's not a big deal -- I've had my share of fun setting interplanetary records, and then some.
I'm more concerned about the possibility of needlessly delaying science experiments, but I decide the right thing to do is to let it go. Since sol 1, my policy has been to honor the most conservative (yet still reasonable) engineering analysis. Today that happened to be Mark's. ("I can't remember the last time I wasn't the most conservative RP," Jeff comments.) I can't help chafing at this particular result, since I disagree with the analysis, but I firmly believe the policy's the right one. So if the drive gets cut short, we'll just have to live with it. We have another sol before we need to be in our final weekend position anyway, and if the drive is cut off 25m short, we should still be able to make it.
"We'll just have to cross our fingers, I guess," I say to Jeff.
"Yeah," he says, "but I'd rather be crossing my fingers this way than the other way, you know what I mean?"
Sad to say, I do. Ah, we'll be fine. Besides, the numbers we picked would have let us do the last couple of sols of driving, so since part of my own argument is that the road ahead is like the one behind, we should have nothing to worry about.
Not that that'll stop me.
Just before the CAM, we get a new image: a beautiful color PANCAM showing the heat shield, apparently broken and bent from the impact. I suddenly want to see the damn thing up close. I have to see it!
"I don't care what Mark says, I'm setting the tilt limit to fifty degrees!"
Jeff laughs. "Hey, if the command dictionary allows it, it must be safe, right?"
Courtesy NASA/JPL/Cornell. One of several approximate-true-color PANCAM images we took as we approached Opportunity's heat shield, and probably the one I was looking at then. (Follow the link for the full-sized image.)
Footnotes:
[1] Just a joke. Don't get excited. :-)
[2] Since you're no doubt wondering: I don't have the budget numbers I'd need for an exact answer, but we're under $5 per second now (amortized over the whole mission).
However, we're not sure exactly how far away it is. Estimates range from 100m to 150m, my personal favorite being the 132m estimate the PANCAM data suggests. Since we're trying to get 40m away as of this next drive, and we also want to keep at least 20m away in order to avoid a possible debris field, this uncertainty is a problem. I decide to drive 90m in its direction, but aimed slightly to the right. That way, if the range to the heat shield is on the low side, we'll still be safe; if I'm right, we'll be smack where we want to be.
It's a gamble, but it's one I feel really good about. Opportunity is eating distance like it was born to (which, come to think of it, is true), and the PANCAMs have been pretty reliable in these circumstances before.
What I don't know, until after I've already planned the drive, is that 90m is a record-setting drive. Not in terms of one-sol distance, but it's the longest blind drive ever -- all previous drives longer than that have used autonav for anything past about 70m. And I thought the days (sols) of record-setting drives were over!
The lengthy blind drive makes some people nervous. To my own surprise, I manage to talk Jeff Biesiadecki (RP-2 today) into it, but Mark Maimone is a tougher sell. Jeff has to leave to support the ongoing depotatoization efforts on the other side of the planet, but Mark and I stay upstairs and argue -- or, as I try to think of it, negotiate.
I look at the terrain and see flat, flat, flat all the way to the horizon. Mark's more conservative: he agrees that what we can see looks flat, but he points out that our imaging resolution diminishes rapidly after 60m or so, and it's theoretically possible there are hazard-sized dips along our path that would be invisible from our starting location.
Unlikely? Sure, but I have to admit it's possible, for sufficiently small values of "possible." We've already configured the rover to stop if it finds itself tilting more than 12 degrees, or if the suspension articulates significantly, which should catch problems like this. But Mark doesn't feel that the limit is set tight enough. So we look back at the actuals from the past few plain drives and choose newer, even tighter limits that we tell the rover to install after it's gone 65m. (Six degrees of tilt or a 6.5-degree bogie angle, and the rover will stop cold. With those limits, it won't take much more than a pebble to halt us.) So it'll be "normally conservative" for the first two-thirds of the drive and ultra-conservative after that. Neither of us is really happy with the result, which is always the sign of a good compromise, so I let it go.
When Jeff returns from the far side of the world, I bring him up to speed on the changes. He notices an extra command I added when making these modifications -- harmless, but unnecessary -- and since it's so late in the process, we just leave it in. When I shake my head in mock-disgust at my mistake, he tries to console me.
"Well, it's harmless," he points out.
"It'll take one second to execute, though." (All commands take at least that long.)
"I did the math once," he says. "I think I figured the rover cost about ten thousand dollars per second during the nominal mission. Of course, it's cheaper than that now."
"Hey, that gives me an idea! We could have each command sponsored by a corporation. Put it in the comments field: 'This command brought to you by Intel.' We'd make a mint!"
If only.
(For the record: The rovers cost about $45 per second during the nominal mission -- that's $4 million per sol, divided by the number of seconds in a sol. The extended mission has reduced the cost by about a factor of three so far, so they're maybe $15 per second.[2] So I guess I owe the project $15 for my mistake. Let 'em take it out of my paycheck.)
Jeff also shows me a video of the rover driving in the testbed with limits similar to the ones Mark and I agreed on. This is disheartening: it seems to stop the instant one wheel begins to dip into a hole. Indeed, the limits aren't far off what the rover can achieve, even on flat terrain, by just steering the wheels to the turn-in-place configuration. Partly, I'm worried that this will ruin my chance to set another record, but that's not a big deal -- I've had my share of fun setting interplanetary records, and then some.
I'm more concerned about the possibility of needlessly delaying science experiments, but I decide the right thing to do is to let it go. Since sol 1, my policy has been to honor the most conservative (yet still reasonable) engineering analysis. Today that happened to be Mark's. ("I can't remember the last time I wasn't the most conservative RP," Jeff comments.) I can't help chafing at this particular result, since I disagree with the analysis, but I firmly believe the policy's the right one. So if the drive gets cut short, we'll just have to live with it. We have another sol before we need to be in our final weekend position anyway, and if the drive is cut off 25m short, we should still be able to make it.
"We'll just have to cross our fingers, I guess," I say to Jeff.
"Yeah," he says, "but I'd rather be crossing my fingers this way than the other way, you know what I mean?"
Sad to say, I do. Ah, we'll be fine. Besides, the numbers we picked would have let us do the last couple of sols of driving, so since part of my own argument is that the road ahead is like the one behind, we should have nothing to worry about.
Not that that'll stop me.
Just before the CAM, we get a new image: a beautiful color PANCAM showing the heat shield, apparently broken and bent from the impact. I suddenly want to see the damn thing up close. I have to see it!
"I don't care what Mark says, I'm setting the tilt limit to fifty degrees!"
Jeff laughs. "Hey, if the command dictionary allows it, it must be safe, right?"
Courtesy NASA/JPL/Cornell. One of several approximate-true-color PANCAM images we took as we approached Opportunity's heat shield, and probably the one I was looking at then. (Follow the link for the full-sized image.)
Footnotes:
[1] Just a joke. Don't get excited. :-)
[2] Since you're no doubt wondering: I don't have the budget numbers I'd need for an exact answer, but we're under $5 per second now (amortized over the whole mission).
2009-12-18
Opportunity Sol 321 (Spirit Sol 341)
I'm back upstairs in the Land of Opportunity. Looks like an easy one: MIing the solar panels (using a canned sequence) and then a 60m drive, mostly straight, toward the heatshield.
Quite a relief, I must admit, to be away from the depotatoization stress over in the Spirit World. When I see Jeng and he asks if I'm on Spirit thisol, I tell him I'm on Opportunity and what we're up to. "Man," he says, shaking his head, "you're always in the middle of the action." Yeah, but this potato thing is something I could stand not to be in the middle of.
On the upside, there's a rumor that we've been named Time's Person of the Year. Other happy news: the ITAR issue is dead, Jim Erickson announces.
The ITAR issue was a concern about the upcoming imaging of the heatshield. ITAR stands for "International Trafficking in Arms Regulations," the set of USA laws covering -- well, what it says. See, since the heatshield is part of a re-entry vehicle, there was some worry that releasing images of it would be tantamount to releasing information about weapons systems. Which is not as crazy as you might think: for good reasons, all spacecraft are legally classified as weapons systems, so we have to be careful about all information we release, same as if they were ICBMs. (Because they easily could be: replace Opportunity with a nuke and change the launch trajectory a hair, and it's goodbye, Luxembourg. Or something like that.) There was also some concern that we'd be unlawfully releasing proprietary information held by Lockheed Martin, who built the reentry system, but that was secondary to the ITAR worries.
Naturally, Jim didn't want his entire uplink team jailed for putting pictures on the Web, so he had the legal department look into it. And in a gratifyingly sensible decision, the legal department told us to take all the pictures we wanted and hand them out like candy canes.
Somehow I decide this would be a good day to experiment with ratcheting down the HAZCAMs to half-a-bit-per-pixel compression. I've been on this kick, off and on, for weeks; it would save us a megabit each drive sol, which we could spend on science or on more important engineering data. The image just has to be good enough to tell us two things: (a) did we move as expected during the final step of the drive (usually the last meter or half-meter), and (b) is there anything blocking deployment of the IDD? A half-bit-per-pixel image should do this just fine, and anything that gets us more science return is worth considering.
Or so you'd think. But I can't get either Frank or Justin -- and Justin's a scientist, damn it -- to get very excited about this. I do manage to talk them into trying, despite themselves, so thisol's our first experiment with it. At the very end of the drive, we'll take a one-bit-per-pixel image -- as already planned -- and then take the reduced-quality image, so that we can compare them on the ground. If the reduced-quality image is good enough, I can start working this into the daily plans, which should get the ball rolling.
Since both the IDD and drive are already mostly sequenced, it's a relatively light sol. I do however, manage to create some work for myself: I redo the drive to spin the rover around and drive backward, following recent guidelines capturing lessons we learned from Spirit's long-standing (and recently solved!) wheel problem.
I am actually somewhat worried about this drive. I think I'm exhausted from the week. There's a lot I should have thought of. I should have made the drive more robust against failure. I should have quadruple-checked my math -- how long has it been since I did the trig I needed for this sequence?
Oh, yeah, and the rumor was false: we're not Time's Person of the Year, after all. Funny, if I hadn't heard the rumor, I never would have cared either way. But now, all I can think is: those bastards!
[Next post: sol 344 (Opportunity sol 324), December 21.]
Courtesy NASA/JPL-Caltech. In the distance, Opportunity's heat shield glints in the sun.
Quite a relief, I must admit, to be away from the depotatoization stress over in the Spirit World. When I see Jeng and he asks if I'm on Spirit thisol, I tell him I'm on Opportunity and what we're up to. "Man," he says, shaking his head, "you're always in the middle of the action." Yeah, but this potato thing is something I could stand not to be in the middle of.
On the upside, there's a rumor that we've been named Time's Person of the Year. Other happy news: the ITAR issue is dead, Jim Erickson announces.
The ITAR issue was a concern about the upcoming imaging of the heatshield. ITAR stands for "International Trafficking in Arms Regulations," the set of USA laws covering -- well, what it says. See, since the heatshield is part of a re-entry vehicle, there was some worry that releasing images of it would be tantamount to releasing information about weapons systems. Which is not as crazy as you might think: for good reasons, all spacecraft are legally classified as weapons systems, so we have to be careful about all information we release, same as if they were ICBMs. (Because they easily could be: replace Opportunity with a nuke and change the launch trajectory a hair, and it's goodbye, Luxembourg. Or something like that.) There was also some concern that we'd be unlawfully releasing proprietary information held by Lockheed Martin, who built the reentry system, but that was secondary to the ITAR worries.
Naturally, Jim didn't want his entire uplink team jailed for putting pictures on the Web, so he had the legal department look into it. And in a gratifyingly sensible decision, the legal department told us to take all the pictures we wanted and hand them out like candy canes.
Somehow I decide this would be a good day to experiment with ratcheting down the HAZCAMs to half-a-bit-per-pixel compression. I've been on this kick, off and on, for weeks; it would save us a megabit each drive sol, which we could spend on science or on more important engineering data. The image just has to be good enough to tell us two things: (a) did we move as expected during the final step of the drive (usually the last meter or half-meter), and (b) is there anything blocking deployment of the IDD? A half-bit-per-pixel image should do this just fine, and anything that gets us more science return is worth considering.
Or so you'd think. But I can't get either Frank or Justin -- and Justin's a scientist, damn it -- to get very excited about this. I do manage to talk them into trying, despite themselves, so thisol's our first experiment with it. At the very end of the drive, we'll take a one-bit-per-pixel image -- as already planned -- and then take the reduced-quality image, so that we can compare them on the ground. If the reduced-quality image is good enough, I can start working this into the daily plans, which should get the ball rolling.
Since both the IDD and drive are already mostly sequenced, it's a relatively light sol. I do however, manage to create some work for myself: I redo the drive to spin the rover around and drive backward, following recent guidelines capturing lessons we learned from Spirit's long-standing (and recently solved!) wheel problem.
I am actually somewhat worried about this drive. I think I'm exhausted from the week. There's a lot I should have thought of. I should have made the drive more robust against failure. I should have quadruple-checked my math -- how long has it been since I did the trig I needed for this sequence?
Oh, yeah, and the rumor was false: we're not Time's Person of the Year, after all. Funny, if I hadn't heard the rumor, I never would have cared either way. But now, all I can think is: those bastards!
[Next post: sol 344 (Opportunity sol 324), December 21.]
Courtesy NASA/JPL-Caltech. In the distance, Opportunity's heat shield glints in the sun.
2009-12-17
Spirit Sol 340
"Do you know what happened?" Matt Golombek asks me. I sigh inwardly. As I figured out a long time ago, that's never good news.
Indeed it isn't. We appear not to have moved -- we turned, but we didn't go anywhere. The data's only just starting to come down, and I can't tell them anything useful without more analysis, so I slip out of the SOWG meeting and head downstairs to the SMSA. Jeng, Khaled and I quickly work out that the right rear wheel is dug in -- looks like another Wopmay situation. Well, I figure we can jog downhill a bit and then head back up, much like what we did on the far side of the planet. So I rush back upstairs and tell them we'll have to work out the details but it looks like we can do a semi-normal drive.
I start looking more closely at the images, and just as the meeting ends I notice something weird, something that looks like a clod of dirt in the right rear wheel. Or -- uh-oh. A rock.
I remember Jeff Biesiadecki telling me early in the mission how we could get a rock of the right size stuck in the wheel, jamming the wheel permanently. This rock's the right size, and it's in the right spot. And it's jammed against the steering actuator.
I go back down to the SMSA to tell them the bad news, but it looks like they've already discovered it: there's a whole crowd of mechanical and mobility experts, plus a mission manager or two, peering at the images and arguing with each other. There's already consensus on what happened: it looks like we picked up the rock while turning at the very beginning of the drive, and as the right rear wheel spun, it shoved our little passenger up against the steering actuator. That got it stuck, and as the wheel continued to try to turn, it drew additional power. Eventually, the current draw exceeded the defined limit, and the rover shut itself down for safety's sake.
But while there's no controversy about what happened, how to fix it is a matter of contention. I do what I usually try to do in these situations: facilitate. For the most part, this means "shut up." But I have a talent for noticing when two people aren't communicating well and getting them back onto the same page, so I pipe up from time to time to do that. When the discussion starts to go in circles, I summarize and ensure everyone agrees on the plan. Step one is to spin the stuck wheel one radian[1] in the opposite direction, in the hope of unjamming the potato (as Jeff Biesiadecki dubs the oblong rock). Step two, steer the wheel straight (it's currently cocked in, in its turn-in-place position). Finally, we'll drive away. Each step will take a sol, and we'll take high-quality images in between to assess our progress. If things aren't going well, we'll stop.
Well, there's no hope of hitting that 4km mark now.
Now that we've agreed on what to do, building the sequence is easy. The guts of it is just one command, turning the now-stuck wheel. The rest is just decoration, sprinkling in a bit of visual odometry just in case the rover actually moves during this operation (highly unlikely). And, as in comedy, timing is crucial: we schedule the sequence to execute late in the sol, when we'll have the best lighting. I spend about half an hour picking the perfect time.
At least I can have any time I want. The rover doesn't have much else to do thisol.
I'm RP-1 thisol and John is RP-2 -- that's been a while. We also have Khaled Ali, our RP-in-training, shadowing us thisol. And Jeff Biesiadecki takes time to carefully review and inspect the sequence. That's a lot of RPs for one command. I hope, between the four of us, we can manage to turn this stupid wheel and dislodge the potato.
I really do.
Footnotes:
[1] For those of you who aren't so much into math, a radian is a unit of angular measuremen. It's a little less than 60 degrees. Because it's computationally convenient, Spirit and Opportunity measure all angles in radians, so we humans on the project do the same here and there.
Indeed it isn't. We appear not to have moved -- we turned, but we didn't go anywhere. The data's only just starting to come down, and I can't tell them anything useful without more analysis, so I slip out of the SOWG meeting and head downstairs to the SMSA. Jeng, Khaled and I quickly work out that the right rear wheel is dug in -- looks like another Wopmay situation. Well, I figure we can jog downhill a bit and then head back up, much like what we did on the far side of the planet. So I rush back upstairs and tell them we'll have to work out the details but it looks like we can do a semi-normal drive.
I start looking more closely at the images, and just as the meeting ends I notice something weird, something that looks like a clod of dirt in the right rear wheel. Or -- uh-oh. A rock.
I remember Jeff Biesiadecki telling me early in the mission how we could get a rock of the right size stuck in the wheel, jamming the wheel permanently. This rock's the right size, and it's in the right spot. And it's jammed against the steering actuator.
I go back down to the SMSA to tell them the bad news, but it looks like they've already discovered it: there's a whole crowd of mechanical and mobility experts, plus a mission manager or two, peering at the images and arguing with each other. There's already consensus on what happened: it looks like we picked up the rock while turning at the very beginning of the drive, and as the right rear wheel spun, it shoved our little passenger up against the steering actuator. That got it stuck, and as the wheel continued to try to turn, it drew additional power. Eventually, the current draw exceeded the defined limit, and the rover shut itself down for safety's sake.
But while there's no controversy about what happened, how to fix it is a matter of contention. I do what I usually try to do in these situations: facilitate. For the most part, this means "shut up." But I have a talent for noticing when two people aren't communicating well and getting them back onto the same page, so I pipe up from time to time to do that. When the discussion starts to go in circles, I summarize and ensure everyone agrees on the plan. Step one is to spin the stuck wheel one radian[1] in the opposite direction, in the hope of unjamming the potato (as Jeff Biesiadecki dubs the oblong rock). Step two, steer the wheel straight (it's currently cocked in, in its turn-in-place position). Finally, we'll drive away. Each step will take a sol, and we'll take high-quality images in between to assess our progress. If things aren't going well, we'll stop.
Well, there's no hope of hitting that 4km mark now.
Now that we've agreed on what to do, building the sequence is easy. The guts of it is just one command, turning the now-stuck wheel. The rest is just decoration, sprinkling in a bit of visual odometry just in case the rover actually moves during this operation (highly unlikely). And, as in comedy, timing is crucial: we schedule the sequence to execute late in the sol, when we'll have the best lighting. I spend about half an hour picking the perfect time.
At least I can have any time I want. The rover doesn't have much else to do thisol.
I'm RP-1 thisol and John is RP-2 -- that's been a while. We also have Khaled Ali, our RP-in-training, shadowing us thisol. And Jeff Biesiadecki takes time to carefully review and inspect the sequence. That's a lot of RPs for one command. I hope, between the four of us, we can manage to turn this stupid wheel and dislodge the potato.
I really do.
Footnotes:
[1] For those of you who aren't so much into math, a radian is a unit of angular measuremen. It's a little less than 60 degrees. Because it's computationally convenient, Spirit and Opportunity measure all angles in radians, so we humans on the project do the same here and there.
2009-12-16
Spirit Sol 339
Lots of slip, little progress. What else is new? At this rate, I don't stand a chance of hitting that 4km mark.
So it's another day of struggling toward Larry's Lookout. Thisol we're driving a little more across the slope, a little less upslope, so maybe we'll slip less.
One thing's for sure: I need to come in earlier when working on Spirit, at least when the sol is going to be complicated (like today). I don't really have enough time to get up to speed, or to influence the RP-1's decisions when I think they're wrong, when I come in later. So I'm going to try to switch to the Opportunity model next time I'm RP-2 on Spirit.
One other thing: I might be making a mistake in keeping a running to-do list and conscientiously trying to take care of the items on it. At least, that's Chris's opinion: in an echo of Andy's advice to me earlier in the mission, Chris says, "I used to do that. But now ..." he shrugs. "I figure anything that really needs doing will come to my attention repeatedly."
Courtesy NASA/JPL-Caltech. Making a mess as we climb Husband Hill.
So it's another day of struggling toward Larry's Lookout. Thisol we're driving a little more across the slope, a little less upslope, so maybe we'll slip less.
One thing's for sure: I need to come in earlier when working on Spirit, at least when the sol is going to be complicated (like today). I don't really have enough time to get up to speed, or to influence the RP-1's decisions when I think they're wrong, when I come in later. So I'm going to try to switch to the Opportunity model next time I'm RP-2 on Spirit.
One other thing: I might be making a mistake in keeping a running to-do list and conscientiously trying to take care of the items on it. At least, that's Chris's opinion: in an echo of Andy's advice to me earlier in the mission, Chris says, "I used to do that. But now ..." he shrugs. "I figure anything that really needs doing will come to my attention repeatedly."
Courtesy NASA/JPL-Caltech. Making a mess as we climb Husband Hill.
2009-12-15
Spirit Sol 338
Yestersol's drive ended about 1m short of the third waypoint, so we didn't execute the stutter-step at the end. But we did turn for comm, so we got lots of downlink and are ready to drive again.
Spirit's total odometry now stands at 3950m. I probably still won't get to take her over the 4km mark, but I'm going to get one more chance at it than I thought: Ashitey can't work tomorrow, so he asks me to work for him. I have a standing policy about this: whenever you get a chance to drive the rover, take it. Much as it pains me to think of it, theoretically they won't be around until the end of time, and I don't want to be kicking myself later for any missed opportunities (or spirits, har har). So of course I say yes.
This introduces a small problem: it means that on the day after that (Thursday), both RPs will be coming in cold. (Normally, we like to have at least one of the RPs be one who was on shift the previous day.) So we do some complicated schedule juggling; I switch with Ashitey on Thursday, so I'll be on Spirit yet another day. What's more, I'll be Spirit's RP-1 (with John as RP-2 -- that brings back memories). Maybe I'll take her over the 4km mark after all.
Spirit and Opportunity have some cultural differences, one of them being the schedule of the rover planners. On Opportunity, both RPs arrive around the same time, but Spirit retained some of the nominal-mission behavior, where the RP-2 comes in later than the RP-1. Normally, when I'm RP-2 on Spirit, I arrive about one hour ahead of the activity plan approval meeting. This usually gives me enough time to come up to speed before the meeting, then the RP-1 hands over to me.
But today I'm startled when, just a couple of minutes after I arrive, Saina asks Chris (RP-1) if he has an animation to show. I look up and realize that the APAM is already happening. I should have known: when Saina's the TUL, she tends to push the process along as fast as it will go. (But, to her credit, she's never given me a moment of hassle when I've needed extra time. When I ask for something, she gives it.) Justin says she's got a hot date (every time she works?), but today it's something different: she's going to Europe for a couple of weeks, starting tomorrow, and she's eager to go home and pack.
We spot a problem at the CAM: in one of the paths through the sequence, the rover won't turn for comm. So, even though it's a pain to do this so late in the process, we fix the problem and redeliver. Then almost everyone goes home, and I stick around to do other work ... when I think of another problem.
As our drives have gotten more complex, we've been working out ways to simplify them. One method is to use "helper sequences" -- take groups of repeating commands, make a subsequence for those, and then insert multiple calls to the helper sequences. This makes the sequences shorter, as well as being easier to read, write, and reason about. We've been getting more and more aggressive about this; we now even have helpers for the helpers.
But we do have to keep something in mind. MER lets you call sequences in one of two modes -- "abort" mode or "no-abort" mode -- depending on what you want the caller to do when the helper sequence isn't there. (Since we're sending the commands across hundreds of millions of kilometers of space, sometimes interference prevents part of the command load from reaching the spacecraft.) In "abort" mode, the calling sequence immediately stops its own execution. In "no-abort" mode, the most common mode, the calling sequence continues. Either of these can be what you want (which is why they're both there). The helper sequence might be doing something relatively innocuous, like taking a picture you'd be willing not to get, in which case you use the no-abort mode. Or the sequence might set up conditions that the caller is relying on, which is often the case in IDD sequences and drive sequences.
And the thing I just realized is that we're calling some of the drive helper sequences in no-abort mode.
OK, no need to panic. Maybe. I take a few minutes to think through the various cases, all combinations: if this sequence doesn't make it but that one does, are we still OK? Most combinations wouldn't hurt us -- but one of them would.
Saina'a still here. I explain the situation to her and, as eager as she is to go home, she says, "Well, it's the right thing to do. Let's fix the problem." It'll be even more painful to fix the problem now than it was to fix the other one at the CAM, but by pure luck we happen to still have the necessary people to do it. Saina prepares to call Leo to advise him of the situation, and I flip over to RSVP to start implementing the changes.
At which point I see that the sequences in question are called in abort mode, after all. There isn't a problem; never was.
I am so embarrassed. Saina just laughs -- and, wisely, leaves.[1]
Footnotes:
[1] Saina recently started a charity called Geeti, which gives people around the world printed photographs of themselves and their loved ones. People in many parts of the world have never seen a photograph of themselves, have no way to show their children a picture of their parents' wedding day -- no way to visually preserve their memories. Surrounded as I am by pictures of my family, of myself, of friends -- in some cases, pictures so important to me that I'd risk my life to save them -- I was touched by the idea of a way to help provide something so priceless to so many. Check out her site, and if you like the idea, as I did, consider donating there.
"Geeti," by the way, is Saina's mom's name -- it means "world" in Persian.
Spirit's total odometry now stands at 3950m. I probably still won't get to take her over the 4km mark, but I'm going to get one more chance at it than I thought: Ashitey can't work tomorrow, so he asks me to work for him. I have a standing policy about this: whenever you get a chance to drive the rover, take it. Much as it pains me to think of it, theoretically they won't be around until the end of time, and I don't want to be kicking myself later for any missed opportunities (or spirits, har har). So of course I say yes.
This introduces a small problem: it means that on the day after that (Thursday), both RPs will be coming in cold. (Normally, we like to have at least one of the RPs be one who was on shift the previous day.) So we do some complicated schedule juggling; I switch with Ashitey on Thursday, so I'll be on Spirit yet another day. What's more, I'll be Spirit's RP-1 (with John as RP-2 -- that brings back memories). Maybe I'll take her over the 4km mark after all.
Spirit and Opportunity have some cultural differences, one of them being the schedule of the rover planners. On Opportunity, both RPs arrive around the same time, but Spirit retained some of the nominal-mission behavior, where the RP-2 comes in later than the RP-1. Normally, when I'm RP-2 on Spirit, I arrive about one hour ahead of the activity plan approval meeting. This usually gives me enough time to come up to speed before the meeting, then the RP-1 hands over to me.
But today I'm startled when, just a couple of minutes after I arrive, Saina asks Chris (RP-1) if he has an animation to show. I look up and realize that the APAM is already happening. I should have known: when Saina's the TUL, she tends to push the process along as fast as it will go. (But, to her credit, she's never given me a moment of hassle when I've needed extra time. When I ask for something, she gives it.) Justin says she's got a hot date (every time she works?), but today it's something different: she's going to Europe for a couple of weeks, starting tomorrow, and she's eager to go home and pack.
We spot a problem at the CAM: in one of the paths through the sequence, the rover won't turn for comm. So, even though it's a pain to do this so late in the process, we fix the problem and redeliver. Then almost everyone goes home, and I stick around to do other work ... when I think of another problem.
As our drives have gotten more complex, we've been working out ways to simplify them. One method is to use "helper sequences" -- take groups of repeating commands, make a subsequence for those, and then insert multiple calls to the helper sequences. This makes the sequences shorter, as well as being easier to read, write, and reason about. We've been getting more and more aggressive about this; we now even have helpers for the helpers.
But we do have to keep something in mind. MER lets you call sequences in one of two modes -- "abort" mode or "no-abort" mode -- depending on what you want the caller to do when the helper sequence isn't there. (Since we're sending the commands across hundreds of millions of kilometers of space, sometimes interference prevents part of the command load from reaching the spacecraft.) In "abort" mode, the calling sequence immediately stops its own execution. In "no-abort" mode, the most common mode, the calling sequence continues. Either of these can be what you want (which is why they're both there). The helper sequence might be doing something relatively innocuous, like taking a picture you'd be willing not to get, in which case you use the no-abort mode. Or the sequence might set up conditions that the caller is relying on, which is often the case in IDD sequences and drive sequences.
And the thing I just realized is that we're calling some of the drive helper sequences in no-abort mode.
OK, no need to panic. Maybe. I take a few minutes to think through the various cases, all combinations: if this sequence doesn't make it but that one does, are we still OK? Most combinations wouldn't hurt us -- but one of them would.
Saina'a still here. I explain the situation to her and, as eager as she is to go home, she says, "Well, it's the right thing to do. Let's fix the problem." It'll be even more painful to fix the problem now than it was to fix the other one at the CAM, but by pure luck we happen to still have the necessary people to do it. Saina prepares to call Leo to advise him of the situation, and I flip over to RSVP to start implementing the changes.
At which point I see that the sequences in question are called in abort mode, after all. There isn't a problem; never was.
I am so embarrassed. Saina just laughs -- and, wisely, leaves.[1]
Footnotes:
[1] Saina recently started a charity called Geeti, which gives people around the world printed photographs of themselves and their loved ones. People in many parts of the world have never seen a photograph of themselves, have no way to show their children a picture of their parents' wedding day -- no way to visually preserve their memories. Surrounded as I am by pictures of my family, of myself, of friends -- in some cases, pictures so important to me that I'd risk my life to save them -- I was touched by the idea of a way to help provide something so priceless to so many. Check out her site, and if you like the idea, as I did, consider donating there.
"Geeti," by the way, is Saina's mom's name -- it means "world" in Persian.
2009-12-14
Spirit Sol 337
"Welcome back to the A-Team," Chris says as I walk in to begin my
first Spirit shift since ... uh ....
It's a stow-and-go sol. The stow takes about five minutes to sequence. As for the go, well, that's another matter. Spirit's in some very complex terrain right now, rocks and slippery sand everywhere. I'd almost forgotten what it's like over here, and it's more like that than it usually is, if you know what I mean. Happily, the check-and-proceed technique I developed for Opportunity proves useful here; we use that trick to help circumnavigate some scary rocks.
The most noteworthy part of the drive is that we increment the site index (a number we have the rover assign to each place it goes) to 100. 100! I'm the only one who seems excited about this.
Well, I can be excited enough for everyone. 100! 100! 100!
Courtesy NASA/JPL-Caltech. As usual, Spirit's got a tough road ahead.
first Spirit shift since ... uh ....
It's a stow-and-go sol. The stow takes about five minutes to sequence. As for the go, well, that's another matter. Spirit's in some very complex terrain right now, rocks and slippery sand everywhere. I'd almost forgotten what it's like over here, and it's more like that than it usually is, if you know what I mean. Happily, the check-and-proceed technique I developed for Opportunity proves useful here; we use that trick to help circumnavigate some scary rocks.
The most noteworthy part of the drive is that we increment the site index (a number we have the rover assign to each place it goes) to 100. 100! I'm the only one who seems excited about this.
Well, I can be excited enough for everyone. 100! 100! 100!
Courtesy NASA/JPL-Caltech. As usual, Spirit's got a tough road ahead.
2009-12-08
Opportunity Sol 311 (Spirit Sol 332)
Spirit is approaching the 4km mark; they might make it as early as today. It's a shame I won't get to do that drive, but I've contributed my share toward this achievement. It's OK if somebody else has a little glory.
A little.
It turns out that there's a logic behind the fact that some Opportunists want out of Endurance Crater and some don't care. Field geologists, Brenda Franklin explains to me, want out. They want to see new rocks. The spectroscopy folks don't care -- they're hunched over their instrument data, and any rock is as exciting as any other because what they really care about is not the rocks but the instruments themselves.
I say that I think I'm the only Opportunist who's happy either way. "So you don't care whether you get to do a three-hundred-meter drive?" one of the scientists asks teasingly. (This is obviously possible only on the plains outside the crater, not inside the crater itself.) Well, maybe I do care, at that.
Thisol we're redoing the RAT on Wharenhui, making it another easy sol. The exciting stuff -- the egress drive -- will happen later in the week. We should know by 15:15 Sunday whether we've egressed, and I volunteer to come in and help assess the drive result. (Like they could keep me away.) It's a bit of a shame that I won't get to do the drive, but as I've said, I'd rather that honor went to someone who badly wants out. There's no shortage of those folks anyway.
Since most of the cost of the mission was up-front costs -- development, hardware, launch, and so on -- the sols are getting cheaper. We originally figured about $4 million per sol -- that's assuming about 200 sols of combined operations and an overall mission cost of about $800 million. But now we've had about 600 sols of operations, with only a modest cost increase. As Art points out, that means sols now cost about $1.3 million each. (And since that's amortized over the life of the mission, they're actually cheaper than that.) "Pretty soon they'll be so cheap we can all afford 'em," he says. "Maybe we should start auctioning them on eBay." [1]
[Next post: sol 337, December 14.]
Footnotes:
[1] Since I know you're wondering ... I don't have exact numbers, but they're now well under a quarter-million apiece. And still falling.
A little.
It turns out that there's a logic behind the fact that some Opportunists want out of Endurance Crater and some don't care. Field geologists, Brenda Franklin explains to me, want out. They want to see new rocks. The spectroscopy folks don't care -- they're hunched over their instrument data, and any rock is as exciting as any other because what they really care about is not the rocks but the instruments themselves.
I say that I think I'm the only Opportunist who's happy either way. "So you don't care whether you get to do a three-hundred-meter drive?" one of the scientists asks teasingly. (This is obviously possible only on the plains outside the crater, not inside the crater itself.) Well, maybe I do care, at that.
Thisol we're redoing the RAT on Wharenhui, making it another easy sol. The exciting stuff -- the egress drive -- will happen later in the week. We should know by 15:15 Sunday whether we've egressed, and I volunteer to come in and help assess the drive result. (Like they could keep me away.) It's a bit of a shame that I won't get to do the drive, but as I've said, I'd rather that honor went to someone who badly wants out. There's no shortage of those folks anyway.
Since most of the cost of the mission was up-front costs -- development, hardware, launch, and so on -- the sols are getting cheaper. We originally figured about $4 million per sol -- that's assuming about 200 sols of combined operations and an overall mission cost of about $800 million. But now we've had about 600 sols of operations, with only a modest cost increase. As Art points out, that means sols now cost about $1.3 million each. (And since that's amortized over the life of the mission, they're actually cheaper than that.) "Pretty soon they'll be so cheap we can all afford 'em," he says. "Maybe we should start auctioning them on eBay." [1]
[Next post: sol 337, December 14.]
Footnotes:
[1] Since I know you're wondering ... I don't have exact numbers, but they're now well under a quarter-million apiece. And still falling.
2009-12-07
Opportunity Sol 310 (Spirit Sol 331)
We're not driving thisol, which is as expected. But we might not drive nextersol either, since the RAT hole we dug over the weekend didn't go very deep. (Ray deems it a "wussy grind.") On a steep slope like this one, we can't push the arm very hard against the surface, so when the RAT encounters resistance, it's very easy for the IDD to skip off. That causes the RAT to sense it's lost contact and stop drilling, and it seems to be what happened this weekend.
If the RAT folks can verify that's what happened, we'll basically redo that work tomorrow. If they can't, then we'll bag it and drive tomorrow instead. Jeff has his fingers crossed; he's already got the egress drive planned out. But as the SOWG meeting ends, the RAT team reports that they can confirm the cause of the wussy grind, and according to them, it's safe to re-grind. "Sorry, Jeff," I mutter, and he shrugs.
Jeff's been here since just after he dropped his kids off at school this morning, and it's now already past 13:00. So he's been a busy beaver; not only has he planned the egress drive, he's also done most of thisol's IDD sequences. Thanks to that, it's another easy day.
Since I can always stand some easy days to balance out the hard ones, that's good news. But there's even better news: Spirit's right front wheel seems to have healed itself.[1] Its current draw has been nominal for some time. So there's no longer a need to drag it, as far as we know, and driving Spirit can become simple again. There are a few, easy-to-honor restrictions, which basically amount to "don't drive it so hard."
"Earth, Earth," says the Mars rover, "it hurts when I go like this." And then the Earth says, "So don't go like that."
Footnotes:
[1] Long before Spirit's right front wheel failed entirely, it was exhibiting a worrisomely high current draw. This manifested itself during her sprint across the plains toward Husband Hill. We'd dealt with it, in part, by employing a 90/10 duty cycle -- drag the wheel for 90cm, then run it for 10cm, which lifted it over the pile of soil you'd build up during the dragging part. This was a real pain to deal with, and it was nice to be free from it for a while.
If the RAT folks can verify that's what happened, we'll basically redo that work tomorrow. If they can't, then we'll bag it and drive tomorrow instead. Jeff has his fingers crossed; he's already got the egress drive planned out. But as the SOWG meeting ends, the RAT team reports that they can confirm the cause of the wussy grind, and according to them, it's safe to re-grind. "Sorry, Jeff," I mutter, and he shrugs.
Jeff's been here since just after he dropped his kids off at school this morning, and it's now already past 13:00. So he's been a busy beaver; not only has he planned the egress drive, he's also done most of thisol's IDD sequences. Thanks to that, it's another easy day.
Since I can always stand some easy days to balance out the hard ones, that's good news. But there's even better news: Spirit's right front wheel seems to have healed itself.[1] Its current draw has been nominal for some time. So there's no longer a need to drag it, as far as we know, and driving Spirit can become simple again. There are a few, easy-to-honor restrictions, which basically amount to "don't drive it so hard."
"Earth, Earth," says the Mars rover, "it hurts when I go like this." And then the Earth says, "So don't go like that."
Footnotes:
[1] Long before Spirit's right front wheel failed entirely, it was exhibiting a worrisomely high current draw. This manifested itself during her sprint across the plains toward Husband Hill. We'd dealt with it, in part, by employing a 90/10 duty cycle -- drag the wheel for 90cm, then run it for 10cm, which lifted it over the pile of soil you'd build up during the dragging part. This was a real pain to deal with, and it was nice to be free from it for a while.
2009-12-04
Opportunity Sol 307 (Spirit Sol 328)
We were supposed to start at 11:00 yesterday, but they moved it up to 10:00. Worse, today we were supposed to start at 13:00, but they moved it up to 08:00. Being a Friday, we're doing a three-sol plan; everybody wants out early, and since we happen not to need the downlink for planning, they choose to start early.
I need to ensure that I'm consulted on this decision in future. Happily, a suggestion to start earlier on Monday is shot down; Ray, who will be the SOWG chair Monday, isn't available to give his approval. So Monday, at least, we'll start at a civilized hour, 13:00. Ahhh ....
While Brian and I are working, Chris Leger comes up from the Spirit World with a tricky mobility question. Brian and I don't know the answer -- it's really a Mark Maimone question. But Mark is getting married today. "Think it would be bad to interrupt that for a mobility question?" Chris asks. He pretends to think about it. "Maybe I'll see if I can reach Jeff instead."
As for us, we won't be driving for several sols. It's all IDD work this weekend; Monday, the IDD work continues. In the best case, on Tuesday we'll back up and image the results of our handiwork, and maybe drive to the base of the egress chute. I'm on shift Monday and Tuesday, which means I won't be doing the egress drive even if all goes perfectly between now and then. Which suits me fine, for two reasons. First, I didn't do the ingress, and there might be special tricks to the driving that I don't know about. And second, I think one of the Opportunity drivers -- one of the guys who's just sick to death of being in a crater -- should be the one to take us out. They'd get so much more satisfaction out of it than I would.
Thisol's sequencing couldn't be much simpler; it's largely similar to yestersol's. Which leaves us time to talk to Kevin Talley about some work he wants us to do for Phoenix. I don't want to think about it, much less talk about it, but Brian wants to make sure (a) that RSVP continues to be used for Mars missions, and (b) that there's funding available in case MER disappears. ("Someday, these rovers will die," he says. "And when it happens, we might not get a lot of warning." Killjoy.)
With Kevin the discussion always eventually (d)evolves toward things that blow up. (He must miss being in the military.) He mentions an idea Larry Soderblom had for MER: drop a few grenade-like explosives just before landing, to expose fresh surface for the rovers to investigate. The technical details were, ah, a little light, and obviously the idea never came to fruition.
In part because large parts of thisol's IDD sequences are built through the magic of copy-&-paste, we cruise to an easy finish. Andy's imposed a ten-hour limit on a single day's sequencing, even when, as today, we're building a three-sol plan. Today we finish in nine hours and fifty-five minutes, five minutes early. And there was much rejoicing.
[Next post: sol 331, December 7.]
I need to ensure that I'm consulted on this decision in future. Happily, a suggestion to start earlier on Monday is shot down; Ray, who will be the SOWG chair Monday, isn't available to give his approval. So Monday, at least, we'll start at a civilized hour, 13:00. Ahhh ....
While Brian and I are working, Chris Leger comes up from the Spirit World with a tricky mobility question. Brian and I don't know the answer -- it's really a Mark Maimone question. But Mark is getting married today. "Think it would be bad to interrupt that for a mobility question?" Chris asks. He pretends to think about it. "Maybe I'll see if I can reach Jeff instead."
As for us, we won't be driving for several sols. It's all IDD work this weekend; Monday, the IDD work continues. In the best case, on Tuesday we'll back up and image the results of our handiwork, and maybe drive to the base of the egress chute. I'm on shift Monday and Tuesday, which means I won't be doing the egress drive even if all goes perfectly between now and then. Which suits me fine, for two reasons. First, I didn't do the ingress, and there might be special tricks to the driving that I don't know about. And second, I think one of the Opportunity drivers -- one of the guys who's just sick to death of being in a crater -- should be the one to take us out. They'd get so much more satisfaction out of it than I would.
Thisol's sequencing couldn't be much simpler; it's largely similar to yestersol's. Which leaves us time to talk to Kevin Talley about some work he wants us to do for Phoenix. I don't want to think about it, much less talk about it, but Brian wants to make sure (a) that RSVP continues to be used for Mars missions, and (b) that there's funding available in case MER disappears. ("Someday, these rovers will die," he says. "And when it happens, we might not get a lot of warning." Killjoy.)
With Kevin the discussion always eventually (d)evolves toward things that blow up. (He must miss being in the military.) He mentions an idea Larry Soderblom had for MER: drop a few grenade-like explosives just before landing, to expose fresh surface for the rovers to investigate. The technical details were, ah, a little light, and obviously the idea never came to fruition.
In part because large parts of thisol's IDD sequences are built through the magic of copy-&-paste, we cruise to an easy finish. Andy's imposed a ten-hour limit on a single day's sequencing, even when, as today, we're building a three-sol plan. Today we finish in nine hours and fifty-five minutes, five minutes early. And there was much rejoicing.
[Next post: sol 331, December 7.]
2009-12-03
Opportunity Sol 306 (Spirit Sol 327)
I've been off shift for the last week and a half, and clearly something changed. When I show up to drive Opportunity, I'm confronted by a closed door with a cipher lock on it. I knock, but there's no answer.
Turns out this is part of our ongoing effort to protect ourselves against those evil, thieving, dirty, despicable foreign nationals. MRO has space on Opportunity's floor now, and they have FNs working for them, so we have to keep the door locked when nobody's home. Federal law. Oy.
Cooper's RP-1 thisol, and he likes to get things to a certain point before anybody else takes a look, so I spin my wheels for a little while. We're going to RAT thisol, and the RAT guys get their grind timings by subtracting the IDD movement overhead from the total time allocated for the observation, so Cooper works out the timings and goes to lunch. A few minutes later I look up at the clock and think, "I'm hungry; I should go to lunch, too," and just then there's a controversy about Cooper's answer. So I have to redo his work, starting from scratch, which I finish just about the same moment he walks in (thus not saving anybody any time). I never do get lunch. D'oh!
But that's nothing new -- I often don't get lunch.1] I don't get much vacation time, either, and I'm not alone in that. Albert Yen's applying for yet another waiver to allow him time to accumulate more vacation than is normally allowed. Cooper's taking three weeks around Christmas, not because he wants to but because he's already way over the maximum and they won't give him another waiver. I haven't bothered with the paperwork to get credit for the JPL holidays I worked this year, and I'm still going to hit the maximum soon.
Then again, I don't care that I don't get vacation. What am I going to do on my vacation that's anywhere near this cool? And besides, it's not like I'm going house to house in Fallujah -- my job is to play with an extraterrestrial Erector set. What the heck could I possibly need a vacation from, even if I wanted one?
Andy says it would be interesting to study the mixed signals JPL as an institution gives us -- "you have to work 80 hours a week to get the mission done, but don't miss any of your vacation."
You don't even get a vacation if you're injured, or so you'd judge by seeing Geoffrey Lake. He broke his clavicle in four places snowboarding, but he's here today playing TAP/SIE all the same. I tell him I did the same after I fractured my clavicle at aikido, and then Rich pipes up to say he broke his, too. Skiing. An odd coincidence, but not as odd as the other one -- it turns out that Geoff is an aikidoka, too. Me, him, and Cooper. Small world, I guess.
Emily sometimes leads off the CAM with "the picture of the sol" -- in this case, it's a picture of Ken Jennings, who just recently ended his 74-game Jeopardy winning streak. (Ken Jennings is the Babe Ruth of geekdom, I think.) In yet another coincidence, it turns out Craig Leff knows him; they used to compete in the College Bowl.
Emily poses the Final Jeopardy answer that lost Ken his crown; since I saw the show, I know the question already ("What is H&R Block?"). I jokingly dismiss Ken as a bonehead, then grudgingly admit he's smarter than I am. "Not smart enough to drive a Mars rover," says Emily.
[1] Since then, MER's gotten really good about this, in the name of making ops sustainable. People who don't ever get a chance to have lunch tend to get grouchy about it and find other things to do.
Turns out this is part of our ongoing effort to protect ourselves against those evil, thieving, dirty, despicable foreign nationals. MRO has space on Opportunity's floor now, and they have FNs working for them, so we have to keep the door locked when nobody's home. Federal law. Oy.
Cooper's RP-1 thisol, and he likes to get things to a certain point before anybody else takes a look, so I spin my wheels for a little while. We're going to RAT thisol, and the RAT guys get their grind timings by subtracting the IDD movement overhead from the total time allocated for the observation, so Cooper works out the timings and goes to lunch. A few minutes later I look up at the clock and think, "I'm hungry; I should go to lunch, too," and just then there's a controversy about Cooper's answer. So I have to redo his work, starting from scratch, which I finish just about the same moment he walks in (thus not saving anybody any time). I never do get lunch. D'oh!
But that's nothing new -- I often don't get lunch.1] I don't get much vacation time, either, and I'm not alone in that. Albert Yen's applying for yet another waiver to allow him time to accumulate more vacation than is normally allowed. Cooper's taking three weeks around Christmas, not because he wants to but because he's already way over the maximum and they won't give him another waiver. I haven't bothered with the paperwork to get credit for the JPL holidays I worked this year, and I'm still going to hit the maximum soon.
Then again, I don't care that I don't get vacation. What am I going to do on my vacation that's anywhere near this cool? And besides, it's not like I'm going house to house in Fallujah -- my job is to play with an extraterrestrial Erector set. What the heck could I possibly need a vacation from, even if I wanted one?
Andy says it would be interesting to study the mixed signals JPL as an institution gives us -- "you have to work 80 hours a week to get the mission done, but don't miss any of your vacation."
You don't even get a vacation if you're injured, or so you'd judge by seeing Geoffrey Lake. He broke his clavicle in four places snowboarding, but he's here today playing TAP/SIE all the same. I tell him I did the same after I fractured my clavicle at aikido, and then Rich pipes up to say he broke his, too. Skiing. An odd coincidence, but not as odd as the other one -- it turns out that Geoff is an aikidoka, too. Me, him, and Cooper. Small world, I guess.
Emily sometimes leads off the CAM with "the picture of the sol" -- in this case, it's a picture of Ken Jennings, who just recently ended his 74-game Jeopardy winning streak. (Ken Jennings is the Babe Ruth of geekdom, I think.) In yet another coincidence, it turns out Craig Leff knows him; they used to compete in the College Bowl.
Emily poses the Final Jeopardy answer that lost Ken his crown; since I saw the show, I know the question already ("What is H&R Block?"). I jokingly dismiss Ken as a bonehead, then grudgingly admit he's smarter than I am. "Not smart enough to drive a Mars rover," says Emily.
[1] Since then, MER's gotten really good about this, in the name of making ops sustainable. People who don't ever get a chance to have lunch tend to get grouchy about it and find other things to do.
2009-11-23
Opportunity Sol 296 (Spirit Sol 317)
We're continuing the trek out of the crater, today with Jeff and me at the helm. Mostly it's a straightforward drive sol, following the now-usual pattern of driving up to a point and automatically continuing if we're where we're supposed to be.
Steve wants to squeeze in a science observation that would cost 10 megabits. The drive is the only place we can scavenge for it, but Steve makes it very clear that the drive itself is the top priority. "If you can't do it, 'no' is a fine answer," he says.
"Well, let's see," I muse. "We could maybe save four on the NAVCAM panorama wings, go down to two bits per pixel there."
Jeff nods. "And one from the penultimate front HAZCAM -- go down to half a bit per pixel, like the last one."
Halfway there already, but we've already picked the low-hanging fruit. "Can you get by with five megabits?" we ask Steve.
"Nah," he says, "thanks anyway, but if we're going to do this, we'd need the whole ten or nothing."
Jeff frowns, thinking about it. "Well ..." he begins.
"No!" says Art.
As we start to get back to it, I think of a way to scrounge another megabit. I turn around and slide over to the phone. "If we --"
"No!" Art snaps again.
Jeff and I shrug and turn back to the drive. A minute later he looks at me. "Well," he says, "if we reduce the motor/IMU data ...."
Art's head snaps up. "No!"
Now I know how a puppy feels.
[Next post: sol 327 (Opportunity sol 306), December 3.]
Steve wants to squeeze in a science observation that would cost 10 megabits. The drive is the only place we can scavenge for it, but Steve makes it very clear that the drive itself is the top priority. "If you can't do it, 'no' is a fine answer," he says.
"Well, let's see," I muse. "We could maybe save four on the NAVCAM panorama wings, go down to two bits per pixel there."
Jeff nods. "And one from the penultimate front HAZCAM -- go down to half a bit per pixel, like the last one."
Halfway there already, but we've already picked the low-hanging fruit. "Can you get by with five megabits?" we ask Steve.
"Nah," he says, "thanks anyway, but if we're going to do this, we'd need the whole ten or nothing."
Jeff frowns, thinking about it. "Well ..." he begins.
"No!" says Art.
As we start to get back to it, I think of a way to scrounge another megabit. I turn around and slide over to the phone. "If we --"
"No!" Art snaps again.
Jeff and I shrug and turn back to the drive. A minute later he looks at me. "Well," he says, "if we reduce the motor/IMU data ...."
Art's head snaps up. "No!"
Now I know how a puppy feels.
[Next post: sol 327 (Opportunity sol 306), December 3.]
2009-11-20
Opportunity Sol 293 (Spirit Sol 314)
"This is an historic turning point in the Opportunity mission," declares Squyres. "Literally a turning point."
What he's referring to is the beginning of our egress from Endurance Crater. We spent weeks getting as close to Burns Cliff as we safely could, days taking pictures -- and now we'll spend weeks getting away again. But we're not just getting away from Burns Cliff, we're leaving the crater itself. Endurance Crater will soon be a thing of Opportunity's past. This is sure to make a lot of Opportunists happy, those who are sick of being in the damn crater already.
We're at the tip of a sort of narrow "safety peninsula," a rough triangle of driving-friendly rock surface pointed roughly at Burns Cliff. Uphill of us the slopes are too steep for driving, and downhill the sand is too soft. When you're between Scylla and Charybdis, you navigate carefully, and so it is with us. This first drive will be very modest: 3.6m backward, mostly in a very gentle uphill curve, then -- if the rover is sure it's right where we wanted it -- we turn a little bit downhill and drive another 3.6m.
Squyres is ready to make tracks. There won't be much more meaningful science until we egress, so the faster we can (safely) do that, the happier he and the rest of the science team will be. So, as he should, he pushes on whether we can safely extend this drive. I tell him I think it's a bad idea: due to the narrowness of this part of the path, we need to take baby steps. We've got maybe 50m between us and the bottom of our egress chute, and the conservative drive we've planned is already covering more of that in one sol than we'd planned. The original plan was to take about 18 sols to egress, averaging maybe 3m per sol; we'll double that on this drive.
He relents, but later he comes up with another idea. "Since you've got the rover testing its position at the midpoint of the drive, did you consider doing another extension in the same way?" Yeah, I tell him, I did consider it, but I didn't think of it until too late -- it would have torqued the plan too much.
"I wouldn't have liked it anyway," Ashitey says. This is in keeping with his reputation as a risk-averse RP.
"If we'd had John Wright here, we'd have done it," I quip.
It's too late to change the drive, and since Ashitey would object to our trying anyway, we leave it alone. 7.2m tomorrow, then -- if we're lucky. It'll be a good start.
Once we're out of the crater and back on the plains, we'll be covering a lot more distance than that. Ashitey points out something interesting: Cooper's eager to set a new single-sol distance record, but he's going on vacation for a month or so, right around the time we're planning to egress. Meaning the odds are that much better that one of the rest of us could set the record instead ....
[Next post: sol 317 (Opportunity sol 296), November 23.]
What he's referring to is the beginning of our egress from Endurance Crater. We spent weeks getting as close to Burns Cliff as we safely could, days taking pictures -- and now we'll spend weeks getting away again. But we're not just getting away from Burns Cliff, we're leaving the crater itself. Endurance Crater will soon be a thing of Opportunity's past. This is sure to make a lot of Opportunists happy, those who are sick of being in the damn crater already.
We're at the tip of a sort of narrow "safety peninsula," a rough triangle of driving-friendly rock surface pointed roughly at Burns Cliff. Uphill of us the slopes are too steep for driving, and downhill the sand is too soft. When you're between Scylla and Charybdis, you navigate carefully, and so it is with us. This first drive will be very modest: 3.6m backward, mostly in a very gentle uphill curve, then -- if the rover is sure it's right where we wanted it -- we turn a little bit downhill and drive another 3.6m.
Squyres is ready to make tracks. There won't be much more meaningful science until we egress, so the faster we can (safely) do that, the happier he and the rest of the science team will be. So, as he should, he pushes on whether we can safely extend this drive. I tell him I think it's a bad idea: due to the narrowness of this part of the path, we need to take baby steps. We've got maybe 50m between us and the bottom of our egress chute, and the conservative drive we've planned is already covering more of that in one sol than we'd planned. The original plan was to take about 18 sols to egress, averaging maybe 3m per sol; we'll double that on this drive.
He relents, but later he comes up with another idea. "Since you've got the rover testing its position at the midpoint of the drive, did you consider doing another extension in the same way?" Yeah, I tell him, I did consider it, but I didn't think of it until too late -- it would have torqued the plan too much.
"I wouldn't have liked it anyway," Ashitey says. This is in keeping with his reputation as a risk-averse RP.
"If we'd had John Wright here, we'd have done it," I quip.
It's too late to change the drive, and since Ashitey would object to our trying anyway, we leave it alone. 7.2m tomorrow, then -- if we're lucky. It'll be a good start.
Once we're out of the crater and back on the plains, we'll be covering a lot more distance than that. Ashitey points out something interesting: Cooper's eager to set a new single-sol distance record, but he's going on vacation for a month or so, right around the time we're planning to egress. Meaning the odds are that much better that one of the rest of us could set the record instead ....
[Next post: sol 317 (Opportunity sol 296), November 23.]
2009-11-19
Opportunity Sol 292 (Spirit Sol 313)
Opportunity's been sitting at the base of Burns Cliff, taking pictures, so they haven't needed a rover driver for a while. We were released yesterday, and we're released today as well. We'll come back tomorrow -- Friday -- to do the three-sol weekend plan, which has a drive at the end of it (on Sunday).[1]
But Ashitey and I stick around long enough to weigh in on the preliminary version of the three-sol plan. The drive plan normally includes a penultimate HAZCAM image -- an image taken just before the rover takes its last half-meter (or so) step. This is used to support IDD operations, since it gives us a picture of what's under the IDD at the final position, telling us whether it's safe to deploy. But since we won't be doing IDD work, I tell them to skip it. This saves a couple of megabits of downlink volume, which we can use to get science data instead, so I try to do this whenever I can.
But Ashitey wants the image. Why? His argument seems to be: because we always take it. This leads us into a huge philosophical discussion in which I try to convince Ashitey that engineering exists for the sake of science, and he tries to convince me that the engineering side has some kind of responsibility to use up everything it's managed to get the science team to relinquish. I think I win. Or at least it's a draw: Ashitey convinces me that in this particular case we do need the penultimate image in some form, to support some engineering analysis he's doing. But I bargain him down to a half-bit-per-pixel image, which gives us about half the savings I was originally shooting for.
Somehow this also turns into a discussion about how different rover drivers differ on risk-taking. This is something we agree on: Ashitey and Jeff Biesiadecki are the most risk-averse; John Wright and Chris Leger are the most risk-tolerant. The rest of us -- Frank, Cooper and I -- are in between. That's a little odd when I think of it. I'm normally a person who takes things to extremes, good or bad. To be in the middle of anything is new to me.
We don't have to produce a drive sequence today, but we know what we'll be asked for tomorrow. So we decide to build the drive sequence today, at least a rough cut. While we're doing this, John Callas tells us he's bringing the current astronaut class around for a tour, and could we arrange something cool to show them?
One of the great things about being a rover driver is that you never lack cool things to show. (This is very different from being a software developer, where the most cool things about your work often don't look like anything special to a lay audience.) We just hang out in the sixth-floor meeting room and put RSVP up on the big projectors, so they'll display our work in progress.
About an hour later, maybe 30 or 40 astronaut wanna-bes pile in and find seats at the big tables and in what I think of as the audience section. The astronauts-in-training are an interesting mix. Maybe a third to half of the total, and nearly all of the men, are the classic Neil Armstrong pattern: ex-military types with buzz cuts and bulky muscles. The women are less uniform. One or two look like ex-military, one or two others look like somebody's mom, most of the rest look like any women you'd see at the grocery store. Two are ridiculously hot, but one of them makes the mistake of talking. I don't think she'll be flying the Space Shuttle. I hope the other one makes it into space. No, to be honest, I actually hope she flunks out of the astronaut corps and decides she was so impressed by RSVP that she wants to work on MER.
I may be a Martian, but I'm only human.
Footnotes:
[1] During the sols that I was off shift, a huge controversy erupted over the wisdom and feasibility of actually driving to the base of Burns Cliff. The upshot was that we might be able to get there, but the intervening terrain created a risk that we wouldn't be able to get Opportunity safely out of Endurance Crater again if we did. The only upside was an improvement to the quality of the imaging we'd get of Burns Cliff, and as scientifically valuable as that was, Opportunity's safety ultimately trumped that consideration. So we pulled up as close as we could safely get and started imaging from there.
I recognize that that was the right decision, as borne out by the tremendous science Opportunity went on to do afterward. We got great imaging of Burns Cliff and then got a whole lot more science after egressing Endurance, continuing to this very day. Despite all that, it still bugs the hell out of me that we never quite made it to that goal.
But Ashitey and I stick around long enough to weigh in on the preliminary version of the three-sol plan. The drive plan normally includes a penultimate HAZCAM image -- an image taken just before the rover takes its last half-meter (or so) step. This is used to support IDD operations, since it gives us a picture of what's under the IDD at the final position, telling us whether it's safe to deploy. But since we won't be doing IDD work, I tell them to skip it. This saves a couple of megabits of downlink volume, which we can use to get science data instead, so I try to do this whenever I can.
But Ashitey wants the image. Why? His argument seems to be: because we always take it. This leads us into a huge philosophical discussion in which I try to convince Ashitey that engineering exists for the sake of science, and he tries to convince me that the engineering side has some kind of responsibility to use up everything it's managed to get the science team to relinquish. I think I win. Or at least it's a draw: Ashitey convinces me that in this particular case we do need the penultimate image in some form, to support some engineering analysis he's doing. But I bargain him down to a half-bit-per-pixel image, which gives us about half the savings I was originally shooting for.
Somehow this also turns into a discussion about how different rover drivers differ on risk-taking. This is something we agree on: Ashitey and Jeff Biesiadecki are the most risk-averse; John Wright and Chris Leger are the most risk-tolerant. The rest of us -- Frank, Cooper and I -- are in between. That's a little odd when I think of it. I'm normally a person who takes things to extremes, good or bad. To be in the middle of anything is new to me.
We don't have to produce a drive sequence today, but we know what we'll be asked for tomorrow. So we decide to build the drive sequence today, at least a rough cut. While we're doing this, John Callas tells us he's bringing the current astronaut class around for a tour, and could we arrange something cool to show them?
One of the great things about being a rover driver is that you never lack cool things to show. (This is very different from being a software developer, where the most cool things about your work often don't look like anything special to a lay audience.) We just hang out in the sixth-floor meeting room and put RSVP up on the big projectors, so they'll display our work in progress.
About an hour later, maybe 30 or 40 astronaut wanna-bes pile in and find seats at the big tables and in what I think of as the audience section. The astronauts-in-training are an interesting mix. Maybe a third to half of the total, and nearly all of the men, are the classic Neil Armstrong pattern: ex-military types with buzz cuts and bulky muscles. The women are less uniform. One or two look like ex-military, one or two others look like somebody's mom, most of the rest look like any women you'd see at the grocery store. Two are ridiculously hot, but one of them makes the mistake of talking. I don't think she'll be flying the Space Shuttle. I hope the other one makes it into space. No, to be honest, I actually hope she flunks out of the astronaut corps and decides she was so impressed by RSVP that she wants to work on MER.
I may be a Martian, but I'm only human.
Footnotes:
[1] During the sols that I was off shift, a huge controversy erupted over the wisdom and feasibility of actually driving to the base of Burns Cliff. The upshot was that we might be able to get there, but the intervening terrain created a risk that we wouldn't be able to get Opportunity safely out of Endurance Crater again if we did. The only upside was an improvement to the quality of the imaging we'd get of Burns Cliff, and as scientifically valuable as that was, Opportunity's safety ultimately trumped that consideration. So we pulled up as close as we could safely get and started imaging from there.
I recognize that that was the right decision, as borne out by the tremendous science Opportunity went on to do afterward. We got great imaging of Burns Cliff and then got a whole lot more science after egressing Endurance, continuing to this very day. Despite all that, it still bugs the hell out of me that we never quite made it to that goal.
2009-11-10
Spirit Sol 304
The trivial IDD sequence went well, so thisol we'll stow and go.
Stowing is easy. The hard part is going. It's hard because Spirit is so power-limited, and the solar insolation maps we have don't show much hope for us on the way to our next target, the imaginatively named Dark Rock. These maps show us a false-color overlay on navigation images, with the different colors reflecting how much solar energy we'll be able to collect at any given location. (This varies with the magnitude and direction of our tilt: tilting toward the sun is good, away is bad.)
In these images, red is bad, blue is good, green is in between. (Where "bad" means the rover doesn't get enough energy and could, in the worst case, die.) General policy is to lay out paths that are blue all along the way, just in case the drive faults out. Failing that, we choose paths that let us "hop" from one blue region (which we call "lily pads" even though they're not green) to another.
But in this case, we're climbing a generally south-facing slope, and there's not much blue to be seen. We're OK for the first few meters of the drive, but after that the landscape is soaked in red.
Well ... when all else fails, cheat. The different colors represent numbers on a scale from 0 to 1, where red is used for anything below 0.85. (The real floor on the insolation number is supposed to be around 0.83, but due to uncertainties in the way the map is computed, we add a small safety margin and use 0.85.) But Jeff and I find that we don't have to lower the low end very far -- only to about 0.78 or so -- before we start to see small patches of blue. It's not much, but it might be enough for us to hopscotch uphill.
If we're allowed to lower the threshold, that is. So I ask Kirk Fleming and Jake Matijevic about this, stressing that I'm OK with telling science we can't get there from here. Kirk and Jake hem and haw a little while, and come up with an absolute bare minimum insolation number of 0.75.
It's just barely enough, but it's enough. So Jeff and I get working on the first part of the overall drive to Dark Rock. Thisol we'll back uphill 4m, then -- using the same drive-extension trick I recently invented for Opportunity -- if we're on track, we'll turn and drive another 2m downhill, setting ourselves up nicely for the next sol's ~5m drive.
Dark Rock is only 20m away, as the Martian crow flies. With all the zig-zagging, and the short drive times imposed by the tiny amount of available solar power, it will take about a week to get there.
Back in my day, we'd laugh at 20 lousy meters.[1]
[Next post: sol 313 (Opportunity sol 292), November 19.]
[1] And now, of course -- Spirit being deeply embedded in Troy, and possibly unable to get out -- we'd be happy with 20 centimeters per sol. Oh, my poor little rover.
Stowing is easy. The hard part is going. It's hard because Spirit is so power-limited, and the solar insolation maps we have don't show much hope for us on the way to our next target, the imaginatively named Dark Rock. These maps show us a false-color overlay on navigation images, with the different colors reflecting how much solar energy we'll be able to collect at any given location. (This varies with the magnitude and direction of our tilt: tilting toward the sun is good, away is bad.)
In these images, red is bad, blue is good, green is in between. (Where "bad" means the rover doesn't get enough energy and could, in the worst case, die.) General policy is to lay out paths that are blue all along the way, just in case the drive faults out. Failing that, we choose paths that let us "hop" from one blue region (which we call "lily pads" even though they're not green) to another.
But in this case, we're climbing a generally south-facing slope, and there's not much blue to be seen. We're OK for the first few meters of the drive, but after that the landscape is soaked in red.
Well ... when all else fails, cheat. The different colors represent numbers on a scale from 0 to 1, where red is used for anything below 0.85. (The real floor on the insolation number is supposed to be around 0.83, but due to uncertainties in the way the map is computed, we add a small safety margin and use 0.85.) But Jeff and I find that we don't have to lower the low end very far -- only to about 0.78 or so -- before we start to see small patches of blue. It's not much, but it might be enough for us to hopscotch uphill.
If we're allowed to lower the threshold, that is. So I ask Kirk Fleming and Jake Matijevic about this, stressing that I'm OK with telling science we can't get there from here. Kirk and Jake hem and haw a little while, and come up with an absolute bare minimum insolation number of 0.75.
It's just barely enough, but it's enough. So Jeff and I get working on the first part of the overall drive to Dark Rock. Thisol we'll back uphill 4m, then -- using the same drive-extension trick I recently invented for Opportunity -- if we're on track, we'll turn and drive another 2m downhill, setting ourselves up nicely for the next sol's ~5m drive.
Dark Rock is only 20m away, as the Martian crow flies. With all the zig-zagging, and the short drive times imposed by the tiny amount of available solar power, it will take about a week to get there.
Back in my day, we'd laugh at 20 lousy meters.[1]
[Next post: sol 313 (Opportunity sol 292), November 19.]
[1] And now, of course -- Spirit being deeply embedded in Troy, and possibly unable to get out -- we'd be happy with 20 centimeters per sol. Oh, my poor little rover.
2009-11-09
Spirit Sol 303
I'm back in the Spirit World, and today's my first chance in a long time to work with Jeff Biesiadecki. Jeff's one of the other rover drivers, a smart and easy-going guy who wrote large chunks of the mobility flight software. He's also the guy who recommended me to Cooper in the first place (Jeff and I had worked together briefly in the Operations Engineering Laboratory, when I started at JPL), thus setting me on the path that eventually led to this, my dream job from childhood.
So I like working with Jeff.
It's an easy sol. Yestersol they did some IDD work and inadvertently closed the APXS doors just before placing it. It's easier to make this mistake than you might think. You close the doors simply by rotating the turret to a certain position. If you're trying to approach a certain target and the only way to get there is to rotate the turret to that point, well, then, the doors close. (Just to make the point: if this were software, they'd call that a bug. When it happens with hardware, everybody just shrugs.) Back when we worked together practically every sol, Bob Bonitz drilled into me the importance of watching out for this, and indeed it's part of my personal checklist.
So thisol we'll just reopen the doors and re-place the IDD. As it happens, the doors closed when the IDD was switching from the MB to the APXS, so it's not an inevitable consequence of measuring this target; we just need skip the usual preceding MB touch. Since the MB touch itself worked fine yestersol, we just need to get the recorded joint angles out of the downlink. After that, writing the sequence itself is something we could do with our eyes closed.
Apropos of nothing, we've known for some time that Opportunity started showing a sudden power boost recently. The reason is something of a mystery, but the likely cause is simply that Opportunity happened to be in the path of a dust devil, which blew some dust off its solar panels. (Badly power-deprived Spirit could use some of that dust devilly goodness, but no such luck.) The MI images Opportunity took of her magnets show that they've been cleaned off as well, which is consistent with the dust-devil explanation.
My favorite explanation involves a homeless Martian with a squeegee and a bottle of Windex. Mark Maimone points us to another possible explanation: a cartoon showing the solar panels with "WASH ME" finger-written in the dust.
So I like working with Jeff.
It's an easy sol. Yestersol they did some IDD work and inadvertently closed the APXS doors just before placing it. It's easier to make this mistake than you might think. You close the doors simply by rotating the turret to a certain position. If you're trying to approach a certain target and the only way to get there is to rotate the turret to that point, well, then, the doors close. (Just to make the point: if this were software, they'd call that a bug. When it happens with hardware, everybody just shrugs.) Back when we worked together practically every sol, Bob Bonitz drilled into me the importance of watching out for this, and indeed it's part of my personal checklist.
So thisol we'll just reopen the doors and re-place the IDD. As it happens, the doors closed when the IDD was switching from the MB to the APXS, so it's not an inevitable consequence of measuring this target; we just need skip the usual preceding MB touch. Since the MB touch itself worked fine yestersol, we just need to get the recorded joint angles out of the downlink. After that, writing the sequence itself is something we could do with our eyes closed.
Apropos of nothing, we've known for some time that Opportunity started showing a sudden power boost recently. The reason is something of a mystery, but the likely cause is simply that Opportunity happened to be in the path of a dust devil, which blew some dust off its solar panels. (Badly power-deprived Spirit could use some of that dust devilly goodness, but no such luck.) The MI images Opportunity took of her magnets show that they've been cleaned off as well, which is consistent with the dust-devil explanation.
My favorite explanation involves a homeless Martian with a squeegee and a bottle of Windex. Mark Maimone points us to another possible explanation: a cartoon showing the solar panels with "WASH ME" finger-written in the dust.
2009-11-04
Opportunity Sol 277 (Spirit Sol 298)
When the rover receives the uplink, it sends us back a simple tone to acknowledge receipt. That's called the nominal beep. There's also an off-nominal beep, in case the rover thinks something went wrong with the uplink.
Today we didn't get either the nominal or the off-nominal beep from Opportunity. This is unusual, but it can, and probably does, mean nothing -- the DSN made a mistake in configuring the antenna, or something like that. But it can also mean the rover's dead, so it's cause for some worry.
As usual, this prompts me to think about ways I might have killed the rover. "Maybe we drove it off a cliff," I suggest to Andy.
"No, we beep before driving, so that can't be the reason."
He pauses. "Of course, maybe you drove over a cliff afterward," he adds helpfully.
Thanks, Andy.
Naturally, Opportunity is just fine. We never find out what happened with the beep, but we get the downlink right on schedule, so nobody cares too much. And what a downlink -- 122 Mbits, a huge amount of data, some as old as sol 207. This downlink even gets us over the "Herkenwall," a huge amount of MI imagery due to Ken Herkenhoff.
More important to me, yestersol's tricky drive seems to have gone well. Very well. Suspiciously well, come to think of it: according to the data, we're less than 20cm from our predicted position. Frank and I hold off on reporting the happy news until we can confirm it against the images, but it turns out we needn't have been so conservative; we really did do that well.
Even better, we got the full 8m out of the drive. After all that worry about the exact size of the post-first-segment target zone, Opportunity was only 10cm from its center -- basically, right on predict.
So it's easy to see we're driving again today. As we work on the sequence, election results are trickling in, and the telepresent scientists report them -- with diminishing enthusiasm -- as the night progresses. They sounded just like this during the American League Championship Series games the Red Sox lost. Looks like a tough year for Yankees fans and Democrats.
[Next post: sol 303, November 9.]
Today we didn't get either the nominal or the off-nominal beep from Opportunity. This is unusual, but it can, and probably does, mean nothing -- the DSN made a mistake in configuring the antenna, or something like that. But it can also mean the rover's dead, so it's cause for some worry.
As usual, this prompts me to think about ways I might have killed the rover. "Maybe we drove it off a cliff," I suggest to Andy.
"No, we beep before driving, so that can't be the reason."
He pauses. "Of course, maybe you drove over a cliff afterward," he adds helpfully.
Thanks, Andy.
Naturally, Opportunity is just fine. We never find out what happened with the beep, but we get the downlink right on schedule, so nobody cares too much. And what a downlink -- 122 Mbits, a huge amount of data, some as old as sol 207. This downlink even gets us over the "Herkenwall," a huge amount of MI imagery due to Ken Herkenhoff.
More important to me, yestersol's tricky drive seems to have gone well. Very well. Suspiciously well, come to think of it: according to the data, we're less than 20cm from our predicted position. Frank and I hold off on reporting the happy news until we can confirm it against the images, but it turns out we needn't have been so conservative; we really did do that well.
Even better, we got the full 8m out of the drive. After all that worry about the exact size of the post-first-segment target zone, Opportunity was only 10cm from its center -- basically, right on predict.
So it's easy to see we're driving again today. As we work on the sequence, election results are trickling in, and the telepresent scientists report them -- with diminishing enthusiasm -- as the night progresses. They sounded just like this during the American League Championship Series games the Red Sox lost. Looks like a tough year for Yankees fans and Democrats.
[Next post: sol 303, November 9.]
2009-11-02
Opportunity Sol 276 (Spirit Sol 297)
Ashitey's drive went just about perfectly over the weekend, leaving us just where we wanted to be: on a broad lane of exposed rocks, like a cobblestone highway leading us straight to Burns Cliff.
But our first stop is the Bermuda Triangle, or Triangle of Doom. This is a broad triangular (duh) expanse of soft sand -- seemingly the same kind of nasty stuff we were caught in for the last a couple of weeks -- whose apex crosses the highway ahead. We're not sure we can cross it, so our first step, over the next couple of days, will be to drive to it so that we can investigate it up close. Whether we can cross it or not, we'll be able to get better pictures of the cliff from there, and maybe decide whether we want to proceed or not.
The triangle is only 15m away, the kind of distance I'm used to thinking of as a single sol's driving -- and an easy one, at that. Of course, there's a catch: at the most recent traversability meeting, Jim Erickson said he didn't want us driving farther than about five meters per sol. The rationale for this is a good one. We don't have a lot of experience driving on this exact terrain, and the price for failure is very high: if we slip off of the rocks into the soft sand downhill, there's no guarantee that we can ever get uphill again, or indeed out of the crater. Even if we can, we'll squander a lot of precious time trying. So I don't disagree with the rule, but if we follow it strictly, it'll take us three sols to do a single sol's worth of driving.
I think I can get it down to two.
I propose my idea to Andy and Frank. "I think we can pick a spot five meters out where we can say, if the rover makes it here, we'd be willing to push a button to say to go another three meters on the same heading. So why not tell the rover that? Pick a point to serve as our prediction for the end of the nominal drive, and tell the rover that if it's within some tight radius of that point, go another three meters."[1]
That automatically cancels out the biggest objection to longer drives, namely, the a priori uncertainty about the rover's behavior in this terrain. If we don't make it to the predicted point, we simply don't do anything additional. But if we do, then we know our predictions are reasonably accurate, and it's reasonable to extend the drive in that case. And in the admittedly unlikely event that we can do this two sols in a row, we can make it to the triangle one sol sooner.
Frank and Andy agree readily. But that leaves us with a project manager to convince. Andy calls Jim and asks him to join us downstairs, and Frank starts pulling up the relevant imagery so we can make some good choices about drive distance and an appropriate radius for the circle. This means we're ready for Jim's questions, which is probably part of the reason we're able to sell him on the idea without a lot of trouble. (Well, "sold" is relative. His official response, delivered with his characteristic suspicious squint, is: "Okay, let's do something dangerous." But in Jim Erickson terms, that's sold.)
So that's our plan: five meters of nominal driving, with an optional three-meter extension at the rover's discretion. The radius of the circle varies through the day. When I we talk to Jim, we're thinking in terms of 25 or 30cm. But the more we look at the terrain, the wider the circle grows, reflecting our increasing confidence -- 50cm, 70cm, 75cm.
Then Squyres gets into the act. He's eager to get to the triangle and loves the idea of pushing the drive as aggressively as possible. He also recognizes that the larger the circle is, the more likely the rover will be within it and decide to drive farther. So he pushes for a bigger circle, and gets us to go back to the drawing board and bias the drive uphill so that we have a larger safe zone. The circle's eventual size: 90cm.
Part of what bounds the circle's size is whether we'll end up on rock. But there's another limiting factor. If we're expecting (say) 20% slip, then we don't want to choose a circle that would allow 50% slip, even if the rover would be safe anywhere within that larger circle. The whole point of the extension is that if the rover ends up anywhere in the chosen zone, it's okay to do another three meters like the first five. But if it slips far more than we expected, then even if it's currently safe, the added slip means we don't understand what the vehicle is doing, and so we don't want it to drive on without our stopping and taking a look at the situation. I think Steve wanted a larger safe zone (indeed, I know he did), but he understands our logic and acquiesces.
After the CAM, he apologizes -- graciously, though unnecessarily. "Hey, I'm sorry for pushing you guys on the drive stuff."
"No problem," I reassure him. "It's always good to have more people we can blame if anything goes wrong."
Footnotes:
[1] Another example of just inventing a technique on the fly and seeing it suddenly become standard practice. Now we do this kind of thing all the time. I'd forgotten that there was a time when it was novel, not one of those background "of course we do that" things.
But our first stop is the Bermuda Triangle, or Triangle of Doom. This is a broad triangular (duh) expanse of soft sand -- seemingly the same kind of nasty stuff we were caught in for the last a couple of weeks -- whose apex crosses the highway ahead. We're not sure we can cross it, so our first step, over the next couple of days, will be to drive to it so that we can investigate it up close. Whether we can cross it or not, we'll be able to get better pictures of the cliff from there, and maybe decide whether we want to proceed or not.
The triangle is only 15m away, the kind of distance I'm used to thinking of as a single sol's driving -- and an easy one, at that. Of course, there's a catch: at the most recent traversability meeting, Jim Erickson said he didn't want us driving farther than about five meters per sol. The rationale for this is a good one. We don't have a lot of experience driving on this exact terrain, and the price for failure is very high: if we slip off of the rocks into the soft sand downhill, there's no guarantee that we can ever get uphill again, or indeed out of the crater. Even if we can, we'll squander a lot of precious time trying. So I don't disagree with the rule, but if we follow it strictly, it'll take us three sols to do a single sol's worth of driving.
I think I can get it down to two.
I propose my idea to Andy and Frank. "I think we can pick a spot five meters out where we can say, if the rover makes it here, we'd be willing to push a button to say to go another three meters on the same heading. So why not tell the rover that? Pick a point to serve as our prediction for the end of the nominal drive, and tell the rover that if it's within some tight radius of that point, go another three meters."[1]
That automatically cancels out the biggest objection to longer drives, namely, the a priori uncertainty about the rover's behavior in this terrain. If we don't make it to the predicted point, we simply don't do anything additional. But if we do, then we know our predictions are reasonably accurate, and it's reasonable to extend the drive in that case. And in the admittedly unlikely event that we can do this two sols in a row, we can make it to the triangle one sol sooner.
Frank and Andy agree readily. But that leaves us with a project manager to convince. Andy calls Jim and asks him to join us downstairs, and Frank starts pulling up the relevant imagery so we can make some good choices about drive distance and an appropriate radius for the circle. This means we're ready for Jim's questions, which is probably part of the reason we're able to sell him on the idea without a lot of trouble. (Well, "sold" is relative. His official response, delivered with his characteristic suspicious squint, is: "Okay, let's do something dangerous." But in Jim Erickson terms, that's sold.)
So that's our plan: five meters of nominal driving, with an optional three-meter extension at the rover's discretion. The radius of the circle varies through the day. When I we talk to Jim, we're thinking in terms of 25 or 30cm. But the more we look at the terrain, the wider the circle grows, reflecting our increasing confidence -- 50cm, 70cm, 75cm.
Then Squyres gets into the act. He's eager to get to the triangle and loves the idea of pushing the drive as aggressively as possible. He also recognizes that the larger the circle is, the more likely the rover will be within it and decide to drive farther. So he pushes for a bigger circle, and gets us to go back to the drawing board and bias the drive uphill so that we have a larger safe zone. The circle's eventual size: 90cm.
Part of what bounds the circle's size is whether we'll end up on rock. But there's another limiting factor. If we're expecting (say) 20% slip, then we don't want to choose a circle that would allow 50% slip, even if the rover would be safe anywhere within that larger circle. The whole point of the extension is that if the rover ends up anywhere in the chosen zone, it's okay to do another three meters like the first five. But if it slips far more than we expected, then even if it's currently safe, the added slip means we don't understand what the vehicle is doing, and so we don't want it to drive on without our stopping and taking a look at the situation. I think Steve wanted a larger safe zone (indeed, I know he did), but he understands our logic and acquiesces.
After the CAM, he apologizes -- graciously, though unnecessarily. "Hey, I'm sorry for pushing you guys on the drive stuff."
"No problem," I reassure him. "It's always good to have more people we can blame if anything goes wrong."
Footnotes:
[1] Another example of just inventing a technique on the fly and seeing it suddenly become standard practice. Now we do this kind of thing all the time. I'd forgotten that there was a time when it was novel, not one of those background "of course we do that" things.
2009-10-30
Opportunity Sol 273 (Spirit Sol 294)
I'm not on shift thisol. But when I'm on shift, I can't get any other work done -- not even reading my email, usually -- and now I've got at least two weeks' worth of stuff demanding my immediate attention. So I come in today just to catch up.
Frankly, I don't really want to know how the drive went. If yestersol (272) didn't work, there's no reason to think thisol has a chance. Maybe we'll get lucky and at least make enough distance on the last leg of the drive to be sure we're safe to deploy the IDD, so they can do the magnet observations they had to cancel last week. Still, I don't want to know. I avoid the sequencing rooms altogether.
But I can't escape Jeff's email:
I stare at this for nearly a minute, not really believing it. Then I jump out of my seat and literally run upstairs to the fifth floor. Cooper and Ashitey are wearing the fancy 3-D goggles, looking at the front HAZCAM images. The front HAZCAM shows rocky stuff, not sand.
"We made it," Cooper says. "Great driving!" ("It was a beautiful drive," echoes Art, "and I hate you for it." Art wants out of the crater, and this success means we'll be staying a while. The traverse to Burns Cliff is back on.)
"Did you see that NAVCAM?" Ashitey asks.
One of the NAVCAM images shows where we started yestersol. It's worse than we thought -- at least three of the wheels were dug in, not just the left front wheel. In 3-D, it's a dramatic picture. It's hard to believe we got out.
It's even harder to believe we got all the way uphill. Much as I want to, I can't really take credit for the success. The imagery confirms our suspicions -- there was just a thick band of impassable soft sand, and we spent a couple of weeks getting repeatedly caught up in it. (Where the sand is deep enough to form an opaque layer -- common in the crater -- you can't tell how deep it is until you drive across it. Maybe the sandy layer is a mere 1cm deep, so the rover won't slip at all. Yestersol we got caught in stuff that was more than 25cm deep. It's like crossing a swamp, where at any minute you might plunge in right up to your neck.)
Sol 270's experimental drive worked, right up to the point where we got caught up in the stuff again, which explains why we made no progress on the following sol. Yestersol we happened to get across a shallower layer of sand and onto friendlier terrain -- apparently early on, too, so that the rest of the drive worked and we're way uphill now.
So what changed was maybe 10% a matter of skill (if that) and 90% a matter of what happened to be under the wheels. I can't take credit, but I can sure be happy about the outcome, and you'd better believe I do that for all I'm worth. It takes sustained conscious effort to wipe the dumb grin off my face.
It's still a long way to Burns Cliff. The promised (threatened?) traversability meeting -- which grows every time we hold it, incidentally -- works out a plan for driving there. Optimistically, it might take another two weeks; more likely, longer. There's still a "Bermuda Triangle" of sand along the way, which we're not sure we can pass. The slopes look worrisome.
But at least we're back in the running.
[Next post: sol 297 (Opportunity sol 276), November 2 (at nearly midnight).]
Courtesy NASA/JPL-Caltech. Holy smoke! It worked! ... Uh, I mean, I never had any doubts.
Courtesy NASA/JPL-Caltech. Well, that explains a lot. Two weeks of struggling, encapsulated in a single NAVCAM image -- with Wopmay lurking in the background, and the crater floor beyond.
Frankly, I don't really want to know how the drive went. If yestersol (272) didn't work, there's no reason to think thisol has a chance. Maybe we'll get lucky and at least make enough distance on the last leg of the drive to be sure we're safe to deploy the IDD, so they can do the magnet observations they had to cancel last week. Still, I don't want to know. I avoid the sequencing rooms altogether.
But I can't escape Jeff's email:
I see we're back on the paving stones - should be smooth
roving from here. Nice driving!
I stare at this for nearly a minute, not really believing it. Then I jump out of my seat and literally run upstairs to the fifth floor. Cooper and Ashitey are wearing the fancy 3-D goggles, looking at the front HAZCAM images. The front HAZCAM shows rocky stuff, not sand.
"We made it," Cooper says. "Great driving!" ("It was a beautiful drive," echoes Art, "and I hate you for it." Art wants out of the crater, and this success means we'll be staying a while. The traverse to Burns Cliff is back on.)
"Did you see that NAVCAM?" Ashitey asks.
One of the NAVCAM images shows where we started yestersol. It's worse than we thought -- at least three of the wheels were dug in, not just the left front wheel. In 3-D, it's a dramatic picture. It's hard to believe we got out.
It's even harder to believe we got all the way uphill. Much as I want to, I can't really take credit for the success. The imagery confirms our suspicions -- there was just a thick band of impassable soft sand, and we spent a couple of weeks getting repeatedly caught up in it. (Where the sand is deep enough to form an opaque layer -- common in the crater -- you can't tell how deep it is until you drive across it. Maybe the sandy layer is a mere 1cm deep, so the rover won't slip at all. Yestersol we got caught in stuff that was more than 25cm deep. It's like crossing a swamp, where at any minute you might plunge in right up to your neck.)
Sol 270's experimental drive worked, right up to the point where we got caught up in the stuff again, which explains why we made no progress on the following sol. Yestersol we happened to get across a shallower layer of sand and onto friendlier terrain -- apparently early on, too, so that the rest of the drive worked and we're way uphill now.
So what changed was maybe 10% a matter of skill (if that) and 90% a matter of what happened to be under the wheels. I can't take credit, but I can sure be happy about the outcome, and you'd better believe I do that for all I'm worth. It takes sustained conscious effort to wipe the dumb grin off my face.
It's still a long way to Burns Cliff. The promised (threatened?) traversability meeting -- which grows every time we hold it, incidentally -- works out a plan for driving there. Optimistically, it might take another two weeks; more likely, longer. There's still a "Bermuda Triangle" of sand along the way, which we're not sure we can pass. The slopes look worrisome.
But at least we're back in the running.
[Next post: sol 297 (Opportunity sol 276), November 2 (at nearly midnight).]
Courtesy NASA/JPL-Caltech. Holy smoke! It worked! ... Uh, I mean, I never had any doubts.
Courtesy NASA/JPL-Caltech. Well, that explains a lot. Two weeks of struggling, encapsulated in a single NAVCAM image -- with Wopmay lurking in the background, and the crater floor beyond.
2009-10-29
Opportunity Sol 272 (Spirit Sol 293)
Because of all the time we've been devoting to driving, several of the instruments have gotten no new data in the last several sols. Flash has been full for a related reason -- in order to gain as much ground as possible, we haven't been turning to optimize downlink, so most of our downlink passes have been unusually poor.
Not that there's much for the instrument teams to do anyway. We've been in essentially the same spot for a couple of weeks now; they've taken pictures of everything there is to take pictures of and already IDDd the hell out of Wopmay, which was the only interesting thing in reach.
Two weeks -- that's how long we've been struggling to get away from Wopmay and on our way to Burns Cliff. Happily, yestersol's successful experiment made us hopeful that we're on our way at last.
We won't know for a while, though; we're in that phase where the downlink is coming in later and later, so our sequencing starts later and later. As we've been doing, we plan the sol in the blind, assuming a successful drive. We're going to do the same thing thisol whether the drive worked or not: we're going to spend a long time driving (or trying to), and fill up the rest of the time with bits and pieces of random science.
For the past few sols, I've been hearing more and more about the possibility of a return to working weekends, at least on Opportunity. Jim Erickson, at least, is getting worried about Opportunity's apparent lack of progress. He seems to be concerned that when they went to Congress and asked for an extended-extended mission, they had this long list of things Opportunity might do, and so far they've barely done any of it. This is likely why Jim's anxious to flee the crater and get on with other things. (Squyres's rejoinder to this is that the list was explicitly a list of things we might do, and not having checked off many items isn't necessarily a problem when we go back for more funding early next year. I don't think Jim's convinced.) Working one weekend day each week would allow for the possibility of an extra drive sol each week.
Both Andy and Jim sound me out about the possibility of working weekends. I'm cool with it -- actually, I even kind of miss it. ("You're all too eager to work weekends," Andy tells me later, his reproachfulness only semi-joking. Apparently, he even gets lots of volunteers to work Thanksgiving. It's fantastic to work with a team that has such high morale.) Jim's blown away when Mark Maimone and I both tell him we miss Mars time.
"I've never slept so much as when we were on Mars time," Mark shrugs as I nod agreement. "And I don't have kids, so ...."
"Don't you have a wife?" Jim asks Mark.
"A girlfriend."
Jim pauses. "And you want to keep her?"
Not long after, we get the downlink, and -- well -- it's not all that we'd hoped for. Not only didn't we make any progress, we actually lost a little bit of ground, ending up maybe 30cm downslope of where we started, and only about 70cm cross-slope. Looks like we don't have a reliable strategy for making uphill progress after all. Or, as Art puts it: "So our euphoria from the previous day's drive has evaporated."
Looks like I was wrong about whether we should continue trying to drive. And that's the general feeling (I mean, not that I was wrong, but that we've about reached the point where we should give up). Since we've already made a plan for tomorrow that includes a drive, we decide we'll give it one last chance. But we're going to meet again tomorrow, after the drive results are in, and decide what to do. And it's hard to see how it's going to go any other way than abandon the cliff and head back out of the crater.
Cooper has to go to the doctor, so we sketch out the first version of a drive together and he heads out. The rover is dug in very deep, worse than I've ever seen, with the left front wheel buried in a hole (of its own making) deeper than it is tall. We can't turn clockwise to escape the hole, since there's a rock on top of the pile of dirt the rover kicked up on that side -- likely, this will be too much for us to get over. We can't turn counterclockwise because we can't see off to that side, though of course it stands to reason that there's a big pile of dirt there as well. And we can't get out by going uphill, because that's how we dug ourselves in.
So the only way out is straight ahead -- downhill. The only sure way to get completely clear of the hole is to drive the rover about 2m forward, losing more altitude than we gained on the experimental drive. This is all very, very discouraging.
After that, we revert to the plan that appeared to work well for us on the experimental drive: a switchback, so that if the rover gets stuck, at least she'll change direction and perhaps make some progress. Three switchback legs of about 4m each, and thanks to the 2m downhill bump at the start, about the best we can hope for is that we'll break even -- that we'll be more or less back where we started before this latest drive. No closer to Burns Cliff -- and, considering what another failure will mean, in a sense we'll be much farther away.
At least we've got one more chance. Just to make it dramatic, Cooper's appointment means I get to do it all by myself. So, great: we're going to fail to get to Burns Cliff, and I'll get to feel like it's all my personal fault.
Well, what can I do to improve the odds? When I look at the drive Cooper and I sketched out, I spot a problem: switchback driving tends to carry you, on average, straight uphill. But the hole we've just dug is straight uphill from where we start the switchback. We might avoid it, but with all the slip we've been encountering, there's no certainty. So I change the drive: instead of three legs of 4m each, I use one leg of 8m and another of 4m. If this gains us enough uphill distance to get us onto the rockier terrain (the so-called "paving stones") we can see there, we won't care that we're a few meters farther from the cliff. (Just a few meters is all it would take, damn it! You could walk it in seconds.) And if it doesn't, then we'll be that much closer to the exit.
Assuming we don't just dig an even deeper hole, that is.
I name the drive "Hail Mary."
As it happens, I don't have to plan it completely solo. As I'm nearing completion, Spirit finishes its planning for the sol, so Jeff Biesiadecki comes upstairs to see how I'm doing, and Art presses him to backstop me on the sequence. Then John Wright joins him, and Art ropes him in as well.
So we decide to thoroughly review the thing among the three of us. This will delay uplink even longer, but considering the science we're trying to preserve, who cares?
Though it's a reasonably short sequence, reviewing the drive takes maybe 45 minutes or an hour. To paraphrase an old joke, if you put three rover drivers in a room, you get four opinions. Jeff and John do excellent work, poking at pretty much every decision I made, and coming up with some points I hadn't thought of. In the end, little changes -- the drive stays mostly intact, except that we convert the switchback turn from a turn-in-place into a K-turn. (The rover backs uphill through the long leg, then arcs halfway into the turn, reverses course, and arcs ahead to swing into the shorter second leg, driving forward.) I don't expect this to make a lot of difference, but it might save us some slippage, and it definitely looks cooler.
The only question is, will it work?
Courtesy NASA/JPL-Caltech. Well, crap. Looks like it was smart to keep that whole "NEENER NEENER NEENER" thing to myself.
Not that there's much for the instrument teams to do anyway. We've been in essentially the same spot for a couple of weeks now; they've taken pictures of everything there is to take pictures of and already IDDd the hell out of Wopmay, which was the only interesting thing in reach.
Two weeks -- that's how long we've been struggling to get away from Wopmay and on our way to Burns Cliff. Happily, yestersol's successful experiment made us hopeful that we're on our way at last.
We won't know for a while, though; we're in that phase where the downlink is coming in later and later, so our sequencing starts later and later. As we've been doing, we plan the sol in the blind, assuming a successful drive. We're going to do the same thing thisol whether the drive worked or not: we're going to spend a long time driving (or trying to), and fill up the rest of the time with bits and pieces of random science.
For the past few sols, I've been hearing more and more about the possibility of a return to working weekends, at least on Opportunity. Jim Erickson, at least, is getting worried about Opportunity's apparent lack of progress. He seems to be concerned that when they went to Congress and asked for an extended-extended mission, they had this long list of things Opportunity might do, and so far they've barely done any of it. This is likely why Jim's anxious to flee the crater and get on with other things. (Squyres's rejoinder to this is that the list was explicitly a list of things we might do, and not having checked off many items isn't necessarily a problem when we go back for more funding early next year. I don't think Jim's convinced.) Working one weekend day each week would allow for the possibility of an extra drive sol each week.
Both Andy and Jim sound me out about the possibility of working weekends. I'm cool with it -- actually, I even kind of miss it. ("You're all too eager to work weekends," Andy tells me later, his reproachfulness only semi-joking. Apparently, he even gets lots of volunteers to work Thanksgiving. It's fantastic to work with a team that has such high morale.) Jim's blown away when Mark Maimone and I both tell him we miss Mars time.
"I've never slept so much as when we were on Mars time," Mark shrugs as I nod agreement. "And I don't have kids, so ...."
"Don't you have a wife?" Jim asks Mark.
"A girlfriend."
Jim pauses. "And you want to keep her?"
Not long after, we get the downlink, and -- well -- it's not all that we'd hoped for. Not only didn't we make any progress, we actually lost a little bit of ground, ending up maybe 30cm downslope of where we started, and only about 70cm cross-slope. Looks like we don't have a reliable strategy for making uphill progress after all. Or, as Art puts it: "So our euphoria from the previous day's drive has evaporated."
Looks like I was wrong about whether we should continue trying to drive. And that's the general feeling (I mean, not that I was wrong, but that we've about reached the point where we should give up). Since we've already made a plan for tomorrow that includes a drive, we decide we'll give it one last chance. But we're going to meet again tomorrow, after the drive results are in, and decide what to do. And it's hard to see how it's going to go any other way than abandon the cliff and head back out of the crater.
Cooper has to go to the doctor, so we sketch out the first version of a drive together and he heads out. The rover is dug in very deep, worse than I've ever seen, with the left front wheel buried in a hole (of its own making) deeper than it is tall. We can't turn clockwise to escape the hole, since there's a rock on top of the pile of dirt the rover kicked up on that side -- likely, this will be too much for us to get over. We can't turn counterclockwise because we can't see off to that side, though of course it stands to reason that there's a big pile of dirt there as well. And we can't get out by going uphill, because that's how we dug ourselves in.
So the only way out is straight ahead -- downhill. The only sure way to get completely clear of the hole is to drive the rover about 2m forward, losing more altitude than we gained on the experimental drive. This is all very, very discouraging.
After that, we revert to the plan that appeared to work well for us on the experimental drive: a switchback, so that if the rover gets stuck, at least she'll change direction and perhaps make some progress. Three switchback legs of about 4m each, and thanks to the 2m downhill bump at the start, about the best we can hope for is that we'll break even -- that we'll be more or less back where we started before this latest drive. No closer to Burns Cliff -- and, considering what another failure will mean, in a sense we'll be much farther away.
At least we've got one more chance. Just to make it dramatic, Cooper's appointment means I get to do it all by myself. So, great: we're going to fail to get to Burns Cliff, and I'll get to feel like it's all my personal fault.
Well, what can I do to improve the odds? When I look at the drive Cooper and I sketched out, I spot a problem: switchback driving tends to carry you, on average, straight uphill. But the hole we've just dug is straight uphill from where we start the switchback. We might avoid it, but with all the slip we've been encountering, there's no certainty. So I change the drive: instead of three legs of 4m each, I use one leg of 8m and another of 4m. If this gains us enough uphill distance to get us onto the rockier terrain (the so-called "paving stones") we can see there, we won't care that we're a few meters farther from the cliff. (Just a few meters is all it would take, damn it! You could walk it in seconds.) And if it doesn't, then we'll be that much closer to the exit.
Assuming we don't just dig an even deeper hole, that is.
I name the drive "Hail Mary."
As it happens, I don't have to plan it completely solo. As I'm nearing completion, Spirit finishes its planning for the sol, so Jeff Biesiadecki comes upstairs to see how I'm doing, and Art presses him to backstop me on the sequence. Then John Wright joins him, and Art ropes him in as well.
So we decide to thoroughly review the thing among the three of us. This will delay uplink even longer, but considering the science we're trying to preserve, who cares?
Though it's a reasonably short sequence, reviewing the drive takes maybe 45 minutes or an hour. To paraphrase an old joke, if you put three rover drivers in a room, you get four opinions. Jeff and John do excellent work, poking at pretty much every decision I made, and coming up with some points I hadn't thought of. In the end, little changes -- the drive stays mostly intact, except that we convert the switchback turn from a turn-in-place into a K-turn. (The rover backs uphill through the long leg, then arcs halfway into the turn, reverses course, and arcs ahead to swing into the shorter second leg, driving forward.) I don't expect this to make a lot of difference, but it might save us some slippage, and it definitely looks cooler.
The only question is, will it work?
Courtesy NASA/JPL-Caltech. Well, crap. Looks like it was smart to keep that whole "NEENER NEENER NEENER" thing to myself.