We didn't make it work. I come in a little early and sit down at my workstation with Richard Kornfeld, who's now working on MSL and wants to gather some lessons from MER to improve MSL's design. Just as we get started, Chris comes over and asks, "Did you hear what happened?"
"Did you hear what happened?" is bad. People don't ask that when what happened is that things went perfectly, they ask it when what happened is that things went wrong. And things went wrong.
What went wrong is pretty embarrassing. The rover tracks its attitude using something called the Inertial Measurement Unit, or IMU. Because the IMU draws a lot of power, we want to turn it off when the rover is going to be sitting still (so that its attitude won't be changing) for more than a minute or so, and on when the rover is moving. Since the rover sits still and thinks for long periods when doing visodom -- as in yestersol's drive -- visodom sequences turn the IMU off and then back on several times.
Before turning the IMU off, you have to wait a few seconds to ensure that the flight software isn't still trying to read data from it. And after turning it on, you have to wait a few seconds for it to warm up.
What happened yestersol was that we screwed up this trivial procedure the very first time we turned the IMU off. Rather than telling the sequence to wait a few seconds and then turn the IMU off, we told it to turn the IMU off and then (uselessly) wait a few seconds. We did this right after a drive command, so the flight software tried to talk to the IMU while it was off, which caused the rover to decide that something was wrong and abort the rest of the drive. The result is that after all that work yestersol, the rover drove a total of 25cm.
I can think of something they could improve on MSL.[1]
To make this especially embarrassing (and worrisome), there were lots of places we should have caught this. Chris and I never should have made the mistake in the first place, but the uplink team also reviewed the sequence twice -- as usual -- before uplinking it, and nobody caught it during either review. Which means we could miss something else, something that could really damage the rover.
Even worse is that I foresaw this scenario the first time I learned there was this procedure for turning the IMU on and off. I knew we'd screw this up eventually -- like anything else that depends on humans to get it right -- and that I should automate the check. Well, better late than never. While Chris works on thisol's drive, I automate the check for this mistake. One less thing to go wrong in the future.
Ordinarily, a recovery sequence in a case like this would be simple enough -- cut out the commands that executed successfully, so that the rover picks up where it left off. But the rover shows signs of having slipped significantly even over the 25cm it tried to drive, which is probably because we're in a sandy hollow. So Chris plans a whole new drive, one that gets us out of the sand trap sooner. This is probably the right thing to do, but it takes time, and it makes us late once again.
Meanwhile, Art is trying to prevent an international incident. In a couple of sols, we have a coordinated experiment with ESA's MEX spacecraft. In order to do our part of the experiment, we need to be oriented at 260 degrees. But that isn't a scientifically useful heading for the experiments we want to perform on the outcrop, and if we make it to the outcrop on this drive, we won't want to move again just to support the experiment. We might be able to get in a single sol of IDDing the outcrop, but then we'll just have to stow and turn to perform the experiment. For some reason, our LTP folks didn't have the MEX experiment included in their plan, which is why we're scrambling now. Not only is the timing awkward, there's also a significant energy hit involved -- the experiment happens about 3 AM Mars time, so the rover has to be awake and well heated -- and our energy is low and not getting better.
They call Ray about it (Ray's on the freeway, on his way home). Art would rather bag the MEX experiment altogether to simplify things for us, but, as he tells Ray, "It's above my badge level -- this has been negotiated by governments on both sides of the Atlantic. You want to complain to [Project Manager] Jim Erickson about it?" Art knows this isn't going to do any good, since he's already complained to Jim himself, and Jim said MEX wins. But if he can get Ray on board, he can at least go back into the fight with more ammo. I don't expect Ray to say yes, but he does: "We'll be at Clovis" -- the outcrop -- "only a limited time; we don't want to blow two or three sols on this MEX thing," Ray says. "MEX will still be there, you know?" Art sympathizes, but notes that Erickson has already used his veto power -- "But we'll retry," he shrugs. In the end, Erickson does the right thing, the only thing he can do -- he plays the bad guy, and says we have to honor our international agreement to support MEX, and that's that.
Though that still leaves room for good-natured dissension in the ranks. If the drive errors out in the right way, we could still make it to Clovis but be at the wrong heading to support MEX, and it would look like an "unfortunate" accident. Chris jokingly volunteers to make this happen ....
Naturally, we do no such thing. And anyway, damn it, Jim, we're engineers, not diplomats. So we go back to doing what we do best.
We're running behind the (unrealistic) schedule, and Saina -- who always wants to just get the heck out of here -- is antsy. After the walkthrough, she asks how soon I can deliver. I look at my watch. It's 18:40 now ... "Well, this is pushing it," I say, "but maybe I can deliver by 19:00?"
"That would be great," Saina says. "If you deliver by 19:00, we will have a party."
Well, with incentive like that ... I get to work, going by the book, and when I deliver I look at my watch again. 18:58. I tell Saina, "You owe us a party."[2]
Footnotes:
[1] I don't actually know whether MSL works this way or not, but they eventually improved this on MER: a later flight software upgrade eliminated the need to wait before turning the IMU off, though you still have to wait after turning it on.
[2] But she never came through. Or, more likely, she did come through, and I just wasn't invited. ;-)
2009-07-31
2009-07-30
Spirit Sol 204
TiVo thinks we're cool. The TiVo user's guide has a screen shot of TiVo being used to display digital photos, and the digital photos in the screen shot are from Dan Maas's video of the MER rovers landing and roving. I show this to everyone, including Chris Leger, who tells me he has a buddy who works for TiVo. "I bet Tim did that," he says.
We're trying to get uphill to a rock outcrop about 7m away, but it's a heck of a painful drive. We're using visual odometry to do it, which always makes things more complex. And to make matters worse, we missed the first 10 minutes of the comm pass. (The rover sends the most critical data first, so if you miss the beginning of the pass, you miss the data you wanted most.) This leaves us without the rear hazcam from the previous sol's drive -- the image that would have shown us the terrain in one of the directions we could have done the drive. It'll show up later -- two hours later -- but the forward path turns out to be safe, so the delay doesn't matter in the end.
Which is good news, since the scientists are awfully eager to reach that outcrop. So is Art, who I think just wants the credit for getting them there. "The Rover Planners better get it right," he says, "or I'm calling the number on the back of that chair." (This is a reference to the "How's My Driving?" sign. The other joke there is, of course, that that number is Art's.)
Ray expresses the science view on this drive more succinctly. "Make it work," he says.
We're trying to get uphill to a rock outcrop about 7m away, but it's a heck of a painful drive. We're using visual odometry to do it, which always makes things more complex. And to make matters worse, we missed the first 10 minutes of the comm pass. (The rover sends the most critical data first, so if you miss the beginning of the pass, you miss the data you wanted most.) This leaves us without the rear hazcam from the previous sol's drive -- the image that would have shown us the terrain in one of the directions we could have done the drive. It'll show up later -- two hours later -- but the forward path turns out to be safe, so the delay doesn't matter in the end.
Which is good news, since the scientists are awfully eager to reach that outcrop. So is Art, who I think just wants the credit for getting them there. "The Rover Planners better get it right," he says, "or I'm calling the number on the back of that chair." (This is a reference to the "How's My Driving?" sign. The other joke there is, of course, that that number is Art's.)
Ray expresses the science view on this drive more succinctly. "Make it work," he says.
2009-07-26
Spirit Sol 200
Sol 200. Sol two hundred. Sol TWO HUNDRED.
Nope, I still can't believe it.
Neither, I think, can John Wright. "We're on sol 200," he remarks the moment I walk in the door.
"And it's directly thanks to you and me," I say sagely.
"True," he mock-agrees. "If it weren't for us, they'd be just about to grind Adirondack right now." We solemnly shake hands on this. "Our talent is exceeded only by our humility," he observes.
In honor of our 200th day on Mars, Emily Eelkema has brought in a yummy chocolate cake to share. She also shows a short presentation entitled, "Things NOT To Do While Driving the Rover." My favorite is a picture of a rusting car stuck halfway down a cliffside. "Do not drive the rover over a cliff," Emily says.
Eric Baumgartner is following Bob Bonitz to MSL, leaving us a rover planner short. They're not going to replace him, either. Instead, as a further cost-cutting move, they're sometimes going to have only three RPs available for a given sol. One rover will get two of us, the other will make do with one, and that uplink team will have to simplify the sol's plan to be something a single RP can do reliably.
We're a long way from the days of free ice cream.
[Next post: sol 204, July 30.]
Nope, I still can't believe it.
Neither, I think, can John Wright. "We're on sol 200," he remarks the moment I walk in the door.
"And it's directly thanks to you and me," I say sagely.
"True," he mock-agrees. "If it weren't for us, they'd be just about to grind Adirondack right now." We solemnly shake hands on this. "Our talent is exceeded only by our humility," he observes.
In honor of our 200th day on Mars, Emily Eelkema has brought in a yummy chocolate cake to share. She also shows a short presentation entitled, "Things NOT To Do While Driving the Rover." My favorite is a picture of a rusting car stuck halfway down a cliffside. "Do not drive the rover over a cliff," Emily says.
Eric Baumgartner is following Bob Bonitz to MSL, leaving us a rover planner short. They're not going to replace him, either. Instead, as a further cost-cutting move, they're sometimes going to have only three RPs available for a given sol. One rover will get two of us, the other will make do with one, and that uplink team will have to simplify the sol's plan to be something a single RP can do reliably.
We're a long way from the days of free ice cream.
[Next post: sol 204, July 30.]
2009-07-25
Spirit Sol 199
Yestersol's RAT operation went perfectly, drilling a hole almost 4mm deep. So thisol we're poking at the hole, of course.
Since most of the science team is going to be remote as of the end of this month, they're practicing for it this month by phoning into meetings from their science rooms as if they were back at their home institutions. But this isn't being enforced rigorously -- Mike Sims, for one, walks into the SMSA more than once to work out issues in person.
Which should act as a signal that having the science team operate remotely is going to be difficult. But there's only so much we can do about it: the money supply is finite, and that's forcing suboptimal solutions like this one.
Pretty frustrating, but I guess I should look on the bright side: we're having this problem because the rovers continue to operate so well. That makes it easier to take.
Courtesy NASA/JPL-Caltech. A front HAZCAM image showing (in shadow) the RAT holes we've drilled in this outcrop.
Since most of the science team is going to be remote as of the end of this month, they're practicing for it this month by phoning into meetings from their science rooms as if they were back at their home institutions. But this isn't being enforced rigorously -- Mike Sims, for one, walks into the SMSA more than once to work out issues in person.
Which should act as a signal that having the science team operate remotely is going to be difficult. But there's only so much we can do about it: the money supply is finite, and that's forcing suboptimal solutions like this one.
Pretty frustrating, but I guess I should look on the bright side: we're having this problem because the rovers continue to operate so well. That makes it easier to take.
Courtesy NASA/JPL-Caltech. A front HAZCAM image showing (in shadow) the RAT holes we've drilled in this outcrop.
2009-07-24
Spirit Sol 198
Today is all about food. It begins with Larry Soderblom's game of Hide-the-Pizza. John goes to the cafeteria to get a pizza, brings it back, then leaves again to fill his water bottle. While he's gone, Larry takes John's pizza and hides it on top of the cabinets.
John notices the missing pizza, but doesn't say anything. I guess he suspects Larry, because a few minutes later, I hear Larry chuckling, "I should know better than to mess with the rover planners." When I look around, John's munching away.
John's doesn't say anything at all about it until the Activity Plan Approval meeting. "Before showing the animation, I'd just like to point out what a fine judge of human character I am," he drawls. "I knew that if I put down my pizza and then filled up my water bottle, the pizza would be gone when I got back ...." We all laugh.
Later, Cindy Oda announces it's Julie Townsend's birthday, and apparently they got her a cake. A yummy, yummy chocolate cake. They got a candle, but none of us smokes, so nobody had a lighter. So I use my laser pointer to fake a flame, which Julie pretends to blow out after we all sing "Happy Birthday."
When Albert Yen asks me a question about something else, I answer it and then ask him what they've learned about the outcrop so far. They've found bromine, which is exciting because bromine gets deposited in rocks by water. Curiously, there's more bromine inside the rock than outside, which doesn't seem to fit the model of water depositing bromine on an igneous rock. One possible explanation is that the rock consolidated from ash-fall over a long period, with the ash condensing around bromine-laden water. He looks at the image on my screen and shakes his head wonderingly. "If you showed me this and didn't tell me what vehicle it was from, I would not have said Spirit," he says. "This is good stuff."
Well, it's nice to have the scientists interested in Spirit, at last.
In honor of Spirit's track record at uncovering new fatal modes -- such as the weird fatal that cropped up last time I was on shift, a few sols ago -- Mark Adler has decided we should rename Spirit the MSTB (Mars Surface Testbed).
John notices the missing pizza, but doesn't say anything. I guess he suspects Larry, because a few minutes later, I hear Larry chuckling, "I should know better than to mess with the rover planners." When I look around, John's munching away.
John's doesn't say anything at all about it until the Activity Plan Approval meeting. "Before showing the animation, I'd just like to point out what a fine judge of human character I am," he drawls. "I knew that if I put down my pizza and then filled up my water bottle, the pizza would be gone when I got back ...." We all laugh.
Later, Cindy Oda announces it's Julie Townsend's birthday, and apparently they got her a cake. A yummy, yummy chocolate cake. They got a candle, but none of us smokes, so nobody had a lighter. So I use my laser pointer to fake a flame, which Julie pretends to blow out after we all sing "Happy Birthday."
When Albert Yen asks me a question about something else, I answer it and then ask him what they've learned about the outcrop so far. They've found bromine, which is exciting because bromine gets deposited in rocks by water. Curiously, there's more bromine inside the rock than outside, which doesn't seem to fit the model of water depositing bromine on an igneous rock. One possible explanation is that the rock consolidated from ash-fall over a long period, with the ash condensing around bromine-laden water. He looks at the image on my screen and shakes his head wonderingly. "If you showed me this and didn't tell me what vehicle it was from, I would not have said Spirit," he says. "This is good stuff."
Well, it's nice to have the scientists interested in Spirit, at last.
In honor of Spirit's track record at uncovering new fatal modes -- such as the weird fatal that cropped up last time I was on shift, a few sols ago -- Mark Adler has decided we should rename Spirit the MSTB (Mars Surface Testbed).
2009-07-19
Spirit Sol 193
Yestersol's drive was a bullseye -- we ended up just where we'd aimed for. "Welcome to the outcrop," says an excited Larry Soderblom. The only problem is that we picked up a little south-facing tilt, meaning the rover is aimed away from the sun. So before we spend a lot of energy investigating the outcrop, we're going to move to a slightly different spot where we can pick up more solar energy.
That makes thisol a touch-and-go sol -- we'll poke at the rock underneath us, then drive to a new location. Our destination is just a couple of meters away, but we'll be north-facing instead, and the difference in tilt will give us dozens of additional Watt-hours per sol. Especially since we'll want to RAT the rock -- an energy-intensive operation -- this is a critical move. It's a bit of a weird drive, too -- they've already picked out the spot they want to RAT, so we're going to drive the rover right on top of that location, then turn around and back away from it.
"Everybody knows we fataled this morning on Spirit?" asks Rick Welch. I'd heard something about this already, but it gets my attention. Rick's exposition is larded with acronyms I'm not familiar with, but what I think happened was this. Possibly because of the weaker solar energy resulting from our south-facing tilt, the sun woke the rover a little later than expected, so Spirit started executing the morning's commands a little late. One of the first commands would have shut it back down for a nap -- sooner after wakeup than we thought when we sent the command, since the rover woke later than expected -- and its wakeup behavior and shutdown behavior got into a fight. They both won, sort of: the rover first decided to ignore the shutdown command because it was too close to the shutdown time, then another part of the software realized it was ignoring the shutdown command and signaled a fatal error, causing a reboot.
Now, normally, the rover records the presence of a fatal error in flash memory, which will cause it to enter fault-protection mode when it reboots. This mode ignores the on-board sequences and basically just tries to call home for help, which would have caused us to lose the sol. But because the rover was still in its wakeup mode when the fatal error was signaled, it wasn't allowing itself to write to flash yet. So it rebooted, but forgot about the fatal, and proceeded normally with the sol's activities. The rover was confused, but this happened to work out in our favor.
They want to write a new flight rule to prevent a recurrence of this, but apparently nobody left on the project knows how to add rules to the flight rule database. The rover's not the only thing that's confused around here.
[Next post: sol 198, July 24.]
That makes thisol a touch-and-go sol -- we'll poke at the rock underneath us, then drive to a new location. Our destination is just a couple of meters away, but we'll be north-facing instead, and the difference in tilt will give us dozens of additional Watt-hours per sol. Especially since we'll want to RAT the rock -- an energy-intensive operation -- this is a critical move. It's a bit of a weird drive, too -- they've already picked out the spot they want to RAT, so we're going to drive the rover right on top of that location, then turn around and back away from it.
"Everybody knows we fataled this morning on Spirit?" asks Rick Welch. I'd heard something about this already, but it gets my attention. Rick's exposition is larded with acronyms I'm not familiar with, but what I think happened was this. Possibly because of the weaker solar energy resulting from our south-facing tilt, the sun woke the rover a little later than expected, so Spirit started executing the morning's commands a little late. One of the first commands would have shut it back down for a nap -- sooner after wakeup than we thought when we sent the command, since the rover woke later than expected -- and its wakeup behavior and shutdown behavior got into a fight. They both won, sort of: the rover first decided to ignore the shutdown command because it was too close to the shutdown time, then another part of the software realized it was ignoring the shutdown command and signaled a fatal error, causing a reboot.
Now, normally, the rover records the presence of a fatal error in flash memory, which will cause it to enter fault-protection mode when it reboots. This mode ignores the on-board sequences and basically just tries to call home for help, which would have caused us to lose the sol. But because the rover was still in its wakeup mode when the fatal error was signaled, it wasn't allowing itself to write to flash yet. So it rebooted, but forgot about the fatal, and proceeded normally with the sol's activities. The rover was confused, but this happened to work out in our favor.
They want to write a new flight rule to prevent a recurrence of this, but apparently nobody left on the project knows how to add rules to the flight rule database. The rover's not the only thing that's confused around here.
[Next post: sol 198, July 24.]
2009-07-18
Spirit Sol 192
I have a problem I haven't had for some time: I get irrationally worried about yestersol's sequence and can't sleep, waking early to fret about it. It was one of our most complex sequences to date, after all. So I log in from home and, of course, it looks just fine. The only fly in the ointment is that the data products report we drove only 4.5m, where we commanded about 10m. But it was a relatively short comm pass, so it's likely we simply don't have all the data yet. In any case, the rover didn't set an error flag, and its heading looks good, which means the sequence almost certainly succeeded completely.
Big surprise. I go back to sleep.
Today's a much simpler sol, a welcome change after yestersol's harried pace. Indeed, John generates thisol's sequence mostly by copying yestersol's and cutting the stuff we don't need.
We zip through the uplink process almost without a hiccup. The only hitch comes near the end, when we're about to deliver. Joe Melko shows up and tries to talk me into changing the drive, to use a new technique where we accommodate slip by lying to the rover about its wheel radius. I take the approach of "Good idea, we'll be sure to do that on the next one" -- it is a good idea, but it's too late to change the drive thisol -- and Joe goes away. "I like the way you handled that one," Bill Bensler (our TUL today) comments, and that's that.
Bill also has a useful piece of advice for a smooth uplink process: "When in doubt," he says, "say you have a standing waiver."
Big surprise. I go back to sleep.
Today's a much simpler sol, a welcome change after yestersol's harried pace. Indeed, John generates thisol's sequence mostly by copying yestersol's and cutting the stuff we don't need.
We zip through the uplink process almost without a hiccup. The only hitch comes near the end, when we're about to deliver. Joe Melko shows up and tries to talk me into changing the drive, to use a new technique where we accommodate slip by lying to the rover about its wheel radius. I take the approach of "Good idea, we'll be sure to do that on the next one" -- it is a good idea, but it's too late to change the drive thisol -- and Joe goes away. "I like the way you handled that one," Bill Bensler (our TUL today) comments, and that's that.
Bill also has a useful piece of advice for a smooth uplink process: "When in doubt," he says, "say you have a standing waiver."
2009-07-17
Spirit Sol 191
"Here, smell this," says Julie. She's holding out an egg sandwich. "Does this smell funny?" It smells fine to me, but she's looking at it like she's not sure which of them is going to bite the other. I have visions of Julie running to the bathroom, incapacitated, just before the CAM. I volunteer to go get her another sandwich.
I have plenty of time to do this. John is struggling with a complex and lengthy touch-and-go. The IDD part wouldn't be especially complicated, but the target is hard to reach without running into one joint limit or another, so he has to work out an alternative, which takes yet more care and time. This, with the constant stream of interruptions that are the normal part of the RP-1 job, is quickly frustrating him. I gently try to persuade him to worry about either the IDD or the drive sequence and let me worry about the other, but he's too irritable to give up.
During the nominal mission, when our roles were reversed, John would do something when I got like this. He'd just quietly say, "This is so cool ...." It was enough to remind me why I'd chosen the job -- a remarkably effective tactic.
So I sit back and say, quietly, "This is so cool ...." He laughs and looks more at ease.
I let him keep working on it. Larry Soderblom's talking to several listeners about the composition of the sparkly stuff we exposed when we drove over it a couple of weeks ago. He refers to it as "iron sulfite and some crap," which provokes laughter.
"Crap would be good," I point out. "At least it would be a sign of life."
"You know," muses Scott Doudrick, "I'm starting to think we're not going to find that trilobite after all."
John finishes the sequences to his satisfaction -- we're running only a couple of hours late -- and hands over to me. It's one of the most complex sequences ever, with over 450 commands, and even though John did excellent work getting it into shape, I still have a lot to do.
We're very late when the CAM ends -- almost three hours behind schedule -- and it's a Friday, and everybody wants to go home. This being that kind of day, they notice a problem right at the end. We have a rule: no IDD movement can take place between 8.5 and 10.5 minutes after the rover wakes up. This strange restriction is a workaround for a known bug in the flight software; there's a vulnerable period where the flight software could crash and forget the position of the IDD, and we'd have to waste several sols recovering. But maybe we'll be completely done before the start of that window. Rick asks me to work out, as nearly as I can, the expected duration of the IDD sequence, and when I do, I figure we have only thirty seconds of margin.
"Is everybody OK with thirty seconds of margin?" he asks. Everybody wants to go home, so they're ready to buy off on it. I want to let them -- but my conscience is bothering me. I don't think it's the right thing to do.
"Yeah," I say reluctantly, "I have a problem with it."
At first they think I'm just kidding, but when they realize I'm serious, they swing into action. "Okay," Rick says, "let's get to work."
They make the needed changes and rebundle the uplink -- the fourth bundle of the day, which I initially think is some kind of record. But it's not: Marc Pack tells me the record was set earlier this week, when they generated no fewer than eight versions of the uplink before they were done. "So this is nothing," he says. "It's been a brutal week to be a TAP or RP on Spirit."
While they're doing this, I redo my calculation and realize I had made a small mistake: I had slightly undercounted the time required by the IDD sequence. The size of my mistake was thirty seconds -- just exactly enough time to have pushed us into the vulnerable window. I tell Rick this, and he says, "Good thing you spoke up!"
It is a good thing I spoke up. Better yet is that Rick asked whether anybody had a concern, and that the rest of the team took my concern seriously, without even a complaint, even though I'm not sure I could have defended it at the moment I raised it. It's late, and it's Friday, and they all want to go home. But even more, they want to do the right thing.
This is a hell of a team.
I have plenty of time to do this. John is struggling with a complex and lengthy touch-and-go. The IDD part wouldn't be especially complicated, but the target is hard to reach without running into one joint limit or another, so he has to work out an alternative, which takes yet more care and time. This, with the constant stream of interruptions that are the normal part of the RP-1 job, is quickly frustrating him. I gently try to persuade him to worry about either the IDD or the drive sequence and let me worry about the other, but he's too irritable to give up.
During the nominal mission, when our roles were reversed, John would do something when I got like this. He'd just quietly say, "This is so cool ...." It was enough to remind me why I'd chosen the job -- a remarkably effective tactic.
So I sit back and say, quietly, "This is so cool ...." He laughs and looks more at ease.
I let him keep working on it. Larry Soderblom's talking to several listeners about the composition of the sparkly stuff we exposed when we drove over it a couple of weeks ago. He refers to it as "iron sulfite and some crap," which provokes laughter.
"Crap would be good," I point out. "At least it would be a sign of life."
"You know," muses Scott Doudrick, "I'm starting to think we're not going to find that trilobite after all."
John finishes the sequences to his satisfaction -- we're running only a couple of hours late -- and hands over to me. It's one of the most complex sequences ever, with over 450 commands, and even though John did excellent work getting it into shape, I still have a lot to do.
We're very late when the CAM ends -- almost three hours behind schedule -- and it's a Friday, and everybody wants to go home. This being that kind of day, they notice a problem right at the end. We have a rule: no IDD movement can take place between 8.5 and 10.5 minutes after the rover wakes up. This strange restriction is a workaround for a known bug in the flight software; there's a vulnerable period where the flight software could crash and forget the position of the IDD, and we'd have to waste several sols recovering. But maybe we'll be completely done before the start of that window. Rick asks me to work out, as nearly as I can, the expected duration of the IDD sequence, and when I do, I figure we have only thirty seconds of margin.
"Is everybody OK with thirty seconds of margin?" he asks. Everybody wants to go home, so they're ready to buy off on it. I want to let them -- but my conscience is bothering me. I don't think it's the right thing to do.
"Yeah," I say reluctantly, "I have a problem with it."
At first they think I'm just kidding, but when they realize I'm serious, they swing into action. "Okay," Rick says, "let's get to work."
They make the needed changes and rebundle the uplink -- the fourth bundle of the day, which I initially think is some kind of record. But it's not: Marc Pack tells me the record was set earlier this week, when they generated no fewer than eight versions of the uplink before they were done. "So this is nothing," he says. "It's been a brutal week to be a TAP or RP on Spirit."
While they're doing this, I redo my calculation and realize I had made a small mistake: I had slightly undercounted the time required by the IDD sequence. The size of my mistake was thirty seconds -- just exactly enough time to have pushed us into the vulnerable window. I tell Rick this, and he says, "Good thing you spoke up!"
It is a good thing I spoke up. Better yet is that Rick asked whether anybody had a concern, and that the rest of the team took my concern seriously, without even a complaint, even though I'm not sure I could have defended it at the moment I raised it. It's late, and it's Friday, and they all want to go home. But even more, they want to do the right thing.
This is a hell of a team.
2009-07-16
Spirit Sol 190
Art's back. He's been on Opportunity for a month or so, so I ask him what's different between the missions. "Opportunity's RPs are more conservative," he says. It's clearly an understatement. "I would think that RPs would like to switch more between vehicles," he adds. I tell him I would. Maybe I'll get a chance, one of these days.
I've come in early for an engineering meeting, which turns out to have been delayed. When it does happen, it turns out I didn't even need to be there. I could have slept in.
But it's interesting, at least. The point of the meeting is to decide where we're going to drive Spirit from here. Or, more precisely, where we're going to drag Spirit from here, since we'll be spending 90% of our time dragging the right front wheel in order to preserve it for when we really need it.
The news that we're going to treat the wheel as already dead -- or, mostly dead -- is unpleasant. Last I heard, things were looking up. But apparently, the mechanical team has decided that there hasn't been enough improvement to declare Spirit's arthritis cured. For now, the plan is to aim for another 2km, and we're assuming Spirit's wheel has only 200m of life left in it, which is why we'll need to drag it 90% of the time. Still, this assessment is generally felt to be ultra-conservative. There's still hope the problem will fix itself -- say, while we're parked for the winter.
To help us during our gimpy drives, we have a new tool, a solar picture. This shows where solar energy is highest, helping us pick out good destinations for the rover. Ideally, we'll end every sol's drive in a spot where we'll collect the maximum solar energy -- which we need; we're so desperate for energy that we can't afford to use the IDD at all thisol, even though we're sitting on interesting rocks. With a kind of cheerful irony in his voice, Matt Golombek laments, "We drove 190 sols to get to outcrop. Now we have outcrop under our wheels and can't do anything with it even if we wanted to."
The meeting keeps getting in John Wright's way. He's trying to be polite, but he just wants to get started sequencing already, and he evidently sees the meeting as a needless delay. When Julie asks if we have a plan for where we're driving, John testily replies, "Yes: I'm going to ignore all recommendations and do it my way."
Well, John -- I think to myself -- you wanted the RP-1 job. This is what it's like.
Julie Townsend keeps getting empty envelopes in her JPL mail. Maybe it's a stalker -- it could be the snail-mail equivalent of calling a girl, breathing heavily, and hanging up. But I take another tack, and suggest she contemplate the empty envelopes as a Zen experience.
The drive is a weird one. For one thing, we drive backward the whole way. For each meter we drive, we turn off Spirit's lame wheel, drive 90% of the way (90cm), then re-enable the wheel and drive the remaining 10% (10cm). Since five-wheel driving causes the rover to yaw, we also have some fancy tricks to fix our heading periodically. It's slow, and the sequencing is complex, but I soon pick up on the patterns and it's not so bad.
Just before the CAM, Julie plays a new Public Service Announcement she found on NASA's Web site. It's a "Stay in School" video featuring Aerosmith. It also features fleeting glimpses of a bunch of MER team members, such as Justin Maki.
In honor of Art's return to the Spirit world, Julie also plays a video in which Art features prominently -- the "Trek to Bonneville" video from the MER home page. We watch through it and cheer good-naturedly for Art when he appears. To my surprise -- I never watched this one before -- I'm in it too. Very briefly, at the end, they have some B-roll of me driving the rover, the stuff the JPL media guy came and filmed months ago so that they'd have B-roll for stuff like this. I'm not saying anything, they're just shooting over my shoulder for two or three seconds as I play around with RSVP. But still --
I'm famous!
Courtesy NASA/JPL-Caltech. We don't seem to have publicly released many images of our "lily pad" maps. This isn't exactly what we were using, but it's close enough: a color-coded map, overlaid on actual imagery, showing areas where we'd expect better (blue) or worse (red) solar energy.
Courtesy NASA/JPL-Caltech. The "Trek to Bonneville" video. The image above links to the MP4 video; it's also available in other formats.
I've come in early for an engineering meeting, which turns out to have been delayed. When it does happen, it turns out I didn't even need to be there. I could have slept in.
But it's interesting, at least. The point of the meeting is to decide where we're going to drive Spirit from here. Or, more precisely, where we're going to drag Spirit from here, since we'll be spending 90% of our time dragging the right front wheel in order to preserve it for when we really need it.
The news that we're going to treat the wheel as already dead -- or, mostly dead -- is unpleasant. Last I heard, things were looking up. But apparently, the mechanical team has decided that there hasn't been enough improvement to declare Spirit's arthritis cured. For now, the plan is to aim for another 2km, and we're assuming Spirit's wheel has only 200m of life left in it, which is why we'll need to drag it 90% of the time. Still, this assessment is generally felt to be ultra-conservative. There's still hope the problem will fix itself -- say, while we're parked for the winter.
To help us during our gimpy drives, we have a new tool, a solar picture. This shows where solar energy is highest, helping us pick out good destinations for the rover. Ideally, we'll end every sol's drive in a spot where we'll collect the maximum solar energy -- which we need; we're so desperate for energy that we can't afford to use the IDD at all thisol, even though we're sitting on interesting rocks. With a kind of cheerful irony in his voice, Matt Golombek laments, "We drove 190 sols to get to outcrop. Now we have outcrop under our wheels and can't do anything with it even if we wanted to."
The meeting keeps getting in John Wright's way. He's trying to be polite, but he just wants to get started sequencing already, and he evidently sees the meeting as a needless delay. When Julie asks if we have a plan for where we're driving, John testily replies, "Yes: I'm going to ignore all recommendations and do it my way."
Well, John -- I think to myself -- you wanted the RP-1 job. This is what it's like.
Julie Townsend keeps getting empty envelopes in her JPL mail. Maybe it's a stalker -- it could be the snail-mail equivalent of calling a girl, breathing heavily, and hanging up. But I take another tack, and suggest she contemplate the empty envelopes as a Zen experience.
The drive is a weird one. For one thing, we drive backward the whole way. For each meter we drive, we turn off Spirit's lame wheel, drive 90% of the way (90cm), then re-enable the wheel and drive the remaining 10% (10cm). Since five-wheel driving causes the rover to yaw, we also have some fancy tricks to fix our heading periodically. It's slow, and the sequencing is complex, but I soon pick up on the patterns and it's not so bad.
Just before the CAM, Julie plays a new Public Service Announcement she found on NASA's Web site. It's a "Stay in School" video featuring Aerosmith. It also features fleeting glimpses of a bunch of MER team members, such as Justin Maki.
In honor of Art's return to the Spirit world, Julie also plays a video in which Art features prominently -- the "Trek to Bonneville" video from the MER home page. We watch through it and cheer good-naturedly for Art when he appears. To my surprise -- I never watched this one before -- I'm in it too. Very briefly, at the end, they have some B-roll of me driving the rover, the stuff the JPL media guy came and filmed months ago so that they'd have B-roll for stuff like this. I'm not saying anything, they're just shooting over my shoulder for two or three seconds as I play around with RSVP. But still --
I'm famous!
Courtesy NASA/JPL-Caltech. We don't seem to have publicly released many images of our "lily pad" maps. This isn't exactly what we were using, but it's close enough: a color-coded map, overlaid on actual imagery, showing areas where we'd expect better (blue) or worse (red) solar energy.
Courtesy NASA/JPL-Caltech. The "Trek to Bonneville" video. The image above links to the MP4 video; it's also available in other formats.
2009-07-10
Spirit Sol 185
Yestersol they saw a weird, sudden temperature decline at about 13:30 LST, as reported by the temperature sensor covering the right front wheel. Scott Doudrick wonders aloud whether it might be a shadowing issue, so I use RSVP to project the shadows on the rover at that time of day. It's not a shadowing issue. Later we hear that this is something that happens a lot, and nobody knows why. I stare at the graph for a while, trying to think of a plausible explanation, but I come up empty.
Thisol is basically similar to yestersol, continuing the testing. Afterward we're supposed to drive to a new location, but this ends up being cut because we can't afford the resource usage. So it turns out to be a fairly light day for us.
The good news is that the testing might be working -- that is, not only are we learning what we wanted to learn, but the experiment to reflow the wheel lubricant might actually be doing that. Daniel Limonadi says that they've already plotted the data, and it's showing the right front wheel's current draw more "in family" with the other wheels.
Daniel mostly works on Opportunity, where he's trying to persuade the science team to take better advantage of Opportunity's temporary power-rich situation. Opportunity's parked on a slope where her solar panels are aimed right at the sun -- even with Opportunity's stuck-open heater (which is being mitigated by their nightly Deep Sleep) they're getting so much energy they have to shunt, wasting some of it. When they leave this crater, that won't be the case any more, and Daniel dreads hearing the scientists start complaining about a lack of energy again. "They should use it while they've got it," he says.
Doudrick commiserates: "I dream of the days when we used to say, 'Here's a huge energy spike, let's put a comm pass here.'" We're farther from the equator than Opportunity and not particularly aimed at the sun, making us the power-poor mission for the time being.
The wheel-current news from Daniel is almost too good to be true, so I go ask Lori Shirashi, one of the Mechanical folks, for more details. She shows me plots of the current draw from the recent test. The right front wheel is higher than the others -- "but much better than it has been," she assures me. In fact, she says -- pointing to a different plot -- the problem might have been slowly getting better on its own since about sol 150.
The current working theory is that the problem was caused by our long drives. Driving pushes lubricant out of the gear box, and since we were driving during the warmest part of the day, the lubricant was too cold to flow back in at night. We're been sitting relatively still since about sol 150, giving the lubricant a chance to flow back in during the warmer parts of the day. Yestersol's "Martian Jiffy Lube" seems to have just helped the process along some.
So maybe, just this once, being in denial was an OK strategy.
[Next post: sol 190, July 16.]
Thisol is basically similar to yestersol, continuing the testing. Afterward we're supposed to drive to a new location, but this ends up being cut because we can't afford the resource usage. So it turns out to be a fairly light day for us.
The good news is that the testing might be working -- that is, not only are we learning what we wanted to learn, but the experiment to reflow the wheel lubricant might actually be doing that. Daniel Limonadi says that they've already plotted the data, and it's showing the right front wheel's current draw more "in family" with the other wheels.
Daniel mostly works on Opportunity, where he's trying to persuade the science team to take better advantage of Opportunity's temporary power-rich situation. Opportunity's parked on a slope where her solar panels are aimed right at the sun -- even with Opportunity's stuck-open heater (which is being mitigated by their nightly Deep Sleep) they're getting so much energy they have to shunt, wasting some of it. When they leave this crater, that won't be the case any more, and Daniel dreads hearing the scientists start complaining about a lack of energy again. "They should use it while they've got it," he says.
Doudrick commiserates: "I dream of the days when we used to say, 'Here's a huge energy spike, let's put a comm pass here.'" We're farther from the equator than Opportunity and not particularly aimed at the sun, making us the power-poor mission for the time being.
The wheel-current news from Daniel is almost too good to be true, so I go ask Lori Shirashi, one of the Mechanical folks, for more details. She shows me plots of the current draw from the recent test. The right front wheel is higher than the others -- "but much better than it has been," she assures me. In fact, she says -- pointing to a different plot -- the problem might have been slowly getting better on its own since about sol 150.
The current working theory is that the problem was caused by our long drives. Driving pushes lubricant out of the gear box, and since we were driving during the warmest part of the day, the lubricant was too cold to flow back in at night. We're been sitting relatively still since about sol 150, giving the lubricant a chance to flow back in during the warmer parts of the day. Yestersol's "Martian Jiffy Lube" seems to have just helped the process along some.
So maybe, just this once, being in denial was an OK strategy.
[Next post: sol 190, July 16.]
2009-07-09
Spirit Sol 184
Today's like yesterday: continuing a canned test that will go on for a couple more days. Today's test is different, though: we're heating the drive actuators for several hours and driving just a couple of meters, in an effort to reflow the lubricant that we believe has congealed. (I dub this the "Martian Jiffy Lube.") We'll try again tomorrow, and if this works, Spirit will be cured.
We're done by about 13:00, but there's another in the series of science lectures at 16:00, so I eagerly stick around for the three hours. When the time comes, it turns out that apparently nobody told the scientists about it, so none shows up.
"There are only, like, five scientists still here," apologizes Daniel Limonadi, who's coordinating these lectures. "And none of them has anything prepared. We'll, uh, reschedule. And next time I'll be sure to tell them about it."
We're done by about 13:00, but there's another in the series of science lectures at 16:00, so I eagerly stick around for the three hours. When the time comes, it turns out that apparently nobody told the scientists about it, so none shows up.
"There are only, like, five scientists still here," apologizes Daniel Limonadi, who's coordinating these lectures. "And none of them has anything prepared. We'll, uh, reschedule. And next time I'll be sure to tell them about it."
2009-07-08
Spirit Sol 183
In contrast to yesterday's hectic pace, today is pretty relaxed. This is because we're running a diagnostic sequence that was already worked out in the testbed; the sequence runs the drive motors at different rates to gather more information about the wheel problem. All we have to do is change a few minor things and uplink it.
Which leaves me plenty of time to read the BBC article on our project. The news is that NASA is seeking MER funding to continue past the end of the fiscal year (September), when our current funding will run out. (This is not news to us, but I guess it's just been officially announced.) If the funding is approved, we'll have money to keep driving the rovers through late 2005.
Which leaves me plenty of time to read the BBC article on our project. The news is that NASA is seeking MER funding to continue past the end of the fiscal year (September), when our current funding will run out. (This is not news to us, but I guess it's just been officially announced.) If the funding is approved, we'll have money to keep driving the rovers through late 2005.
2009-07-07
Spirit Sol 182
We're back on a tight schedule: when I arrive, it's around 2AM LST, leaving us only about 6 hours to get everything done and uplinked.
So naturally it's a complex sol. Yestersol's IDD sequence faulted out when the IDD flight software detected a pending self-collision, precluding the remainder of the IDD work and the subsequent drive. Which means we have both IDD and drive sequences to do thisol -- a touch-and-go -- on a tight schedule.
Fortunately, we're able to salvage a lot from yestersol's sequences, which makes the problem manageable. Which is good, because it leaves time for Andy to get paranoid about the drive's not completing. If it doesn't complete, we'll be facing in a particularly bad direction for comm. So I go back and make the sequence more robust; in the most likely cases, it will stop trying to drive and just skip ahead to the turn for comm. That, and some other last-minute changes that come up, should help me sleep tonight ....
Astonishingly, I voluntarily stay late for a meeting. This is a meeting Justin Maki called to discuss the problem we've been seeing in the terrain meshes. I've been interested in this problem ever since we noted it for the first time at Mazatzal. ("When this happened at Mazatzal, we said, 'Someday this will happen again, and then we'll have two data points to compare,'" I point out. We now have enough data that we should be able to solve it.)
Chris shows a preliminary graph of the error in MB placements since the start of the mission. There's no universal agreement on the trend of the graph, but it looks to me as if it took a sharp uptick around sol 140 or so: we go from having most placement errors under 1cm to having a lot of errors of 2-3cm or more.
The possible problems are these:
(a) Movement/flexure between cameras and rover/IMU since Earth calibration.
(b) Movement/flexure between cameras and IDD since Earth calibration.
(c) Movement/flexure between cameras and the left and right front hazcams since Earth calibration. Todd Litwin, who wrote most of the flight software for dealing with the cameras, thinks this is the leading candidate.
(d) Maybe the problem is with the IDD, not the cameras.
(e) Maybe the errors are due to ground processing of the images, not with the vehicle.
We need to explore all of these. As the meeting is wrapping up, I think of a way to help narrow down the options that we can do with the information we already have. Because the terrain meshes show the terrain as farther away from the rover than it really is, the rover seems to "float" a couple of centimeters above the terrain if you look closely. So I suggest taking a similar look at the rear hazcam meshes, and maybe at the navcam meshes. If the rover appears to be floating above those in the same way, then the problem almost certainly has nothing to do with the front hazcams themselves; alternatively, if only the front hazcam meshes are messed up, then the problem is almost certainly in the cameras. "Ooo, good idea," Justin says, and he assigns someone to look into it.
For now, we're not going to update the camera models on board. The only part of the flight software that uses them for on-board processing is visual odometry, and we can't usefully use visual odometry with the hazcams anyway. (Except for turns in place, and the visodom we can do with turn-in-place isn't affected by this hazcam problem.) But we're going to use updated camera models on the ground, for Spirit at least, which should reduce the appearance of this problem in the future.
So naturally it's a complex sol. Yestersol's IDD sequence faulted out when the IDD flight software detected a pending self-collision, precluding the remainder of the IDD work and the subsequent drive. Which means we have both IDD and drive sequences to do thisol -- a touch-and-go -- on a tight schedule.
Fortunately, we're able to salvage a lot from yestersol's sequences, which makes the problem manageable. Which is good, because it leaves time for Andy to get paranoid about the drive's not completing. If it doesn't complete, we'll be facing in a particularly bad direction for comm. So I go back and make the sequence more robust; in the most likely cases, it will stop trying to drive and just skip ahead to the turn for comm. That, and some other last-minute changes that come up, should help me sleep tonight ....
Astonishingly, I voluntarily stay late for a meeting. This is a meeting Justin Maki called to discuss the problem we've been seeing in the terrain meshes. I've been interested in this problem ever since we noted it for the first time at Mazatzal. ("When this happened at Mazatzal, we said, 'Someday this will happen again, and then we'll have two data points to compare,'" I point out. We now have enough data that we should be able to solve it.)
Chris shows a preliminary graph of the error in MB placements since the start of the mission. There's no universal agreement on the trend of the graph, but it looks to me as if it took a sharp uptick around sol 140 or so: we go from having most placement errors under 1cm to having a lot of errors of 2-3cm or more.
The possible problems are these:
(a) Movement/flexure between cameras and rover/IMU since Earth calibration.
(b) Movement/flexure between cameras and IDD since Earth calibration.
(c) Movement/flexure between cameras and the left and right front hazcams since Earth calibration. Todd Litwin, who wrote most of the flight software for dealing with the cameras, thinks this is the leading candidate.
(d) Maybe the problem is with the IDD, not the cameras.
(e) Maybe the errors are due to ground processing of the images, not with the vehicle.
We need to explore all of these. As the meeting is wrapping up, I think of a way to help narrow down the options that we can do with the information we already have. Because the terrain meshes show the terrain as farther away from the rover than it really is, the rover seems to "float" a couple of centimeters above the terrain if you look closely. So I suggest taking a similar look at the rear hazcam meshes, and maybe at the navcam meshes. If the rover appears to be floating above those in the same way, then the problem almost certainly has nothing to do with the front hazcams themselves; alternatively, if only the front hazcam meshes are messed up, then the problem is almost certainly in the cameras. "Ooo, good idea," Justin says, and he assigns someone to look into it.
For now, we're not going to update the camera models on board. The only part of the flight software that uses them for on-board processing is visual odometry, and we can't usefully use visual odometry with the hazcams anyway. (Except for turns in place, and the visodom we can do with turn-in-place isn't affected by this hazcam problem.) But we're going to use updated camera models on the ground, for Spirit at least, which should reduce the appearance of this problem in the future.
2009-07-03
Spirit Sol 178
"Could you do the IDD sequence?" Chris asks. "I'm worried about the drive."
Ah, it's nice to be back in RP-1 territory. Even though writing the sequence is almost completely a matter of expanding macros -- very little problem-solving required, in this case -- it's still a nice change of pace.
The main reason Chris is worried about the drive is that thisol will be our first use of visual odometry (visodom) on Spirit. After two or three rounds of very nearly using it, only to cancel the experiment at the last minute (breaking Mark Maimone's heart each time), it looks like it really is on for the day.
So naturally, when Mark stops by to help out, I look at him and deadpan, "You know we're not using visodom today, right?" The look of horror on his face before he realizes I'm just fooling is priceless. Then he starts to laugh. "Very funny," he admits, "you got me."
The chair with the "How's My Driving?" sign on it has been in the RP-1 spot since the sign was first taped to it. I think I broke the spell, though -- I moved it to the RP-2 workstation a few days ago, and now it's begun to float around the room. Like just anybody can sit in it! Today it's Ray's chair. Later, when everyone else has left, I steal it back.
While the software is cranking away, getting things ready for the CAM, Jeff Favretto comes in to ask for everyone's help. The flight director webcast for the day is going to have a special feature: at the end of the report, they want a whole bunch of MERfolk to pop up and shout, "Congratulations, Cassini!" So Jeff is rounding up participants.
We all troop over to the SMSA and pack in around Jeff. As we arrange ourselves, I realize that Jeff -- who's taller than I am -- is standing between me and the camera, so I'm not even going to be visible. I caption this event in my mind: "The other MER members and I congratulate Cassini. Not pictured: me."
But as it happens, Jeff repositions himself, and I'm clearly visible in the webcast after all. At the crucial moment, on cue, the camera pulls back, a couple of dozen additional team members pop out from hiding places, and we all wave and shout, "Congratulations, Cassini!" It looks a lot less cheesy than I'd envisioned.
Mark Maimone has been working in the sandbox, testing something or other. Since this is Spirit's first use of visodom, Mark Adler asks me to have him present at the CAM to answer any questions. So I call Mark -- who has just gotten the system up to the point where he can start useful testing -- and ask him to come to the CAM.
Mark arrives on time, but -- to neither his surprise nor mine -- the CAM starts 20 minutes late. Still, he's in a good mood; the stuff he's worked on for years is getting its day in the sun. Naturally, in the middle of the CAM, Adler starts asking questions about resource usage. Then he turns to Mark and says, "You know what, we're out of time -- we're going to cut visodom."
But he's kidding. Mark laughs his good-natured, rueful laugh, saying "I was waiting for that. Thanks a lot."
In the end, the commands go to the spacecraft. Visodom, at last.
On my way out, I run into Trina Ray on the steps of Building 264. I've known Trina for a long time -- she's a Cassini radio science team member, energetic and funny. She rants about the poor job the Cassini project has done of coordinating with the media. Only a day or two into Saturn orbit, they're already making amazing discoveries about the rings and the moons, and very little of it is to be seen in the press. "We've got Prometheus pulling material off the F-ring, we've got Epimetheus venting oxygen into space ...." She buries her face in her hands and issues a frustrated groan. "I gotta go," she says, and she walks into the building.
[Next post: sol 182, July 7.]
Ah, it's nice to be back in RP-1 territory. Even though writing the sequence is almost completely a matter of expanding macros -- very little problem-solving required, in this case -- it's still a nice change of pace.
The main reason Chris is worried about the drive is that thisol will be our first use of visual odometry (visodom) on Spirit. After two or three rounds of very nearly using it, only to cancel the experiment at the last minute (breaking Mark Maimone's heart each time), it looks like it really is on for the day.
So naturally, when Mark stops by to help out, I look at him and deadpan, "You know we're not using visodom today, right?" The look of horror on his face before he realizes I'm just fooling is priceless. Then he starts to laugh. "Very funny," he admits, "you got me."
The chair with the "How's My Driving?" sign on it has been in the RP-1 spot since the sign was first taped to it. I think I broke the spell, though -- I moved it to the RP-2 workstation a few days ago, and now it's begun to float around the room. Like just anybody can sit in it! Today it's Ray's chair. Later, when everyone else has left, I steal it back.
While the software is cranking away, getting things ready for the CAM, Jeff Favretto comes in to ask for everyone's help. The flight director webcast for the day is going to have a special feature: at the end of the report, they want a whole bunch of MERfolk to pop up and shout, "Congratulations, Cassini!" So Jeff is rounding up participants.
We all troop over to the SMSA and pack in around Jeff. As we arrange ourselves, I realize that Jeff -- who's taller than I am -- is standing between me and the camera, so I'm not even going to be visible. I caption this event in my mind: "The other MER members and I congratulate Cassini. Not pictured: me."
But as it happens, Jeff repositions himself, and I'm clearly visible in the webcast after all. At the crucial moment, on cue, the camera pulls back, a couple of dozen additional team members pop out from hiding places, and we all wave and shout, "Congratulations, Cassini!" It looks a lot less cheesy than I'd envisioned.
Mark Maimone has been working in the sandbox, testing something or other. Since this is Spirit's first use of visodom, Mark Adler asks me to have him present at the CAM to answer any questions. So I call Mark -- who has just gotten the system up to the point where he can start useful testing -- and ask him to come to the CAM.
Mark arrives on time, but -- to neither his surprise nor mine -- the CAM starts 20 minutes late. Still, he's in a good mood; the stuff he's worked on for years is getting its day in the sun. Naturally, in the middle of the CAM, Adler starts asking questions about resource usage. Then he turns to Mark and says, "You know what, we're out of time -- we're going to cut visodom."
But he's kidding. Mark laughs his good-natured, rueful laugh, saying "I was waiting for that. Thanks a lot."
In the end, the commands go to the spacecraft. Visodom, at last.
On my way out, I run into Trina Ray on the steps of Building 264. I've known Trina for a long time -- she's a Cassini radio science team member, energetic and funny. She rants about the poor job the Cassini project has done of coordinating with the media. Only a day or two into Saturn orbit, they're already making amazing discoveries about the rings and the moons, and very little of it is to be seen in the press. "We've got Prometheus pulling material off the F-ring, we've got Epimetheus venting oxygen into space ...." She buries her face in her hands and issues a frustrated groan. "I gotta go," she says, and she walks into the building.
[Next post: sol 182, July 7.]
2009-07-02
Spirit Sol 177
Normally, I come in an hour before the activity plan approval meeting -- which is when I really need to be here -- so I can have some time to get up to speed. Not today: today, we were out looking at houses with our realtor, and I'm just barely on time.
"I never used to really understand why people who are buying a house were late for work," I tell Chris ruefully. "I kept thinking, 'Why can't they get it together and not let this interfere with their job?'"
"Now you get it, huh?"
"Now I get it."
At least I'm not late. And there's not much to get up to speed on today, luckily for me -- we're just IDDing, continuing to poke at what's nearby before we move along.
I also have time to catch up on my email, including notes from the Spirit traversability meeting I missed. If the problem with Spirit's right front wheel is not fixed, we only have 100m to 300m of mileage -- er, meterage -- left before we lose it. I can tell I'm still in denial. Spirit's going to be fine, I keep thinking. They'll fix it.
I only hope that's true.
There's been some progress in unraveling the mystery of the bright scratches Spirit's wheels left behind when climbing the rocks in this area. Larry Soderblom says he thinks the "sparkly bits," as he calls them, are chemical precipitates. I.e., they're chemicals left behind when the local water evaporated -- the clearest evidence yet, he says, for fluids in this area.
Saina Ghandchi is our TUL today, which lately has meant we're on a fast track to uplink. Saina pushes the process along as fast as she can -- today, for instance, we skip the animation at the CAM and, since the IDD sequences haven't changed since the earlier walkthrough, we only skim them. Saina wants to go home!
"I never used to really understand why people who are buying a house were late for work," I tell Chris ruefully. "I kept thinking, 'Why can't they get it together and not let this interfere with their job?'"
"Now you get it, huh?"
"Now I get it."
At least I'm not late. And there's not much to get up to speed on today, luckily for me -- we're just IDDing, continuing to poke at what's nearby before we move along.
I also have time to catch up on my email, including notes from the Spirit traversability meeting I missed. If the problem with Spirit's right front wheel is not fixed, we only have 100m to 300m of mileage -- er, meterage -- left before we lose it. I can tell I'm still in denial. Spirit's going to be fine, I keep thinking. They'll fix it.
I only hope that's true.
There's been some progress in unraveling the mystery of the bright scratches Spirit's wheels left behind when climbing the rocks in this area. Larry Soderblom says he thinks the "sparkly bits," as he calls them, are chemical precipitates. I.e., they're chemicals left behind when the local water evaporated -- the clearest evidence yet, he says, for fluids in this area.
Saina Ghandchi is our TUL today, which lately has meant we're on a fast track to uplink. Saina pushes the process along as fast as she can -- today, for instance, we skip the animation at the CAM and, since the IDD sequences haven't changed since the earlier walkthrough, we only skim them. Saina wants to go home!
2009-07-01
Spirit Sol 176
This is Cassini's big day -- SOI[1] is upon us. Maybe it's my imagination, but it doesn't seem to me that they're getting the same kind of media coverage we got. Which is a shame: Saturn's a beautiful planet, and Cassini's an exciting mission that will help unravel its mysteries. Still, it seems that the public likes us better.
Nyaah nyaah.
I have my own reasons for being more interested in MER. As usual, today brings something new. As part of solving the problem we've had with the terrain not being where the images made us think it was, we're recalibrating the IDD today. This involves commanding the arm to a series of twenty or so positions, and taking a picture of each one. Later, Eric Baumgartner will compare the commanded position with the actual position as seen in the HAZCAMs, and he'll use that information to recalibrate the IDD.
Steve Squyres has a simpler description: "Martian tai-chi," he calls it.
Since Eric already worked out the sequence of moves, there's not much for Chris and me to do, except modify a few of the poses slightly so that the arm doesn't scrape the ground. Similarly to yesterday, I end up showing the resulting animation to more NASA Code T types who walk through late in the afternoon, guided by Jim Erickson. When they show up, I think they're the entourage of a Congressman who was visiting today -- I waste a minute or two trying to figure out which of them is the Congressman before I realize they're a separate party.
Darn, I was really hoping to see the Congressman and not quite get my picture taken with him. I hate to break with tradition.
Footnotes:
[1] Saturn Orbit Insertion.
Nyaah nyaah.
I have my own reasons for being more interested in MER. As usual, today brings something new. As part of solving the problem we've had with the terrain not being where the images made us think it was, we're recalibrating the IDD today. This involves commanding the arm to a series of twenty or so positions, and taking a picture of each one. Later, Eric Baumgartner will compare the commanded position with the actual position as seen in the HAZCAMs, and he'll use that information to recalibrate the IDD.
Steve Squyres has a simpler description: "Martian tai-chi," he calls it.
Since Eric already worked out the sequence of moves, there's not much for Chris and me to do, except modify a few of the poses slightly so that the arm doesn't scrape the ground. Similarly to yesterday, I end up showing the resulting animation to more NASA Code T types who walk through late in the afternoon, guided by Jim Erickson. When they show up, I think they're the entourage of a Congressman who was visiting today -- I waste a minute or two trying to figure out which of them is the Congressman before I realize they're a separate party.
Darn, I was really hoping to see the Congressman and not quite get my picture taken with him. I hate to break with tradition.
Footnotes:
[1] Saturn Orbit Insertion.
Subscribe to:
Posts (Atom)
Posts
