Digitally Indigenous Content

Scrum and the Copernican Revolution

Does the Agile / Scrum software development paradigm encourage innovation, or discourage it? We continue to explore this topic in part II of our little fantasy. As you’ll recall from part I, we are imagining that the Copernican revolution in Astronomy happened during the course of a Scrum-driven effort to add yet more epicycles to the then-conventional “Geocentric” celestial model. This model, popular in Europe until the 16^th century, predicted the motion of the planets by assuming the Earth was the center of the Universe, with the planets rotating around the Earth in a complex combination of circles-on-other-circles called “epicycles”.

In part I we saw a junior engineer, Nick Copernicus, suggest a radical refactoring of the architecture of the armillary sphere your team is helping to construct. The “Mars” team, of which you are Scrum Master, is working along with other Scrum teams to produce a product update meeting the requirement: “As a Monarch, I want my armillary sphere to accurately reflect the new planetary observational data before any other Monarch has such a sphere—or else”. The King is coming to see a demo in six sprints, and sprint one is nearly over. Sprints 3 through 6 are reserved in the product plan for the construction and testing of the mechanical device.

Nick Copernicus has made the suggestion that the complexities of planetary motion could more naturally be described by assuming that planets move in concentric circular orbits around the Sun, rather than the complex circle-upon-circle pattern the current Earth-centered architecture now requires. You see the elegance of his proposal and have become a believer. As scrum master of Nick’s team, you’ve taken his proposal to the Product Owner. The PO does not argue with you about whether Nick’s architecture idea is good or bad, but instead points out that you’re in a schedule-driven release and that the schedule is tight. Of course this is correct. What is the right thing to do in this situation?

Here are some of the thoughts that run through your head:

• You’ve got the wrong architecture—you are now convinced. Completing this release using the current architecture will simply be adding patch upon patch, making the architecture more and more cumbersome, and harder and harder to maintain over time.
• If you don’t fix it now, when will you? Sure the schedule is tight, but it’s always tight. Last release was aggressive and this release will certainly be followed by yet another aggressive release. It’s always been that way for this Monarch, and will be for the foreseeable future.
• On the other hand, if you don’t meet the business requirement by the release date you’ve been told “heads will roll”. You are not sure if this is literal or figurative, but either way—is it worth risking your job or worse just because the new architecture is more “elegant”? To put it bluntly: what’s in it for you?
• The actual business requirement is to build an “armillary sphere” and to make it as accurate as possible. This new thing Nick came up with will indeed predict planetary motion, you think, but it won’t be an “armillary sphere”. By definition an armillary sphere is based on the assumption the Earth is stationary at the center of the Universe. You think this new heliocentric model should be accurate—qualitatively at least it explains the motion of the planets correctly and it just seems right—but the truth is that you haven’t tested it in action yet. All you’ve got is pictures on a napkin. Suppose you moved to this new architecture and it was less accurate—or didn’t work at all! The current model has been refined over nearly 2,000 years. The architecture may be a complicated mess, but one thing you can say for it is that it’s “battle tested”.
• As you think about it, the business requirement is pretty lousy. It actually dictates an architecture—namely a geocentric model of the Universe as embodied in an armillary sphere. A good requirement should talk about the end result, not the means. It seems pretty clear to you that the King’s intent should be to model the Universe most accurately, and as it really is. But you don’t know that from the requirement as written. Maybe the King (you look around cautiously when you think this) doesn’t understand the architecture or astronomy and just wants something fancy to look at, like a piece of furniture. Maybe he only wants to one-up his fellow Monarchs, and for this he might see having the world’s most complicated armillary sphere as a good thing. Or maybe he really is interested in the ultimate in accuracy and that’s his key goal. The King’s true purpose is not at all clear from the requirement you’ve been given, now that you have a new perspective.
• If part or all of the King’s real goal is to one-up his fellow Monarchs, moving to this new architecture could be one heck of a way to do it! If the Sun-centered “heliocentric” model turns out to be better, it could instantly make every armillary sphere in the world obsolete. The King would have the latest-and-greatest in models of the Universe, and the ultimate bragging rights.
• If you don’t suggest this architecture, and some other King’s engineering team comes up with it, then your armillary sphere will become obsolete. While you probably won’t literally lose your head over this, at a minimum the King is going to ask some hard questions like “Why didn’t you guy think of this!” Now that a better architecture is obvious, doing nothing seems like another, though slower, path to job loss.

Bottom line—it seems like a risk to refactor, and a risk not to. Your sense is that the refactored architecture is better, but it does not meet the “business requirement” as written, which you now realize contains a built-in assumption that you will use the old architecture. It may or may not meet the goal of improved accuracy—you just don’t know. And now that you see things from a new perspective you’re painfully aware that you really don’t understand the “strategy” driving the business requirements.

There’s no way you would talk to the King himself about this—you’ve never even met the King, and he would have no idea who you were if his guards even let you near him. Despite no real answer to your question about “what’s in it for me”, you somehow feel that advocating this new architecture is the right thing to do. The Product Owner is the King’s representative on this project, so you decide you need to go talk with him again, if for no other reason than to understand what is behind the business requirement in the first place. Before you have that conversation, though, you decide to think the problem over from the Product Owner’s point of view.

You know that the Product Owner’s key goal is to please the King—first, last and always. It’s not so much that the PO doesn’t understand what you and Nick are proposing, or your arguments about engineering elegance, architectural cohesiveness, and astronomical correctness. Rather it’s that the PO sees these arguments as totally beside the point. The PO’s only goal is shipping something on-time that meets the business requirements, and making the King happy. Your guess is that the PO will be thinking:

• I do not want to do ANYTHING that adds risk to the schedule. It’s already tight, and we have fixed scope, and a hard end date. It’s also too late to recruit and train anyone, at least for this release, so we have a fixed team size.
• The King may be pleased that we used our initiative and showed him a new approach that lets him one-up his fellow Monarchs. I—and the team—may get rewarded for this.
• On the other hand the King may think we wasted his time and money even thinking about this, let alone doing anything about it. He hasn’t asked us for it. If it costs very much we could even get fired for doing it, if he doesn’t like it. It’s very high risk.
• If I don’t say anything and some other King gets it, he will think we’re a bunch of idiots for not thinking of it first. He may fire us all. He’ll certainly fire me (or worse) as Product Owner for not keeping the product current.
• If I ask the King for permission before we have anything to show him—while it’s still just an idea on a napkin—he will (the PO secretly suspects) have no idea what we’re talking about. The idea is too abstract and technical. If we are to communicate the concept at all, we would need some sort of working model or prototype to make it concrete.

In sum, you realize the Product Owner also sees a risk either way, though in his case the safest course is probably to do nothing. On the other hand, if there was some low-risk way to develop Nick’s model, the PO might go for it because there’s upside for him and the project as well—if it works. You can’t think of what such a low-risk course might be, however. That seems like the key to this whole dilemma—how to take out the risk.

You don’t need to wonder what Nick Copernicus is thinking because he told you at lunch the other day.

“You know, I’ve been thinking about the refactored architecture” he said. “In fact, I can’t stopthinking about it.”

“What are you thinking, Nick?” you replied.

“I’m really convinced this is the right thing to do. I mean, I’m really convinced. In fact…” he hesitated.

“In fact?” you prompt.

Nick took a deep breath. “In fact, I’ve been thinking that if we don’t do this… Well, if we don’t do this—I’m going to move to another kingdom and start my own shop.”

You gasp “Nick, nobody quits this Monarch! And how do you know you’ll even get funded in a new kingdom? Or even get a new job! Times are really tough, especially for job-hoppers.”

“Yeah, I know” Nick replied, looking worried but resolute. “I would hate to do it. I really love this place. But if that’s the only way I can get this new architecture built, that’s what I’ll have to do.”

Leaving that conversation, you are really conflicted. Nick is one of your best guys, and you’d hate to lose him. You are also genuinely concerned about his future. He’s a great kid, and the odds against successfully launching a new workshop in a new kingdom and being in a position to build his heliocentric model are, well, truly astronomical. You have thought about it for several days, talked over the pros and cons with your spouse, and now you’ve made your decision. You pull Nick aside and say:

“Nick, I know how much this refactoring means to you. I’m willing to support you—back you all the way. But I want you to agree that if I do it, win or lose, you will wait at least until the end of this project before you even think about leaving. Will you promise that?”

Making this offer, you figure that if you get shot down—again—then at least this will buy time for him to cool down and for you to try to change his mind.

Taking a deep breath, Nick says: “OK.”

The die is cast—you and Nick will once again approach the product owner and try to sell the idea of a major architectural refactoring, moving from a geocentric armillary sphere to a heliocentric model. The two of you talk long after the shop has closed about your options and your approach, until you at last come up with an alternative you think the PO just might be willing to endorse. You leave the workshop late in the evening, with a handshake and a vow to speak with the product owner tomorrow.

To be continued…