Software penetrates every pore of human existence. We look up the weather info over the web, giving up on outdoor thermometers. We’re driving to destinations with GPS navigator (forget paper maps with their G7 sections on page 59). We turn on RunKeeper when riding a bike to calculate the average speed and run and boast in Twitter. We’re using software every single day of our lives. It seems we’re hugging our dear gadgets a lot more than our loved ones.
No one knows the exact how-to of writing great software fast, that’s the problem. Waterfall passed away at the crossing of 2 centuries, whereas new software development methodologies (agile) fail at solving the fundamental problems so far. We’re living in very interesting times. Software development industry grows fast right here, right now, and the foundation for a quantitative leap is building up.
Now it’s absolutely clear that agile is turning into a mainstream methodology, and soon only a few extinct mammoths will do waterfall. Agile has won the race. Microsoft is now working to improve agile methodology support in TFS, they also do iterative development for many of their products. IBM has launched Jazz agile platform. Even SAP implement agile (I’d love to take a sneak peek at this).
See what Wolfgang Gattner, the CIO of Deutsche Bank, says:
We have to get there. The traditional ways are not fast enough, they are too complex and they don’t surface the innovation. So that is very clear.
About each and every organization is at least becoming aware of the fact that old-fashioned software development is dead. They’re trying to change the way things work and they see agile as the best possible solution for the time being. But here comes a question: are agile improvements good for software development? They are good. But are they good enough? No. Agile is not solving all the problems. They release crap over and over again. Development speed still leaves much to be desired. Many software solutions fail, abound in bugs and their poor quality is letting us down at best, actually, worst possible times.
Remember the recent outage of Amazon servers. Our server was down for about a week! Skype is failing time and again (especially as acquired by Microsoft). Everyone must have been doing some shaman passes one way or another to make it work.
iOS issues are quite common. My iPhone wouldn’t go to sleep mode after updates and would eat up the battery charge in just 8 hours, let alone accidental slips and calls. That was terribly annoying. They’ve been dragging with the fix for almost a month!
Do Right Things
Let’s take a closer look at the 3 fundamental problems of software development. The ultimate goal is — do right things.
What does it mean — right things? These are workable things, things that solve specific problems, and solve them well. Several thousand (!) project management tools are available on the market. How many of these are really good? 10? 20? Hardly more than 20.
Why all these horrible monsters with appalling interface have ever come to life? They torture both their users and their creators, and they still hang in there. Why hundreds of primitive tools that do the same basic things and only vary in their color, platform and login screen? I have no answer. But I know for sure: this world is inundated with useless software that stupidly replicates each other, drives people nuts with their UI and pretends to be the best.
We have to learn to create the right software. This is of paramount importance. We need to get into people’s hearts with first release.
Do Things Right
This ranks second. Do things right.
If we do the right thing, the job is only half-done. If we do the right thing in a wrong way, we will not complete our mission. Yes, users will be able to solve their problems with our software, but randomly, in between system crashes.
A poorly written software is amazingly ambiguous. It claims it’s good and helpful, but a reality check reveals it’s an old wreck, just like a bad used car where engine oil likes to mingle with coolant; AC is running in winter only, the wipers – when there’s no rain only, and axle squeaks at 80 mph speed. Smart managers have done a great pre-sales job with this wreck, and sold it in an instant. AC off? Nevermind, it’s still OK in winter. Besides, 65 mph is the maximum speed limit so a squeaking axle is not a critical bug at all.
A year ago I tried OmmWriter, a minimalistic editor to process large amounts of text. Relaxing music. Distraction-free writing. I liked the app and started using it.
One day I wrote a large article in one go. About 5 pages. I’ve been saving it often and there was no back-up, of course. Next day I came to the office, opened my MacBook and OmmWriter hang joyfully. I killed the process, restarted OmmWriter and opened my document. It was empty. Have you ever had those moments, a flashing insight that you forgot to unplug the iron? So I got chills running down my spine, felt my adrenaline raised and faced the fatal destiny. My hands shaking, I found the file in Finder and checked its size — 0 bytes it was. That was the last day I used OmmWriter. The app that loses my data loses my trust. Recently I purchased Byword, a very similar app. It works faultlessly, and I’m happy with it.
Syncing badly written software to a new reality is very hard, with all the costs and time involved.
We wrote TargetProcess quickly, back in old times. As we worked after-hours and on week-ends, our main goal was to test the idea, its viability. There were few unit tests. The architecture was quite lame. О/R framework turned out to be a very bad choice. We didn’t think of scalability and performance at all. But we did what we wanted: the product concept has proven itself viable.
Eventually we had to re-write everything from scratch and it took 8 months of work full-time. That’s how TargetProcess 2.0 emerged. Was it good, or was it bad? On the one hand, we appeared to have lost 8 months. On the other hand, we tested the concept and made sure it works.
Speed
We’re smoothly proceeding to the third problem — speed.
Software development fails to catch up with users. As projects are getting more complex, the speed is dropping. The speed factor gets ever so important for any company.
If we create software fast, we get a chance to try various options, to tune in to market changes and to find the right way faster than others. With slow work, there’s no second chance.
There’s a legendary vaporware in the industry: Duke Nukem Forever.
They started it in 1996, and delivered the long-awaited sequel in 2011. After several missed deadlines the development team held back on public estimates and just kept saying the game would be released when done. Regretfully, Duke Nukem hasn’t lived up to the expectations of gamers. Gamespot rated it at 3.5. The game appeared boring, unoriginal and out-dated.
A team should be able to release new products in several months, in one year at most.
“A slow sort of country!” said the Queen. “Now, here, you see, it takes all the running you can do, to keep in the same place. If you want to get somewhere else, you must run at least twice as fast as that!”
— Black Queen, Alice in Wonderland
Actually, we need to run fast just to keep up with others. If we want to be the first, we need to run faster, way faster.
Any software development process should zoom in on those 3 goals, and here’s an appropriate headline:
Do the right things right and fast
If we do the right product right, but slowly, we might miss out on market opportunities. What if we run late and there’s a reality shift, and no one will need our product anymore?
If we do the right product fast, but in a wrong way, this means putting off problems till later. There will be only 2 options: stitch fixes over fixes and bury the product in 10 years, or re-write it from scratch, which is costly.
If we do the wrong product right and fast, hardly anyone will need it, unless we come to understand what’s wrong and make fast changes.
Ideally, we should get to the center, where the 3 circles overlap.
The future of agile development
Check out the short-term evolution chart for agile software development processes. At the end of the day, it all gets down to 2 main trends: speed and scale.
We have to learn to do right things right and fast in any scale.
That’s the basics.
Why I highlighted speed as the core entity? My definition of speed here implies thetimeframe to solve customer problems. What if we don’t do the right thing? We’d have to re-do or to give up completely. What if we do the right thing in a wrong way? We’d have to re-do to move forward. Overhauls and fixes reduce speed dramatically. That’s why doing right things right brings about faster speed in the long run.
So let’s start with speed. Speed is derived from 2 variables: expertise and fast feedback.
Feedback
Why is feedback important? Here is a nice story:
Dr. Anders Ericsson, a Swedish psychologist, has been doing numerous researches on learning and expertise. In particular, he has noted that most doctors actually perform worse the longer they are out of medical school. Surgeons, however, are an exception. That’s because they are constantly exposed to two key elements of deliberate practice: immediate feedback and specific goal-setting.
The same is not true for, say, a mammographer. When a doctor reads a mammogram, she doesn’t know for certain if there is breast cancer or not. She will be able to know only weeks later, from a biopsy, or years later, when no cancer develops. Without meaningful feedback, a doctor’s ability actually deteriorates over time. Ericsson suggests a new mode of training. “Imagine a situation where a doctor could diagnose mammograms from old cases and immediately get feedback of the correct diagnosis for each case,” he says. “Working in such a learning environment, a doctor might see more different cancers in one day than in a couple of years of normal practice.”
Feedback gives us a solid knowledge of 2 concepts: whether we’ve done the right thing and whether we’ve done it right. We want to know this as early as possible. Ideally, right after it’s done. So how do we know?
Solutions sketching
Sketches are a powerful tool for requirements management. It’s not only about drawings. Sketch is a quick way to emulate the real system. Sometimes it’s enough to hand-draw a sketch. Sometimes a sophisticated model should be designed.
In the early 80s IBM decided they should come up with a voice interface. Zillions of users dreamed about giving commands to OS, not mouse clicks. They were about to kick out with the project, as they decided to run a safety check: what if users don’t like it? The sketch was basic. A real user was suggested to give voice commands to computer. The user would go enthusiastically, as someone in another room listened and fulfilled the commands. The user watched processes running and commands accomplished on his screen. Brilliant! Eventually, most users didn’t like to interface with computer like that, as this was slow and ineffective. Keyboard and mouse won, they cancelled the project. One can imagine the costs of voice recognition algorithms, embedding voice commands to the OS etc. Just one simple sketch saved tons of money and helped take the right decision.
Sketching skills are vital for software development professionals, and we often lack these skills. How we develop now? Requirements stacked to a large backlog, important user stories picked out, and off we go drawing the UI and writing the code. We’d do the prototype (at best) which will evolve into the real system (at worst). This is not at all enough. First, the team should make sure they’re doing the right thing. Someone might say, this task is for business, not for development. But business has a very vague idea about software development and UI prototyping. Business alone is quite helpless here.
Jumping in and immediately starting to build the product, even if it does get completed and ship, is almost guaranteed to produce a mediocre product in which there is little innovation or market differentiation. When you have only one kick at the can, the behaviour of the entire team and process is as predictable as it will be pedestrian.
— Bill Buxton in “Sketching the User Experience”
There’re virtually no tools for rapid prototyping of modern web applications at the moment. With lame tools, it takes weeks to create prototypes of average complexity. It’s waaay long. New tools that will allow to create interactive prototypes in a couple of days should appear in next coming years. With such a dramatic cut down on time, there will be a faster feedback, as several alternative solutions are considered. THIS will make us sure we’re doing the right thing.
Continuous delivery — an utterly bold idea — is another way to speed up feedback.
Continuous Delivery
Delivery with every single commit reduces feedback loop cycle to a minimum.
This does not make any big difference for end users. But for the team, such extreme conditions create a drastically different development environment, making them improve the quality of code a lot. The company should really get set to stand up to continuous delivery.
Continuous Delivery brings along almost full testing automation: unit tests, functional tests, performance tests, acceptance tests. Delivery should be automated as well. Such an extensive automation requires solid team efforts as well as total process overhaul, from requirements management to marketing. This is about scalability, which makes it even more complicated.
Expertise
Let’s move on to more human-centric things. Software is developed by people. Attempts to put software to a line production had no success, which is not surprising at all. Software development industry blooms in tender age, and the first signs of puberty are just about to protrude.
So far Everything is about people. A team of high-end professionals will make cool software, no matter how lame the process. A bunch of dummies have a vague chance to do the same only at times of total solar eclipse, even if they extreme program like hell.
It’s people who possess expertise and problem solving skills, and that’s where they should develop and improve.
Domain knowledge
Domain knowledge helps us understand if we’re doing the right thing. If you know how a car is set up, you’d hardly agree to add a 5th wheel to its center. If you see a car for the first time in your life, adding a 5th wheel would not rush your heart headlong, and you’d do it ruthlessly.
Domain knowledge is about the most epic fail for software developers. Well, one way or another they look into it, but hardly anyone takes focused time to study their domain. As I was working on software for insurance business, it occurred to me only in a year that I could actually read up on how insurance business is run, how Carrier and Account differ in real life, not only in classes. Well, this never occurred to my co-workers.
Usually it doesn’t matter for a developer, what is this entity doing outside the code. What matters is a set of business rules. Developers want to get consistent requirements, and to implement them. They want to design a nice architecture and write the clean code.
That’s where company should take the lead. They should arrange for domain knowledge training. In our company, we send people to conferences and strongly recommend reading books on agile project management, but this is not enough.
Everyone should clearly understand what are we doing, why are we doing this, and why is it important at this very moment.
If at least several people in a team have decent domain knowledge, they can expertly discuss business problems and come up with solutions. Such a team wins a lot more trust. They are much less likely to do the wrong thing, hence the increase in speed.
Craftsmanship
A Craftsmanship concept is closer to a programmer’s heart. This concept is about improving the quality of solutions and honing professional skills. Craftsmanship almost entirely focuses on doing things right.
The only way to go fast is to go right.
— Uncle Bob Martin
A thoughtfully designed architecture, excellent test coverage, keeping it as simple as possible – all that makes for a higher quality.
There have been some attempts to apply athletic and martial arts training techniques to software development, e.g. Coding Dojo and Coding Kata. Well, it’s a big question if these can be used to train programmers effectively. But some sort of training is definitely needed. If you’re working on the production code all the time, you solve same problems same way and get stuck in your professional development. Looks like we really need some workshops focused on mastering certain skills, not on production. The most effective technique can be solving one and the same problem in different ways, using various technologies. Constant improvements to architecture also work well. You need to always question the current setup, try to improve the solution, make it simpler, clearer, and more flexible.
It’s obvious, that the more complex a system, the more challenging it gets for people. Nowadays it’s not enough to know just C language, data structures and algorithms. You need to have a good command of OOP, patterns, libraries, a bunch of related technologies and sciences. More and more knowledge is required with each year. However, the level of abstraction hasn’t changed dramatically, whereas systems got a lot more complex. The architecture of Facebook and Twitter is very complicated; I don’t think any developer working for them has a consistent idea of how Facebook or Twitter functions as a whole. So,
developers must set themselves to lifelong learning.
Problem solving
To me, problem solving is the most fundamental skill of all. With this skill, you can fill both your own gaps and the gaps of development process in your company.
Let’s look at retrospective meetings. People get together, raise some issues which frequently lie on the surface, and offer surface solutions likewise. Root-cause issues are diagnosed and resolved very rarely. Brilliant, off-the-beaten-path solutions come up too seldom. Almost no problem identification tools are used at retrospectives. Systems thinking, TRIZ, brainstorming techniques — most teams and developers ignore these, as they have no such skills and knowledge. No one teaches them TRIZ. No one teaches them brainstorming. No one teaches them lateral thinking.
Problem solving skills are totally left out of formal education, be it in high school, or in college, or in university. That’s just incredible! Not a single lecture makes mention of these skills, as vital as they are for any professional activities!
If you ask developers, they like to study anything that has to do with strategy and tactics of programming. All the rest is of virtually no interest for most developers. They use algorithms to solve their tasks, missing out on heuristic, creative ways. If you want to improve a process, it’s an elaborate creative task; you can’t solve it with algorithms. Designing architecture also calls for creative thinking.
Experienced coaches try to introduce new practices to retrospectives, quite often with no success. It’s either they are not really experts in problem solving or they fail to pass their knowledge on to the team.
Agile is putting its bet on self-organization in teams. Team should identify and resolve their issues effectively, or they will make the same mistakes over again. However, no agile methodology comes equipped with problem solving tools.
Curiously enough, they use these tools in UX and graphical design; it’s pretty straight forward, just another tool set. Developers lack this quite often.
Great developers think creatively and originally. They don’t have to compose poems, but comprehensive writing skills are a must. Train your right brain hemisphere, ladies and gentlemen!
Scale
Scale is a major crush test for agile processes. Scalability problems are mostly related to company as a whole and to distributed teams.
Company as a whole
Agile works quite well on team level. A standalone team can really improve their productivity as they transition to Scrum + XP or Kanban. But if you try to scale it up across the entire company, this will be very often followed by grave resistance and misunderstanding.
In the first place, agile clashes with corporate culture. Business world eyes fundamental agile principles suspiciously. Transparency? Trust? Humility? Hmmm… Political games, incompetency, bragging are common in most companies. For them, agile is like a life-threatening virus.
Agile tends to break the homeostasis of old-school organizations and brings all the underwater currents to the surface, revealing hidden flaws and some substances that normally don’t drown.
Even if a company is doing quite well with transparency and trust, there’s a handful of other issues. HR department should get busy not only with skills screening, but with culturally fit new hires. Marketing people should decide what to do with frequent releases. Design team should find a way to integrate with development team. Top management should facilitate interaction between multiple agile teams in their organization. It all gets very complicated, and many questions still remain unanswered. Large companies are just starting out with agile adoption.
UX and development teams tend to merge very well lately. Progress is good here.
But when it goes about multiple teams interacting with each other, things are much worse. Scrum of Scrum has proven itself completely inconsistent. There’re no clear guidelines, everyone is bumping the same walls and re-inventing the same wheels.
Distributed teams
Globalization has been an all-time issue for agile, with distributed teams powered by WWW evolving. Some companies run offices in dozen countries, syncing the work of hundreds people. Since agile originally implied co-location, it’s having hard time with distributed development. Seamless communication and reduced feedback cycle are hardly possible if product owner wakes up at 9 am, and developers come to work at 4 pm.
There’s no easy solution to this problem. First off, this is about increased density of communication channels. Permanent video connection works best of all. A freshly unshaven exterior will honestly tell how’s it going and you will always know if it’s a good or bad time to ask questions.
Software for remote pair programming, brainstorming and testing is coming up as well. Tools such as TargetProcess are indispensable for distributed teams.
Summary
Software development industry should hit the center, it’s all about that. Learn to do right things fast. I’m sure it’s possible, and here’s what needs to be done:
- study problem-solving techniques, such as TRIZ, lateral thinking and
systems thinking - reduce feedback cycle by any means
- study domain
- learn to scale agile-mindset to the whole company and to distributed teams
0 comentaris a «The future of agile software development, by Michael Dubakov»