Lessons from Agile Development with an Off-shore Team

Lessons from Agile Development with an Off-shore Team by Andrew Schultz (17/10/12)

My main role at my company is to deliver subject-matter expertise on Dynamics CRM (which includes solution design for companies building products on top of CRM) and to manage the delivery of the resulting product by our development team in Kiev, Ukraine. I enjoy both of these roles, but the actual project management piece, which is a part of the “managing delivery” role, I could honestly do without.

So when I saw how our team worked as I started my employment here, I was concerned. As might be expected from an off-shored professional services organization, the approach to work was a bit scattered at first. Projects would come in, the least-busy people were assigned to those projects, we tried to maximize the time each person spent working on billable work, etc. This meant that the team never had time to gel (as there was no consistency in the makeup of the teams from project to project). It also meant that each person was typically working on several projects at once. Projects were difficult to close out, and I don’t think it was a great environment for developers who want a more planned approach to their careers, the competencies they build, and the type of work they do.

So I was really happy when we had 2 projects start that were both multi-month development projects. The larger one has been going for 8.5 months and has 6 developers/testers working on it. Most of that team has been very consistent throughout the project. So at the beginning, I decided that this would be the perfect opportunity to introduce agile to the team. I had several particular goals:

1. Avoid gantt charts [shudder]. I hate them. And to try to manage an 8-month project that started with a 230-page technical spec with waterfall and a gantt chart and a critical path … I’d die first. The technical spec was written by a 3rd-party architect, so this created a sticky business triangle, but outside of that, we tried to use this architect as our on-site customer.
2. Overcome the wall I felt between myself and the devs. It seemed like they were afraid to speak their minds to me, to contribute their ideas to our projects, to tell me when they had a better idea than I did … I assumed this was cultural. I know they didn’t agree with me all the time, because previously, from time to time, they had followed my way of doing things until they kind of exploded with frustration at doing it my way when they had a better idea. I wanted a more open dev environment where we could work together as equal contributors, even if I am technically above them in the chain of reporting.
3. Instill agile skills in the team. This was for their own benefit as much as it was for mine. I consider agile experience a plus for any developer’s resume. This is especially true if he has truly experienced agile (not just adopted a few mechanisms like sprints and scrums).

So … how did it go? The culture gradually changed. I was gratified to see the technical lead on our team start to take over my role as the person who led the planning meetings, started writing the stories, etc. His confidence in this role grew as the project went on. We faced some challenges, such as going way over budget, but as I investigated the causes (and asked the direct question multiple times about whether this was a result of using the unfamiliar agile approach), we honestly determined that the project had just been poorly estimated at the beginning. Estimated at the beginning, you might ask? Yes … we couldn’t do everything the right way according to agile. I’ll discuss this below.

What Went Well

1. We finally got the art of agile estimation down. We started by looking directly at the spec and trying to group the sections from there into our iterations. This worked marginally well until the sections got too big to do in one iteration. We started re-writing sections from the spec as user stories that the developers could understand more readily and give the gut-feeling estimates we needed to generate our story-points. As the weeks went on, the estimates began to coalesce. At the beginning, there would generally be one cluster of estimates with 2-4 days difference between the highest and the lowest, and then one outlier from the “pessimistic guy” that was usually 5 days higher than everyone else. We averaged the estimates together to get our story points. While we were fairly consistent in the number of story points we accomplished from iteration to iteration, it could have been better. At the end, the estimates were much closer together, usually all within 1 to 1.5 days of each other.
2. Planning improved. The major challenge with this project was the complexity of the technical spec. Did I say it was 230 pages long? And completely unreadable by normal humans. After starting with 2-week iterations and really struggling to consistently deliver the planned functionality due to problems that arose mid-iteration, we pared the iterations down to 1-week. We had our demo and planning every Tuesday, so the developers really had only 4 development days plus a few hours in an iteration. This dramatically improved our consistency. Keeping the planning cycles short left much less room for us to get bogged down in complexity we didn’t understand when we planned.
3. Code quality was great. We re-factored constantly, we tested the functionality for each iteration diligently so we could release it at the end of the iteration (even though the customer never took the product out of the demo into production). I honestly think the code quality in this project was the best I’ve seen from our team.
4. Cooperation was awesome. Since this project had most of our team working on it, I feel like it’s actually transformed our team. Our leader (who was appointed the technical development lead shortly after the project started) has been able to develop a great collaborative atmosphere with the other team members, and that’s not something that would have happened had they all just been working on 3 projects at once like they were before. They spoke to each other, walked to each-others workstations, and generally had a much higher level of interaction than on other projects.

What didn’t go well

There’s only so much you can do in your first attempt to implement agile. The core of agile is the cooperation between business people and developers, and this is where we fell short, unfortunately. So, while I’m hesitant to say we were fully practicing agile with these shortcomings, I think we made great strides, and we’ll learn from these things moving forward.

1. Collaborative workspace – while our developers have this among themselves, there was simply no way for us to put our “onsite customer” (the 3rd-party architect who wrote the spec) in the same room as the developers in Kiev for more than 3 days at the beginning of the project. At first, he was present via Skype at our daily standup meetings. Then, he was only present at our demos. He was always available for questions, and was very helpful, but it wasn’t the same as having him in the room. When could you ever have the customer in the room all the time for a professional services project, especially an off-shored one? I would say never. Unfortunately, this is a challenge that may not have a perfect solution. The best thing to do is to try to keep the customer as involved as possible, especially at the points where input is needed and decisions are made. Demand his availability, and, if possible, insist that he not require questions to be scheduled.
2. Full implementation of my favorite concepts- there were some XP concepts that I just couldn’t get the team to try – including test-driven development and pair programming. I’ve spoken to developers who have successfully used these practices and will attest to their effectiveness, but my developers in Kiev were not ready for them. Well, we change gradually, right? Maybe sometime in the near future. But it’s important to have the right financial configuration on a project to introduce radically different development practices, and this probably wasn’t it.
3. Welcoming change – this is another pillar of agile, and unfortunately, we fell short here. Why? We were working on a fixed-fee project. At first this was good for our adoption of agile, because instead of being accountable for every hour we spent, we could focus more on the result. However, as it became apparent the estimates we had used to price the project were way off, the financial burdens of the project required us (me, mostly) to start really challenging any input our onsite customer gave us that changed the scope of the project. Enter the business triangle I mentioned earlier. We had to document any changes as change requests to the paying customer, who then had to go back to the onsite customer (the third-party architect), who then came back to us, and the politics would commence.

The third item here is especially telling. One of the biggest challenges I see in a professional services team doing agile is the project’s financial arrangement. A fixed-fee project seemed like a good arrangement at first, and it probably would have been somewhat OK had we done a better job up front estimating. But the best arrangement would be for the customer to understand agile and the value it can create in his own go to market strategy. He should understand the advantage of not spending his full development budget before his product ever lands at an end-customer. There is a huge benefit financially and strategically in understanding the core functionality an end-customer needs in order to give your customer money (minimum releasable functionality). If your customer understands this principle, he can pay you to develop 10% of his product’s planned functionality and then have paying end-customers funding all or part of the other 90%. The alternative is paying for all 100% himself, and then realizing that 80% of it is irrelevant, and that he’s missing another 60% of things his end-customers want (I’m making all of these numbers up, but you get the point).

If everything were perfect, I would see a fixed-seat project, where the customer simply pays for x number of full-time developers and testers, and then commits the resources from his own company to work with that dev team daily. There’s no scope, no milestones, and no fixed fee. The customer knows what he’s going to pay for development each week or month, but he doesn’t know what the product is going to look like. Most people will be uncomfortable with this. However, if they’re smart, they’ll gladly accept that uncertainty for the increased agility that will allow them to get early feedback from customers and develop the solution that makes money the fastest.