Revisiting my long dormant toy that uses analysis of song lyrics and Flickr or Google Images to do contextual visualizations. I have implemented a simple ti.idf frequency counter and am getting decent results but wanting for more. In this web 2.0 world maybe I should be looking for services to do this?
The struggle continues. Faced with being increasingly outnumbered by PMI habituated and prediction minded project managers we've hired lately I'm returning to this question of why we iterate.
People are using the word. But too often not groking the motives.
We have things like 9 month long efforts to build an internally deployed version of a currently external system being called an iteration. To me that's a whole other project or a "phase" but certainly not an iteration.
We have PM candidates that my peers are gushing about. I get asked to interview. I ask about things like how they think requirements work should progress in an iterative methodology. I get back ideas about getting tomes of requirements first and then planning iterations.
We have months long breadth first swipes through the requirements for a whole system being called an iteration along side plans for a second iteration to do "design".
We have business users insisting that it's pointless to test an incomplete system. "I dont want to see it until it's done".
We have directors asking for MS project plans in the first weeks of a project detailing how many iterations it will take and what will be done in each.
Seems like we have a future destined for many more failed projects and missed "estimates" before we start to figure this out.
I have dreams about some speech, presentation or as of yet un-conceived prop that will suddenly bring the idea home to folks. But in waking life am running thin on optimism that we can get this concept across and make it work.
So, maybe one more time I'll try to collect some nuggets of persuasion and contemporary thought about why emergent software planning is more healthy than predictive planning.
The neglected practice of iteration Jeff Patton sends a reminder that software developers who neglect the practice of iteration will get caught either delivering poor quality software or delaying schedules in order to make time to iterate. We kick ourselves, or others, for not "getting [software] right up front" when we all know that the hardest part of software development is figuring out what to build. But there's hope, and it comes in the form of prototypes and frequent iterations
Good quip about devs not being the constraint From David Anderson's blog article Why We Lost Focus on Development Practices In my Zen of Agile Management class, I teach participants about constraints. I then ask them, in a collaborative exercise, to speculate about the bottleneck in their organization and discuss how they would prove it and what they would do about it. In almost 3 years of teaching this class, over 4 continents, and around 12 occassions with groups ranging from 16 people to 250 people (at Javapolis in 2007), I have concluded that developers are the constraining factor on project and team performance less than 10% of the time. In some groups it is as low as 3%.
My belief around this is quite simple. Basic agile practices focusing on quality including collaborative working such as pair programming, and a strong focus on unit testing and early defect detection with continuous integration and test automation, have greatly improved software development to the point where initial software quality (i.e. bug insertion rates) is not the constraining factor on team performance. With immature teams, with sloppy development practices and poor initial quality (i.e. high defect insertion rates) development is the constraint. Agile has successfully fixed this!
So we can declare victory!
In this sense, the crowd who argue for a continued focus on better development practices are fighting the last war.
The rest of the community moved to other areas - most notably project management and business analysis / value-based software engineering. The community tends to get sucked to where the constraining problems are occurring. This is the natural and correct behavior. And it shows that many in the community interpret better ways of developing software in the broad sense rather than the narrow programming-centric sense.
Metaphors & Quips OOPA - Observe, Orient, Plan, Act (or maybe P really should be Decide) Shoot, Move, Communicate Fail early and often.
Sometimes it seems my entire career as a software professional has been framed in this long suffering debate about development methodologies. I think I subscribed to the big planning, big requirements, big design upfront concept for a few early years. Eventually I noticed the soul crushing death marches, huge overruns, and scandalous politics so common to PMI/waterfall projects and nearly went back to law school.
It was my good fortune to work on several dot com startups and the works of Booch, Rumbaugh on UP and later Ambler, Beck and Fowler on agile approaches that convinced me that there was indeed a more profitable and humane way to produce software.
It worked well when the focus was on innovation and entrepreneurial ventures. When there was a mix of people focused and actually engaged in delivering product, not satisfying prediction. Once the companies dragged in a few too many investors and the obligatory sr. management that things started to come off the rails and the focus became, to my mind, too much focused on prediction, rigorous estimation, and unrealistic expectations.
Again I find myself in an organization struggling with how to manage the software process. In this case we are playing from a technical deficit and have a service orientation rather than a product orientation. We try to say yes to everything and often over commit and under deliver. We have the typical mix of middle management that has, to my mind, spent dangerously little time involved in the craft of software development and tends to see software as a manufacturing problem, not a creative one.
This debate will heat up in the coming months and I'm not sure I have the strength to fight it again. Scott Ambler once told me "You either change your organization or you change your organization."
This post is both a place for me to dmup some links and fodder for the coming discussions and a plea to anyone who's listening to chime in on which end of Ambler's paradox I should take up.
[]
Iterative vs. waterfall software development: Why don't companies get it? [Computerworld]
[]
The problem with waterfall methodologies is that they don't work all that well. Trying to create complete, perfect system specifications before you start any coding simply ensures you've wasted a bunch of effort, because once developers start coding, design flaws become evident and require that you revisit earlier stages.
Except that you can't, because that stage is complete and there's no budget for going back.
This isn't something that can be fixed by getting better at design and specification, because the more you try to design everything before you start coding anything, the longer the delay between requirements and production. That delay means some of the design "flaws" aren't flaws at all - they're changes that are the result of the future not looking exactly like the past. [Infoworld: What "waterfall" means ]
[]
Perhaps not authoritative but a good bullet list of when iterative makes sense and a (shorter) list of when waterfall makes sense: [http://www.joisinc.com]
[]
Good, if cheesy, flash video contrasting Agile vs. Waterfall: [A Tale Of Two Teams]
[]
Good metaphors for incremental vs. iterative and some witty agilist thoughts: [agileproductdesign]
[]
Fabulous origins of "The New Methodology" by Fowler. Plainly lays out the motivations for agile & iterative.
[]
Ambler's rants about agile planning. Explains the problem with WBS, GANTT, PMI and MS Project.
Explains the schedule by value vs. task here: Agilists typically schedule the development of requirements (user stories, features, use cases, ...) into iterations as the line items, not tasks such as design, test, .... For example, the line items of iteration 5 might be "Implement Search for Books", "Implement Search for DVDs", "Implement Process Mastercard Payment", and "Implement Capture Different Billing and Shipping Addresses" for an online e-commerce system. These line items would stretch the length of the iteration, I would trust the team to schedule the actual work appropriately and wouldn't go into any more detail in the project schedule.
[]
In the end, laughter might be necessary: waterfall2006
Reuter's new Calais API/service provides ontology generation for your docs. They tout it as a tool for the vaulted semantic web. This might be a worthwhile alternative to the simple frequency analysis I've been doing on my iTunes / Flickr mashup tool which presents a slide show of pertinent images from the lyrics of the now playing song.
