Archive for the ‘Process & Methodology’ Category
This post aims to answer some of the questions we frequently get from executives on what we do, the business and process behind information technology in general and software in particular. First a preamble. I don’t expect an executive to understand the intricacies and details of what we do as software engineers and consultants. My job is to understand what an executive requires, what “pain points” might exist in the operation of the business, what opportunities might lie ahead and to devise and implement solutions through information technology. My job is to understand and communicate the nature of the solution, scope it, price it, build it and integrate it. Our primary expertise is software, more specifically custom software.
What is custom software?
You can buy “off-the-shelf” software. Software of this type is often, quite literally, available on a shelf at a computer or electronics store. Other times it is downloaded or procured from a commercial or open source vendor. Most people are familiar with this type of software because of the ubiquitous availability of some well known off-the-shelf software. If you have used Microsoft Office, you have used off-the-shelf software. Custom software is purpose built, or rather purpose “assembled” from readily available and custom-built libraries. You don’t buy or download it ready-made.
Is custom software built from scratch?
Not at all. Today’s application development is more accurately described as application “assembly”. Architects and developers combine readily available libraries and components to meet the business and functional requirements of the system and the needs of the organization. The widespread availability of these (often open-source) components has created a new breed of software development, one that relies on rapid prototyping and frequent iterations. Good developers don’t reinvent the wheel. They use tried and true readily available components, libraries and best practices. They don’t make, they assemble.
Why do I need custom software, can’t I customize off-the-shelf software?
It does depend on the software but in the vast majority of cases, you can, to some degree. Appearances can be deceiving however. Making changes to a large one-size-fits-all software application or platform can often be more expensive than purposefully assembling an application from loose components. The economics of “buy vs build” hinge on the nature of the application. This is why neither should be a foregone conclusion. Always start with your business and functional requirements, initially ignoring what you think is doable, perceived costs and complexity.
Brand “X” off-the-shelf software does 80% of what I need. How expensive will it be to build the remaining 20%?
As I said above, I can’t quite answer this question for each and every situation without further analysis. But I can say this with absolute certainty: a lot longer than you could ever imagine and often a lot longer than the vendor is willing to admit. Nowhere does the 80/20 rule apply more so than in systems. You will meet 80% of your requirements in 20% of the time and budget. Commercial vendors know this and are quick to sell you those features that come to mind. Don’t assume what you didn’t see is easy to get; it isn’t. The remainder will be expensive and difficult because by design, off-the-shelf software is meant to fit most organizations’ needs, not yours specifically. Custom built software has a more predictable and linear complexity curve. While not all features are equal in complexity and scope, building custom software has few or no limitations. Any experienced professional can accurately scope and estimate the time and costs involved in building the features needed.
How do I kick-start a software project?
Every software project needs a mandate. Software exists to serve a business and functional purpose. Elliciting requirements is a job fit for professionals. Any good software consulting organization will put forth experienced individuals in this area. They will meet your stakeholders, will interview current and future users, will seek to understand the current business and functional processes the new piece of software is meant to support, alleviate or replace. From this process, a list of mandatory business and functional requirements emerge. Be specific and get everything in writing. Upon delivery, the software will be passed by user acceptance. User acceptance ensures that the system meets all requirements stated and is fit for deployment.
How do I measure success on a software project?
Sofware should be easy to use. What goes into usability is open for debate but the outcome isn’t. Are your users productive? Do they (the people actually using the software, be it your employees or clients) find it easy to use? Have previously difficult and time consuming tasks become easier and faster? Is the software intuitive? Does it lend itself to experienced users and novices alike (a difficult balance by the way)?. Usability is important. Make sure you work with people who read, think and speak usability. There are other facets, but this one cannot be overlooked.
Software needs to be fast. Give the most patient person in the world a web browser and make him wait 4 seconds and you have a frustrated irate user. Rightly so. Software needs to be fast. People are used to thinking fast. Customers demand highly responsive interactions or they move on. Fast requires proper software engineering and infrastructure. Don’t assume any piece of software can scale. That is simply not true. Principles of scalability must be embedded in the application itself. We listen to every word Google, You Tube and Facebook software engineers have to say because scalability is very much a science that relies on software patterns, design and infrastructure decisions. You may not be as large as Google but scaling down is easier than scaling up. In this regards, there is absolutely no substitute for experience. Don’t hire an organization that hasn’t built something comparable in size or scope. They will learn on the job, they won’t meet your expectations and you will miss your target. Software engineers are worth every penny you pay them. Expensive? Just adopt the agile methodology and ensure most of your dollars are going towards the end product not superfluous management (not that management is superfluous but in agile development, extra process can in fact be detrimental).
Software must be easy to change. If I had to pick one symptom of poorly engineered software I would say, without a doubt, a pattern of “I asked how long it would take to make small change X and they said it would be Y weeks”. The truth is not all software is created equal. Good software is what we call “declarative”. It can be changed easily because only key functions are “hard” coded, the interactions between code modules and functions to actually create processes are “soft” coded, typically in XML or configuration files. If your vendor consistently tells you it will take days and weeks to do simple things, they may in fact be honest but (regrettably) incompetent. Talk to a vendor’s existing or previous clients. Was the software written on time? Did it perform? Were changes easily accommodated? If any of these answers is negative, move on.
Can my IT department write software?
Some can. However most IT departments are usually barely keeping up with the ongoing needs of the business. Freeing up resources to write and integrate complex software is often prohibitive. Another angle is while some IT departments have in house talent able to write software, writing enterprise software is complex and very much a profession in itself. Technical skills and development methodologies are taxing and time consuming to learn and master. A little knowledge can indeed be a dangerous thing. If a system is mission critical and/or will affect your bottom line, leave it to people who do nothing but software development.
How do I choose a vendor?
In operational and logistical areas, the size of the vendor is often proportional to the size of the project. Software is different however. Software scales, not according to the number of people on the team but according to the experience of the engineers who architected it. I once worked for a large consulting organization. The pitch to your new clients was often the same: at the first meeting, they’d bring out the “stars”, the experts. The client was wowed. Clearly this is money well spent they felt. Unfortunately, the contract would be signed, the stars would disappear, never to be seen again, and the client would be stuck with a team of recently hired “B” developers. Projects at these large consulting houses go notoriously wrong and over budget. Not to say that there isn’t a place for them. But when working with software, get to know who you are working with. And keep in mind that small teams do great things.
What vendor would you recommend?
I thought you’d never ask! We at Betterdot Systems practice what we preach. We’re a small company of ultra-motivated highly-experienced software professionals who do great things. Speaking with us is not cheap. It’s free. We want to understand your business and your needs before commitments are made or sought. There are other vendors out there. In fact if we feel that your requirements don’t fit our expertise and skill set, we’ll happily recommend a few. Speak with your peers and ask them about their experience with software vendors. And as I mentioned above, ask to speak with a vendor’s clients. A good vendor has happy clients. Happy clients are willing to talk.
Success in the web 2.0 sphere makes for an easy sales pitch. You need to spend less money. You need to get a product out faster. And the catch is…? None, but spending less is not as easy as it seems. You need the right framework, the right partners, the right methodology. Here are some principles Betterdot Systems abides by in everything we do…
1) Build great teams. As Marcus Daniels once told me, “You are better off starting a business with an A team and a B plan than a B team with an A plan”. Why? Because the A team will eventually gravitate to the A plan anyways. The A team will innovate, will find the way, the shortest path. Hire great people. Work with great partners. Build your A team.
2) Build small teams. Small teams do great things. The mighty Google is said to have no team larger than seven. Meanwhile Microsoft used armies of developers to produce the much vilified Vista operating system. Build small teams. Assign them a clear mandate. Give them responsibilities. Let them shine.
3) Be agile. Don’t fight change, embrace it. Focus on the user experience. Try rapid prototyping and commit to frequent releases. Be nimble, be agile.
The principles above allow us to do more, do it faster and do it for less. Sales pitches are so web 1.0… 😉
My blog post on The Case for Nearshoring to Canada seems to have hit a nerve, at least with unbiased Canadian readers ;). Unfortunately the article was cross-posted to LinkedIn so some interesting comments were posted there instead. As a follow-up to the original article, here is a compilation of the most interesting comments, from various sources. Thanks for your contribution.
Gary Toste writes: “For offshoring to be successful, you truly need to commit to it and integrate it into your processes and commit to going there regularly. So yes there are additional costs outside of just what you initially see to make it successful. As soon as you outsource anything, be it down the street, or half way around the world, there are complexities that need to be managed or things can fall apart. The further away though, the more issues that can happen. Timezone is by far the biggest factor in my opinion. ”
Sean Abel writes: “In my experience with offshoring (about 15 small to medium custom software projects with total combined budget of $1 million USD), the costs savings were negligible. In order to get the correct product on time with an acceptable level of quality, there is more time spent on the front end specifying (never a bad thing) and on the backend in quality assurance than is my experience with in-house talent. The one positive that I noticed was that the paper trail was more comprehensive than a typical in-house project. The biggest negative was the very few hours of in-office overlap between my North American assets and my overseas developers. This required resources on both ends to work some odd hours when resolving complicated issues. All in all I find that with my typical engagement I can deliver a superior product in less time for the same cost with a local development team. Manufacturing is an entirely different animal and not my core expertise.”
Marcio Ferrini writes: “…in Brazil the time zones, work ethics and business values are close as well. I think our main problem is related to instability of our currency. The exchanges rates vary so much every day and even we having contracts in dollars, we end up having expenses in Reais which has some time to be adjust.”
Peter Giblet writes: “In addition to your comments I would also say there are other advantages to leveraging Canadian skills in development projects. In addition to the reasons pointed out corporations engaging R&D resources do have significant tax advantages for new developments, provided the terms and conditions of an employees contract are correctly worded.”
Alex Boudarov writes: “There is no cost saving in outsourcing whatsoever. Indian devs cost between 20-60$/hr. Resources at lower price range overbill heavily, which brings the effective price point to North American standards. In my experience (I have seen how outsourcing in major financial institutions) it is more expensive then local development, especially considering lack of face to face communication, timezone difference and lack of control over the development process on their end.”
Steve Sykes writes: “When offshoring (and outsourcing, to a degree) got started as concepts, there tended to be a lack of a “total cost” view. Such a view would take into account all the great points made thus far here, and also things like vendor management – which requires visits to vendor facilities, etc. I’d like to see a “carbon impact” factor introduced to such analyses, as well as the productivity cost associated with an executive traveling to Russia, China or India. A week trip is usually stretched into 10 days or 2 weeks – and then there is the wind-down before the trip and wind-up afterwards. Whole lot easier (and less expensive) to manage a vendor in the same metro area (or at least, province). I would suggest that each such trip impairs between 1% and 5% of annual productivity. I’d also like to see all of the human costs sufficiently considered. “
Agile software development is a methodology that relies on small iterations and frequent releases, teamwork and collaboration. Agile development embraces change as the rule not the exception. It is generally nimble and adaptive. Agile typically favours face to face communication to documentation. Agile measures success according to working software (hence the importance of rapid prototyping), not abstract (from a software end-product standpoint) project milestones.
By putting the emphasis on the end-product, on collaboration and by embracing change, the agile methodology is perhaps the best software development methodology available today (for most projects). We use some form of agile development and project management on virtually all projects. One size does not fit all however. I sure hope the avionics of the plane I am flying on were not developed by a team who believes that paper specifications and formal documentation were unnecessary project overheads. Yet most projects, even the largest and most complex (perhaps better served by the waterfall method), benefit from adopting some agile principles.
My biggest concern with strict agile is the insistence on human interactions over documentation, as if one was a substitute for the other. Take requirements analysis for example (a topic recently covered in a LinkedIn Q&A). Gathering requirements is an iterative process, a series of passes. After each pass (a discussion, interview or meeting with stakeholders), requirements must be documented and compiled. Provisional conclusions and principles are drawn. At the next meeting, it is important to review and vet the requirements, assumptions and principles as understood and documented. This will typically elicit further qualifications and uncover other requirements and assumptions. The process requires multiple iterations, each advancing the requirements-gathering further towards finalization. The process goes on until all parties believe they have exhausted all qualifications and that the requirements, as documented, fully encapsulate the business, functional and technical needs of the particular feature or component being discussed. This does not mean the specifications are ever “final” or set in stone (one of the biggest arguments against the waterfall method. “finalizing” requirements assumes changes are the exception not the rule). It does however support the idea that documentation is an extension of the collaborative thought-process not a substitute for it. I believe without this important qualifier, the agile methodology cannot scale to large and complex projects where dependencies, complexities (business rules and complex processes) and risks abound.
In conclusion, I believe the agile methodology to be sound and effective, cost and otherwise. But documentation does not undermine the methodology, it makes it better.
I am often reminded of the old real-estate adage regarding the three factors that matter most in real estate: location, location, location. Likewise, while credit is often given to software development and integration efforts (and the teams/individuals behing them), every succesful application and every on-time deployment is rooted in three factors: analysis, analysis, analysis. Analysis determines who will use the system, what the system will do, what it won’t do, how it will integrate with other systems, other functional processes and other departments. Analysis uncovers invalid assumptions, unforeseen dependencies and potential risks and road blocks. As such analysis allows a precise functional and technical blueprint to be produced. This blueprint (technical specifications, typically a set of use cases, UML diagrams, class diagrams and database model) means the systems fits the business and functional requirements. It allows the system to be designed to meet the requirements of the business which translates into reduced development cycles, fewer change requests and fewer bugs.
In software development as in life, it’s good to have principles. Design principles are essential to discuss and document in the early days of a software development project. Principles support and protect the vision of the architecture; that which will allow it to be stable, flexible and/or scaleable. Without them (and this is particularly true in larger teams, new teams and teams with high turnover), it is easy to make day to day decisions that will weaken the application, introduce bugs or won’t scale. Design principles are guardian angels for software. Discuss them, review them often and go back to them when alternate implementations are being discussed.
Use cases are an effective way of capturing business and functional specifications. In fact, coupled with user interface mock-ups and wireframes, use cases are typically the key constituents of a business and functional specifications document. Use cases often expose dependencies and workflow paths that may not otherwise surface. For all their worth, use cases are time consuming to write. They tend to be bulky and can be repetitive. In fact, we have found there are diminishing returns with producing ever more use cases for a given system. The question is when is not enough and how much is too much? We generally produce detailed use cases around key subsystems such as authentication/authorization and wherever one application interfaces with another (such as when web services are used for example). In such critical or complex areas, there is value in investing and writing detailed use cases. Elsewhere, we have found that it is often more effective to focus on written documentation and mock-ups.