Managing Digital Initiatives: Case of ECM

The author provides a quick guide to Quality Assurance leads, Solution Architects or Project Managers on how they should solution-to-deliver, to ensure quality in Enterprise Content Management projects. The paper propagates that QA is all encompassing and an integral part of end-to-end project life-cycle.

Project objectives: 

Project Key Performance Indicators (KPIs) to be defined at start of project to measure success at project closure. These should be quantifiable/measurable.

• A project objective that is written as “Enhanced Look & Feel/ UI”: You should define clearly how Look & Feel/ UI can be enhanced? Benchmark against similar sites. Quantitatively define how the project can enhance customer good will (increase in number of visitors by 10%; increase stickiness/residence time on site by 20%)
• A project objective that is written as“Improved demand generation”: Can be re-written as – Increase of 20% in ‘Book a Test Drive’ (in case of a Car website); Increase of 30% in ‘Request a Quote’; 
• “Effective public communication” can be enhanced as: Compare Cars functionality to be implemented to help customers take informed purchase decision. Embed Special Offers in all E-Mail communication to customers.
• “Contextual collaboration”: Avoid gibberish. Keep objectives simple, understandable and measurable.
• “Seamless process integration”: Keep project objectives relevant. Use business parlance. Avoid technical language.
• “Mitigate immediate challenges to create, control, deliver, and maintain the rapidly growing and changing volume of information on company’s websites”: Avoid long descriptions in Project Objectives/ KPIs. It can seem incoherent and open to misinterpretation.

Assign a full-time Business Analyst

Project to assign full-time Business Analyst at Project site for entire duration of the project.

• Responsibilities include: Understand Scope of Work (Information Architecture document). Gather business requirements & documenting/updating business blueprint (To-Be document or Functional Specification document). Responsible for walking thru requirements to development team; and clarifying their queries. Be the conduit between Business team & development team. Lead all activities with Business team viz. Conference Room Pilot/ Business Acceptance Test/ User Acceptance Test and End User Training.
• Recommended profile for Business Analyst:Overall 10-15 years of work experience. At least 5-7 years in similar role. Excellent communication skills. Excellent functional knowledge of ECM Product. At least ECM implementation experience. Relevant experience:Website deployment driven by Sales & Marketing team. Industry/Domain knowledge (e.g. work experience with clients in private sector/ automobile/automotive industry). Good knowledge of Community and Commerce dimensions of portal solution. Preferred experience: 2-3 years as Solution Architect (Solution to sell and/or deliver)

Business team to assign a SPOC

Business Team to assign at least 1 representative who is available at project site full-time during key project phases.

• Responsibilities include: Participate in requirements gathering session with implementation partner and business to ensure there is no ‘leakage’ of business requirements in terms of understanding or solutioning. Highlight requirements which may need business review, as early as in development phase. These requirements would largely be in Look & Feel aspects of the site, which require iterative process for acceptance. Participate in demo/dry-run/conference room pilot that maybe planned by implementation partner. Participate in User Acceptance Test. Be the conduit between Business Team providing timely input and sign-offs to design, content sourcing /verification et al. Attend meetings on need basis for any clarifications et al. Project phases during which this/these representative(s) would be required: Full-time during Requirement Gathering Phase & UAT. Need basis for all other activities mentioned above. 

Review proposal submitted by implementation partner

• The Scope of Work that is understood and documented in the proposal should be reviewed by IT and Business Team to ensure all stakeholders are aware of  what’s in scope and what’s not
• The gestation period from Sales presentation to proposal negotiation & contracting may be high – with scope being ‘lost’, ‘negotiated out of scope’, delivered as work-around or later. It is therefore important for all stakeholders to review the final draft before contracting
• Example: Implementation Partner sales team presents slide on Marketing Campaign capabilities with an eye on future road-map & to demonstrate capabilities of product suite. These functionalities may catch imagination of certain stakeholders but may not find place in scope of current project.

Quality Assurance 

There are 4 key areas which need to be under QA purview:

Content

Content Verification is a unique testing that needs to be conducted by business team along with or before User Acceptance Test. Content Verification will test or verify that the content that will be published is actually the content that was provided. UAT (User Acceptance Test) will test functionalities and not Content.  

• Content Inventory should be maintained to track content (images, copy, video) provided to implementation team
• The inventory tracker should contain Content details, folder where stored, date when provided, versioning
• This inventory tracker should be referred by all stakeholders within the Content Eco-system (business team, creative team and implementation partner) and shall be trusted source for all content items
• All content provided should be QA before being delivered to implementation partner. There is a time-gap of several weeks between content being provided and being published to site. Publishing to site usually happens close to UAT and/or Go-Live
• It is therefore important that content provided is checked at source and not after being published. Any defects identified after being published may impact UAT and/or Go-live as there is insufficient time for fixing content defects if they exceed the control limit stipulated by Project Management team (usually 2% defects are acceptable)

HTML & Design

A performance test is key to ensure Design team has not gone ‘overboard’ is creating ‘heavy’ designs that take time to render. This is especially critical in countries where there are internet bandwidth constraints. An example of an automotive website for an auto company at Dubai is shown below:

• Implementation partner must advise creative team on development standards to be followed by creative team for HTML coding
• Creative team must follow these guiding principles (provided by performance test team):
• High number of CSS and JS Scripts will impact performance of site and impact page load
• Design should follow best practice, in effect rendering online Look & Feel/ UI with low size images. High size images, on account of poor design requirements will impact performance of site and page load

Some statistics that can be food for thought for our design team: For every 1 second of delay in page load translates to:

• 7% loss in conversion
• 11% fewer page views
• 16% decrease in Customer Satisfaction
• 57% users perceive the site negatively if it takes beyond 5s to load.
• 80% of these users resist returning to the site and spread negative opinion.
• 5sec + page loads result in degradation of Business performance.
• Search Engines (including Google) penalize ranking of slow sites

Delivery & Testing

Some of the QA metrics that maybe measured and reported include:

• Defect Removal Efficiency: {(No. of defects identified by implementation team)/(No. of defects identified by implementation team and business/user team)} * 100
• Defect Rate: {(Total defects/Total delivery effort )}
• Delivered Defect Rate: {(Defects identified by business/user team)/ (Total delivery effort)}

The implementation partner must follow strict QA processes:

• All Unit Test conducted by developers should be recorded and published
• All Peer Review conducted by development team leads should be published
• Peer Review should QA against Development Standards (e.g. Naming convention used for Title Pages); and verification of test condition results prescribed for each development object
• The Business Analyst, onsite, should conduct a final Functional Acceptance Test for each business requirement
• And finally the QA team should also publish their test results when they perform QA after development
• It is crucial that Look & Feel related items are run past client. Client should conduct Functional Acceptance Test for such items. This is crucial as Look & Feel items follow an iterative process of acceptance.

Delivery model

Large ECM projects are usually delivered by teams working out of multiple locations. A project site and offshore delivery centres. In case of ECM projects, especially for Development phase, it is important that development items for Back-end processes (transactions) and ‘Look & Feel’ requirements follow a different development cycle. ‘Look & Feel’ requirements are subjective and therefore it would be useful if these are reviewed by business/user teams during Development phase, instead of waiting till UAT phase.

Arrange a Conversation