An Iterative, Non-Linear Approach to BI Projects

What often happens in BI development is that the business environment changes before the solution can be delivered. A straightforward explanation for this is the reliance on the traditional “waterfall” development approach.

First used in the late 1970s in scholarly articles and later adopted as the standard by the US Department of Defense for working with software development contractors, this method focuses on long-term results and relies on standards, stability, and centralized control.

The idea is that no stage of development should be repeated since that would be inefficient and costly. This linear approach is beneficial when the project is clearly defined and end users don’t need to change its scope after it has begun, which is not the case for most BI development projects.

The BI development life cycle instead consists of several different stages, each of which can evolve and change through the project’s lifetime:

• Gathering of Requirements
• Data Source Identification
• Data Integration (ETL) Development
• Data Warehouse Development
• Reporting and Analytics Development
• BI Solution Testing, Deployment, and Implementation

To view the DW/BI Lifecycle, click here

Agile development, in contrast to the waterfall method, enables teams to prioritize, plan, execute, and review work to deliver solutions to users in usable increments. Beyond that, 71% of organizations use Agile on their projects sometimes, often, or always.

Through the Agile BI development model, BI tools deliver valuable information and data to users, along with business insights, team collaboration, and most importantly, the ability to adjust to change as it occurs.

Scrum and extreme programming are both different approaches to Agile, but this article will focus on the Scrum method. Scrum projects are divided into sprints, which include planning, review, and retrospective sessions. Sprints have different categories with work items that can be classified according to the following:

• Requirement Analysis
• Data Presentation and Visualization
• Data Integration and Modelling
• Data Augmentation
• Testing and Validation
• System Configuration

Each of these work items can be combined, leaving the question of how to integrate them into each sprint. Agile BI operates on the assumption that user feedback and requirement refinement are crucial in determining which items to prioritize. In particular, most BI teams expect to iterate on the first three categories the most.

Work items with less value to the user would be pushed to a later sprint, helping to facilitate a more valuable BI solution to the user in less time.

Incorporate Insights from Business Experts Early On

Sprints aren’t the only way to ensure successful BI development projects. Selecting the right team is also essential.

Too often, the focus for developing many business tools is confined to technical experts, such as data architects and software engineers. However, these technical experts may not necessarily have a clear understanding of what the end-user is looking for in terms of a BI tool.

The primary task of the product owner for any BI development project is to ensure the project delivers value to the business. The ideal product owner should not only be a subject matter expert but also possess a comprehensive understanding of the entire business.

Since the end goal of Agile BI development is to provide data to the user – often in a visual form, allowing them to take immediate action – this business understanding is essential.

The process of choosing the right product manager and incorporating business insights, however, requires buy-in from senior management, who must give them the necessary amount of time and level of authority needed to work on and take responsibility for the project’s success.

Once the right product manager is identified, the iterative model approach is crucial for ensuring that business value is incorporated from the earliest stages of development.

Since the goal of Agile development is close collaboration between team members, it is also essential that the entire team is well-suited to the project. The Scrum Master, for example, must manage the project’s scope and ensure sufficient time is allocated for prioritized tasks.

In the fast-paced and dynamic environment of Scrum projects, it is essential to recognize that many team members possess multiple skills and can support one another, providing necessary assistance to enhance team productivity and capacity.

Listen to Stakeholders at Every Step of the Way

One of the most frequent complaints of businesses is that by the time IT has implemented the reporting and analytics features, the issue is no longer relevant. Since Agile shortens the development cycle and provides a quicker time to market, it can positively affect how your users perceive the value of your BI services.

However, Agile cannot be implemented without the active participation of stakeholders, who must provide their requirements and collaborate with the team. After all, they are the ones most affected by the project’s outcome. Stakeholders can be anyone, from team members to product owners, to the head of marketing and sales, or the head of customer service. They could be internal or external.

The key to successful Agile BI development is the ability to incorporate feedback from multiple and diverse stakeholders at every step of your project or sprints. As business needs change and evolve to constantly meet customer demands, Agile BI development projects will continue to drive value by keeping these stakeholders in mind.