Sunday, December 9, 2007

Agile Business Rule Development Methodology

I will publish this month the Agile Business Rule Development plug-in for the Eclipse Process Framework. This plugin presents a pragmatic approach to the development of a Business Rule Application using Rule Engine and Business Rules Management System (BRMS) technology such as ILOG JRules, Rules for .Net or any other Rule Engine and BRMS on the market.

Business rules are an expression of business policy in a form that is both comprehensible to business users and executable by a rule engine. From a business perspective, a business rule is a precise statement that describes, constrains, or controls some aspect of the business. Let’s illustrate this with a simple example coming from the lending industry. The following business policy can be established to limit the purchase of loans by a bank or lender:

Only prime loans are eligible for purchase

This policy may be split into two rules, one defining what a prime loan is and the second taking the decision on purchasing it.

R1: If the loan amount is less than the prime loan limit, then the loan type is Prime

R2: If the loan type is not Prime then reject the loan

Implementing these rules using an Object Oriented Language may look like two if statements testing conditions on objects, and applying actions on the same or other objects:

if (loan.amount <>

loan.type = PrimeType;

}

if (loan.type != PrimeType) {

case = new Case(“Loan is not Prime”);

loan.status = REJECT;

}

The concept of business rules is not new, and for decades analysts and developers were implementing them within business applications. What is more recent is the introduction of a Business Rule Management System application which provides tools to manage the rule as a standalone element, outside of the core business application, and in a form which is executable by a rule engine.

Business rules are packaged as a rule set which can be considered as a piece of software executed by a rule engine. Integrated in a business application, the rule set implements a sub-set of the business logic. This logic is externalized from the traditional code and can be easily changed, and maintained by business analysts instead of developers.

As opposed to business logic embedded and hidden in functions, procedures, macros or databases, business rules are clearly separated from the rule engine, a software component which executes and activates rules based on matching conditions on a set of data.

Figure 1 below illustrates this fundamental approach in implementing business logic, with the old embedded model on the left and the new architecture on the right. In many ways the approach is similar to the separation of code and data which Data Base Management Systems (DBMS) brought in the 1970




Figure 1: Buried vs. Externalized Business Rules

A piece of business logic or rule embedded in code will be more difficult to identify, locate, and change, therefore increases costs of maintenance.

A rule which is externalized in a BRMS platform, and whose versions are controlled in a rule repository and deployed and executed in a rule engine, will be much easier to modify, thus reducing dramatically the time and cost to implement changes required by the business.

Traditional software life cycles such as the V cycle or waterfall model are notorious for leading to systems which costs more than expected, come to market later than planned and with outcomes not matching the clients’ initial expectations. With such approaches, the business users have little ownership of the solution, and the time required to implement changes can easily range from 3 to 6 months if not more. As a result, policy owners don’t feel comfortable with change which translates into a loss of agility to respond to new business and competitive situations: too much time passes between policy owners submitting requirements to IT and the actual deployment of new rules.

Also, with all the fine tuning and iterations, policy owners cannot really be sure that the policy is being implemented according to their needs. The only way to actually know is to keep performing tests and hope that all the cases are covered in such tests.

As important element of the business decision, the business rules need strong management processes and tools to support their life cycle.

A BRMS provides solutions to make this management more efficient, both for developers and for the business users of the applications. Obviously tooling is not the single answer to the software and business challenges behind the development of decision-support systems. Deploying BRMS, BRE, and BPM components into business applications requires to modify current development methodologies and practices in order to take into account new concepts as well as providing a better collaboration framework between IT and business.

The Agile Business Rule Development (ABRD) methodology corresponds to such a need. It is an incremental and iterative software development process leveraging the Agile Alliance manifesto. Here are some of the Agile development values particularly relevant to the implementation of a rule set using the ABRD approach:

- Individuals and interactions over processes and tools. The rule discovery, analysis and validation activities require an active and efficient communication between the rule developer and the Subject Matter Experts (SME). Such processes are defined as light as possible.

- Working software over comprehensive documentation. The proposed rule set development done per iteration, with its validation step, is based on the evidence that a working and executable rule set has much more business value than a rule description manual. All the project stakeholders benefit from such a principle but in particular the business users who are then sure that what they see (the rules) is what gets executed in the deployed system.

- Customer collaboration over contract negotiation. Subject Matter Experts who define the business policies and the business rules are strongly involved in the development process. They are the customers of the final system, owners of the policies and are conveniently collocated with the development team during the project.

- Responding to change over following a plan. Business rules evolve, more often and faster than other standard pieces of software. This is actually one of the key values of the business rule approach. For this fundamental reason the methodology to support the rule set development has to be tailored to such rapid life cycle and include the appropriate activities, processes, best practices and work products to support such changes efficiently.

The Agile Business Rule Development methodology addresses in more detail the following goals:

- Separate rules as a manageable artifact using discovery, analysis and authoring activities and work products
- Trace rules during their full life cycle from requirement to deployment and maintenance
- Link rules to business context and motivation
- Develop the rule description using business terms and high level rule language

- Prepare the logical data model for the rule engine (The concept of Business Object Model in ILOG JRules)

- Prepare the Rule set implementation and deployment as decision services in a SOA or BPM application

- Articulate the rule governance processes

Two fundamental drivers govern successful rule set developments:

- The unforgiving honesty of executable rules

- The effectiveness of people working together with goodwill, shared vision and common interests (the business user and the development team).

Executable or working rules tell the developers and the Subject Matter Experts what they really do as opposed to promise through a paper-based design or specification.

I started to develop ABRD in June 2002 to address customer's request on a methodology for BRMS. Extended after that it uses extensively at ILOG to support successful implementation and deployment of BRMS by its customers. It is now offered as a free open source methodology as part of the Eclipse Process Framework plug-in library.

ABRD extends OpenUP to avoid redefining some standard roles, tasks, work products, guidelines and processes which are also relevant to rule development. As an EPF plug-in, development teams can leverage it for their own purpose and tailor it to the specific context of their own project, leveraging the EPF Process Composer.

Agile IT Architecture

This blog is about discussing Agile IT Architecture and sharing best practices - comments, how tos with IT architects, project managers, developers. Working from personal experience, experience of people working with me, and articles on the web I hope this blog will be used by people to get good information on this interesting subject. I'm author of a methodology used internally at my company, ILOG, but I'm in the final process to publish and share a piece of this methodology as an open source project part of Eclipse Process Framework. (EPF)

I'm also in relationship with a nice initiative around sustainable architecture which promotes restructuring of IT architecture by focusing on reference data, process and business rules and the methodology to do so.

There is a real need to share knowledge and experience on those subjects.

Agile IT architecture is about designing IT architecture that supports the quick changes in the business goals. Business users wants to run their business better, generate more revenue, reduce cost, deploy quickly new product, new marketing campaign...
It is interesting to note that despites the progresses done on product and technologies business is still complaining about getting quick change of the underlying supporting IT software stack to support change in business.

This is not so dark!. There are a lot of successful projects and applications which are bringing agile application: web front end and portal help to deploy quick marketing campaign, publish content and best practices to help current and future customer on our product....

Early SOA, BPM and BRMS deployments are bringing values and facilities to support business, but still it looks there is some needs and failures. One of the major issue is to be able to access to how-to and knowledge sharing and practices sharing.

SOA, BPM, BRMS, MDM, ESB,.. all those components are needed to deploy an Agile IT architecture and there are a lot of companies using those products to support this ultimate goal: "close the gap as much as possible between IT and business". But it is not so easy to do.

SOA design and deployment are not easy to do. Most of current SOA initiatives are about exposing a legacy application as a set of services we may be able to reuse. SOA is not short term thing, and needs to be approached as a top down initiative to support the future of the enterprise.

BPM helps to design, orchestrate and control the execution of a business process. Well! The business users need to define their processes, spend time on it, and then we can use BPM to partly implement it. Once a process is defined and implemented in a BPM, I'm not so sure it will change a lot. This is interesting in this domain to see that BPM suite comes from workflow, document processing, and are migrating to BPEL and orchestration of services. Does orchestration of service a business process? I can see some parallel but examples are more on application logic flow implemented in BPEL.

BRMS, younger player in the IT architecture, it is still not used enough. This component is bringing a lot of agility, and business users can understand how their policies are mapped to executable rules. One of the issues as of today is the lack of concrete, practical methodology to address the how do I use those set of tools to be able to develop, maintain my business policies. I will discuss about that in next post.

As of today it is easy to forecast, despites the marketing efforts, it will never happen: Business users do not want to change their process, do not want to code or write business rules. They want to run their business better.
And I guess it is better like that. There are different job. I can discuss with my VP sales and study how he is working, his team focus to make sales, manage lead, close deal. They want accurate data on lead, revenue forecast, sale unique view of a customer, track time or results of team members... CIO and IT team has different job. So why trying to merge both? Can you imagine having 150 different business applications designed, developed by business who did that to server a special business need at a given time? This is the excel macro and worksheet pattern. A lot of people can do nice things with excel and get efficient supporting data to manage his own business decision. But is it the IT we want?

In fact I'm pushing the envelop here. There is one important person in a company which is the strong liaison between IT and business: the business analyst. This is true that BPM, BRMS, exposed SOA services will help a business analyst to understand how the business application is running, and even change the policies, but IT is still owner of the application, and manage its availability.

So the subject is big. Lets' discuss about it.