How Process Mining can support your BPM project

Process Mining is an important new capability that supports and enhances BPM initiatives. This article describes what process mining is, how it fits into the BPM methodolgy and why organisations should start mining their business processes.

What is Process Mining?

Process mining is the reverse engineering of business processes based on event logs.

Event logs are produced by "process aware information systems". These event logs contain; case ids, event names and types, originators and timestamps. Event logs may also contain; customer ids, product types and other decision attributes. Event logs are often kept as an audit trail.

Organisations need to map their workflows as the first step. It's a daunting task. There are workshops with key people. Where do you get the data from? How do you know it’s correct? Projects are often abandoned because they are seen to be taking too long and consuming valuable resource time (even after initial successes).

An event log from an application is converted to an MXML (mining xml) format by an interface. Now it’s ready to be mined and analysed. This requires about a day per log file to code and test. It provides performance data that answers the following questions;

• How long does each event take?
• What are the wait times?
• What is the throughput time?

So if you have poorly documented legacy systems which need to be analysed, improved or converted into new systems then process mining is your starting point.

These tools can save a significant amount of time while providing accuracy and analysis that would be almost impossible to determine otherwise. In order to carry out a process improvement project you will need this data to benchmark your processes.


What is BPM?

BPM is the achievement of an organisation’s objectives through the improvement, management and control of essential business processes.

What are the key elements of the BPM discipline?

According to Gartner they are:

•Optimizing the performance of end-to-end business processes that span functions, as well as processes that might extend beyond the enterprise to include partners, suppliers and customers

•Making the business process visible (and thus explicit) to business and IT constituents through business process modeling, monitoring and optimization

•Keeping the business process model in sync with process execution

•Empowering business users and analysts to manipulate a business process model to modify instances of the process

•Enabling rapid iteration of processes and underlying systems for continuous process improvement and optimization

Process mining supports these key elements. Some examples;

End to end business processes can be discovered using process mining’s workflow discovery capabilities. Extracted event logs can be mined and converted to various workflow formats and exported to standard BPM modeling tools.

Process mining can make the business processes visible and explicit by annotating workflows with performance and organisational data.Using process mining’s conformance capabilities the current process model can be compared with the log from the latest execution of the application. The conformance reports will highlight variations between the model and the log.

Process mining tools allow inquiry and discovery of individual process instances using various tools.Extraction of an event log, conversion to process mining format (mxml) and mining and analysis by a process mining tool is a rapid. This allows for a quick response and round tripping between model and log.
 
How can you use process mining in your organisation?

Process Mining is an important part of the BPM tool kit. There are various opportunities for organisations to make use of this capability.

Legacy systems are often the backbone of organisations. They are often poorly understood and not well documented. Nevertheless they are important to the running of the business. By using process mining these applications systems can be brought to life by the creation of workflows and performance analysis.

Process mining will show what the application is doing and what resources are being used. From this analysis management can decide to rewrite, reengineer or make changes to the system or human side of the process. Once this is done the new business process can be executed and compared to the new process model to ensure the changes had the desired effect.

Legacy systems are often linked in a complex web. Using the semantic capabilities of process mining this can be unraveled and the applications better understood.
Auditors can make use of process mining to better understand the process and eliminate risks to the organisation.

Many organisations run a standard application to support their business. They are often unhappy with the performance of their processes without actually knowing what those processes are. Process mining can help discover what the issues are and where changes can be made.

For organisations that are moving from one application (eg. ERP or CRM) to another process mining offers an opportunity. It is important in a BPM context to be able to compare the “As Is” business processes with “To Be”. Process mining provides this capability.

Process mining is an important part of BPM. Organisations often embark on BPM projects from small process improvement projects to full scale Six Sigma efforts without knowing what their processes look like. The time taken to determine the processes with traditional methods is often inaccurate and time consuming.
Process mining provides a quick start to these projects that is risk free. There are many opportunities for organisations that use process mining!

References:
Gartner RAS Core Research Note G00164485 18 Feb 2009, RA3 02192010

Process Mining and Call Centres

Process Mining can be applied to almost any area where a system produces event logs.

In this powerpoint presentation a call centre is used as an example.

The presentation demonstrates the various steps from process discovery, ensuring process capability and to the performance analysis. This example shows how Process Mining can be of use to Six Sigma practioners.

http://www.authorstream.com/Presentation/GDVarv-283836-callcentre-entertainment-ppt-powerpoint/

Process Mining, Knowledge Creation, Ontologies and Frameworks

Can these capabilities be used for a better understanding of ERPs?

This article discusses how these “capabilities” can be combined to facilitate the use of ERPs.

Overview

Process mining allows the reverse engineering of event logs into workflows that describe how a process behaves. Traditional process mining techniques are based on syntax. Typically process mining requires; a unique process id, event names and types, date and time and user names.

Most ERPs are not suited to this kind of analysis. They are either without any accessible process id or a reliable audit trail. However, applications running on the IBM i (previously AS400 or iSeries) have a journaling capability that can provide an answer to this problem.

Using semantic knowledge allows us to extend the process mining capability and explore other disciplines. ERPs are rich in knowledge and meaning. There are structures that are known to IT users but often not well understood by the business. One of the issues that business users have with ERPs is their complexity. An objective must be to introduce greater clarity and simplicity to the ERP.

Capabilities such as the use of ontologies and standard frameworks (APQC PCF) create the structure that allows semantic process mining and the creation of new knowledge.

These capabilities allow an organisation to ask new questions;
How is SAP Purchasing different to other ERPs?
What capabilities does SAP have compared to other ERPs?

Knowledge Creation

One of the difficulties in creating new knowledge is the conversion of tacit knowledge to explicit knowledge. Tacit knowledge is about know-how. It involves mental models. The sort of knowledge that organisation find hard to document and share within an organisation. This is true of application systems such as an ERP.

For example, every ERP has a Purchasing module that allows the organisation to purchase materials and manage relationships with vendors. There may be a set of applications that form a process called “Process Purchase Orders”. However this knowledge may not be readily available to business users. One way to resolve this is by building ontologies. This will allow the creation of explicit knowledge that can be shared and understood throughout the organisation.

AQPC Process Classification Framework

“Created in collaboration with expert and practitioner panels in 1992 and updated annually, APQC's Process Classification FrameworkSM (PCF) is a globally recognized business process model that defines activities and processes across 12 enterprise-level operating and management categories.” (APQC)

The PCF is also a good benchmark to make sure you haven’t missed anything when reviewing an ERP. Any process that is discovered should exist by a similar name in the PCF. See reference for PCF details.

Ontologies

The basis of semantic business process mining is the ontology. Ontologies can represent applications, organisation or business objects.

There are two main benefits in using this kind of ontology. Firstly, workflows are created the mid level descriptors (Change Purchase Order etc) can be used rather than the codes. Secondly, like processes are grouped in one place so that business users can see the applications capabilities.

Conclusion

These capabilities are useful on their own but are much more powerful when used together. If the goal is knowledge creation, process mining provides new information that is organised through frameworks and ontologies. If the goal is business process management, process mining provides discovery, the ontologies provide clarity and structure and the frameworks provide a checklist and a standard.

References
http://www.ip-super.org/
http://www.infor.com/
http://www.sap.com/australia/index.epx
The Knowledge Creating Company; Nonaka; Takeuchi, OUP 1995

Semantic Process Mining and SAP on the IBM i (AS400 iSeries)

Background

Process Mining allows for the reverse engineering of business processes. By the creation of domain specific ontologies that support the mined data, greater meaning can be revealed using semantics concepts. Process Mining tools like ProM allow for the creation of these application ontologies and their subsequent analysis.

Issues

Most modern ERP applications are large and complex. SAP is no exception. Thousands of transaction codes are linked to activities, roles, components and applications through various types of documentation. To complicate this further the link between transaction codes and activities is not always a one to one relationship.

Methodology

This paper will discuss how this structure can be made be available in a machine readable format that will both simplify and clarify this structure for SAP users. A top down approach is used.

Examples will demonstrate how process mining tools can provide concise analytical data based on this ontology. Using a semantic process mining tool (ProM) both performance and conformance data can be extracted.

Ontologies provide a way to structure SAP’s ERP. The SAP documentation uses the following terms; Application, Module, Component, Role, Activity and Task to group the various entities in SAP. The Ontology described here will use those terms in the same way, as well as adding an extra term called Process, which is based on the APQC Process Classification Framework.

This is not an exhaustive list of all components in the MM module. A subset of the 400+ concepts in Materials Management is used here. A high level view of the complete ontology is attached. It is not legible due to size but it gives an idea of the scale and structure of the ontology (Fig 8).(Note that ontologies can be created for any organisational structure, product codes, or practical business object.)

SAP Application Architecture

The name of the ontology (Fig 1) is “SAP_Application_Architecture”. The selected SAP defined application name is “Logistics”. The module name used in this example is “Materials Management”.



Fig 1

The components of the Materials Management module are; MM_Inventory_Management, MM_Invoice_Verification, MM_Purchasing and MM_Service as seen below in (Fig 2).


Fig 2

The component MM_Purchasing is further split into its process groups in Fig 3 below.

The process descriptions (right hand side) are from the APQC Process Classification Framework. This framework provides a standardized naming convention as well as a way of grouping similar roles.


Fig 3

In the next step the process “Manage Contracts” will be split into its individual SAP defined roles.

Fig 4 shows the SAP defined roles for “Manage Contracts”.


Fig 4

Fig 5 shows the next step; activities and transactions for “Process Contracts”.


Fig 5


Examples

The question is (apart from having a complete hierarchy of components, processes, roles, activity and tasks) what is the benefit of semantic process mining? Here are two illustrations.

Performance Metrics in Ontologies

Fig 6 ‘Visualize performance metrics in ontologies’ gives a visual representation of the mined data using the discovered ontology. Also provided is an analysis of the time and frequency of each task SAP task summarized at each level of the ontology.


Fig 6

Semantic LTL Checker

From a conformance perspective, Linear Temporal Logic (LTL) is an important tool of process mining. The user can ask the question show instances where "activity A eventually executed"?

The result will be two lists of instances; one where activity A is executed and one where it is not. You can click on the individual instances in either list and review their details.

In this example we have selected the activity Process Purchase Orders. The LTL inquiry will select all the sub concepts of this activity in its search.

It will select (Create) ME21N and (Change) ME22N as it “knows” they are part of the ontology. There is an extensive range of inquiries possible through the use of the drop down menu.


Fig 7

In both these examples ‘abstractions’ have been used to simplify and illustrate.

Fig 8 - Complete MM Ontology



This is illegible in A4 but is produced to show structure and size. A pdf is available on request.


References & Links


http://prom.win.tue.nl/tools/prom/
http://www.apqc.org/portal/apqc/ksn?paf_gear_id=contentgearhome&paf_dm=full&pageselect=detail&docid=152203
http://www.ebookspdf.com/computers/2004/sap-mm-material-management-manual/

BPCS and SBPM on the IBM i (AS400 iSeries)

Many organisations are mapping and documenting their business processes as a first step towards managing their business processes. This can be a difficult task as there is often no useful documentation to describe these processes and the applications that support them.

This paper discusses how a new approach, Semantic Business Process Mining (SBPM), can combine with not so new technology (BPCS), to provide knowledge that is of benefit to both the business and information technology members of an organisation.

Background

The BPCS ERP LX application is the biggest selling mid-range ERP application with over 6,500 applications at 18,000 sites. It runs on the IBM i (S/38 and AS/400 and i Series) the biggest selling mid-range computer with over 750,000 installed sites.

Infor’s BPCS ERP LX supports high-volume, repetitive, multimode manufacturing. This product has an installed base in the specialty chemical, food and beverage, pharmaceutical, CPG and automotive supplier industries.

SBPM is new. Only recently developed as part of the SUPER research project (Semantics Utilized for Process Management within and between Enterprises) it has evolved as a methodology to bridge the gap between the different ‘languages’ used by business and IT. SBPM is part of the larger field called Semantic Business Process Management.See http://www.ip-super.org/

Issues

How does the BPCS Inventory Management (or any BPCS application) behave over a monthly cycle? Is the process behaving as expected? What does the workflow look like? Which users are working on specific tasks? Which groups of users are working on Inventory Transaction Posting? What programs are used in Cycle Counting or Month end? How does the organisation audit the application?

BPCS does not have any workflow diagrams. There are over 1000 programs and there is no defined hierarchy. There is also no link between the different users of BPCS.

Process Mining

Process Mining is a capability that allows the reverse engineering of event logs to generate workflow diagrams. We have developed an interface between BPCS and the Process Mining application ProM. http://prom.win.tue.nl/tools/prom/

Findings

The BPCS customer used in this example manufactures and sells products in Australia as part of a larger US conglomerate. They use all the BPCS modules and no other ERP applications.In order to discover BPCS processes we need to mine its event logs IBM i journals.

Using the BPCS to Prom Interface, BPCS event logs were mined at the end of a one month cycle (June 2009) from the client’s IBM i. Two tasks had to be completed before a semantic analysis could take place;

Create an Originator Ontology – Using WSMO Studio, an ontology was created for the Client. There were about 120 users, split into several groups including; Sales, Manufacturing, Finance and Logistics.

Create a BPCS Ontology – Again using WSMO studio, an ontology of BPCS tasks and groups was created. This was quite time consuming for several reasons; it is difficult to get a comprehensive list of the BPCS tasks and their descriptions, the descriptions are often not clear and we are only interested in those that update the database. It is then important to create ‘super’ concepts in the ontology that are meaningful and useful.

A section of the BPCS ontology is below.

BPCS Inventory Management Ontology

A further problem was to take the program descriptions and turn them from a program name to something more like a process name. For example; INV100D2 is the program called Item Master Maintenance. As a process name ‘Maintain Item Master’ is more appropriate.

Why Use Semantics?

Process Mining on its own will reveal workflows. Process Mining is based on syntax; Semantic Business Process Mining is based on meaning. It allows us to turn these Ontologies into a machine readable format. So the mining process application knows how all the pieces of BPCS fit together and who works for what department in the organisation.

Benefits

Semantic abstractions provide meaning. BPM raises understanding from a technical level to a business level. Rather than using codes we can now have descriptions in the analysis. They also provide clarity. We can group like codes into a process group.

In the next three diagrams we can see three levels of abstraction in a BPCS workflow.

This diagram shows the BPCS Inventory process as a workflow using program names.


The diagram below is exactly the same but using descriptions instead of codes.


The following diagram is a further abstraction where the tasks have been summarized into a higher level of the ontology.



For example, all the Cycle Count tasks appear as just one task, Perform Cycle Counting.

Clearly there can be as many levels of abstraction required and users can tailor their own ontologies to meet their individual needs. There are many other analyses available.

Note that the above applies to any iSeries application including Oracles ERP OneWorld and EnterpriseOne

What are Event logs?

It is probably appropriate to begin here. Event logs are produced by "process aware information systems". These event logs usually contain case ids, event names, originators and timestamps. They will contain decision data, ie customer ids, product types, information necessary for determing workflow decision attributes. The event logs are usually kept as an audit trail. So why would an organisation want to mine them?

When I first heard about Process Mining I was curious too. Process Mining in many ways addresses the initial issue organisations face when they start with Business Process Managment . They need to map their organisation's workflows but it's such a big task! Lots of workshops. Key people tied up for weeks. Where do you get the data from? Finally once its done how do you know its correct? Projects are often abandoned because it is seen to be taking to long even after initial success.

A Process mining tool takes a log file from an application and after a interface converts it to an MXML (mining xml) format its ready to be mined and analysed. This requires about a day per log file to code and test. Once this is done a mining tool can carry out literally dozens of different analysis and convert it to a workflow diagram in minutes.

Not only does it create the workflow but it provides performance data as well; how long does each event take, what are the wait times, what is the throughput time etc.

It is also possible to discover sociometry based analysis that will show the relationships between the users of the systems. You can mine emails to map your social networks!

So the answer to the original question becomes clear. If you have poorly documented legacy systems which need to be analysed, improved or converted into new systems then this is your starting point. These tools can save an enormous amount of time whilst providing accuracy and analysis that would be almost impossible to determine otherwise. In order to carry out a process improvement project you will need this data to benchmark your processes as you proceed.

This blog contains links to the Business Process Management Groups at the Eindhoven University of Technology & Queensland University of Technology plus other BPM sites. These two organisations are working together as leaders in Process Mining & BPM.