Why an "engine" for workflow ?

[jay vas]

Hi guys. Ive been banging my head against these workflow technologies for some time, trying to find what it is about them that makes them so special.

For me, workflow can mean any procedural logic that is implemented programmatically. In that sense, isnt any programming language a workflow engine ?

I really want to learn how to automate workflows using something like JBPM but havent been able to understand what it is that JBpm does which good old object oriented code doesnt also do. For example, the JBPM website sais :

-Pluggable architecture
-Extensible and customizable on every level
-Easy programming model

This reminds me of what youre instructor sais when you first take a class in Java programming and OOP.

Then after they cite these reasons, they mention a whole bunch of other acronyms which are confusing, and if you google those acronyms, you the get exact same list of pros (i.e. extensibility, pluggability, modularity, etc etc). It seems like its an endless circle.

When working on large projects, I always look to the relational data model as the "end point" i.e., the place which gives immediate relevance, structure, and scalability to any software component in a system. But it seems like these WSDL BPM XPDL types only reference databases as secondary services in their frameworks, making it difficult to understand what it is that is being solved by their frameworks.

Ive been researching this because Im tired of reinventing the wheel. I learned hibernate because it seemed like it would save me time and it did, just as did learning OO design principles. But the next stage in creating better software seems like it is "workflow technology" - since all applications have a workflow about them... but where in all these tools is the concrete benefit of adaption ?

Will somebody please give us a real world example of how xpdl/Jbpm/etc speeds deployment, development, etc. ?

[Peer Reynders]

Originally posted by jay vas:
When working on large projects, I always look to the relational data model as the "end point" i.e., the place which gives immediate relevance, structure, and scalability to any software component in a system. But it seems like these WSDL BPM XPDL types only reference databases as secondary services in their frameworks, making it difficult to understand what it is that is being solved by their frameworks.

This actually suggests that you are "data-(structure-)oriented". The database is the crown-jewel of the corporation but without process/behavior that data serves nobody just like process cannot produce results without data. WSDL exposes operations which imply that they are based on data; the data-structures exposed are limited to the ones that are necessary to communicate with the operations - the relational model is hidden away, "encapsulated" so that it is free to change in response to pressures unrelated to the operations that are exposed in the WSDL.

A workflow engine facilitates "workflow" by orchestrating (i.e. it is the conductor) the interactions between disparate document sources and sinks. Now your document sources and sinks could simply be databases - however that would mean that the workflow is providing all of the process (unless you are using DB stored procedures). However a good workflow engine should also be able to use existing business logic that may already exist as components or services, using them as document source and sinks and access them with the required interfacing technologies (JAR, RMI, IIOP/CORBA, COM, DCOM, TCP/IP, HTTP, SOAP, etc.). It also needs document transformation and aggregation facilities so that it can feed the output from one or more document sources in a reformatted fashion into another document sink. So in fact the workflow is largely procedural. A workflow instance may need to track its current position in the flow and it may even have to buffer some documents that have already beed produced and won't be used until later - but is doesn't have "state" in the object sense, especially considering that the flow usually simply starts and runs to completion or fails (rather than exposing an "interface protocol" as an object does to its clients).

It may be worth your while to skim through J�rgen Thelin's: A Comparison of Service-oriented, Resource-oriented, and Object-oriented Architecture Styles, especially the slides following "Comparison of 3 Distributed
Architecture Styles" (Slide 37).

Object-Orientation is tightly coupled which means it is suitable for closed systems under the control of a single "owner".

Resource-Orientation is loosely coupled due to the universal (CRUD) interface and suitable for "read-mostly" data that can be linked to between different "owners"

Service-Orientation is loosely coupled*** and suitable for sharing a system between owners.

(*** a lot of the loose coupling is lost on the client side when you bind the service contract "early" through the use of a code generator to an OO interface)

The point being is that OO isn't always the right tool for the job. It is great for managing complexity in "closed" systems but the fact that objects (as well as services) have to be interacted through an "arbitrary" interface (i.e. specific to that class) leads to tight-coupling that can be difficult to deal with in "shared" systems (See also Transitioning From Object-Oriented to Resource-Oriented Programming; while a bit extreme I can see his point).

Now if you are saying that you see workflow engines exclusively used to pluck data out of one database, transform it (i.e. apply some business logic) and then stuff it into another database then I would tend to agree with you - I don't think workflow engines were supposed to be used for "micro-managing" business logic at that level; I'd imagine that the resulting set of workflows could become unmanageable quite quickly. Better to compartmentalize the business logic and implement it in a technology (OO solution, Business Rule Engine, etc.) more capable of dealing with the complexity in such a way that it can be shared by many workflows.

Then of course there is the matter of audience. While it will take software developers to implement business logic components and services, I don't think that people creating the workflows for workflow engines were supposed to be software developers - even though developer skills are definitely an asset.
[ April 18, 2008: Message edited by: Peer Reynders ]

[Anil Vupputuri]

Workflow engine: Its like sevices depend on each other, with the requirement of interdependent processing tasks, one task is dependent on outcome of another task. BPEL engines are just one example. There are many others: event processing, business rules engines, extract/transform/load (ETL) engines, etc. These and many other technologies are all perfect candidates to be implemented as JBI compliant service engines. A service engine is a runtime process to which a developer deploys an artifact (BPEL code, business rules, etc.).

That artifact can interact with services provided by other JBI compliant service engines. For example, a BPEL process could invoke a script over in an ETL engine in order to move some data around.

We can achieve this in sequential programming using if-else logic but this is not effecient so samething can break up a sequential process into individual activities which are tied together using workflow engine like BPEL. And another example of workflow engine is Spring framework using Inversion control, where all the activities in workflow process are defined in configuration (xml) document.

[Jimmy Clark]

I really want to learn how to automate workflows using something like JBPM but havent been able to understand what it is that JBpm does which good old object oriented code doesnt also do.

Will somebody please give us a real world example of how xpdl/Jbpm/etc speeds deployment, development, etc. ?

Business Process Management/Business Rules and a Business Engine enable non-programmers to create business applications without programming. They create composite applications using web services.

This directly speeds up product development because there is no traditional analysis, programming, development, and testing cycles.

In order for this to work, web services must exist already. For example, if a company has a library of 1000 web services, business analysts will be able to create applications "on-demand". This results in a streamlined product development process with efficiencies stemming from service reuse, smaller IT staff, reduction in errors, and a bunch of other stuff, including business agility and ability to compete better.

Traditional analysis, programming, development, and testing cycles still occur when creating the web services. However, these activities are not part of the product development and delivery cycle. Composite applications are built using web services that have been created, tested and deployed already.
[ April 23, 2008: Message edited by: James Clark ]

[jay vas]

Okay... thats a good explanation, but Im still wondering - how is this different than a traditional application ? Dont normal applications have services (we used to call them methods) ?

[Jimmy Clark]

The methods of an object and a Web service are different concepts. Basically, the concepts of service-oriented architecture are an evolution of the object-oriented paradigm.

Composite applications are designed and created with Web services that exist already. When you build a new Composite application, there is no coding, unless you need something that does not exist already in your library of Web services.

There are all types of characteristics for Web services and decision points to be considered when designing them. Too many to cover in a forum post, but there is ample material for you to study to get a good understanding of service-oriented concepts. When you get a chance, read the following:

Service-Oriented Architecture (SOA) Compass: Business Value, Planning, and Enterprise Roadmap
Hardcover: 272 pages
Publisher: IBM Press
Language: English
ISBN-10: 0131870025
ISBN-13: 978-0131870024

Enjoy!
[ June 09, 2008: Message edited by: James Clark ]

[samy omar]

Hi dears,
Can anybody tell me please how to implement a workflow engine ??
i searched google for any help but all I found was speaking about how to
use the built engines.
I am doing my master and I have to build a workflow engine

Waiting your contributions

Thanks in advance
[ July 24, 2008: Message edited by: Ulf Dittmer ]

[jay vas]

As far as I'm concerned, a workflow engine is a for loop.

1) A "smart" workflow engine is a for loop with a couple of if/else statements in it.

2) A "dyanmic" workflow engine is a for loop with an if/else statement in it that calls a couple of methods somewhere else in your code.

3) A "modular" workflow engine is a for loop with an if/else statement in it, that calls a couple of objects somewhere else in your code.

4) An "extensible" workflow engine is a program someone else wrote , that automates the extremely difficult process of writing a for loop, so that you may now implement a for loop in ~100 lines of code (not including several XML configuration documents). This is a significant productivity enhancement over the 3 lines of code which normally are required to write a for loop.