Process orchestration with Flowable and Spring Boot Chavdar Baykov Solution Architect (Digital Wallets) Paysafe Group
About Me • Solution Architect in Paysafe Group • Digital Wallets • 20+ years of Software Development experience • Java & Backend Services • Business Process Analytics • Cloud PaaS & SaaS • In my free time I like to • Spend time with the family • Watch movies • Play with tech gadgets
“ ” A series of actions or steps taken in order to achieve a particular end. We may not realize it, but processes are everywhere and in every aspect of our leisure and work What is a Process ?
Implementing Processes using Microservices Orchestration Choreography
Orchestration vs Choreography Orchestration • Process Engine node manages execution • Considered SPOF and bottleneck • Prone to increase of complexity • Process-agnostic services • Do not know their place in the process • Less prone to change • Process ownership • Distributed vertically • Issues fixed in the orchestrator instead in services Choreography • No central process • No SPOF and considered more scalable • Smart Services • Know their role in the process • Prone to change and increase in complexity • Process ownership • Lack of the ‘whole picture’ • Distributed horizontally across teams • Services are up, but process is broken effect
Hybrid Approach • Orchestration in Business Domains • micro-orchestrators • End-to-end process ownership • Choreography across Business Domains • Responsibility boundaries • More Stable contract • Less context required • Examples: • Know Your Customer processes • Customer levels processes
Introducing • Compact and highly efficient workflow and BPM platform (Process Engine) • Written in Java • Free Open Source • Apache 2.0 License • Can run • Embedded or on Dedicated servers • Clustered • Cloud or Locally • Highly customizable and pluggable • Only one required dependency • Database
Flowable Architecture • Persistency engine (MyBatis) • H2, Oracle, PostgreSQL, MySQL • Services • Segregated by functionality for each Engine • Engines • BPMN, CMMN, DMN, Form, Content • REST API • Flowable Applications • Modeler, Task , IDM, Admin
Flowable APIs
When is appropriate ? • Java ecosystem • Process as part of the service • Orchestrator role • Drives and tracks the business processes in some domain • Orchestrate single purpose microservices • Shared services • Embedded engine benefits • Allows some business logic to run embedded • Test driven development
Flowable and Spring • Spring Boot property-based auto configuration • https://flowable.com/open-source/docs/bpmn/ch05a-Spring-Boot/ • All expressions in the BPMN processes will also 'see' all the Spring beans, by default. • what Beans Flowable ‘see’ is configurable • Unit & Integrating testing • H2 in-memory database by default • Spring Boot tests – for Deployment, Configuration and Bindings • Spring tests - for Testing Flow logic with Mocked Task implementations • Unit test - for tasks and services
Process Definition • Describe the process steps • Tasks • Flows • Gateways • BPMN 2.0 XML format • each node has id • flowable extension elements • Diagram element contains visual metadata • Executions are called Process Instances • Instance state is stored in Process Context
Start Event • Indicates where process starts • Process start through API • Other Start Event Types (Triggers) • Timer Start Event • Signal Start Event • Message Start Event • Graphical Notation • XML Representation
Sequence Flow • Connector between elements • sourceRef - start element reference • targetRef – end element reference • Conditional Sequence Flow • Continues only if expression evaluates to TRUE • Graphical Notation • XML Representation
Java Service Task • Invokes a Java logic implementing a task • Can invoke • class – Java Class • delegateExpression - Java Bean • expression - Java Bean method • Use to orchestrate internal and external logic • Graphical Notation • XML Representation
Implementing Java Service Tasks • Not Instantiated During Deployment • Unit test them to prevent runtime errors • Stateless • Get and store state in Process Context (or somewhere else) • Retriable (if configured) • Short Running Tasks (sync, in transaction) • Implement org.flowable.engine.delegate.JavaDelegate • Long Running Tasks (async, out of transaction) • Implement org.flowable.engine.delegate.FutureJavaDelegate
User Task • Task, that needs to be done by Human • Execution stops until task is completed • Actual task logic is outside of the engine • API to query tasks • API to complete tasks • Might be assigned (delegated) to • Users • Groups of Users • Graphical Notation • XML Representation
End Event • End of a process path • Terminate End Event • Terminates the process instance • Graphical Notation • XML Representation
Parallel Gateway • Model concurrency in a process • fork – all outgoing sequence flows executed • join – wait until all incoming flows arrive • Conditions on incoming and outgoing flows are ignored • Graphical Notation • XML Representation
Exclusive Gateway • Model a decision in the process (XOR) • Only One sequence flow is executed • Expression must evaluate to TRUE • Execute first defined if multiple evaluate true • Graphical Notation • XML Representation 
Process Deployment • Process packaged in the Application • Activates the updated Process • Only deploys if there are changes • Automatic versioning • All referenced classes must exist • Watch out for cluster consistency issues • Trigger deploy after update is finished • Spring Auto Deployment • Deployment through API
Demo: Verified Phone Registration Process • Build a microservice, that provides verified phone registration • The phone validation process happens through secret token • Delivered as SMS (mocked) • 60 seconds validity • User should input the received token to complete the process • User can re-generate token (invalidating the old token) up to 3 times • User can abandon the process at any time • Existing microservices (mocked) • SMS Delivery, User Profile, eMail Delivery
Verified Phone Registration Process
Verified Phone Registration REST API • Create phone registration • POST /user/phone/registration/{userId} • Get ongoing phone registrations for User • GET /user/phone/registration/{userId} • Create new token for registration • POST /user/phone/registration/{userId}/{registrationId} • Submit token for registration • POST /user/phone/registration/{userId}/{registrationId}/token
Flowable Best Practices • Unit & Integration test the processes • Decouple interactions with process engine classes • Store only what is needed in the context • Optimize the process for execution and reliability • Fetch dependencies earlier, do modification later in the process • Serialize complex objects as JSON in context • Move business logic out of tasks • Services might be reusable, but tasks are not! • Add Error Handling to processes • Failed processes must complete at some point • Take care of process history
Thank you! Контакти Чавдар Байков chavdarb@yahoo.com LinkedIn https://www.linkedin.com/in/chavdar-baykov-8322385/ Github https://github.com/chavdarb Следващо събитие: Test Coverage: Why, What, Who, How
Resource Links • Flowable Home • https://flowable.com/open-source/ • Flowable Spring Boot Samples • https://github.com/flowable/flowable-engine/tree/master/modules/flowable-spring-boot/flowable- spring-boot-samples • Phone Registration Sample Application (Presentation Demo) • https://github.com/chavdarb/flowable-phone-registration-demo • Introduction to Flowable • https://www.baeldung.com/flowable

Process Orchestration with Flowable and Spring Boot

  • 1.
    Process orchestration with Flowableand Spring Boot Chavdar Baykov Solution Architect (Digital Wallets) Paysafe Group
  • 2.
    About Me • SolutionArchitect in Paysafe Group • Digital Wallets • 20+ years of Software Development experience • Java & Backend Services • Business Process Analytics • Cloud PaaS & SaaS • In my free time I like to • Spend time with the family • Watch movies • Play with tech gadgets
  • 3.
    “ ” A series ofactions or steps taken in order to achieve a particular end. We may not realize it, but processes are everywhere and in every aspect of our leisure and work What is a Process ?
  • 4.
    Implementing Processes usingMicroservices Orchestration Choreography
  • 5.
    Orchestration vs Choreography Orchestration •Process Engine node manages execution • Considered SPOF and bottleneck • Prone to increase of complexity • Process-agnostic services • Do not know their place in the process • Less prone to change • Process ownership • Distributed vertically • Issues fixed in the orchestrator instead in services Choreography • No central process • No SPOF and considered more scalable • Smart Services • Know their role in the process • Prone to change and increase in complexity • Process ownership • Lack of the ‘whole picture’ • Distributed horizontally across teams • Services are up, but process is broken effect
  • 6.
    Hybrid Approach • Orchestrationin Business Domains • micro-orchestrators • End-to-end process ownership • Choreography across Business Domains • Responsibility boundaries • More Stable contract • Less context required • Examples: • Know Your Customer processes • Customer levels processes
  • 7.
    Introducing • Compact andhighly efficient workflow and BPM platform (Process Engine) • Written in Java • Free Open Source • Apache 2.0 License • Can run • Embedded or on Dedicated servers • Clustered • Cloud or Locally • Highly customizable and pluggable • Only one required dependency • Database
  • 8.
    Flowable Architecture • Persistencyengine (MyBatis) • H2, Oracle, PostgreSQL, MySQL • Services • Segregated by functionality for each Engine • Engines • BPMN, CMMN, DMN, Form, Content • REST API • Flowable Applications • Modeler, Task , IDM, Admin
  • 9.
  • 10.
    When is appropriate? • Java ecosystem • Process as part of the service • Orchestrator role • Drives and tracks the business processes in some domain • Orchestrate single purpose microservices • Shared services • Embedded engine benefits • Allows some business logic to run embedded • Test driven development
  • 11.
    Flowable and Spring •Spring Boot property-based auto configuration • https://flowable.com/open-source/docs/bpmn/ch05a-Spring-Boot/ • All expressions in the BPMN processes will also 'see' all the Spring beans, by default. • what Beans Flowable ‘see’ is configurable • Unit & Integrating testing • H2 in-memory database by default • Spring Boot tests – for Deployment, Configuration and Bindings • Spring tests - for Testing Flow logic with Mocked Task implementations • Unit test - for tasks and services
  • 12.
    Process Definition • Describethe process steps • Tasks • Flows • Gateways • BPMN 2.0 XML format • each node has id • flowable extension elements • Diagram element contains visual metadata • Executions are called Process Instances • Instance state is stored in Process Context
  • 13.
    Start Event • Indicateswhere process starts • Process start through API • Other Start Event Types (Triggers) • Timer Start Event • Signal Start Event • Message Start Event • Graphical Notation • XML Representation
  • 14.
    Sequence Flow • Connectorbetween elements • sourceRef - start element reference • targetRef – end element reference • Conditional Sequence Flow • Continues only if expression evaluates to TRUE • Graphical Notation • XML Representation
  • 15.
    Java Service Task •Invokes a Java logic implementing a task • Can invoke • class – Java Class • delegateExpression - Java Bean • expression - Java Bean method • Use to orchestrate internal and external logic • Graphical Notation • XML Representation
  • 16.
    Implementing Java ServiceTasks • Not Instantiated During Deployment • Unit test them to prevent runtime errors • Stateless • Get and store state in Process Context (or somewhere else) • Retriable (if configured) • Short Running Tasks (sync, in transaction) • Implement org.flowable.engine.delegate.JavaDelegate • Long Running Tasks (async, out of transaction) • Implement org.flowable.engine.delegate.FutureJavaDelegate
  • 17.
    User Task • Task,that needs to be done by Human • Execution stops until task is completed • Actual task logic is outside of the engine • API to query tasks • API to complete tasks • Might be assigned (delegated) to • Users • Groups of Users • Graphical Notation • XML Representation
  • 18.
    End Event • Endof a process path • Terminate End Event • Terminates the process instance • Graphical Notation • XML Representation
  • 19.
    Parallel Gateway • Modelconcurrency in a process • fork – all outgoing sequence flows executed • join – wait until all incoming flows arrive • Conditions on incoming and outgoing flows are ignored • Graphical Notation • XML Representation
  • 20.
    Exclusive Gateway • Modela decision in the process (XOR) • Only One sequence flow is executed • Expression must evaluate to TRUE • Execute first defined if multiple evaluate true • Graphical Notation • XML Representation 
  • 21.
    Process Deployment • Processpackaged in the Application • Activates the updated Process • Only deploys if there are changes • Automatic versioning • All referenced classes must exist • Watch out for cluster consistency issues • Trigger deploy after update is finished • Spring Auto Deployment • Deployment through API
  • 22.
    Demo: Verified PhoneRegistration Process • Build a microservice, that provides verified phone registration • The phone validation process happens through secret token • Delivered as SMS (mocked) • 60 seconds validity • User should input the received token to complete the process • User can re-generate token (invalidating the old token) up to 3 times • User can abandon the process at any time • Existing microservices (mocked) • SMS Delivery, User Profile, eMail Delivery
  • 23.
  • 24.
    Verified Phone RegistrationREST API • Create phone registration • POST /user/phone/registration/{userId} • Get ongoing phone registrations for User • GET /user/phone/registration/{userId} • Create new token for registration • POST /user/phone/registration/{userId}/{registrationId} • Submit token for registration • POST /user/phone/registration/{userId}/{registrationId}/token
  • 25.
    Flowable Best Practices •Unit & Integration test the processes • Decouple interactions with process engine classes • Store only what is needed in the context • Optimize the process for execution and reliability • Fetch dependencies earlier, do modification later in the process • Serialize complex objects as JSON in context • Move business logic out of tasks • Services might be reusable, but tasks are not! • Add Error Handling to processes • Failed processes must complete at some point • Take care of process history
  • 26.
  • 27.
    Resource Links • FlowableHome • https://flowable.com/open-source/ • Flowable Spring Boot Samples • https://github.com/flowable/flowable-engine/tree/master/modules/flowable-spring-boot/flowable- spring-boot-samples • Phone Registration Sample Application (Presentation Demo) • https://github.com/chavdarb/flowable-phone-registration-demo • Introduction to Flowable • https://www.baeldung.com/flowable

Editor's Notes

  • #5 Orchestration Dedicated system responsible to drive, coordinate and track the process execution (using REST or Messaging). It might be centrally managed orchestration engine or implemented in microservice performing this role. For example Amazon step functions. Choreography No dedicated system has responsibility for driving the process execution This implicates this knowledge is implemented into the micro services, so they must take care to push the process execution
  • #6 Orchestration Too much responsibility given to the Orchestrator Context unification issues Choreography Responsibility unclear Complex processes hard to track Multiple processes might affect the same service driving complexity up
  • #7 Business Domain -> Set of very closely related processes.