Friday, November 15, 2013

Running Scenarios asynchronously/parallel in ODI

Running Scenarios asynchronously/parallel in ODI 

In this post, I will demonstrate how to run interfaces, procedures or packages asynchronously inside a ODI package. 

By default for interfaces, procedures or packages, ODI doesn't provide an option to run them asynchronously (run it in parallel along with the initiating session) . To run them asynchronously, we need to covert interface, procedure or package to scenarios and scenarios can be run asynchronously inside a package or through command.

In this post, I will demonstrate how to asynchronously run the scenarios through package as well as through command.

1. In my tutorial, I am going to use five interfaces which does table to table loading for regions, countries, locations, departments and employees from hr schema to data_target schema. 
2. I will convert these five interfaces to scenarios and run them asynchronously inside a package and through procedure command.

Following are the five interfaces, I will use in this tutorial. I will also use the Jython execution report procedure (created in my previous post  Jython execution report) to write the execution report to file after the parallel run.


Lets convert all the interfaces to scenarios so that we can run them asynchronously. Right click on each interface and select Generate scenario.




1. Running the scenarios asynchronously inside a package 

Lets create a package and drag and drop the generated scenarios into the package in sequence by connecting them with ok in the regions, countries, locations, departments and employees order. After employee, connect it to wait for child session and then to the execution report procedure.

wait for child session event detection will wait for all child sessions (asynchronous sessions) to finish before putting the control to the next step which execution report procedure here.

I have presented them vertically in the diagram since we are going to make them run in parallel. But we will connect them together with ok sequentially.



Lets configure each scenario in the package diagram to run asynchronously. Select each scenario and select the synchronous/asynchronous option to asynchronous for each scenario in the package. By default this mode is set to synchronous.



Also we can set options for the odi wait for child sessions event detection. If we don't specify a session id, it takes the current session id (which is the package in which it is running)

Keywords property can be used to wait for child sessions which match the selected keyword here. Keyword can be set for each scenario in its properties and we can group the scenarios with keywords and wait for child session will wait only for scenario child sessions which match the keyword.


Lets run the package and see the results. Run is successful.
We can see that child session is created for each asynchronous scenario run but no child session session created for the procedure, it is part of the parent session. Each parellel run is assigned a new session id which is child to the parent session id 29001.


Lets check the execution report to see the timing in which each interface triggered. we could see that all the scenarios started at the same time and waitforchildsession event detector waited until the last scenario parallel run has completed.




2. Running the scenarios asynchronously using command in procedure 

In this case, I want to solve a use case where we should be able to run any number of scenarios dynamically in parallel but we will come to know the scenarios only  at run time.

In this use case, a table will be created at run time with all the scenario names with versions which need to be run in parallel. We can simply read this table in procedure source section and run the scenario asynchronously in target section through procedure binding (executed the command in target section for each row from the source query).

This is how the dynamic table which stores the scenario name and version looks like.


Lets create the procedure with source and target sections.

Source runs the query on the dynamic table with scenario and version names.


For each from source query target start scenario command is executed in asynchronous mode for the scenario.


Lets run the procedure and verify the results. we can see that child session is created for each scenario and it ran in parallel.



24 comments:

  1. If anyone of the scenario is failed how it react ?
    I have situation like i am runiing 5 sessions running parallel having wait child session suppose one scenario is failed i dont want to continue but child is checking only status is not equal to running and proceeding tonext steps even though one step is failed.

    ReplyDelete
    Replies
    1. Oracle Data Integrator Tutorials: Running Scenarios Asynchronously/Parallel In Odi >>>>> Download Now

      >>>>> Download Full

      Oracle Data Integrator Tutorials: Running Scenarios Asynchronously/Parallel In Odi >>>>> Download LINK

      >>>>> Download Now

      Oracle Data Integrator Tutorials: Running Scenarios Asynchronously/Parallel In Odi >>>>> Download Full

      >>>>> Download LINK sI

      Delete
  2. I was searching for exactly the same information.Thanks for sharing.Good work.Keep it up.These days Big data is trending technology.“Without big data, companies are blind and deaf, wandering out onto the web like deer on a freeway.”If you are looking for any online courses on big data visit our site.
    Big Data Hadoop Online Training Courses

    ReplyDelete
  3. Quickbooks manage all accounting related aspects for the user of their business.Reach to QuickBooks Customer Service Phone Number and let your problems resolve by experts for you.Call +1-855-9O7-O4O6. For know more vist : https://g.page/QuickBooks-Support-Number-Texas?gm

    ReplyDelete
  4. Nice & Informative Blog ! Are You facing issue in QuickBooks? We resolve all sorts of issues with effective solutions provided by our QuickBooks experts. Just give a call on QuickBooks Customer Service Number 1-855-907-0406.

    ReplyDelete
  5. Thanks for the great publish. If you going through QuickBooks related Error then Dial dial Quickbooks Customer Service Phone Number.

    ReplyDelete
  6. Oracle Data Integrator Tutorials: Running Scenarios Asynchronously/Parallel In Odi >>>>> Download Now

    >>>>> Download Full

    Oracle Data Integrator Tutorials: Running Scenarios Asynchronously/Parallel In Odi >>>>> Download LINK

    >>>>> Download Now

    Oracle Data Integrator Tutorials: Running Scenarios Asynchronously/Parallel In Odi >>>>> Download Full

    >>>>> Download LINK

    ReplyDelete
  7. Nursing jobs in Germany present a wealth of opportunities for professionals seeking to expand their careers in a robust healthcare system. With a growing demand for qualified nurses, positions are available in various settings, including hospitals, clinics, and long-term care facilities. Candidates interested in this field must hold relevant nursing qualifications and often speak German, as effective communication with patients and colleagues is crucial. The roles offer competitive salaries, favorable working conditions, and continuous education opportunities. Nurses in Germany benefit from a strong emphasis on work-life balance, making it an appealing choice for many. Additionally, the country’s commitment to healthcare innovation ensures career growth and personal development. Join the vibrant nursing community in Germany and make a significant impact on patient care!
    https://www.dynamichealthstaff.com/nursing-jobs-in-germany

    ReplyDelete