Introduction to ODI Agents and Creating a ODI Standalone agent

Introduction to ODI Agents and Creating a ODI Standalone agent

In my last post Creating ODI Repository, I have provided the details of creating repository in ODI.
In this post, I am going to talk about ODI agents and how to configure them.

ODI Agent

ODI Agent is a run time component which orchestrates the data integration process. At design time all ODI objects which need to be run are converted into scenarios and those scenarios are submitted to ODI agent for execution. When it receives request for running scenarios  it takes the required source, target connectivity  information from master repository and also scenario , KM information from work repository and starts orchestrating the data integration. Orchestration of ETL process involves, generating SQL statements based ODI object definitions, KM configurations and sending it to remote databases (source, stage or target) for execution. It also collects the status of execution and updates into the work repository.

Agent also exposes public web services which can be used by a java client or BPEL to invoke the scenarios from external application for adhoc synchronous or asynchronous invocations. Agent supports both synchronous and asynchronous callback web services. User need to provide user id, password in the web service payload along with scenario details to get authenticated into ODI and then execute the web service.

There are two types of ODI Agents available.
1. Standalone Java Agent - It is a java process which runs and listens on a particular port. When request is received for a scenario execution, it start orchestrating the scenario and updates the status to work repository. A parent and child standalone agents can be configured in ODI studio to do the load balancing if there is a need to load balance between stand alone agents.

2. JavaEE agent - This agent is part of web logic and listens on a particular port. When request is received for a scenario execution, it start orchestrating the scenario and updates the status to work repository. Load balancing is supported to through web logic clusters for a Java EE Agent.

Configuring  a standalone agent

When it is a odi companion download or complete odi enterprise product download, once the install-able is extracted, agent directory can be found under "oracledi" directory. Whichever machine is identified to run the standalone agent (windows or unix) which has compatible java installed can run the standalone agent. Just zip and copy the agent folder to the target machine and extract it. Under agent folder there are three directories available.

1. bin -  folder has all the utilities and configuration files for the agent.

2. lib  - folder has all the libraries which are shipped with odi and needed for the agent.

3. drivers - folder is used to place any custom user libraries or any new jdbc drivers which are needed for running the scenarios by the agent. 

This is how the bin folder looks like for the standalone agent.  Bin folder has the following types of files.

1. encode - this is used to encode passwords which are to be used in configuration files for security purposes.

2. odiparams and logging-config are used to configure agent and logging levels for the agent.

3. agent start and stop scripts - used to stop and start the agent

4. jps-config-jse - used to configure external security configuration for ldap etc.

5. opmn*  - used to configure the agent in OPMN.

6. command line tools - commandline tools to start, stop and restart a scenario, session, loadplan. startcmd is used to invoke the odi tools like email, ftp etc in command line.

To configure a standalone agent, we need to configure the master and work repositories in the odi params file. (use .bat for windows and .sh for unix). encode the password using encode utility before putting the password in configuration.

Increase the java heap memory for the agent if needs more memory to run the scenarios. Usually if you receive OutOfMemoryError in the logs, we need to increase the heap memory to resolve it.

Close the odiparams and start the odi studio to define the standalone agent in the repository. Go to topology and create a agent.

Provide name, portno, hostname (where standalone agent runs), web application context (used for accessing public web service WSDL) and save the agent.

Now, lets go ahead and start the new standalone agent in the machine where we need to run it.
Go to agent bin directory and run the command agent -NAME=<agentname> -PORT=<portno>.

Now, test the running agent from odi client whether it is able to connect to it successfully.

Lets try to access the WSDL for the agent from the url http://<hostname>:<port>/<web appl context>/OdiInvoke?WSDL and wsdl is accessible. You can see that in wsdl odi supports both synchronous and asynchronous operations for invocation through BPEL or Java.

To stop the agent, run agentstop -NAME=<agentname>

With this we have configured a standalone agent successfully. In my next post, i will demonstrate how to configure ODI standalone agent with OPMN for monitoring and restarting purposes and also how to configure a Java EE odi agent.