ODI 12c: Setting up Standalone Agent

Warning! This post is written by using ODI 12.1.2, in version ODI 12.1.3 there are changes in agent setup. A post about ODI 12.1.3 will be written soon.

Hello all,

In this post I am going to tell you how you can set up a standalone agent for ODI 12c. Oracle provided a new tool for configuring our ODI 12c setup elements, such as WLS, Console, Agent, RAC. But to use this configuration manager you will need to create some schemas for OPSS, AUDIT, AUDIT VIEWER and luckily we have RCU for this task, you may do it manual also. But I prefer automation and I will describe in this post how you do it with RCU.

Important! Notice that even if you have a database which can work with ODI 12c, it can be unsuitable with RCU. I faced this problem, I had Oracle for my ODI repository but RCU needed a newer version, so I installed ODI 12c database.

Remember! The paths I will give in this post are absolute paths for my installations, your path may change depending on where you have installed ODI 12c.

Shortly we are going to do

  1. Create required repository and schemas by using RCU
  2. Create a domain to run agent by using Configuration Manager
  3. Define physical agent on ODI Studio
  4. Run agent
  5. Test it from ODI Studio

1. Creating necessary schemas


You will find RCU tool in /home/oracle/Oracle/Middleware/Oracle_Home/oracle_common/bin for Linux OS and C:OracleMiddlewareOracle_Homeoracle_commonbin for Windows OS. File name is RCU.cmd or RCU.sh depending on OS. Start the RCU, select “Create Repository” and continue.


Fill all the spaces for the database you want to connect, be sure to enter full service name. In screenshot you will see it is filled as “orcl122” but full service name is “orcl122.localdomain”. I took the screenshot a bit early. Don’t let it trick you.


You will need to select schemas for RCU to create, Oracle Platform Security Services, Audit Services, Audit Services Viewer must be selected. When you check Audit Services, Audit Services Viewer and Audit Services Append will be checked automatically. Also in this step you can choose RCU to create Master and Work Repository for you. Since I have already created my repositories long ago I will skip it.


Set passwords for schemas that are going to be created. I will choose to set same password for every schema.


Set tablespace for schemas. If you leave it default, RCU will name them with a prefix which is set on third step. If tablespaces does not exist RCU will create them for you.


Now RCU will show you a summary of settings, check the list if it seem fine click on “Create”. It will take some time to create schemas, tablespaces, tables then it will show you a report. And you are ready to close RCU and continue with Configuration Manager.

2. Creating Domain for Agent with Configuration Manager

Conf Man
Conf Man

Start Configuration Manager, you can find it in /home/oracle/Oracle/Middleware/Oracle_Home/oracle_common/common/bin on Linux OS and in C:OracleMiddlewareOracle_Homeoracle_commoncommonbin in Windows OS. File name is config.sh or config.cmd depending on OS. Select “Create a new domain” and select where you want to put domain’s files. I will leave it as it is.

Conf Man
Conf Man

Next page is Template Selection page, we will need to check “Oracle Data Integrator – Standalone Collocated Agent” and when you check this option there will be two more options checked automatically.

Conf Man
Conf Man

Next step will want you to create a login for WLS. Password should be at least eight characters, including at least one letter and one number.

Conf Man
Conf Man

Select Domain Mode and JDK, I will leave them default.

Conf Man
Conf Man

Now we will select RCU Data option, since we created necessary schemas with RCU. Enter connection information for the database schema where RCU worked on before. Then click on “Get RCU Configuration” and wait for Configuration Manager to read from database. Then click “Next” when it is available.

Conf Man
Conf Man

Enter all passwords and schemas we have created before. Also you will need to fill database connection information again.

Conf Man
Conf Man

Configuration Manager will check for connections if they are working.

Conf Man
Conf Man

Now we need to select components which we are going to need, In this case we will only select “System Components” and “Deployments and Services” and continue.

Conf Man
Conf Man

Set name for the agent I will use OracleDemoAgent for this demo.

Conf Man

Next select the component from dropdown list, which we have created one step ago. Set listener information you can leave port as 21910 it is default for configuration manager. Fill in supervisor user information. And continue, for this demo we will leave settings default on other steps then click on “Create” on last step.

Conf Man
Conf Man

Creation will take sometime after that click on “Next” and “Finish” and we are almost ready to run the agent.

3. Defining Physical Agent in ODI Studio

Run ODI Studio, and login to repository, where you have Topology rights. We will need this rights to create agents.


Go to Topology tab, in Physical Architecture accordion, right click on “Agents” and click on “New Agent” a new window will appear on right panel.


Name the agent as the same name as we selected in Configuration Manager for the Agent component. When we try to run agent, it will search for its name in Master Repository so names should match exactly. Set the port as you set in Configuration Manager. Save the agent.

4. Start up the agent

Open a terminal/command line window and go to the domain folder that you have created. Under this folder you will find bin folder. For me this folders path is /home/oracle/Oracle/Middleware/Oracle_Home/user_projects/domains/demo_domain/bin in Linux OS or C:OracleMiddlewareOracle_Homeuser_projectsdomainsdemo_domainbin in Windows OS. Run the command agent with NAME parameter as following, or as you can see in screenshot.  agent -NAME=OracleDemoAgent

Command Line
Command Line

Agent will start to rise up, it will connect to Master Repository, check for its name, read other configurations and start the service and listener. When agent is fully started, command line will seem like screenshot below.

Command Line
Command Line

5. Test the agent from ODI Studio

Now we have the agent up and running, also we have a definition for it in our Master Repository and a GUI in ODI Studio. It’s time to test if ODI Studio can connect to agent service. Go to Topology tab and open the agent we have created. There you will see a button written “Test” click on it. If everything is right until here you should see the screen below.

Test Agent
Test Agent


We have completed everything now. Agent is up and ready to run your jobs on databases for you.

Thanks for reading and please do not hesitate to write your comments or ask questions in section below.




ODI 12c: After TROUG BI/DW SIG Day 2014

It has been about one week since TROUG BI/DW SIG Day. And I just found some time to write about the event.

First there was ODI, OGG, EDQ and Exadata Integration presentation by Gurcan Orhan, it was a really helpful and compact presentation. Architectural designs, tips and tricks about these tools and general information about them was the content of this presentation.

Then we had a session about Graphical Data Mining on Financial Data, this was a bit boring and out of the concept actually. It has an acamedic perspective and presentation of a tool about graphical data mining. And unfortunately they ran out of time and had an overtimed session about 15 minutes, which caused my session to start late.

Third, session is mine, it was just before the lunch break, and unfortunately I had to start late. So it went real quick and I was excited since it was my first presentation in this area. I believe I forgot some of the key points I had to say. You can find the presentation below.

After lunch break we had series of very useful sessions. Starting with Hadoop & DWH by Ersin İhsan Ünkar, which replied some false myths about hadoop, and show a road map about Hadoop in DWH. Also some use cases about where to position Hadoop in enterprise DWH projects.

Sessions continued with Compression In Hadoop by Selahattin Güngörmüş. In my opinion this is the most useful session. He talked about compression algorithms, how to use them in Hadoop environment, pros and cons of compressing files in Hadoop.

Last session was about Analytic Functions for DWH by Emrah Mete. He talked about some useful functions, which most of the DWH developers use I believe. But it was nice to see most used and most useful functions all together in one session.

At the end it was a nice and useful day for DWH developers, there were really nice presentations. Thanks TROUG for the organization and also giving me chance to speak at such a nice event.


Hello all,

On 3rd April 2014 TROUG is organizing BI/DW SIG Day at İTÜ Arı3 Teknokent. You can have detailed information on : http://www.troug.org/?p=684 and you can register for the event on eventbrite : https://www.eventbrite.com/e/troug-bidw-sig-meeting-tickets-10986690487

I willl be there as a speaker, my session will be about Requirements, Installation and New Features of ODI 12c, hope to see you there.

My presentation will be up in this post on 3rd evening. 
You can reach presentation by this link.

Mark your calendars, register for event and have a nice day.

ODI 11g: Implementing Loops

While using ODI to implement your ETLs, you may need to have loops. Let’s look at the examples, where I will implement loops that will iterate n times (for loops) and loops that will iterate while they ensure the condition. (while loops)

For Loop

In programming we implement for loop as follows,

for (i = 0; i < 10; i++){

This is a simple loop which iterates ten times, if we parse the part in the parenthesis we can see in the first part we assign a value to a variable, second part we define the condition and the last part is change of variable value per iteration.

In ODI 11g we can implement this as follows:

1- Create a variable
I created a variable called V_FOR_LOOP which is numeric and does not have a refreshing code.

2- Create a package
I create a package and name it as P_FOR_LOOP, I will put a screenshot of package’s final status when we complete all steps.

3- Set variable
Set a value to our variable V_FOR_LOOP as an initialization value. I will set it as 0. Also name the step as set initial.

4- Evaluate variable
Evaluate V_FOR_LOOP against iteration condition. I will use “less than 5” as iteration value. You can choose between the options as you wish or your requirement. Name step as Evaluate Value.

5- Place your statements
Now it is time to place your statements which will iterate. I will only put one interface.

6- Increment your variable
Increment your variable one step using SET VARIABLE object’s Increment option, I will increment by one and name this step as Increment.

7- Connect your Increment step to Evaluate Value step
Until this step every object was connected to its following object with an OK line, now connect Increment to Evaluate Value with an OK line. Now it will go back to evaluation and iterate until the evaluation is false.

Here is how our package looks in final form:

For Loop Package
For Loop Package

And the operator screen when we run the package:


For Loop Operator View
For Loop Operator View

As seen above steps numbered 1,2,3 repeats 5 times, then Evaluate Value runs one more time, decides that V_FOR_LOOP < 5 is not true enough and package finishes its run.

While Loop

In programming we can implement while loop as follows:

while (flag == true){

So this will iterate unknown times until its condition becomes incorrect. Confession time : I have to admit that I have never felt need of using while loop in ETL/ODI but you may need.

Before implementing this step-by-step, I created a table includes two columns c1 and flag, where I will use flag as my condition. My data is as follows :

C1 F
— –
1 T
2 T
3 T
4 T
5 F
6 T
7 T
8 T
9 T
10 F

Now let’s implement while loop:

1- Create a variable to hold flag value
I create a variable called V_WHILE_LOOP which is alphanumeric and refreshing by : select flag from variable.test where c1 = #V_FOR_LOOP
I will use my V_FOR_LOOP to select flag values, in this sample case. Your case will contain different logic than this sample for sure.

2- Create a package
I create a package named P_WHILE_LOOP.

3- Set Variable (in my case)
Since I am refreshing my flag depending on  V_FOR_LOOP, I set this as first step.

4- Refresh Flag
Refresh your flag variable.

5- Check Flag
Evaluate flag variable.

6- Statements
Place your statements, I will put my sample interface and also I will increment V_FOR_LOOP as I will need this to reach an invalid flag.

7- Set your connections
Until the end of your statements every step will be connected by an OK, when you reach the end connect it to Refresh Flag step, so you will refresh, check and start your statements again and again until flag is false.

Here is a view of package :

While Loop Package
While Loop Package

And the view from operator:


While Loop Operator
While Loop Operator

You can see it hits the end when we refresh flag for the 5th time since it will return F as flag value, which is not suitable to our condition.

So here we are at the end of the post, now with the knowledge of “How to implement loops in ODI 11g”

Thank you for your patience to read, and if you have any questions or comments please drop a comment and I will read (and reply if it’s a question) it for sure.

ODI 12c: First Look and Repository Creation


After a long pause on blog, here I am again. Oracle Data Integrator 12c is finally available for everyone to download. So in this post I will discuss about my first impressions and I will explain how to create repositories, both master and work. Actually it is pretty simple and almost same with 11g which I told in this post.

So first impressions, when you download ODI 12c through this page, you will get odi_121200.jar (numbers can differ with time since it’s version number) and some opatches bundled with it. Actually it is a bit disturbing for me to have a jar file which is 1.8GB. I’d like to have an exe for Windows.

Anyway I had some problems with running this jar also, first I tried it on my VM which has 32bit Windows 7 and got an error that states it could not reach jar file. So I moved to my physical machine that is 64bit Windows 7 and OUI could not recognize the platform and exited everytime, until I download and install Java 1.7. So after solving the problem with Java, I moved to my VM again to solve other problem where it came out that my path is problematic, since my user name is Canburak Tümer, space created a problem to reach file. I created another user without space that can run the installer.

Finally I could see the installer UI. It was a pretty straight forward installation, I just selected enterprise and went on. After installation, I ran the ODI Studio, it has a really clean and elegant splash screen and it asks to migrate any user settings from old installations. After splash screen, ODI workbench has been load:

ODI 12c Start Screen
ODI 12c Start Screen

Creating Master Repository

As I mentioned before, repository creations are almost same with ODI 11g, we will start by clicking File > New and we will see screen below:

Master - 1
Master – 1

Select “Master Repository Creation Wizard” in ODI tab and click “OK”.

Master - 2
Master – 2

You will see screen above, where we will enter database information, schema where we will create repository and DBA user to run some of the creation scripts.

Master - 3
Master – 3

Define and confirm password for SUPERVISOR user. DO NOT FORGET THIS PASSWORD UNLESS YOU HAVE ANOTHER USER WITH SUPERVISOR PRIVILEGES. For this reason many ODI developers/admins make this password “SUNOPSIS” as an old habit. I prefer to have it as “SUPERVISOR” in my VM and personal development environment.

Master - 4
Master – 4

Select password storage as you wish. Then click finish, it will run scripts now to create master repository, it took around 4 minutes in my VM, probably it will take around 2-3 minutes in your physical machines. Now it’s time to create a connection to master repository.


Click on “Connect to Repository” then click to green plus in the pop-up window, then fill required information in the form. Use SUPERVISOR as ODI user and DB user which you have created the master repository with. Make sure you have selected “Master repository only” radio button. Then click “OK”.


ODI 12c will ask you to if you want to keep passwords in a secure wallet with a master password. I do not have enough information about this wallet yet, but I will learn and write another post about it. I prefer the less secure way which does not include the wallet. Now we have master repository and connection to master repository. Now it’s time to create work repository.

Creating Work Repository

To create work repository, connect your master repository then go to Topology tab and expand Repositories menu.

Work - 1
Work – 1

Right click to “Work Repositories” and click to “New Work Repository” from menu.

Work - 2
Work – 2

Insert connection information of schema which you want to create Work Repository in. (I had a problem with this step, actually I wanted to use odiw_c user but ODI 12c keeps filling the form in upper case so it gives invalid credentials error.)

Work - 3
Work – 3

In final step, insert repository name and select repository type.

Work - 4
Work – 4

You can also define a password for repository, which is different from ODI user password or DB user password. This password is just to secure the repository connection. When you click “Finish” it will run scripts to create work repository and will ask you if you want to create a connection to work repository. It will create a connection without ODI user information. So you will need to edit connection to insert ODI user information.

After all these steps we have installed ODI 12c and setup both master and work repositories for our environment. And we have a final view as below :

We are ready to develop.
We are ready to develop.

Now, it’s time to create our topology connections, models, projects; import or reverse engineer data sources; develop mappings (new name for interfaces), packages and more.

Welcome to ODI 12c, keep following my blog for further posts and please do not hesitate to contact me through comment form below.




ODI 11g: Step by Step Creating a User

In the last post here: http://www.canburaktumer.com/blog/odi-11g-step-by-step-master-and-work-repository-creation/ , we have created our master and work repository. As we only had one user, which is SUPERVISOR, we need other users to have a safe environment. For a safe environment give every user privileges, only  what they need.

For example, an OPERATION ADMIN role should not be able to use designer tab to edit interfaces, packages and procedures. A DEVELOPER role should not have privileges to create or alter users in security tab.

Creating a Typical DEVELOPER

So now let’s create a DEVELOPER user now, who will have the rights to Designer , Operator and Topology tabs. First connect to your master or work repositories with SUPERVISOR.

Create a User - 1
Create a User – 1

Expand the Users accordion to see user list, in our case there is only SUPERVISOR in list. Click on the little man with plus on it, a pop-up menu will appear with only one option; New User. Click that option.

Create User -2
Create User -2

So this is the main screeen where you enter information of a new user. There is some little details I like to tell about. You can create an account with an expiration date. So when the day comes, user will become an invalid user. And its icon will be red. (See second picture below. I created DEVELOPER user with expiration date, and let it expire to have a screenshot.)

If you check Supervisor checkbox than this user will have SUPERVISOR privileges, and actually you don’t have to assign any other privileges to this user as it has all of them.

Last detail is the Password tex box, which you can not edit. There is a button below text box, which says Enter a Password. When you click it you’ll be able to create a password for user. Password also has an expiration option. You can create passwords that expire to increase level of security. Also there is other password policies you can set, but we will be talking about it in next post. By default ODI has only one password policy, which is ‘Passwords should have six or more characters.’

Create User -3
Create User -3
Expired User
Expired User
Password Window
Password Window

After selecting a password, we save and close and user appears in our user list.

Assigning Privileges to a User

Now it’s time for our user to have privileges a developer will need. We will assign privileges by using predefined roles, there is also a longer way to do it; giving privileges object by object like giving ‘view interface’, ‘edit interface’, ‘new interface’ privileges. But it is a longer method also it requires more attention and knowledge, so in this post we are going to use roles, as we are learning about the basics.

If you have used ODI before, but not assign a role to a user, it may become confusing in the ways of ODI’s user experience perspective. In ODI, when you are doing something in accordions, then usually you click, right-click or double click and open the edit window and do whatever you want. However, in security tab while you are assigning a profile to a user, you simply drag profile on to the user. It is a simple way to do it, on the other hand it is not the obvious way in terms of ODI’s user experience.

So I will drag and drop CONNECT, DESIGNER, METADATA ADMIN, OPERATOR and TOPOLOGY ADMIN privileges to our DEVELOPER user. Final view should be like below:


If I should briefly explain these privileges:

CONNECT : The basic profile to connect an ODI repository, it is like CREATE SESSION privilege on Oracle Databases. It has some more rights like viewing some objects. You can see objects if you click + next to CONNECT.

DESIGNER : This is where our codes are. There are interfaces, procedures, scenarios, packages, projects, variables in designer. And designer profile gives privileges to create, edit, delete ability for these objects.

METADATA ADMIN : This is actually about Model accordion. Model is the place where data stores’ metadata are being held. Like tables, files, web services and other data stores. METADATA ADMIN gives you ability of create, edit, view and delete of Model objects or Model folders.

OPERATOR : Operator has sessions’ information, load plans, scenarios … etc. Actually running codes, and their sessions are being held in this tab. By using operator you can see errors, see successful runs, re-run scenarios…

TOPOLOGY ADMIN : Topology is where you have connection information to data stores, TNS info for a table’s database or path of a file or URL of a web service. All connections are stored here. We will see topology tab in detail in upcoming posts.

So now we learned how we create a user, how to assign privileges to it. But how can we connect to repository with our new user. OK, nice question, let’s create a connection for our new user.

Creating a Connection

So we have already created connections, so I will make a quick review of it.

Click on connect to repository, you should see login window, click on green plus to create a new connection. Now enter your DEVELOPER user’s information like you see in figure below.

Connection of DEVELOPER
Connection of DEVELOPER

Our connection is ready.

Next post will be about security tab in detail. Before we begin to set up our environment for development, we should secure it.

Questions in the comment area please.




ODI 10g: Installation Errors

Installing ODI 10 on Windows 7

When you are trying to install ODI 10 on Windows 7, it will check Windows version, and raise an error like below:

“This version of operating system is not supported. Supported versions : 4.0,5.0,5.1,5.2,6.0. Your version: 6.1″

Here is the solution:

  • Open file %ODISETUP%setupWindowsDisk1installoraparam.ini
  • Find the lines below and add “,6.1″ at the end. (without quotation)
    [Certified Versions]
    #You can customise error message shown for failure, provide value for CERTIFIED_VERSION_FAILURE_MESSAGE
  • After modification line should be like this:
  • You are ready to install ODI 10g on Windows 7 now.

Source : http://gurcanorhan.wordpress.com/2011/02/07/installing-odi-to-windows-7/

ODI 11g : Step by Step Master and Work Repository Creation

Hello all,

So first post, first steps. We are going to create master and work repository for your Oracle Data Integrator. I assume that you have already have an ODI installation up and running and also an Oracle Database.

First I am going to create schemas for my repositories, you can put your repositories in one schema but it is common to have them in separate schemas. If you run the script below you will have two schemas for your repositories with all privileges.

create user ODIM_BLOG identified by ODIM_BLOG;
create user ODIW_BLOG identified by ODIW_BLOG;
grant all privileges to ODIM_BLOG;
grant all privileges to ODIW_BLOG;

Creating Master Repository

So if your schemas are ready, we can continue with the master repository. In the main ODI screen, go to File > New.

Master Repository Creation Step 1

Now you’ll see three options, we’ll continue with Master Repository Creation Wizard, select it and click OK to continue.

Master Repository Creation Step 2

Now it is time to setup our connection to database, you will need DBA connection to setup your repository. Fill in the required fields, you can see my blog environment’s information below. A little caution here: if you are going to use SYS user as DBA account, you have to fill DBA User area like ‘SYS as sysdba’ otherwise it will not connect. If you are using another DBA account then you don’t need to declare ‘as sysdba’. Hit the Test Connection button to see if you can connect.

There is another text box which asks for the repository ID, be sure to remember this ID, as you’ll also be asked for another ID while creating work repository. And it is important that all repositories have different IDs. As ODI does every operation with ID’s, it’s crucial not to have duplicated IDs.

Master Repository Creation Step 3

Next step is the generating a password for SUPERVISOR user, this user will be our ODI admin until you will create another user with SUPERVISOR privileges. It is an old tradition to set this user’s password as SUNOPSIS, it is also a tribute like behavior for ODI’s creator Sunopsis. I set it as SUPERVISOR to remember it easily.

Master Repository Creation Step 4

Here it goes, now we will wait until ODI finishes its job. If you check your master repository schema you can see it is being populated by SNP_% tables. SNP prefix is also a heritage from Sunopsis.

Master Repository Creation Step 5

And here we have created our master repository. Now it is time to setup a connection and create a work repository.

Master Repository Creation Step 6

Creating a Master Repository Connection

So for connecting master repository, click Connect to a Repository in ODI Home Screen. A little window will pop-up. This is the window where you can select between your predefined connection, edit them or create new connection. As we do not have any predefined connection, we will create one from scratch. Click on +(plus) sign to add a new connection.

Master Repository Connection Step 1

Now this is the connection details window. First blank will be the alias for this connection so it should be something to remind you to which repository you are connecting. First login information is for ODI user we are going to use, which is the SUPERVISOR in our case. And the second login information is for the Master Repository’s database schema, this is ODIM_BLOG in our case.Make sure Master Repository only radio button is checked before testing the connection. Because we do not have any other repositories yet. Test your connection then connect your master repository with SUPERVISOR. It’s time to move a step further.

Master Repository Connection Step 2

Creating Work Repository

When you are connected to aster repository go into Topology view and extend the repositories accordion. Right click on Work Repositories and click on the New Work Repository on pop-up menu.

Work Repository Creation Step 1

Enter your schema information, test your connection and move on. In our case we are going to use ODIW_BLOG schema as we want our repositories to be separated.

Work Repository Creation Step 2

Give a name and ID to your work repository. DO NOT forget to give a different ID from your master repository’s ID. Select your work repository type in this step.

There is two types of work repositories: Development and Execution. Execution repository does not have a development ability and does not have an active Designer tab. It only holds, scenarios, load plans, sessions and topology information. Development repository has an active Designer tab, which adds this repository to hold, create, edit, delete ability of interfaces, packages, variables, scenarios, knowledge modules, model objects like tables and views.

We will get into deeper about development and execution repository in another post. So I will make it short here.

Work Repository Creation Step 3

Now let ODI to finish creating work repository.

Work Repository Creation Step 4

Now ODI will ask if you want it to create a connection for new created work repository.

Work Repository Creation Step 5

When you click Yes, it will ask for connection name, which we said it is connection alias. And when you click OK. It will create a connection.

Work Repository Creation Step 6

Connecting to Work Repository

Work Repository Connection Step 1

ODI has created a connection for us. When you disconnect your master repository. (Under ODI > Disconnect) You can connect to work repository, click on Connect to a Repository, select your work repository connection from dropdown list. Password field will be empty, enter your SUPERVISOR password and click connect.

As we do not have any other user yet, we will connect to work repository by SUPERVISOR user.

Thanks for your patience to finish reading. Next ODI post will be about creating a user in detail, and assigning profiles to a user.

If you have any questions, please use comment section to ask me.