ODI 11g, ODI 12c : What’s an Agent?

What is an agent?

An agent is a java process which usually located on server and listens to a port for incoming requests. It runs the requested scenario, reverse engineers requested datastores etc.

When a job submitted through ODI Studio GUI or through startscen.sh agent gets scenario from work repository and topology definitions from master repository, combines and converts them into a runnable job, usually consisting more than one code block. Then it sends code blocks to destination environments, which may be DB servers, file servers, hadoop name node etc. Finally agent gets job statuses from these environments and writes into work repository tables for us to see from Operator tab of ODI Studio.

Agent diagram from Oracle A-Team Blog

Agent Types

Standalone Agent

It is the basic agent of ODI. It does not require an application server like JEE Agent. It is easy to configure/start/stop this agent from shell. Since today I’ve always used this agent and never tried other versions. A-Team article says this is the most light weight and low footprint choice.

JEE Agent

“Java Enterprise Edition” agent, which requires an application server, in most documentation you can see the name of WebLogic Server since it’s another Oracle product. (Some search results: depending on this CertMatrix of Oracle http://www.oracle.com/technetwork/middleware/data-integration/odi-11gr1certmatrix-ps6-1928216.xls ODI 11.1.1.7.0 only supports WLS and does not support Tomcat or other application servers. You may -or may not- configure them to run together but it is not supported.) This agent is first delivered with ODI 11g. And still exists in ODI 12c.

Some pros of JEE agent which are written in A-Team blog are:

  • High availability : Through Web Logic Server’s cluster architecture, even a node is down agents may run on other nodes.
  • Configurable connection pooling : Connection pool can be configured through WLS.
  • Monitoring : Oracle Enterprise Manager can monitor, configure, alert, manage ODI JEE Agents. But I believe, there is a plug-in to be installed to achieve this tasks from OEM.

Colocated Agent

The newest agent type which has arrived with ODI 12c. This type is like a combo of other two types. Agent is a standalone agent, but can be monitored and configured through WLS. Unfortunately it does not take advantage of connection pooling, high availability. Our agent will be in WLS domain, can be managed through WLS and that’s all. It is lighter than JEE Agent. In my opinion companies which prefers JEE Agent as production agents can choose colocated agent as their DEV/TEST agent.

Agent Types diagram from Gerard Nico's website
Agent Types diagram from Gerard Nico’s website

Where to locate an agent?

To decrease network I/O it is better to locate agent to target DB server. Since agent submits code to DB Engine, it is better for them to be on same machine. Don’t forget that ODI is an E-LT tool. Which means it will load data into target server, then it will transform your data. So most of the load will be on target server. Which also means most of the codes will be submitted to target server.

Also since an agent is a local java process, agent will write files to the machine, which it is set-up on. If you have a file server seperate than DB server, then it is better to have another agent on file server to handle read/write file processes. Or to mount file servers directory to DB server as a directory and setting up only one agent is another solution.

Also these solutions will prevent any firewall related problems.

 

Thanks for reading, don’t forget to share & comment.

 

 

Resources I’ve read before writing this post:
http://www.ateam-oracle.com/odi-agents-standalone-jee-and-colocated/
http://www.ateam-oracle.com/understanding-where-to-install-the-odi-standalone-agent/
http://odibeginnertutorials.blogspot.com.tr/2013/11/2-introduction-to-odi-agents-and.html
http://www.odigurus.com/2011/10/defining-odi-agents.html
http://gerardnico.com/wiki/dit/odi/agent

[Random Thoughts] 7 Things to do Before Graduation

1- Erasmus Exchange

Erasmus is an European Union supported exchange programme. For one semester or for one academic year students go abroad to study there. EU is paying students some scholarship, which is quite enough to live. All the ask for is to success your lectures, even not all the lectures, just some percent of them is enough.

Erasmus was one of the best experiences I’ve ever had till today. I started to stand on my own feet. I learned to keep my budget. I learned to properly cook, clean, iron etc. all by myself. I learned about Italian culture, (some German culture also). I taste different foods, experienced different teaching methods, learned new languages, got new friends, saw too many cities with little money. And became a confident, self-aware guy.

In Turkey, you need a GPA and also you need to pass some exams to participate in Erasmus programme. My advice is, whatever it takes for you participate in Erasmus exchange.

Life changing experience
Life changing experience

Erasmus really changed my life, and the way I see the life. I feel more open-minded, more independent, more welcoming now. And it is too hard to stay in a place for a long time without traveling now.

2- Part-time work or long-term internship

Internships-Wordle-v3

Having a part-time job or a long-term internship will make you more experienced. especially a work in the branch that you want to become professional will be a good choice.

You will taste making money, you will become less dependent to your family.

You will start building your network. You will learn the difference between university life, university projects and the real life, real projects.

You will have some experience to write on your CV. And probably you’ll get your first job, if you are lucky and hard working enough.

3- Join student clubs & organizations

Students clubs & organizations will improve your network and soft skills. I got most of my non-IT friends thanks to student organizations. I’ve been in two different student clubs through my university life, the first one is EHAVK (Ege University Aviation Association Paragliding Team). I’ve been in this association for 5 years. I started my journey from the base to the top. From beginner pilot to instructor & tandem pilot. This association taught me about organizing events, leading a group, instructing to people, team work, presentation skills.

I am an ex-AIESECer
I am an ex-AIESECer

My second association was AIESEC. I’ve only stayed for one year in AIESEC, but it improved me alot. Because it gave me lots of responsbilities, I had the roles of Project Manager and Communication Responsible of Local Committee. I was leading a team of 3 persons. We were organising events, recruiting new members, publishing newsletter, keeping our website updated, participating in weekly meetings. It taught me a lot.

Being a member of student associations will teach you lots of things, improve your network, let you improve yourself in every manner. Try it.

4- Go to events

One of my regrets about my university life. I’ve begun attending to events in my job life. But there is never enough time to participate in the events I want.

Networking-events-blog-image

Attending to events will let you enlarge your network, stay updated about your professions matters and development. My suggestion for IT sector is to attend events like hackathons, Google Developer Group events, hackerspaces/makerspaces and vendor events.

5- Chose graduation project / thesis wisely

Another regret for me.

Your project/thesis will effect your job. It will effect your chance of getting selected for jobs and Master/PhD programmes. Choose your thesis/project on your branch, make contribution to this area.

thesis-binding-dublin-1bc

6- Have fun

remember-to-have-fun  Study hard, work harder but have fun even harder. It was our motto in AIESEC “Work hard, party harder.” It is one of the best  proverbs I’ve ever heard. Party hard, have fun hard, enjoy your university and social life while you can.

Keep in mind that these days will not come back. And there is another thing, every person I met says they were richer when      they were students. Some how it is true. I was not earning money like I am doing today. But I was living a better life. Also I had much more time for everything.

While you still have time and money, enjoy your life. Get out with friends, get crazy.

7- Move out

I also suggest you to move out if you can. Leaving without your parents  will teach you so much. You will learn to plan a budget, being tidy, cooking, cleaning, washing, ironing etc etc. You will stand on your own feet. You will face with bitter truth of bills, rent, house maintenance etc. But you’ll grow up by handling them. You’ll get mature.

543x269xmoving-out.png.pagespeed.ic_.D_d-NFibnY

That’s all from me. Thanks for reading.
Share, comment, join conversation.

 

 

 

Monthly Challenges : June ’15 Weight Loss and Standing Desk

Hello all,

I had two challenges for June and luckily I completed both of them.

1. Lose 3 kg.s in a month.

This one was easy, I will not get into details. I was 87.6 kg.s, I cut off eating junk food, I began to count my steps and put a target of 10.000 steps daily. And at the end I was 83.8 kg.s. I’ve reached and passed my target.

2. Work on Standing Desk

This was the real challenge. Actually,  when I am working at home, I am already using my DIY standing desk,  but I am working at most two hours at home. So it is not challenging.

Standing desk in the office has at least two challenges. Which are a) convincing manager and colleagues, b) really standing for 8 hours straight.

For the first part, luckily we were moving to a new office, I had the most isolated part for myself. So I will not get too much attention 🙂 And my manager is an open-minded man, he is also very friendly (once we were colleagues). Convincing manager was not that hard. It took more time for my colleagues to get used to me 🙂

Second part was hard sometimes. When you google ‘standing desk’ and read some blog posts about them. You will see some tips, which come true when I experience it. First of all just standing and working like that is not possible. I often transfer my weight from one leg to another, take some small walks in office, have some small sitting pauses when I am taking notes or a friend ask for some help.

Shoes are really important when you stand all day long. Hard soled shoes will make your day a nightmare, some people use yoga mats to soften the floor. I choose more soft soled shoes. This way my feet don’t cry for help.

I also built my standing desk in office, so it is a DIY standing desk like the one at home. This have some pros, like I have two spare shelves to put my notebooks, pens etc.

My DIY Standing Desk
My DIY Standing Desk

Now, I am happy with the result.

Have a look at also my post on LinkedIn about standing desk : https://www.linkedin.com/today/author/130746977
Leave comments, keep in touch.

New Series : Monthly Challenges

Hello all,

I am going to kick off new blog post series. By this way, I believe I will also keep the blog updated. The series are about Monthly Challenges, I’ve already completed my first challenge and its post is coming today.

I will not reveal monthly challenges until the month ends. Every challenge will start on first monday of the month and will end on last day of the month. So I may have a few days between two challenges to rest. 🙂

I got the idea from this blog post (Turkish), and he wrote that he was inspired by Matt Cutts.

Challenge posts list (will be updated every month) :

June ’15 Challenge : http://www.canburaktumer.com/blog/monthly-challenges-june-15-weight-loss-and-standing-desk/
July ’15 Challenge : http://www.canburaktumer.com/blog/monthly-challenges-july-15/

Random Thoughts: Generation Y and Job Hopping

Nowadays I hear a lot complains about my generation: Generation Y. People say it is hard to keep us happy. We become upset quickly, we expect too much etc. etc. So in this post I will try to focus on this topic with an internal view.

Generations

Let’s look at the “generations” closer. Scientist divided human race to generations, they’re separated by birth years and every group has common thought and behavior which differs from other generations.

Generations
Generations

When we check the generation list of western world (America, Europe) from Wikipedia we can see the list below:

  • Lost Generation (1883 – 1900): Who fought in WWI.
  • Greatest Generation (1901 – 1924): Who fought in WWII.
  • Silent Generation (1925 – 1942): Who fought in Korean and Vietnam wars.
  • Baby Boomers (1943 – 1960): Generation born after WWII is finished.
  • Generation X (1960 – 1980)
  • Generation Y (1980 – 2000)
  • Generation Z (2001 – …)

Last three generation does not have any distinctive historic event, they only have date intervals given by demographers. But I will write some common behaviors for these three generations depending on my observations in my home country, Turkey.

Generation in Workforce
Generations in Workforce

Generation X members are mostly have a stable professional life. Most wanted jobs are civil servant, branch manager in a bank or officer in army. They are hardworking in the sense of body power. For sure they are not lazy but they afraid of changes. They’ve seen military interventions in 70’s and 80’s. So they want a guaranteed lifestyle. They start to work in somewhere and usually get retired from the same organization.

Generation Y is the main topic of this post so I will have a detailed text below about them and their professional life.

Generation Z members are still in high school or in college. They were born into technology. They are mostly lazy because of “smart” devices. They are getting less social and more connected day by day. I wonder about their professional lives.

So, what Generation Y wants?

Generation Y is more free then Generation X. It is certain that we have much more possibility and opportunity than Generation X. We were born when there were homeworks done by help of encyclopedias, then we evolve with technology. First come the PCs and then internet connection. And our lives became easier and faster. That’s why we value and love the technology.

Members of Generation Y have an innovative and entrepreneurial soul. Early members of Generation Y earned some easy money using and developing technology. So late comers believed that it is easy to get rich and famous. And that’s what they wanted; become rich and famous easily.

Wishlist of Generation Y
Wishlist of Generation Y

But we’re raised by Generation X. We’ve raised with the belief of guaranteed standards is best thing in our lives. Entrepreneurship is a risk, and taking risks is stupidity. But at the same time we want money and fame. So we became the generation of white collars. We believed, partly guaranteed life standards, money and fame can be achieved by working in enterprise companies. That was true until a certain point in Turkey.

After crisis in 1999 and 2011 enterprise world renewed their budgets with strict ones. And to me, it is the point where they lost the Generation Y.

Why Generation Y changes job quickly?

Because we are not as stable as Generation X members. They’re determined. When we see that we cannot go any further in the organization we are in, we decide to quit simply for a better one but Generation X waits patiently even until their retirement.

Should I stay or should I go?
Should I stay or should I go?

I will give examples from my short professional life. It is short but it has enough job changes to use. It has been a little more than three years and I am in my fourth organization. Let’s have a look why I quit from every single one:

  1. Part-time ETL developer in Turkcell Technology, I was waiting for a full time position, but since the budget was cut it was a low possibility to find a full time position inside. My manager was asking for me to wait but I was already graduated and wanted a serious job with a stable paycheck to pay my rent and other expenses. (13 months)
  2. Java developer in OBSS, OBSS is really one of the best places for a consultant to work. But I wanted to become an expert in data warehouses, ETL and data mining so I changed my job when I got acceptance from Oracle Consulting (2 months)
  3. Associate Consultant in Oracle Turkey Consulting, It was absolutely a dream job and a dream company for me until I became a part of it. It seems like it is the best source that I can learn DW/ETL so I made a sacrifice from my paycheck (it was a mistake I admit.) and I accepted a lower paycheck then my job in OBSS. I thought I am going to learn so many new things, I will become an expert in DW and Oracle. Unfortunately reality is not like that, I was alone in customer location, there were no seniors to ask questions when I lost in code. I began to feel I am not going any better and also I was losing money since the salary was not a competitive one. I decided to quit and start to search for consulting firms, where they pay you more money but no training. I was thinking to save money to pay for my own trainings to improve myself. Then I found my next job. (15 months)
  4. BI & DW Specialist in MAKEIT Software & Consulting, finally the job I love and a competitive enough salary to give me a life standard, and company also encourages me to improve myself. Encourages me to present in events, prepare training materials for new-grad employees, give challenging projects and enough support when I feel lost. (9 months and counting up)

Looking through my experience, I can say Generation Y aims something and when it feels things are not working, it changes strategy to reach its aim.

My aim is to become an expert in DW, ETL and Data Mining as I said. So I always took the steps which is better in my path to my aim. I worked with an AceD (Mr. Gürcan Orhan) and tried to learn as much as I can while I was a part-time, the for a better future I felt I need to move to Oracle and I did, there I saw I am not going anywhere and I changed tracks for the destination. There is only one thing that did not change through these three years, I kept working hard to learn new things and improve myself, in every free time I made demos with new tools, I wrote blog posts, I prepared presentations, I read others blogs and presentations.

Conclusion

To sum up I can say, Generation Y members have an aim, which is a guaranteed life standard and fame. And they will change the path they walk when they fell they cannot get any closer to their aim. For Generation Y work life is not aim of the life, it is just a tool to reach to their aim. It is like a long flight to destination we may need to change planes on the way.

From graduation to life aim.

 

Thanks for reading and please share your thoughts in comment section.

You can also find this post on LinkedIn.

Sources for images:






Android : MAKEIT Rehber Project

Hello,

Last night I developed a new Android application for inhouse use of my company. Since we are a small company which aims to get bigger, our population keeps growing. This brings some problems with it. For example, some older employees don’t know about newcomers and their contact information.

One day I needed to call a new employee and I realised I don’t have her contact information. And before that point I was not aware of situation. Then I decided to make a phonebook that reads information from a shared file on internet, which our HR will update when somebody joins or leaves the company. So every employee would have every other’s contact information.

This is how MAKEIT Rehber was born. – Rehber means phonebook in Turkish. – Yesterday night I developed the first version of application, where the contact info was hardcoded, then added reading a remote file, then I added sending SMS or Whatsapp messages directly and lastly I added my own open source library : tellal to send notifications through MAKEIT Rehber. This notifications can be just new version notifications or something about company.

There is still some development to do. But it has more than enough abilities now.

Below you can find a screenshot.

MAKEIT Rehber
MAKEIT Rehber

 

Thanks for reading, please do not hesitate to ask your questions or comment on my posts in the comment area you can find below.

Have a nice day.

ODI 12c: Setting up Standalone Agent

Warning! This post is written by using ODI 12.1.2, in version ODI 12.1.3 there are changes in agent setup. A post about ODI 12.1.3 will be written soon.

Hello all,

In this post I am going to tell you how you can set up a standalone agent for ODI 12c. Oracle provided a new tool for configuring our ODI 12c setup elements, such as WLS, Console, Agent, RAC. But to use this configuration manager you will need to create some schemas for OPSS, AUDIT, AUDIT VIEWER and luckily we have RCU for this task, you may do it manual also. But I prefer automation and I will describe in this post how you do it with RCU.

Important! Notice that even if you have a database which can work with ODI 12c, it can be unsuitable with RCU. I faced this problem, I had Oracle 11.2.0.3 for my ODI repository but RCU needed a newer version, so I installed ODI 12c database.

Remember! The paths I will give in this post are absolute paths for my installations, your path may change depending on where you have installed ODI 12c.

Shortly we are going to do

  1. Create required repository and schemas by using RCU
  2. Create a domain to run agent by using Configuration Manager
  3. Define physical agent on ODI Studio
  4. Run agent
  5. Test it from ODI Studio

1. Creating necessary schemas

RCU
RCU

You will find RCU tool in /home/oracle/Oracle/Middleware/Oracle_Home/oracle_common/bin for Linux OS and C:OracleMiddlewareOracle_Homeoracle_commonbin for Windows OS. File name is RCU.cmd or RCU.sh depending on OS. Start the RCU, select “Create Repository” and continue.

RCU
RCU

Fill all the spaces for the database you want to connect, be sure to enter full service name. In screenshot you will see it is filled as “orcl122” but full service name is “orcl122.localdomain”. I took the screenshot a bit early. Don’t let it trick you.

RCU
RCU

You will need to select schemas for RCU to create, Oracle Platform Security Services, Audit Services, Audit Services Viewer must be selected. When you check Audit Services, Audit Services Viewer and Audit Services Append will be checked automatically. Also in this step you can choose RCU to create Master and Work Repository for you. Since I have already created my repositories long ago I will skip it.

RCU
RCU

Set passwords for schemas that are going to be created. I will choose to set same password for every schema.

RCU
RCU

Set tablespace for schemas. If you leave it default, RCU will name them with a prefix which is set on third step. If tablespaces does not exist RCU will create them for you.

RCU
RCU

Now RCU will show you a summary of settings, check the list if it seem fine click on “Create”. It will take some time to create schemas, tablespaces, tables then it will show you a report. And you are ready to close RCU and continue with Configuration Manager.

2. Creating Domain for Agent with Configuration Manager

Conf Man
Conf Man

Start Configuration Manager, you can find it in /home/oracle/Oracle/Middleware/Oracle_Home/oracle_common/common/bin on Linux OS and in C:OracleMiddlewareOracle_Homeoracle_commoncommonbin in Windows OS. File name is config.sh or config.cmd depending on OS. Select “Create a new domain” and select where you want to put domain’s files. I will leave it as it is.

Conf Man
Conf Man

Next page is Template Selection page, we will need to check “Oracle Data Integrator – Standalone Collocated Agent” and when you check this option there will be two more options checked automatically.

Conf Man
Conf Man

Next step will want you to create a login for WLS. Password should be at least eight characters, including at least one letter and one number.

Conf Man
Conf Man

Select Domain Mode and JDK, I will leave them default.

Conf Man
Conf Man

Now we will select RCU Data option, since we created necessary schemas with RCU. Enter connection information for the database schema where RCU worked on before. Then click on “Get RCU Configuration” and wait for Configuration Manager to read from database. Then click “Next” when it is available.

Conf Man
Conf Man

Enter all passwords and schemas we have created before. Also you will need to fill database connection information again.

Conf Man
Conf Man

Configuration Manager will check for connections if they are working.

Conf Man
Conf Man

Now we need to select components which we are going to need, In this case we will only select “System Components” and “Deployments and Services” and continue.

Conf Man
Conf Man

Set name for the agent I will use OracleDemoAgent for this demo.

Config10
Conf Man

Next select the component from dropdown list, which we have created one step ago. Set listener information you can leave port as 21910 it is default for configuration manager. Fill in supervisor user information. And continue, for this demo we will leave settings default on other steps then click on “Create” on last step.

Conf Man
Conf Man

Creation will take sometime after that click on “Next” and “Finish” and we are almost ready to run the agent.

3. Defining Physical Agent in ODI Studio

Run ODI Studio, and login to repository, where you have Topology rights. We will need this rights to create agents.

Topology
Topology

Go to Topology tab, in Physical Architecture accordion, right click on “Agents” and click on “New Agent” a new window will appear on right panel.

Topology
Topology

Name the agent as the same name as we selected in Configuration Manager for the Agent component. When we try to run agent, it will search for its name in Master Repository so names should match exactly. Set the port as you set in Configuration Manager. Save the agent.

4. Start up the agent

Open a terminal/command line window and go to the domain folder that you have created. Under this folder you will find bin folder. For me this folders path is /home/oracle/Oracle/Middleware/Oracle_Home/user_projects/domains/demo_domain/bin in Linux OS or C:OracleMiddlewareOracle_Homeuser_projectsdomainsdemo_domainbin in Windows OS. Run the command agent with NAME parameter as following, or as you can see in screenshot.  agent -NAME=OracleDemoAgent

Command Line
Command Line

Agent will start to rise up, it will connect to Master Repository, check for its name, read other configurations and start the service and listener. When agent is fully started, command line will seem like screenshot below.

Command Line
Command Line

5. Test the agent from ODI Studio

Now we have the agent up and running, also we have a definition for it in our Master Repository and a GUI in ODI Studio. It’s time to test if ODI Studio can connect to agent service. Go to Topology tab and open the agent we have created. There you will see a button written “Test” click on it. If everything is right until here you should see the screen below.

Test Agent
Test Agent

 

We have completed everything now. Agent is up and ready to run your jobs on databases for you.

Thanks for reading and please do not hesitate to write your comments or ask questions in section below.