ODI 12c: Renaming a Project

Hello all,

Working as a consultant has its own challenges which makes me try things at home, which drives me to play more with Oracle and ODI and learn new things. Today a new challenge appeared in front of me. I am working in an upgrade project, where the upgrade from ODI 10g to ODI 12c is already completed and my task is to test the codes in ODI 12c to see if they are still good. (There are some problems but they will be topic of another post.)

So today, our client admitted that they do not have only ODI 10g to upgrade, they also have an ODI 12c repository other than the upgraded one. And they want to merge two 12c repositories together. But there are some problems.

  1. Projects have the same name
  2. Some variables have the same name
  3. Some custom KMs have the same name but different codes.

So we thought of some methods and their possible outcomes. One option was to change the Project Name (and Code off course) but we could not decide how will ODI react to this change. So I decided to give it a try in my training VM tonight.

What I was expecting?

  1. ODI will recognize and change codes of variables in mappings
  2. ODI will recognize and change codes of variables in packages (refresh, evaluate, set, declare variable components)
  3. ODI may recognize the variable usage in procedure options
  4. ODI will not recognize and change variables in procedures

Let’s see what happened together:

First I created a new project and I created a variable called MY_VAR and put some refresh code in it, which does not matter since I am not going to run any of the objects for this demo.

Project Creation
Project Creation

Second I created MY_PROC with one task TASK 1 and one option MY_OPT. Task has one line of code, which is the MY_VAR itself, just to see if it will change.

MY_VAR
MY_PROC

Then I created a mapping called MY_MAP, put two random datastores in it. Made a link between them and mapped a column with MY_VAR.

MY_MAP
MY_MAP

Then I created a package MY_PACK which includes a Refresh Variable, MY_PROC with MY_OPT set to MY_VAR, and finally MY_MAP.

MY_PACK
MY_PACK

Finally I proceed to change Project Name and Project Code. ODI warns user in this step, that errors in objects may occur especially with project variables.

Rename
Rename

After I accepted the changing the name and the code, let’s see what happened. Refresh Variable in the pack is changed as I expected. You can see its new path in the lower right corner of the image. (click on image to see it bigger)

Refresh MY_VAR
Refresh MY_VAR

Option MY_OPT of MY_PROC did not change.

MY_OPT
MY_OPT

Code in MY_PROC stays the same after change.

MY_PROC
MY_PROC

Column mapping in MY_MAP did not change also which is a disappointment for me.

MY_MAP
MY_MAP

After seeing the results let’s check how many guesses I’ve done right. I will copy/paste the list from above.

  1. ODI will recognize and change codes of variables in mappings DID NOT HAPPEN
  2. ODI will recognize and change codes of variables in packages (refresh, evaluate, set, declare variable components) HAPPENED
  3. ODI may recognize the variable usage in procedure options DID NOT HAPPEN
  4. ODI will not recognize and change variables in procedures HAPPENED

So I achieved a score of 2 out of 4 in my guesses. And unfortunately ODI achieved to detect only 1 of the four scenarios. So be cautious about changing the name and the code of a project in ODI, actually be careful about changing the code of any othe objects too. (such as models, contexts etc.)

This demo has been done using ODI 12.1.2.0.0 and results may vary depending on the version.

Version
Version

Thanks for reading and please do not hesitate to write your comments or ask questions in section below.

ODI 11g : Results of Deleting a User

One of my customers had a high turnover rate. Sometimes we ran into some ODI objects which are locked by a developer who has gone. That triggered a question. What if I delete an ODI user while it has locks on objects. So I gave it a try, and while trying this other cases also pop-up. I also tried them on my VM.

Case 1: Deleting a user with locked objects

My expectation was being unable to delete a user. I was expecting there should be a FK between user and lock tables in repository. But I was able to delete. Then I thought about two scenarios,
a) object will be unlocked
b) object will stay locked and when I try to visualize lock, ODI will crash since there will be nothing returned from lock query.

I was wrong. You may find the screenshot about how ODI responds to deleting a user with locks.

Locked Object by deleted user
Locked Object by deleted user

 

I can open object, ODI will pop-up “This object is locked by DEVELOPER. You can not edit” dialog. You can view everything in the object. But to begin using object again, you need supervisor to unlock it.

Case 2: Deleting a user who created some objects

In that case I was expecting to find “Created By:” text box in object’s version tab to be empty. But I was wrong again. ODI just worked fine. When I checked the repository, I saw that ODI holds usernames for these text box not the user ID’s.

Case 3: Deleting an online user

 

After first two cases and failures of my expectations, now I had nothing to expect. I would just try and see. I created a user, open another ODI instance, connect with new user and deleted the user. Then I tried to take some actions in ODI. Some I was able to, some I was not.

 

 

Create Scenario
Create Scenario

Things I was able to:

  • View operator logs
  • View topology definitions
  • View models
  • Create scenarios
View interface
Save an open interface

Things I was not able to :

  • View interfaces
  • Run interfaces or scenarios
  • Selective Reverse Engineering a model
  • Edit topology definitions.

 

 

Exevute scenario
Exevute scenario
View interface
View interface

 

 

If you have some other cases for me to try please write in the comments area.

Thanks for reading, dont forget to share & comment.

Monthly Challenges : July ’15

So another month is over. My challenges and statuses are listed below.

On Going Challenges

– Work another month with Standing Desk, checked. I believe I won’t return to a sedentary work life, I really got used to my desk and working without a seat.
– Lose three more kilograms, failed. I’ve only lost around 600-700 grams.

New Challenges

– Tidy up my house, checked. Since we are having a really hot summer, I didn’t want to do anything but resting when I got home. So it got really untidy, but with this month’s challenge now I have a tidy house and it effects my life also.
– Exercise for at least 19 days (which is 2/3 of challenge range), failed. I only exercised 4 or 5 days.
– Publish two drafts, checked. You can find them on :
http://www.canburaktumer.com/blog/odi-11g-odi-12c-whats-agent/
http://www.canburaktumer.com/blog/x-things-graduation/
– Publish two Vlogs about extreme sports, checked.

Most of the challenges completed for July ’15. See you on the 7th of September for August Challenge Report.

ODI 11g, ODI 12c : What’s an Agent?

What is an agent?

An agent is a java process which usually located on server and listens to a port for incoming requests. It runs the requested scenario, reverse engineers requested datastores etc.

When a job submitted through ODI Studio GUI or through startscen.sh agent gets scenario from work repository and topology definitions from master repository, combines and converts them into a runnable job, usually consisting more than one code block. Then it sends code blocks to destination environments, which may be DB servers, file servers, hadoop name node etc. Finally agent gets job statuses from these environments and writes into work repository tables for us to see from Operator tab of ODI Studio.

Agent diagram from Oracle A-Team Blog

Agent Types

Standalone Agent

It is the basic agent of ODI. It does not require an application server like JEE Agent. It is easy to configure/start/stop this agent from shell. Since today I’ve always used this agent and never tried other versions. A-Team article says this is the most light weight and low footprint choice.

JEE Agent

“Java Enterprise Edition” agent, which requires an application server, in most documentation you can see the name of WebLogic Server since it’s another Oracle product. (Some search results: depending on this CertMatrix of Oracle http://www.oracle.com/technetwork/middleware/data-integration/odi-11gr1certmatrix-ps6-1928216.xls ODI 11.1.1.7.0 only supports WLS and does not support Tomcat or other application servers. You may -or may not- configure them to run together but it is not supported.) This agent is first delivered with ODI 11g. And still exists in ODI 12c.

Some pros of JEE agent which are written in A-Team blog are:

  • High availability : Through Web Logic Server’s cluster architecture, even a node is down agents may run on other nodes.
  • Configurable connection pooling : Connection pool can be configured through WLS.
  • Monitoring : Oracle Enterprise Manager can monitor, configure, alert, manage ODI JEE Agents. But I believe, there is a plug-in to be installed to achieve this tasks from OEM.

Colocated Agent

The newest agent type which has arrived with ODI 12c. This type is like a combo of other two types. Agent is a standalone agent, but can be monitored and configured through WLS. Unfortunately it does not take advantage of connection pooling, high availability. Our agent will be in WLS domain, can be managed through WLS and that’s all. It is lighter than JEE Agent. In my opinion companies which prefers JEE Agent as production agents can choose colocated agent as their DEV/TEST agent.

Agent Types diagram from Gerard Nico's website
Agent Types diagram from Gerard Nico’s website

Where to locate an agent?

To decrease network I/O it is better to locate agent to target DB server. Since agent submits code to DB Engine, it is better for them to be on same machine. Don’t forget that ODI is an E-LT tool. Which means it will load data into target server, then it will transform your data. So most of the load will be on target server. Which also means most of the codes will be submitted to target server.

Also since an agent is a local java process, agent will write files to the machine, which it is set-up on. If you have a file server seperate than DB server, then it is better to have another agent on file server to handle read/write file processes. Or to mount file servers directory to DB server as a directory and setting up only one agent is another solution.

Also these solutions will prevent any firewall related problems.

 

Thanks for reading, don’t forget to share & comment.

 

 

Resources I’ve read before writing this post:
http://www.ateam-oracle.com/odi-agents-standalone-jee-and-colocated/
http://www.ateam-oracle.com/understanding-where-to-install-the-odi-standalone-agent/
http://odibeginnertutorials.blogspot.com.tr/2013/11/2-introduction-to-odi-agents-and.html
http://www.odigurus.com/2011/10/defining-odi-agents.html
http://gerardnico.com/wiki/dit/odi/agent

[Random Thoughts] 7 Things to do Before Graduation

1- Erasmus Exchange

Erasmus is an European Union supported exchange programme. For one semester or for one academic year students go abroad to study there. EU is paying students some scholarship, which is quite enough to live. All the ask for is to success your lectures, even not all the lectures, just some percent of them is enough.

Erasmus was one of the best experiences I’ve ever had till today. I started to stand on my own feet. I learned to keep my budget. I learned to properly cook, clean, iron etc. all by myself. I learned about Italian culture, (some German culture also). I taste different foods, experienced different teaching methods, learned new languages, got new friends, saw too many cities with little money. And became a confident, self-aware guy.

In Turkey, you need a GPA and also you need to pass some exams to participate in Erasmus programme. My advice is, whatever it takes for you participate in Erasmus exchange.

Life changing experience
Life changing experience

Erasmus really changed my life, and the way I see the life. I feel more open-minded, more independent, more welcoming now. And it is too hard to stay in a place for a long time without traveling now.

2- Part-time work or long-term internship

Internships-Wordle-v3

Having a part-time job or a long-term internship will make you more experienced. especially a work in the branch that you want to become professional will be a good choice.

You will taste making money, you will become less dependent to your family.

You will start building your network. You will learn the difference between university life, university projects and the real life, real projects.

You will have some experience to write on your CV. And probably you’ll get your first job, if you are lucky and hard working enough.

3- Join student clubs & organizations

Students clubs & organizations will improve your network and soft skills. I got most of my non-IT friends thanks to student organizations. I’ve been in two different student clubs through my university life, the first one is EHAVK (Ege University Aviation Association Paragliding Team). I’ve been in this association for 5 years. I started my journey from the base to the top. From beginner pilot to instructor & tandem pilot. This association taught me about organizing events, leading a group, instructing to people, team work, presentation skills.

I am an ex-AIESECer
I am an ex-AIESECer

My second association was AIESEC. I’ve only stayed for one year in AIESEC, but it improved me alot. Because it gave me lots of responsbilities, I had the roles of Project Manager and Communication Responsible of Local Committee. I was leading a team of 3 persons. We were organising events, recruiting new members, publishing newsletter, keeping our website updated, participating in weekly meetings. It taught me a lot.

Being a member of student associations will teach you lots of things, improve your network, let you improve yourself in every manner. Try it.

4- Go to events

One of my regrets about my university life. I’ve begun attending to events in my job life. But there is never enough time to participate in the events I want.

Networking-events-blog-image

Attending to events will let you enlarge your network, stay updated about your professions matters and development. My suggestion for IT sector is to attend events like hackathons, Google Developer Group events, hackerspaces/makerspaces and vendor events.

5- Chose graduation project / thesis wisely

Another regret for me.

Your project/thesis will effect your job. It will effect your chance of getting selected for jobs and Master/PhD programmes. Choose your thesis/project on your branch, make contribution to this area.

thesis-binding-dublin-1bc

6- Have fun

remember-to-have-fun  Study hard, work harder but have fun even harder. It was our motto in AIESEC “Work hard, party harder.” It is one of the best  proverbs I’ve ever heard. Party hard, have fun hard, enjoy your university and social life while you can.

Keep in mind that these days will not come back. And there is another thing, every person I met says they were richer when      they were students. Some how it is true. I was not earning money like I am doing today. But I was living a better life. Also I had much more time for everything.

While you still have time and money, enjoy your life. Get out with friends, get crazy.

7- Move out

I also suggest you to move out if you can. Leaving without your parents  will teach you so much. You will learn to plan a budget, being tidy, cooking, cleaning, washing, ironing etc etc. You will stand on your own feet. You will face with bitter truth of bills, rent, house maintenance etc. But you’ll grow up by handling them. You’ll get mature.

543x269xmoving-out.png.pagespeed.ic_.D_d-NFibnY

That’s all from me. Thanks for reading.
Share, comment, join conversation.

 

 

 

Random Thoughts: Generation Y and Job Hopping

Nowadays I hear a lot complains about my generation: Generation Y. People say it is hard to keep us happy. We become upset quickly, we expect too much etc. etc. So in this post I will try to focus on this topic with an internal view.

Generations

Let’s look at the “generations” closer. Scientist divided human race to generations, they’re separated by birth years and every group has common thought and behavior which differs from other generations.

Generations
Generations

When we check the generation list of western world (America, Europe) from Wikipedia we can see the list below:

  • Lost Generation (1883 – 1900): Who fought in WWI.
  • Greatest Generation (1901 – 1924): Who fought in WWII.
  • Silent Generation (1925 – 1942): Who fought in Korean and Vietnam wars.
  • Baby Boomers (1943 – 1960): Generation born after WWII is finished.
  • Generation X (1960 – 1980)
  • Generation Y (1980 – 2000)
  • Generation Z (2001 – …)

Last three generation does not have any distinctive historic event, they only have date intervals given by demographers. But I will write some common behaviors for these three generations depending on my observations in my home country, Turkey.

Generation in Workforce
Generations in Workforce

Generation X members are mostly have a stable professional life. Most wanted jobs are civil servant, branch manager in a bank or officer in army. They are hardworking in the sense of body power. For sure they are not lazy but they afraid of changes. They’ve seen military interventions in 70’s and 80’s. So they want a guaranteed lifestyle. They start to work in somewhere and usually get retired from the same organization.

Generation Y is the main topic of this post so I will have a detailed text below about them and their professional life.

Generation Z members are still in high school or in college. They were born into technology. They are mostly lazy because of “smart” devices. They are getting less social and more connected day by day. I wonder about their professional lives.

So, what Generation Y wants?

Generation Y is more free then Generation X. It is certain that we have much more possibility and opportunity than Generation X. We were born when there were homeworks done by help of encyclopedias, then we evolve with technology. First come the PCs and then internet connection. And our lives became easier and faster. That’s why we value and love the technology.

Members of Generation Y have an innovative and entrepreneurial soul. Early members of Generation Y earned some easy money using and developing technology. So late comers believed that it is easy to get rich and famous. And that’s what they wanted; become rich and famous easily.

Wishlist of Generation Y
Wishlist of Generation Y

But we’re raised by Generation X. We’ve raised with the belief of guaranteed standards is best thing in our lives. Entrepreneurship is a risk, and taking risks is stupidity. But at the same time we want money and fame. So we became the generation of white collars. We believed, partly guaranteed life standards, money and fame can be achieved by working in enterprise companies. That was true until a certain point in Turkey.

After crisis in 1999 and 2011 enterprise world renewed their budgets with strict ones. And to me, it is the point where they lost the Generation Y.

Why Generation Y changes job quickly?

Because we are not as stable as Generation X members. They’re determined. When we see that we cannot go any further in the organization we are in, we decide to quit simply for a better one but Generation X waits patiently even until their retirement.

Should I stay or should I go?
Should I stay or should I go?

I will give examples from my short professional life. It is short but it has enough job changes to use. It has been a little more than three years and I am in my fourth organization. Let’s have a look why I quit from every single one:

  1. Part-time ETL developer in Turkcell Technology, I was waiting for a full time position, but since the budget was cut it was a low possibility to find a full time position inside. My manager was asking for me to wait but I was already graduated and wanted a serious job with a stable paycheck to pay my rent and other expenses. (13 months)
  2. Java developer in OBSS, OBSS is really one of the best places for a consultant to work. But I wanted to become an expert in data warehouses, ETL and data mining so I changed my job when I got acceptance from Oracle Consulting (2 months)
  3. Associate Consultant in Oracle Turkey Consulting, It was absolutely a dream job and a dream company for me until I became a part of it. It seems like it is the best source that I can learn DW/ETL so I made a sacrifice from my paycheck (it was a mistake I admit.) and I accepted a lower paycheck then my job in OBSS. I thought I am going to learn so many new things, I will become an expert in DW and Oracle. Unfortunately reality is not like that, I was alone in customer location, there were no seniors to ask questions when I lost in code. I began to feel I am not going any better and also I was losing money since the salary was not a competitive one. I decided to quit and start to search for consulting firms, where they pay you more money but no training. I was thinking to save money to pay for my own trainings to improve myself. Then I found my next job. (15 months)
  4. BI & DW Specialist in MAKEIT Software & Consulting, finally the job I love and a competitive enough salary to give me a life standard, and company also encourages me to improve myself. Encourages me to present in events, prepare training materials for new-grad employees, give challenging projects and enough support when I feel lost. (9 months and counting up)

Looking through my experience, I can say Generation Y aims something and when it feels things are not working, it changes strategy to reach its aim.

My aim is to become an expert in DW, ETL and Data Mining as I said. So I always took the steps which is better in my path to my aim. I worked with an AceD (Mr. Gürcan Orhan) and tried to learn as much as I can while I was a part-time, the for a better future I felt I need to move to Oracle and I did, there I saw I am not going anywhere and I changed tracks for the destination. There is only one thing that did not change through these three years, I kept working hard to learn new things and improve myself, in every free time I made demos with new tools, I wrote blog posts, I prepared presentations, I read others blogs and presentations.

Conclusion

To sum up I can say, Generation Y members have an aim, which is a guaranteed life standard and fame. And they will change the path they walk when they fell they cannot get any closer to their aim. For Generation Y work life is not aim of the life, it is just a tool to reach to their aim. It is like a long flight to destination we may need to change planes on the way.

From graduation to life aim.

 

Thanks for reading and please share your thoughts in comment section.

You can also find this post on LinkedIn.

Sources for images: