My takes from System Design Interview by Alex Xu

System Design Interview by Alex Xu

Since 2015, I am attending interviews as an interviewer for recruiting new talents, for the selection of new partner companies etc. And since 2019 I am conducting interviews for my current company. The topics I interview for are usually General Tech and Culture Fit. I enjoy also system design and thought I can also help our company by taking over a few System Design Interviews hence I started to prepare myself for the next step. I saw “System Desing Interview: An Insider’s Guide” by Alex Xu is suggested for interviewees to prepare for upcoming challenges so I thought I should give it a shot to sharpen my skills and understanding of the system design domain.

The book starts with the basics like designing for scaling, back of the envelope calculations then suggests a framework for system design and provides 12 different system design examples. I took some notes and also recorded some resources to read further.

Let’s look into the suggested framework for system design. It’s a four-step framework and actually, I found it quite useful to have such a structure.

  1. Understand the problem and establish design scope
  2. Propose a high-level design and get buy-in
  3. Design deep dive
  4. Wrap-up

Time allocation suggested by the author per step is

  1. 3 – 10 minutes
  2. 10 – 15 minutes
  3. 10 – 25 minutes
  4. 3 – 5 minutes

This time allocation would allow a candidate to complete a design in around 50 minutes and would provide some additional time for Q&A in a 60 minutes interview. From my perspective suggested framework and time allocations would work in an interview setup.

Let me summarize each step with one paragraph, then I can provide my thoughts, notes and feedback about the book.

Understand the problem and establish design scope

Nobody would expect you to design the new Facebook or Netflix in the short time of an interview. So asking clarifying questions, setting up the boundaries of design and noting down the assumptions are the way to start the design. All those steps should be discussed with the interviewer, get their ideas about the important parts of the system to design and get their approval on the assumptions.

Propose a high-level design and get buy-in

This is the step where the interviewee should lay the foundations of the system and provide a blueprint with the collaboration of the interviewer. Doing some back of the envelope calculations and depending on those calculations system size and components should appear on the design. This step should not include details like data format yet but should have a storage layer for example.

Design deep dive

This is the time to think about handling the load, choosing the correct data store, scaling the API servers etc. Depending on the discussion with the interviewer, you should focus on a few key areas and deliver a detailed design and even a few implementation details if required.

Wrap up

In this last step, you should go the extra mile and tell about the flaws in your design and how it may be improved, what can be expected to change in the business in the future and how the design can evolve to cope with those changes.

My notes and thoughts about the book

  • There are some mistakes or not so best-practices in the book e.g.
    • Author claims NoSQL should be chosen to store massive amount of data, this is open for discussion. A technology selection should not be justified only by data size but also use cases and requirements.
    • In one of the API design sections, author does not follow the REST best practices and uses verb in the URL
    • Author claims a scale-up would not help with the load, in my experience it helps.
  • Back of the envelope numbers are great shortcuts during the design time.
  • There is an unfortunate use of magic number in the book while calculating peak system load. There is no explanation of why we use this factor to multiply our assumption.
  • 12 different system design examples backed by real life resources are really handy and informative.
  • There are some missing information, for example author says “token bucket is memory efficient” but does not explain why and how. Or while discussing quorum of a distributed system author overlooks possibility of a Byzantine Fault.
  • Reference materials are useful, almost every section I noted down some of the reference materials to read further.
  • In some designs there are security flaws which may cause data leak, which have been overseen. Though this can be defended as security is not in the scope and time is short. Still I believe security should start in the design phase.
  • Also in some designs sections, author does not follow the framework they suggested.

All in all, I got some fresh ideas and a good framework for system design interviews from the book. I would consider recommending this book to mid-level engineers who are looking for different aspects of system design and would use a few real-life examples. For me, a 10+ year engineer, the advantages of reading this book was limited.

Thanks to Alex Xu for putting all the effort and collecting those examples in a well-structured format. And thank you for reading this blog post, feel free to comment or share this post if you find it useful.

Coming Back…

It has been more than 5 years since my last blog post in here.

A few years ago I moved my writing habits into another website, which was medium.com. Now I decided to come back to my own web site to post my thoughts.

I can not promise about the intervals for new posts, it can be bi-weekly or monthly. But it will be about my personal career growth including but not limited to technical thoughts and demos, learnings about management, book reviews and maybe some whisky tasting notes. (People need their hobbies ☺)

Let’s start slow today and close this “Welcome back home” post. See you in the future posts and feel free to comment and share.

Featured Image Photo by Danielle MacInnes on Unsplash

ODI 12c: Renaming a Project

Hello all,

Working as a consultant has its own challenges which makes me try things at home, which drives me to play more with Oracle and ODI and learn new things. Today a new challenge appeared in front of me. I am working in an upgrade project, where the upgrade from ODI 10g to ODI 12c is already completed and my task is to test the codes in ODI 12c to see if they are still good. (There are some problems but they will be topic of another post.)

So today, our client admitted that they do not have only ODI 10g to upgrade, they also have an ODI 12c repository other than the upgraded one. And they want to merge two 12c repositories together. But there are some problems.

  1. Projects have the same name
  2. Some variables have the same name
  3. Some custom KMs have the same name but different codes.

So we thought of some methods and their possible outcomes. One option was to change the Project Name (and Code off course) but we could not decide how will ODI react to this change. So I decided to give it a try in my training VM tonight.

What I was expecting?

  1. ODI will recognize and change codes of variables in mappings
  2. ODI will recognize and change codes of variables in packages (refresh, evaluate, set, declare variable components)
  3. ODI may recognize the variable usage in procedure options
  4. ODI will not recognize and change variables in procedures

Let’s see what happened together:

First I created a new project and I created a variable called MY_VAR and put some refresh code in it, which does not matter since I am not going to run any of the objects for this demo.

Project Creation
Project Creation

Second I created MY_PROC with one task TASK 1 and one option MY_OPT. Task has one line of code, which is the MY_VAR itself, just to see if it will change.

MY_VAR
MY_PROC

Then I created a mapping called MY_MAP, put two random datastores in it. Made a link between them and mapped a column with MY_VAR.

MY_MAP
MY_MAP

Then I created a package MY_PACK which includes a Refresh Variable, MY_PROC with MY_OPT set to MY_VAR, and finally MY_MAP.

MY_PACK
MY_PACK

Finally I proceed to change Project Name and Project Code. ODI warns user in this step, that errors in objects may occur especially with project variables.

Rename
Rename

After I accepted the changing the name and the code, let’s see what happened. Refresh Variable in the pack is changed as I expected. You can see its new path in the lower right corner of the image. (click on image to see it bigger)

Refresh MY_VAR
Refresh MY_VAR

Option MY_OPT of MY_PROC did not change.

MY_OPT
MY_OPT

Code in MY_PROC stays the same after change.

MY_PROC
MY_PROC

Column mapping in MY_MAP did not change also which is a disappointment for me.

MY_MAP
MY_MAP

After seeing the results let’s check how many guesses I’ve done right. I will copy/paste the list from above.

  1. ODI will recognize and change codes of variables in mappings DID NOT HAPPEN
  2. ODI will recognize and change codes of variables in packages (refresh, evaluate, set, declare variable components) HAPPENED
  3. ODI may recognize the variable usage in procedure options DID NOT HAPPEN
  4. ODI will not recognize and change variables in procedures HAPPENED

So I achieved a score of 2 out of 4 in my guesses. And unfortunately ODI achieved to detect only 1 of the four scenarios. So be cautious about changing the name and the code of a project in ODI, actually be careful about changing the code of any othe objects too. (such as models, contexts etc.)

This demo has been done using ODI 12.1.2.0.0 and results may vary depending on the version.

Version
Version

Thanks for reading and please do not hesitate to write your comments or ask questions in section below.

ODI 11g : Results of Deleting a User

One of my customers had a high turnover rate. Sometimes we ran into some ODI objects which are locked by a developer who has gone. That triggered a question. What if I delete an ODI user while it has locks on objects. So I gave it a try, and while trying this other cases also pop-up. I also tried them on my VM.

Case 1: Deleting a user with locked objects

My expectation was being unable to delete a user. I was expecting there should be a FK between user and lock tables in repository. But I was able to delete. Then I thought about two scenarios,
a) object will be unlocked
b) object will stay locked and when I try to visualize lock, ODI will crash since there will be nothing returned from lock query.

I was wrong. You may find the screenshot about how ODI responds to deleting a user with locks.

Locked Object by deleted user
Locked Object by deleted user

 

I can open object, ODI will pop-up “This object is locked by DEVELOPER. You can not edit” dialog. You can view everything in the object. But to begin using object again, you need supervisor to unlock it.

Case 2: Deleting a user who created some objects

In that case I was expecting to find “Created By:” text box in object’s version tab to be empty. But I was wrong again. ODI just worked fine. When I checked the repository, I saw that ODI holds usernames for these text box not the user ID’s.

Case 3: Deleting an online user

 

After first two cases and failures of my expectations, now I had nothing to expect. I would just try and see. I created a user, open another ODI instance, connect with new user and deleted the user. Then I tried to take some actions in ODI. Some I was able to, some I was not.

 

 

Create Scenario
Create Scenario

Things I was able to:

  • View operator logs
  • View topology definitions
  • View models
  • Create scenarios

View interface
Save an open interface

Things I was not able to :

  • View interfaces
  • Run interfaces or scenarios
  • Selective Reverse Engineering a model
  • Edit topology definitions.

 

 

Exevute scenario
Exevute scenario

View interface
View interface

 

 

If you have some other cases for me to try please write in the comments area.

Thanks for reading, dont forget to share & comment.

Monthly Challenges : July ’15

So another month is over. My challenges and statuses are listed below.

On Going Challenges

– Work another month with Standing Desk, checked. I believe I won’t return to a sedentary work life, I really got used to my desk and working without a seat.
– Lose three more kilograms, failed. I’ve only lost around 600-700 grams.

New Challenges

– Tidy up my house, checked. Since we are having a really hot summer, I didn’t want to do anything but resting when I got home. So it got really untidy, but with this month’s challenge now I have a tidy house and it effects my life also.
– Exercise for at least 19 days (which is 2/3 of challenge range), failed. I only exercised 4 or 5 days.
– Publish two drafts, checked. You can find them on :
http://www.canburaktumer.com/blog/odi-11g-odi-12c-whats-agent/
http://www.canburaktumer.com/blog/x-things-graduation/
– Publish two Vlogs about extreme sports, checked.

Most of the challenges completed for July ’15. See you on the 7th of September for August Challenge Report.

ODI 11g, ODI 12c : What’s an Agent?

What is an agent?

An agent is a java process which usually located on server and listens to a port for incoming requests. It runs the requested scenario, reverse engineers requested datastores etc.

When a job submitted through ODI Studio GUI or through startscen.sh agent gets scenario from work repository and topology definitions from master repository, combines and converts them into a runnable job, usually consisting more than one code block. Then it sends code blocks to destination environments, which may be DB servers, file servers, hadoop name node etc. Finally agent gets job statuses from these environments and writes into work repository tables for us to see from Operator tab of ODI Studio.

Agent diagram from Oracle A-Team Blog

Agent Types

Standalone Agent

It is the basic agent of ODI. It does not require an application server like JEE Agent. It is easy to configure/start/stop this agent from shell. Since today I’ve always used this agent and never tried other versions. A-Team article says this is the most light weight and low footprint choice.

JEE Agent

“Java Enterprise Edition” agent, which requires an application server, in most documentation you can see the name of WebLogic Server since it’s another Oracle product. (Some search results: depending on this CertMatrix of Oracle http://www.oracle.com/technetwork/middleware/data-integration/odi-11gr1certmatrix-ps6-1928216.xls ODI 11.1.1.7.0 only supports WLS and does not support Tomcat or other application servers. You may -or may not- configure them to run together but it is not supported.) This agent is first delivered with ODI 11g. And still exists in ODI 12c.

Some pros of JEE agent which are written in A-Team blog are:

  • High availability : Through Web Logic Server’s cluster architecture, even a node is down agents may run on other nodes.
  • Configurable connection pooling : Connection pool can be configured through WLS.
  • Monitoring : Oracle Enterprise Manager can monitor, configure, alert, manage ODI JEE Agents. But I believe, there is a plug-in to be installed to achieve this tasks from OEM.

Colocated Agent

The newest agent type which has arrived with ODI 12c. This type is like a combo of other two types. Agent is a standalone agent, but can be monitored and configured through WLS. Unfortunately it does not take advantage of connection pooling, high availability. Our agent will be in WLS domain, can be managed through WLS and that’s all. It is lighter than JEE Agent. In my opinion companies which prefers JEE Agent as production agents can choose colocated agent as their DEV/TEST agent.

Agent Types diagram from Gerard Nico's website
Agent Types diagram from Gerard Nico’s website

Where to locate an agent?

To decrease network I/O it is better to locate agent to target DB server. Since agent submits code to DB Engine, it is better for them to be on same machine. Don’t forget that ODI is an E-LT tool. Which means it will load data into target server, then it will transform your data. So most of the load will be on target server. Which also means most of the codes will be submitted to target server.

Also since an agent is a local java process, agent will write files to the machine, which it is set-up on. If you have a file server seperate than DB server, then it is better to have another agent on file server to handle read/write file processes. Or to mount file servers directory to DB server as a directory and setting up only one agent is another solution.

Also these solutions will prevent any firewall related problems.

 

Thanks for reading, don’t forget to share & comment.

 

 

Resources I’ve read before writing this post:
http://www.ateam-oracle.com/odi-agents-standalone-jee-and-colocated/
http://www.ateam-oracle.com/understanding-where-to-install-the-odi-standalone-agent/
http://odibeginnertutorials.blogspot.com.tr/2013/11/2-introduction-to-odi-agents-and.html
http://www.odigurus.com/2011/10/defining-odi-agents.html
http://gerardnico.com/wiki/dit/odi/agent

[Random Thoughts] 7 Things to do Before Graduation

1- Erasmus Exchange

Erasmus is an European Union supported exchange programme. For one semester or for one academic year students go abroad to study there. EU is paying students some scholarship, which is quite enough to live. All the ask for is to success your lectures, even not all the lectures, just some percent of them is enough.

Erasmus was one of the best experiences I’ve ever had till today. I started to stand on my own feet. I learned to keep my budget. I learned to properly cook, clean, iron etc. all by myself. I learned about Italian culture, (some German culture also). I taste different foods, experienced different teaching methods, learned new languages, got new friends, saw too many cities with little money. And became a confident, self-aware guy.

In Turkey, you need a GPA and also you need to pass some exams to participate in Erasmus programme. My advice is, whatever it takes for you participate in Erasmus exchange.

Life changing experience
Life changing experience

Erasmus really changed my life, and the way I see the life. I feel more open-minded, more independent, more welcoming now. And it is too hard to stay in a place for a long time without traveling now.

2- Part-time work or long-term internship

Internships-Wordle-v3

Having a part-time job or a long-term internship will make you more experienced. especially a work in the branch that you want to become professional will be a good choice.

You will taste making money, you will become less dependent to your family.

You will start building your network. You will learn the difference between university life, university projects and the real life, real projects.

You will have some experience to write on your CV. And probably you’ll get your first job, if you are lucky and hard working enough.

3- Join student clubs & organizations

Students clubs & organizations will improve your network and soft skills. I got most of my non-IT friends thanks to student organizations. I’ve been in two different student clubs through my university life, the first one is EHAVK (Ege University Aviation Association Paragliding Team). I’ve been in this association for 5 years. I started my journey from the base to the top. From beginner pilot to instructor & tandem pilot. This association taught me about organizing events, leading a group, instructing to people, team work, presentation skills.

I am an ex-AIESECer
I am an ex-AIESECer

My second association was AIESEC. I’ve only stayed for one year in AIESEC, but it improved me alot. Because it gave me lots of responsbilities, I had the roles of Project Manager and Communication Responsible of Local Committee. I was leading a team of 3 persons. We were organising events, recruiting new members, publishing newsletter, keeping our website updated, participating in weekly meetings. It taught me a lot.

Being a member of student associations will teach you lots of things, improve your network, let you improve yourself in every manner. Try it.

4- Go to events

One of my regrets about my university life. I’ve begun attending to events in my job life. But there is never enough time to participate in the events I want.

Networking-events-blog-image

Attending to events will let you enlarge your network, stay updated about your professions matters and development. My suggestion for IT sector is to attend events like hackathons, Google Developer Group events, hackerspaces/makerspaces and vendor events.

5- Chose graduation project / thesis wisely

Another regret for me.

Your project/thesis will effect your job. It will effect your chance of getting selected for jobs and Master/PhD programmes. Choose your thesis/project on your branch, make contribution to this area.

thesis-binding-dublin-1bc

6- Have fun

remember-to-have-fun  Study hard, work harder but have fun even harder. It was our motto in AIESEC “Work hard, party harder.” It is one of the best  proverbs I’ve ever heard. Party hard, have fun hard, enjoy your university and social life while you can.

Keep in mind that these days will not come back. And there is another thing, every person I met says they were richer when      they were students. Some how it is true. I was not earning money like I am doing today. But I was living a better life. Also I had much more time for everything.

While you still have time and money, enjoy your life. Get out with friends, get crazy.

7- Move out

I also suggest you to move out if you can. Leaving without your parents  will teach you so much. You will learn to plan a budget, being tidy, cooking, cleaning, washing, ironing etc etc. You will stand on your own feet. You will face with bitter truth of bills, rent, house maintenance etc. But you’ll grow up by handling them. You’ll get mature.

543x269xmoving-out.png.pagespeed.ic_.D_d-NFibnY

That’s all from me. Thanks for reading.
Share, comment, join conversation.

 

 

 

Monthly Challenges : June ’15 Weight Loss and Standing Desk

Hello all,

I had two challenges for June and luckily I completed both of them.

1. Lose 3 kg.s in a month.

This one was easy, I will not get into details. I was 87.6 kg.s, I cut off eating junk food, I began to count my steps and put a target of 10.000 steps daily. And at the end I was 83.8 kg.s. I’ve reached and passed my target.

2. Work on Standing Desk

This was the real challenge. Actually,  when I am working at home, I am already using my DIY standing desk,  but I am working at most two hours at home. So it is not challenging.

Standing desk in the office has at least two challenges. Which are a) convincing manager and colleagues, b) really standing for 8 hours straight.

For the first part, luckily we were moving to a new office, I had the most isolated part for myself. So I will not get too much attention 🙂 And my manager is an open-minded man, he is also very friendly (once we were colleagues). Convincing manager was not that hard. It took more time for my colleagues to get used to me 🙂

Second part was hard sometimes. When you google ‘standing desk’ and read some blog posts about them. You will see some tips, which come true when I experience it. First of all just standing and working like that is not possible. I often transfer my weight from one leg to another, take some small walks in office, have some small sitting pauses when I am taking notes or a friend ask for some help.

Shoes are really important when you stand all day long. Hard soled shoes will make your day a nightmare, some people use yoga mats to soften the floor. I choose more soft soled shoes. This way my feet don’t cry for help.

I also built my standing desk in office, so it is a DIY standing desk like the one at home. This have some pros, like I have two spare shelves to put my notebooks, pens etc.

My DIY Standing Desk
My DIY Standing Desk

Now, I am happy with the result.

Have a look at also my post on LinkedIn about standing desk : https://www.linkedin.com/today/author/130746977
Leave comments, keep in touch.

New Series : Monthly Challenges

Hello all,

I am going to kick off new blog post series. By this way, I believe I will also keep the blog updated. The series are about Monthly Challenges, I’ve already completed my first challenge and its post is coming today.

I will not reveal monthly challenges until the month ends. Every challenge will start on first monday of the month and will end on last day of the month. So I may have a few days between two challenges to rest. 🙂

I got the idea from this blog post (Turkish), and he wrote that he was inspired by Matt Cutts.

Challenge posts list (will be updated every month) :

June ’15 Challenge : http://www.canburaktumer.com/blog/monthly-challenges-june-15-weight-loss-and-standing-desk/
July ’15 Challenge : http://www.canburaktumer.com/blog/monthly-challenges-july-15/