Oracle Data Integrator: Best practices for Life Cycle Management.

Oracle Data Integrator offers many possibilities to re-use objects. One and the same model can be used by many different projects, scenarios generated by one project can be re-used by other projects, models can use knowledge modules from projects and so on... While this powerful feature has a positive impact on your workflow, it does come at a price.

ODI panels examples

Chances are since you're using ODI, you won't be working on just two simple projects, you will be working on many and large data integration projects which might change over time. Not to mention the creation of new projects which will in their turn be dependent on objects created for other projects.

So the question is: how do I manage the Life Cycle for all these projects without having to worry about all these interdependencies? The answer is simple: use Best Practices but also try keep the use of these interdependencies to a bare minimum.

The most effective "best practice" procedure is, without doubt, the initial step in our ODI project's Life Cycle: versioning. Versioning will not only safeguard your ODI development efforts and grant transparency, it will also serve as the ideal backup plan when something goes terribly wrong and you need to rollback your ODI repository.

And as you know, especially in meticulous operations such as data integration development, something is bound to wrong, eventually. So how do we go about managing the Life Cycle of our ODI projects?

Managing the Life Cycle starts with a first step, being versioning your objects. In order to have a simple, manageable versioning strategy and structure you could do the following:

First up: the Version Control Repository structure

Work project-based. By project based we mean that you should create a version control repository project for each ODI project. This version control repository project will contain the ODI project and the related project scenarios. Next to the version control repository project, you should create a "Global Objects" project that contains your Models and Global Objects. The latter will allow you to version your projects and Global Objects and Models separately.

Why you should use this VCR structure

  • Each ODI project can have its own life cycle and can travel at his own pace
  • You can version projects (projects and scenarios) and global objects (models and global objects) while the management of object dependencies is taken care off at commit and restore time
  • You don’t have to worry about merging objects from one ODI project to another

A typical Subversion repository setup for working with the ODI versioning component

Next: Oracle Data Integrator folder structure best practice

  • Since you have all of your projects in one ODI project directory, be aware that you don’t use project scenarios generated by a different project.
  • The same goes for your Models. Make sure you don’t use another project's Knowledge Modules.
  • In your Scenarios and Load Plan folder you should create a folder per project. Also make sure your project scenarios are part of that scenario project folder.

The ideal folder structure when using automated deployment in Oracle Data Integrator

Putting everything together

After applying the structures and using the proposed way of working explained above for both Oracle Data Integrator and your Version Control Repository, it is necessary to have a system that can respond to changes in your ODI codebase. Using the IKAN ALM DevOps framework (combined with its versioning component) we will have the possibility to use automated deployment for ODI and we will get the following benefits:

  • For each project you can have a trunk and multiple branches which enables proper release management
  • Testing, database updates, and any additional custom procedures can be part of your ODI life cycle
  • Transparent ODI deployments and rollbacks which can be approval-based

Using the IKAN ALM DevOps framework to deploy and rollback ODI projects

Let us show you how this works

Would you like to see a live demo of how Life Cycle Management and versioning for ODI works, and how it can help you develop faster and more secure? Get your free live demo here.

 

Get free insight!

Let's analyze your current development and release process, and show you how to make it much faster and 100% reliable.

Yes, I want insight

Do you want to recieve the latest news and updates?