Oracle Data Integrator offers many possibilities to re-use objects. One and the same model can be used by many different projects, scenarios generated by one project can be re-used by other projects, models can use knowledge modules from projects and so on... While this powerful feature has a positive impact on your workflow, it does come at a price.
Chances are since you're using ODI, you won't be working on just two simple projects, you will be working on many and large data integration projects which might change over time. Not to mention the creation of new projects which will in their turn be dependent on objects created for other projects.
So the question is: how do I manage the Life Cycle for all these projects without having to worry about all these interdependencies? The answer is simple: use Best Practices but also try keep the use of these interdependencies to a bare minimum.
The most effective "best practice" procedure is, without doubt, the initial step in our ODI project's Life Cycle: versioning. Versioning will not only safeguard your ODI development efforts and grant transparency, it will also serve as the ideal backup plan when something goes terribly wrong and you need to rollback your ODI repository.
And as you know, especially in meticulous operations such as data integration development, something is bound to wrong, eventually. So how do we go about managing the Life Cycle of our ODI projects?
Managing the Life Cycle starts with a first step, being versioning your objects. In order to have a simple, manageable versioning strategy and structure you could do the following:
First up: the Version Control Repository structure
Work project-based. By project based we mean that you should create a version control repository project for each ODI project. This version control repository project will contain the ODI project and the related project scenarios. Next to the version control repository project, you should create a "Global Objects" project that contains your Models and Global Objects. The latter will allow you to version your projects and Global Objects and Models separately.
Why you should use this VCR structure- Each ODI project can have its own life cycle and can travel at his own pace
- You can version projects (projects and scenarios) and global objects (models and global objects) while the management of object dependencies is taken care off at commit and restore time
- You don’t have to worry about merging objects from one ODI project to another
Next: Oracle Data Integrator folder structure best practice
- Since you have all of your projects in one ODI project directory, be aware that you don’t use project scenarios generated by a different project.
- The same goes for your Models. Make sure you don’t use another project's Knowledge Modules.
- In your Scenarios and Load Plan folder you should create a folder per project. Also make sure your project scenarios are part of that scenario project folder.
Putting everything together
After applying the structures and using the proposed way of working explained above for both Oracle Data Integrator and your Version Control Repository, it is necessary to have a system that can respond to changes in your ODI codebase. Using the IKAN ALM DevOps framework (combined with its versioning component) we will have the possibility to use automated deployment for ODI and we will get the following benefits:
- For each project you can have a trunk and multiple branches which enables proper release management
- Testing, database updates, and any additional custom procedures can be part of your ODI life cycle
- Transparent ODI deployments and rollbacks which can be approval-based
Get a live demo to see how this works
Would you like to see how Life Cycle Management and versioning for ODI works? We will show you how it will help you develop faster. Get a free live demo.
About the author
Hello, my name is René De Vleeschauwer.
Throughout my career I've been responsible for the development of enterprise software. Since 12 years I've been leading the development of IKAN ALM, an open DevOps framework.
Do you have any questions about this post? Just ask me!