ODI - Knowledge Modules

After long time doing some assignment on ODI,so thought to write something about Knowledge modules.

You can find more about ODI Architecture and how to configure from here .

• Knowledge Modules (KMs) are code templates. Each KM is dedicated to an individual task in the overall data integration process.
• A KM will be reused across several interfaces or models.the benefit of Knowledge Modules is that you make a change once and it is instantly propagated to hundreds of transformations. 
• KMs are based on logical tasks that will be performed. They don’t contain references to physical objects (data stores, columns, physical paths, etc.)

Six Types of Knowledge Modules:

1.  Reverse-engineering Knowledge Modules (RKM)
2.  Check Knowledge Modules (CKM)
3.   Loading Knowledge Modules (LKM)
4.   Integration Knowledge Modules (IKM)
5.   Journalizing Knowledge Modules (JKM)
6.   Service Knowledge Modules (SKM)

Reverse-engineering Knowledge Modules (RKM) : 

The RKM is in charge of connecting to the application or metadata provider then transforming and writing the resulting metadata into Oracle Data Integrator’s repository. The metadata is written temporarily into the SNP_REV_xx tables.

Check Knowledge Modules (CKM) :

 

The CKM is in charge of checking that records of a data set are consistent with defined constraints they can check either an existing table or the temporary "I$" table created by an IKM.

CKM operates in both STATIC_CONTROL and FLOW_CONTROL

In STATIC_CONTROL mode, the CKM reads the constraints of the table and checks them against the data of the table. Records that don’t match the constraints are written to the "E$" error table in the staging area.

In FLOW_CONTROL mode, the CKM reads the constraints of the target table of the Interface. It checks these constraints against the data contained in the "I$" flow table of the staging area. Records that violate these constraints are written to the "E$" table of the staging area.

Loading Knowledge Modules (LKM):

An LKM is in charge of loading source data from a remote server to the staging area.It is used by
interfaces when some of the source data stores are not on the same data server as the staging area.The LKM creates the "C$" temporary table in the staging area. This table will hold records loaded
from the source server.

Integration Knowledge Modules (IKM):

The IKM is in charge of writing the final, transformed data to the target table. Every interface uses a single IKM.all remote source data sets have been loaded by LKMs into "C$" temporary tables in the staging area, or the source datastores are on the same data server as the staging area. 

Therefore, the IKM simply needs to execute the "Staging and Target" transformations, joins and
filters on the "C$" tables, and tables located on the same data server as the staging area. The resulting set is usually processed by the IKM and written into the "I$" temporary table before loading it to the target .


IKM will written into the "I$" temporary table before loading it to the target.
These final transformed records can be written in several ways depending on the IKM selected in your interface. They may be simply appended to the target, or compared for incremental updates or for slowly changing dimensions.





Finally data flow looks like

Journalizing Knowledge Module(JKM):

JKMs create the infrastructure for Change Data Capture on a model, a sub model or a data store. JKMs are not used in interfaces, but rather within a model to define how the CDC infrastructure is initialized.


Service Knowledge Modules (SKM):

SKMs are in charge of creating and deploying data manipulation Web Services to your Service Oriented Architecture (SOA) infrastructure.

Several Knowledge Modules are provided to export data to a target file or to read data from a source file.

Reading from a File:

• LKM File to SQL
• LKM File to DB2 UDB (LOAD)
• LKM File to MSSQL (BULK)
• LKM File to Netezza (EXTERNAL TABLE)
• LKM File to Oracle (EXTERNAL TABLE)
• LKM File to Oracle (SQLLDR)
• LKM File to SalesForce (Upsert)
• LKM File to SAS
• LKM File to Sybase IQ (LOAD TABLE)
• IKM File to Teradata (TTUs)
• LKM File to Teradata (TTUs)

Writing to a File:

• IKM SQL to File Append
• IKM Netezza To File (EXTERNAL TABLE)
• IKM SalesForce to File (with filter)
• IKM SalesForce to File (without filter)
• IKM Teradata to File (TTUs)

Sources : ODI Developers Guide

                KM Users Guide