12.1向11.2传输数据

While GoldenGate is used most often to replicate database to database there are times when writing out to a file could be advantageous.

Replicating to an unsupported database is one example as well as creating flat files for Hadoop. In this post we will take an existing install of GoldenGate 12.1.2 running against a 12.1.0.1 Oracle database and send the data to a server running the GoldenGate 11.2.1.5 flat file adapter. Not going to cover the install of GoldenGate against 12c as that has already been covered in an earlier post. After installing GoldenGate and creating an extract and a pump we are ready to create the configuration files for the source. Below is the parameter file for the extract. EXTRACT e022

USERID ggs_user,password *************** TRANLOGOPTIONS DBLOGREADER

EXTTRAIL dirdat/e2, FORMAT RELEASE 11.2 TABLE SOE.*;

Its a pretty basic parameter file. The only thing to note is since the target side is running GoldenGate 11.2 we need to ensure the trail file is written in a 11.2 format to avoid

OGG-01411 Oracle GoldenGate Capture for Oracle, wpump.prm: Cannot convert input file dirdat/e2000000 with format LEVEL 4 to output file dirdat/ff000001 with format LEVEL 3.

Next is the pump parameter file EXTRACT wpump

RMTHOST ****, MGRPORT 7809

RMTTRAIL dirdat/ff, FORMAT LEVEL 3 PASSTHRU

TABLE SOE.*;

Again a very basic pump with the sole exception of FORMAT LEVEL 3 again since we are replicating from GoldenGate 12.1 to 11.2 we need to use a compatible format to avoid the following error on the target side. OGG-01332 Oracle GoldenGate Capture, ffwriter.prm: File

dirdat/ff000000, with compatibility level 4, is not compatible with the current software version's compatibility level of 3. Modify the file writer's parameter file to generate the appropriate format using the FORMAT LEVEL 3 option.

With the source side ready we can focus on the target. Since the target isn’t running a database we are going to use generic version of GoldenGate that includes the flat file adapter. You can find the adapter on

edelivery.oracle.com under the fusion middleware. The adapter download is under one of the non Oracle packs. Download the file and run the usual unzip, untar. Once untarred is still a basic ggsci, create subdirs, setup manager parameter file.

If you are running a Unix/Linux you should see flatfilewriter.so in the GoldenGate installation directory (on Windows its flatfilewriter.dll). In the case of a database as the target you have a replicat that reads the trail files and applies the data to the database. With the file adapter its an extract that reads the trail files and calls a library (flatfilewriter.so) to write out the data to the flat files.

To create the extract we run the command

add extract ffwriter, exttrailsource dirdat/ff

Next is the parameter file for the extract and as you will see its not a simple standard parameter file. EXTRACT ffwriter

SOURCEDEFS dirdef/oracle.def

CUSEREXIT flatfilewriter.so CUSEREXIT PASSTHRU, INCLUDEUPDATEBEFORES, PARAMS \TABLE SOE.*;

First you will notice we do need a definition file (not covered in this post). Next is the CUSEREXIT line. This line states the extract is to perform a C call out. The file being called is the flat file adapter library. The function being called in the library is USEREXIT, PASSTHRU states the extract doesn’t need to write a trail file. The INCLUDEUPDATEBEFORES send both before and after image data to the library and is required for consistency purposes. The file parameter is the PARAMS followed by a file name. This file is the configuration file for the flat file library and is where most of the configuration for controlling how the flat files are written out set.

Below is the contents of the ffwriter.properties file goldengate.flatfilewriter.writers=dsvwriter dsvwriter.mode=DSV

dsvwriter.rawchars=false

dsvwriter.includebefores=false dsvwriter.includecolnames=false dsvwriter.omitvalues=false dsvwriter.diffsonly=false

dsvwriter.omitplaceholders=false

dsvwriter.files.onepertable=true dsvwriter.files.data.ext=_data.dsv

dsvwriter.files.data.tmpext=_data.dsv.temp dsvwriter.dsv.nullindicator.chars= dsvwriter.dsv.fielddelim.chars=;

dsvwriter.dsv.fielddelim.escaped.chars= dsvwriter.files.data.rootdir=./out

dsvwriter.files.data.rollover.size=10000

Not going to even attempt to explain what all the parameters as this is only a small sample of what is possible and chances are you will need to do something different. Highly recommend reading the documentation. But I will cover a few important ones.

goldengate.flatfilewriter.writers=dsvwriter – using a dsvwriter, there a couple of other types of writers to choose from

dsvwriter.mode=DSV – writing a character delimited file, could also choose LSV for fixed length

dsvwriter.includecolnames=false – not writing column names to the file dsvwriter.files.data.ext=_data.dsv – extension used for naming the files

dsvwriter.files.data.tmpext=_data.dsv.temp – extension for the temp files

dsvwriter.files.data.rootdir=./out – where are the files stored

dsvwriter.files.data.rollover.size=10000 – the size of the temp before it rolls over to a final file

dsvwriter.files.onepertable=true – every table being replicated will have its own text file

Basically how the flat file adapter works is as transactions are received they are written to a temp text file. After so many minutes or the file grows to a certain file size the temp file is renamed. Now start up the source and target and check to see if you are running text files. ls

SOE.ADDRESSES_2014-05-28_15-16-26_data.dsv SOE.CUSTOMERScontrol SOE.ORDER_ITEMS_2014-05-28_15-16-26_data.dsv

SOE.ADDRESSEScontrol SOE.INVENTORIES_2014-05-28_15-16-26_data.dsv SOE.ORDER_ITEMScontrol

SOE.CARD_DETAILS_2014-05-28_15-16-26_data.dsv SOE.INVENTORIEScontrol SOE.ORDERS_2014-05-28_15-16-26_data.dsv

SOE.CARD_DETAILScontrol SOE.LOGON_2014-05-28_15-16-26_data.dsv SOE.ORDERScontrol

SOE.CUSTOMERS_2014-05-28_15-16-26_data.dsv SOE.LOGONcontrol

Above is a directory listing. Files ending in .dsv are the data files but you will notice that for each table being replicated there are two files, a data and a control. Below is a sample of one of the control files. cat SOE.ADDRESSEScontrol

./out/SOE.ADDRESSES_2014-05-28_15-16-26_data.dsv

Basically control files handle the problem of when is the text file finished being written? The answer is when the file name appears in the control file. Below is a sample of one of the text files. 503745;1003383;\Name\1503746;1003384;\Name\1503747;1003385;\Name\1503748;1003386;\Name\1503749;1003387;\

Name\

With the data now in text form can be loaded into its final destination. You will need to create some process that monitors the control file and as new files come in the dsv files are processed. In this case the data doesn’t have an operation type so we would only use this setup for an initial load or if GoldenGate was transforming all operations into insert statements. In a future post will show how to handle operations like update and deletes.

本文来源：https://www.bwwdw.com/article/8i57.html