Correspondence Transfer for the Registration of Multimodal Images

Gene expression data provide information on the location where certain genes are active; in order for this to be useful, such a location must be registered to an anatomical atlas. Because gene expression maps are considerably different from each other – t

CorrespondenceTransferfortheRegistrationofMultimodalImages

ZhaoYiStefanoSoatto

ComputerScienceDepartment,UniversityofCalifornia

LosAngeles,CA90095

{zyi,soatto}@cs.ucla.edu

Abstract

Geneexpressiondataprovideinformationonthelo-cationwherecertaingenesareactive;inorderforthistobeuseful,suchalocationmustberegisteredtoananatomicalatlas.Becausegeneexpressionmapsareconsiderablydifferentfromeachother–theydisplaytheexpressionofdifferentgenes–andfromtheanatomicalatlas,thisproblemiscurrentlyaddressedeitherman-uallybytrainedexperts,orbyneglectingallimagein-formationandonlyusingthepre-segmentedboundaries.Inthismanuscriptweconcentrateondatadiscrepancymeasuresthattakeintoaccountimageinformationwhenthisispresentinboththetargetandtemplateimages.Weexploitsuch“bi-lateral”structurestodrivethecor-respondenceprocessinregionswheretheintensityin-formationisinconsistent,analogouslytoa“motionin-painting”task.Althoughnogroundtruthcanbeestab-lished,andpriorinformationclearlyplaysakeyrole,weshowthatourmodelachievesdesirableresultsonsubjectivetestsvalidatedbyexpertsubjects.

1.Introduction

Establishingcorrespondencebetweendifferentim-agesiskeyforustoinferpropertiesoftheunderlyingscene.Thebasicassumptionisthatthereissomethingcommonbetweentheimages,modulodomaindeforma-tions(e.g.inducedbyviewpointchangesorbyscenede-formations)andrangedeformations(e.g.contrasttrans-formationsinducedbychangesinillumination,orbychangesofimagingmodality).Suchcommonalitymaybeabstract,ratherthanphysical,forinstancewhentheimagesportrayobjectsinthesamecategory,say“hip-pocampus,”eventhougheachimageportraysadifferentphysicalobject.Acrucialcomponentofanyapproachtoregistrationisthemechanismusedtocomparetwo(de-formed)images:Whilerange(intensity)similarityisa

naturalchoice,forinstancemeasuredinthesenseofL2[30]orTotalVariation[26],extremecontrastchangeshavebeensuccessfullytackledusingMutualInforma-tion[22].

Themostrecentdevelopmentsinmedicalimag-ing,however,arechallengingthesepremisesaltogether:Geneexpressiondataaregeneratedwithdifferentstains,highlightingdifferentgenes,withtheexpressgoalofmakingeachresultingimageasdifferentaspossiblefromtheothers,inordertomaximizetheirinformationcontent.Nevertheless,thepractitionerrequiresregister-ingsuchimagestoanatomicalatlases,inordertoascribetheactivityofagenetoaparticularanatomicalstruc-ture(Fig.1).Thesamegoesforregisteringfunctionalimaging(e.g.F-MRI)toanatomicalatlases,ataskthatisbyandlargeperformedmanuallybytrainedphysicians.Whilethisisdoableforahandfulofsubjects,system-aticstatisticalstudiesofgeneexpressiondatainlargepopulationscallforsomedegreeofautomation.Butwhatdoesitmeantoestablishcorrespondence,whenthereisnocommonunderlyingstructure,andwhenthedataaredesignedtobeasdifferent(“indepen-dent”)fromeachotheraspossible?Clearlyexpertpriorknowledgeofanatomyandbiologicalfunctionalityisin-dispensable,andseveralresearchgroupsareactivelyen-gagedinmodeling,learningandenforcingshapepriorsinsegmentationandregistration[16,25].Nevertheless,anyregistrationalgorithmmustalsotakeintoaccounttheavailabledata,andthisproblemhasbeenlargelyoverlookedintheliterature,wheremostlystandarddatatermsareused[15,24],orwhereonlytheboundaryin-formationistakenintoconsiderationandtherestofthedeformation eldisdeterminedbygenericregulariza-tion[8,20,27].Therefore,inthismanuscriptwefocusourattentionondevisingsuitabledatatermsforregister-ingmulti-modalimages.Ourgoalistodesignaschemetotakevisiblegeometricstructures(onecouldcallthem“landmarkregions”)intoaccountwhentheyarepresent

Gene expression data provide information on the location where certain genes are active; in order for this to be useful, such a location must be registered to an anatomical atlas. Because gene expression maps are considerably different from each other – t

Figure1.Geneexpressiondata(top-right)andamanuallyse-lectedanatomicaltemplate(top-left).Detailedcorrespondenceiscurrentlyperformedlargelybyhandbytrainedphysicians.Syntheticphantoms(bottom):Certainregionsarevisibleinboththetargetandthetemplate,althoughdeformed,whereasotherregionsarevisibleonlyinoneofthetwo.Thegoalistoexploit“bilateral”regionstodrivethecorrespondenceprocessfor“unilateral”regions.

inbothimages,andusetheirregistrationasboundaryconditionto“guide”thestructuresthatarepresentinoneimage(e.g.geneexpression),butnottheother(e.g.theatlas).Theproblemisbestillustratedwithaphantom,or“cartoon”(Fig.1).Atemplateimage(left)exhibitssomevisiblestructures(e.g.greyregiononthetopleft),andoneisinterestedindeterminingwheresuchstruc-tureislocatedonanatlas(right).Unfortunately,suchastructureisabsentintheatlas!Therefore,weneedto“transfer”correspondenceinformationfromcommon(or“bilateral”)structuresinordertoinferthemotionanddeformationof“unilateral”ones.Onecouldthinkofthisproblemas“motioninpainting”[3],althoughonewheredomainknowledgeplaysaconsiderablerole.Anyapproachthatreliesonrawintensityinformationfailsthistaskbecause,bydesign,oneassumesthatim-agesareequivalentuptodiffeomorphicdomaintrans-formations[2,16].InFig.2weshowtheeffectsofacommonintensity-basedalgorithmonregisteringthephantomsofFig.1:Bilateralstructuresaremappedcor-rectly,butunilateralonesshrinktoapoint,inducingasingularity(sink)inthewarpingthatisnotphysicallyplausibleinthiscontext(althoughitwouldbeappropri-ateina“growth”model[10]).Thisproblemis

mostly

Figure2.CorrespondenceforthephantomsinFig.1usingonlyintensityinformation(top),usingonlygeometricinfor-mation(middle),andusingthecombinedmodelwepropose(bottom).Ineachcaseweshowthedeformation eld(left)andthemappedtemplate(right).Inthecaseofintensityin-formationalone,unilateralregionsdisappear(top).Inthecaseofgeometricinformationalone,bilateralregionsarenotde-formedcorrectly(middle).Inthecombinedmodel,bilateralregionsaredeformedaccordingtothedata,whereasunilateralregionsaremappedaccordingtogeometricinformation(bot-tom).

addressedincurrentliteraturebyneglectingintensityin-formationaltogether,usinginsteadtheouterboundaryoftheslice.Inthiscase,thedeformationissmooth,butbilateralstructuresarenotmappedcorrectly(Fig.2).Ourgoalistobridgethisgap:Wherebilateralstructuresarepresent,wewanttousethemtoguideourwarping.Unilateralstructures,ontheotherhand,shouldbepre-servedandmappedontotheatlas.Whatweneedisaspatially-varyingcriterionthatusesintensityinforma-tiononlywhereavailable.Wewillformulatethisprob-lemasaprobabilisticinference,wherethelikelihoodofthedataisweightedateachpointbytheprobabilityof

Gene expression data provide information on the location where certain genes are active; in order for this to be useful, such a location must be registered to an anatomical atlas. Because gene expression maps are considerably different from each other – t

therebeingabilateralstructure.2.Formalizationoftheproblem

LetI1,I2:D R2→R+betwoimagesx→Ij(x),j=1,2,andw:D→Dadiffeomorphismofthedomainofoneontotheother.Withinthedomainofeachimageliesaregionofinterest j D,whereasthe“background”D jisassumedsegmented(orequiva-lentlyIjcanbeassumedtohavezerovalueoutside j.)Werepresentregions Dusingthesigneddistance

functionφ(x| )=.

±miny∈ (|x y|),x∈Dwiththepositivesignforx∈ andnegativeotherwise.ThefunctionφisatleastLifschitzcontinuous[19].

Withineachdomain j,assumethatthereareregionsBj j,j=1,2(notnecessarilysimplyconnected)thathaveadistinctphotometricsignaturesothattheycanbedetectedbyalow-levelimageprocessingalgo-rithm.Wewillmakethisprecise,andindeedwewillrelaxthisassumptionlater;fornow,assumethattheregionsBj,j=1,2areknown.Wecallthesebilat-eralregions,inthesensethattheyaredetectedinbothimages(Fig.1).Ontheotherhand,thereareregionsU jthataredetectedinoneimagebutnottheother,whichwecallunilateral.ForthesakeofillustrationwewillassumethatU 1.ThisscenarioisdisplayedinitsmostelementaryforminFig.1.

Ourmodelisbasedonthepremisethat,locallyaroundbilateralregions,thewarpingwisdeterminedbyintensityinformation,whereasawayfrombilateralregions,whereintensityisconstantorinconsistentbe-tweenthetwoimages(e.g.aroundunilateralregions),thediffeomorphismisdeterminedbythegeometryoftheregions 1, 2,aswellasbygenericregularizers.Theseassumptionscanbetranslatedintoasimplegen-erativemodel

I1(w(x))=I2(x),x∈ 2∩Bσ(B2)

B1=w(B2)

(1)φ(w(x)| 1)=φ(x| 2),x∈DwhereBσ(C)isaregionincludingCbyamarginσ>0(e.g.theunionofCwithacoveringofballsofradiusσaround C).Wewill rstreviewcriteriatoinferthediffeomorphismwbasedongeometricinformationonlyinSect.2.2(i.e.neglectingthevalueofIj(x),x∈ j);thenbasedonphotometricinformationonly(i.e.ontheintensityvalueoftheimages)inSect.2.3,and nallydiscussourmodelinSect.2.4,forwhichwewillprovideaprobabilisticinterpretationinSect.2.5.Beforedoingso,however,wediscusstheissueofvalidation.

2.1.Onvalidation

Naturally,becausethereisnodatatosupportcor-respondenceofunilateralstructures,theresultwillbeadirectconsequenceofourassumptions(or“model”,“prior”or“regularizers”,dependingontheparlanceofthescienti cdomainofpreference).Inthissense,theproblemisbothscienti callyill-de ned(i.e.non-falsi able),andmathematicallyill-posed(therearein- nitelymanysolutionsthatarenon-continuouslyde-pendentontheinitialconditions).Asaconsequence,“groundtruth”cannotbeestablished–similartoIn-painting[3]–andtheproblemistautologicallyde nedbyitssolution.Ultimately,thequalityofourresultcanonlybejudgedsubjectivelyonexperimentsperformedbyhighlytrainedanatomiststhatcanestablishsuchacorrespondencebasednotontheavailabledataalone(aswedo),butbasedonhigh-levelknowledgethatisnotavailabletotheuntrainedeye,re ectingtheclinicalvalueofaproposedscheme.

2.2.Geometry-drivencomponentcost

Thesimplestmodeltoperformregistrationbasedonlyontheshapeoftheregions jcrepancytermbetweenφ1(x)=.istominimizeadis-φ(x| 1)andφ2(x)=.

φ(x| 2),forinstancetheL2norm.Becausethedif-feomorphismisin nite-dimensional,theproblemisill-posed,henceweneedtoimposesomeregularization,forinstancetheL2normofitsgradient:

w =.argminΦ.

w

geom+βΦreg=

=.1|φβ1(w(x)) φ2(x)|2+| w(x)|2(2)

D2

2dx

whereβ>0isatuningparameterand|v|2=.

vTvde-notesthesquaredtwo-normofavector.Ofcoursemoreelaboratemodelsandtechniquescanbeemployed,andthereaderisreferredtotheliterature,forinstance[1].However,thissimpleonesuf cesforustointroduceourmodeltocombinegeometricwithintensityinformation.Wereviewthesole-intensitymodelnext.

2.3.Intensity-drivencomponentcost

Correspondencebasedonintensityinformationisa eldalmostasbroadasComputerVisionitself,soobvi-ouslynofairreviewoftheliteraturecanbeprovidedinthisvenue.Wewillchooseoneofthesimplestmodelsthatcanserveourpurpose,namelytheL2matchingcri-terionthatcorrespondstorestrictingtheclassicalHorn

Gene expression data provide information on the location where certain genes are active; in order for this to be useful, such a location must be registered to an anatomical atlas. Because gene expression maps are considerably different from each other – t

andShunk ow[11]totheregionsofinterest:

w =argmin.

w

Φint+βΦreg=

=.

1|Iβ1(w(x)) I2(x)|2+| w(x)|2dx.

2∩Bσ(B2)2

2(3)Thismodelisratherrestrictive,inthatitdoesnotal-lowintensityvariationsamongthetwoimagesandas-sumesthattheyaresimplydiffeomorphicallyequiva-lent[10,16].Itcanberelaxedbyallowingsimple(global)changesincontrastandscaling,eitherviapre-processing,orbyaugmentingthemodelwithadditionalparametersthatcanbeinferredalongwiththewarpingw .Moregenerally,thecostfunctioncanbemodi edbyallowingtheintensitiestobedifferent,solongasthemutualinformationbetweenthetwoimagesismax-imized[15,24,29].OfcourseothervariationsusingotherLpnorms[12,28],totalvariation[26],Kullback-Lieblerdivergence[6],Bhattacharyadistancesbetweenregionhistograms[7],oramyriadofdifferentregular-izerscanalsobeemployed.Also,thefunctionalabovecanbemadesymmetricwithrespecttowhichimage(inourcaseI1)ismappedtowhich,byallowingtwodiffeo-morphismstowarpbothimagestoacommontemplate[17].

binedfunctionalandbilateralregion

detection

Aswehaveanticipated,ourapproachconsistsofus-ingintensityinformation(orotherintensitystatistics,forinstancegradienthistograms)wheresalientregionsaredetectedinbothimages.Followingthemodelabove,thissimplytranslatesintoafunctionaloftheform

Φ(w)=

1

2

|I1(w(x)) I2(x)|2χ D

2∩Bσ(B2)(x)++α

2

|φ1(w(x)) φ2(x)|2χ c2∪Bcσ(B2)(x)++β

2

| w(x)|2dx(4)

whereχS(x)isthecharacteristicfunctionofasetS,α>0isatuningmultiplier,andthesuperscriptcde-notesthecomplementinD.

Now,theuseofthecharacteristicfunctionsaboveas-sumesthatthebilateralregionsBjhavebeendetected,andthisisusuallyaccomplishedbyalow-levelvisionalgorithm.Likeanyotherdecisionproblem,thiswillin-volveselectingathresholdonsomestatisticoftheimageintheneighborhoodofBj,involvingtheprobabilitythat

xbelongstoit.Forinstance,onecancomputelikelihoodratiosbasedonthegradientoftheimage,orbetteryetlookforextremaofoperatorsinscale-space[14].Ratherthanassumingthatthisdecisionhasbeenmadeforus,wewillsimplyweightthegeometricandintensitytermsatapointxbytheprobabilitythatsuchapointbelongstoa“structure,”usingthesamecriterionthatalow-levelstructurecriterionwoulduse.Forthesakeofillustra-tion,wewillusethenormalizedgradientofGaussianscale-spaceoftheimage,following[14],thatisequiva-lenttoassumingP(x∈B2)=| normI2(x)|∈[0,1].Naturally,forB2tobeabilateralregion,itwillhavetohaveacorrespondenceinimageI1,soitisnotsuf -cienttoevaluatethegradientatI2,wemustalsoevalu-ateitatI1,warpedviaw,sothatthecriterionbecomes| normI1(w(x))|·| normI2(x)|.Wewillwritethisintermsofprobabilitiesinthenextsubsection,anddiscusshowtoextendittomoregeneraldiscrepancyfunctionssuchasmutualinformationinSect.3;fornowwejustnoticethatthecostfunctionalabovebecomes

Φ(w)=

1

|I1(w(x)) I2(x)|2| normI1(w(x))|·D2

·| x)|+α

normI2(2

|φ1(w(x)) φ2(x)|2·

·(1 | Iβ

norm1(w(x))|·| normI2(x)|)+| w(x)|22

dx.

(5)Thisfunctionalonlyconsidersintensitywherethenor-malizedscale-spacegradientislargebothinthetargetimageandinthewarpedtemplate.Thisonlyhappensonandaroundbilateralregions,toanextentthatdependsonthescaleofsuchregions(see[14]fordetailsonau-tomaticscaleselection).Wheresuchconditionsarenotsatis ed,thegeometrictermandthegenericregularizerdrivetheenergy.Anaddedbene tisthat,becausewehaveassumedthattheimageshavebeenmaskedsothatthebackgroundiszero,wecansimplyperformtheinte-gralonDwithoutrestrictingportionsofitto 2.

TominimizeΦ(w),variationalcalculusyieldsthe rstvariation(forsimplicityweonlyconsiderU 1)δΦ

δw

=(I1(w(x)) I2(x)) I1(w(x))··| normI2(x)|+α(φ1(w(x)) φ2(x)) φ1(w(x))·

·(1 | normI2(x)|) β 2w(x)

(6)

with 2theLaplacianoperator.Bygradientdescentwithbacktrackinglinesearch[23]weobtaintheasso-ciatedEuler-Lagrangeequations,parameterizingthede-

Gene expression data provide information on the location where certain genes are active; in order for this to be useful, such a location must be registered to an anatomical atlas. Because gene expression maps are considerably different from each other – t

scentdirectionbyanarti cialtimet≥0:

w t= δΦ

δw

.(7)Thetemporalandspatialgradientsareapproximatedby nitedifferencemethods.

Again,thismodelonlyusesthesimplestintensityterm,andthesimplestgeometricterm.Ouremphasisisinhowtocombinethetwo.Onecanconceivewaysinwhichthisapproachcanbeextendedtomorecomplexfunctionals,anissuewediscussinSect.3.

2.5.Probabilisticinterpretation

Informalterms,ourgoalcanbestatedasseekingthemaximuma-posterioriprobabilityofadiffemorphicwarping,thatis

w =.arginfw

logp(w|I1,I2)=

=arginfw

logp(I1,I2|w)p(w).

(8)

Thesecondterm,logp(w),canbeeasilyrecog-nized, inthemodel(5),asthegenericregularizer

12likelihoodD| w(x)|dx.Soweconcentrateonthelog-termlogp(I1,I2|w).Thiscanbeobtainedviap(I1,I2|w)=p(I1|I2,w)p(I2).Tothisend,themod-elsproposedby[18]couldbeemployed,inprinciple,sowhatweneedtocomputeisp(I1|I2,w).Now,againinpurelyformalterms,wecouldrepresenttheprobabil-ityofmatchingbilateralregionsasP(B1 B2);thenwhatwewishtocomputeis

p(I1|I2,w)=p(I1|I2,w,B1 B2)P(B1 B2)+

+p(I1|I2,w,U )(1 P(B1 B2)).(9)Now,thisisjustformalnotation.Thedif cultycomesinwhenwetrytowriteexplicitlytheprobabilitiesabove,becausetheconditionB1 B2isspeci ctoeachpointx∈D,soagainwehavetospecifythespatialstatisticsoftheimage,whichwouldleadtoaninferenceproblemwhereallpossiblecombinationsofstatesarepossibleandtime-consumingMarkov-ChainMonteCarlometh-odsbecomenecessaryratherthansimplelocaldescentalgorithms.

So,insteadofattemptingtocomputetheabovelike-lihood,wewillapproximateitbyassumingthatallpix-elsareindependent,andcomputinganaverage(expec-tation) overpixelsoftheprobabilitye 1|I1(w(x)) I2(x)|2 e 1

)) φ2(x)|2|φ1(w(xB1 B2

U

(10)

which,modulotechnicalities,shouldconvergeto(4),the“stiffversion”ofourfunctional(5).

3.Extensions

Whiletheapproachdescribedintheprevioussec-tioncanbeapplieddirectlytosimplesyntheticphantomssuchasthoseinFig.1,applicationtorealmulti-modalimagesrequiresamorepowerfulmodel.

Speci cally,ratherthanL2,weusemutualinforma-tion[21]betweenthedeformedtemplateI1 wandthetargetI2,denotedby

Φp(I1,I2|w)

MI(w)=

log

p(I(IdP(I1,I2|w).(11)

1|w)p2)

Estimationforthejointimageintensitydistribution

p(I1,I2|w)iscarriedontheregionofoverlapAofbothimagesbyusing2-DParzenwindowingwithGaussiankernelGσ:

p(I1

1(w(x)),I2(x)|w)=|A|Gσ(I1(w(x))

A

I1(w(y)),I2(x) I2(y))dy.(12)Itisequivalenttoconvolvingthejointintensityhis-togramwithadiscreteapproximationofGσ.ThejointhistogramsofI1 wandI2withintheirregionofoverlapareconstructedbybinningthecorrespondingintensitypairs(I1(w(x)),I2(x)),andthemarginalhistogramsareobtainedbyintegratingoverrowsandcolumns,re-spectively.

Substituting(12)into(11)andrearrangingfollowing[4,9],yieldsthe rstvariationofΦMI:δΦMI

=1 δw |A|·I1,I ·G Lσ w

2

I(I1(w(x)),I2(x)) I1(w(x))

(13)

1

with theconvolutionoperator,andLIw1,I2

givenby

LI(I1,I2|w)

w1,I2

=1+log

pp(Iw)p(I.

(14)

1|2)

InplaceofanL2regularizer,weusea uidmodel[5]wherethedeformationvelocityv(x,t)isgovernedbythesimpli edNavier-Stokesequations

µ 2v+(µ+λ) ( ·v)+f=0

(15)

withµandλviscosityconstants.Herefistheforce eldwhichdrivesthewarpingwintheappropriatedi-rection.Itisderivedfromimageinformation,usuallyset

Gene expression data provide information on the location where certain genes are active; in order for this to be useful, such a location must be registered to an anatomical atlas. Because gene expression maps are considerably different from each other – t

tothe rstvariation.Inthecombinedmodel,wehavef(x,w(x))= δΦMIδΦgeom

δw δw

=

=1|A|G LI1,I2

σ w

I(I1(w(x)),I2(x)) I1(w(x))·1·P(x∈B2) α (φ1(w(x)) φ2(x)) φ1(w(x))·

·(1 P(x∈B2)).(16)Thedeformationu(x)=x w(x)issuccessivelyaccu-mulatedthrough

u

t

=v (v· )u.(17)

Thisextensionisratherstraightforward,theonlysigni cantchangebeingtheexpressionoftheweightP(x∈B2).Sincemutualinformationbetweentwoim-agesisrelatedtopixellocationsthroughintensitydistri-butions,weneedtonotonlyconsidertheintensitygra-dients,butalsothespatialgradientsofintensitydistribu-tions.Adirectextensionoftheargumentfor(5)leadstomultiplyingthetargetintensitygradientsbythespatialgradientofaParzenwindow-basedjointdensityestima-torfromthesamplehistogramsofthetwoimages,i.e.,P(x ∈B2)=| normI· 2(x)|·

LI1,I2 Gσ w I(I1(w(x)),I2(x)) .(18)

2

norm

4.Experiments

Inthissectionwereportasubsetoftheexperimentswehaveconductedtovalidatethemodelproposed.AsdiscussedinSect.2.1,ground-truthcannotbeestab-lishedforcorrespondencetransfersincethereisnocor-respondenceforunilateralregions,sovalidationisper-formedsubjectivelybydomainexperts,andultimateperformancewillhingeonhowourapproachisinte-gratedwithshapepriorsandotherhigh-levelinforma-tion.

Fig.2showstheresultsofapplyingourapproachtothesimplesyntheticphantomsshowninFig.1.Itispatentthatourapproachhasthedesirablepropertyofnotmakingtheunilateralregionsdisappear,andatthesametimeofproperlydeformingbilateralregions.

InFig.3weillustrateourapproachonrealdata.Inordertodoso,wemustuseamoreelaboratediscrep-ancymeasure,asdiscussedinSect.3.Inthetoprowweshowtheresultsobtainedusingonlyanintensitytermwherediscrepancyismeasuredusingmutualinforma-tion.Asitcanbeseen,thedeformationgridis

rather

parisonwithMutualInformation:(top)warping

andregistrationusingmutualinformation,and(bottom)usingourapproach.Noticethatthedarkregioninthetemplateismappedinahighlyirregularfashiontothetarget,andtheitsgeometricstructureisnotpreserved.Ourcombinedmodel,displayedinthebottompartoftheimage,showsamoreplau-sibledeformation eld,withunilateralregionsbeingsmoothlymappedintotheanatomicaltemplate,andbilateralregionscor-rectlydeformed.

irregular,andinparticularthedarkregionoftheleftbe-comes“turbulent”inawaythatisnotcompatiblewithhigh-levelknowledgeofanatomy.Ourapproach(bot-tom),ontheotherhand,showsthatunilateralregionsaremappedsmoothlywhilebilateralfeaturesaredeformedconsistently.AdditionalexamplesondifferenttestdataarereportedinFig.4.

InFig.5weshowsomerepresentativeexperimentswhereourapproachfailstoyieldameaningfulcorre-spondence.Asitcanbeseen,thetemplateisdeformedlocally,sothefoldvisibleinthetargetisnotcorrectlymapped.Thiskindofbehavioristobeexpectedsinceouralgorithmisbasedonadatadiscrepancytermalone,whichencodesbottom-up,low-levelinformationandisoblivioustoanyknowledgeoftheanatomyorphysicsoftheunderlyingstructures.Thiscanbeobviatedbytakinghigh-levelpriorinformationintoaccount,anissuethatisbeyondourscopeinthispaper,whereconsiderableef-fortsareundergoinginthemedicalimagingcommunity.

Gene expression data provide information on the location where certain genes are active; in order for this to be useful, such a location must be registered to an anatomical atlas. Because gene expression maps are considerably different from each other – t

Figure4.Somemorerepresentativeexampleswhereourcom-binedmodelyieldssubjectivelysuccessfulcorrespondenceofunilateralregions.Thetemplate(top-left),andtarget(top-right)aremappedoneontotheotherbyadeformation eld(bottom-left)thatyieldsthedeformedtemplate(bottomright).

4.1.Validationanddiscussion

AswehavediscussedinSect.2.1,nogroundtruthcanbeavailablefortheproblemweaddress.Thatis,unlesshigh-levelpriorknowledgeisbroughttobearthatouralgorithmdoesnotexploit.Ultimately,ouralgo-rithmwillhavetobecomplementedwithshapepriors,similarlytowhatdoneby[13].Fornow,however,welimitourselvestosubjectiveevaluationofourresultsbyexpertanatomists.Atthisstage,cross-validationandotherstatisticaltestscannotbeperformedbecause

there

Figure5.Exampleoflimitationsofourmodel.Fine-scalege-ometricfeatures(e.g.thesmallgaponthetoprightportionoftheatlas)arenotmappedcorrectlybecauseourmodelonlyuseslow-levelinformationandisnotcognizantofanatomicalstructureandconstraints.Itisclearthatpriorknowledgeofthegeometryoftheunderlyinganatomyhastobeenforcedtoachievethereliabilityandprecisionofhumanexperts.

isnosecondarytask(e.g.classi cation)forwhichwecouldhavegroundtruth.Itispossiblethatthesewillbecomeavailableinthefuture(e.g.post-mortemstud-iesofeachindividualsubject),butthatwouldnotbepractical.Ourhopeisthatourmodel,integratedwithsuitableshapepriors,canhelpsciencebyautomatingmulti-modalregistrationtasks.Ourexperimentsshowthatincludinganexplicitmodelofthecorrespondencehypothesisforbilateralversusunilateralregionsthereareimprovementsoverbothtraditionalintensity-basedregistration,aswellasovermutualinformation-basedapproaches.

Acknowledgements

ThisworkwassupportedbyAFOSRFA9550-06-1-0138.WethankGregorioGuidiandAndreaVedaldifortheircommentsandsuggestions,andErh-FangLeeandIvoDinovforprovidingdataandfeedback.

References

[1]M.A.Audette,F.P.Ferrie,andT.M.Peters.Anal-gorithmicoverviewofsurfaceregistrationtechniquesformedicalimaging.MedicalImageAnalysis,4:201–217,2000.3

Gene expression data provide information on the location where certain genes are active; in order for this to be useful, such a location must be registered to an anatomical atlas. Because gene expression maps are considerably different from each other – t

[2]M.F.Beg,M.I.Miller,A.Trouve,-putinglargedeformationmetricmappingsviageodesic owsofdiffeomorphisms.IJCV,61:139–157,2005.2[3]M.Bertalmio,L.Vese,G.Sapiro,andS.Osher.Simul-taneousstructureandtextureimageinpainting.IEEETrans.ImageProcessing,12:882–889,2003.2,3[4]C.Chefd’Hotel,G.Hermosillo,andO.Faugeras.A

variationalapproachtomulti-modalimagematching.InIEEEWorkshoponVariationalandLevelSetMethodsinComputerVision,pages21–28,2001.5[5]G.E.Christensen,R.D.Rabbitt,andM.I.Miller.De-formabletemplatesusinglargedeformationkinematics.IEEETrans.ImageProcessing,5:1435–1447,1996.5[6]A.C.S.Chung,W.M.WellsIII,A.Norbash,and

W.E.L.Grimson.Multi-modalimageregistrationbyminimisingkullback-leiblerdistance.InMICCAI,pages564–571,2002.4[7]aniciuandP.Meer.Meanshift:arobustap-proachtowardfeaturespaceanalysis.PAMI,24(5):603–619,2002.4[8]C.M.Cyr,A.F.Kamal,T.B.Sebastian,andB.B.Kimia.

2d-3dregistrationbasedonshapematching.InMath-ematicalMethodsinBiomedicalImageAnalysis,pages198–203,2000.1[9]E.D’Agostino,F.Maes,D.Vandermeulen,and

P.Suetens.Aviscous uidmodelformultimodalnon-rigidimageregistrationusingmutualinformation.Med-icalImageAnalysis,7:565–575,2003.5[10]U.Grenander,A.Srivastava,andS.Saini.Characteriza-tionofbiologicalgrowthusingiterateddiffeomorphisms.IEEEInt.Symp.BiomedicalImaging:NanotoMacro,pages1136–1139,2006.2,4[11]B.K.Horn.RobotVision.MITPress,1986.4

[12]D.P.Huttenlocher,G.A.Klanderman,paringimagesusingthehausdorffdistance.PAMI,15:850–856,1992.4[13]M.E.Leventon,W.E.L.Grimson,andO.Faugeras.Sta-tisticalshapein uenceingeodesicactivecontours.InCVPR,volume1,pages316–323,2000.7[14]T.Lindeberg.Principlesforautomaticscaleselection.

Technicalreport,RoyalInstituteofTehnology,Stock-holm,ComputationalVisionandActivePerceptionlabo-ratory,1998.4[15]F.Maes,A.Collignton,D.Vandermeulen,G.Marchal,

andP.Suetens.Multimodalityimageregistrationbymaximizationofmutualinformation.IEEETrans.Med-icalImaging,16:187–198,1997.1,4[16]putationalanatomy:shape,growth,and

atrophycomparisonviadiffeomorphisms.NeuroImage,23:S19–S33,2004.1,2,4

[17]M.I.MillerandL.Younes.Groupactions,homeo-morphismsandmatching:ageneralframework.IJCV,41:61–84,2001.4[18]D.MumfordandB.Gidas.Stochasticmodelsforgeneric

municationsinPureandAppliedMathe-matics,54(1):85–111,2001.5[19]S.OsherandJ.Sethian.Frontspropagatingwith

curvature-dependentspeed:putationalPhysics,79:12–49,1988.3[20]N.Paragios,M.Rousson,andV.Ramesh.Matchingdis-tancefuctions:ashape-to-areavariationalapproachforglobal-to-localregistration.InECCV,pages775–789,2002.1[21]J.P.W.Pluim,J.B.A.Maintz,andM.A.Viegever.Im-ageregistrationbymaximizationofcombinedmutualin-formationandgradientinformation.IEEETrans.Medi-calImaging,19:809–814,2000.5[22]J.P.W.Pluim,J.B.A.Maintz,andM.A.Viegever.Mu-tualinformationbasedregistrationofmedicalimages:asurvey.IEEETrans.MedicalImaging,22:986–1004,2003.1[23]W.H.Press,B.P.Flannery,S.A.Teukolsky,andW.T.

Vetterling.NumericalRecipesinC.CambridgeUniver-sityPress,1988.4[24]A.Rangarajan,H.Chui,andJ.S.Duncan.Rigidpoint

featureregistrationusingmutualinformation.MedicalImageAnalysis,3:425–440,1999.1,4[25]M.RoussonandN.Paragios.Shapepriorsforlevelset

representations.InECCV,pages78–92,2002.1[26]L.I.Rudin,S.Osher,andE.Fatemi.Nonlineartotalvari-ationbasednoiseremovalalgorithms.Physica,60:259–268,1992.1,4[27]R.Stefanescu,mowick,G.Malandain,P.-Y.Bon-diau,N.Ayache,andX.Pennec.Non-rigidatlastosub-jectregistrationwithpathologiesforconformalradio-therapy.InMICCAI,pages704–711,2004.1[28]B.C.Vemuri,J.Ye,Y.Chen,andC.M.Leonard.Image

registrationvialevel-setmotion:Applicationstoatlas-basedsegmentation.MedicalImageAnalysis,7:1–20,2003.4[29]P.ViolaandW.M.WellsIII.Alignmentbymaximization

ofmutualinformation.IJCV,24:137–154,1997.4[30]B.ZitovaandJ.Flusser.Imageregistrationmethods:

asurvey.ImageandVisionComputing,21:977–1000,2003.1

本文来源：https://www.bwwdw.com/article/c61j.html