Integrating Dynamic Deformations into Interactive Volume

Non-linear geometric deformation (or warping) is a useful tool for working with volumes. Unfortunately, the computational expense of performing the resampling needed to implement volume deformation has precluded its use in interactive applications. In this

Eurographics/IEEE-VGTCSymposiumonVisualization(2006)ThomasErtl,KenJoy,andBeatrizSantos(Editors)

IntegratingDynamicDeformationsintoInteractiveVolume

Visualization

TomBrunet

K.EvanNowak

MichaelGleicher

DepartmentofComputerSciencesUniversityofWisconsin,Madison

Abstract

Non-lineargeometricdeformation(orwarping)isausefultoolforworkingwithvolumes.Unfortunately,thecom-putationalexpenseofperformingtheresamplingneededtoimplementvolumedeformationhasprecludeditsuseininteractiveapplications.Inthispaper,weshowhownon-lineardeformationscanbeintegratedintointeractivevolumevisualizationallowingfordynamicdeformationstobeusedalongwithinteractiveviewing,exploration,andmanipulationtools.Wedescribehowhardwareassistedvolumerenderingcanbeadaptedtoresamplevolumedeformations,leveragingprogrammableshaderstocomputedeformationsandthelocalcoordinatetransforma-tionsrequiredforshadingeffects.Wedescribehowvolumeinteractiontechniques,suchasraypickingandplaneslicing,canbeusedinconcertwithourdeformationmethods.Ourmethodsextendtosimultaneousdisplayofmultiplevolumesenablingcomparisons.Wedemonstratedynamicvolumedeformationatinteractiveratesoncommodityhardwareforinteractivedeformationcontrol,animateddeformations,andvolumewidgets.CategoriesandSubjectDescriptors(accordingtoACMCCS):I.3[ComputerGraphics]:

1.Introduction

3DScalarField(e.g.Volume)dataisimportantandcom-moninscienceandmedicine.Applyingnon-lineargeomet-rictransformationstovolumedataisavaluablepartofitsuse,allowingforthecorrectionofimagingdeformations,alignmentofdifferentobjects,andmodellingforanimation.Unfortunately,applyingdeformationstovolumesrequiresa3Dresamplingoperationthatiscomputationallyexpensive.Tointeractivelyvisualizeadeformedvolume,thedeforma-tionsandresamplingaretypicallyprecomputed,precludingdynamicdeformations.Applicationswherethedeformationschangeatinteractiverates,suchasinteractivenon-linearreg-istration,real-timevolumeanimation,andanimateddefor-mationinteractiontechniqueshavebeenrestricted.Thispaperprovidesmethodsthatintegratespatialtrans-formationsintoarangeofinteractivevolumevisualizationtools.Theprinciplecontributionistoshowtheeffectivenessofprogrammablegraphicshardwarefortherenderingofde-formedvolumesandforencapsulatingthedeformation.De-formationsarecomputedper-pixelinfragmentshaders.Wewillshowthatplacingthedeformationcomputationinthis

cTheEurographicsAssociation2006.

innermostloopofrenderinghasseveralbene tsandcanbe

doneatinteractiverates.

Thebasicrenderingofdeformedvolumesusingfragmentshadersisstraightforward,modulosomeissueswewillad-dress.Akeyadvantageoftheapproach,however,ishowthedeformationcanbeencapsulatedintheshader,makingiteasytosupportarangeoftoolsthataredesirableininterac-tivevolumevisualizationsuchasarbitraryslicing.Wealsocontributeanoveltechniqueforperformingpickingbyraycastingintothedeformedvolumes.

Previouswork,suchas[FHSR96]and[RSSSG01],hasusedgraphicshardwaretoprovideinteractivedisplayofdeformedvolumes.However,ournewmethodprovidesagreaterrangeofvolumevisualizationtoolsincludingstyl-izedshading,slicingandprobing.Otherpriorwork,suchas[GTB03]hasshowntheusefulnessofdynamicdeforma-tionstoprovideinteractiontechniques,butprovidesalim-itedimplementation.Ourapproachenablestheintegrationofthesemethodswithothervolumetools.

Followingadiscussionofrelevantrelatedwork,wede-scribehowvolumedeformationsarerealizedaspartofthegraphics-hardware-basedvolumevisualizationshadingpro-

Non-linear geometric deformation (or warping) is a useful tool for working with volumes. Unfortunately, the computational expense of performing the resampling needed to implement volume deformation has precluded its use in interactive applications. In this

TomBrunet&K.EvanNowak&MichaelGleicher/IntegratingDynamicDeformationsintoInteractiveVolume

Visualization

Figure1:VisualizationofanMRIofamousetorso.Theoriginalvolume(left)istoneshaded.(right)VolumedeformedusingaHierarchicalB-Spline.Redcirclesdenotesourcepoints,andbluecirclesdenotetargetpoints.cess.Wespeci callyaddresstheissuesofrepresentingthedeformationfunctionsinfragmentprogramsandevaluat-ingthegradientsrequiredforshadingeffects.Section4de-scribeshowslicingandraypickingcanbeprovidedforthedeformedvolumes.Section5discussessomeapplicationsofdynamicdeformations,includinginteractivedeformationandspatialvolumewidgets.Weconcludewithadiscussionofthebene tsandissuesinperformingthedeformationaspartoffragmentshaderprograms.2.RelatedWork

Theimportanceofvolumetricdatasetshasleadtoanen-tire eldofvisualizationtechniquesfortheirdisplay,see[SML98]forasurvey.

Akeyissueinthedisplayofvolumedatasetsistopro-videvisualcuesforcomprehensibility.Directdisplaymeth-odsrelyontransferfunctionsthatdescribehowrayspassingthroughthevolumeareaffectedbythevaluesinthevolume.Thegradientofthescalar eldisoftenusedintransferfunc-tionsasananalogtothesurfacenormalforlightingcom-putations[DCH88].Thisallowsdirectvolumerenderingtoprovideavarietyofsimulatedlightingandstylizeddisplayeffects[ER00].Thevolumedisplaymethodswepresentinthispaperaredesignedtoallowthisrangeofeffects.Displayingvolumesdirectlyatinteractiverateswasorigi-nallyaccomplishedbyspecialpurposehardware,suchasthePixarImageComputerortheVolumeProcard[PHK 99].Currently,mostinteractivedirectvolumerenderingusesstandardgraphicshardwaretocompositeasetoftexturemappedpolygons.ThisideaoriginatedwithCullipandNeu-mann[CN94]andCabraletal.[CCF94],andhasbeenex-tendedovertheyearstomakeuseofnewergraphicshard-warefeatures.See[EHK 04]forasurvey.Ourworkinte-gratesvolumedeformationsintothisubiquitousapproach.Deformationsareoftenusefulinworkingwithvolumedata.Anexampleofrenderingdeformedvolumesthroughprecomputationisshownby[LGL95].Methodsforef cientdisplayofdeformedvolumesinclude[KY97]whichpro-videsapiece-wiselinearapproximationtodeformationwithtesselatedproxygeometry.[FHSR96]and[RSSSG01]stly,[KPH 03]discussestheuseofadeformationcontroltexturetoaddper-turbationdeformationeffectstovolumeshading.3.RenderingDeformedVolumes

Ourapproachrendersdeformedvolumesdirectly.Ratherthan rstdeformingthevolumeandrenderingthedeformedvolume,weintegratethedeformationprocessintothevol-umerenderingprocess.AschematicoverviewoftheprocessisillustratedinFigure2.

Wedenotetheinitial,undeformedvolumeasv(x,y,z).Thescalarfunctionisrepresentedasa3Darrayofsamples,andpointsbetweenaredeterminedthroughtrilinearinter-polation.Thecoordinatesystemofthisdataisundeformedobjectspace,classicallyreferredtoastexturespace.Ade-formationisde nedbyamappingbetweenpointsintheun-deformedobjectspaceandtheirresultingpositionsinthede-formedobjectspace,classicallyreferredtoassimplyobjectspace.Wedenotethismappingasthedeformationfunctiond:R3→R3.Forthepurposesofrendering,wewill ndthattheinversedeformationismorerelevant,sowedenoteu=d 1.Inmostapplications,u,notdisprovided,intherarecaseitisnot,scattereddatainterpolationcanbeusedtoinvertthedeformation.Thedeformedvolumeisthen

v′(x,y,z)=(v u)(x,y,z).

Thebasicideaofourapproachtorenderingdeformedvol-umesisthatratherthanpre-computingv′andapplyingavolumerenderingtotheresult,wemodifythevolumeren-deringprocessreplacingvwithv u.Theapproachsimplyaugmentstheaccessestothearrayoftexturesamplesby rstapplyingtheinversedeformationfunctiontothecoordi-nates.Afterabriefreviewoftheaugmentedvolumerender-ingmethodinSection3.1,wedescribethehurdlesthatwefaceinaddingdeformations.

cTheEurographicsAssociation2006.

Non-linear geometric deformation (or warping) is a useful tool for working with volumes. Unfortunately, the computational expense of performing the resampling needed to implement volume deformation has precluded its use in interactive applications. In this

Function

Tx,Ty,Tz = d-1(Ox, Oy, Oz)

Figure2:Overviewofdeformationrendering.Samplingplanesarespaceduniformlyinviewspace.Polygonsaredrawnineachsamplingplane.Theirtexturecoordinatesprovidepositionsinobjectspacetobesampled.Torenderdeformedvolumes,thedeformationfunctionisusedtomapbetweenobjectspacecoordinatesandvolumetexturecoordinates.3.1.HardwareVolumeRendering

Acommonapproachforhardwarerenderingofvolumesplacesthevolumeina3Dtexture,see[EHK 04]forde-tails.Brie y,theintegraloflightattenuationovereachraythroughthevolumeisapproximatedbysampling.Thesam-plesaregeneratedbyrenderingproxygeometry,andaccu-mulatedintheframebufferwithcompositingoperations.Toprovideforcorrectsamplingthroughtheobjectspace,asetofparallelgeometricelementsareusedfortheproxygeometry.Mostoften,theseareviewalignedplanes.Un-derperspectiveprojection,thespacingofthesamplesisnon-uniform(Figure2).Whiletheseerrorsareoftensmallenoughtobeignored,theycanbeaddressedthroughtheuseofnon-planarproxygeometryorapplyingaper-pixel(e.g.ray)correctionfactor.Ourimplementationdoesthelatter.Volumerenderingisimplementedbyrenderingpolygonsintheviewalignedplanesinbacktofrontorder.Eachpoly-gonisassignedtexturecoordinatessuchthatitsfragmentssampletheappropriatelocationinobjectspace.3.2.DeformedVolumeRendering

ThedeformedvolumerenderingprocessisschematizedinFigure2.First,theundeformedvolumedataisplacedina3Dtexture.Therenderingprocessdrawsaseriesofpolygonsthatarealignedwiththeimageplane.Eachpolygonisas-signedtexturecoordinatesthatareitsdeformedobject-spacecoordinates.Whenthepolygonisrasterized,eachfragment’sexecutionisprovideditsdeformedobject-spacecoordinatesandthevolumetrictextureasinputparameters.Tosamplethedeformation,itmustthereforeconvertthedeformedob-jectspacecoordinatesintoundeformedobjectspace,ortex-turespace,coordinatesbyapplyingtheinversedeformationfunctionbeforeperformingthetexturelookup.

cTheEurographicsAssociation2006.

Renderingadeformedvolumerequiresreplacingalleval-uationsoftheundeformedvolume,v,withthedeformedvol-ume,v′,meaningthatreferencestovarereplacedbyv u.

Fromanimplementationpointofview,foreachfragment,weapplytheinversedeformationfunctiontothedeformedobjectspacecoordinatesandusetheresultingundeformedobjectspacecoordinatestosamplethetexture.Thatis,thefragmentprogramforanundeformedtexturehasthefollow-ingstructure:

Vec3uvw=textCoord;

floatd=texture3D(texD,uvw);Vec4color=transfer(d);

thedeformedrenderingcanbeachievedsimplybyinsertingtheinversedeformationintotheevaluation:

Vec3uvwo=textCoord;

Vec3uvw=invDeform(uvwo);

floatd=texture3D(texD,uvw);Vec4color=transfer(d);

Thelookupintothevolumetextureperformsapointsam-plingwhichmayleadtoaliasing.Thisproblemshouldbeaddressedwhetherornotdeformationisused.Anyvolumetexturesamplingsolution,suchasa3Danalogtoamipmap,appliestothedeformedcaseaswell.Tocorrectly lterthewarp,thekernelradiusmustbedeterminedforthespacingofthesamplingintexturespace,notobjectspace.Ourpresentimplementationimplementspointsampling.

Withthedeformationfunction“encapsulated”insideofthefragmentprogram,otheraspectsofthevolumerender-ingprocessareunchanged.Theundeformedvolumedataisstillstoreddirectlyinthe3Dtexture.Anyproxygeometrycanbeused,althoughitismostsensibletouseview-alignedplanestoavoidartifacts.Proxygeometryisprovidedwithobjectspacelocationsastexturecoordinates,justasbeforedeformationsbecamepartoftherenderingprocess.Thefactthatthisobjectspaceisdeformedobjectspaceishiddenas

Non-linear geometric deformation (or warping) is a useful tool for working with volumes. Unfortunately, the computational expense of performing the resampling needed to implement volume deformation has precluded its use in interactive applications. In this

thetransformationbetweendeformedandundeformedob-jectspaceinthefragmentprogram.

Unfortunately,thereareseveralhurdlesthatwemustad-dressinordertorealizesuchanapproach:

1.Fragmentprogramshavelimitedresourcesmakingsomedeformationfunctionsimpracticaltoimplement.2.Shadingneedsthegradientofthedeformedvolume.3.Giventhelargenumberoffragmentsthatmustberen-dered,theamountofcomputationmightleadtoperfor-manceissues.Thefollowingtwosectionsconsiderhowweaddressthese rsttwohurdles.TheperformanceconsiderationisdeferreduntilSection6.2.

3.3.DeformationFunctions

Onedif cultyinourmethodisthatthedeformationfunc-tionmustbeencodedintothefragmentprogram.While,inprinciple,thefragmentprogramsmaybegeneralpurposecomputations,inpracticetheresourcesavailabletofragmentprogramsaremorelimitedthanthosetotheCPU.Afurtherpracticallimitationisthatsincetheseprogramsareexecutedforeveryfragmentrendered,theymustbeef cient.Thecontinuedevolutionofhardwareandshadinglan-guagesexpandsthesetoffunctionsthatcanbeimplementedeffectivelyasfragmentprograms.However,theremayal-waysbesomefunctionsthataretoocomplexorcomputa-tionallyexpensivetoapplyinthefragmentprograms.Weevaluatesuchfunctionsusingadatacentricrepresentationofstoringatableofsamplesandinterpolating.

Theideaofstoringasampledrepresentationofthefunc-tionuina3Dtexturewassuggestedin[RSSSG01].Priortorendering,theinversedeformationfunctionisevaluatedontheCPUforallpointsonaregular3Dgrid.Thesesam-plesarestoredina3Dtexturethatisaccessedbythefrag-mentprograms.Becausetextureaccessprovidestrilinearin-terpolation,thisapproacheffectivelyconstructsanef cienttoevaluate,piecewise-linearapproximationtothedeforma-tionfunction.Evaluationoftheinversedeformationfunctioninthefragmentprogramsrequiresonlyasingle3Dtexturesamplingoperation,independentofthecomplexityofthefunctionitself.Thisdeformationtextureneednothavethesameresolutionasthevolumedata.

Theuseofasampleddeformationfunctionhasdraw-backs.Forone,itcomputesapiecewiselinearapproximationthatmayfailtocapturedesiredsmoothnessorhighfrequen-ciesunlesslargenumbersofsamplesareused(Figure3).Second,theentiretablemustbeevaluateddensely,whichmaybeexpensive.However,formanycategoriesoffunc-tions(suchaspolynomialsplines),methodsforcomputingregularsamplescanbemoreef cientthancomputinginde-pendentsamples.Third,fragmentprogramsoftenbecometexture-lookup

bound.

(a)163control

texture(b)323control

texture

(c)643control

texture

(d)Fragmentdeformation

Figure3:Imagesofasolidbrickinsideofa2563volumeun-derasinedeformation.Thedeformationisperformedwithcontroltexuresof:a)163,b)323,c)643.d)performsthedeformationinthefragmentshader.

Withourcurrent(circa2005)hardware(anNVIDIAGeForce6800GT)andshadinglanguagetechnology(GLSL),we ndthatsimpledeformationsarebestper-formeddirectlyintheshaders.Forexample,weimplementbendsandtwistsinthismanner.Weusedeformationtex-turestodisplayHierarchicalB-SplineandThin-PlateSplinedeformationsthatarecomputedontheCPU.Incaseswhereonlytheforwarddeformationfunctionisavailable,weusescattereddatainterpolationtoapproximatetheinverse.ThiscomputationisdoneontheCPUandappliedusingadefor-mationtexture.

3.4.GradientComputations

Becausethegradientisdependentonthedeformation,wecannotprecomputethegradientsthatwillbeusedforshad-ing.Wechoosetocomputethegradients’onthe y’inthefragmentshader,usingaforward nitedifferencescomputa-tion:

v(u(O))≈

(v(u(Ox+ Ox,Oy,Oz)) v(u(O)),(1)v(u(Ox,Oy+ Oy,Oz)) v(u(O)),v(u(Ox,Oy,Oz+ Oz)) v(u(O)))

whereOrepresentstheobjectspacecoordinates.Thismethodallowsustosampleagradientinviewspacewithoutcomputingdeformationderivatives.

The rst-order nitedifferencespoorlyestimatesgradi-ents,oftenleadingtovisualartifacts(Figure4).Toimple-mentbetterkernelsef ciently,thevolumetextureispre- ltered.OurimplementationusesaGaussianblurforthepre lter.Boththeoriginaltextureandthe lteredversion

cTheEurographicsAssociation2006.

Non-linear geometric deformation (or warping) is a useful tool for working with volumes. Unfortunately, the computational expense of performing the resampling needed to implement volume deformation has precluded its use in interactive applications. In this

(a) rst nite

differences(b)pre- ltered nitedifferences

Figure4:Aspherewithdiffuseilluminationillustratesis-suesingradientcomputation.(a)forwarddifferencespoorlyestimategradientsyieldingablockyappearance,thisissolvedbypre- lteringthevolumebeforegradientcompu-tation(b).

aresuppliedtothefragmentshader,andthelatterisusedforgradientcomputation.Becauseofitsband-limitation,theresolutionofthepre- lteredvolumecanbereducedtore-ducetexturememoryuse.4.SamplingTechniques

Theencapsulationofthedeformationintothefragmentshaderintheprevioussectionseparatestheprocessofgen-eratingthesamplesfromwarping.Wehavethe exibilitytouseproxygeometrysuitedtothetask.Intheprevioussec-tion,viewalignedplanesgeneratedasamplingappropriatefordirectvolumerendering.Inthissection,weusethefree-dominproxygeometrytoimplementsamplingthatachievesothervolumeinteractionmethods.

Anyproxygeometrycanbeusedtogeneratethefrag-mentsthatsamplethevolume.Thisenablessamplingalongarbitrarylinesandplanes.Toensurethatthesesamplescanbereadfromtheframebuffer,itisimportantthatthelineorplaneisviewaligned.Ourstrategyforsamplinganarbitrarylineorplaneistorotatetheviewsothatitisparalleltotheimageplane,rendertheelement,andthenreadtheresultsfromtheframebuffer.

Wenotethatotherapproachesforinteractivevolumede-formationsthatrelyondeformingthemeshoftheproxyge-ometry(§6.1)wouldrequiremorecomplexapproachestorealizingthealternatesamplingstrategies.

Wediscussmethodsthatexploitthefreedominproxyge-ometrytoperformray-castingandarbitraryslicing,andtodisplaymultipledeformedvolumessimultaneously.4.1.PickingandProbing

Onesimplesamplingtechniqueinvolvestheproxygeome-tryofaline.Theuseofalineasproxygeometryallowsustodoalinearprobe,whichissimilartothatofaray

cast.

cTheEurographicsAssociation2006.Figure5:ArenderingofahumanheadcapturedviaCT

imaging(512x512x106).Thisrenderingdisplaysaclippingplanewithasamplingplanerenderedbothontheclippingplaneandinadetachedviewportontheimageplane.Weobtainlinearprobesbyrotatingobjectspacetoalignthedesiredlineparalleltotheimageplane.Wethenrasterizethisline,directingfragmentshaderoutputandhencevolumesamplingoutputtothebackframebuffer,givingusahigh-resolutionsamplingofthedeformedspace.Wecanthencopythislinetomainmemoryforusebyuser-interfaces.Ourmethodsimpli essuchlinearprobessinceweonlyneedtodrawasinglelinetoobtaintheraycastinformationthroughthedeformedvolume.Withoutencapsulatingthede-formationinsideofthefragmentshader,thislinewouldneedtobepidedintoseverallinestoperformapiece-wiselinearapproximationofthedeformation.

Theabilitytocomputeaprobecreatesanumberofuserinteractionpossibilities.Forexample,auser-interfacecalledraypickingmayneedtodetermineatwhatdepththe rstdeformed“surface”occursunderthemouse.Anumberofdifferentprobescouldbeusedtodeterminethisinforma-tion.Onesuchprobewouldscanthegeneratedlineforthe rstnon-zeroalphavalue.Anothersuchprobemightscanthelineforthe rstgradientwith“large”magnitude.Athirdprobecouldsumalphavaluesuntiltheyexceedone,imply-ingin niteabsorptionofanythingfartherback.4.2.Slicing

Asecondproxygeometrythatisusefulforuserinterfacesisthatofaplane.Thoughplanesareusedasproxygeometryinthefullvolumerendering,theyhaveanother,commonlysoughtafteruse.Slicingplanesareidealforremovingdepthcomplexitywhenexploringvolumes.

Ourimplementationcansamplearbitraryplanesbyview-aligningthemandrenderingtothebackbuffer.Thiscanei-therbedisplayedinaseparateviewport,orappliedasatex-turedpolygoninobjectspace.BothdisplaysareillustratedinFigure5.

Non-linear geometric deformation (or warping) is a useful tool for working with volumes. Unfortunately, the computational expense of performing the resampling needed to implement volume deformation has precluded its use in interactive applications. In this

Figure6:AcruderegistrationofamousetorsocapturedviaMRI(256x256x192),showninblue/grey,toasecondmousecapturedviaCT(256x256x385),showninblack/red.Bothdatasetsandtheirassociatedwarptexturesarepassedtothefragmentshaders,wheretheyareshadedandmixed.

Toillustratetheusefulnessoftheseslicingplanes,con-sideravolumeobtainedthroughCTimagingasshowninFigure5.Aslicingplanealignedalongthedeformedobjectspaceaxiscanallowascientisttocomparethedeformedsliceagainstastandardatlasofthehead.Additionally,arbi-trarilyorientedslicescanbeexaminedandcompared.4.3.MultipleVolumeDisplay

Athirdanduniquebene tofarbitraryproxygeometriesisforthedisplayofmultipledeformedvolumes.Tocorrectlysamplespace,aproxygeometrysamplecannotoverlapotherproxygeometrysamples.Thisimpliesthatinordertocor-rectlyrendertwovolumesinthesamespace,theirsamplesmustbetakenandcombinedfromthesameproxygeometry.Therefore,techniquesthatadaptivelytessellatetheproxyge-ometriesdependentonthedeformationwouldhaveto ndanadaptivetessellationthatsatis esthedeformationsofeachvolume.

Ourmethodallowsustorendermultiplevolumesusingstandardproxygeometries,passingbothtexturevolumestothefragmentshaders.Thefragmentshadersthenhavetheadditionalfreedomtocomputetheiremissionsandabsorp-tionsbasedondifferentcombinationstrategies:sum,differ-ence,emittedcolormixing,etc.AscanbeseeninFigure6,theserenderingscanbevisuallycomplexanddif culttoin-terpret.Makinguseofsimultaneousvolumedisplayisanareaforfuturework.

esofDynamicVolumeDeformation

Therenderingandsamplingtechniquesdescribedinsections3and4providethebuildingblocksneededforanumberofapplications.Wewilloutlinesomefeaturesthatwehaveimplementedthatshowtheversatilityofourapproach.5.1.InteractiveControlofVolumeDeformationsWehaveimplementeddeformationsthatareinteractivelycontrolledbyadjustingtheirparameters.Theprimaryuse

ofthisislandmarkdeformationwhereasetofuserspeci- edpointsarecontrolled.Thismaybeusedforperforminginteractiveregistration[FRR96]

Landmarkdeformationfunctionsuseasetofpointpairsasparameters,whereonepointineachpair,thesource,isinundeformedobjectspace,andthesecondpoint,thetarget,isindeformedobjectspace.Thegoaloftheselandmarkbaseddeformationfunctionsisto ndamappingthateitherexactlyorapproximatelymapsbetweenallsourcesandtheircor-respondingtargets.TwoexamplesofdeformationsthatcanbeusedaslandmarkdeformationfunctionsareThinPlateSplinesandHierarchicalB-Splines.

Toaddmeaningfulpointpairs,weneedtobeabletoeas-ilyspecifysigni cantsurfacefeaturepoints.Theraypicker(§4.1)ingaraypicker,wecanallowtheusertoholddownthemouseandmoveapointalongthesurfaceofthevolume,evenifithasbeendeformed.Thisallowstheusertospecifyacoordinatein2D,andallowsthesystemtoinferthedepthalongtheraythattheuserwantstoselectandwith nerresolutionthanisactuallyusedtorenderthevolume.

Onceplaced,controlpointscaneitherbemanipulatedbyusingtheraypickertodragthepointsalongaspeci edsur-face,ordraggedinviewalignedplanes.Sinceavastmajorityoftheworkfordisplayingthedeformedvolumeisof oadedtotheGPU,theCPUcanbeutilizedmoreforsolvingthede-formation.Therefore,theusercanreceiveimmediatefeed-backofhowhisorheractionsareaffectingthedeformationbyobservingthechangesinvolumedeformation.

Raypickingcanalsobeofusetoauserinterestedininter-activeregistration.Whileviewingmultiplevolumes,sourcepointscanbeplacedalongthesurfaceofonevolume,andtargetpointscanbeplacedalongthesurfaceofasecondvol-ume.Thisispossiblesincetheraypickercanchangeshadersandpickwhichvolumeisbeinginteractedwith.

5.2.VolumeAnimations

Theabilitytorenderanddeformvolumesatinteractiveratesnaturallyleadstodeformationbasedanimations.Ourexplo-rationofthisareahasbeenminimal,however,wehaveim-plementedapulsing, sh-eyelikedeformationasaproofofconcept.Thisparticularanimationcausesalocaldeforma-tionwithinasphereofin uencearoundthepointofinterest.Sincethehumaneyeisattractedtomovement,weenvisionsuchadeformationasauser-interfacetoolthatisusefulfordrawingattentiontoaregionofinterest.

Thisparticularanimationisattractivebecausethedefor-mationfunctioncanbecomputedcompletelyintheshader.Theapplicationsimplyhastopassafew oatingpointpa-rametersde ningthedeformationforthatparticularframe.Therefore,thecostofanimationoverrenderingisnegligible.

cTheEurographicsAssociation2006.

Non-linear geometric deformation (or warping) is a useful tool for working with volumes. Unfortunately, the computational expense of performing the resampling needed to implement volume deformation has precluded its use in interactive applications. In this

Figure7:Aleaferwidgetfrom[GTB03]implementedasa

deformation.5.3.VolumeWidgets

Wehaveimplementedseveralvolumewidgetsasdiscussedin[GTB03],includingtheoneshowninFigure7.Weim-plementthesedeformationdirectlyintheshader.Thedis-continuitiesinthedeformationfunctionsthatrepresentthesewidgetsrequireconditionalbranchesthatareinef cientonsomecurrenthardware.6.Discussion

Inthissection,wewilladdressourthirdhurdlefordefor-mationrendering:performance.First,wewillcompareourmethodwithotherhardwareacceleratedmethods,parisonwithPer-VertexMethods

Ourapproachprovidesinteractivedisplayofdeformedvol-umesbyperformingaper-fragmentevaluationofthede-formationfunction.Thealternativeistoapplythedefor-mationtotheproxygeometryonaper-vertexbasis.Ex-amplesofsuchanapproachinclude[FHSR96],[KY97],and[RSSSG01].Here,wecompareourper-fragmentap-proachwiththeseper-vertexapproaches.

Per-vertexapproachesrelyonatesselationoftheproxygeometrytoprovidethesetofverticestodeform.Iftheproxygeometryistobeviewdependent,itmustbere-tesselatedwhenevertheviewchanges.Thismakesmethodsthatrenderfrommultipleviewpoints(§4)dif cult,particu-larlyifweareconcernedwithusingthesamesamplingsothatmultipleviewscanbeusedseamlessly(asinFigure5).Italsocomplicatestheuseofview-alignedproxygeometry.Suchgeometryisadvantageousasituniformlysamplestheobjectspace,leadingtomoreconsistenttransparentshadingeffects.

Per-vertexmethodsrelyonsubpisiontoreducethenum-berofdeformationevaluationsthatneedtobeperformed.Sinceweareinterestedindynamicdeformations,thismeansthatchangesinthedeformationwouldrequireanewsub-pision,anewtessellation,andnewdeformationevalua-tions.Wecaneliminatethesubpisionconsiderationifwe

cTheEurographicsAssociation2006.ShaderSine29fps7.1fpsTexture21.7fps5.7fps

Table1:Performancesummary,discussedinsection6.2.Case1representsasimpletransferfunctionandCase2rep-resentsagradientcomputationanddiffuseshading.

ControlTextureSizeSamp/Vox

163264FSTable2:Frameratesinfpsofcontroltexturesizevs.sam-pleplanespervoxel,usingthemousetorsoofFigure1,a256x256x192volume.

assumea xed,uniformsubpisionisused,whichissensi-bleforevaluationarapidlychanging,unknowndeformation.Wecanalsoeliminatethetesselationconsiderationifweuseobject-orientedgeometry.Undertheseconditions,ourmeth-odsarethemostsimilar.

Ourmethods,therefore,haveadvantagesinprovidingfor exibleproxygeometrythatenablesinteractiontechniquesandavoidsrecomputationwhendeformationschange.Intermsofperformance,ourmethodoffersadifferentsetoftradeoffsaswemovemoreofthecomputationfromtheCPUtothefragmentshaders.Becausefragmentshadersexecuteinparallel,theyaremorelikelytoprovideperformancein-creasesinfuturegenerations.6.2.PerformanceandAccuracy

Weevaluatedtheperformanceofourprototypeimplemen-tationonaPCwitha3GhzIntelPentium4ProcessorandanNVIDIAGeForce6800GTgraphicscard.Whileourim-plementationadjustssamplingratestoprovidefasterperfor-manceattheexpenseofimagequality,we xthesamplingrateatthesizeofthesmallestobject-spacevoxelfortheseperformancemeasurements,exceptwherenotedotherwise.Allmeasurementsarefortheachievedsystemframeratein-cludinganycomputationofthedeformations.

Therenderingratesonarealisticexample(the512x512x106CTHead,Figure5)aresummarizedinTa-ble1.Weevaluatetwoshadingtypesandthreedeformations.Forrenderingcase1,adensityvalueisconvertedtoacolorandalphavaluebasedonatransferfunctiontexturelookup.Forrenderingcase2,weaddouron-the- ygradientcom-putationandadotproductisperformedfordiffuseshading.Thedeformationsweconsiderareatrivialtranslationandasinewave,bothimplementedintheshader,andaHierarchi-calB-Splinethatisevaluatedandstoredina163texture.Thetrivialtranslationdefomationisinterestingbecause

Non-linear geometric deformation (or warping) is a useful tool for working with volumes. Unfortunately, the computational expense of performing the resampling needed to implement volume deformation has precluded its use in interactive applications. In this

Samp/Vox

81632Table3:Frameratesinfpsofcontroltexturesizevs.sampleplanespervoxel,whilechangingtheHBSplinedeformationofthe256x256x192mousetorsoofFigure1.

althoughitadds(almost)nocomputationtotheshadingpro-cess,itdoeseffectperformance.Directlyfeedingthetexturecoordinatetothetexturelookupachieves39fpsincase1.Performingatrivialcomputationonthecoordinate rstre-ducestheperformanceto29.2fps.Thissuggeststhattheun-derlyinggraphicssystemprovidessomeoptimizationfordi-recttexturelookups.Sinceweareunsureifthisoptimizationcouldbeleveragedfordeformations,wereporttheshiftedcaseinthetable.

Thedropinframeratebetweencase1andcase2,whereweaccessanadditionaltexturefourtimes,suggeststhatourperformanceisboundbytextureaccess.Thesetexturelookupsmayhavepoormemorycoherence.Slowdownsinadditiontothisincreasedshadercomplexitycomefromwrit-ingtothedeformationtexture,andfrommemoryband-widthtothegraphicscardtoupdatethedeformationtex-tures.Overall,we ndthattheapplicationsdescribedinsec-tion5areinteractiveifweusethereducednumberofsam-plingplanesduringinteraction.

ToshowtheperformanceimpactofcontroltexturesizeweusethedatasetseeninFigure1.Performanceforasim-pledeformationisshowninTable2.Asexpected,largercon-troltexturesslowrenderingslightly,andthefragmentshaderdeformation,usingnodeformationtexturelookup,isnearlytwiceasfast.

Whentheevaluationsofthedeformationfunctionbecomemoreexpensive,bettersamplingofthedeformationfunc-tionshavemoreofaperformanceimpact.Table3,weshowtheframeratewhilechangingacontrolpoint,solvingforthenewHBSplinedeformation,storingtheevaluationsinthecontroltexture,andrenderingtheimage.

Performanceofourprototypeshowsthatinteractiveview-ingofvolumeswithdynamicdeformationsispracticaloncurrenthardware.Ourmethodsarewell-posedtoleveragetrendsingraphicshardware.Acknowledgments

ThisresearchwassupportedinpartbyNSFgrantsIIS-0416284andCCF-0540653.TBwassupportedbyanNLMCIBMtraininggrant(NLM5T15LM007359).WethankJamieWeichert’slabforprovid-inguswiththemousevolumesseeninFigure1and6.TheCTHeadinFigure5wasobtainedfromOpenQVis.

References

[CCF94]CABRALB.,CAMN.,FORANJ.:Acceleratedvolume

renderingandtomographicreconstructionusingtexturemappinghardware.InProceedingsofthe1994symposiumonVolumevi-sualization,pp.91–98.[CN94]CULLIPT.J.,NEUMANNU.:AcceleratingVolumeRe-constructionWith3DTextureHardware.Tech.rep.,ChapelHill,NC,USA,1994.[DCH88]DREBINR.A.,CARPENTERL.,HANRAHANP.:Vol-umerendering.InProceedingsofSIGGRAPH1988,pp.65–74.

[EHK 04]ENGELK.,HADWIGERM.,KNISSJ.M.,LEFOHNA.E.,REZK-SALAMAC.,WEISKOPFD.:Real-timevolumegraphics.SIGGRAPHCourseNotes,Course28,2004.[ER00]EBERTD.,RHEINGANSP.:Volumeillustration:non-photorealisticrenderingofvolumemodels.InProceedingsoftheconferenceonVisualization2000,pp.195–202.[FHSR96]FANGS.,HUANGS.,SRINIVASANR.,RAGHAVANR.:Deformablevolumerenderingby3Dtexturemappingandoctreeencoding.InProceedingsofthe7thconferenceonVisual-ization1996,pp.73–ff.[FRR96]FANGS.,RAGHAVANR.,RICHTSMEIERJ.:Volumemorphingmethodsforlandmarkbased3dimagedeformation.InSPIEInternationalSymposiumonMedicalImaging(1996).[GTB03]GUFFINM.,TANCAUL.,BALAKRISCHNANR.:Usingdeformationsforbrowsingvolumetricdata.InProceedingsofIEEEVisualization2003,pp.401–408.[KPH 03]KNISSJ.,PREMOZES.,HANSENC.,SHIRLEYP.,MCPHERSONA.:Amodelforvolumelightingandmodeling.InIEEETransactionsonVisualizationandComputerGraphics2003,pp.150–162.[KY97]KURZIONY.,YAGELR.:Interactivespacedeformationwithhardware-assistedrendering.IEEEComputerGraphicsAp-plications17,5(1997),pp.66–77.[LGL95]LERIOSA.,GARFINKLEC.D.,LEVOYM.:Feature-basedvolumemetamorphosis.InProceedingsofSIGGRAPH1995,pp.449–456.[PHK 99]PFISTERH.,HARDENBERGHJ.,KNITTELJ.,LAUERH.,SEILERL.:Thevolumeproreal-timeray-castingsystem.InProceedingsofSIGGRAPH1999,pp.251–260.[RSSSG01]REZK-SALAMAC.,SCHEUERINGM.,SOZAG.,GREINERG.:Fastvolumetricdeformationongeneralpurposehardware.InProceedingsoftheACMSIG-GRAPH/EUROGRAPHICSworkshoponGraphicshardware,pp.17–24.[SML98]SCHROEDERW.,MARTINK.M.,LORENSENW.E.:Thevisualizationtoolkit(2nded.):anobject-orientedapproachto3Dgraphics.Prentice-Hall,Inc.,UpperSaddleRiver,NJ,USA,1998.

cTheEurographicsAssociation2006.

本文来源：https://www.bwwdw.com/article/0yxq.html