Automatic reconstruction of colored 3d models

Abstract. A basic issue of mobile robotics is the automatic generation of environment maps. This paper presents novel results for the reconstruction of textured 3D maps with an autonomous mobile robot, a 3D laser range finder and two pan-tilt color cameras

AutomaticReconstructionofColored3DModels

KaiPerv¨olz,AndreasN¨uchter,HartmutSurmann,andJoachimHertzberg

FraunhoferInstituteforAutonomousIntelligentSystems(AIS)

SchlossBirlinghoven

D-53754SanktAugustin,Germany

{pervoelz,nuechter,surmann,hertzberg}@ais.fraunhofer.de

Abstract.Abasicissueofmobileroboticsistheautomaticgenerationofenvironmentmaps.Thispaperpresentsnovelresultsforthereconstructionoftextured3Dmapswithanautonomousmobilerobot,a3Dlaserrange nderandtwopan-tiltcolorcameras.Building3Dmapsinvolvesanumberoffundamentalscienti cissues.Thispaperadressestheissueofhowtofusethegeometrydataofthe3Dlaserrange nderwithcameraimages.Theproposedalgorithmallowstotexturizegeometrical3Dscenes-models.

1Introduction

Onefundamentalprobleminthedesignofautonomousmobilecognitivesystemsistheper-ceptionoftheenvironment.Abasicissueofmobileroboticsisautomaticmapbuildingofenvironments.Digital3Dmodelsoftheenvironmentareneededinrescueandinspectionrobotics,facilitymanagementandarchitecture.Autonomousmobilerobotsequippedwith3Dlaserscannersarewellsuitedforthegagingtask[14].Tocreaterealisticvirtualrealitiesfromgeometricmodels,textures,i.e.,photosoftheenvironments,havetobeacquiredandmustbepreciselymappedontothescene.Thismappinghastobecomputedautomaticallyfromthe3Dpointcloudofthescannedsceneandtheacquiredphotographs.

Tocomputethecorrecttextureforascannedscene,fourstepsarenecessary:First,thecamerasarecalibrated;second,ameshingmethodgeneratesatrianglemeshofthe3Ddata,andthird,thetextureforeverytriangleischosenandmapped.DrawingtothescreenisdonebyOpenGL.Finallyandfourth,globalcolorcorrectionsaremadetoremovethesystematiccolorandilluminationdi erencesbetweentheindividualtexturemaps.Afterdiscussingthestateoftheartin3DreconstructionandpresentingtherobotKurt3Dthesefourstepsaredescribedindetail.

2StateoftheArt

Somegroupshaveattemptedtobuild3Dvolumetricrepresentationsofenvironmentswith2Dlaserrange nders.Thrunetal.[15],Fr¨uhetal.[6]andZhaoetal.[17]usetwo2Dlaserrange nderforacquiring3Ddata.Onelaserscannerismountedhorizontallyandoneismountedvertically.Thelatteronegrabsaverticalscanlinewhichistransformedinto3Dpointsusingthecurrentrobotpose.Sincetheverticalscannerisnotabletoscansidesofobjects,Zhaoetal.[17]usetwoadditionalverticalmounted2Dscannershiftedby45 toreduceocclusion.Thehorizontalscannerisusedtocomputetherobotpose.Theprecisionof3Ddatapointsdependsonthatposeandontheprecisionofthescanner.Alltheseapproacheshavedi culties

Abstract. A basic issue of mobile robotics is the automatic generation of environment maps. This paper presents novel results for the reconstruction of textured 3D maps with an autonomous mobile robot, a 3D laser range finder and two pan-tilt color cameras

tonavigatearound3Dobstacleswithjuttingoutedges.Theyareonlydetectedwhilepassingthem.

Afewothergroupsuse3Dlaserscanners[12,2].A3Dlaserscannergeneratesconsistent3Ddatapointswithinasingle3Dscan.TheRESOLVprojectaimedatmodelinginteriorsforvirtualrealityandtelepresence[12].TheyusedaRIEGLlaserrange nderontworobots,calledESTandAEST(AutonomousEnvironmentalSensorforTelepresence).TheyusetheIterativeClosestPoints(ICP)algorithm[3]forscanmatchingandaperceptionplanningmoduleforminimizingocclusions.TheAVENUEprojectdevelopsarobotformodelingurbanenvironments[2]usingaCYRAXlaserscanner.Theymatch3Dscanswithcameraimagessemiautomaticallytoyieldatexturedmodel.

Othertechniquesforacquiringrangedataarestereovisionandphotogrammetry.Stereovisionhasdi cultieswithproducingdensedepthmapsanddependsontheilluminationtosomedegree.Photogrammetricmethodsproducehighqualitymodelsthataretextured.Nevertheless,thesemethodsareusuallymanualandcomputationallyexpensive,thuscannotbecomputedinrealtimeoronamobilesystem,i.e.,onarobot.Tab.1compareslaserscanningwithphotogrammetry.Stateoftheartphotogrammetricmethods[4]arecombinedwithlaserscanningtoyieldameasuringmethodologythatismore esthiscombinationtoextracttextureandtore nethemodelbasedon3Dlaserscanningbyimages[4].Itispossibletocompletethemodelsinareaswheredataismissingortoincreasetheresolutioninareasofhighinterestand3Dcontents.

Resolution

large

sensors

Photo-

grammetryhighreso-lutionphotosext.lightindependentextract3Dfromphotosbycorrespondences3DmeasurmenthightextureisrequiredReliabilityTable1:Comparisonoflaserscanningwithphotogrammetricmodelacquisitionmethods.Advantagesareprintedinreditalic.

3

3.1TheAutonomousMobileRobotKurt3DTheRobotPlatform

Kurt3D(Fig.1,topleft)isamobilerobotplatformwithasizeof45cm(length)×33cm(width)×26cm(height)andaweightof15.6kg.Equippedwiththe3Dlaserrange ndertheheightincreasesto47cmandtheweightincreasesto22.6kg.1Kurt3D’smaximumvelocityis5.2m/s(autonomouslycontrolled4.0m/s).Two90Wmotorsareusedtopowerthe6wheels,whereasthefrontandrearwheelshavenotreadpatterntoenhancerotating.Kurt3Doperatesforabout4hourswithonebattery(28NiMHcells,capacity:4500mAh)charge.ThecoreoftherobotisaPentium-III-600MHzwith384MBRAM.Anembedded16-BitCMOSmicrocontrollerisusedtocontrolthemotor.

3.2TheAIS3DLaserRangeFinder

TheAIS3Dlaserrange nder(Fig.1,middle)[14,13]isbuiltonthebasisofa2Drange nderbyextensionwithamountandasmallservodrive.The2Dlaserrange nderisattachedinthecenterofrotationtothemountforachievingacontrolledpitchmotionandreducing

Abstract. A basic issue of mobile robotics is the automatic generation of environment maps. This paper presents novel results for the reconstruction of textured 3D maps with an autonomous mobile robot, a 3D laser range finder and two pan-tilt color cameras

Figure1:Left:TheautonomousmobilerobotKurt3DequippedwiththeAIS3Dlaserrange nder.Middle:TheAIS3Dlaserrange nder.ItstechnicalbasisisaSICK2Dlaserrange nder(LMS-200).Right:Scannedsceneaspointcloud(viewingpose1meterbehindscannerpose).

torsionalmoments.Ontheleftside,thehighgradeservoisconnected.Onebatterycharge(Scanner:17W,20NiMHcellswithacapacityof4500mAh,Servo:0.85W,4.5Vwithbatteriesof4500mAh)issu cientfor5hoperatingtime.

Theareaof180 (h)×120 (v)isscannedwithdi erenthorizontal(181,361,721)andvertical(210,420)resolutions.Aplanewith181datapointsisscannedin13msbythe2Dlaserrange nder(rotatingmirrordevice).Planeswithmoredatapoints,e.g.,361,721,duplicateorquadruplicatethistime.Thusascanwith181×210datapointsneeds2.8seconds.Fig.1(topright)showsanexampleofapointcloudwithaviewingposeonemeterbehindthescannerpose.

3.3TheCameraSystem

Thecamerasystem(Fig.2,left)consistsoftwoTerraCAMUSBProwebcams.Theyareequippedwithamanualfocuslensandtheresolutionislimitedto640×480pixelswith7fpsasthemaximumframerate.Tocoverthewholearea,scannedbythelaserrange nder,eachcameraneedstotake6di erentimages(Fig.2,right).Tohandlethis,thewebcamsaremountedonpan-tiltunitseachofwhichisbasedon2servodrives(VolzMicro-Maxx),oneforthehorizontalaxisandtheotherfortheverticalaxis.Eachaxiscanberotatedby±45 .Duetothehigh-gradeservodrives,anexcellentrepeataccuracyinpositioningis

guaranteed.Figure2:Left:Thepan-tiltcamerasystem.Right:Scannedsceneaspointcloud(viewingpose2meterbehindscannerpose).Thesceneiscoveredby12cameraimageswithsomeoverlappingareas.

Abstract. A basic issue of mobile robotics is the automatic generation of environment maps. This paper presents novel results for the reconstruction of textured 3D maps with an autonomous mobile robot, a 3D laser range finder and two pan-tilt color cameras

ThewebcamsarepoweredovertheUSBinterfaceandtheservodrivesarefedbythesamebatteriesasthe3Dlaserrange nderservo(cf.section3.2).

4CameraCalibration

Thecameraismodeledbytheusualpinholeapproach.Acameraprojectsa3Dpointp∈3tothe2Dimage,resultinginp ∈2.Therelationshipbetweena3Dpointpanditsimageprojectionp isgivenby

αγupRtps=AwithA= 0βv .100011001

Aisthecameramatrix,i.e.,internalcameraparameters,withtheprincipalpointwiththecoordinates(u,v).Randtspecifytheexternalcameraparameters,i.e.,theorthonormal3×3rotationmatrixandtranslationvectorofthecameraintheworldcoordinatesystem.Inadditiontotheseequations,weconsiderthedistortion,resultingin4additionalparameterstoestimate[16].

Cameracalibrationusesanewtechniquebasedon

Zhang’s

method.Wegiveabriefsketchhere,detailscanbefoundin

[16].ThekeyideabehindZhang’sapproachistoestimate

theintrinsic,extrinsicanddistortioncameraparametersby

asetofcorrespondingpoint.These3D-to-2Dpointcorre-

spondencesare rstusedtoderiveananalyticalsolution,i.e.,

thegeneral4×4homographymatrixHisestimated:

pp=Hs.11

Theestimationisdonebysolvinganoverspeci edsystem

oflinearequations.Sincethepointsof3D-to-2Dpointcor-

respondenceshaveusuallysmallerrorsandonlyafewpoints

areusedtosolvetheequationsforH,this rstestimation

needstobeoptimized.Anonlinearoptimizationtechnique,

i.e,theLevenberg-Marquardtalgorithmbasedonthemaxi-

mumlikelihoodcriterionisusedtooptimizetheerrorterm:

mn pi,j p i,j(Hj) .

i=1j=1

Herebyp i,jarei,jarethepointspi,jprojectedbyHj,andp

thegivencorrespondingpoints;iisthepointindexandj

istheimageindex.AfterthecalculationofH,thecamera

matrixA,therotationmatrixRandthetranslationtare

calculatedfromH.Again,anoverspeci edsystemoflinear

equationissolved,followedbyanonlinearoptimizationof

n m p p . (A,R,t)jji,ji,j

i=1j=1Figure3:Top:Chessboardplaneforcalibration.Bottom:Chessboarddetectionina3Dlaserrangescan.

Abstract. A basic issue of mobile robotics is the automatic generation of environment maps. This paper presents novel results for the reconstruction of textured 3D maps with an autonomous mobile robot, a 3D laser range finder and two pan-tilt color cameras

Finallythetermforoptimizationissetto

n m pi,j p i,j(A,Rj,tj,k1,k2,l1,l2) ,

i=1j=1

wherek1,k2areparametersfortheradialdistortion,andl1,l2theonesfortangentialdistor-tion.TheminimumisfoundbytheLevenberg-MarquardtalgorithmthatcombinesgradientdescentandGauss-Newtonapproachesforfunctionminimization[5,11].Animportantcon-ceptoftheLevenberg-Marquardtalgorithmisthevectorofresiduals,i.e.,e(a)={Ei(a)}i=1N,sothatE(a)=||e(a)||isoneoftheerrortermsabove.Thegoalateachiterationistochooseanupdatextothecurrentestimateac,suchthatsettingac+1=ac+xreducestheerrorE(a).ATaylorapproximationofE(a+x)resultsin

E(a+x)=E(a)+( E(a)·x)+1

.Thetaskateachiterationistodeterminea

stepxthatwillminimizeE(a+x).UsingtheapproximationofEdi erentiatingwithrespecttoxequatingwithzero,yields aj

xE(x+a)=JTe+JTJx=0.

SolvingthisequationforxyieldsthenewGauss-Newtonupdatex=(JTJ) 1JTe.Incontrast,theupdatewithanacceleratedgradientdescentisgivenbyx=λ 1JTewithλdenotingtheincrementbetweentwogradientdescentsteps.Ineveryiterationthisnewupdateiscalculatedbyacombination,i.e.,by

x=(JTJ+λ) 1JTe.

Fortheabovecameracalibrationalgorithmthe3D-to-2Dpointcorrespondencesareessential.Calibrationisdonewithachessboard.Fromtheimagetheboardpatterncornersareex-tractedautomatically(Fig.3top).Thecorresponding3Dpointsareautomaticallyextractedbasedonthecornersofaquadin3D(Fig.3bottom).Thecalibrationalgorithmextractsthe3Dquadfromthescannedpointcloudwithamodi edICP(IterativeClosestPoints)algo-rithm[3,9].Givenasetof3DscanpointsM,aquadismatched.ThealgorithmcomputestherotationRandthetranslationt,suchthatthedistancesthescanpointsmi∈3andtheirprojectiontothequaddi∈3isminimized,i.e.,

Nm i=1||mi (Rdi+t)||2.

TheICPalgorithmaccomplishestheminimizationiteratively.Itcomputes rsttheprojectionsandthenminimizestheaboveerrorterminaclosedformfashion[3,9].TheclosedformsolutionisbasedontherepresentationofarotationasaquaternionasproposedbyHorn[7].

Abstract. A basic issue of mobile robotics is the automatic generation of environment maps. This paper presents novel results for the reconstruction of textured 3D maps with an autonomous mobile robot, a 3D laser range finder and two pan-tilt color cameras

Figure4:Left:Schematicillustrationofthetexturemappingprocess.Middleandright:Final3Dtexturemappingfromtwodi erentviewposesofthescenegiveninFig.2,right.

5MeshGeneration

The3Dscannergagestheenvironmentbyatiltrotation,thusthescanslicesareordered.Furthermore,ineveryscanslicethedataisorderedcounterclockwise.So,asimplealgorithmcreatesatrianglemeshbyconnectingneighboringdatapoints.Athresholdforthesidelengthofatriangleisusedtohandlejumpegdes,i.e.,rgermeshesarecreatedbyconnectingthe(i,j)thneighbor,respectively.

6TextureMapping

Inordertoassigntexturetothepreviouslygeneratedtrianglemesh,thealgorithmprojectstheverticesofeachtriangletotheimagesbyutilizingtheextrinsic,intrinsicanddistortionparameters,computedduringthecameracalibrationprocess(cf.section4).DuetothelensqualityofanordinaryUSBwebcam,modelingthelensdistortionisessential.

To gureoutwhichofthe12imagesholdsthecorrecttextureinformation,theverticesareprojectedtoallimages.Ifoneimagedoesnotcovertheaccordingpartofthe3Dscenethecomputedvertexcoordinateswillbeoutoftheimagerange.Ifthealgorithmgivesvalidcoordinatesformorethanoneoftheimages(e.g.,fortrianglesinoverlappingareas,compareFig.2),itusestheimageinwhichtheprojectedvertexcoordinatesareclosesttotheimagecenter.

Basedontheseprojectedcoordinates,anOpenGL-basedviewerapplicationcutsthetextureoutoftheimagesand”glues”itontothe3Dtrianglemesh.Duetothisrelationof3Ddataandtextureinformation,thescenecanberenderedfromdi erentperspectives(Fig.4,middleandright)asatextured3Dscene.2

7GlobalColorCorrection

Di erentilluminationconditionsandthecameratechnologypreventcolorcontinuityatthebordersofeachimage,leadingtoobservablediscontinuitiesinthecolorandalsobrightness.BasedontheideasofAgathosandFisherweuseglobalcorrectionsinordertodi usethetexturefromeachtwodi erentviews[1]andtoreducetheobservablediscontinuities.TheymotivatetheassumptionthatthereexistsalineartransformationmatrixTj→ktocorrectthejthviewtothekthview,i.e.,

(j)(k)Tj→kNi(λ)=Ni(λ),

video[10].

Abstract. A basic issue of mobile robotics is the automatic generation of environment maps. This paper presents novel results for the reconstruction of textured 3D maps with an autonomous mobile robot, a 3D laser range finder and two pan-tilt color cameras

Global Texture Correction

without correction

corrected image

Figure5:Left:Magni cationofthetextured3DsceneinFig.4,right.Right:UncorrectedandcorrectedphotoNo.B3.ThecorrectionisbasedonthepixelssharedwithphotoNo.A3.Thephotonumbersarede nedinFig.2.

fori∈{R,G,B}.TwovectorsofpixelsVk,Vjareformedfromtheviewskandjrespectively.TheycontainR,G,Bpixelsfromtheoverlapoftheviews.TheglobalcorrectionmatrixTj→kisestimatedasfollows[1]:

Vk=Tj→kVk TTj→k=(VkVj)(VjVj) 1

Fig.5showsanimagepartofthesceneofFig.4withanuncorrectedandcorrectedimage.TheresultispresentedinFig.4andshowsstillsomecolorincontinuitiesthatcannotberesolvedwiththecorrection.Themethodrequiresenoughimageoverlap,precise3D-to-2Dcalibrationandsu cientinputimagequality.

8Conclusions

Thispaperhaspresentedaframeworkforthereconstructionoftextured3Dmapswithanautonomousmobilerobot,a3Dlaserrange nderandtwopan-tiltcolorcameras.Thedevelopedsystemsallowstogageenvironmentsin3Dandfusethedatawithcameraimages.Awiderangeofapplicationsusing3Dmodelsbene tfromtheproposedautomaticacquisitionmethod,e.g.,virtualrealityapplications,architecture,factoryandfacilitymanagementandrescueandinspectionrobotics.

Needlesstosay,muchworkremainstobedone.Futureworkwillconcentrateonfouraspects:

Build3Dmapswithtextureinformationinvolving6Drobotposes,includinglooptours,i.e.,6DSLAM[9].

Uselocalcolor/texturecorrectioninadditiontoourglobalcorrectionmethod.

Calibratethecameraparametersdynamicallywithoutthechessboardquad,i.e.,calcu-lateitfromthe3Dsceneandthecorrespondingimages.

Generatehighleveldescriptionsandsemanticmapsofenvironmentsincludingthe3Dinformation,e.g.,inXMLformat.Thesemanticmapscontainspatial3Ddatawithdescriptionsandlabels[8].

Acknowledgment:

ourwork.SpecialthankstoKaiLingemannandMatthiasHennigforsupporting

Abstract. A basic issue of mobile robotics is the automatic generation of environment maps. This paper presents novel results for the reconstruction of textured 3D maps with an autonomous mobile robot, a 3D laser range finder and two pan-tilt color cameras

References

[1]A.AgathosandR.B.Fisher.ColourTextureFusionofMultipleRangeImages.InProceedingsof

the4thIEEEInternationalConferenceonRecentAdvancesin3DDigitalImagingandModeling(3DIM’03),Ban ,Canada,October2003.

[2]P.Allen,I.Stamos,A.Gueorguiev,E.Gold,andP.Blaer.AVENUE:AutomatedSiteModelingin

UrbanEnvironments.InProceedingsofthethirdInternationalConferenceon3DDigitalImagingandModeling(3DIM’01),QuebecCity,Canada,May2001.

[3]P.BeslandN.McKay.AmethodforRegistrationof3–DShapes.IEEETransactionsonPattern

AnalysisandMachineIntelligence,14(2):239–256,February1992.

[4]P.Dias,V.Sequeira,F.Vaz,andJ.G.M.Goncalves.RegistrationandFusionofIntensityand

RangeDatafor3DModellingofRealWorldScenes.InProc.ofthe4thIEEEInt.Conf.onRecentAdvancesin3DDigitalImagingandModeling(3DIM’03),Ban ,Canada,October2003.

[5]A.W.Fitzgibbon.RobustRegistrationof2Dand3DPointSets.InProceedingsofthe11thBritish

MachineVisionConference(BMVC’01),2001.

[6]C.Fr¨uhandA.Zakhor.3DModelGenerationforCitiesUsingAerialPhotographsandGround

LevelLaserScans.InProceedingsoftheComputerVisionandPatternRecognitionConference(CVPR’01),Kauai,Hawaii,USA,December2001.

[7]B.Horn.Closed–formsolutionofabsoluteorientationusingunitquaternions.Journalofthe

OpticalSocietyofAmericaA,4(4):629–642,April1987.

[8]A.N¨uchter,H.Surmann,andJ.Hertzberg.AutomaticClassi cationofObjectsin3DLaserRange

Scans.InProc.8thConf.onIntelligentAutonomousSystems,pages963–970,Amsterdam,TheNetherlands,March2004.

[9]A.N¨uchter,H.Surmann,K.Lingemann,andJ.Hertzberg.6DSLAM-PreliminaryReporton

closingtheloopinSixDimensions.InProceedingsofthe5thSymposiumonIntelligentAutonomousVehicles(IAV’04),Lissabon,Portugal,June2004.

[10]A.N¨uchter,H.Surmann,K.Lingemann,K.Perv¨olz,andJ.Hertzberg.Video:Autonomous

MobileRobotsfor3DDigitalizationofEnvironments.InProceedingsofthe4thIEEEInternationalConferenceonRecentAdvancesin3DDigitalImagingandModeling(3DIM’03),October2003.

[11]W.H.Press,B.P.Flannery,S.A.Teukolsky,andW.T.Vetterling.NumericalRecipesinC:

TheArtofScienti cComputing.CambridgeUniversityPress,January1993.

[12]V.Sequeira,K.Ng,E.Wolfart,J.Goncalves,andD.Hogg.Automated3Dreconstructionof

interiorswithmultiplescan–views.InProc.ofSPIE,ElectronicImaging’99,TheSocietyforImagingScienceandTechnology/SPIE’s11thAnnualSymp.,SanJose,CA,USA,January1999.

[13]H.Surmann,K.Lingemann,A.N¨uchter,andJ.Hertzberg.A3Dlaserrange nderforautonomous

mobilerobots.InProceedingsoftheofthe32ndInternationalSymposiumonRobotics(ISR’01),pages153–158,Seoul,Korea,April2001.

[14]H.Surmann,A.N¨uchter,andJ.Hertzberg.Anautonomousmobilerobotwitha3Dlaserrange

nderfor3Dexplorationanddigitalizationofindoorenvironments.RoboticsandAutonomousSystems,45:181–198,December2003.

[15]S.Thrun,D.Fox,andW.Burgard.Areal-timealgorithmformobilerobotmappingwithappli-

cationtomultirobotand3Dmapping.InProceedingsoftheIEEEInternationalConferenceonRoboticsandAutomation(ICRA’00),SanFrancisco,CA,USA,April2000.

[16]Z.Zhang.A exiblenewtechniqueforcameracalibration.IEEETransactionsonPatternAnalysis

andMachineIntelligence,11(22):1330–1334,2000.

[17]H.ZhaoandR.Shibasaki.ReconstructingTexturedCADModelofUrbanEnvironmentUsing

Vehicle-BorneLaserRangeScannersandLineCameras.InSecondInternationalWorkshoponComputerVisionSystem(ICVS’01),pages284–295,Vancouver,Canada,July2001.

本文来源：https://www.bwwdw.com/article/bgw1.html