智利国家基金 - 文献计量

ResearchPolicy41 (2012) 1461–1475

ContentslistsavailableatSciVerseScienceDirect

ResearchPolicy

journalhomepage:www.elsevier.com/locate/respol

Theimpactofnationalresearchfunds:AregressiondiscontinuityapproachtotheChileanFONDECYT?

JoséMiguelBenaventea,GustavoCrespib,LucasFigalGaronec,d,AlessandroMaf?olic,?

a

BusinessSchool,UniversidadAdolfoIbá?nez,Santiago,Chile

CompetitivenessandInnovationDivision,Inter-AmericanDevelopmentBank,Washington,DC,UnitedStatesc

StrategyDevelopmentDivision,Inter-AmericanDevelopmentBank,Washington,DC,UnitedStatesd

UniversidaddeSanAndrés,Victoria,BuenosAires,Argentina

b

article

info

abstract

Articlehistory:

Received15April2009

Receivedinrevisedform7April2012Accepted19April2012

Available online 15 May 2012

JELclassi?cation:O30O38H43C21

Thispaperanalysestheroleofnationalresearchfundsinpromotingscienti?cproductioninemergingeconomies.ThestudyfocusesontheimpactoftheChileanNationalScienceandTechnologyResearchFund(FONDECYT).Theanalysisusesdatadrawnfrominternationalsourcesofbibliometricinformationcombinedwiththeadministrativerecordsoftheprogram’sexecutingunit.Tomeasuretheprogram’simpact,weimplementaregressiondiscontinuity(RD)designonprincipalresearcherswhoappliedforfundingbetween1988and1997consideringasoutcomesbothquantity(publicationsupto2002)andquality(citationsupto2005)oftheirscienti?cproduction.Ourresultsshowsigni?cantandpositiveimpactintermsofpublications,butnoimpactintermsofqualityofscienti?cproductionintheproximityoftheprogram’sthresholdranking.

? 2012 Elsevier B.V. All rights reserved.

Keywords:FONDECYTChile

EconomicsofScienceScienti?cgrants

Regression-discontinuitydesignPolicyevaluation

1.Introduction

Academicsandpolicymakershaveincreasinglyrecognizedtheimportanceofscienti?cresearchinprovidingthefounda-tionsforbothinnovationandcompetitiveness.Thisrecognitionhasbeenmatchedbyanincreasingamountofpublicfundingforsuchresearchbygovernmentsandinternationaldonors.However,thereisapersistentlackofevidencethattheseinvestmentslead

?Thisstudywasdevelopedaspartof“IDB’sScienceandTechnologyPrograms:anEvaluationofTechnologyDevelopmentFundsandCompetitiveResearchGrants”,aprojectcoordinatedbytheOf?ceofEvaluationandOversight(OVE)oftheInter-AmericanDevelopmentBank(IDB).WearegratefultoRobertoIunes,InderRuprah,YuriSoares,DiegoUbfalandGonzaloVázquezBaréforusefuldiscussionsandcom-ments.Wealsothankfouranonymousrefereesfortheirhelpfulsuggestionsandcomments.J.M.BenaventealsowantstoacknowledgeNucleoMilenioInitiativeNS100017“IntelisCentre”forpartialfunding.The?ndingsandinterpretationoftheauthorsdonotnecessarilyrepresenttheviewsofInter-AmericanDevelopmentBank.Theusualdisclaimerapplies.

?Correspondingauthor.Tel.:+12026232802.

E-mailaddresses:jmbenavente@uai.cl(J.M.Benavente),gcrespi@iadb.org(G.Crespi),l?gal@iadb.org(L.FigalGarone),alessandrom@iadb.org(A.Maf?oli).

togreaterscienti?coutputand,ultimately,tobettereconomicperformance.Infact,whilemuchoftheavailableliteratureconcen-tratesonmeasuringanddescribingtheresultsofdifferenttypesofsciencepoliciesintermsofscienti?coutputs,evidenceonthecausalrelationshipbetweenthebudgetsspentonscienceandsci-enti?coutputsisstillscarce.1Thislackoriginatesnotonlyfromthedynamicnatureofscienti?cresearch,whichinvolvesrecur-sivefeedbacksbetweeninputsandoutputs,butalsofrompracticalrequirements,whichinvolvesde?ningcounterfactualsandgainingaccesstodataongroupsofbene?ciariesandcontrolgroupsofnon-bene?ciaries.2Thesepracticalrequirementsareprobablythemainreasonwhyexperimentalandquasi-experimentaldesignsarenotcommoninsciencepolicy.

Thescienti?cprocessresultsinseveralresearchoutputsthatcanbeclassi?edintothreebroadlyde?nedcategories:(1)new

Examplesofrecentimpactevaluationsoffundingofscienti?cactivityareChudnovskyetal.(2008),JacobandLefgren(2011a,b)andUbfalandMaf?oli(2011).2

Inthisframework,thecontrolgroupusuallyreferstoresearchersthatdonotreceivetheprogram’sfunding.

1

0048-7333/$–seefrontmatter? 2012 Elsevier B.V. All rights reserved.http://dx.doi.org/10.1016/j.respol.2012.04.007

1462J.M.Benaventeetal./ResearchPolicy41 (2012) 1461–1475

knowledge,(2)highlyquali?edhumanresources,and(3)newtech-nologies.Althoughtherearenodirectmeasuresofthesetypesofresearchoutputs,severalproxieshavebeenusedinpreviousstudies,amongthempublications,citationsandnumbersofPhDdegreesawarded.Thispaperfocusesonthecausaleffectthatonespeci?cpolicyinterventionsuchasanationalcompetitiveresearchfundhasonthecreationofnewknowledge–the?rsttypeofoutput–usingaquasi-experimentaldesign.Onthebasisofbibliometricdataandimplementingaregressiondiscontinuity(RD)approachwewanttoanswerthefollowingresearchquestions:

(i)Doesanationalcompetitiveresearchfundincreasetheamountofscienti?cproductioninanemergingcountry?

(ii)Doesanationalcompetitiveresearchfundimprovethequality

ofscienti?cproduction?

Toexploretheabovequestions,weevaluatetheimpactoftheChileanNationalScienceandTechnologyResearchFund(FONDE-CYT).Inadditiontosheddingsomelightontheeffectivenessofthispolicyinstrument,weaimatcontributingtotheexistingliteraturealsofrommethodologicalpointofview,showinghowarigorousquasi-experimentalevaluationcanbeimplementedanddiscussingthetypeofdatarequiredforthispurpose.Throughouranalysis,weareabletogenerateconsistentandrobustresultsthatshowsignif-icantandpositiveimpactonquantityofpublications(i.e.quantityofscienti?cproduction),butnoimpactonqualityoftheresearch.Therestofthepaperisorganizedasfollows.Afterthisbriefintroduction,Section2providessomeinformationontheChileaninnovationpolicyandontheFONDECYTinstitutionalsetting.Sec-tion3presentsourtheoreticalframework.Section4discussestheevaluationstrategy,datasourceandpreparation,andtheeconometricmodel.Section5discussesthemain?ndingsoftheprogram’simpactevaluation.Finally,Section6concludesandiden-ti?espotentialextensionsofthisresearch.

2.TheChileaninnovationpolicyandFONDECYT’sinstitutionalsetting

InChile,asinmanyothercountriesofLatinAmerica,theinstitutionalsettingforScienceandTechnology(S&T)hasbeenhistoricallybasedonanetworkofuniversitiesandtechnologicalinstitutessupportedandcontrolledbythepublicsector.Indeed,sincethemid-19thcentury,universitiesprovidedthemainnet-workforbasicresearchandthenucleiformostoftheappliedresearchconductedinthecountry(BenaventeandCrespi,1996).Lateron,duringthe20thcentury,anetworkoftechnologicalinsti-tuteswascreatedbytheStatewiththeaimofsupportingtheindustrializationprocessandtoincreaseproductivityofthenat-uralresourcebase(inparticularintheagriculture,forestryandminingsectors).Inadditiontothis,mostofthetechnologicalactiv-itiesdevelopedintheproductivesectorswerecarried-outbyasetofpubliclyownedenterprisesestablishedafterWWII.TheNationalDevelopmentAgency(CORFO),establishedin1939playedapivotalroleinthecoordinationand?nancingoftheoverallindustrial-izationprocess,includingtechnologicaldevelopment.Tosupportthisprocess,theNationalScienceandTechnologyResearchCouncil(CONICYT)wasestablishedin1967,undertheMinistryofEdu-cation,withtheaimofsupportinghumancapitalformationandstrengtheningthescienceandtechnologybaseofthecountry.Theprocessofeconomicreformsofmid-70sledtoincreasedparticipationoftheprivatesectorinmanystrategicindustries(withtheexceptionofcoppermining),toareductioninthepublicfund-ingavailabletothepreviouslycreatedtechnologicalinstitutesandtoadramaticchangeinthewaypublicresourcesforS&Twereallo-cated,movingfromdirecttransfersofbudgetappropriationsto

competitivemechanisms.TheNationalScienceandTechnologyResearchFund(FONDECYT),establishedin1982underCONICYT,wasaclearmanifestationofthesechanges.Indeed,FONDECYT’smainobjectiveis“tomaintain,strengthenanduseofthenationalcapacityforhigh-qualityresearch”throughsupportingindividualandcollectiveresearchinitiativesofnationalresearchersand?nancingthemonaqualityandexcellencebasis.

Itisimportanttosaythatdespitethechanges,theactualimplementationofsciencepolicyduringthemilitarygovernment(1973–1989)followedasupplyorientedapproachinwhichtheori-entationoftheeffortswasstilldeterminedbythepublicsectorandtheacademiccommunity.Thisapproachwasclearlyinspiredbythelinearmodelunderwhichitwasassumedthatknowledgewasasortofpublicgoodthatonceproducedbyscienceandtechnol-ogyorganizationswouldautomatically?owtowardtheproductivesector.ThisapproachtranspirestothedesignofFONDECYTwhichisahorizontalfundthatsupportmostly“curiositydriven”research.

Withthearrivalofthedemocraticgovernments,sinceearly90sanewvisionontheimportanceofthescience,technologyandinnovationfordevelopmentstartedtoemerge.Thisvisioninaddi-tiontorecognizingtheimportancethattechnologicalchangeandinnovationplayforlongtermeconomicgrowth,alsolocatedthe?rmatthecoreoftheinnovationprocess.So,amajorprocessofinstitutionalconstructiontookplace,?rstthroughthelaunchingofaseriesnationalprograms3andthenthroughtheestablish-mentofaNationalCouncilforInnovationandCompetitivenessandtheapprovalofalongtermscience,technologyandinnovationstrategy.4

Alongthisprocessofinstitutionaldevelopmentamajorshiftinscience,technologyandinnovationpoliciestookplaceputtingastrongeremphasisonmarketincentives,designinginterventionsthatfocusedonthedemandsideoftheinnovationprocess–the?rm–andonfosteringthelinkagesamongthedifferentactorsinthesystem.Newfundingmechanismswerecreatedandtheland-scapeofpublicprogramsbecamefarmorecompleteandcomplex.5

Despitethisprocess,throughoutthisperiod,theFONDECYT’scallsforproposalsremainedthemostimportanthorizontalmech-anismforthefundingofscienti?cresearchinChile.Sinceitsinceptionin1982,FONDECYTsupported10,372researchprojects(Fig.1)withaverageapprovalratesaround35%ofallthesubmittedproposals.

Toselecttheprojectstobefunded,FONDECYToperatesonthebasisofannualcompetitions,opentobothindividualresearchersandresearchinstitutions.Bydesign,FONDECYTisatotallyneu-tralinstrumentandthequalityoftheresearchproposalistheonlycriteriontoallocatefunding.ANationalCouncilforScienti?candTechnologicalDevelopment–whichincludestheMinistersofEducation,Planning,andFinance–isresponsibleforfunds’allocation.Thefund’soperationsaremanagedbytwoSuperiorCouncils–oneforscience(sevenmembers)andonefortechnologi-caldevelopment(?vemembers).Inturn,thesearesupportedby23studygroupsmadeupofsubjectspecialistsinthevarious?eldsof

3

SuchastheScienceandTechnologyProgram(1990–1995),theTechnologyInnovationProgram(1996–2000)andtheInnovationandTechnologyDevelopmentProgram(2001–2006).4

SeeNationalCouncilforInnovationandCompetitiveness(2010),NationalAgendaforInnovationandCompetitiveness(2010–2020).5

ParticularlyimportantwasestablishmentoftheNationalTechnologyDevelop-mentFund(FONTEC)managedbyCORFOandoftheNationalFundforScienti?candTechnologicalDevelopment(FONDEF)managedbytheCONICYT.ThefocusofFONTECwasonthefundingofdemanddrivenR&Dprojectssubmittedby?rms.Ontheotherhand,FONDEF’sfocusedonfundingcollaborativeR&Dprojectsperformedbypublicinstitutionsanduniversitiesincollaborationwiththeprivatesector.

J.M.Benaventeetal./ResearchPolicy41 (2012) 1461–1475

1463

005004stcejo0r0p30020011980199020002010yearSource: CONICYT (2012): http://www.conicyt.cl/573/article-27736.html Fig.1.NumberofresearchprojectssupportedbyFONDECYTperyear.

research.6ThetwoSuperiorCouncilsappeartofunctionindepen-dentlyofanyotherformallinkagewithCONICYT,exceptthroughthebudgetpreparationprocess,andthrough“specialinitiatives”.Theamountoffundingperprojectsystematicallygrewovertime.Indeedwhileduringthe?rst?veyearsoftheprogramaver-agefundingwasonlyaboutUS$10,000perproject,sincemid-80sandduringthe90saveragesupportgrewtoaboutUS$50,000per-project.Duringthelastphase,andinparticular,sincetheestablishmentofthenewinnovationstrategy,averagefundingper-projectgrewtoaboutUS$100,000.Despitethegrowingtrendsintheaveragefunding,evenunderthebestscenariooftheendoftheperiod,actualfundingperprojectislessthanonethirdofwhatisthenorminmanydevelopedcountries.7So,givenitsmeagerfund-ingperprojectincomparisonwithinternationalstandards,itisacontentiousissuewhetherthesupportprovidedbythisprogramtoChileanresearchershasbeenrelevantenoughastohaveanimpactinscienti?cproductivityand,moreimportantly,onthequalityoftheresearchconductedinthecountry.8

3.Thetheoretical

9

framework3.1.Therationalebehindpublicfundingofscience

ThetwofundamentalfeaturesofFONDECYTrationalearethepublicgoodnatureofscienti?cresearchandtheparticularincen-tivesystemthatgovernsthegenerationofhighqualitycodi?edknowledge.Fig.2syntheticallyrepresentstheFONDECYTrationale.SincetheseminalworksbyNelson(1959)andArrow(1962),schol-arshavefrequentlyde?nedscienti?cknowledgeasadurablepublicgood,i.e.non-excludable,non-rivalandcumulative.Thispublicgoodnatureofscienti?cknowledgeprovidestheprimeandbasicjusti?cationforthepublic?nancingofresearchprojects.Inpar-ticular,theimpossibilityofcompletelyappropriatingthebene?tsarisingfromthegenerationofscienti?cknowledgecausesadiffer-encebetweentheprivateandthesocialmarginalreturnofscience,makinginvestmentfallshortofoptimallevels.Furthermore,thenon-rivalandcumulativecharacterofnewknowledgeintensi?es

6

FONDECYTisadministratedbyadedicatedadministrativeunit,whichincludes23staffmembersandisledbyanexecutivedirector,withthesupportofCONICYTstaff(especiallyfromtheInformationDepartmentandAccountingUnit).7

Wearegratefultooneoftheanonymousrefereesforpointingoutthiskeyissue.8

However,theaveragesupportgivenbyFONDECYThasinrecentyearssigni?-cantlyincreasedanditisnowclosertotheaveragesupportprovidedbyothersimilarprogramsinLatinAmerica,suchastheArgentina’sFONCYT(Codner,2011).9

ThissectionfollowsthesurveyonEconomicsofSciencebytheOf?ceofEvalu-ationandOversightoftheIDB(IDB,2006)andChudnovskyetal.(2008).

thedif?cultyofcreatingincentivesthatcancompensateforthenon-appropriablepro?ts.Finally,theuncertaintyandindivisibilityofknowledgeinvestmentscauseanevengreatersub-optimalityintheallocationofresources.

However,DasguptaandDavid(1994)stressthatthejusti?-cationforthepublicsupporttoscienti?cresearchgoesbeyondtheneedofcorrecting“marketfailures”,arguingthatthequalityofbeingpublic(nontacit)isnotinherenttoknowledge.Indeed,researcherscanmakethedecisiontocodifyknowledgeandmakeitavailableasinformationinsteadofmaintainingitintacitform.Inthisframework,publicfundingshouldalsoaimtoprovidetherightincentivesformaintainingabalancedallocationofresearcheffortbetweenscienti?candtechnologicalresearch.Finally,theworksofevolutionaryscholarsprovidealternativejusti?cationsforthepublicfundingofscienceandcriticizesthepublicgoodargumentbyclaimingthatlearningandtransmissioncostscouldsigni?cantlydiminishboththenon-rivalandthecumulativecharacterofknowl-edge(Pavitt,2001).Therationaleforpublicsupportofscienti?cresearchhastobefoundinthedynamicandevolutivenatureoftheknowledgecreationprocess.Publicinvestmentinscienceshouldfostersystemiclearningcapabilitiesbytrainingscientists(SalterandMartin,2001),developingnewmethods(Rosenberg,1992),creatingknowledgenetworks(Lundvall,1992)andincreasingthecapacitytosolveproblems(PatelandPavitt,2000).

TheFONDECYTfundsupportsscienti?cresearchinChilebyprovidinggrantsthroughacompetitivemechanismbasedonqual-ityandexperience.10Nevertheless,somepotentialtrade-offsareembeddedinthespeci?cparametersadoptedintheselectionpro-cess.Gambardella(2001)providesevidenceoftheeffectivenessoffundingscientistsonthebasisoftheirpublicationpro?le(excel-lenceorexperiencecriterion).Thisselectioncriterionnotonlyleadstomoreandbetterpublications,butitalsoreducestheprobabilityofprojectfailure.However,theadoptionoftheexperiencecriterionmayleadtotheconcentrationoffundingamongalimitednum-berofscientistsortopicsandlessvarietyintheresearchportfolio.Indeed,Mollas-GallartandSalter(2002)criticizetheexcellencecriterionarguingthatresearchvarietyincreasestheprobabilityofobtainingvaluableresearchresults.Inthisdirection,criteriabasedonthequalityoftheresearchproposalsusuallyassumemorerisksofprojectfailureasacounterpartforresearchdiversi?cation.Addi-tionalallocationcriteria,suchastheportfoliosystemproposedbySchererandHarhoff(2000)aimatpursuingagreatervarietyofresearchtopics–diversi?cation–andaccesstofundingfornewgroupsandyoungerresearchers,thusincreasingtheprogram’sout-reach.

TheFONDECYTprogram,asotherscienti?cresearchfunds,adoptsamixofcriteriathattrytobalancetheabovementionedtradeoffs.TheFONDECYTevaluatorsranktheresearchproposalsonthebasisoffourweightedcriteria:(i)thequalityoftheresearchproposal(35%);(ii)theprojectviability(25%);(iii)theabilityandproductivityoftheprincipalresearcher(29%),and(iv)theabil-ityandproductivityofthesecondaryresearchers(11%).Ascanbeseenfromtheweightsdistribution,theFONDECYTselectionpro-cessgivessomepreferencetothequalityoftheresearchproposalcriterion.However,forwellrenownedresearchers,especiallythosewithpreviousbutnotnecessarilycurrentgoodpublicationperfor-mance,theirCVmayaffectmorethantheprojectqualitycompared

10

TheFONDECYT’sstrategyclearlyfallsintoasubsidycategory:thegovernmentgrantssubsidiestoscientiststo?nancetheirresearchactivities.Otherstrategiescommonlyidenti?edbyscholarsandpolicymakershavebeentheFrenchCNRSmodel(thegovernmentusespublicresourcestodirectlyproduceanddiffusesci-enti?cresearchthroughstate-ownedorganizationswherethescientistsarepublicemployees)andthegrantingofpropertyrights(thegovernmentgrantspropertyrightstoprivateresearcherstofostertheproductionofscienti?cknowledge).

1464J.M.Benaventeetal./ResearchPolicy41 (2012) 1461–1475

?Scientific knowledge is a public good and private investment falls short?Non-balance between codified and tacit knowledgeFunding of R&D projects through competitive funds Researchers compete on the basis of quality of projects ↑publications↑patentsCountry↑Externalities↑Spill-over effectsSource: CONICYT (2012): http://www.conicyt.cl/573/article-27736.html.

Fig.2.TheFONDECYTrationale.

toayounger,unknownresearcher.11AnotherinterestingfeatureofFONDECYTfundingmechanismisthatprincipalresearchersarerequiredtoshowevidenceofsubmissionoftheproject’soutputstoanindexedjournalasaconditionfortheformalcompletionoftheproject.

3.2.Expectedoutcomesanddeterminantsofscienti?cproductivity

Howshouldweevaluatetheresultsofpublicsupporttoscience?Themostgenerallyacceptedapproachhasbeentousebibliometricdata,sinceingeneralitisacceptedthatthenumberofpublicationscanbeagoodmeasureoftheproductionofcodi?edknowledgeandthepossibilityofaccesstothisknowledge.12FollowingStephan(1996),ausualformtomeasuretheimportanceofascientist’scontributionisthroughthenumberofpublicationswithsomeweightingtocorrectforthequalityofoutputs.Althoughthetypicalwaytocontrolforqualityhasbeentousetheimpactfactor13ofthejournalswherethoseoutputswerepublished,thismeasurementofqualitydependsonthe?eld,thetypeandthesizeofthejour-nal,?uctuatesfromyeartoyearanditdoesnotalwaysprovideanappropriatemeasureofthequalityofscientist’spublications(AminandMabe,2000).Forthisreason,adirectmeasurementofcitationstothearticlesproducedbyeachscientisthasbeenincreasinglyusedtocomplementthenumberofpublications.

CrespiandGeuna(2004)pointouttheimportanceofincorpo-ratinglagsintheestimationsoftheresultsofscienti?cresearch,inordertocatchthedelaybetweenthereceptionofthefundsandactualpublication.InaneconometricstudyofapanelofOECDcoun-tries,theyshowthattoidentifythemaximumimpactofagivenincreaseinthesciencebudgetonpublicationsandcitationsitisnecessarytowaitbetween?vetosixyears.14Moreover,CrespiandGeuna(2005)emphasizethatdifferent?eldsarecharacterizedbydifferentpropensitiestopublishinrecognizedjournals,aswellasbydifferenttimelagsinreachingpublication.

Theliteratureontheeconomicsofsciencehasprovidedanincreasingamountofempiricalevidenceregardingthevariablesthataffectthenumberofpublicationsbyscientists(Stephan,2010).

11

In2006,theFONDECYTestablishedanewandseparatedcompetitionforyoungresearcherscalled“FONDECYTiniciación”.12

Diamond(1986)providesempiricalevidencefortherelevanceofthismecha-nismtodeterminesalariesincreasesandpromotionsatuniversities.DebackereandGl?nzel(2004)analyzetheresultsofanexperimentthatconsistedindistributingfundstoFlemishuniversitiesonthebasisofbibliometricoutput.13

Impactfactorsarenomorethanameasurementofthefrequencywithwhichthe“average”articleofajournalwasmentionedinacertainyear.Inparticular,theimpactfactoriscalculateddividingthenumberofcitationsreceivedbyarticlespub-lishedduringthetwopreviousyearsinagivenjournalbythenumberofpublicationsinthoseyearsinthesamejournal.14

Inthisdirection,AroraandGambardella(2005)considertheimpactofpub-licfundingtoeconomicresearchersintheUnitedStatesusingthepublicationsweightedbycitationsinawindowof5yearsafterthedecisiontograntthefundingwastaken.

Indeed,thisliteraturesuggeststhatscientists’publicationproduc-tivityisnon-linearlyaffectedbyage(Stephan,1996),15gender(Long,2001),levelofeducationoftheresearcher(Buchmuelleretal.,1999),theavailabilityofothersourcesofincome(Stephan,1996),thesizeandqualityoftheresearchlaboratory(TurnerandMairesse,2005)andthescoreobtainedduringtheapplicationpro-cessforresearchfunding(AroraandGambardella,2005).Inourempiricalanalysiswecontrolformostofthesein?uentialcovari-ates.

Finally,tofullyunderstandtheeffectsofnationalresearchfundssuchasFONDECYTonthedifferentoutputsoneneedstocon-siderhowfundingandtheinteractionbetweendifferentsourcesoffundingmayaffectresearcherincentivesanddecisions.Fundsforscienti?cresearchmightcomefrommanysources(contractswithothergovernmententities,consultancywithprivatecompanies,supportfrommultilateralorganizations,aidfrominternationaldonors,etc.)andthesedifferentsourcesoffundscanbeeithercomplementorsubstitute.16However,duringouranalysisperiodFONDECYTwaspracticallytheonlynationalsourceoffundingforscienti?cresearchinChile.AlthoughtwootherChileanprogramshadpotentiallysimilargoals,theirspeci?cobjectivesandtargetswerequitedifferentand,therefore,theirpotentialoverlapwithFONDECYTalmostnonexistent.Infact,theFONDEFprogram,alsomanagedbyCONICYT,targetedonlyuniversity-industrytechnol-ogytransfer,whilethePresidentoftheRepublicScholarshipaimedatsupportingpost-graduatestudiesoutsideofChile.

Somecomplementarityorsubstitutionstillmightcomefromthefundingforresearchprovidedbyinternationalcooperation.AtthattimeChilewasemergingfromadictatorshipandsointerna-tionalcooperationfundswereavailabletosupportS&TinChile.Nevertheless,internationalcooperationfundsusuallyfocusedonmoreappliedresearchprojectsorproblemsolvingactivitiesandtheywerenotaimedatgeneratingpublicationsinindexedjour-nals.Therefore,inprincipleweexpectthattheoverlapbetweenFONDECYTandinternationalcooperationfundingtoberatherweakaswell.Moreover,althoughboththecomplementaryandthesub-stitutioneffectofthisinternationalfundingcannotbecompletelydiscarded,oneshouldconsiderthatthisfundingwasavailableundertheexactsameconditionsforbothresearchersthatweresupportedbyFONDECYTandthosethatwerenot.

15SeealsoGonzalez-BrambilaandVeloso(2007).

16

Atypeof“crowdingin”(or“crowdingout”)phenomenoninrelationtootheravailableresearchfundsmayoccur.Ontheonehand,science,technologyandinno-vationactivitiesareusuallycharacterizedby“supermodularity”orcomplementary(MohnenandRoller,2005).Theremightbesituationsinwhichapositivecorrela-tionbetweennationalresearchfundsandothersourcesisexpected,e.g.resourcescanbeusedto?nance?xedcapitalcostsortheriskiercomponentoftheresearchproject.Ontheotherhand,thisrelationshipmightalsoleadtoasubstitutioneffect.Thiswouldoccurinsituationsinwhichthetypeofprojectbeingfundedbyresearchcouncilswasverysimilartothetypesofprojectsfundedbyothersourcesoffunds.Inthiscontext,publicfundsmightinfactbesuper?uous(Lach,2002).

J.M.Benaventeetal./ResearchPolicy41 (2012) 1461–1475

1465

Ques

tionsIndicatorsDataSourcesQuantityofScientificProductionNumberofPublicationi)ISI-SCI BibliometricDatabasesii)AdministrativeRecords

QualityofScientificProductionCitations to publicatio

nsSource: Au thors’ el aborati on

Fig.3.FONDECYTresearchquestions,indicatorsanddatasources.

4.Theevaluationstrategy

AfterreviewingtherationaleoftheFONDECYTprograminlightoftheeconomicsofscienceliteratureandhavingidenti?editspotentialoutcomes,inthissectionweaddressthemainresearchquestionsofthisstudy:(i)whatistheFONDECYT’simpactonthenumberofpublicationsproducedbythe?nancedscientists?(ii)WhatistheFONDECYT’simpactonthequalityofthepublicationsproducedbythe?nancedscientists?

Toanswerthesequestions,wehavetodealwiththeproblemthattheFONDECYTlacksofamonitoringsystemforcollectingdataandtrackingoutcomesofinterestforbothbene?ciaryandnon-bene?ciaryresearchers.WeovercomethisproblembyusingFONDECYT’sadministrativerecordscombinedwithsecondarydataonthepublicationpro?leofscientistsprovidedbytheScienceCita-tionIndexfromtheInstituteforScienti?cInformation(ISI-SCI)(Fig.3).

Wethenusethisinformationtoperformaquasi-experimentalimpactevaluationthroughadiscontinuityregressiondesigncon-sideringresearcherswhoappliedtotheprogrambetween1988and1997.

4.1.Datadescription

Tocarryoutthisevaluationwesetupaspeci?cdatabaseincooperationwiththeCONICYT’sInformationUnit.Thedatagath-eringprocesswasorganizedintothreephases:(i)identi?cationofthepopulationofreference;(ii)attributionofthescientiststothetreatmentandcontrolgroups;(iii)attributionofpublicationsandcitationstotheselectedscientists.

i.Identi?cationofthepopulation:thepopulationofreferenceincludes(i)allPrincipalResearchers(PRs)whoseprojectsreceived?nancialsupportoftheprogrambetween1988and1997,and(ii)allPRswhoappliedtotheprogramduringthatperiod,butwerenot?nancedbecausetheirprojectsrankedbelowthethresholdforbeingadmittedtofunding.

ii.De?nitionofthetreatedandcontrolgroups:thetreatedgroupisformedbyPRsthatreceivedFONDECYT’sfundingforthe?rsttimebetween1988and1997–i.e.theydidnotreceivefund-inginthe?rstsixyearsoftheprogram’slife(1982–1987)–anddidnotreceivefundingfromthisprogramagainatleastinasix-yearwindowfromtheyearinwhichtheir?rstprojectwasapproved.ThecontrolgroupisformedbyPRsthatdidnot

receiveFONDECYT’sfundingbetween1982and2002.Forthiscontrolgroup,weconsiderthe?rstcompetitioninwhichPRsparticipatedinourtreatmentperiod(1988–1997).17Giventhattheprogramallowsforthere-submissionofprojects,wedonotincludeinthesamplePRswhoseprojectswererejectedintheir?rstsubmission,butwereacceptedinlatercompetitions.18

iii.Attributionofthebibliometricdata:forPRsinboththetreated

andthecontrolgroupswecountthefullsetofpublicationsincludedintotheISI-SCIandproducedinthesix-yearwindowafterthedecisiontograntthefundingordenyitwastaken.19ItisworthnotingthattheselectionincludesallpublicationsuptoDecember2002.Inthisframework,youngerpublicationsmayexperiencesomecensoringintheircitations.Therefore,wecon-siderallcitationsmadeuptoDecember2005tominimizethispotentialproblem.

Table1showsthemaindescriptivestatisticsofourworkingsample.Oursampleincludes3143PRs,886PRsinthetreatmentgroupand2257PRsinthecontrolgroup.Whenanalyzingthemaindescriptivestatisticsofthetwogroupswegetthat,onaverage,treatedPRsproducedaround2morepublicationsthanPRsinthecontrolgroup.Ifweconsiderthoseresearcherswithapositivenum-berofpublications,20publicationsofthecontrolgroupreceived,onaverage,slightlymorecitationsthantheonesofthetreatedgroup.Projectsinthetreatmentgrouphadamuchlowerrankthanprojectsinthecontrolgroup(thatis,theywereclosertothetopintermsofevaluationscores),howeverbothgroupsareverysimilaronaverageinobservedcharacteristicsofthePR–ageinthesub-missionyearandgender–andtheirprojects–sizeintermsofthenumberofresearchersintheresearchteam,durationandpresenceofaforeignresearcherintheteam.

4.2.Theeconometricstrategy

Theimpactofpublic?nancingofresearchfundssuchastheFONDECYTisamuchlessexplored?eldthanotherpolicyeval-uations.Theidenti?cationoftheimpactofpublic?nancinghastodealwithaquiteclearpotentialselectionbias:researcherswhoseprojectsarethebestcandidatesforfundingarealsothoseresearchersthatwouldhavethelargestexpectedoutputintheabsenceoffunding.Inordertounderstandtheevaluationproblem,itisusefultospecifythefollowingknowledgeproductionfunction:

Yi,t+j=??Di,t+Zi,tB+?i+εi,t+j

(1)

whereYi,t+jistheresearchoutput(thenumberofpublicationsinrefereedjournalsbyaPRiduringjyearsafterthedecisiontograntthefundingordenyitwastaken),Ditisadummyvari-ablethattakesthevalueof1ifresearcherireceivesFONDECYT’sgrantattimet,??istheimpactfortheresearcherifromreceiv-ingagrant,Zitisavectorofobservabledeterminantsofresearch

17

WeallowPRstohaverejectedprojectsbefore1988forthecontrolgroupandbeforeits?rstapprovedprojectforthetreatedgroup.18

Theinclusionof“switchingprojects”wouldhavecontaminatedthetreatmentandcontrolsamples.19

ByconsideringthefullportfolioofpublicationsweareabletoestimatethepotentialspillovereffectsbetweenthoseprojectsfundedbyFONDECYTandotherresearchprojectsconductedbythesamePR.20

Thereisanon-negligiblereductionofthesampleafterconsideringonlythoseresearcherswithatleastonepublicationintotheISI-SCI(foragooddiscussionontheusageofISIbibliometricdata,seeCameron,2005).ThismightindicatethatChileanresearchersarepublishinginotherlanguage,suchasSpanish.ThisiscertainlyalimitationofusingISI-SCIdata,givenitsfocusonscienti?cliteraturewritteninEnglish(wearegratefultooneoftherefereeswhopointedoutthisimportantissue).However,becausewearecomparingChileanPRsinboththetreatmentandcontrolgroup,thisandotherproblems–suchasthe“cronyismeffect”(Narin,1976)–thatequallyaffectbothgroupsshouldnotbeathreattoourevaluationstrategy.

1466

Table1

Descriptivestatistics.

J.M.Benaventeetal./ResearchPolicy41 (2012) 1461–1475

Variable

GRANT=1

GRANT=0

Total

SD

Mean

SD

Obs.

Mean

Obs.

Mean

SD

Obs.

PublicationsCitationsaRankingbAge

Genderc

ResearchersDuration

Foreignresearcherd

(Count)(Mean)

(Standardized)(Years)(Dummy)(Count)(Years)(Dummy)

5.496.88?0.5142.070.712.922.100.13

16.6211.350.459.740.451.860.690.33

886465886886886886886886

3.117.421.2643.350.713.262.050.08

10.1025.770.729.740.451.910.710.27

2257803225722572257225722572257

3.787.220.7642.990.713.162.060.09

12.3421.621.039.750.451.910.710.29

31431268314331433143314331433143

Source:Authors’elaboration.a

ThisoutcomeisthemeannumberofcitationsperpublicationincludedintheISI-SCI,whenthenumberofISIpublicationsispositive.Thatis,itexcludesthecasesofzerocitationsforzeropublications.b

Tofacilitatetheinterpretationwehavenormalizedtherankingofeachyearrelativetothecut-offpoint.Inotherwords,ineachyearthecut-offcorrespondstothezeroranking.c

“Gender”isadummyvariablethattakesvalueoneiftheprojectleadergenderismaleandzerootherwise.d

“Foreignresearcher”isadummyvariablethattakesvalueoneiftheresearchteamhasatleastoneforeignmemberandzerootherwise.

outputsand?irepresentsnon-observables(totheevaluator)fromtheresearcherandthereforeunobservedprojectquality.Apri-maryobstacletoidenti?cationisthenon-randomassignmentoftreatments.Inparticular,selectionforsupportonthebasisofunob-servables(bybothapplicantsandCONICYT)mayleadtoanon-zerocorrelationbetweenthetreatmentandthosenon-observablesintheerrorterm,Cov(Di,t,?i)=/0.Inthiscase,thetreatmenteffectestimatedusingOLSmightnotre?ecttheprogram’scausaleffectonPRs’performance.

Inordertoovercometheselectionproblem,weadoptedtheso-calledregressiondiscontinuity(RD)design.Thistechniqueuti-lizesadiscontinuityintheprobabilityofselectionthatoccursataparticularthresholdwithrespecttosomeindexofqualitytoiden-tifythetreatmenteffectseparatelyfromtheimpactofquality.21Accordingtothisdesign,assignmentissolelybasedonwhetherapre-interventionmeasureisabove/belowanestablishedthreshold.Forinstance,considerthecaseinwhichcandidatesaresplitintotwogroupsaccordingtowhetherthepre-interventionmeasure(forexample,averageevaluationbypeersintheFONDECYTcase)isaboveorbelowaspeci?edthreshold.Thosewhoscoredabovethethresholdreceivedthegrantwhilethosewhoscorebelowaredeniedit.

Theregressiondiscontinuitydesignapproachreliesonthemaintainedhypothesisthatindividualswithascorejustbelowthethresholdscoreareverysimilarintheirobservedandunobservedcharacteristicstoindividualswithscorejustabovethethresholdscore.Thisdesignfeaturesbothadvantagesanddisadvantages.Ontheonehand,byexploitingthefactthatwithinaneighborhoodaroundthecut-offsubjectsassignedtothecontrolandtreatmentgroupsdiffersolelywithrespecttothevariableonwhichtheassignmenttotheinterventionisestablished(andwithrespecttoanyothervariablecorrelatedtoit),onecancontrolfortheconfoundingfactorsjustbycontrastingmarginalparticipantstomarginalnon-participants.22Bydoingthis,themethodallowsforidentifyingthemeanimpactofinterventionlocallywithrespecttothecut-offpoint.Intuitively,fortheidenti?cationtoholditmustnotbethecasethataspuriousdiscontinuityintherelationshipbetweentheoutcomeandthevariableonwhichselectionisbased

happenstocoincidewiththecut-offpoint.Ontheotherhand,thedesignfeaturestwomainlimitations.Firstly,itsapplicabilityisbyde?nitioncon?nedtothoseinstancesinwhichselectiontakesplacebasedonanobservablepre-interventionmeasureoronthebasisofadeterministicprocess.Secondly,evenwhenthedesignisfeasibleitonlyidenti?esthemeanimpactatthethresholdforselection.ConsideringEq.(1)theimplementationofthediscontinuityregressionapproachisbasedonestimatingthefollowingregres-sion:

Yi,t+j=??Di,t+(xi,t?ct)+??(xi,t?ct)Di,t+Zi,tB+ui,t+j

?=i,t(ct?h≤xi,t≤ct+h)

(2)

wherethenewvariablexi,tistherankinggiventotheresearchprojectduringtheassessmentandcisthecut-offranking(thatchangesfordifferentyears).Thetreatmentestimatorisgivenby??whiletheinteractiontermcontrolsbythepossibilitythattheslopesoftheoutcomefunctionatbothsidesofthecut-offbedifferent.Finallyhisthe“bandwidth”thatdetermineshowfareachobser-vationshouldbefromthecut-offpointinordertobeincludedintheestimationsample.Thisversionofthediscontinuityregressionapproachiscalledintheevaluationliteratureas“SharpRegressionDiscontinuity”(SRD)design.

InordertoprovideunbiasedresultsoftreatmentimpactsaSRDdesignrequiresthattheforcingvariableorrankingcapturesalltheinformationregardingthequalityofboththeprojectsandtheirresearchersandthatfundingdecisionbebasedonlyonthisrank.Thesituationbecomesmorecomplicatedwhenthegrantingagency,basedonadditionalinformationnotcapturedontherankassessment,makesadecision“topass”someprojectsthatother-wisewouldhavebeenrejected–projectsjustabovethecut-offrank–andtorejectsomemarginalprojectsthatotherwisewouldhavebeenpassed–projectsjustbelowthecut-offrank.Inthiscasesomesortofsampleselectionbias“atthemargin”couldstillremain.Thiscontextisknownasa“FuzzyRegressionDiscontinuity”(FRD)design.Inthiscaseweneedtoestimate(2)byusinginstru-mentalvariables.23FollowingJaffe(2002)agoodinstrumentunderthissituationisadummyvariablethattakesavalueof1whenevertherankingisbelowthecut-off.Inotherwordstheinstrumentwouldbe:

TheregressiondiscontinuitydesignestimatorwasintroducedbyThistlethwaiteandCampbell(1960).IntheeconomicliteraturethisestimatorhasbeenrecentlyusedbyBlack(1999),AngristandLavy(1999),DiNardoandLee(2004),VanDerKlaauw(2002),Lalive(2008)andUrquiolaandVerhoogen(2009).Theidenti?cationandestimationoftreatmenteffectsarediscussedinHahnetal.(2001),ImbensandLemieux(2008),LeeandLemieux(2009)andImbensandWooldridge(2009).22

Inthiscontext,thetermmarginalreferstothoseresearcherswhoseprojectsarenottoofarfromthethresholdorcut-offpointforselection.

21

1{xi,t≤ct}

(3)

Thetreatmenteffectcanberecoveredbycomputingtheratioofthejumpintheoutcomevariableatthethreshold(discontinuityjumpintherelationbetweenYiandxi)tothejumpintheprobabilityofparticipationinFONDECYTalsoatthethreshold(discontinuityjumpbetweenDiandxi).

23

J.M.Benaventeetal./ResearchPolicy41 (2012) 1461–1475

1467

Mean Citations per publicationPublications4321-2 -1 Relative Standardized Rankingh=globalh=h*=1.37h=0.50 1 2 24683-2-1Relative Standardized Rankingh=globalh=h*=0.88h=0.50 1 2 3 lpoly smooth: Publicationslpoly smooth: Publicationslpoly smooth: Publications lpoly smooth: Citationslpoly smooth: Citationslpoly smooth: Citations 4.5Mean Citations per publicationPublications43.532.5-2 -1 Relative Standardized Ranking0 1 4567892-1-.5 Relative Standardized Ranking0 .5 1h=h*=1.37h=1h=0.5 lpoly smooth: Publicationslpoly smooth: Publicationslpoly smooth: Publications h=h*=0.88h=0.5h=0.25 lpoly smooth: Citationslpoly smooth: Citationslpoly smooth: Citations Source: Authors’ elaboration

Fig.4.Therelationshipbetweenproject’srankingandresearchoutputsofFONDECYT’sprincipalresearchers.

Thisinstrumentshouldbehighlycorrelatedwiththetreatmentdummy(Di,t)byconstruction,butbecausewearealreadycontrol-lingfortherankingintheregression(xi,t),itshouldnotbecorrelatedwiththeerrorterm.BothSRDandFRDapproacheswillbeappliedinthispaper.

5.Empiricalresults

5.1.GraphingdiscontinuitiesandtestingtheRDdesign

Inthe?rststageofouranalysiswestudytherelationshipbetweentherankingobtainedbyprojectsandthenumberofISIpublications(citations)generatedbythePRs.Forthispurpose,weestimatethisrelationshipthroughalocallinearregressionjusttotheleftandrightofthecut-offpoint,24usingasdependentvariablethefrequencyofpublications(meancitationsperpublication)andasindependentvariabletherankingobtainedbytheprojects.25Weconsiderseveralbandwidthsincludingthe“optimalbandwidth”

Welettheregressionfunctiondifferonbothsidesofthecut-offpoint(seeImbensandLemieux,2008;LeeandLemieux,2009;andImbensandWooldridge,2009).Foralllocallinearregressionsweuseatriangle(edge)Kernel.25

Forallestimationsonpublicationsandcitationstheoutliersweredroppedforeachoutcomeseparatelyusingthefollowingrule-of-thumb:YiisanoutlierifYi>E(Yi)+3SD(Yi).AppendixIdisplaysthedistributionofpublicationsandcitationsafterthisprocedure.

24

proposedbyImbensandKalyanaraman(2009).26Fortheregres-siondiscontinuityapproachtobevalidweshouldobservesomesortofdiscontinuityoftheoutputfunctionatornearthecut-offpoint.

ResultscanbeseeninthetwotoppanelsofFig.4.Thetopleftpanelshowsthelocallinearregressionresultsforpublications.Inthiscase,the“jump”atthecut-offpointindicatesthatthediscon-tinuouschangeineligibilityincreasesthenumberofpublicationsoftheresearchers.SupportedPRswhoseprojectsareatthetopoftherankingproducedmorepublications.

ThetoprightpanelinFig.4showsthelocallinearregressionresultsforcitations.ThegraphsuggeststhatPRswithprojectswiththehighestscorenotnecessarilyproducedhigherqualitypublica-tions(publicationsthatwerecitedmoreoften).Quitesurprisingly,we?ndthataroundthecut-offpoint,wherethediscontinuityissmaller,thoseresearcherswhoseprojectswererankedabovethecut-offpointhadhigherqualitypublicationsthanthosebelowit.AninterestingconclusionofcombiningthesetwographsisthatFONDECYT’sreviewerscouldhavetendedtogivemoreweighttoresearchersthatwereexpectedtoproducemorepublicationsregardlessoftheirquality.

OnepotentialproblemwiththetwographsinthetoppanelsofFig.4isthatthelocallinearregressionisestimatedacrossthefullsupportoftheforcingvariable.AccordingtoImbensandLemieux

Weestimatethe“Optimalbandwidth”foreachoutcomeseparatelyfollowingtheroutineforStata(“rdob”)developedbyFujietal.(2009).

26

1468J.M.Benaventeetal./ResearchPolicy41 (2012) 1461–1475

1ytili8b.aborP6. tnem4t.aerT2.0-2 -1 03Relative Standardized Ranking 1 2 h=globa l lpo ly smoo th: Treatment prob abili th: Treatment prob abili tyh=1tyh=0.25 lpoly smoolpoly smooth: Treatment probabili tySource: Authors’ elaboration

Fig.5.FONDECYTtreatmentprobabilityandproject’sranking.

(2008)thiscouldharmvisualclarityofanydiscontinuityintheoutcomefunctions.Toimproveouranalysis,wecomputelocallin-earregressionsconsideringonlythoseresearcherswithaproject’srankingwithinthe“optimalbandwidth”aroundthecut-offpoint.BothbottomgraphsofFig.4reinforceourpreviousconclusions.Thebottomleftpanelsuggeststhatthereisanimportantdiscon-tinuityforpublications.However,thebottomrightpanelindicatesthatimpactcouldevenbenegativeforcitations.

Beforeproceedingwiththeestimations,weneedtodeterminewhatsortofdiscontinuityregressiondesignshouldbeusedfortheanalysis.Onewayofdoingthisisbyplottingthetreatmentvariablebyproject’srankingtoseeifthereisasuddendropinthetreatmentprobabilityatthecut-offpoint.

Fig.5showstheresultsofthisanalysis.The?gureplotsthetreatmentprobabilitybyproject’srankingusinglocallinearregres-sionwithseveralbandwidths.Thegraphsuggeststhatnotalltheresearcherswithprojectswithranksbelowthecut-offpointwereselectedfor?nancingandthatnotallresearcherswithprojectswithranksabovethecut-offpointwererejected.Inotherwords,theresultsindicatethatthereissigni?cantmobility“atthemargin”andthataFRDapproachmightbemoreadequatefortheestimations.

WethenimplementtwoteststovalidatetheuseoftheRDdesign.First,althoughtheunderlyingassumptionthateachresearcherhasimprecisecontrolovertheforcingvariablecannotbetestdirectly,27wecantestwhetherthereisadiscontinuityinthedensityoftheforcingvariable,followingMcCrary(2008).Forthispurpose,weinspectthehistogramoftheforcingvariableandthekerneldensityestimatesoverbinswithabandwidthof0.05(80binsinthegraph–Fig.6,topleftpanel)andthenweusethefrequencieswithinthebinsgeneratedasadependentvariableinalocallinearregressiononbothsidesofthecut-offpoint(Fig.6,toprightpanel).Wealsocomputeasimilarprocedurewithstan-darderrorbandsinthebottompanelofFig.6.28We?ndnosignofajumpindensityofobservationsatornearthecut-offpoint,andthereforeconcludethatthedatashownoevidenceofmanipulationoftheassignmentvariable.

WealsotestthevalidityoftheRDdesignbyinspectingwhetherourmainobservedcharacteristicsofPRsandprojectsare“locally”balancedoneithersideofthecut-offpoint.Fig.7displaysthe

27

Thisassumptionsupportstheideaoflocalrandomassignmentofthetreatmentaroundthecut-offpoint.SeeLeeandLemieux(2009).28

WeestimatethebottompanelofFig.6byusingtheroutine“DCdensity”proposedbyMcCraryandNovakinhttp://emlab.berkeley.edu/～jmccrary/DCdensity/.

locallinearregressionestimationforeachofthosecharacteristics(speci?callyfromthelefttoright:ageandgenderofthePR,num-berofresearchersinteam,durationoftheprojectandpresenceofaforeignresearcherintheteam).Althoughthegraphsshowingeneralnoevidenceofdiscontinuityatornearthecut-off,somesmalldiscontinuitiesappearinparticularwhenweusethesmall-estbandwidths.29Toavoidanypossiblebias,weincludeallthesecovariatesascontrolvariablesinourestimation.

5.2.Estimationresults

Inwhatfollowswepresenttheresultsfortheimpactsonpublicationsandcitationsusingparametric(OLSandIV)andnon-parametrictechniques.30Forbothoutcomesweconsiderseveralbandwidths.31Table2summarizestheresultsoftheSRDwhenthedependentvariableisthenumberofpublicationsandtheestima-tiontechniqueisOLS.We?ndthatinthesamplearoundthecut-offpointFONDECYThasapositiveandsigni?cantimpactofaround2publications.

Table3showstheresultswhenweadoptaFRDdesignandestimatetheprogram’simpactonpublicationsusinginstrumentalvariables.Weusethedummyvariablede?nedbyEq.(3)asinstru-mentforthetreatmentdummyanditsinteractionwiththerankingasinstrumentfortheinteractionbetweenthetreatmentdummyandtheranking.TheresultsinTable3showthattheimpactisstat-icallysigni?cantandnumericallylarger(upto6publications)thantheimpactestimatedwithOLS.Overall,theresultsestimatedwithaFRDdesigncon?rmthattheprogramstillhasasigni?cantimpactonpublicationsevenwhenonecontrolsforselectionatthemargin.Adoptingthesameapproach,weestimatetheimpactofFONDE-CYTonthemeannumberofcitationsperpublication.Inthiscase,we?ndnosigni?cantimpact,consistentlywiththeoutcomeplotsofFig.4.Tables4and5summarizetheresultsforthecitationout-comevariableandshowthatwhenusingbothOLS(Table4)andIV(Table5)techniques,thetreatmentvariableisnegativeinalmostallcases,thoughneverstatisticallysigni?cant.32

Our?ndingsarecon?rmedandreinforcedwhenwere-estimateboththeimpactonnumberofpublicationsandmeannum-berofcitationsperpublicationusinganon-parametricapproach(Table6).33Indeed,westill?ndapositiveandsigni?cantimpactofFONDECYTonnumberofpublicationswhileamainlynegativeandnon-signi?cantimpactonmeannumberofcitations.34

29

However,itisworthmentioningthatwhilediscontinuitiesintheoutcomesarerobusttodifferentbandwidthsandtypesofKernels,inthegraphsofFig.7thesmalldiscontinuitiesareverydependentonbandwidthselectionandtendtodisappearwhenusingadifferentKernel.30

Forthenon-parametricapproachweestimatelocallinearregressionsonbothsidesofthecut-offusingatriangleKernelinaFRDframework.Wecomputetheratioofthejumpinoutcometothejumpintreatment(LocalWaldestimate).31

We?rstconsiderthefullsample.WethenusetheImbens–Kalyanaraman“opti-malbandwidth”thatdetermineshowfareachobservationshouldbefromthecut-offpointinordertobeincludedintheestimationsample.Finally,toseehowrobusttheseresultsareweestimatetheimpactonbothoutcomesbyusingfouraddi-tionalbandwidthsthatarisefromincreasingandreducingthe“optimalbandwidth”in0.25and0.50points.32

Giventhedistributionofpublicationsandcitations–rightskewedwithclump-ingatzeroandavariancenearly3timesgreaterthanthemean,AppendixI–wereplicateestimatesinTables2–4byusingaNegativeBinomialModel–thismodelaccountsbetterforoverdispersionthanthePoissonregressionmodel,whichassumesthatthemeanandvariancearethesame.Theresultsaresimilartothosepresentedinthepaperintermsofthemagnitude,signandsigni?canceoftheimpactoftheprogram.ForthesakeofbrevitywepresenttheminasummarytableinAppendixIV.33

Non-parametricresultsarerobusttodifferenttypesofKernels.34

InAppendixV(Figs.A2andA3)weevaluatethedependenceofthenon-parametricestimatesonbandwidthselection.Bothgraphsshowtheusualtrade-offbetweenbiasandvariance.However,theestimatedeffectforcitationsisclearlymoresensitivetobandwidthselection.

J.M.Benaventeetal./ResearchPolicy41 (2012) 1461–1475

1469

Table2

Treatmenteffectsonnumberofpublications,OLSestimates.

Coef?cient

(1)All

(2)

h=1.87

(3)

h=1.62

(4)

h*=1.37

(5)

h=1.12

(6)

h=0.87

DXDXAgeGender

No.ofresearchersDuration

ForeignmemberConstant

1.5709***(0.439)?0.1379(0.156)1.0422*(0.543)?0.0178*(0.010)0.1314(0.232)0.1029*(0.060)0.2394*(0.145)?0.4198(0.360)1.6854**(0.789)

1.7464***(0.453)0.1003(0.205)0.9228(0.571)?0.0155(0.012)?0.0632(0.263)0.0853(0.069)0.1875(0.165)?0.3674(0.391)1.6342*(0.940)

1.7485***(0.459)0.0289(0.243)0.9997*(0.592)?0.0191(0.013)?0.1267(0.282)0.1130(0.077)0.1645(0.182)?0.2546(0.436)1.8775*(1.051)

1.8714***(0.474)0.1450(0.305)1.0115(0.617)?0.0243*(0.015)?0.0732(0.318)0.0971(0.087)0.2657(0.195)?0.4275(0.462)2.1329*(1.202)

1.7767***(0.493)0.1279(0.373)0.9635(0.711)?0.0279(0.017)?0.1660(0.363)0.1052(0.094)0.3023(0.216)?0.0804(0.524)2.0928(1.335)

1.3067**(0.584)?0.9314(0.648)1.9960*(1.035)?0.0215(0.020)?0.2883(0.424)0.0423(0.107)0.4506*(0.266)?0.0554(0.584)2.7497*(1.617)

ObservationsR-squared

31010.056

26340.054

22720.060

19420.063

16320.064

12080.073

Source:Authors’elaboration.

Robuststandarderrorsinparentheses.*

p<0.10.**

p<0.05.***

p<0.01.

Scienti?cDisciplines,RegionandYeardummiesincluded–Thescienti?cdisciplinesaredividedinto:Agronomy/Zoology,SocialSciences,NaturalSciences,EarthSciences,EngineeringSciences,MedicalSciences,Legal-Economic-ManagementSciencesandHumanities(seeAppendixII).Theregionsincludedare:RegionofTarapaca,Antofagasta,Atacama,Coquimbo,Valparaiso,LibertadorG.B.O’Higgins,Maule,Bio-Bio,Araucania,LosRios,LosLagos,MagallanesandtheAntarticaChilena,AricaandParinacota,MetropolitanZoneandparticularresearchers.Finally,weconsideryeardummiesfrom1988to1997(seeAppendixIII).“D”isthetreatmentvariable,“X”istheforcingvariable(relativestandardizedranking)and“DX”istheinteractionterm.

Table3

Treatmenteffectsonnumberofpublications,IVestimates.

Coef?cient

(1)All

(2)

h=1.87

(3)

h=1.62

(4)

h*=1.37

(5)

h=1.12

(6)

h=0.87

DXDXAgeGender

No.ofresearchersDuration

ForeignmemberConstant

2.9938***(1.065)0.1886(0.299)1.9100***(0.727)?0.0203**(0.010)0.1405(0.232)0.0872(0.061)0.2307(0.145)?0.4830(0.365)0.9575(0.988)

3.5689***(1.192)0.6557(0.424)1.6728**(0.739)?0.0190(0.012)?0.0521(0.263)0.0637(0.071)0.1757(0.166)?0.4484(0.398)0.5511(1.192)

3.7512***(1.325)0.7520(0.567)1.6704**(0.744)?0.0235*(0.013)?0.1038(0.283)0.0837(0.080)0.1468(0.183)?0.3549(0.444)0.6755(1.340)

4.2902***(1.583)1.2963(0.854)1.3610*(0.786)?0.0316**(0.015)?0.0478(0.319)0.0610(0.091)0.2298(0.197)?0.5374(0.474)0.6537(1.559)

4.9642**(1.985)2.1680(1.364)0.6928(1.025)?0.0423**(0.019)?0.1277(0.366)0.0535(0.101)0.2398(0.223)?0.2228(0.539)0.3048(1.750)

6.3432*(3.474)3.2483(2.867)1.0658(1.501)?0.0439*(0.026)?0.2812(0.438)?0.0217(0.124)0.3177(0.296)?0.4471(0.668)0.0295(2.490)

ObservationsR-squared

31010.052

26340.047

22720.051

19420.049

16320.036

12080.007

Source:Authors’elaboration.

Robuststandarderrorsinparentheses.*

p<0.10.**

p<0.05.***

p<0.01.

Scienti?cDisciplines,RegionandYeardummiesincluded.“D”isthetreatmentvariable,“X”istheforcingvariable(relativestandardizedranking)and“DX”istheinteractionterm.

Table4

Treatmenteffectsonmeannumberofcitationsperpublication,OLSestimates.

Coef?cient

(1)All

(2)

h=1.38

(3)

h=1.13

(4)

h*=0.88

(5)

h=0.63

(6)

h=0.38

DXDXAgeGender

No.ofresearchersDuration

ForeignmemberConstant

?0.5738(0.663)?0.6040(0.707)?0.1738(0.381)?0.0414(0.028)?0.2074(0.504)?0.0353(0.117)0.2670(0.362)0.9518(0.778)4.9463*(2.567)

?0.5629(0.704)?1.2005(0.949)0.2871(0.622)?0.0260(0.030)?0.4311(0.584)0.2087(0.163)?0.2802(0.380)1.0715(0.990)4.4102*(2.419)

?0.7688(0.732)?1.5890(1.136)0.3037(0.697)?0.0353(0.035)?0.5511(0.636)0.0627(0.148)?0.2915(0.415)1.3015(1.031)5.7221**(2.695)

?0.7650(0.780)?2.0052(1.711)0.3597(1.080)?0.0494(0.040)?0.6856(0.777)0.0006(0.164)?0.0364(0.455)0.7581(1.021)4.1168(2.528)

?0.2398(0.894)?2.5329(2.144)2.0913(1.826)?0.0420(0.045)0.0289(0.772)0.0859(0.202)?0.5287(0.574)0.7858(1.107)3.9966(2.816)

?0.7209(1.095)?3.5919(4.206)3.9062(3.908)?0.0520(0.053)0.5075(0.848)0.3596(0.262)?0.8526(0.739)0.8577(1.444)4.8012(3.543)

ObservationsR-squared

12610.057

8960.050

7770.064

5760.080

4180.100

2800.145

Source:Authors’elaboration.

Robuststandarderrorsinparentheses.*

p<0.10.**

p<0.05.***p<0.01.

Scienti?cDisciplines,RegionandYeardummiesincluded.“D”isthetreatmentvariable,“X”istheforcingvariable(relativestandardizedranking)and“DX”istheinteractionterm.

1470J.M.Benaventeetal./ResearchPolicy41 (2012) 1461–1475

.6Frequency of ProjectsDensity of Projects.4.2020406080-20 -1 0 1 2 3Relative Standardized Ranking-2 -1 0 1 kdensity X2 3Relative Standardized Ranking = XDensity h=globalh=1h=0.5h=0.05 lpoly smooth: Frequency of Projectslpoly smooth: Frequency of Projectslpoly smooth: Frequency of Projectslpoly smooth: Frequency of ProjectsDensity of projects0.2.4.6-2 -1 Relative Standardized Ranking0 1 2 3Source: Authors’ elaboration

Fig.6.Densityandfrequencyofprojectsbyranking.

Summingup,ourresultsclearlyindicatethatFONDECYTsuccessfullycontributedtoincreasethequantityofscienti?cpro-ductionoftheChile’sNationalInnovationSystem(NIS),butnotitsquality.AsdiscussedinSection3,onewouldexpectthegrants

offeredbyanationalresearchfundtoaffecttherelativerewardsfromresearchcomparedwithotheractivitiessuchasteachingandconsulting.Fromtheindividual’spointofviewtherefore,receiv-ingagrantmayinducetheresearchertoallocatetimeawayfrom

Table5

Treatmenteffectsonmeannumberofcitationsperpublication,IVestimates.

Coef?cient(1)

(2)

(3)

(4)

(5)

(6)

All

h=1.38

h=1.13

h*=0.88

h=0.63

h=0.38

DXDXAgeGender

No.ofresearchersDuration

ForeignmemberConstant

?1.1455(2.086)?0.2528(0.989)?0.4755(0.798)?0.0407(0.029)?0.2062(0.504)?0.0361(0.120)0.2635(0.362)0.9586(0.771)5.4987*(2.835)

?0.8072(3.570)?0.8405(1.710)0.0172(2.318)?0.0252(0.036)?0.4297(0.584)0.2083(0.169)?0.2827(0.381)1.0690(0.984)4.6178(3.432)

?2.6081(4.833)?0.3826(2.471)?1.2949(3.659)?0.0248(0.048)?0.5448(0.644)0.0863(0.169)?0.2858(0.416)1.3945(1.034)6.8602*(3.780)

?14.3506(17.196)3.3230(7.219)?11.6855(15.331)0.0537(0.141)?0.3099(1.013)0.1369(0.253)0.1759(0.602)1.7953(1.603)10.5930(8.922)

7.4605(14.594)?8.8006(6.484)12.4258(15.791)?0.0897(0.092)0.0011(0.835)0.0008(0.286)?0.7077(0.677)0.1339(1.829)0.2906(8.422)

?2.6692(6.569)?4.8268(8.993)2.9194(10.663)?0.0363(0.070)0.4585(0.893)0.3880(0.270)?0.6950(0.878)1.0719(1.767)5.4121(4.233)

ObservationsR-squared

12610.056

8960.050

7770.054

576

418

2800.133

Source:Authors’elaboration.

Robuststandarderrorsinparentheses.*

p<0.10.**

p<0.05.***

p<0.01.

Scienti?cDisciplines,RegionandYeardummiesincluded.“D”isthetreatmentvariable,“X”istheforcingvariable(relativestandardizedranking)and“DX”istheinteractionterm.

J.M.Benaventeetal./ResearchPolicy41 (2012) 1461–1475

1471

4535.55Gender proportions (Male).6.65.7.75Age40-2 -1 01Relative Standardized Ranking h=globa l h=1 h=0.5 2 lpo ly smoo th: Agelpo ly smoo th: Agelpo ly smoo th: Age3-2 -1 01Relative Standardized Ranking 2 3h=globalh=1h=0.5 lpoly smooth: Gender (Male)lpoly smooth: Gender (Male)lpoly smooth: Gender (Male) 3.5No. of Researchers22.531.51.92Project duration2.22.12.3-2 -1 01Relative Standardized Ranking 2 3-2 -1 01Relative Standardized Ranking h=globalh=1h=0.5 lpoly smooth: No. of researcherslpoly smooth: No. of researcherslpoly smooth: No. of researchers h=globalh=1h=0.5 2 lpo ly smooth: Dura tionlpo ly smooth: Dura tionlpo ly smooth: Dura tion3Proportion of teams with at least one foreign member.06.08.1.12.14-2-101Relative Standardized Ranking h=globalh=1h=0.5 2 lpo ly smooth: F ore ign memberlpo ly smooth: F ore ign memberlpo ly smoo th: F ore ign member3 Source: Authors’ elaboration

Fig.7.Mainobservedcharacteristicsbyproject’sranking.

consultingtowardresearch,ortoputmoreeffortinagivenresearch

time.Inthisway,agrantmaystimulateresearcheffortsandaffectbothquantityandqualityofoutputs.InthecaseofFONDECYTthiseffectisre?ectedinagreaternumberofpublications,butnotinhigherquality.Why?

Theexplanationforthismixedimpactshouldbe?rstsoughtinhowthespeci?ccharacteristicsofFONDECYT’sfundingprocessmayormaynotaffectdifferentresearchoutputs.Twoissuesmightbeparticularlyrelevant:(i)theaverageamountofthefundingper

project,and(ii)theincentiveembeddedintheselectionanddis-bursementmechanismsoftheprogram.

AspointedoutinSection2,duringtheevaluationperiod,theactualfundingperprojectgrantedbyFONDECYTisrelativelylow,especiallywhencomparedtoresearchfundingindevelopedcoun-tries.Aproblemoflowper-projectfundingseemsconsistentwithourresults.TheresourcesgrantedbyFONDECYTareprobablyenoughtosupportasigni?cantincreaseinthevolumeofpublica-tionsinindexedjournals,butnotenoughtogenerateasigni?cant

1472J.M.Benaventeetal./ResearchPolicy41 (2012) 1461–1475

Table6

Treatmenteffectsonpublicationsandcitations,non-parametricestimates.

Publications

(1)

(2)

(3)

(4)

(5)

(6)

h=global

h=1.87

h=1.62

h*=1.37

h=1.12

h=0.87

D

Observations

3.4666***(1.271)3101

4.1736***(1.616)2634

4.7345**(1.959)2272

5.5982**(2.593)1942

7.2836*(3.984)1632

14.0306(11.747)1208

CitationsD

(1)

h=global

(2)

h=1.38

(3)

h=1.13

(4)

h*=0.88

(5)

h=0.63

(6)

h=0.38

?1.7525(2.569)?4.2902(8.874)?18.8162(45.858)7.9279(16.158)?0.2571(3.746)6.5890(12.435)Observations

1261

896

777

Source:Authors’elaboration.*

p<0.10.**

p<0.05.***

p<0.01.

“D”isthetreatmentvariable.Allcontrolvariablesincluded.

jumpinqualityofthesepublications.Infact,whilethenumberofpublicationscanbeincreasedalsothroughrelativelyinexpensiveactivities–forinstance,networking,promotionofstudiesatcon-ferencesandseminarsandhiringofresearchassistants,improvingthequalityofresearchmayrequiremuchmoreexpensiveinvest-ments–suchasneworrenewedlaboratories,datacollections,?eldworks.

Inaddition,theFONDECYT’sfundingallocationanddisburse-mentincludeexplicitincentivetopublishmoreinindexedjournals,butnottoproducebetterpublications.Theresearchersareaskedtoindicateinwhatjournaltheyexpecttopublishtheirresultsandtheclosingoftheprojectissomehowconditionaltothesubmissionofthemanuscripttothesejournals.Again,theincentivespoten-tiallycreatedbysuchprovisionseemconsistentwithourresults.Indeed,thisprovisionintroducesaclearandspeci?cincentivetopublishmoreinindexedjournal,butnotnecessarilytoseekforhigherqualityproducts.Onecouldclaimthatthequalityincentiveisembeddedinthequalityofthetargetedjournals.Evenconsider-ingthis,theincentivesclearlyseemmorebindingonthequantityratherthanqualityofpublications.

6.Conclusion

TheimpactevaluationoftheFONDECYTshowsclearevidenceofapositiveandsigni?cantimpactintermsofPRs’publications.Indeed,usingthemostconservativeestimatesproducedbyaSRDdesign,we?ndanimpactofabouttwoadditionalpublicationsinthesix-yeartimewindowafterthereceptionofthegrant.These?gurescomparesfavorablywiththeresultsobtainedfortheArgentina’sFONCYT(Chudnovskyetal.,2008)–aboutoneadditionalpublication–andtheNIHpost-doctoralandregularresearchprograms(JacobandLefgren,2011a,b)–alsoofaroundoneadditionalpublication.So,inprinciple,wecanconcludethattheFONDECYTprogramhadasigni?cantimpactonthescienti?cproductivityofChileanresearchers.

However,thisincreaseofresearchoutput’svolumemighthavecomeatthecostofalackofimpactintermsofquality.Indeed,eitherusingdifferentRDdesignsorconsideringseveraladditionalband-widths,wefailedto?ndanysigni?cantevidenceofanimpactonresearchqualityasmeasuredbytheaveragecitationsperpublica-tion.This?ndingisatoddswiththeresultsobtainedbyChudnovskyetal.(2008)andJacobandLefgren(2011a,b)fortheArgentina’sFONCYTandtheNIHprograms,wherepositiveimpactsoncita-tionswereobtained.However,thisdifferencemaybeduetothefactthatthesetwobenchmarkprogramsprovidedmuchhigherfund-ingperprojectthanFONDECYTdidduringtheperiodweconsider.Therefore,theimpactofthesetwoprogramsoncitationsseemstoreinforcetheconclusionthatthelackofimpactofFONDECYToncitationscouldcomefromtherelativelylowfundingper-project.

576

418

280

BeforediscussinganypolicyrecommendationsitisimportanttoemphasizethatbecauseinternationalfundingandadditionalfundingmechanismsestablishedbytheChileanauthoritieswereavailable,ourresultsmaynotyieldtheimpactofFONDECYTrel-ativetoaclean“nonsupport”situation,buttheimpactrelativetothenext-bestfundingoption.35However,whiletheseadditionalsourcesoffundingwereavailable,theyclearlyfocusedontechno-logicallyappliedormissionorientedresearchandnotonthekindofbasicandcuriositydrivenresearchsupportedbyFONDECYT.So,weexpectthatthedegreeofoverlaporactualcompetitionbetweenFONDECYTandtheseothersourcesoffundingbequitesmall,inparticularduringourevaluationperiod.Inaddition,oneneedstorememberthatthesealternativefundswereavailableforboththetreatedandthecontrolgroupsofresearchers.36

Our?ndingsonthelackofimpactonresearchqualityarecer-tainlyconcerningandcallforacriticalreviewoftheprogram.Itisverylikely,thatthemeageramountoffundingprovidedbythepro-grammayhaveledtotargetthewrongtypeofresearchprojects.Excessiveriskaversionmayhavealsoledtotargetresearchprojectswithlower?xedcosts,lowerriskand,insomecase,lowerorig-inality.Onthetopofthis,theruleofprojectcompletioncouldhavecontributedtoinducetheresearchertosubmitprojectsthatminimizetheriskoffailure,projectsthatcangeneratepublicationvolume,butnotasmuchresearchquality.

TheresultsofthisevaluationsuggestthatanyreviewofFONDE-CYTshouldfocusontheinclusionofspeci?cincentivestofosterthequalityofscienti?cproduction.Thiscouldbedonebyeitherincreasingtheaveragesizeofthegrantprovidedbytheprogramtoa?gureclosertotheinternationalstandards37togetherwithaddinganex-postrewardsystembasedoncitations.

Thisstudyiscertainlyjusta?rststeptowardbetterunderstand-ingoftheimpactsofsciencepolicyindevelopingcountrycontexts.Amorecomprehensiveapproachshouldalsoincludetheimpactsonthescienti?ccapabilitiesofthecountry,inparticularinthepro-gressionoftheresearchers’academiccareers,andontheformationofadvancedhumancapital–e.g.PhDstudents.Furthermore,moreresearchisalsoneededforafullassessmentofthesocialreturnsofaprogramasFONDECYT.Thiswouldrequireconsideringalltheprogram’sbene?ts,includingthepotentialexternalitiesgeneratedbytheresearchprojects,andfullamountofthepublicandprivate

35

Thisproblemcouldbeeliminatedinanexperimentalframework.However,thistypeofdesignisnotcommoninsciencepolicy.36

Anotherreasontobecarefulwiththeinterpretationofourresultsisthatthespeci?ctechniqueadoptedfortheimpactestimationonlycapturestheso-calledLocalAverageTreatmentEffect(LATE).Thismeansthatwearemeasuringtheeffectsoftheprogramonarestrictedgroupofbene?ciariescomparedwitharestrictedgroupofnon-bene?ciariesatthethresholdlevel.37

ChangesinthisdirectionhavealreadybeenintroducedbyCONICYT.

J.M.Benaventeetal./ResearchPolicy41 (2012) 1461–1475

1473

economicresourcesinvestedintheprogram,bothinthesupportedprojectandintheadministrationofthefundingagency.AfutureevaluationofFONDECYTimpactwouldalsobene?tfromcontrol-lingforothercharacteristicsoftheprincipalresearcher,researchteamandresearchprojectandalsoforalternativesourcesoffund-ing.Inthesamedirection,futureextensionsofthisstudycouldfocusontheheterogeneityofimpactfordifferentcategoriesofresearchersandresearchproject.Onlyasigni?cantlyexpandedsetofinformationthattheonecurrentlyavailablewouldallowtheseresearchextensions.

AppendixIII.

PercentagesofPRsbyyearandtreatment(fullsample).

Year

GRANT=1

GRANT=0

AppendixI.

SeeFig.A1.

1988198919901991199219931994199519961997

6.43%9.59.61%9.81.16.95%9.03.88%8.80.72%

14.22%8.15.36.39.63%8.64%9.39%7.58%7.93%7.71%

Source:Authors’elaboration.

AppendixII.

PercentagesofPRsbyscienti?cdisciplineandtreatment(fullsample).a

Scienti?cdiscipline

GRANT=1

GRANT=0

Agronomy/zoologySocialsciencesNaturalsciencesEarthsciences

EngineeringsciencesMedicalsciences

Legal-economic-managementsciencesHumanities

7.67 .990.70%2.91.16.22%6.32%8.01%

8.37'.96.75%3.15.39.59%8.86%7.93%

Source:Authors’elaboration.a

Thesepercentagesbecomemorebalancedbydisciplinewhenrestrictingthesampletosmallerbandwidthsaroundthecut-offpoint.ThisappliesinparticularforthecaseofNaturalSciences.

Distribution of Publications and Citations

Density of Principal ResearchersDensity of Principal Researchers.150.05.1.30.1.20 10 Density20Publications 30 400 20 Density40Citations 60 80 kdensity Publications kdensity Citations Source: Authors’ elaboration

Fig.A1.Distributionofpublicationsandcitations.

1474J.M.Benaventeetal./ResearchPolicy41 (2012) 1461–1475

AppendixIV.

Treatmenteffectsonpublicationsandcitations,negativebinomialestimates.

Publications

(1)

(2)

(3)

(4)

(5)

(6)

Coef?cient

All

h=1.87

h=1.62

h*=1.37

h=1.12

h=0.87

D

D(IV)

Observations

0.5430***(0.140)0.8740***(0.321)3101

0.6239***(0.141)1.1433***(0.356)2634

0.6243***(0.145)1.1063***(0.403)2272

0.6894***(0.146)1.2895***(0.478)1942

0.6573***(0.149)1.5325***(0.590)1632

0.5684***(0.171)1.9369*(1.045)1208(6)h=0.38

CitationsCoef?cientD

D(IV)

Observations

(1)All

?0.1285(0.105)?0.2281(0.333)1261

(2)h=1.38

(3)

(4)

(5)

h=1.13

h*=0.88

h=0.63

?0.0968(0.109)?0.0935(0.556)896

?0.1291(0.110)?0.4056(0.748)777

?0.1480(0.120)?2.1350(1.993)576

?0.0564(0.144)0.9075(1.982)418

?0.1390(0.167)?0.7100(1.026)280

Source:Authors’elaboration.

Robuststandarderrorsinparentheses.*

p<0.10.**

p<0.05.***

p<0.01.

Allcontrolvariablesincluded.“D”isthetreatmentvariable.

AppendixV.

SeeFigs.A2andA3

40Estimated effect on PublicationsEstimated effect on Citations3020-1000-5010050100-10 .19 .38 .63 .88 1.13 1.38CI .87 1.12 1.37 1.62 CI 1.87 2.2 2.6Bandwidth 1.8Est 2.1 2.6Bandwidth Est* The vertical li ne indicate s the “Optimal bandwidth”.

* The vertical li ne indicate s the “Optimal bandwidth”.

Source: Au thors’ el aborati on

Fig.A3.DependenceoftheestimatedeffectsonCitationsonbandwidthselection.

Au thors’ el aborati on Source:

Fig.A2.DependenceoftheestimatedeffectsonPublicationsonbandwidthselec-tion.

.

J.M.Benaventeetal./ResearchPolicy41 (2012) 1461–1475

1475

References

Amin,M.,Mabe,M.,2000.ImpactFactors:UseandAbuse.PerspectivesinPublishing

No.1.

Angrist,J.D.,Lavy,V.,1999.UsingMaimonides’ruletoestimatetheeffectofclass

sizeonscholasticachievement.QuarterlyJournalofEconomics114(2),533–575.

Arora,A.,Gambardella,A.,2005.TheimpactofNSFsupportforbasicresearchin

economics.Annalesd’EconomieetdeStatistiqueNo.79/80,91–117.

Arrow,K.,1962.EconomicsWelfareandtheAllocationofResourcesforInvention.

In:Nelson,R.(Ed.),TheRateandDirectionofInventiveActivity.PrincetonUni-versityPress,pp.164–181.

Benavente,J.M.,Crespi,G.,1996.TheChileannationalsystemofinnovation.Estu-diosdeEconomía,UniversityofChile,DepartmentofEconomics23(2Year19),223–254.

Black,S.,1999.Dobetterschoolsmatter?Parentalvaluationofelementaryeduca-tion.QuarterlyJournalofEconomics114(2),577–599.

Buchmueller,T.,Dominitz,J.,Hansen,L.,1999.Graduatetrainingandtheearlycareer

productivityofPhDeconomists.EconomicsofEducationReview14,65–77.Cameron,B.,2005.TrendsintheUsageofISIBibliometricData:Uses,Abuses,and

Implications.LibrarianandStaffPublications,Paper3.

Chudnovsky,D.,López,A.,Rossi,M.,Ubfal,D.,2008.Moneyforscience?Theimpact

ofresearchgrantsonacademicoutput.FiscalStudies29(1),75–87.

Codner,D.,2011.Alcance,resultadoseimpactosdelFONCYTentre2006y2010.In:

Porta,F.,Lugones,G.(Eds.),InvestigaciónCientí?caeInnovaciónTecnológicaenArgentina:impactodelosfondosdelaagencianacionaldepromocióncientí?caytecnológica.UniversidadNacionaldeQuilmes(Editorial).

Crespi,G.,Geuna,A.,2004.TheProductivityofScience.SPRU,Brighton,Reportpre-paredfortheOf?ceofScienceandTechnology(OST),DepartmentofTradeandIndustry(DTI),UK.

Crespi,G.,Geuna,A.,2005.ModelingandMeasuringScienti?cProduction:Results

foraPanelofOECDCountries.SPRUElectronicWorkingPaperSeries,No.133,TheFreemanCentre,UniversityofSussex,Brighton.

Dasgupta,P.,David,P.,1994.Towardaneweconomicsofscience.ResearchPolicy

23(5),487–521.

Debackere,K.,Gl?nzel,W.,2004.Usingabibliometricapproachtosupportresearch

policydecisions:thecaseoftheFlemishBOF-key.Scientometrics59(2),253–276.

Diamond,A.,1986.Thelife-cycleresearchproductivityofmathematiciansandsci-entists.JournalofGerontology41(4),520–525.

DiNardo,J.,Lee,D.S.,2004.Economicimpactsofnewunionizationonprivatesector

employers:1984–2001.QuarterlyJournalofEconomics119(4),1383–1441.Fuji,D.,Imbens,G.W.,Kalyanaraman,K.,2009.NotesforMatlabandStataRegression

DiscontinuitySoftwareManuscript.HarvardUniversity.

Gambardella,A.,2001.EconomicToolsandMethodologiesfortheDesignofResearch

ProgrammesintheSocio-economicField.Sant’AnnaSchoolofAdvancedStudies,Pisa,Expert’sEvaluationpreparedfortheDGXII.

Gonzalez-Brambila,C.,Veloso,F.M.,2007.Thedeterminantsofresearchoutputand

impact:astudyofMexicanresearchers.ResearchPolicy36(7),1035–1051.Hahn,J.,Todd,P.,VanderKlaauw,W.,2001.Identi?cationandestimationof

treatmenteffectswitharegressiondiscontinuitydesign.Econometrica69(1),201–209.

IDB,2006.IDB’sScienceandTechnologyPrograms:AnEvaluationoftheTechnol-ogyDevelopmentFunds(TDF)andCompetitiveResearchGrants(CRG).Mimeo,WashingtonDC.

Imbens,G.W.,Kalyanaraman,K.,2009.OptimalBandwidthChoicefortheRegression

DiscontinuityEstimator.NBERWorkingPaperNo.14726.

Imbens,G.W.,Wooldridge,J.M.,2009.Recentdevelopmentsintheeconometricsof

programevaluation.JournalofEconomicLiterature47(1),5–86.

Imbens,G.,Lemieux,T.,2008.Regressiondiscontinuitydesigns:aguidetopractice.

JournalofEconometrics142(2),615–635.

Jacob,B.A.,Lefgren,L.,2011a.TheimpactofNIHpostdoctoraltraininggrantson

scienti?cproductivity.ResearchPolicy40(6),864–874.

Jacob,B.A.,Lefgren,L.,2011b.Theimpactofresearchgrantfundingonscienti?c

productivity.JournalofPublicEconomics95(9–10),1168–1177.

Jaffe,A.,2002.Buildingprogramevaluationintothedesignofpublicresearchsup-portprograms.OxfordReviewofEconomicPolicy18(1),22–34.

Lach,S.,2002.DoR&DsubsidiesstimulateordisplaceprivateR&D?Evidencefrom

Israel.JournalofIndustrialEconomics50(4),369–390.

Lalive,R.,2008.Howdoextendedbene?tsaffectunemploymentduration?Aregres-siondiscontinuityapproach.JournalofEconometrics142(2),785–806.

Lee,D.S.,Lemieux,T.,2009.Regressiondiscontinuitydesignsineconomics.Journal

ofEconomicLiterature48(2),281–355.

Long,J.S.(Ed.),2001.FromScarcitytoVisibility:GenderDifferencesintheCareers

ofDoctoralScientistsandEngineers.NationalAcademyPress,Washington,DC(Chapter6:TheAcademicCareer).

Lundvall,B.,1992.NationalSystemsofInnovation.PinterPublishers,London.

McCrary,J.,2008.Manipulationoftherunningvariableintheregressiondisconti-nuitydesign:adensitytest.JournalofEconometrics142(2),698–714.

Mollas-Gallart,J.,Salter,A.,2002.DiversidadyExcelencia:Consideracionessobre

PolíticaCientí?ca.TheIPTSReport,n.66,Julio.

Mohnen,P.,Roller,L.-H.,2005.Complementaritiesininnovationpolicy.European

EconomicReview49(6),1431–1450.

Narin,F.,1976.EvaluativeBibliometrics:TheUseofPublicationandCitationAnalysis

intheEvaluationofScienti?cActivity.ComputerHorizons,CherryHill,NJ.

Nelson,R.,1959.Thesimpleeconomicsofbasicscienti?cresearch.JournalofPolitical

Economy67(3),297–306.

Patel,P.,Pavitt,K.,2000.Nationalsystemsofinnovationunderstrain:theinterna-tionalizationofcorporateR&D.In:Barrell,R.,Mason,G.,O’Mahoney,M.(Eds.),Productivity,InnovationandEconomicPerformance.CambridgeUP.

Pavitt,K.,2001.Publicpoliciestosupportbasicresearch:whatcantherestofthe

worldlearnfromUStheoryandpractice?(Andwhattheyshouldnotlearn).IndustrialandCorporateChange10(3),761–779.

Rosenberg,N.,1992.Scienti?cinstrumentationanduniversityresearch.Research

Policy21(4),381–390.

Salter,A.,Martin,B.,2001.Theeconomicbene?tofpubliclyfundedbasicresearch:

acriticalreview.ResearchPolicy30(3),509–532.

Scherer,F.,Harhoff,D.,2000.Technologypolicyforaworldofskew-distributed

outcomes.ResearchPolicy29(4–5),559–566.

Stephan,P.,1996.Theeconomicsofscience.JournalofEconomicLiterature34(3),

1199–1235.

Stephan,P.,2010.Theeconomicsofscience.In:Hall,B.,Rosemberg,N.(Eds.),Hand-bookontheEconomicsofInnovation.NorthHolland.

Thistlethwaite,D.,Campbell,D.,1960.Regression-discontinuityanalysis:analter-nativetotheexp-postfactoexperiment.JournalofEducationalPsychology51(6),309–317.

Turner,L.,andMairesse,J.,2005.Individualproductivitydifferences

inpublicresearch:Howimportantarenon-individualdetermi-nants?AneconometricstudyofFrenchphysicists’publicationsandcitations(1986-1997).WorkingPaper.http://www.jourdan.ens.fr/piketty/?chiers/semina/lunch/Turner2005.pdf.

Ubfal,D.,Maf?oli,A.,2011.Theimpactoffundingonresearchcollaboration:evi-dencefromadevelopingcountry.ResearchPolicy40(9),1269–1279.

Urquiola,M.,Verhoogen,E.,2009.Class-sizecaps,sorting,andtheregression-discontinuitydesign.TheAmericanEconomicReview99(1),179–215.

VanDerKlaauw,W.,2002.Estimatingtheeffectof?nancialaidoffersoncol-legeenrollment:aregression-discontinuityapproach.InternationalEconomicReview43(4),1249–1287.

本文来源：https://www.bwwdw.com/article/t8jx.html