Nonlinear adaptive control using nonparametric Gaussian process prior models

NONPARAMETRICGAUSSIANPROCESSPRIORMODELS

RoderickMurray-SmithDanielSbarbaro

DepartmentofComputingScience,UniversityofGlasgow,GlasgowG128QQ,Scotland,UK.&HamiltonInstitute,NUIMaynooth,Ireland

E-mail:rod@dcs.gla.ac.uk

´El´DepartamentodeIngenieriaectrica,UniversidaddeConcepci´on,

Chile.E-mail:dsbarbar@die.udec.cl

Abstract:NonparametricGaussianProcesspriormodels,takenfromBayesianstatisticsmethodologyareusedtoimplementanonlinearadaptivecontrollaw.Theexpectedvalueofaquadraticcostfunctionisminimised,withoutignoringthevarianceofthemodelpredictions.Thisleadstoimplicitregularisationofthecontrolsignal(caution),andexcitationofthesystem.Thecontrollerhasdualfeatures,sinceitisbothtrackingareferencesignalandlearningamodelofthesystemfromobservedresponses.Thegeneralmethodanditsmainfeaturesareillustratedonasimulationexample.Copyrightc2002IFAC.

Keywords:Gaussianprocesspriors,nonparametricmodels,dualcontrol,nonlinearmodel-basedpredictivecontrol.

1.INTRODUCTION

Manyauthorshaveproposedtheuseofnon-linearmodelsasabasetobuildnonlinearadaptivecon-trollers.Inmanyapplications,however,thesenon-linearitiesarenotknown,andnon-linearparameter-isionmustbeusedinstead.Apopularchoicehasbeentheuseofartificialneuralnetworksforesti-matingthenonlinearitiesofthesystem(NarendraandParthasarathy,1990;LiangandElMargahy,1994;ChenandKhalil,1995;BittantiandPiroddi,1997).Alltheseworkshaveadoptedforcontrollerdesign,thecertaintyequivalenceprinciple,wherethemodelisusedinthecontrollaw,asifitwerethetruesys-tem.Inordertoimprovetheperformanceofnonlinearadaptivecontrollersbasedonanonlinearmodels,theaccuracyofthemodelpredictionsshouldalsobetakenintoaccount.

Mostengineeringapplicationsarestillbasedonpara-metricmodels,wherethefunctionalformisfullyde-scribedbyafinitenumberofparameters,oftenalinearfunction.Eveninthecaseswhereflexibleparamet-ricmodelsareused,suchasneuralnetworks,spline-

basedmodels,multiplemodelsetc,theuncertaintyisusuallyexpressedasuncertaintyofparameters(eventhoughtheparametersoftenhavenophysicalinterpre-tation),anddonottakeintoaccountuncertaintyaboutmodelstructure,ordistanceofcurrentpredictionpointfromtrainingdatausedtoestimateparameters.ThispaperdescribesanapproachbasedonGaussianpro-cesspriors,asanexampleofanon-parametricmodelwithusefulanalyticproperties.Theseallowustoan-alyticallyobtainacontrollawwhichperfectlymin-imisestheexpectedvalueofaquadraticcostfunc-tion,whichdoesnotdisregardthevarianceofthemodelpredictionasanelementtobeminimised.Thisleadsnaturally,andautomaticallytoasuitablecom-binationofregularisingcautionincontrolbehaviourinfollowingthereferencetrajectory,andexcitationofcontroleffort,dependingonmodelaccuracy.Theaboveideasarecloselyrelatedtotheworkdoneondualadaptivecontrol,wherethemainefforthasbeenconcentratedontheanalysisanddesignofadaptivecontrollersbasedontheuseoftheuncertaintyasso-ciatedtoparametersofmodelswithfixedstructure(Wittenmark,1995;FilatovandUnbehauen,2000).

2.NON-PARAMETRICMODELSAND

UNCERTAINTYPREDICTIONParametricmodelscommonlyusedincontrolcontextsassumethatthesystem’sfunctionalformcanbede-scribedbyafinitesetofparameters.Non-parametricmodels(called‘smoothing’insomeframeworks)donotconstrainthemodeltoapre-specifiedfunctionalform,anddonotprojecttheobserveddatadowntoafiniteparameterisation.Theweakerpriorassumptionsandflexibleformtypicallyappliedinanon-parametricmodelmaketheapproachwell-suitedforinitialdataanalysisandexploration.

Asignificantadvantagewhenmodellingnonlineardy-namicsystemsisthatnonparametricapproachesretaintheavailabledataandperforminferenceconditionalonthecurrentstateandlocaldata.Thisisadvanta-geousinoff-equilibriumregions,sincenormallyintheseregionstheamountofdataavailableforiden-tificationismuchsmallerthanthatavailableclosetoequilibriumconditions.Theuncertaintyofmodelpredictionscanbemadedependentonlocaldataden-sity,andthelocalmodelcomplexityautomaticallyrelatedtotheamountofavailabledata(morecomplexmodelsneedmoreevidencetomakethemlikely).Bothaspectsareveryusefulwhenmodellingsparsely-populatedtransientregimesindynamicsystems,andwillhavesignificanteffectsoncontrolbehaviour.

3.GAUSSIANPROCESSPRIORS

Anexampleoftheuseofaflexiblenon-parametricmodelwithuncertaintypredictionsisaGaussianPro-cessprior,introducedin(O’Hagan,1978)andre-centlyreviewedin(Williams,1998).UseofGPsinacontrolsystemscontextisdiscussedin(Murray-Smithetal.,1999;Leithetal.,2000).AvariationwhichcanincludeARMAnoisemodelsisdescribedin(Murray-SmithandGirard,2001).

Inthefollowing,thefullmatrixofstateandcontrolinputvectorsisdenotedby,andthevectorofoutputpointsis.Thediscretedataoftheregressionmodelareand.Intheexampleusedinthispaper,,and

.Thegivendata

pairsusedforidentificationarestackedinmatrices

andthedatapairsusedforpredictionare.Insteadofparameterisingasaparametricmodel,wecanplaceapriordirectlyonthespaceoffunctionswhereisassumedtobelong.AGaussianprocessrepresentsthesimplestformofprioroverfunctions–weassumethatanypointshavea-dimensionalmultivariateNormaldistribution.Wewillassumezeromean,soforthecasewithpartitioneddataandwewillhavethemultivariateNormaldistribution(wewillassumezeromean),

(1)

whereisthefullcovariancematrixforthetraininginputs,andwhereforthecross-covariancebetweenand.LiketheGaussiandistribution,theGaussianProcessisfullyspecifiedbyameananditscovariancefunction.Thecovariancefunctionexpressestheexpectedcovariancebetweenand–wecanthereforeinfer’sfromconstantgiven’sratherthanbuildingexplicitparametricmodels.

Asinthemultinormalcase,wecandividethejointprobabilityintoamarginalGaussianprocessandaconditionalGaussianprocess.Themarginaltermgivesusthelikelihoodofthetrainingdata,

where,asinthestraightforwardmultinormalcasedescribedearlier,

(3)

(4)

sowecanuse

astheexpectedmodel

output,withavarianceof

.

3.1Thecovariancefunction

Themultivariatenormalassumptionmayseemre-strictive,butwecanactuallymodelawiderangeoffunctions,dependingonthechoiceofcovariancefunc-tion.Themodel’sprioroverpossiblefunctionscanbeadaptedtoagivenapplicationbyalteringthestructureorparametersofthecovariancefunction.Thechoiceoffunctionisonlyconstrainedinthatitmustalwaysgenerateanon-negativedefinitecovariancematrixforanyinputs,sowecanrepresentaspectrumofsys-temsfromverynonlinearmodels,tostandardlinearmodelsusingthesameframework.Thecovariancefunctionwillalsooftenbeviewedasbeingthecombi-nationofacovariancefunctionduetotheunderlyingmodelandoneduetomeasurementnoise.Theentriesofthismatrixarethen:

(5)

wherecouldbe,whichwouldbe

addinganoisemodel

tothediagonalentriesof.

Wediscusscovariancefunctionsforcorrelatednoisemodelsin(Murray-SmithandGirard,2001).Inthispaper,weuseastraightforwardcovariancefunction,

sothattheparametervector.

isadistancemeasure,whichshouldbeoneatandwhichshouldbeamonotonicallydecreasingfunctionof.Theoneusedherewas

Withmostmodels,estimationofVar

Var

(10)

,or

VarVar

4.1Example:linearinparametersGP

Asanintroductoryexample,ifwehaveaGPtorepresentalinearmodel,foratargetsystemoftheform

where,and,whereinthiscaseweusethesimplelinearbasis

,andisadiagonalmatrixwith

thevectoraselements

Var

(11)

where,

fornotationalconvenience.

(SISO).

Var

At

where

derivative:

and

.Takingthepartial

.At

asinabovebutweusedifferentvaluesofthehyperparameterstothoseusedin.

5.SIMULATIONRESULTS

Toillustratethefeasibilityoftheapproachweusedittocontrolasimplesystembasedonnoisyobservedre-sponses.Westartoffwithonlytwotrainingpoints,andaddsubsequentdatatothemodelduringoperation.Themodelhashadnoprioradaptationtothesystembeforetheexperiment,andthecovariancefunctionschosenareverygeneral.Modelhyperparametersareadaptedaftereachiterationusingaconjugatedescentalgorithm.ThepriorityhereistoshowthattheGPmodelsarecapableofadaptivenonlinearcontrol,evenwhennopriorshavebeenplacedontheirhyperpa-rameters.AmorecompleteBayesianapproachwouldbetousethefullpriorstructureinO’Hagan(1978),whichwouldalsoleadtoincreasedrobustnessandhigherperformanceintheearlystagesofadaptation.Letthenon-linearsystembe:

where

,

and,subjecttonoisewithvariance

(FabriandKadirkamanathan,1998).

NotehowinFigure2islargeintheearlystagesoflearning,butdecreasingwiththedecreaseinvariance,showinghowtheregularisingeffectenforcescautioninthefaceofmodeluncertainty,butreducescautionasthemodelaccuracyincreases.isalsolargerintheearlystagesoflearning,essentiallyaddinganexcita-torycomponent.Weexpectwillplayamoresig-nificantrolewhenthecovarianceofthecontrolsignaldependsonthestate.Figure3showsthedevelopmentinthemeanmappingfromtoasthesystemacquiresdata.

6.CONCLUSIONS

Thisworkhaspresentedanoveladaptivecontrollerbasedonnon-parametricmodels.Thecontroldesignisbasedontheexpectedvalueofaquadraticcostfunction,leadingtoacontrollerthatnotonlywillminimisethesquareddifferencebetweenthereferencesignalandtheexpectedvalueoftheoutput,butwillalsotrytominimisethevarianceoftheoutput.Thisleadstoarobustcontrolactionduringadaptation.Simulationresults,consideringlinearandnon-linearsystems,demonstratetheinterestingcharacteristicsofthistypeofadaptivecontrolalgorithm.

GP’shavebeensuccessfullyadoptedfromtheirstatis-ticsoriginsbytheneuralnetworkcommunity(Williams,1998).ThispaperisintendedtobringtheGPapproachtotheattentionofthecontrolcommunity,andtoshowthatthebasicapproachisacompetitiveapproachfor

Nonlinear GP model, Nonlinear IIR system5Y Yd 4µ µ+2σµ−2σ3u 210−1−2−3−4020406080100120140160180200(a)SimulationofnonlinearGP-basedcontrollerα0.50−0.5−1−1.5020406080100120140160180200β10110010−110−210−3020406080100120140160180200(b)and(regularisationterm)Fig.2.Simulationresultsshowingmodellingaccu-racy,controlsignals,trackingbehaviourandlev-elsofandateachstage.

modellingandcontrolofnonlineardynamicsystems,evenwhenlittleattempthasbeenmadetoanalysethedesigner’spriorknowledgeofthesystem–thereismuchmorethatcanbetakenfromtheBayesianapproachtouseinthedualcontrolandnonlinearcontrolareas,e.g.useoftheframeworksuggestedin(O’Hagan,1978)foroptimalexperimentdesignwithGaussianprocesspriorscouldimproveexploration.TheGaussianProcessapproachtoregressionissimpleandelegant,andcanmodelnonlinearproblemsinaprobabilisticframework.Thedisadvantageisitscom-putationalcomplexity,asestimatingthemean

requiresamatrixinversionoftheco-variancematrix,whichbecomesproblematicforiden-tificationdatawhere.Intransientregimes,however,wehaveveryfewdatapointsandwewishtomakerobustestimatesofmodelbehaviour,whicharenowpossible.TherobustinferenceoftheGPap-proachinsparselypopulatedspacesmakesitpartic-82)1+0t(y−2−4−6−83213201−10−2−1−2u(t)−3−3x(t)(a)Truemapping

54.54433.523)1+1)2.51+(y(σ201.5−11−20.5−3033213232120101−10−10−2−1−2−2−1−2u(t)−3−3x(t)u(t)−3−3x(t)(b)At2datapoints(c)

80.760.0.52)1)0.41+0+(y(σ−20.3−40.2−60.1−8033213221320101−10−10−2−1−2−2−1−2u(t)−3−3x(t)u(t)−3−3x(t)(d)At20datapoints(e)

810.820.6)1)1+0+(y(σ−20.4−40.2−6−8033213232120101−100−2−1−1−1−2−2−2u(t)−3−3−3x(t)u(t)−3x(t)(f)At100datapoints(g)

Fig.3.Meansurface(mesh)

overthespace,withuncer-

taintysurfaces.

ularlypromisinginmultivariableandhigh-ordersys-tems.Furtherworkisunderwaytoaddressthecontrolofmultivariablesystems,nonminimum-phasesystemsandimplementationefficiencyissues.

7.ACKNOWLEDGEMENTS

BothauthorsaregratefulforsupportfromFONDE-CYTProject700397.RM-SgratefullyacknowledgesthesupportoftheMulti-AgentControlResearch

TrainingNetworksupportedbyECTMRgrantHPRN-CT-1999-00107,andtheEPSRCgrantModernsta-tisticalapproachestooff-equilibriummodellingfornonlinearsystemcontrolGR/M76379/01.

8.REFERENCES

Bittanti,S.andL.Piroddi(1997).Neuralimplemen-tationofGMVcontrolshemesbasedonaffineinput/outputmodels.Proc.IEEControlTheoryAppl.(6),521–530.

Chen,F.C.andH.Khalil(1995).Adaptivecontrolof

aclassofnonlineardiscrete-timesystems.IEEE.Trans.AutomaticControl40(5),791–801.

Fabri,S.andV.Kadirkamanathan(1998).Dualadap-tivecontrolofstochasticsystemsusingneuralnetworks.Automatica14(2),245–253.

Filatov,N.M.andH.Unbehauen(2000).Surveyof

adaptivedualcontrolmethods.Proc.IEEControlTheoryAppl.(1),119–128.

Leith,D.J.,R.Murray-SmithandW.E.Leit-head(2000).Nonlinearstructureidentification:AGaussianProcessprior/Velocity-basedapproach.In:Control2000,Cambridge.

Liang,F.andH.A.ElMargahy(1994).Self-tuning

neurocontrolofnonlinearsystemsusinglocal-izedpolynomialnetworkswithCLIcells.In:ProceedingsoftheAmericanControlConfer-ence.Baltimore,Maryland.pp.2148–2152.

Murray-Smith,R.andA.Girard(2001).Gaussian

ProcesspriorswithARMAnoisemodels.In:IrishSignalsandSystemsConference,Maynooth.pp.147–152.

Murray-Smith,R.,T.A.JohansenandR.Shorten

(1999).Ontransientdynamics,off-equilibriumbehaviourandidentificationinblendedmultiplemodelstructures.In:EuropeanControlConfer-ence,Karlsruhe,1999.pp.BA–14.

Narendra,K.S.andP.Parthasarathy(1990).Identifi-cationandcontrolofdynamicalsystemsusingneuralnetworks.IEEE.Trans.NeuralNetworks1(1),4–27.

O’Hagan,A.(1978).Oncurvefittingandoptimalde-signforregression.JournaloftheRoyalStatisti-calSocietyB40,1–42.

O’Hagan,A.(1992).SomeBayesiannumericalanal-ysis.In:BayesianStatistics4(J.M.Bernardo,J.O.Berger,A.P.DawidandA.F.M.Smith,Eds.).pp.345–363.OxfordUniversityPress.Williams,C.K.I.(1998).PredictionwithGaussian

processes:Fromlinearregressiontolinearpre-dictionandbeyond.In:LearningandInferenceinGraphicalModels(M.I.Jordan,Ed.).pp.599–621.Kluwer.

Wittenmark,R.(1995).Adaptivedualcontrolmeth-ods:Anoverview.In:Proceedingsofthe5thIFACSymposiumonAdaptivesystemsinControlandSignalProcessing.Budapest.pp.67–92.