Dj3P dZddlZddlmZddlmZdZdZdZ dZ d Z d Z d Z d Zd ZdZdZedk(rdZerdZej*dZej.j1ddZeddddfZej6eeeedddfzzZgdZee eeeeej6eez dddfezZeeeeeej6eede eeej6eez ddlm Z m!Z!dZ"dZ#dZ$dZ%dZ&dZ'eredee!jPe"dddd d!ed"edddeded#edddedee!jPe"d$d%dd&d!ed#ed$dded'ee!jPe"d$d%dd(d!ed#ed$dded'ee!jPe"d$d%dd)d!ed#ed$dded'ee!jPe#dd%dd*d!ee!jPe$dd%dd+d!d,\ZZ)Z*Z+ed#eee)e*ed'ee!jPe#e)d%dee*e+fd!ee!jPe$e*d%dee)e+fd!ed-ee!jPe%dddd.d!eeddde d/\ZZ)Z*ed#eee)e*e ee!jPe&e)d%dee*fd!ee!jPe'e*d%dee)fd!d0\ZZ)Z*ed#eee)e*e ee!jPe&e)d%dee*fd!ee!jPe'e*d%dee)fd!ed1ee dejXddgd2e"dddd2d3ee dejXddgde"ddddd3eedejXddgde"ddddee dejXdd4gde"d5dd5deedejXddgd2e"dddd2ee dejXddgde"ddd5ejZd6zddd7l.m/Z/gd8Zej`d9d:d;Z1eD]b\Z)Z*e!jPe&e)d%de1e*fd!Z2e!jPe'e*d%de1e)fd!Z3ee1e)e*e Z4e/e2e4dde/e3e4dddd@D]Z+eD]\Z)Z*e!jPe#e)d%de1e*e+fd!Z2e!jPe$e*d%de1e)e+fd!Z3ee1e)e*ee+Z4e/e2e4ddAd=>e/e3e4ddAd?>e/ee1ejXe)e*dzge+e"e1e)e*e+de/e e1ejXe)e*dzge+e"e1e)e*ejZe+d:z e+z ze+dyy)Cagradient/Jacobian of normal and t loglikelihood use chain rule normal derivative wrt mu, sigma and beta new version: loc-scale distributions, derivative wrt loc, scale also includes "standardized" t distribution (for use in GARCH) TODO: * use sympy for derivative of loglike wrt shape parameters it works for df of t distribution dlog(gamma(a))da = polygamma(0,a) check polygamma is available in scipy.special * get loc-scale example to work with mean = X*b * write some full unit test examples A: josef-pktd N)special)gammalnc|j\}}dtjdtjztj|z||z dz|z zz}|S)aynormal loglikelihood given observations and mean mu and variance sigma2 Parameters ---------- y : ndarray, 1d normally distributed random variable params : ndarray, (nobs, 2) array of mean, variance (mu, sigma2) with observations in rows Returns ------- lls : ndarray contribution to loglikelihood for each observation g)Tnplogpi)yparamsmusigma2llss hC:\Crop_Prediction\Backend\crop-ai-system\venv\Lib\site-packages\statsmodels/sandbox/regression/tools.pynorm_llsrsO JB qw"&&.0AbD19V3CC DC Jc|j\}}||z |z }||z dz|z dz tj|z }tj||fS)aBJacobian of normal loglikelihood wrt mean mu and variance sigma2 Parameters ---------- y : ndarray, 1d normally distributed random variable params : ndarray, (nobs, 2) array of mean, variance (mu, sigma2) with observations in rows Returns ------- grad : array (nobs, 2) derivative of loglikelihood for each observation wrt mean in first column, and wrt variance in second column Notes ----- this is actually the derivative wrt sigma not sigma**2, but evaluated with parameter sigma2 = sigma**2 r)rrsqrt column_stack)r r r rdllsdmu dllsdsigma2s r norm_lls_gradr/sV,JBtVmGbD19V#a'8K ??G[1 22rc|S)z-gradient/Jacobian for d (x*beta)/ d beta )xbetas r mean_gradrKs  Hrc>|dd}|dtjt|dfz}t||}tj||}tj ||f}t ||}tj |ddddf|z|ddddff} | S)aJacobian of normal loglikelihood wrt mean mu and variance sigma2 Parameters ---------- y : ndarray, 1d normally distributed random variable with mean x*beta, and variance sigma2 x : ndarray, 2d explanatory variables, observation in rows, variables in columns params : array_like, (nvars + 1) array of coefficients and variance (beta, sigma2) Returns ------- grad : array (nobs, 2) derivative of loglikelihood for each observation wrt mean in first column, and wrt scale (sigma) in second column assume params = (beta, sigma2) Notes ----- TODO: for heteroscedasticity need sigma to be a 1d array Nr)roneslenrdotrr) r rr rrdmudbetar params2dllsdmsgrads rnormgradr(Ps0 #2;D BZQ + +FD!H 4Boor&k*GAg&G ??GAbqbDM(2GAbqbDMB CD Krc`|j\}}|dz}t|dzdz t|dz z dtj|dz tjzzz }||dzdz tjd||z dz|dz z |z zzdtj|zzz}|S)at loglikelihood given observations and mean mu and variance sigma2 = 1 Parameters ---------- y : ndarray, 1d normally distributed random variable params : ndarray, (nobs, 2) array of mean, variance (mu, sigma2) with observations in rows df : int degrees of freedom of the t distribution Returns ------- lls : ndarray contribution to loglikelihood for each observation Notes ----- parametrized for garch ?r@?r)rrrr r r r dfr rrs rtstd_llsr/ts,JB CB 2a4) wr"u~ -BFFBqD"%%<4H0H HCBqD"9rvvbAbD19bd#3F#::; ;cBFF6N>R RRC Jrc| S)z?derivative of log pdf of standard normal with respect to y r)r s r norm_dlldyr1s  2IrcLtj|dz}tjtj|dzdz tj|dz z tj |dz tj zz }|d|dz|dz z z|dzdz zz}|S)zIpdf for standardized (not standard) t distribution, variance is one r*rr+r)rarrayexprrrr )rr.rPxs rtstd_pdfr7s CA 1b)'//!B$*?? @!A#ruuAU UB1adQqS\>ac2X &&B Ircnt||||j\}}|dz}t|dzdz t|dz z dtj|tj zzz }||dzdz tjd||z dz|z |z zzdtj|zzz}|S)at loglikelihood given observations and mean mu and variance sigma2 = 1 Parameters ---------- y : ndarray, 1d normally distributed random variable params : ndarray, (nobs, 2) array of mean, variance (mu, sigma2) with observations in rows df : int degrees of freedom of the t distribution Returns ------- lls : ndarray contribution to loglikelihood for each observation Notes ----- parametrized for garch normalized/rescaled so that sigma2 is the variance >>> df = 10; sigma = 1. >>> stats.t.stats(df, loc=0., scale=sigma.*np.sqrt((df-2.)/df)) (array(0.0), array(1.0)) >>> sigma = np.sqrt(2.) >>> stats.t.stats(df, loc=0., scale=sigma*np.sqrt((df-2.)/df)) (array(0.0), array(2.0)) r*rr+r,r)printrrrr r r-s rts_llsr:s: !VRJB CB 2a4) wr"u~ -BFFB:4F0F FCBrE2:rQrTAIrN6$99: :S266&>=Q QQC Jrc<|dz}|dz |z d|dz|z zz |zS)aderivative of log pdf of standard t with respect to y Parameters ---------- y : array_like data points of random variable at which loglike is evaluated df : array_like degrees of freedom,shape parameters of log-likelihood function of t distribution Returns ------- dlldy : ndarray derivative of loglikelihood wrt random variable y evaluated at the points given in y Notes ----- with mean 0 and scale 1, but variance is df/(df-2) r*rrrr r.s rts_dlldyr=s5, BBT7B<1q!tRy= )A --rc>|dz |dz z d|dz|dz z zz |zS)aderivative of log pdf of standardized t with respect to y Parameters ---------- y : array_like data points of random variable at which loglike is evaluated df : array_like degrees of freedom,shape parameters of log-likelihood function of t distribution Returns ------- dlldy : ndarray derivative of loglikelihood wrt random variable y evaluated at the points given in y Notes ----- parametrized for garch, standardized to variance=1 rr+rrr<s r tstd_dlldyr?s3.T7BrE?a!Q$2,. /! 33rcj||z |z }||g| |z }d|z ||g|||z z|dzz z }||fS)aderivative of log-likelihood with respect to location and scale Parameters ---------- y : array_like data points of random variable at which loglike is evaluated loc : float location parameter of distribution scale : float scale parameter of distribution dlldy : function derivative of loglikelihood fuction wrt. random variable x args : array_like shape parameters of log-likelihood function Returns ------- dlldloc : ndarray derivative of loglikelihood wrt location evaluated at the points given in y dlldscale : ndarray derivative of loglikelihood wrt scale evaluated at the points given in y grr)r locscaledlldyargsystdlldloc dlldscales r locscale_gradrHs]4 S5%-CS 4  5(GE E#--37q@@I I r__main__g?r r)rrr)statsmisccntjtjj ||||SN)rArBrr rLtpdf)r rArBr.s rlltrS2&vveggkk!RSk>??rcntjtjj ||||SrOrP)rAr rBr.s rlltlocrV4rTrcntjtjj ||||SrOrP)rBr rAr.s rlltscalerX6rTrcltjtjj |||SrOrr rLnormrR)r rArBs rllnormr\9$vvejjnnQCun=>>rcltjtjj |||SrOrZ)rAr rBs r llnormlocr_;r]rcltjtjj |||SrOrZ)rBr rAs r llnormscalera=r]rz gradient of tgư>)rrrJ)dxnrDorderzt ts?g|=)rrrg)rrrg)rrrg)rfrrg)rfrrg)rfrrrgz gradient of normrr)rfrr)rfrrz loglike of tdzdifferently standardizedg?r*g?)assert_almost_equal)rh)r*r*)gr+)r*r+gr+ z deriv loc)err_msgz deriv scale)rKrJriloglike)5__doc__numpyrscipyr scipy.specialrrrrr(r/r1r7r:r=r?rH__name__verbosesigr!rrandomrandnrvsrr#r r r9 dllfdbetarLrMrSrVrXr\r_ra derivativerArBr.r3r numpy.testingrjlinspaceytdlldlodlldscgrrrrrs*!(38 H> $N.644> zGrwwqziioob# !"I BFF1TNSQqS\ ) hq!V$%vrvva&$/1  i mAvrvva:>? aq$ !@@@???   odooc1KL dM!Q:r:; dM!Q8R89 odooc35AHANP dM#q!Xr:; odooc35AHANP dM#q!Xr:; odooc35AHANP dM#q!Xr:; odoofaEQZqQS odooheqzQRST&#eB dM!Cx<= odoofceq%|STUW odooh%1Ac":UVWX "# odoofaDAEKL mAq!Z01 #e dM!Cz:; odooi!1U)STUW odook5UaqgUVWX #e dM!Cz:; odooi!1U)STUW odook5UaqgUVWX  hq("((Aa5/3/Qq3A[\ hq("((Aa5/2.Aa" ?YZ fQ!A,c!Aam< hq("((Ai=126BqBH fQ!A-s1Qq~> hq("((Aa5/2.Aa72775>8I"0MN2 1F SB BE E CEQbZWXY e"SYZ[ 2sE: 6FBqE1kBFBqE1mD E  3 3IC$T__VSUar%PRm[\]F$T__Xu!2cRT+]^_Fr3x