DjgldZddlZddlmZddlmZmZmZm Z m Z gdZ GddeZ Gd d eZ y) aZ Multivariate Conditional and Unconditional Kernel Density Estimation with Mixed Data Types. References ---------- [1] Racine, J., Li, Q. Nonparametric econometrics: theory and practice. Princeton University Press. (2007) [2] Racine, Jeff. "Nonparametric Econometrics: A Primer," Foundation and Trends in Econometrics: Vol 3: No 1, pp1-88. (2008) http://dx.doi.org/10.1561/0800000009 [3] Racine, J., Li, Q. "Nonparametric Estimation of Distributions with Categorical and Continuous Data." Working Paper. (2000) [4] Racine, J. Li, Q. "Kernel Estimation of Multivariate Conditional Distributions Annals of Economics and Finance 5, 211-235 (2004) [5] Liu, R., Yang, L. "Kernel estimation of multivariate cumulative distribution function." Journal of Nonparametric Statistics (2008) [6] Li, R., Ju, G. "Nonparametric Estimation of Multivariate CDF with Categorical and Continuous Data." Working Paper [7] Li, Q., Racine, J. "Cross-validated local linear nonparametric regression" Statistica Sinica 14(2004), pp. 485-512 [8] Racine, J.: "Consistent Significance Testing for Nonparametric Regression" Journal of Business & Economics Statistics [9] Racine, J., Hart, J., Li, Q., "Testing the Significance of Categorical Predictor Variables in Nonparametric Regression Models", 2006, Econometric Reviews 25, 523-544 N)kernels) GenericKDEEstimatorSettingsgpke LeaveOneOut _adjust_shape)KDEMultivariateKDEMultivariateConditionalrcFeZdZdZd dZdZdfdZd dZd dZd Z d Z y) r a Multivariate kernel density estimator. This density estimator can handle univariate as well as multivariate data, including mixed continuous / ordered discrete / unordered discrete data. It also provides cross-validated bandwidth selection methods (least squares, maximum likelihood). Parameters ---------- data : list of ndarrays or 2-D ndarray The training data for the Kernel Density Estimation, used to determine the bandwidth(s). If a 2-D array, should be of shape (num_observations, num_variables). If a list, each list element is a separate observation. var_type : str The type of the variables: - c : continuous - u : unordered (discrete) - o : ordered (discrete) The string should contain a type specifier for each variable, so for example ``var_type='ccuo'``. bw : array_like or str, optional If an array, it is a fixed user-specified bandwidth. If a string, should be one of: - normal_reference: normal reference rule of thumb (default) - cv_ml: cross validation maximum likelihood - cv_ls: cross validation least squares defaults : EstimatorSettings instance, optional The default values for (efficient) bandwidth estimation. Attributes ---------- bw : array_like The bandwidth parameters. See Also -------- KDEMultivariateConditional Examples -------- >>> import statsmodels.api as sm >>> nobs = 300 >>> np.random.seed(1234) # Seed random generator >>> c1 = np.random.normal(size=(nobs,1)) >>> c2 = np.random.normal(2, 1, size=(nobs,1)) Estimate a bivariate distribution and display the bandwidth found: >>> dens_u = sm.nonparametric.KDEMultivariate(data=[c1,c2], ... var_type='cc', bw='normal_reference') >>> dens_u.bw array([ 0.39967419, 0.38423292]) Nc||_t|j|_t||j|_||_t j|j\|_|_|j|jkr td| tn|}|j||js|j||_y|j||_y)NzGThe number of observations must be larger than the number of variables.)var_typelenk_varsr data data_typenpshapenobs ValueErrorr _set_defaults efficient _compute_bwbw_compute_efficient)selfrrrdefaultss lC:\Crop_Prediction\Backend\crop-ai-system\venv\Lib\site-packages\statsmodels/nonparametric/kernel_density.py__init__zKDEMultivariate.__init__es  $--( !$ 4 !!#$))!4 4; 99 #=> >*2*:$& 8$~~&&r*DG--b1DGcd}|dt|jzdzz }|dt|jzdzz }|d|jzdzz }|d|jzdzz }|S) Provide something sane to print.z KDE instance zNumber of variables: k_vars =  zNumber of samples: nobs = zVariable types: BW selection method: )strrrr _bw_methodrrprs r__repr__zKDEMultivariate.__repr__usy /#dkk2BBTII -DII>EE &6== &84?? r c|SNxs rzKDEMultivariate.~r c t|j}d}t|D]<\}}t|| |j|ddf |j}|||z }>| S)aX Returns the leave-one-out likelihood function. The leave-one-out likelihood function for the unconditional KDE. Parameters ---------- bw : array_like The value for the bandwidth parameter(s). func : callable, optional Function to transform the likelihood values (before summing); for the log likelihood, use ``func=np.log``. Default is ``f(x) = x``. Notes ----- The leave-one-out kernel estimator of :math:`f_{-i}` is: .. math:: f_{-i}(X_{i})=\frac{1}{(n-1)h} \sum_{j=1,j\neq i}K_{h}(X_{i},X_{j}) where :math:`K_{h}` represents the generalized product kernel estimator: .. math:: K_{h}(X_{i},X_{j}) = \prod_{s=1}^{q}h_{s}^{-1}k\left(\frac{X_{is}-X_{js}}{h_{s}}\right) rNr data_predictr)rr enumeraterr)rrfuncLOOLiX_not_if_is rloo_likelihoodzKDEMultivariate.loo_likelihood~sl6$))$ #C. JAwr !Q$7G $ /C cNA  r r c j| |j}nt||j}g}tt j |dD]R}|j t|j|j||ddf|j|jz Tt j|}|S)aT Evaluate the probability density function. Parameters ---------- data_predict : array_like, optional Points to evaluate at. If unspecified, the training data is used. Returns ------- pdf_est : array_like Probability density function evaluated at `data_predict`. Notes ----- The probability density is given by the generalized product kernel estimator: .. math:: K_{h}(X_{i},X_{j}) = \prod_{s=1}^{q}h_{s}^{-1}k\left(\frac{X_{is}-X_{js}}{h_{s}}\right) Nrr2 rr rrangerrappendrrrrsqueeze)rr3pdf_estr8s rpdfzKDEMultivariate.pdfs,  99L(t{{CLrxx -a01 EA NN4dii-9!Q$-?)-8:>))D E E **W%r c p| |j}nt||j}g}tt j |dD]U}|j t|j|j||ddf|jddd|jz Wt j|}|S)a Evaluate the cumulative distribution function. Parameters ---------- data_predict : array_like, optional Points to evaluate at. If unspecified, the training data is used. Returns ------- cdf_est : array_like The estimate of the cdf. Notes ----- See https://en.wikipedia.org/wiki/Cumulative_distribution_function For more details on the estimation see Ref. [5] in module docstring. The multivariate CDF for mixed data (continuous and ordered/unordered discrete) is estimated by: .. math:: F(x^{c},x^{d})=n^{-1}\sum_{i=1}^{n}\left[G(\frac{x^{c}-X_{i}}{h})\sum_{u\leq x^{d}}L(X_{i}^{d},x_{i}^{d}, \lambda)\right] where G() is the product kernel CDF estimator for the continuous and L() for the discrete variables. Used bandwidth is ``self.bw``. Nr gaussian_cdfaitchisonaitken_cdf wangryzin_cdf)rr3rckertypeukertypeokertyper=)rr3cdf_estr8s rcdfzKDEMultivariate.cdfs>  99L(t{{CLrxx -a01 GA NN4dii-9!Q$-?)-)7)>)8 : =AII F G G**W%r c zd}ttjtjtj}|j }|j }|j}tj|Dcgc]}|dk( c}}||j} tj|j} t|D]d} t|D](\} } || || |dd| f|| | f| dd| f<*| jd| z }|jd}||z }fttj tj"tj$}t'|j }d}tj|jddz |jdf} t|D]f\} }t|D])\} } || || |dd| f || | f| dd| f<+| jd| z }||jdz }h||dzz d|z||dz zz z Scc}w)a Returns the Integrated Mean Square Error for the unconditional KDE. Parameters ---------- bw : array_like The bandwidth parameter(s). Returns ------- CV : float The cross-validation objective function. Notes ----- See p. 27 in [1]_ for details on how to handle the multivariate estimation with mixed data types see p.6 in [2]_. The formula for the cross-validation objective function is: .. math:: CV=\frac{1}{n^{2}}\sum_{i=1}^{n}\sum_{j=1}^{N} \bar{K}_{h}(X_{i},X_{j})-\frac{2}{n(n-1)}\sum_{i=1}^{n} \sum_{j=1,j\neq i}^{N}K_{h}(X_{i},X_{j}) Where :math:`\bar{K}_{h}` is the multivariate product convolution kernel (consult [2]_ for mixed data types). References ---------- .. [1] Racine, J., Li, Q. Nonparametric econometrics: theory and practice. Princeton University Press. (2007) .. [2] Racine, J., Li, Q. "Nonparametric Estimation of Distributions with Categorical and Continuous Data." Working Paper. (2000) r)courMNraxis)dictrgaussian_convolutionwang_ryzin_convolutionaitchison_aitken_convolutionrrrrarrayprodemptyrr>r4sumgaussian wang_ryzinaitchison_aitkenr)rrFkertypesrrrrMix_cont_bw_cont_productKvalr8iivtypedens k_bar_sumr6r7r9s rimsezKDEMultivariate.imses@f '66!88!>>@yy z==((h7AH78g;++-xx #t A&x0 ; E-huobf.21b5k.21b5k;QU  ; 99!9$'77Da(I NA '**!,,!224$))$ xxAq$**Q-89#C. "JAw&x0 ; E-huobf/6q"u~o.21b5k;QU  ;99!9$'77D q! !A  "D!G a!ettax'899:98s3 H8c(d}|jf}||fS)@Helper method to be able to pass needed vars to _compute_subset.r )rr class_type class_varss r_get_class_vars_typez$KDEMultivariate._get_class_vars_typeNs& mm& :%%r NNr+ __name__ __module__ __qualname____doc__rr)r;rBrKrgrmr,r rr r )s5:v2 '2"H"H.`V;p&r r cHeZdZdZ d dZdZdfdZd dZd dZd Z d Z y) r a Conditional multivariate kernel density estimator. Calculates ``P(Y_1,Y_2,...Y_n | X_1,X_2...X_m) = P(X_1, X_2,...X_n, Y_1, Y_2,..., Y_m)/P(X_1, X_2,..., X_m)``. The conditional density is by definition the ratio of the two densities, see [1]_. Parameters ---------- endog : list of ndarrays or 2-D ndarray The training data for the dependent variables, used to determine the bandwidth(s). If a 2-D array, should be of shape (num_observations, num_variables). If a list, each list element is a separate observation. exog : list of ndarrays or 2-D ndarray The training data for the independent variable; same shape as `endog`. dep_type : str The type of the dependent variables: c : Continuous u : Unordered (Discrete) o : Ordered (Discrete) The string should contain a type specifier for each variable, so for example ``dep_type='ccuo'``. indep_type : str The type of the independent variables; specified like `dep_type`. bw : array_like or str, optional If an array, it is a fixed user-specified bandwidth. If a string, should be one of: - normal_reference: normal reference rule of thumb (default) - cv_ml: cross validation maximum likelihood - cv_ls: cross validation least squares defaults : Instance of class EstimatorSettings The default values for the efficient bandwidth estimation Attributes ---------- bw : array_like The bandwidth parameters See Also -------- KDEMultivariate References ---------- .. [1] https://en.wikipedia.org/wiki/Conditional_probability_distribution Examples -------- >>> import statsmodels.api as sm >>> nobs = 300 >>> c1 = np.random.normal(size=(nobs,1)) >>> c2 = np.random.normal(2,1,size=(nobs,1)) >>> dens_c = sm.nonparametric.KDEMultivariateConditional(endog=[c1], ... exog=[c2], dep_type='c', indep_type='c', bw='normal_reference') >>> dens_c.bw # show computed bandwidth array([ 0.41223484, 0.40976931]) Nc||_||_||z|_t|j|_t|j|_t ||j|_t ||j |_tj|j\|_ |_tj|j|jf|_ tj|jd|_| tn|}|j!||j"s|j%||_y|j)||_y)Nr)dep_type indep_typerrk_depk_indepr endogexogrrr column_stackrrrrrrrr)rrzr{rvrwrrs rrz#KDEMultivariateConditional.__init__s  $!J.' 4??+ "5$**5 !$ 5 " 4 4:OOTZZ$;< hhtyy)!, *2*:$& 8$~~&&r*DG--b1DGr c<d}|dt|jzdzz }|dt|jzdzz }|dt|jzdzz }|d|jzdzz }|d|j zdzz }|d|j zdzz }|S) r"z$KDEMultivariateConditional instance z+Number of independent variables: k_indep = r#z'Number of dependent variables: k_dep = zNumber of observations: nobs = z!Independent variable types: zDependent variable types: r$)r%ryrxrrwrvr&r's rr)z#KDEMultivariateConditional.__repr__s5 <4<< !#'( ( 84::!%& & 03tyy>ADHH 2T__DtKK 04==@4GG &84?? r c|Sr+r,r-s rr/z#KDEMultivariateConditional.r0r c t|j}t|jj}d}t |D]\}}t |}t || |j|ddf |j|jz} t ||jd| |j|ddf |j} | | z } ||| z }| S)a Returns the leave-one-out conditional likelihood of the data. If `func` is not equal to the default, what's calculated is a function of the leave-one-out conditional likelihood. Parameters ---------- bw : array_like The bandwidth parameter(s). func : callable, optional Function to transform the likelihood values (before summing); for the log likelihood, use ``func=np.log``. Default is ``f(x) = x``. Returns ------- L : float The value of the leave-one-out function for the data. Notes ----- Similar to ``KDE.loo_likelihood`, but substitute ``f(y|x)=f(x,y)/f(x)`` for ``f(x)``. rNr2) rrr{__iter__r4nextrrvrwrx) rrr5yLOOxLOOr7r8Y_jr9f_yxf_xr:s rr;z)KDEMultivariateConditional.loo_likelihoods2499%499%..0 o FAs4jG#TYYq!t_4D"&--$//"ADDr$**+gX%)YYq!t_$4 $1C*C cNA r r c j| |j}nt||j}| |j}nt||j}g}t j ||f}tt j|dD]}t|j|j||ddf|j|jz}t|j|jd|j||ddf|j}|j||z t j|S)aQ Evaluate the probability density function. Parameters ---------- endog_predict : array_like, optional Evaluation data for the dependent variables. If unspecified, the training data is used. exog_predict : array_like, optional Evaluation data for the independent variables. Returns ------- pdf : array_like The value of the probability density at `endog_predict` and `exog_predict`. Notes ----- The formula for the conditional probability density is: .. math:: f(y|x)=\frac{f(x,y)}{f(x)} with .. math:: f(x)=\prod_{s=1}^{q}h_{s}^{-1}k \left(\frac{x_{is}-x_{js}}{h_{s}}\right) where :math:`k` is the appropriate kernel for each variable. Nrr2)rzr rxr{ryrr|r>rrrrrvrwr?r@)r endog_predict exog_predictrAr3r8rrs rrBzKDEMultivariateConditional.pdfs<   JJM)-DM  99L(t||DL |'DE rxx -a01 'Adii%1!Q$%7"&--$//"ADDtwwtzz{+$))$0A$6 $1C NN4#: & 'zz'""r c @| |j}nt||j}| |j}nt||j}t j |d}t j|}t|D]}t|j|jd|j||ddf|j|jz }t j|}t|jd|j|j||ddf|jdddd}t|j|jd|j||ddf|jd }||zjd } | |j|zz ||<|S) a Cumulative distribution function for the conditional density. Parameters ---------- endog_predict : array_like, optional The evaluation dependent variables at which the cdf is estimated. If not specified the training dependent variables are used. exog_predict : array_like, optional The evaluation independent variables at which the cdf is estimated. If not specified the training independent variables are used. Returns ------- cdf_est : array_like The estimate of the cdf. Notes ----- For more details on the estimation see [2]_, and p.181 in [1]_. The multivariate conditional CDF for mixed data (continuous and ordered/unordered discrete) is estimated by: .. math:: F(y|x)=\frac{n^{-1}\sum_{i=1}^{n}G(\frac{y-Y_{i}}{h_{0}}) W_{h}(X_{i},x)}{\widehat{\mu}(x)} where G() is the product kernel CDF estimator for the dependent (y) variable(s) and W() is the product kernel CDF estimator for the independent variable(s). References ---------- .. [1] Racine, J., Li, Q. Nonparametric econometrics: theory and practice. Princeton University Press. (2007) .. [2] Liu, R., Yang, L. "Kernel estimation of multivariate cumulative distribution function." Journal of Nonparametric Statistics (2008) Nrr2rDrErFF)rr3rrGrHrItosumrr3rrrP)rzr rxr{ryrrrYr>rrrwrr@rvrZ) rrrN_data_predictrJr8mu_x cdf_endogcdf_exogSs rrKzKDEMultivariateConditional.cdfsyR   JJM)-DM  99L(t||DL,/2((>*~& 0A ,499%1!Q$%7!%248II>D::d#DTWWQtzz2*71*=&*mm&4&;&5U DIDGGDJJK0tyy)5ad);%)__ECHX%***2Adii$./GAJ! 0$r c ^t|j}d}t|j}t j |jdz df}t |D]\}}|dd|jdf}|ddd|jf} t j| |} t j|| } t j||} t j||} t||jd| |j|ddf|jd}t||jd| |j|ddf|jd}t|d|j| | |jdddd }||z|zj|d zz }t|| |j|ddf |j|jz |z }t||jd| |j|ddf |j |z }|||d zz d ||z zz z }||z S) a The integrated mean square error for the conditional KDE. Parameters ---------- bw : array_like The bandwidth parameter(s). Returns ------- CV : float The cross-validation objective function. Notes ----- For more details see pp. 156-166 in [1]_. For details on how to handle the mixed variable types see [2]_. The formula for the cross-validation objective function for mixed variable types is: .. math:: CV(h,\lambda)=\frac{1}{n}\sum_{l=1}^{n} \frac{G_{-l}(X_{l})}{\left[\mu_{-l}(X_{l})\right]^{2}}- \frac{2}{n}\sum_{l=1}^{n}\frac{f_{-l}(X_{l},Y_{l})}{\mu_{-l}(X_{l})} where .. math:: G_{-l}(X_{l}) = n^{-2}\sum_{i\neq l}\sum_{j\neq l} K_{X_{i},X_{l}} K_{X_{j},X_{l}}K_{Y_{i},Y_{j}}^{(2)} where :math:`K_{X_{i},X_{l}}` is the multivariate product kernel and :math:`\mu_{-l}(X_{l})` is the leave-one-out estimator of the pdf. :math:`K_{Y_{i},Y_{j}}^{(2)}` is the convolution kernel. The value of the function is minimized by the ``_cv_ls`` method of the `GenericKDE` class to return the bw estimates that minimize the distance between the estimated and "true" probability density. References ---------- .. [1] Racine, J., Li, Q. Nonparametric econometrics: theory and practice. Princeton University Press. (2007) .. [2] Racine, J., Li, Q. "Nonparametric Estimation of Distributions with Categorical and Continuous Data." Working Paper. (2000) rrNFrgauss_convolutionwangryzin_convolutionaitchisonaitken_convolution)rr3rrGrIrHrrRr2)rrfloatrronesr4rxkronrr{rwrvrZ)rrzLOOCVrexpanderrcZXYYe_LYe_RXe_LXe_RK_Xi_XlK_Xj_XlK2_Yi_YjGf_X_Ym_xs rrgzKDEMultivariateConditional.imse[s!^499% TYY77DIIM1-.t_ 5EB!TZZ[.!A![djj[.!A771h'D778Q'D771h'D778Q'D2djjk?(, "a%(8$(OO5BG2djjk?(, "a%(8$(OO5BGBq,4)- %8%<%B"' )H 7"X-224tQw>A1"DIIb!e4D3D#'==4??#BEGKLEr$**+aR%)YYr1u%5$5 $1378C 1sax<1 #44 4B3 56Dyr cTd}|j|j|jf}||fS)rir )rxrvrwrjs rrmz/KDEMultivariateConditional._get_class_vars_types*1 jj$--A :%%r r+rnror,r rr r Us;?D2& '2&P2#hFPN`&r r )rsnumpyrr _kernel_baserrrrr __all__r r r,r rrsA< Qi&ji&X Z&Z&r