Dj)ddlmZddlmZddlmZmZddlmZm Z m Z m Z m Z ddl Z ddlmZmZddlmZddlmZddlZdd lmZmZmZdd ZGd d eZdd ZdZdZ dZ!dZ"dZ#ddZ$dZ%ddZ&ddZ'ddZ(ddZ)dZ*d dZ+y)!)lrange)StringIO)environmakedirs)abspathdirnameexists expanduserjoinN) HTTPErrorURLError)urljoin)urlopen)Indexread_csv read_statac6t||dz}t|S)a Download and return an example dataset from Stata. Parameters ---------- data : str Name of dataset to fetch. baseurl : str The base URL to the stata datasets. as_df : bool Deprecated. Always returns a DataFrame Returns ------- dta : DataFrame A DataFrame containing the Stata dataset. Examples -------- >>> dta = webuse('auto') Notes ----- Make sure baseurl has trailing forward slash. Does not do any error checking in response URLs. z.dta)rr)databaseurlas_dfurls ^C:\Crop_Prediction\Backend\crop-ai-system\venv\Lib\site-packages\statsmodels/datasets/utils.pywebusers6 '4; 'C c?ceZdZdZdZy)Datasetc d|_d|_d|_d|_tj ||||_ |jjt|_ y#YyxYwN) endogexogrnamesdict__init____dict__astypefloatraw_data)selfkws rr#zDataset.__init__/sX     dB   II,,U3DM  s $A A$c,t|jSr)str __class__)r(s r__repr__zDataset.__repr__?s4>>""rN)__name__ __module__ __qualname__r#r-rrrr.s  #rrc|j}t|trF||}||j}||j |gd}n|||j}n|j dd|fj}t |j}||j |d}n=t|tr|||j}n|||j}|Lt|jdd|f}||_ |j|_ |j||}t |j} t|t ||||| } | S)N)axis)rr!rr endog_name exog_name) columns isinstanceintcopydroploclistrilocindex set_indexr) r endog_idxexog_idx index_idxr!r5rr r?r6datasets rprocess_pandasrECsO LLE)S!9% Z %%'  99j\92Dh(--/DI&++-%--(  99Za90D # &h(--/Dh(--/Ddii9 -. ZZ\ ~~eI./T\\"I4tE{%J)MG Nrc |jjttdt |dzr|j d}|S)z All the Rdatasets have the integer row.labels from R if there is no real index. Strip this for a zero-based index r3T)r;)r?equalsrrlen reset_index)rs r_maybe_reset_indexrJcsB  zzvaTQ789T* KrcN|durd}|S|dur td}|St|}|S)NFT) get_data_home)caches r _get_cacherNms@ ~ L $d# Le$ Lrcddl}t|d5}|j|j|dddy#1swYyxYw)Nrwb)zlibopenwritecompress)r cache_pathrQzfs r _cache_itrWxs; j$ &2 t$%&&&s !;Acddl}t|d5}|j|jcdddS#1swYyxYw)Nrrb)rQrR decompressread)rUrQrVs r _open_cacher\~s8 j$ *2rwwy)***s :Acd}||jddjdd}|jd}t|dkDr|dxxd z cc<n |d xxd z cc<dj|d z}t||} t |}d }|s)t |d j }| t||fS#Y3xYw)z Tries to load data from cache location otherwise downloads it. If it downloads the data and cache is not None then it will put the downloaded data in the cache path. Fz:///,.r3z-v2rz.zipT)timeout)splitreplacerHr r\rr[rW)rrM from_cache file_namerUrs r_urlopen_cachedris J IIe$R(00c: OOC( y>A  bMU "M aLE !LHHY'&0 %+  z*DJ sA&++-   dJ '    s  CCc||dz|zz} t||\}}|j dd}t ||fS#t$r"}dt|vrtd|z|d}~wwxYw)Nz.%s404zDataset %s was not found.utf-8strict)rir r+ ValueErrordecoder)base_urldatanamerM extensionrrrgerrs r _get_datarts h&)3 3C*36j ;;w )D D>: %%  CH 88CD DI s; A&A!!A&c \d}t||\}}|jdd}tt|}t j |j |k(|j|k(}|jstd|d|d|d|j|}|djd S) NzRhttps://raw.githubusercontent.com/vincentarelbundock/Rdatasets/master/datasets.csvrlrmzItem z from Package z& was not found. Check the CSV file at z to verify the Item and Package.Titler) rirorrnp logical_andItemPackageanyrnr<r>) rqpackagerM index_urlr_r?idx dataset_metas r_get_dataset_metars1Ii/GD! ;;w )D Xd^ $E ..x/'1I JC 779H:^G95(k)I K  99S>L  % %a ((rcd|zdz}d|zdz}t|}t|||\}}t|d}t|}t |||}t|||d\}} t ||j |||S) a7download and return R dataset Parameters ---------- dataname : str The name of the dataset you want to download package : str The package in which the dataset is found. The default is the core 'datasets' package. cache : bool or str If True, will download this data into the STATSMODELS_DATA folder. The default location is a folder called statsmodels_data in the user home folder. Otherwise, you can specify a path to a folder to use for caching the data. If False, the data will not be cached. Returns ------- dataset : Dataset A `statsmodels.data.utils.Dataset` instance. This objects has attributes: * data - A pandas DataFrame containing the data * title - The dataset title * package - The package from which the data came * from_cache - Whether not cached data was retrieved * __doc__ - The verbatim R documentation. Notes ----- If the R dataset has an integer index. This is reset to be zero-based. Otherwise the index is preserved. The caching facilities are dumb. That is, no download dates, e-tags, or otherwise identifying information is checked to see if the data should be downloaded again or not. If the dataset is in the cache, it's used. zJhttps://raw.githubusercontent.com/vincentarelbundock/Rdatasets/master/csv/r_zJhttps://raw.githubusercontent.com/vincentarelbundock/Rdatasets/master/doc/z/rst/r) index_colrst)r__doc__r|titlerg)rNrtrrJrrr[) rqr|rM data_base_url docs_base_urlrrgrdocr~s r get_rdatasetrsJ##*++./M##*++23M u E %@D* DA &D d #D h 7E }hu =FC chhj'( **rc| tjdtdd}t|}t |s t ||S)a Return the path of the statsmodels data dir. This folder is used by some large dataset loaders to avoid downloading the data several times. By default the data dir is set to a folder named 'statsmodels_data' in the user home folder. Alternatively, it can be set by the 'STATSMODELS_DATA' environment variable or programatically by giving an explicit folder path. The '~' symbol is expanded to the user home folder. If the folder does not already exist, it is automatically created. STATSMODELS_DATA~statsmodels_data)rgetr r r r data_homes rrLrLsFKK 2 $S*< =? 9%I )  rcDt|}tj|y)z.Delete all the content of the data home cache.N)rLshutilrmtreers rclear_data_homersi(I MM)rcT|dn|} t|y#t$r }Yd}~yd}~wwxYw)zCheck if internet is availableNzhttps://github.comFT)rr )rrss rcheck_internetrs4"%+ 3C   s  ''cg}|D]h}|jdr|jdr|dd}n-|jdr|dd}n|jdr|dd}|j|j||_|S)a  Remove leading and trailing single quotes Parameters ---------- df : DataFrame DataFrame to process Returns ------- df : DataFrame DataFrame with stripped column names Notes ----- In-place modification 'r3r^N) startswithendswithappendr7)dfr7cs rstrip_column_namesr"s$G  << !**T"2!BA \\$ !"A ZZ #2AqBJ Irctt|}t||}|dk7rdnd}i}|dk(rddi}t|f||d|}|r|j t }|S)zStandard simple csv loaderr`pythonrfloat_precisionhigh)sepengine)rrr rr%r&) base_filecsv_namer convert_floatfilepathfilenamerrrs rload_csvrAslwy)*HHX&HXFO },f5 H H#f H HD{{5! Kr)z%https://www.stata-press.com/data/r11/T)rNN)csv)datasetsFr)r`F),statsmodels.compat.pythonriorosrros.pathrrr r r r urllib.errorr r urllib.parserurllib.requestrnumpyrwpandasrrrrr"rrErJrNrWr\rirtrrrLrrrrr1rrrs, >> , "..>#d#*@& *< &)$2*n0 > r