Example: Running DplPy with Jupyter Notebook¶

This Jupyter Notebook must be run within the dplPy directory. Refer to README.md for instructions on how to run Jupyter Notebook and the required conda Environment on your OS.

Import packages and load test data from the test directory.

Note: choose your own example file from /tests/data/<format>/<file.format>. In this example, the test file is rwl/ca533.rwl.

In [11]:

# Import statements
# loading test example as data
# Find examples in `tests` folder

import os
directory = os.getcwd().split("/")
if directory[-1] != 'src':
    os.chdir("./src")
import dplpy as dpl
data  = dpl.readers("../tests/data/rwl/ca533.rwl")

# Import statements # loading test example as data # Find examples in `tests` folder import os directory = os.getcwd().split("/") if directory[-1] != 'src': os.chdir("./src") import dplpy as dpl data = dpl.readers("../tests/data/rwl/ca533.rwl")

Attempting to read input file: ca533.rwl as .rwl format


SUCCESS!
File read as: .rwl file

Series names:
['CAM011', 'CAM021', 'CAM031', 'CAM032', 'CAM041', 'CAM042', 'CAM051', 'CAM061', 'CAM062', 'CAM071', 'CAM072', 'CAM081', 'CAM082', 'CAM091', 'CAM092', 'CAM101', 'CAM102', 'CAM111', 'CAM112', 'CAM121', 'CAM122', 'CAM131', 'CAM132', 'CAM141', 'CAM151', 'CAM152', 'CAM161', 'CAM162', 'CAM171', 'CAM172', 'CAM181', 'CAM191', 'CAM201', 'CAM211']

Summarize data using the summary function.

In [12]:

# Summarizes input data

dpl.summary(data)

# Summarizes input data dpl.summary(data)

Out[12]:

	CAM011	CAM021	CAM031	CAM032	CAM041	CAM042	CAM051	CAM061	CAM062	CAM071	...	CAM151	CAM152	CAM161	CAM162	CAM171	CAM172	CAM181	CAM191	CAM201	CAM211
count	454.000000	551.000000	628.000000	549.000000	301.000000	446.000000	737.000000	627.000000	459.000000	947.000000	...	749.000000	229.000000	504.000000	1000.000000	758.000000	797.000000	781.000000	791.000000	593.000000	1343.000000
mean	0.439581	0.424465	0.349156	0.293224	0.525648	0.439148	0.273012	0.462281	0.441939	0.249071	...	0.445648	0.533799	0.339464	0.396710	0.450264	0.482296	0.282638	0.366271	0.473929	0.356813
std	0.221801	0.185397	0.213666	0.162930	0.222568	0.347705	0.139691	0.201785	0.188389	0.109357	...	0.272561	0.194947	0.148916	0.184057	0.209848	0.249002	0.148853	0.335788	0.180967	0.182086
min	0.000000	0.050000	0.000000	0.000000	0.100000	0.070000	0.000000	0.000000	0.000000	0.000000	...	0.000000	0.060000	0.000000	0.000000	0.080000	0.080000	0.000000	0.000000	0.000000	0.000000
25%	0.290000	0.290000	0.180000	0.180000	0.350000	0.270000	0.180000	0.335000	0.330000	0.180000	...	0.240000	0.410000	0.230000	0.260000	0.300000	0.310000	0.170000	0.170000	0.350000	0.220000
50%	0.400000	0.400000	0.290000	0.260000	0.530000	0.360000	0.250000	0.470000	0.450000	0.250000	...	0.390000	0.520000	0.330000	0.370000	0.400000	0.420000	0.250000	0.250000	0.470000	0.340000
75%	0.540000	0.520000	0.510000	0.390000	0.680000	0.460000	0.330000	0.600000	0.580000	0.320000	...	0.610000	0.660000	0.430000	0.510000	0.580000	0.590000	0.380000	0.455000	0.580000	0.470000
max	1.360000	1.110000	1.030000	0.850000	1.380000	3.030000	1.320000	1.090000	0.920000	0.620000	...	1.640000	1.250000	0.900000	1.040000	1.540000	1.980000	0.800000	2.540000	1.490000	1.100000

8 rows × 34 columns

Show general statistics of data through the stats function.

In [3]:

# General statistics of input data

dpl.stats(data)

# General statistics of input data dpl.stats(data)

Out[3]:

	series	first	last	year	mean	median	stdev	skew	gini	ar1
1	CAM011	1530	1983	454	0.440	0.40	0.222	1.029	0.273	0.698
2	CAM021	1433	1983	551	0.424	0.40	0.185	0.946	0.237	0.702
3	CAM031	1356	1983	628	0.349	0.29	0.214	0.690	0.341	0.809
4	CAM032	1435	1983	549	0.293	0.26	0.163	0.717	0.309	0.665
5	CAM041	1683	1983	301	0.526	0.53	0.223	0.488	0.238	0.710
6	CAM042	1538	1983	446	0.439	0.36	0.348	3.678	0.324	0.881
7	CAM051	1247	1983	737	0.273	0.25	0.140	1.836	0.262	0.705
8	CAM061	1357	1983	627	0.462	0.47	0.202	-0.111	0.247	0.510
9	CAM062	1525	1983	459	0.442	0.45	0.188	-0.266	0.240	0.529
10	CAM071	1037	1983	947	0.249	0.25	0.109	0.027	0.247	0.578
11	CAM072	1114	1983	870	0.309	0.29	0.163	0.698	0.292	0.735
12	CAM081	1081	1983	903	0.327	0.31	0.124	0.555	0.211	0.723
13	CAM082	977	1983	1007	0.285	0.29	0.114	0.312	0.223	0.771
14	CAM091	1460	1983	524	0.532	0.52	0.255	0.425	0.267	0.632
15	CAM092	1591	1983	393	0.349	0.34	0.226	0.337	0.369	0.561
16	CAM101	1727	1983	257	0.568	0.56	0.260	0.254	0.259	0.716
17	CAM102	1665	1983	319	0.604	0.62	0.261	0.082	0.243	0.677
18	CAM111	1446	1983	538	0.625	0.62	0.249	0.196	0.225	0.625
19	CAM112	1471	1983	513	0.570	0.56	0.211	0.223	0.207	0.583
20	CAM121	1000	1983	984	0.259	0.26	0.106	0.042	0.231	0.594
21	CAM122	1000	1983	984	0.271	0.27	0.109	0.346	0.223	0.653
22	CAM131	695	1970	1276	0.552	0.53	0.198	0.330	0.202	0.788
23	CAM132	710	1232	523	0.397	0.38	0.148	0.871	0.203	0.810
24	CAM141	1030	1970	941	0.627	0.60	0.204	0.695	0.177	0.746
25	CAM151	1222	1970	749	0.446	0.39	0.273	1.068	0.332	0.765
26	CAM152	1221	1449	229	0.534	0.52	0.195	0.297	0.203	0.695
27	CAM161	1106	1609	504	0.339	0.33	0.149	0.633	0.243	0.794
28	CAM162	971	1970	1000	0.397	0.37	0.184	0.647	0.259	0.840
29	CAM171	1213	1970	758	0.450	0.40	0.210	1.250	0.250	0.799
30	CAM172	1174	1970	797	0.482	0.42	0.249	1.622	0.268	0.847
31	CAM181	1190	1970	781	0.283	0.25	0.149	0.706	0.293	0.805
32	CAM191	1180	1970	791	0.366	0.25	0.336	2.359	0.429	0.876
33	CAM201	990	1582	593	0.474	0.47	0.181	0.772	0.208	0.709
34	CAM211	626	1968	1343	0.357	0.34	0.182	0.513	0.286	0.683

In [6]:

# Detrends a series by fitting to spline and calculating residuals.
# !!! Note: spline is the current detrent default method;
# !!! Note: Line graph is defaulted to show residuals.
# 
# Detrend funtion can modified to fit Hugershoff, modified negative exponential, linear, horizonal.
# This option will be available in the short term future.
#
# The detrend function accepts a specific series as input in the following format:

dpl.detrend(data["CAM191"])

# Detrends a series by fitting to spline and calculating residuals. # !!! Note: spline is the current detrent default method; # !!! Note: Line graph is defaulted to show residuals. # # Detrend funtion can modified to fit Hugershoff, modified negative exponential, linear, horizonal. # This option will be available in the short term future. # # The detrend function accepts a specific series as input in the following format: dpl.detrend(data["CAM191"])

In [9]:

# Returns parameters
# As default max lag is set to 5; 
# add second parameter (integer) to change max lag.

dpl.autoreg(data["CAM191"], 10)

# Returns parameters # As default max lag is set to 5; # add second parameter (integer) to change max lag. dpl.autoreg(data["CAM191"], 10)

/Users/michelecosi/miniconda3/envs/dplpy4/lib/python3.8/site-packages/statsmodels/tsa/base/tsa_model.py:471: ValueWarning: An unsupported index was provided and will be ignored when e.g. forecasting.
  self._init_dates(dates, freq)
/Users/michelecosi/miniconda3/envs/dplpy4/lib/python3.8/site-packages/statsmodels/tsa/base/tsa_model.py:471: ValueWarning: An unsupported index was provided and will be ignored when e.g. forecasting.
  self._init_dates(dates, freq)

Out[9]:

const         0.022210
CAM191.L1     0.503373
CAM191.L2     0.087230
CAM191.L3     0.143716
CAM191.L4     0.020119
CAM191.L5    -0.027769
CAM191.L6    -0.010029
CAM191.L7     0.001373
CAM191.L8     0.025588
CAM191.L9     0.042340
CAM191.L10    0.136916
dtype: float64

In [10]:

# Returns residuals+mean after 
# choosing best AR model fit with chosen max lag (default=5).

dpl.ar_func(data["CAM191"])

# Returns residuals+mean after # choosing best AR model fit with chosen max lag (default=5). dpl.ar_func(data["CAM191"])

/Users/michelecosi/miniconda3/envs/dplpy4/lib/python3.8/site-packages/statsmodels/tsa/base/tsa_model.py:471: ValueWarning: An unsupported index was provided and will be ignored when e.g. forecasting.
  self._init_dates(dates, freq)
/Users/michelecosi/miniconda3/envs/dplpy4/lib/python3.8/site-packages/statsmodels/tsa/base/tsa_model.py:471: ValueWarning: An unsupported index was provided and will be ignored when e.g. forecasting.
  self._init_dates(dates, freq)

Out[10]:

array([-0.23485391,  0.61703427,  0.07848781,  0.30139532,  0.31970781,
        0.7709527 , -0.13122202,  0.73735556,  0.79259251,  1.03150292,
        0.19654108, -0.13481225,  0.61573077,  0.65068195,  0.43323283,
        0.95951473, -0.37325966,  0.50079401, -0.10560507,  0.05038   ,
        0.56974055,  0.3394715 ,  0.51772866,  0.28269221,  0.42226138,
        0.63341811,  1.4739965 ,  0.28602606,  0.2604808 ,  0.27934854,
       -0.34367123,  0.33895547,  0.19844979, -0.07263543,  0.45164677,
        0.85279497,  1.01238366,  0.74549344,  0.51996229,  0.77010877,
        0.46936605,  0.0785818 ,  0.09314398,  0.47463727,  0.83439219,
        0.03987213,  0.81898116,  0.58663827,  0.30382456,  0.43938001,
        0.53303445,  0.23979872,  0.42268213,  0.6094634 ,  0.44863191,
        0.52719227,  0.14005905,  0.32252156,  0.14725286,  0.29913277,
        0.61191963,  0.44957831,  0.42429965,  0.5959667 ,  0.49178689,
        0.45410314,  0.25085001,  0.42356281,  0.55757744, -0.24367237,
        0.15117969,  0.66712127,  0.69258454, -0.12784539,  0.6789434 ,
        0.60783217,  0.61407111,  0.62246195,  0.41272014, -0.00936233,
       -0.0480617 ,  0.0694901 ,  0.33897886,  0.36166267,  0.30446064,
        0.48141051,  0.56882759,  0.64385542,  0.30465005,  0.51569189,
        0.79791196,  0.39735395,  0.31585581,  0.15895399,  0.58666957,
        0.40978068,  0.15985022,  0.40607871,  0.40388826,  0.3096272 ,
        0.0342205 ,  0.55427596,  0.52377703,  0.26738502,  0.53938707,
        0.33790184,  0.5382865 ,  0.41514302,  0.35265856,  0.30646022,
        0.50990241,  0.4243464 ,  0.34850027,  0.18307126,  0.2732797 ,
        0.51736136,  0.54959262,  0.45413807,  0.29612966,  0.45052545,
        0.46909323,  0.36982143,  0.39153275,  0.64037135,  0.27772417,
        0.31826222,  0.52523135,  0.51617064,  0.45047841,  0.49865942,
        0.45413743,  0.05947446,  0.28437018,  0.38299106,  0.22614538,
        0.36022031,  0.50037608,  0.66379821, -0.08412939,  0.80206774,
        0.55463454,  0.65119228,  0.55409197,  0.41811455,  0.18790989,
        0.31775581,  0.42356545, -0.11198417,  0.61155616, -0.00400045,
        0.42381569,  0.16696088,  0.42339441,  0.27222434,  0.34009664,
        0.37453456,  0.47170985,  0.40264874,  0.51659048,  0.36462353,
        0.38726786,  0.41915095,  0.43958695,  0.2692216 ,  0.4572227 ,
        0.37663572,  0.37516709,  0.51114637,  0.51301848,  0.20921054,
        0.38756127,  0.50750609,  0.34038451,  0.22999705,  0.41815438,
        0.1613977 ,  0.09181197,  0.28494723,  0.39606824,  0.51561671,
        0.36811293,  0.35614156,  0.56900867,  0.40354229,  0.61000538,
        0.42411197,  0.19417072,  0.45693544,  0.59061735,  0.46628091,
        0.42696804,  0.4041577 ,  0.08013931,  0.43943249,  0.14019481,
        0.48190889,  0.58643436,  0.55826101,  0.5105765 ,  0.48199429,
        0.54310525,  0.27927251,  0.30634321,  0.32827562,  0.52612092,
        0.35138125,  0.31463398,  0.54032327,  0.32644925,  0.52935574,
        0.37253887,  0.31671287,  0.25945758,  0.24138538,  0.29070475,
        0.48241831,  0.07597918,  0.5339914 ,  0.43070055,  0.38209941,
        0.54220723,  0.41888816,  0.13905795,  0.37808339,  0.5182685 ,
        0.33383431,  0.43731525,  0.49870236,  0.39623926,  0.20947203,
        0.43548597,  0.33406516,  0.61692086,  0.20888847,  0.43691344,
        0.40875671,  0.28687993,  0.33627451,  0.45303114,  0.39374527,
        0.30778941,  0.28913639,  0.52006099,  0.29244499,  0.50501874,
        0.30752383,  0.23357822,  0.51230839,  0.49609901,  0.03676054,
        0.66377245,  0.46695285,  0.28583492,  0.24910179,  0.38565254,
        0.51333396,  0.34451973,  0.30623028,  0.28624853,  0.2484529 ,
        0.30577825,  0.23987747,  0.33826147,  0.21603312,  0.39775025,
        0.24295094,  0.37695061,  0.35095463,  0.42067853,  0.29581275,
        0.43572216,  0.26531939,  0.30630855,  0.30089194,  0.3770091 ,
        0.29272387,  0.4289114 ,  0.35323252,  0.37325316,  0.33271626,
        0.35549783,  0.39572473,  0.35150282,  0.37620792,  0.35962842,
        0.37867256,  0.31254057,  0.30823814,  0.33498754,  0.30991824,
        0.22106116,  0.47600479,  0.36901305,  0.35061435,  0.34308667,
        0.40811558,  0.42101334,  0.30019725,  0.38620366,  0.35702432,
        0.38213919,  0.46361132,  0.32010587,  0.35373268,  0.32502378,
        0.44608853,  0.38144384,  0.32546815,  0.47334522,  0.447194  ,
        0.2980056 ,  0.23089079,  0.10410034,  0.34967115,  0.45274025,
        0.25222561,  0.49008612,  0.40675568,  0.41047231,  0.29493453,
        0.32821789,  0.28024849,  0.35079935,  0.37324237,  0.3868993 ,
        0.37054072,  0.39523113,  0.44287362,  0.31951994,  0.3604808 ,
        0.25996029,  0.48716987,  0.3405697 ,  0.36588728,  0.36170745,
        0.47566618,  0.36956056,  0.12281409,  0.17040604,  0.43080876,
        0.45339988,  0.38657195,  0.51062833,  0.24663067,  0.2944111 ,
        0.42325223,  0.30723836,  0.39412815,  0.11526089,  0.45728477,
        0.49026851,  0.44408903,  0.52517481,  0.30415652,  0.44375   ,
        0.36124269,  0.10234557,  0.14085553,  0.38944291,  0.33393161,
        0.24747679,  0.38142452,  0.41099257,  0.42110084,  0.33544361,
        0.42189318,  0.31541684,  0.45510977,  0.34895568,  0.37032153,
        0.394427  ,  0.27837366,  0.4047497 ,  0.10612834,  0.4954011 ,
        0.37753079,  0.46006873,  0.25295007,  0.44859502,  0.36381464,
        0.20880894,  0.48459165,  0.43634343,  0.27762589,  0.42387602,
        0.25771294,  0.14473909,  0.32079832,  0.31786878,  0.4091164 ,
        0.38066256,  0.35516329,  0.33930981,  0.21244847,  0.29353493,
        0.32229358,  0.49335149,  0.34781998,  0.46855312,  0.32200119,
        0.34367487,  0.36297008,  0.36864722,  0.35925875,  0.31742445,
        0.26701958,  0.3770456 ,  0.29160236,  0.39071092,  0.46679997,
        0.31216462,  0.44482003,  0.38258064,  0.45239854,  0.36552033,
        0.21686622,  0.37011469,  0.20408543,  0.4766801 ,  0.37603546,
        0.40307278,  0.26970016,  0.18577225,  0.5308118 ,  0.41354986,
        0.35416121,  0.30267815,  0.2431837 ,  0.38263567,  0.43256545,
        0.28026539,  0.47917932,  0.36702031,  0.12126338,  0.41128376,
        0.36910108,  0.32816472,  0.39654297,  0.2728162 ,  0.39017996,
        0.41235812,  0.32996245,  0.3658953 ,  0.39350694,  0.41380087,
        0.33389311,  0.28319668,  0.28699192,  0.38468972,  0.45064504,
        0.43507548,  0.34339383,  0.38741558,  0.29479123,  0.38417023,
        0.42920096,  0.21992131,  0.35415206,  0.41429713,  0.28045376,
        0.34746292,  0.3414929 ,  0.30219699,  0.39595103,  0.36762076,
        0.35817275,  0.43388609,  0.34128867,  0.29889916,  0.25351852,
        0.20326752,  0.42197456,  0.47608529,  0.42410472,  0.38622311,
        0.43329866,  0.38954634,  0.26966697,  0.31011112,  0.41435408,
        0.3312214 ,  0.30561928,  0.34896042,  0.14830434,  0.47081585,
        0.16646207,  0.41601987,  0.37207901,  0.42193349,  0.25398928,
        0.24741244,  0.33292854,  0.27909425,  0.44545975,  0.36996414,
        0.40320527,  0.23548363,  0.46307916,  0.35668605,  0.37330783,
        0.37192971,  0.29535696,  0.41388927,  0.34920842,  0.35240151,
        0.17638227,  0.46377493,  0.3939198 ,  0.46448029,  0.34974532,
        0.4091069 ,  0.2996696 ,  0.31868948,  0.27776093,  0.27969009,
        0.31001004,  0.34651993,  0.36511498,  0.25641987,  0.36787472,
        0.25061601,  0.43874015,  0.30907333,  0.30742187,  0.43356778,
        0.37184625,  0.37781472,  0.38811771,  0.36018255,  0.32513911,
        0.41765669,  0.32689887,  0.28030158,  0.39163757,  0.33158315,
        0.3595052 ,  0.46889063,  0.32822815,  0.32566163,  0.33101464,
        0.37216688,  0.39733423,  0.44800362,  0.40265194,  0.36828679,
        0.3945686 ,  0.25542368,  0.35761989,  0.23298255,  0.35897608,
        0.38835324,  0.35940801,  0.33920676,  0.38774871,  0.35665835,
        0.40969949,  0.37855848,  0.26505381,  0.3750314 ,  0.3614608 ,
        0.29583819,  0.387177  ,  0.45308406,  0.35631867,  0.3471497 ,
        0.31944481,  0.40949886,  0.32453882,  0.27349772,  0.31247103,
        0.28736743,  0.33053127,  0.28445865,  0.42163137,  0.39165334,
        0.48125848,  0.39958453,  0.36121433,  0.25596448,  0.35028976,
        0.37800471,  0.29054553,  0.39901677,  0.28566321,  0.40228394,
        0.28094665,  0.38282782,  0.32080927,  0.36732028,  0.28021801,
        0.43386441,  0.36566424,  0.17309174,  0.2864465 ,  0.48588328,
        0.4379513 ,  0.31807606,  0.22276896,  0.3837975 ,  0.35437269,
        0.41095687,  0.35536757,  0.36256703,  0.41053693,  0.19875665,
        0.35931273,  0.34938392,  0.35938082,  0.29932014,  0.36026023,
        0.29420832,  0.41583381,  0.41836997,  0.35236038,  0.36478763,
        0.37422349,  0.42237656,  0.35491203,  0.38401865,  0.32478055,
        0.39639871,  0.30754248,  0.28298515,  0.35355333,  0.38764711,
        0.24149446,  0.44736186,  0.26846159,  0.30305692,  0.37937295,
        0.35399136,  0.37436065,  0.48975852,  0.32763145,  0.35771297,
        0.24457454,  0.43817467,  0.32318189,  0.40939673,  0.30904956,
        0.4058264 ,  0.31548055,  0.34778588,  0.27216749,  0.35959523,
        0.35308367,  0.39804884,  0.31123224,  0.37074699,  0.31098905,
        0.32787846,  0.22225314,  0.36189398,  0.38569857,  0.40244394,
        0.37431475,  0.37646296,  0.30050867,  0.38853535,  0.26827023,
        0.30120785,  0.3924199 ,  0.31654751,  0.39963708,  0.35394093,
        0.37008765,  0.34325678,  0.46986591,  0.37089893,  0.37767187,
        0.38052207,  0.21070825,  0.26081478,  0.26066993,  0.43560227,
        0.47730778,  0.26494492,  0.44390498,  0.31396022,  0.46117816,
        0.22760724,  0.45842222,  0.33492516,  0.46903585,  0.32518185,
        0.41382035,  0.33071743,  0.32781117,  0.42697473,  0.29922905,
        0.39353705,  0.34316083,  0.45248865,  0.31556241,  0.29688595,
        0.19042043,  0.30801811,  0.43209191,  0.38171683,  0.36874542,
        0.33110284,  0.43408485,  0.35350848,  0.41717183,  0.3795025 ,
        0.33800292,  0.40788355,  0.3487189 ,  0.36334129,  0.36773897,
        0.35716652,  0.31958933,  0.35229354,  0.38192844,  0.131064  ,
        0.46945613,  0.50452416,  0.40170409,  0.33910162,  0.2446623 ,
        0.47178829,  0.35272988,  0.47105281,  0.32434132,  0.29563633,
        0.3489984 ,  0.4313384 ,  0.26458517,  0.28972957,  0.37328946,
        0.3513375 ,  0.4182225 ,  0.40488257,  0.35984623,  0.35157567,
        0.36490789,  0.36145163,  0.36219304,  0.31249589,  0.31993087,
        0.34506224,  0.33977483,  0.43941594,  0.34300712,  0.11407017,
        0.55013731,  0.36031895,  0.35651344,  0.3536409 ,  0.39214983,
        0.35685251,  0.32191647,  0.42147809,  0.28167082,  0.39399796,
        0.38954763,  0.29320292,  0.28894349,  0.367458  ,  0.33458904,
        0.38906359,  0.41711617,  0.46963755,  0.23272096,  0.29462233,
        0.39546486,  0.4524549 ,  0.31592476,  0.45280139,  0.33135251,
        0.36175169,  0.36998016,  0.35009763,  0.34600311,  0.40323174,
        0.22948486,  0.29787165,  0.47074459,  0.44792308,  0.43539402,
        0.40254804,  0.2987047 ,  0.374543  ,  0.37842883,  0.40753657,
        0.34595611])

---

Last update: 2024-01-09