Northeast Brook Trout Occupancy

Table of Model Results

Fixed Effects:

Parameter	Estimate	Std. Error	z value	P-value
(Intercept)	0.314	0.11	2.84	0.00445
area	-0.416	0.0591	-7.04	1.89e-12
summer_prcp_mm	0.385	0.0978	3.94	8.14e-05
meanJulyTemp	-0.706	0.0719	-9.82	9.03e-23
forest	0.413	0.0686	6.02	1.71e-09
surfcoarse	0.165	0.0586	2.81	0.00494
allonnet	-0.291	0.0568	-5.13	2.83e-07
devel_hi	-0.0996	0.0569	-1.75	0.0799
agriculture	-0.664	0.0995	-6.67	2.57e-11
area:summer_prcp_mm	0.0217	0.0503	0.432	0.666
meanJulyTemp:forest	-0.034	0.0501	-0.678	0.498
summer_prcp_mm:forest	0.127	0.0585	2.17	0.0302

Random Effects (HUC10):

Parameter	SD	Variance
(Intercept)	1.34	1.79
area	0.211	0.0443
agriculture	0.353	0.124
summer_prcp_mm	0.534	0.285
meanJulyTemp	0.235	0.0553

These results indicate that mean July stream temperature had the largest (negative) effect on the probability of Brook Trout occupancy. Forest cover within the 200 foot riparian buffer had a strong positive effect on occupancy, whereas agriculture within the entire upstream drainage had a negative effect on occupancy. Mean summer precipitation has a positive effect on occupancy and the effect was larger with increasing levels of riparian forest cover, but was not dependent on stream drainage area. The total impounded area on the stream network had a negative effect on Brook Trout occupancy as did the upstream drainage area. Surficial coarseness was positively correlated with the presence of Brook Trout, which may be a result of better physical habitat structure or as an indication of local groundwater upwelling.

The average occupancy across the range of observed catchments was 0.58.

The effects of these landscape and climate characteristics are similar to what has been observed in other Brook Trout studies.

Model Fit

We examined the false positive and false negative rates and used the Area Under the Receiver Operating Characteristic ROC) curve (AUC) to assess the model fit.

Definitions

• AUC - measures a model’s ability to determine which locations are occupied (Zipkin et al. (2012) Ecological Applications)
• Sensitivity - true positive rate (=“recall rate”)
• Specificity - true negative rate
• 1-Specificity - false positive rate (Type I Error rate; =“fall-out rate”)
• 1-Sensitivity - false negative rate (Type II Error rate; =“miss rate”)
• Accuracy - ability to identify true positives and true negatives
• ROC - The curve is created by plotting the true positive rate (sensitivity) against the false positive rate (1-specificity) at various threshold settings

The model output (predictions) are the probability of occupancy but the data are observed presence and absence (1 or 0). Therefore, it is difficult to evaluate how well the model predicts the data. The probabilities of occupancy must be converted to presence-absences for comparison. We do this over a range of thresholds (= cutoffs). The threshold is the probability above which the stream is assumed to be occupied (Brook Trout = present). For example, if the probability of occupancy for a stream is 0.45 and we set a threshold = 0.50, we would assign the stream as unoccupied (absent). However, if we used a threshold of 0.4 then this same stream would be assigned as occupied (present). If the true (observed) state of the stream was occupied, then using a threshold of 0.5 would result in a false absence (predicted absent when really present) but if we used a threshold of 0.4 we would correct assign the stream as occupied (true positive). Assigning a threshold is a balance of trade-offs between false positives and false negatives. The balance is based on the risk tolerance to the consequences of type I and type II errors.

AUC can range from 0-1. An AUC value of 0.5 indicates the model does no better than random chance in discriminating occupancy. Models with AUC >0.7 are considered to have good discrimination in assessing the probability of occupancy.

Our model is a good fit (AUC = 0.95) to the data and has a very strong ability to assess the probability of occupancy.

Model Validation

More important than how well the model works with the data used to fit the model is the ability of the model to predict occupancy at unsurveyed locations. To assess this predictive power, we used data from the 1933 stream reaches withheld from model fitting. We use the term “fitted data” to refer to the data we used to fit (estimate) the model. For comparison, others use the terms “training data” or “calibration data” synonymously. Validation data are the independent data withheld from model fitting for the purpose of understanding how well a model predicts to unobserved space and time. To evaluate this predictive power, we plotted the false positive rate (1-specificity) vs. the true positive rate (sensitivity) and calculated the AUC.

The AUC when predicting for the validation data was 0.75, which indicates that the model has good ability to discriminated between occupied and unoccupied stream reaches (catchments) for locations without survey data. From this we are highly confident in the model’s ability to predict occupancy across the region.

Direct Comparison

Our predictions were on slightly different flowlines and catchment delineations than those used by Downstream Strategies. To make explicit comparision of predictions from the two models, we rasterized the DSS predictions and performed zonal statistics to calculate the area-weighted average in each of our catchments (associated with each stream reach).