Accuracy assessment of burned area maps has been traditionally performed using pixel-based metrics, with the objective of assessing the accuracy and precision of burned area estimates at local and regional scales. While these assessments are helpful for obtaining consistent estimates of the burned area across many fires and over large areas, pixel-based approaches do not necessarily characterize how well individual fires are mapped. At the individual fire scale, other factors like the shape of the fire have significance regarding ecology, fire succession, and landscape management and determining other fire properties such as the spread rate. We propose a method for evaluating wildfire classification maps, which retains the spatially explicit properties of the burn scar. Our method quantifies the edge error (EE) of burned area classifications and reference maps by calculating the average geometric normal of the evaluated burned area boundary along the burn edge and the two nearest neighbor samples from the reference burn boundary. The metric is a physically meaningful quantification of the EE, which represents the average distance between the boundaries of the reference and evaluated burn scars. The methods are demonstrated by comparing MODIS Burned Area (MCD64A1) maps to Monitoring Trends in Burn Severity (MTBS) maps for 173 total wildfires in the United States. The results indicate that when accounting for the minimum achievable EE (MAEE) due to differing spatial resolutions, the mean EE is less than two MODIS pixels and the magnitude of the errors does not appear to be related to fire size.