Modern Parking Guidance Systems Need Modern Accuracy Assessment Practices

by Peter Filice, Senior Consultant

Many of us will remember the old TV Public Service Announcement from the late 60’s: “It’s 10:00 pm – Do You Know Where Your Children Are?” Bringing this into the modern era and in the realm of parking technology, we can use some poetic license to rewrite it: “It’s 2020 – Do You Know How Accurate Your Parking Guidance System Is?”

One of the many areas of significant advancement in parking technology over the last five years is modern Automated Parking Guidance Systems (APGS) for parking structures. Many of these modern APGS are based on advanced sensor technology such as still or video camera images. Such systems, now available from many vendors, leverage the power of image processing and artificial intelligence (AI) to offer the dual advantages of very high accuracy plus value-added features like object type classification, license plate capture, and others.

The higher accuracy of these camera-based systems is a significant benefit compared to more traditional sensor technologies like loops or infrared/ultrasonic in-motion sensors. With any APGS, accuracy is absolutely paramount. It’s extremely unproductive and disappointing to invest in an APGS only to have parkers distrust it or require parking operations personnel to constantly reset the system count, or worst of all, to have parking managers turn the system completely off because the accuracy becomes so poor. The newer generation systems promise to improve on the state of the art relative to accuracy, but their accuracy still needs to be validated. What best practices should parking professionals use to measure and monitor the accuracy of the latest generation of APGS systems?

Let’s start by reviewing the two main classes of camera-based APGS systems. We will then examine best practices for assessing the accuracy of each and identify potential sources of inaccuracy.

In-motion Camera-Based APGS

These systems use a real-time video stream from a camera to track vehicles crossing designated “count points” such as entrance lanes, exit lanes, and ramps. The video stream is analyzed by image processing software which determines the direction of vehicle travel and the number of vehicles passing the count point. Certain in-motion counting systems also offer the ability to classify the type of vehicle such as truck vs. car vs. motorcycle. These systems essentially substitute a camera sensor for what was done traditionally with embedded magnetic loops or infrared/ultrasonic overhead in-motion sensors. However, it is important to understand that in-motion camera-based APGS still use the “count accrual” method of determining the occupancy of a facility or garage level, just like more traditional sensor systems. The in/out or up/down counts at each count point are accrued or aggregated to calculate the overall parking occupancy. As a result, high accuracy at each count point is very important to achieve because counting errors can accrue over time, eventually rendering the system count unreliable and useless.[1]

The accuracy of a properly installed and calibrated in-motion camera-based APGS at each individual count point should be >99% over a large sample size of counted vehicles during a typical day, including peak traffic periods. Because count points must be aggregated and errors may accrue, it is to be expected that the overall system accuracy will slowly degrade over time. For a system in which each individual count point is >99% accurate and assuming some error cancellation occurs, a reasonable expectation is that the system accuracy may decline to around 85% over a 30-day period without manual intervention such as a recount.

Camera-based Single Space APGS

These systems use a camera that senses the occupancy status of a given parking stall “at rest”; that is, when a vehicle has occupied the space and is now still, or a vehicle has recently left the stall, or the stall was never occupied. This type of APGS, while generally more expensive than an in-motion camera-based APGS, provides significantly more granular space occupancy information and can drive overhead space indicator lights guiding motorists visually to open stalls. From an accuracy perspective, single space systems generally offer higher accuracy than in-motion counting systems for two reasons:

• Accurately detecting vehicle presence is generally easier at rest than in motion.
• Errors are limited to specific stalls and do not accrue across the entire parking garage.

The second point is very critical in helping single space systems achieve high and stable accuracy. However, it is not an assurance that a camera-based APGS is always accurate. Everyone who has utilized a garage with such a system has undoubtedly run across some stalls that are tagged as empty when they are full or vice versa.

A properly installed and calibrated Single Space APGS should achieve accuracy >99.5% and should remain stable over an extended period with no accrual of errors.

So How Should APGS Accuracy Be Assessed?

The following are some best practices that are recommended for camera-based APGS systems of both types.

For Camera-Based Systems

An extremely powerful tool of camera-based systems is the ability to compare direct observation with stored or streamed camera images. Many systems from the leading camera-based APGS vendors allow this capability to be used during installation phase calibration and over time for system monitoring or troubleshooting. Vendors should be willing to make such camera images available to their customers or consultants. The images should show a clear view of the counting point or parking stalls as well as a super-imposed “system count”; that is, what the image processing software believes it is counting. Such images or videos captured over a long period of time can be examined remotely using tools that allow accelerated retrospective viewing. Using this method, it is relatively simple to create a tally of actual observed vehicle counts vs. system vehicle counts, compare the two, and even detect underlying scenarios that may be causing inaccuracies.

Contrast this approach with the accuracy checking methods that must be used with more traditional APGS systems like loops and infrared/ultrasonic sensors. These systems require manual tests, observations and data recording while present at the customer site, which can be very time-consuming, resource intensive and costly.

For In-Motion Systems

Start the accuracy assessment process by checking the accuracy of each individual “count point.” These points correlate to the locations of the in-motion counting cameras. Use the camera stream to record a large amount of count events (usually several hundred data points per count point) to ensure statistical validity. Record data at various times of day that offer varying traffic loads, lighting conditions, and presence of foreign objects such as pedestrians. Replay the recorded videos and observe and tabulate the system count vs. actual count as seen with your eyes. Continue and repeat this testing process until each count point achieves >99% accuracy.

Now, begin a system wide accuracy check. Over a period of days and weeks, monitor the accuracy of the entire parking facility and the parking levels or zones, if a level/zone counting system has been installed. Every few days, take a physical count of the facility and compare it to the count shown on the system’s signs and/or on-line administrative interface. It is most practical and easiest to conduct the physical count off-hours when there are few vehicles in the garage and little to no vehicle movement into, out of, or within the garage. Calculate the accuracy percentage and track it over time to see if it remains stable and within a reasonable goal such as 85% over a 30-day period. Ensure that the APGS vendor has not activated any “automatic count resets” during this test period, as those will invalidate the test results.

For Single Space Systems

The methodology for accuracy assessment in single-space systems is somewhat different from in-motion systems. It still relies on the ability to record and retrospectively review camera images. But in this case, they are still images of the parking stalls used to ascertain if they are correctly being detected as occupied or vacant. The leading single space APGS vendors have established procedures that they use both during calibration and accuracy testing which employ comparing the system-reported stall status vs. actual stall status seen in the camera image, initially for all stalls and over time for a statistically valid subset of the stalls. The accuracy should converge to >99.5% and then remain stable over time. Check with your vendor and ask for their detailed testing process. Audit the results by requesting the detailed test information and stall camera images.

In addition to that testing protocol, conduct spot checks of the parking garage looking for incorrect red/green indicator lights. Do this at both idle and very busy times in terms of garage occupancy. If you witness an error on a particular stall or set of stalls, take a picture of the stall(s) that have the error including a view of the sensor/indicator light, and also record the sensor number which is generally labeled on the bottom of the camera sensor. Report these details to the APGS vendor and ask for an analysis and correction of the issues.

What Can Cause Camera-Based APGS Inaccuracy?

Experience has shown that the following sorts of issues may cause inaccuracies in camera-based APGS systems. These are presented roughly in the frequency of occurrence, based on Walker Consultants’ experiences.

• Improper installation or calibration of the cameras or count zones.
• Unusual counting zone traffic patterns for in-motion systems.
• Unusual stall striping orientation for single-space systems.
• Impact of unusual or high-contrast lighting conditions.
• Network wiring issues causing camera image deterioration.
• Poor quality sensors or inferior image processing software.

The underlying causes can often be gleaned from a combination of direct observation and review of recorded camera images.

Conclusion

Walker Consultants applies best practices when assisting clients in specifying, assessing and testing the accuracy of APGS that were previously installed, are retrofitted into an existing garage, or are installed as part of a newly constructed parking facility. We also can assess the underlying causes of inaccuracy and recommend plans to improve it. By applying the knowledge and best practices as outlined in this article, customers, consultants and vendors can increase their confidence in answering the crucial question posed at the beginning of this article: “Do You Know How Accurate Your Parking Guidance System Is?”

About the Author

Based out of Walker’s San Francisco office and as a member of Walker’s Consulting Resources Group, Peter Filice covers Northern California and the Pacific Northwest with expertise in parking technology solutions across multiple vertical markets — municipal, university, commercial garages, hotel, event and stadium, on-street, mixed-use, and airport. In particular, Filice has deep expertise in Automated Parking Guidance Systems (APGS).

[1] In fairness, counting errors can cancel one another (e.g. an overcount of 1 vehicle cancels an undercount of 1 vehicle), but in real-life system experience this does not always occur. Usually no more than 50% of errors cancel.

---

Any photos provided by APGS vendors are used with permission and are for illustrative purposes only. Inclusion of these photos does not constitute an endorsement of these vendors.