A new evaluation framework promises to capture the full value of security technology systems across their lifecycle, meeting stakeholder needs and driving innovation
System evaluation is a critical phase in the life of any technology. It guides development, procurement, deployment, and continuous improvement by helping decision-makers understand which parts of a system are worth enhancing, replacing, or investing in.
At its core, evaluation ensures that solutions deliver real value and meet the expectations of stakeholders. This becomes especially important in the case of surveillance systems, which are among the most advanced and complex software-driven technologies in use today.
“Their sophistication means that evaluation is not just helpful, it is essential to ensure these systems are developed to their best advantage and provide maximum value. Video surveillance offers one of the most interesting and challenging cases for such evaluation,” says doctoral researcher Aleksandra Karimaa.
Unlike typical data systems, its performance is tied to inherently ambiguous and qualitative nature content and factors, such as for example video and its quality. In example of video quality-focused evaluation, while resolution or signal properties can be measured easily, capturing deeper aspects like perceptual quality, environmental conditions, or the system’s ability to detect specific objects and movements require more complex measurements.
Complex measurements might be not practical to many system developers and customers, which means different stakeholders will evaluate system’s video quality and the overall value of the system differently. They draw different conclusions and will prioritize its improvements differently.
“In cases like this the developers, customers, end-users and researchers might not work together close enough to create innovations that makes practical sense. Additionally, when focusing only on selected aspects of evaluation, such as video quality, we miss holistic view to the system and therefore we risk overlooking how these video quality improvements might increase architectural complexity, costs, or maintenance burdens, ultimately compromising the system’s overall value,” Karimaa says.
In her dissertation, Aleksandra Karimaa addressed this challenge by developing a holistic evaluation framework. She introduced a metrics-based scoring system that allows video surveillance technologies to be assessed in a structured and transparent way. The framework combines measurable indicators into a scoring model that reflects technical performance, usability, and stakeholder priorities across the entire lifecycle of a system.
Karimaa demonstrated the framework through case studies, showing how metrics can be applied to capture system properties and overall performance. Her research highlights how this approach can support innovation by providing a practical tool for evaluating new solutions from business, technology, and research perspectives.
“The goal was to create a system that captures the full value of surveillance technologies, not just one aspect of performance,” explains Karimaa. “This framework offers a common evaluation language that helps industry, researchers, policymakers, and innovators work together more effectively. And while video surveillance is the clearest example, the same approach can be applied to other complex systems where evaluation is equally challenging.”
With its holistic approach and broad applicability, the dissertation offers a valuable resource for both industry and academia. It provides a way to improve collaboration, accelerate innovation, and speed up the commercialization of new technologies, not only in video surveillance, but across the wider landscape of complex security and defence systems.
Dissertation defence on Friday 5 December
MSc, MBA Aleksandra Karimaa defends the dissertation in Information and Communication Technology titled “On Evaluating Video Surveillance Systems” at the University of Turku on 5 December 2025 at 12.00 (University of Turku, Publicum, Pub3, Assistentinkatu 7, Turku).
Opponent: Associate Professor Jussi Kasurinen (LUT University, Finland)
Custos: Associate Professor Tuomas Mäkilä (University of Turku)
