Evaluation

Evaluation is critical because digital communities comprise highly interconnected subsystems with complex dependencies. Interactions across components can produce behaviors that are difficult to predict during operations. Evaluating technologies in realistic settings enables researchers to validate effectiveness, identify unintended consequences, and understand system-level impacts before deployment. It also ensures that security solutions remain robust under real-world conditions, where scale, heterogeneity, and operational constraints introduce additional challenges.

Thrust 4 advances this vision by bringing together two complementary efforts: (1) a living lab security architecture for evaluating technologies safely, realistically, and at system scale, and (2) methods for evaluating the security of digital communities through continuous monitoring, distributed data collection, and vulnerability assessment. The three core capabilities developed in Thrust 4 are:

• A scalable security monitoring platform that provides real-time visibility into network activity, device vulnerabilities, and potential attack paths, enabling informed and timely operational decisions.
• A Graph Convolutional Network (GCN)-based method integrated with real-time monitoring to identify and rank critical assets whose compromise could have the greatest impact on the wider system.
• An adaptive anomaly detection approach that combines Temporal Convolutional Networks (TCNs) with variational autoencoders (VAEs) to distinguish between normal operational drift and malicious activity, allowing the system to adapt while maintaining resilience.