This file was created by the TYPO3 extension bib --- Timezone: CEST Creation date: 2024-07-04 Creation time: 00-20-25 --- Number of references 2 article 2023_lamberts_metrics-sok Journal of Systems Research 2023 10 31 3 1 Industrial systems are increasingly threatened by cyberattacks with potentially disastrous consequences. To counter such attacks, industrial intrusion detection systems strive to timely uncover even the most sophisticated breaches. Due to its criticality for society, this fast-growing field attracts researchers from diverse backgrounds, resulting in 130 new detection approaches in 2021 alone. This huge momentum facilitates the exploration of diverse promising paths but likewise risks fragmenting the research landscape and burying promising progress. Consequently, it needs sound and comprehensible evaluations to mitigate this risk and catalyze efforts into sustainable scientific progress with real-world applicability. In this paper, we therefore systematically analyze the evaluation methodologies of this field to understand the current state of industrial intrusion detection research. Our analysis of 609 publications shows that the rapid growth of this research field has positive and negative consequences. While we observe an increased use of public datasets, publications still only evaluate 1.3 datasets on average, and frequently used benchmarking metrics are ambiguous. At the same time, the adoption of newly developed benchmarking metrics sees little advancement. Finally, our systematic analysis enables us to provide actionable recommendations for all actors involved and thus bring the entire research field forward. internet-of-production, rfc https://www.comsys.rwth-aachen.de/fileadmin/papers/2023/2023-lamberts-metrics-sok.pdf eScholarship Publishing 2770-5501 10.5070/SR33162445 1 OlavLamberts KonradWolsing EricWagner JanPennekamp JanBauer KlausWehrle MartinHenze inproceedings 2023-wolsing-xluuvlab 2023 10 Roughly two-thirds of our planet is covered with water, and so far, the oceans have predominantly been used at their surface for the global transport of our goods and commodities. Today, there is a rising trend toward subsea infrastructures such as pipelines, telecommunication cables, or wind farms which demands potent vehicles for underwater work. To this end, a new generation of vehicles, large and Extra-Large Unmanned Underwater Vehicles (XLUUVs), is currently being engineered that allow for long-range, remotely controlled, and semi-autonomous missions in the deep sea. However, although these vehicles are already heavily developed and demand state-of-the-art communi- cation technologies to realize their autonomy, no dedicated test and development environments exist for research, e.g., to assess the implications on cybersecurity. Therefore, in this paper, we present XLab-UUV, a virtual testbed for XLUUVs that allows researchers to identify novel challenges, possible bottlenecks, or vulnerabilities, as well as to develop effective technologies, protocols, and procedures. Maritime Simulation Environment, XLUUV, Cyber Range, Autonomous Shipping, Operational Technology https://www.comsys.rwth-aachen.de/fileadmin/papers/2023/2023-wolsing-xluuvlab.pdf IEEE 1st IEEE LCN Workshop on Maritime Communication and Security (MarCaS) Daytona Beach, Florida, USA 1st IEEE LCN Workshop on Maritime Communication and Security (MarCaS) Oktober 1-5, 2023 accepted en 10.1109/LCN58197.2023.10223405 1 KonradWolsing AntoineSaillard ElmarPadilla JanBauer