The IoT has emerged as a significant target for cyber-attacks, particularly with a focus on the routing protocol for low-power and lossy networks (RPL) within Wireless Sensor
Networks (WSNs). These attacks can disrupt network topologies and compromise data transmission. Early detection of routing attacks is crucial, particularly in resource-constrained RPL networks. This study employed a simulated dataset encompassing Hello Flood, Version Number, and Worst Parent attacks to develop a robust detection model for resource-based routing attacks in IoT networks. In this research, a novel cross-layer feature analysis was conducted, identifying 12 key features crucial for distinguishing between normal and malicious nodes within the network out of the 29 features examined. Various machine learning algorithms, including random forest, CatBoost, and extreme gradient boosting, were evaluated for precise classification. The optimized CatBoost model, a gradient-boosting decision tree (DT)
algorithm, demonstrated outstanding performance with a 99% of detection rate, 0.8% of false positive rate, 98% of sensitivity, and 98% of positive predictive values on an independent test
dataset. Furthermore, an advanced intrusion prevention algorithm leveraging cross-layer feature-induced intrusion detection was introduced to effectively combat prevalent routing attacks. This study significantly contributes to enhancing cybersecurity in IoT networks, particularly in smart cities, by offering robust intrusion detection and prevention mechanisms.