IMPROVING THE RELIABILITY OF RELEVANT RAILWAY SIGNAL DETECTION BY COMBINING ARTIFICIAL INTELLIGENCE AND CLASSICAL COMPUTER VISION METHODS

Abstract

Transferring visual monitoring tasks from the driver to onboard systems is an important research direction for autonomous railway operation. Reliable recognition of wayside signals could support drivers and, following further development and safety validation, contribute to driverless operation on conventionally signalled routes. This requires more than recognising a light’s colour: the system must identify the signal that applies to the train’s own track and provide a timely, stable interpretation. The task becomes more difficult when several tracks and signals are visible in the same scene, particularly under varying lighting conditions. This paper presents a hybrid framework combining deep learning, classical computer vision and temporal verification. TEP-Net estimates the train’s ego-path, supported by Canny edge detection and Hough-based geometric checks. In the proposed configuration, YOLO detects red, yellow and green signal lights across the full image. The estimated ego-path defines a narrower region of interest for additional candidate detection using histogram of oriented gradients features and logistic regression, and a wider region for assessing signal-to-track association. After candidate matching, local colour and shape checks complement the detector outputs. Signal evidence, track quality and spatial evidence are combined to score each candidate. Candidates are tracked across successive frames before confirmation. When the evidence is insufficient to distinguish between them, the system leaves the choice unresolved rather than selecting a signal solely on the basis of its detection score. Digital infrastructure maps combined with train localisation are considered as a future extension to support verification of the estimated track and association of detected candidates with expected signals. The experimental protocol uses two datasets from railway environments with different infrastructure, terrain, signalling and vegetation. Each includes nested training sets of 500, 1,500 and 3,000 images, a fixed 300-image validation set and separate video test sequences. Planned comparisons between the detector alone and different fusion configurations will assess detection accuracy, incorrect signal-to-track association, missed restrictive signals, temporal stability and decision delay. The intended contribution is a modular framework whose decisions can be examined through individual module outputs, together with a reproducible procedure for evaluating its benefits and limitations.