Although this should be in its own research doc, here’s a summary of two separate things I’d like to get done with model trains image detection. There’s some overlap yet they are different.

The first research subject is automated mapping: given a static image, find all tracks in the image, and create a visual schematic map of the network. There are two possible representations:

• Tracks form single paths (both lines and curves) with node intersections matching the frog of turnouts.
• Track segments -- lines, curves, and turnouts.

The first part of the problem is finding tracks in an image.

• A track is defined as two parallel rails with perpendicular ties in the middle. An additional constraint is that rails always have the same distance. Some obvious complications arise when we have parallel tracks. Outlier cases such as heavily weathered tracks where ties are invisible could be omitted. The outlier case of road crossings should be optionally treated though.
• Turnouts may be more difficult to detect. One approach is consider them the intersection of tracks, using a pure path/node approach. Turnouts geometry can be complex (e.g. double slip, X crossings). A first approach could just detect the “typical” simple turnout cases -- yard ladders, single sidings, and simple crossovers.
• Elevation is expected to be a problem, as detailed below.

The second part of the problem is to be able to map a layout by taking a series of “aerial” pictures, mapping track on each picture, and then merging them by detecting the edge overlaps.

• One simplification is to get images that are as much “from the top” as possible. Some kind of perspective should be expected and corrected. However this erases elevation and would make it nearly impossible to detect.
• Eye-level images would not be ideal as perspective is distorted and can make tract analysis a lot more difficult. There might be some compromise e.g. 45 degree view that would make the track still visible yet elevation possible to account for.

The second research is detecting trains on the track.

There are two axes for this: simplified for block detection, or fully automated.

Simplified mode:

• User draws / manually places markers on the image delineating block limits. Can include both linear track and curved portions. Interactive/visual confirmation is needed.
• Some level of automated calibration is desired e.g. if camera/track moves (specific to my home segment test tracks).
• Block detection can use a simple image diff algorithm detecting motion on the specified blocks.Stopped trains become invisible.
• Block detection can rely on a diff algorithm by comparing the track vs an “idle” state registered e.g. when the system starts.
• In all cases, we should be able to detect start/end parts of a train with respect to the defined block.

Fully automated mode:

• Relies on the previous research to detect track in the image.
• User still needs to map block boundaries, but this time against the schematic track.
• Assuming we have an algorithm that detects tracks by detecting parallel rails and ties in between, block detection can be done by running the algorithm again on known block boundaries and checking where track cannot be detected because a train hides the track.
• We should be able to detect start/end parts of a train with respect to the defined block.

Latency and analysis time become interesting. The automation goal is to be able to not only detect when blocks are occupied, but also to stop trains. For example, detect a train is arriving in a station block, slow it down, and then use the start/end occupancy feedback to stop it where desired.

Related CV keywords:

• Background segmentation.
• Distance finding via the “triangle” algorithm.

A third potential research is being able to identify trains. That part is a lot more open ended:

• Track-level camera and reading text of cars/engines going by.
• Obvious question of speed making the image fuzzy.
• Aerial view and color/pattern matching known cars/engines top views.
• This would be more of a fuzzy detection / only detecting classes.

The idea here is to have dedicated “detection” hotspots (e.g. entrance/exit of yards) that can detect some engine numbers. Then some semantic model associates the presence of a track somewhere to that detected number based on continuity as the sole logic predicament. This only fails if e.g. trains are added/moved manually.