In the old Conductor 1, anytime the train sequence did not work as expected, the engine simply stopped everything, and we had to manually replace the trains as expected and restart everything.
Thus, when I redesigned the new Conductor 2, I added some “recovery” functionality -- the idea is that if the train sequence does not happen as expected, the automation engine should do something to try to recover the trains.

Unfortunately, the idea of “recovery” is basically to more or less predict the unpredictable and decide what the automation program should do in that case. Which means there are always cases that one never thought about because that “could never happen” and thus such cases are not handled. Today is one such case.

Orion and I have been slowly working on fixing the Branchline T324 turnout. That involves crawling through one of the mountain access holes, and then we need to reach the Branchline over the Mainline track. To make it more fun, that’s one of the tracks used by the Passenger train automation.

Twice I’ve seen the Passenger train go into error mode while we were trying to work on the Branchline T324 turnout. What was even more puzzling is that the Passenger was at Summit and then the Freight train just… started! Suddenly, on the same track, I had the Passenger train go into recovery and thus reversing towards the station whilst at the same time I had the Freight train leaving the station… Each going towards each other in the opposite direction. Not exactly a good scenario.

Hilarity ensues.

The first part of the puzzle was easy to find: when we crawl into the mountain access hole, we somehow activate the block B340. It’s not exactly clear why and how that happens, yet that sure is the result.

Here’s an excerpt from conductor-log-2024-10-12-08-40-03.txt:

This happened once, when Allen and Jim ended their Saturday session:

• At 1:40pm, they turned the "Saturday mode" switch off, turned the Mainline Automation toggle on. Consequently, the Freight train correctly moved back to its normal position.

So far that seems normal. However...

• At 1:42pm and 1:43pm, the computer registered multiple on/off instances of block detection on Block B330 -- there were about a dozen on/off activations for 2 seconds -- but also B360 registers as on -- see below.
• Cameras show they were not running anything at that time, and that block B330 is clearly empty. I cannot see B360 however nothing on the video indicates there should be anything on it (no trains going in or out in the following minutes).
• Camera shows Allen moving trains to Richmond via B160/B150 a minute later. I don’t think it’s related.

Here’s an excerpt from conductor-log-2024-10-12-08-40-03.txt:

⇒ This starts just after the Freight train has reached its default parking location at the Station:
13:42:34.779 R Idle Mainline #0 ML Ready : ACTIVE

13:42:35.972 S S/NS773 B330 : ON

13:42:36.937 S S/NS773 B330 : OFF

13:42:37.833 S S/NS773 B330 : ON

13:42:38.962 S S/NS773 B330 : OFF

13:42:39.659 S S/NS773 B330 : ON

13:42:40.639 S S/NS773 B330 : OFF

13:42:41.520 S S/NS773 B330 : ON

13:42:42.416 S S/NS773 B330 : OFF

13:42:43.348 S S/NS773 B330 : ON

Orion is working on a project: creating an internal intercom system for the Randall Museum Model Railroad.

One of his requirements is to make the phone rings using their internal ringer, without modifying the phones, or without using the easy solution of simulating the ringer using an arduino+speaker setup.

That means we need to simulate the POTS signaling for the ringer on his closed phone line: such phones ring when they receive a signal 90 V AC at 20-40 Hz:

• https://www.jkaudio.com/article_10.htm 
• https://pbxbook.com/other/trunks.html (fascinating details including railroad origins)

Orion and I looked at his BlueTooth phone adapter -- the ring does provide a 90~100 V signal at about 20 Hz. We checked that with a real old-school scope. One thing I did forget to measure was the current draw, doh. But since the adapter is powered via a regular USB brick, it can’t be more than 5 W (1A x 5V).

Thus the goal is to make some circuitry, the simplest possible, that will generate a 90~100 V DC square signal switched at 20 Hz. And, as an extra precaution, I want this circuitry to be isolated from the power grid.

The Software Defined Blocks project uses an ESP32 with sensors to emulate block activations for a train model railroad.

The first prototype automation test has been overwhelmingly problematic:

• At first, I could not get the ToF sensor to reliably detect the engine. It would seem to work, then most of the time it would not detect a decreasing distance as the trolley was approaching till it was right next to the sensor.
• At the same time, my mere presence a couple feet on the side would influence the result. That’s not good.
• The response time in JMRI seems extremely slow.

Let’s detail that last one.

The Software Defined Blocks project uses an ESP32 with sensors to emulate block activations for a train model railroad.

I brought my prototype board to Randall today and gave it a try with the desired trolley:

The SDB prototype detected the trolley properly. When placed this way against the end bumper, we can adequately measure the trolley distance between 10 mm and 400 mm (1 cm up to 40 cm). After that, we always get a value in the 400 range.

Note that in this test, I did not hook SDB to the main JMRI computer for actual control. The goal was to determine which sensor placement would fulfill the detection need. Whatever is on the picture above seems like it will work correctly, so I’ll go ahead and create some kind of sensor support -- probably out of foam/cardboard first and then later 3d-printed or stuck to a little building.

Here’s an overview of the track I want to control:

The Software Defined Blocks project uses an ESP32 with sensors to emulate block activations for a train model railroad.

The initial design called for the use of two ToF sensors, one on each side of the piece of track to automate. That would allow the automation to precisely stop the train on either side. The piece of track I want to monitor is about 3 meters long.

I selected the Adafruit VL53L0X “Time of Flight Distance Sensor” for this application.

The specification lists the sensor as having a 30 mm to 1 meter range. The idea was thus to have 2 sensors, one on each side of the track to monitor, thus creating 3 “virtual blocks”: each sensor would monitor about 1 meter of the track, and if the engine is not detected there, we would assume it would be in the middle.

Unfortunately, as soon as I tried the sensor, I realized that it does not work for this specific application.

The Software Defined Blocks project uses an ESP32 with sensors to emulate block activations for a train model railroad.

The initial design calls to use two ToF sensors, one on each side of the piece of track to automate. That would allow the automation to precisely stop the train on either side.

The sensor I use is the Adafruit VL53L0X. It connects on the I2C bus at a specific 0x29 address. However, it is possible to change the address in software: one of the sensor’s pins named XSHUT is to “shutdown” the sensor. The procedure is to connect both sensors on the I2C bus, and have the two sensors’ XSHUT pins connected to different GPIO pins. This makes it possible to selectively turn them on at boot, changing their address one by one.

The Shay running at Randall after DCC conversion.

I recently got a Bachmann Spectrum Shay -- the “80-ton 3 truck Shay” #81901. The one I got is an unlettered black version, used, and in good condition. All gears work. It has an 8-pin NMRA / NEM652 connector, and it screams to be converted to DCC so we’ll discuss this effort here.

The Bachmann Shay before DCC conversion, as unboxed.

Initial Observations

Here are the initial observations:

In the latest version of Conductor 2, I’ve added “end-of-route statistics”. I’ve now accumulated enough data to start looking at it.

First, I needed to parse a JSON log file (a log file with one JSON entry per line per day per run), and I needed a way to convert that to CSV to dump it in a Google Spreadsheet. I briefly considered writing a short python script to do that, then realized that nothing can beat the awesomeness of sed, so I ended up with this:

I regret nothing. <insert appropriate meme>

I shall note that this expression purposely only selects log lines that have no errors, that is, successful runs that have completed with no issue. This will become relevant later.

Now let’s look at the collected data.

Freight Route

This is a simple shuttle route over two blocks: we’re moving from block B311 (the Stockton Station) to B321 (the Sonora Station), stopping there, and then back again to B311. The way I wrote the route stats aggregator, I add an index number when a block is repeated, so “B311.1” means “block B311 when leaving”, and “B311.2” means “block B311 when coming back”.

I have 98 runs for this one, and when we compute the stats we get a very nice result:

The latest fix for Conductor 2 using a virtual block for B504 is now installed and has been working very well.

The latest change I did to Conductor 2 is to compute end-of-route statistics. Each time a route has finished, we log the time spent on each block as well as the error status of the route. This is sent to a little server over a basic JSON RPC, and in the statistics/status page we get a quick overview of the latest route -- how many times we ran that route during the day, when the last one finished, and how many seconds the train spent on each block:

This also keeps track of all the recovery attempts.

Having the time per block should help me fine-tune the automation script later. We can see the time spent on B321 for example, and directly relate that to the observations made in the previous post.

Finally I also have the info in the server log, which is rotated daily, which means I could compute statistics later. I’m interested in getting the time spent on each block for long term analysis. Can we get trends out of it? How consistent are these running times? For example, could we detect erratic behavior, or do engines suffer some kind of loss of performance before they stop working?