The place where random ideas get written down and lost in time.
2017-10-27 - Mesh Networks
Category DEV- ESP8266 (wifi / mesh, e.g. library easyMesh)
- NRF24L01 (in the 2.4 GHz band)
- RFM69 (< 1 Ghz, from last thread here)
2017-09-24 - State of the Onion
Category DEVCurrent projects or ideas and their status, more or less from higher to lower priority:
- Randall: Setup the linux box to auto-start with JMRI faceless and my controller script.
- High priority, EOM / EOY.
- A sub-project is having a non-authenticated display on the laptop, which could be either in the form of a screen saver or a non-privileged user in auto-login running a single full screen app that can't be closed. Ideally: EOY. Useful for me.
- Randall: RTAC android app for tablet.
- High Priority MVP by EOM / EOY on at least one tablet. Can be fine tuned later.
- Home Monitor:
- Redo circuit board for alarm interface + box. Still want it. Low-priority because works well and kinda like the hackish look.
- Current RPi setup works well.
- Monitor android app using Firebase is meh ⇒ Rebuild without Firebase, direct App Engine web + Android w/ notifications.
- Cab Engineer:
- A v2 of Cab Throttle. Rewrite app using my RX instead of event bus.
- MVP should have server save/recall, engine save/recall, soft-consist, wear.
2017-08-12 - Simple Rx
Category DEVI should try to make my own little Rx clone lib.
There are a few good things in the RxJava lib but it also has these issues:
- It does too much for my needs.
- On paper, and when viewing a few first tutorials, it's all good. But then it breaks down. There's RxJava 1 vs 2. The vocabulary confuses me to no end. There's too much of an initial learning curve for what looks like a simple concept on the paper.
- Example of vocabulary confusion:
- An "observable" is actually a stream of events. Then call it a stream.
- "Observers" actually subscribe to a stream. Then call them subscribers.
- A source or an emitter is the same thing.
- A consumer is an observer (subscriber).
- Then RxJava 2 adds its own confusion by having Flowable vs Observable, and their version of the Observer is now called a Subscriber (as it should).
- RxStream uses even different names: Publisher, Subscriber, Subscription, Processor.
- The issue with this is the same as with languages like Ruby and Go: They are "almost" easy to get but yet different. When I move away from one for 6 months or a year, I need to go through the ropes again to get familiar with what I wrote a year ago. A "good" framework is something that is intuitive for me and I can just re-read my old code without wondering what it means or why I wrote it that way.
I'll use the RxStream terminology:
- A Publisher is the source, emitter, generator. It takes a stream and publishes new events onto it.
- A Subscriber is the reader, consumer, observer. It takes a stream and reads from it.
- Schedulers (from RxJava) express which thread a publisher or subscriber runs onto.
- A Stream is a pipe that sends events from the publisher(s) to the subscriber(s). It can be a 1-1 or a N-N combination.
- Streams can be created empty and publishers and subscribers added to them in any order.
- Streams are either open, pause or closed (aka "completed").
- Closing is final. Once closed, publishing generates an error.
- Subscribers can pause the stream, which blocks subscribers.
- Processors are stream filters/maps. They take one or more streams and combine events together into a new output stream.
- Example: map(lambda X ⇒ Y), take(N), delay(N), merge().
- Subscribers are notified when publishers are added or removed.
- Subscribers are notified when publishers pause the stream.
- Publishers are notified when subscribers are added or removed.
- Streams are by default asynchronous and multithreaded.
- When there are multiple publishers or subscribers, they could all be on different schedulers.
Canonical examples:
S = Stream.create() # can be exported via dagger
Or
@Inject Stream S;
S.pause(true);
S.addSubscriber( … ).on( Scheduler );
S.addPublisher( … ).on( Scheduler );
S.addProcessor( map( … ) ).on( Scheduler );
S.pause(false);
S.isPaused();
S.close();
Async task:
Stream.on( Scheduler.io() ) # sets default scheduler unless overridden
.publish( fixed data, e.g. some url )
.publishOn( Schedulers.io() ) # affects the last added publisher
.map( lambda worker: url ⇒ http request ⇒ body response )
.processOn( Schedulers.io() ) # affects the last added processor
.subscribe( subscriber lambda )
.subscribeOn( Schedulers.androidMainThread() ); # affects last subscriber
[Update 2017-08-20]
Once this is implemented, I have one issue: should users be able to publish directly on the stream, or always via subscribers?
Example:
_pub = MyCustomPublisher()
_stream = Stream.on(Schedulers.io())
.publishWith( _pub )
.subscribe( SomeSubscriber() );
_pub.publish(42);
_stream.publish(43);
Now we have 2 ways to inject events in the stream: via the publisher and via the stream directly.
The latter overrides whatever behavior from the publisher: we might have used a Just() publisher that only provides its constructor arguments yet we can still add more.
In essence, the publisher does not control whether the stream is "read-only".
So a different strategy is:
- Only Publishers can publish on a stream.
- Users that hold a stream can't publish directly.
Furthermore, we're reducing the scope of each objects:
- Stream methods only serve to configure the stream itself.
- Publishers decide if they expose a public publish method.
- When they don't, they become closed generators.
2017-08-12 - Cab Throttle v2
Category DEVI'll reorganize the Cab Throttle project under this structure:
- A WiThrottleLib repo (destined to maybe open source later).
- A Cab Throttle v2 repo. The app itself.
Part of the app such as the server list and the default throttle view should go in the lib.
The app repo will keep things that are branding specific. For example if I want to extend the default view with plugins, that goes in the app.
The lib provides hooks that the app extends.
One core change is to move away from the event bus structure and go directly to daggerization.
2017-08-11 - Cab Throttle
Category DEVApp on G Play: "DCC Cab Throttle"
Non-goal: Not trying to compete with Engine Driver. Invariably people will compare it to that. Let it be. Do what's useful to me.
What’s left, missing requirements for an ideal 1.0:
- Server list needs startup hints.
- "No JMRI server found. Automatic server discovery has started."
- Plus some bullet points: Make sure one is connected to the same wifi network as the JMRI server /or/ Use the + button to enter a server address manually.
- Remember recently used servers. Recall + delete.
- Suggestion: "star" to memorize a server, or make it automatic?
- Suggestion: tie server to wifi network.
- Auto-connect feature (if not connected and on same wifi network).
- Edit server button: Full page edit server ⇒ change name, IP/port, star/unstar, wifi network, auto-connect, delete entry.
- Remember recently used engines. Recall + delete.
- Q: Per server or across servers? Should it be an option?
- (v2) Export / import servers or engines list.
- "soft" consists.
- F to lead or all.
- Replace F icons. They suck.
- Replace program icons (wear, trash, wifi)
- Different view modes. Look at iOS, people like its clunkiness.
- Definitely consider having easily switchable views. Full-screen mode with themes.
- (v2) user plug-in views.
- (v2) Settings to remap F icons to different F numbers.
- Create profiles and then associate profile to dcc address.
- Provide a few default profiles (eg MTH, Athearn, Bachmann)
- (v3) experiment with plug-ins for profiles, comm, views.
- (v3) Ability to use F > 9
- Or instead of 1..9, select which 8 numbers to display.
Code wise:
- Reboot project using structure from RTAC.
- Separate WiThrottleLib in its own open repo.
- Optional open-source cab throttle except branding? ⇒ split into an app lib.
2017-05-23 - RxJava
Category DEVStill trying to wrap around the basics.
Some high level vocabulary:
- Observable = iterator == the stream.
- Source / emitter == produce values (into an Observable) via onNext / onCompleted.
- Observer = consumer == callback (onNext / onError / onCompleted).
- "Subscribe to an observable using an observer".
⇒ This is my main grip with RxJava: the vocabulary is nonsensical and confusing. It means as soon as I stop using it I will get confused when I get back to it later.
The flow in a nutshell:
(emitter) ⇒ Observable ⇒ Observer[s].
But in fact it is implemented as such:
Observable.onSubscribe(emitter code) ⇒ Observer/Subscriber(receiving code)
with various intermediate steps in the stream.
A Disposable/Subscription represents the link from an Observer/Subscriber to an Observable/stream.
Core to remember:
Observable.{from|create}(source) ---> the observable
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(an observer).
RxJava2 has Observable vs Flowable:
- Observable = no back pressure.
- Observer type: #onNext / #onComplete / #onError / #onSubscribe(Disposable).
- Disposable: #dispose.
- Specialized types: Single, Completable, Maybe (types of Observable)
- Source: fromCallable(() -> return value),
- Source: Observable.create() using a "subscriber" with an "emitter" object.
- Flowable = has back pressure built-in.
- Subscriber type: #onNext / #onComplete / #onError / #onSubscribe(Subscription).
- Subscription: #cancel, #request.
RxStreams:
- Publisher: #subscribe(Subscriber)
- Subscriber: #onSubscribe(Subscription)
- Subscription: #request, #cancel
- Processor<T, R>: converts a Subscriber T into a Publisher R.
A few links:
- The missing intro: https://gist.github.com/staltz/868e7e9bc2a7b8c1f754
- RxJava Android tips: http://futurice.com/blog/top-7-tips-for-rxjava-on-android
Using it to replace an Event Bus:
- http://blog.kaush.co/2014/12/24/implementing-an-event-bus-with-rxjava-rxbus/
- https://lorentzos.com/rxjava-as-event-bus-the-right-way-10a36bdd49ba
Useful videos:
The most obvious pattern is to replace async tasks:
Observable.just( input data )
.map( async lambda, transforming input to output type )
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe( observer using the output )
To replace a timer task:
subscription = Observable.timer(delay, interval, TimeUnit.unit)
.take(20) / map, etc. # for example
.subscribe( observer doing something at interval )
subscription.unsubscribe() # equivalent to timer.cancel
2017-04-21 - RxJava / RxAndroid
Category DEVLooking into RxJava 2 for usage in CabThrottle and derived apps.
RxJava 2 has 2 kinds of streams: "observables" (no back pressure, no blocking) and "flowable" (back pressure, blocking). Still trying to figure the APIs correctly, but let's see where these would be needed:
- Mountain throttle: stream from the hardware throttle to notify of throttle changes.
- CabThrottle:
- NSD/Discovery stream (add/remove).
- Throttle UI to controller.
- Throttle controller to network.
- Network back to throttle controller/UI.
- KeyServer changes.
Some of these were done using the event bus and instead would map nicely in the concept of a stream with subscribers being notified. They are not pull events, mostly push events.
All of these map on the observable (no back pressure / no blocking) pattern.
2017-03-21 - Event Bus
Category DEVCabThrottle uses the new Event Bus design I made up in 2014.
Pros and cons:
- Pro: The event bus implementation is simple and minimalist.
- Pro: There is no annotation processor to find subscribers, so no complicated build setup.
- Cons: It suffers from the typical good/bad part of a disconnected sub-system. The whole point is to remove hard dependencies, but the side effect is that it hides the dependencies. They are still there, just not so visible.
- Pro: The "user" dependencies are simple enough to find. Search for code using any of the bus constants to find the users. Done.
- Cons: The bus messages rely on a numeric ID and the simple naive approach used is to create "namespaces", a.k.a. ID ranges. E.g. 1000 is the base for the Discovery bus messages. But it doesn't say what is the max, and there's no centralized place to look at all the usage ranges (on purpose, because decentralized). That means it's hard to place it in a library and be sure there won't be conflicts down the road with something else.
- Cons: There's some inefficiency in sending messages to subscribers without knowing if they want to listen to them. The receivers can subscribe to message class types, but that's not too useful when for example sending a string notification.
So really the weakness is the ID range issue. Ideally each message would use an enum but the Java implementation of enums kind of sucks -- can't make a "generic" enum, and they are too heavy for serialized number sequences. Another alternative is to use a hash code as a base (e.g. NSD DiscoveryMixin.class.getHashCode() but even that is not guaranteed to not have collisions and it would prevent from making static final int IDs.
It does bring back some potential for dagger.
In usage, I mostly use the event bus for notification, obviously. E.g. NSD discovery mixing sends bus messages when a new host appears or vanishes. Well I can just register a listener and be done with it, since that's essentially what the event bus does. One point of the event bus is that the subscriber could be a view or something that does not have access to the sender object because it would require passing everything via globals or something, as I used to do before. Well that particular problem is well solved by dagger. E.g. there would be a module for the discovery client and whatever code need to use it, be it the activity to start the client or the UI to display its state, can simply access it via injection, then subscribe a listener as needed.
Same goes for throttles for example.
The other realization is that this seems to also fit the Reactive pattern of having observable streams. Each bus is essentially a stream, and users can get the stream via dagger.
Worth looking into it.
2017-03-10 - Dagger
Category DEVIt still feels like shoe-horning to use dagger in my apps (e.g. CabThrottle).
Reactive … Potentially interesting or totally hyped:
- http://reactivex.io/ and https://github.com/ReactiveX/RxJava -- an observer pattern (the "reverse" of a subscriber pattern) to do async operations.
- It has stuff for Android (https://github.com/ReactiveX/RxAndroid).
- Potential: for tasks that involve multiple AsyncTasks (e.g. google play login) or for replacing my event bus. (huh, not really?)
Unrelated:
- Dokan, a user-mode file system for Windows. https://dokan-dev.github.io/
2017-01-01 - CMRS Touch Panels
Category DEV- Option A: use the cheaper Intel “Ubuntu” stick. Fairly good price at $39. Use it as a dumb remote and just display a web page served by the JMRI server.
- Option B: use the beefier Intel Windows 10 stick. $125 or such. Run a C# or Java app that displays an SVG map and communicates with JMRI via the JSON server or the WiThrottle server.
Should look into option A, software side:
- Server side, a dedicated HTTP server. Could be either IIS or a custom Java servlet. The latter is probably easier to deal with, on a custom non-80 port.
- Explore what the JSON server in JMRI can and cannot do. Eg it can serve states, but does it allow performing changes? ⇒ yes, e.g. JMRI\java\src\jmri\server\json\turnout\JsonTurnoutHttpService.java doPut().
- Another option is to do a Java/Jython bridge like used with Conductor. This makes it trivial to start/shutdown at the same time as JMRI and can use the Jython interface to access turnouts.
- On the client side, we just need to display some SVG or bitmaps. Multiple maps can be trivially handled by having different query parameters.
Server side:
- Jetty / JSP
- Restlet
- Swing for UI
- JMRI Jython script dynamically linked to a Java JAR
Client side:
- jQuery
- SVG
- vanilla JS