I should try to make my own little Rx clone lib.

There are a few good things in the RxJava lib but it also has these issues:

• It does too much for my needs.
• On paper, and when viewing a few first tutorials, it's all good. But then it breaks down. There's RxJava 1 vs 2. The vocabulary confuses me to no end. There's too much of an initial learning curve for what looks like a simple concept on the paper.
• Example of vocabulary confusion:
• An "observable" is actually a stream of events. Then call it a stream.
• "Observers" actually subscribe to a stream. Then call them subscribers.
• A source or an emitter is the same thing.
• A consumer is an observer (subscriber).
• Then RxJava 2 adds its own confusion by having Flowable vs Observable, and their version of the Observer is now called a Subscriber (as it should).
• RxStream uses even different names: Publisher, Subscriber, Subscription, Processor.
• The issue with this is the same as with languages like Ruby and Go: They are "almost" easy to get but yet different. When I move away from one for 6 months or a year, I need to go through the ropes again to get familiar with what I wrote a year ago. A "good" framework is something that is intuitive for me and I can just re-read my old code without wondering what it means or why I wrote it that way.

I'll use the RxStream terminology:

• A Publisher is the source, emitter, generator. It takes a stream and publishes new events onto it.
• A Subscriber is the reader, consumer, observer. It takes a stream and reads from it.
• Schedulers (from RxJava) express which thread a publisher or subscriber runs onto.
• A Stream is a pipe that sends events from the publisher(s) to the subscriber(s). It can be a 1-1 or a N-N combination.
• Streams can be created empty and publishers and subscribers added to them in any order.
• Streams are either open, pause or closed (aka "completed").
• Closing is final. Once closed, publishing generates an error.
• Subscribers can pause the stream, which blocks subscribers.
• Processors are stream filters/maps. They take one or more streams and combine events  together into a new output stream.
• Example: map(lambda X ⇒ Y), take(N), delay(N), merge().
• Subscribers are notified when publishers are added or removed.
• Subscribers are notified when publishers pause the stream.
• Publishers are notified when subscribers are added or removed.
• Streams are by default asynchronous and multithreaded.
• When there are multiple publishers or subscribers, they could all be on different schedulers.

Canonical examples:

        S = Stream.create()  # can be exported via dagger

Or

        @Inject Stream S;

        S.pause(true);

        S.addSubscriber( … ).on( Scheduler );

        S.addPublisher( … ).on( Scheduler );

        S.addProcessor( map( … ) ).on( Scheduler );

        S.pause(false);

        S.isPaused();

        S.close();

Async task:

        Stream.on( Scheduler.io() )                # sets default scheduler unless overridden

                .publish( fixed data, e.g. some url )

                .publishOn( Schedulers.io() )  # affects the last added publisher

                .map( lambda worker: url ⇒ http request ⇒ body response )

                .processOn( Schedulers.io() )  # affects the last added processor

                .subscribe( subscriber lambda )

                .subscribeOn( Schedulers.androidMainThread() ); # affects last subscriber

[Update 2017-08-20]

Once this is implemented, I have one issue: should users be able to publish directly on the stream, or always via subscribers?

Example:

        _pub = MyCustomPublisher()

        _stream = Stream.on(Schedulers.io())

                .publishWith( _pub )

                .subscribe( SomeSubscriber() );

        _pub.publish(42);

        _stream.publish(43);

Now we have 2 ways to inject events in the stream: via the publisher and via the stream directly.

The latter overrides whatever behavior from the publisher: we might have used a Just() publisher that only provides its constructor arguments yet we can still add more.

In essence, the publisher does not control whether the stream is "read-only".

So a different strategy is:

• Only Publishers can publish on a stream.
• Users that hold a stream can't publish directly.

Furthermore, we're reducing the scope of each objects:

• Stream methods only serve to configure the stream itself.
• Publishers decide if they expose a public publish method.
• When they don't, they become closed generators.

---

I'll reorganize the Cab Throttle project under this structure:

• A WiThrottleLib repo (destined to maybe open source later).
• A Cab Throttle v2 repo. The app itself.

Part of the app such as the server list and the default throttle view should go in the lib.

The app repo will keep things that are branding specific. For example if I want to extend the default view with plugins, that goes in the app.

The lib provides hooks that the app extends.

One core change is to move away from the event bus structure and go directly to daggerization.

---

App on G Play: "DCC Cab Throttle"

Non-goal: Not trying to compete with Engine Driver. Invariably people will compare it to that. Let it be. Do what's useful to me.

What’s left, missing requirements for an ideal 1.0:

• Server list needs startup hints.
• "No JMRI server found. Automatic server discovery has started."
• Plus some bullet points: Make sure one is connected to the same wifi network as the JMRI server /or/ Use the + button to enter a server address manually.
• Remember recently used servers. Recall + delete.
• Suggestion: "star" to memorize a server, or make it automatic?
• Suggestion: tie server to wifi network.
• Auto-connect feature (if not connected and on same wifi network).
• Edit server button: Full page edit server ⇒ change name, IP/port, star/unstar, wifi network, auto-connect, delete entry.
• Remember recently used engines. Recall + delete.
• Q: Per server or across servers? Should it be an option?
• (v2) Export / import servers or engines list.
• "soft" consists.
• F to lead or all.
• Replace F icons. They suck.
• Replace program icons (wear, trash, wifi)
• Different view modes. Look at iOS, people like its clunkiness.
• Definitely consider having easily switchable views. Full-screen mode with themes.
• (v2) user plug-in views.
• (v2) Settings to remap F icons to different F numbers.
• Create profiles and then associate profile to dcc address.
• Provide a few default profiles (eg MTH, Athearn, Bachmann)
• (v3) experiment with plug-ins for profiles, comm, views.
• (v3) Ability to use F > 9
• Or instead of 1..9, select which 8 numbers to display.

Code wise:

• Reboot project using structure from RTAC.
• Separate WiThrottleLib in its own open repo.
• Optional open-source cab throttle except branding? ⇒ split into an app lib.

---

Still trying to wrap around the basics.

Some high level vocabulary:

• Observable = iterator == the stream.
• Source / emitter == produce values (into an Observable) via onNext / onCompleted.
• Observer = consumer == callback (onNext / onError / onCompleted).
• "Subscribe to an observable using an observer".

⇒ This is my main grip with RxJava: the vocabulary is nonsensical and confusing. It means as soon as I stop using it I will get confused when I get back to it later.

The flow in a nutshell:

(emitter) ⇒ Observable ⇒ Observer[s].

But in fact it is implemented as such:

Observable.onSubscribe(emitter code) ⇒ Observer/Subscriber(receiving code)

with various intermediate steps in the stream.

A Disposable/Subscription represents the link from an Observer/Subscriber to an Observable/stream.

Core to remember:

Observable.{from|create}(source) ---> the observable

        .subscribeOn(Schedulers.io())

        .observeOn(AndroidSchedulers.mainThread())

        .subscribe(an observer).

RxJava2 has Observable vs Flowable:

• Observable = no back pressure.
• Observer type: #onNext / #onComplete / #onError / #onSubscribe(Disposable).
• Disposable: #dispose.
• Specialized types: Single, Completable, Maybe (types of Observable)
• Source: fromCallable(() -> return value),
• Source: Observable.create() using a "subscriber" with an "emitter" object.
• Flowable = has back pressure built-in.
• Subscriber type: #onNext / #onComplete / #onError / #onSubscribe(Subscription).
• Subscription: #cancel, #request.

RxStreams:

• Publisher: #subscribe(Subscriber)
• Subscriber: #onSubscribe(Subscription)
• Subscription: #request, #cancel
• Processor<T, R>: converts a Subscriber T into a Publisher R.

A few links:

• The missing intro: https://gist.github.com/staltz/868e7e9bc2a7b8c1f754
• RxJava Android tips: http://futurice.com/blog/top-7-tips-for-rxjava-on-android

Using it to replace an Event Bus:

• http://blog.kaush.co/2014/12/24/implementing-an-event-bus-with-rxjava-rxbus/ 
• https://lorentzos.com/rxjava-as-event-bus-the-right-way-10a36bdd49ba 

Useful videos:

• https://www.youtube.com/watch?v=k3D0cWyNno4 
• https://www.youtube.com/watch?v=htIXKI5gOQU 

The most obvious pattern is to replace async tasks:

Observable.just( input data )

        .map( async lambda, transforming input to output type )

        .subscribeOn(Schedulers.io())

        .observeOn(AndroidSchedulers.mainThread())

        .subscribe( observer using the output )

To replace a timer task:

subscription = Observable.timer(delay, interval, TimeUnit.unit)

        .take(20) / map, etc. # for example

        .subscribe( observer doing something at interval )

subscription.unsubscribe() # equivalent to timer.cancel

---

Looking into RxJava 2 for usage in CabThrottle and derived apps.

RxJava 2 has 2 kinds of streams: "observables" (no back pressure, no blocking) and "flowable" (back pressure, blocking). Still trying to figure the APIs correctly, but let's see where these would be needed:

• Mountain throttle: stream from the hardware throttle to notify of throttle changes.
• CabThrottle:
• NSD/Discovery stream (add/remove).
• Throttle UI to controller.
• Throttle controller to network.
• Network back to throttle controller/UI.
• KeyServer changes.

Some of these were done using the event bus and instead would map nicely in the concept of a stream with subscribers being notified. They are not pull events, mostly push events.

All of these map on the observable (no back pressure / no blocking) pattern.

---

CabThrottle uses the new Event Bus design I made up in 2014.

Pros and cons:

• Pro: The event bus implementation is simple and minimalist.
• Pro: There is no annotation processor to find subscribers, so no complicated build setup.
• Cons: It suffers from the typical good/bad part of a disconnected sub-system. The whole point is to remove hard dependencies, but the side effect is that it hides the dependencies. They are still there, just not so visible.
• Pro: The "user" dependencies are simple enough to find. Search for code using any of the bus constants to find the users. Done.
• Cons: The bus messages rely on a numeric ID and the simple naive approach used is to create "namespaces", a.k.a. ID ranges. E.g. 1000 is the base for the Discovery bus messages. But it doesn't say what is the max, and there's no centralized place to look at all the usage ranges (on purpose, because decentralized). That means it's hard to place it in a library and be sure there won't be conflicts down the road with something else.
• Cons: There's some inefficiency in sending messages to subscribers without knowing if they want to listen to them. The receivers can subscribe to message class types, but that's not too useful when for example sending a string notification.

So really the weakness is the ID range issue. Ideally each message would use an enum but the Java implementation of enums kind of sucks -- can't make a "generic" enum, and they are too heavy for serialized number sequences. Another alternative is to use a hash code as a base (e.g. NSD DiscoveryMixin.class.getHashCode() but even that is not guaranteed to not have collisions and it would prevent from making static final int IDs.

It does bring back some potential for dagger.

In usage, I mostly use the event bus for notification, obviously. E.g. NSD discovery mixing sends bus messages when a new host appears or vanishes. Well I can just register a listener and be done with it, since that's essentially what the event bus does. One point of the event bus is that the subscriber could be a view or something that does not have access to the sender object because it would require passing everything via globals or something, as I used to do before. Well that particular problem is well solved by dagger. E.g. there would be a module for the discovery client and whatever code need to use it, be it the activity to start the client or the UI to display its state, can simply access it via injection, then subscribe a listener as needed.

Same goes for throttles for example.

The other realization is that this seems to also fit the Reactive pattern of having observable streams. Each bus is essentially a stream, and users can get the stream via dagger.

Worth looking into it.

---

It still feels like shoe-horning to use dagger in my apps (e.g. CabThrottle).

Reactive … Potentially interesting or totally hyped:

• http://reactivex.io/ and https://github.com/ReactiveX/RxJava -- an observer pattern (the "reverse" of a subscriber pattern) to do async operations.
• It has stuff for Android (https://github.com/ReactiveX/RxAndroid).
• Potential: for tasks that involve multiple AsyncTasks (e.g. google play login) or for replacing my event bus. (huh, not really?)

Unrelated:

• Dokan, a user-mode file system for Windows. https://dokan-dev.github.io/ 

---

Proposal doc as been sent:

• Option A: use the cheaper Intel “Ubuntu” stick. Fairly good price at $39. Use it as a dumb remote and just display a web page served by the JMRI server.
• Option B: use the beefier Intel Windows 10 stick. $125 or such. Run a C# or Java app that displays an SVG map and communicates with JMRI via the JSON server or the WiThrottle server.

Should look into option A, software side:

• Server side, a dedicated HTTP server. Could be either IIS or a custom Java servlet. The latter is probably easier to deal with, on a custom non-80 port.
• Explore what the JSON server in JMRI can and cannot do. Eg it can serve states, but does it allow performing changes? ⇒ yes, e.g. JMRI\java\src\jmri\server\json\turnout\JsonTurnoutHttpService.java doPut().
• Another option is to do a Java/Jython bridge like used with Conductor. This makes it trivial to start/shutdown at the same time as JMRI and can use the Jython interface to access turnouts.
• On the client side, we just need to display some SVG or bitmaps. Multiple maps can be trivially handled by having different query parameters.

Server side:

• Jetty / JSP
• Restlet
• Swing for UI
• JMRI Jython script dynamically linked to a Java JAR

Client side:

• jQuery
• SVG
• vanilla JS

---

Part of the project goal was to explore Firebase. From what I see right now:

• Firebase auth looks bolted on. I understand the main point is having different login systems (google, email, etc.) and the Firebase server generates a unique user id for new users, trying to regroup them across login system (e.g. email vs google account.)
• The pro is having a single user id regardless of the authentication method.
• The obvious cons is the secondary server authentication lookup to convert the googler user id into a firebase user id.
• Pros: not having to implement it using my own app engine instance + db.
• The "realtime" database seems to work as promised. However:
• User compartmentalization is done at the app level and by server-side rules. It seems fragile in the sense that a programming error or a design flaw could possibly expose all data to all users.
• All user data is visible as-is in the console, with a limit of 500 users or similar. Search can be done by user id. That seems like a potential PII issue and at the same time not sure how much it can really scale. What happens with 1 million users?
• How much "realtime" is the database? I wasn't even able to sign-in on a low cell phone connection (vacation).
• Pros: not having to implement it using my own app engine instance + db.
• I did not try "remote app settings". That seems like a generalization of the realtime db readable by all apps. Maybe that's the point and just enough.
• I implemented the firebase analytics but I can't see the results yet, the console is quite unresponsive over a low cell connection.
• I don't see anything obvious that is an advantage over pulling google analytics, except it's one less library and in theory it's tied to firebase user ids.

Overall I'll have to defer to see how this behaves with a better connection.

I do have my concern with the speed of the firebase requests.

The other thing not so clear is how does that scale.

Overall I guess the point of Firebase is having to avoid implementing an extra app-engine backend server. If it were used in an application that requires such a service, then the point seems rather moot.

For example I started a project with the idea of having both Firebase and App Engine. I thought I needed app engine for the storage of users ACLs, themes, and live instances but I see these could be replicated in the Firebase realtime db using denormalized storage & rules. The other reason I wanted app engine was to have all the processing on the server and the web site to be merely a "dump" presenter (using the MVP distributed pattern) .

Using Firebase, it makes more sense to have all the logic client side, with only storage on the server side.

⇒ It turned out to be disappointing. One issue is the login time on the Android app. It takes several seconds (10-20 s?) for the app to authenticate via google auth then via firebase before it can do any queries.

---

What I have at home is an RPi monitoring some status. The interesting properties of Firebase in this case are receiving notifications (aka GCM) and the use of the distributed "realtime" database. The question is how do the events get generated. Can a script running on the RPi send data to Firebase?

• "Firebase Notifications" are sent solely using the Firebase console web UI. Not adequate.
• "Firebase Messages" can be generated via an HTTP POST with an OAuth token; however this requires getting the registration tokens for each device that should receive, which doesn't seem practical.
• "Firebase Database" is a synchronized JSON store. There are many REST wrappers (Go, Python, or just curl) that seem appropriate for the RPi to generate data. Authentication rules are set in the database console. See https://firebase.google.com/docs/database/rest/start 

---