Extending "PETE" with Reusable Components So We Can Build People-Centric Standards-Compliant Data-Driven Apps Easier/Faster/Reliably

# Context 🤔

We have been using the ["**PETE**" Stack](https://github.com/dwyl/technology-stack/issues/37) for the past **18 Months** and it has been a really _good_ experience
for end-users (_people using the apps we've built_), developers and product owners/clients!
We _love_ Phoenix and Elixir and feel it's been a [**_good_ choice**](https://github.com/dwyl/learn-elixir/issues/102) of framework and programming language.

My/our _one_ "***regret***" is not _pushing_ for more **modularity** earlier on in our journey with Phoenix,
which has meant that we have "locked up" _most_ of the useful functionality in the Apps we have built
and thus have to "re-implement" lots of "boilerplate" each time we start a new project. 😞

## _Example_ 💊

A _good_ example of the PETE stack is Healthlocker https://github.com/healthlocker/healthlocker
The end-users like using and is _much_ better than anything the NHS had before (_for this use-case_). 🥇
Everyone who worked on the project can be _proud_ of the work that has been done. 🎉

_However_ if you look at the [**`mix.exs`**](https://github.com/healthlocker/healthlocker/blob/becff6feafb3e10ebe061f7056bcc04bf4755c0c/mix.exs#L55) file, you will see that while we used several open source modules created by other people, we did not _create_ any reusable components/packages/modules ourselves ...
That's "OK" because it was not the (_stated_) "goal" of the project to "_create **reusable code**_",
the goal of the project was to create an App that helps people "self-manage their [mental] health" and "effectively communicate with their clinicians".

_However_ it was an aim of the project to be ***Open Source*** so that _anyone_ could contribute!
And while we "checked the box" of Open Source by making the project **`public`** on GitHub,
we have not had any _contributions_ from anyone _outside_ of the "core" team.
This raises a _separate_ question around what constitutes "success" in Open Source: https://github.com/dwyl/technology-stack/issues/66 ...

Our _hypothesis_ is that if we had created Healthlocker as a collection of reusable components which then get _assembled_ into an App, we _could_ have aided the "core" goal of the project because by making reusable components, each time one of those components is reused and _improved_, the _original_ project receives all the improvements "for free".

By creating Healthlocker as a one "monolithic" repository (_AKA "**monorepo**"_)
we were able to move fast(er) _initially_ and deliver the App to users/stakeholders quicker ... 🐎
but _now_ when we need to implement Auth, Permissions or Chat in a new project 🆕
we have to do it all "from scratch" ... ⏳

## _Analogy_: Smart Phones 📱

Think about the Smart/Mobile Phone in your hand/pocket, it's made up of [_millions_ of components](https://fossbytes.com/whats-inside-smartphone-depth-look-parts-powering-everyday-gadget/).
The software stack that goes into making the smartphone possible is _billions_ of lines of code.
Each time one of the elements in the software stack (Kernel, APIs, UI Libraries, Build Tools) improves,
_everyone_ in the ecosystem (developers, users, companies) benefits from the improvement.
If the stack of software for the smartphone was a "monorepo" it would have failed _long_ ago;
because it would have become unmaintainable and no _individual_ person would be able to understand it!

> **Note**: For this analogy, focus on the _software_ and ignore the fact that for the most part the individual hardware components (screen, battery, processor, camera etc.) are not upgradeable. 🙄

## Lessons Learned 📚

We have _all_ learned a lot from our work over the last few years. A few lessons
+ [x] The "flexibility" of "NoSQL" rapidly becomes a burden when data is inconsistent and queries slow!
> We felt this pain when we tried to use ElasticSearch as a _primary_ DB, Redis to store user data, or JSON in PostgreSQL which resulted in painfully slow queries! MongoDB was a *pig* when we used Meteor; they sacrificed reliability for "bigness".
+ [x] Don't delay/procrastinate on figuring out the schema for your App.
The sooner you think in "data structures" the better. see: https://github.com/dwyl/learn-elm/issues/120
+ [ ] **_Embrace_ Email** as a means of connecting/communicating with and getting _feedback_ from users!
_Everyone_ that has access to the internet has an email address. Android (_the most popular operating system **by far**_) ***requires*** people to use a GMail account. Email is _still_ a good way of reaching people, even if just to loop them back into the App/service you are building.
+ [ ] **_Proactively_ Implement Analytics** and capture metrics for every aspect of your app(s)!
> Trying to _retrospectively_ apply "GTM" as an _afterthought_ is like using a tennis racket to paddle a canoe _upstream_! It might "kind work" but you'll ultimately "lose" drowning in data noise!
+ [ ] Building reusable components/modules/packages might take a little more effort up-front, but it makes life _much_ easier the _next_ time we need to build a similar feature/app and invites other people from the community to give feedback and _improvements_!

# Proposal 💡

Our mission for the rest of 2018 is to create the reusable stack we will be deploying in 2019:


> Diagram: https://drive.google.com/file/d/1gfvgK1Mn7EuamHJroS8G7FUqauRgA9EH
Note: the colors of the chevrons are not significant. they are merely to help differentiate between the different elements in the stack. If anyone wants to make this look better, please go for it!

If this _feels_ like "a lot", don't be discouraged or overwhelmed, we already have a _massive_ head start!
A _lot_ of the work on this has already been started and in some cases shipped!
Break it down into "chunks" and start from the bottom: the Append-only Log is the _core_ of all apps.
see: [https://github.com/dwyl/**phoenix-ecto-append-only-log-example**](https://github.com/dwyl/phoenix-ecto-append-only-log-example) (_if you haven't already_)

@Danwhy is already making great progress with Append-only Log: https://github.com/dwyl/alog
Both @Cleop and @RobStallion have made great contributions to https://github.com/dwyl/autoform
and we are already _actively_ ["dogfooding"](https://en.wikipedia.org/wiki/Eating_your_own_dog_food) elements in our client projects!

We'll build a "minimalist" analytics system "from scratch" on top of alog: https://github.com/dwyl/atm
We have done _some_ work towards this in: https://github.com/dwyl/hits

Auth will need to be assembled from the ground up based on alog, fields and autoform.
So for now, _ignore_ the work that had been previously done.

Once we have the "base" layer of alog, Fields, ATM, Autoform, Auth and Admin, we can build the more "feature rich" tools: Contact and Feedback.

Hopefully this will clarify the "strategic" direction we are taking when building our most recent projects in a _modular_ way.

# Name? 💭

For now, we don't have a _name_ for the _collection_ of modules/packages.

> Open to suggestions as to what we should call this collection of modules.

I'd love to call this the "**_Obvious_ Stack**" because most of these things are _obviously_ useful to most apps we have (already) built and will give us a _massive_ head start for anything we build in future.

Considering "_repurposing_" https://github.com/dwyl/abase as it was the "spiritual grandfather" of this.
"**aBase**" was built and used used for a couple of Node.js Apps in 2016/17 but not developed further, as transitioned to Phoenix in early 2017.
It's not the _best_ name (mea culpa), but at least it conveys the meaning of what we are building.

# Feedback / Thoughts / Questions / Discussion ? 💬

As always _your_ feedback is very much invited/requested/appreciated!
If you have ideas, thoughts, questions, concerns, please share!!

Thanks! ✨

***Related*** to: https://github.com/dwyl/learn-elm/issues/121 (Creating Elm Packages) ❤️ and https://github.com/dwyl/hq/issues/497 (Learning Goals). 🎯
***Required*** by: https://github.com/dwyl/feedback/issues/96 (Feedback Widget) and https://github.com/dwyl/product-roadmap/issues/7 (Metrics)