console.blog

Your Monorepo Dependencies Are Asking for Trouble

Andrew Brey — Wed, 12 Oct 2022 18:55:15 GMT

When I wrote the title of this post, I had planned on saying “please forgive me for using such a click-bait title” but I’ve decided that, no, I actually do kinda think your (or, well the Royal “Your”) dependency management just might be asking for trouble.

The topic I want to discuss in this post, monorepo workspaces with incompatible dependency versions, definitely doesn’t apply to every monorepo, and also, even if it does apply to your monorepo, there’s more than one way to solve for it.

This post will talk about the method I use in my own projects to ensure my monorepo workspaces don’t use conflicting dependency versions. Any choice of how to handle this topic in your own monorepo will come with tradeoffs, and my method is no exception. We’ll talk about those tradeoffs at the end 👍.

Note — this topic isn’t exclusive to the JavaScript ecosystem, but it is a topic I’ve encountered as I work to build the Lemmy App, the code for which lives in a TypeScript (mostly) monorepo, so discussion and examples in this post will be from the perspective of a Node monorepo.

The Problem

You’re building an application in a monorepo so that you can share as much code as possible between different concerns of your project. Let’s say you’ve got a marketing-website workspace and a webapp workspace and you want them both to use the same UI components (buttons, accordions, typography, etc), so you decide you’ll make a shared ui-lib workspace that both other workspaces can depend on.

For simplicity, we’ll assume that these three workspaces all use React so there’s nothing complicated about getting the marketing-website to render out the <Button /> component from the shared ui-lib, and similarly there’s nothing complicated about getting the webapp to render out the <Sidebar /> component from the shared ui-lib…right?

Early in the project, yeah, that’s probably true! You’ll be able to just import { Button } from 'ui-lib'; in either of the other workspaces and you’ll see a button render.

The problems lurk a little down the road when you want to bump the version of React that your ui-lib uses to take advantage of the awesome new features available in the latest version of React. The components in ui-lib don’t exist in a vacuum — they are only relevant in the context of the applications that import them, and what happens when the applications that import them, in our example marketing-website and webapp, are on a version of React that is incompatible with the newest React features?

Spoiler Alert!!! Only click here if you want spoilers!!1!1

💣 (It blows up at runtime...Sad App 😢)

“Sad App” Demonstration

FYI — as of this writing, StackBlitz doesn't (can't) fully support embedded WebContainers in anything except Chromium-based browsers. If you're in Firefox, Safari, or something else non-Chromium, you can still read the project code right here but unfortunately the in-page preview won't run and you'll have to open the "Sad App Monorepo" demo directly to run it. Sorry 'bout that!

Hey! — right here on larger screens there is an embedded demo using StackBlitz, but to be effective it needs a little more space than your device provides. You'll be better served by opening the demo directly in another browser tab: https://stackblitz.com/github/andrewbrey/blog-monorepo-deps-sad-app

^ that demo is a bare-bones version of the exact scenario I talked about above (minus the “marketing website”), and what we see is that when we upgraded the ui-lib so that we can use the super awesome new React.useId() API added in react@18 within the <Button /> component, but don’t or can’t upgrade the webapp from where it’s at (react@16) to also be on React version 18, we get a “crash” at runtime when the webapp tries to render the <Button />…Sad App 😢.

What Now

Of course, in this exact scenario, the fix seems pretty simple and clear - just upgrade the webapp to use react@18, the project is tiny and it’ll take about 5 seconds to safely make the change. Also, of course, this exact scenario is not very realistic to the real world where you’ve got a lot more code of your own, a lot more code from 3rd party libraries, and usually more people/teams involved in contributing to the project. Saying “just upgrade lol” is almost rudely dismissive of the complexity that’s potentially involved in mitigating this pain point.

So what should we do? There’s lots of things we might want to look into that can help fix our broken webapp. We could have the ui-lib declare react@^18 as a peerDependency and that would at least potentially help us realize at author-time that we’re careening towards a pitfall…that would be helpful, but it doesn’t actually solve the issue, and in a lot of ways it kind of makes it worse because now we’ve got another classification of dependency we need to worry about managing. You don’t have to look hard on Stack Overflow to find threads filled with people frustrated and confused by peer dependencies.

Maybe there’s something we could do with package resolutions if our package manager supports them (yarn does, which is what I use)? I don’t really see how that would help in this scenario, but it’s the kind of thing people toss out dismissively when you try and seek help on the internet for this type of issue.

Maybe we should just switch all of the projects to svelte so that we’re shipping compiled-to-vanilla JavaScript and all of our problems are solved??? (just jokin’ svelte is super neat, and I’m eager to build something real with it, but it’s also really common for people to suggest “just use a different tool” as the solution to a problem like this, and it annoys me)

I contend that there’s really only two options available that actually solve for this pain point:

don’t upgrade the ui-lib to use a newer version of React (and by extension, don’t get access to new features, bug fixes, security patches, etc)
upgrade everything that uses React to use the newer version (and by extension, mandate upgrades to everything else that’s downstream and impacted by your choice of React version)

as you might agree, both of these options suck. In my opinion, option 1 is like the tautological choice where “not making a choice is a choice”, and while technically a possibility especially for a short-term solution, hopefully it’s clear that it’s not a viable long-term option.

That leaves us with option 2, which is extra annoying, because like 2 minutes ago I said:

Saying “just upgrade lol” is almost rudely dismissive of the complexity that’s potentially involved in mitigating this pain point

— Andrew, 2 minutes ago

so, instead I’m saying the following:

I’m very sorry to say this dear stranger, but unfortunately, the best, or really least-bad, option available to solve the issue of “incompatible and shared 3rd party dependencies” is to ensure that all 3rd party shared dependencies have the same (or at least definitely compatible) version in every project that shares them.

The Fix

Taking a Step Back Before Going Forward

If you take another look at the workspaces in the “Sad App” Demo above, you’ll notice that each workspace has a package.json in which it declares its dependencies. In the package.json at the root of the monorepo, we tell our package manager where to find child workspaces, and it handles the intricacies of “hoisting” everything it can from the child workspaces into the root node_modules when you do an install (thus saving space on disk only keeping 1 copy of any dependencies where it’s possible to do so)…there’s nothing new here, this is standard Node monorepo stuff and it’s the exact structure you get out of tools like Turborepo and Lerna when you initialize a brand new monorepo project.

Unfortunately, it’s this structure which ends up leading down the road to “The Problem” above. You see, by allowing each child workspace to declare its own dependencies we’re opening the door for divergence from one workspace to another when it comes to dependencies that they “share” - in fact when using this kind of project structure, I am to the point now where I even push back on the nomenclature of “share” in that last statement.

If two workspaces declare different versions of the same dependency name, can we really say they “share” that dependency? Your package manager is potentially going to end up resolving two completely different sets of files to satisfy each workspaces’ declared dependencies, and at runtime, the code that gets executed could be wildly different! In my opinion, these are “shared” dependencies “in name only”.

I’ll keep using the term dependency name going forward, and all I mean by that is just divorcing the “name” of a dependency from any particular version. The difference between “my project uses React” and “my project uses react@18”

Not only that, but this structure of monorepo makes it harder even for diligent maintainers to keep the versions of a given dependency name in sync between workspaces because as your project grows and you add more workspaces that also need to use that same dependency name, you have more places to check and keep up to date. If you miss one or make a mistake, well, bad news, your package manager isn’t going to complain at you about that because it’s perfectly happy and capable of resolving more than one version, even though it’s not what you want to happen - and if you’re unlucky, you won’t find out about the mistake or missed update until it explodes at runtime.

Tools and scripting are definitely helpful here, at least with the mechanics of performing these updates, but tools can’t safely do this on their own (yet) and one way or another a human needs to be involved in this maintenance task. Mistakes will happen.

Implementing a Solution

At long last, time to discuss how I now go about solving “The Problem”. The key insight for me was when I had that thought from earlier about dependency names. Really, what I want is to have, for any given dependency name, 1 and only 1 version for the entire monorepo. I don’t even want the opportunity for my child workspaces to declare different versions of a given 3rd party dependency name.

It turns out, it’s really easy to accomplish that — just put all of your 3rd party dependencies into your root package.json, and none of them into any child workspace package.json! With this style of declaring workspace dependencies, when a child workspace has some code that imports React, the Node require() resolution logic will traverse up the file tree until it gets to the root node_modules where it’ll always resolve to the same exact version of React, regardless of which child workspace did the import.

In many cases, there are literally no additional consideration to make with regards to the machinery of managing your dependencies^** and you no longer need to worry about ui-lib using a version of React that’s incompatible with webapp or marketing-website…because they use the exact same version always.

^**oh, for sure there are more considerations overall, namely the tradeoffs involved here (and we’ll talk about those at the end), but I just mean with respect to the mechanics of ensuring you use one version everywhere

“Happy App” Demonstration

FYI — as of this writing, StackBlitz doesn't (can't) fully support embedded WebContainers in anything except Chromium-based browsers. If you're in Firefox, Safari, or something else non-Chromium, you can still read the project code right here but unfortunately the in-page preview won't run and you'll have to open the "Happy App Monorepo" demo directly to run it. Sorry 'bout that!

Hey! — right here on larger screens there is an embedded demo using StackBlitz, but to be effective it needs a little more space than your device provides. You'll be better served by opening the demo directly in another browser tab: https://stackblitz.com/github/andrewbrey/blog-monorepo-deps-happy-app

Discussion

In the “Happy App” demo, you see that any 3rd party dependency that any of the child workspaces needs to function is just tossed into the root package.json under the dependencies key.

Care about why I only use the "dependencies" key?

It's because, from the perspective of the workspace root, there's no way to know if a given dependency is for runtime or author-time...they're all just "dependencies" of one child workspace or another.

It also helps simplify some tools that need to look up dependency versions by their name (discussed further down in this section)

Further, each child workspace declares no 3rd party dependencies directly. They do declare 1st party “workspace” dependencies which enables tools like Turborepo and Nx to build a relationship graph between workspaces and optimize task running, but technically, even these “workspace” dependencies could be omitted.

You might also notice a strange shadow key in the child workspaces package.json, under which you’ll see a structure that looks suspiciously like the top level “dependency” declaration keys, except instead of being an object declaring names with versions, it’s just an array declaring names.

This declaration of shadow dependencies (just a name I chose and made sense in my head) lets each child workspace keep track of exactly which 3rd party packages it depends on, both at runtime as well as author-time, and further enables construction of the "effective" package.json for a given workspace (by looking up the declared shadow dependencies in the root package.json which knows the concrete version information).

Depending on what you’re building and in particular, what your build tooling looks like, construction of the "effective" package.json might not matter for you. For the Lemmy App, one of the child workspaces contains an Electron-based desktop app, and I’m using the electron-builder package to compile the app into its distributable form for production builds.

Unfortunately, electron-builder doesn’t play seamlessly with the ”go look in the root of the workspace to find all dependencies” idea out of the box when building for production, so I use the ability to construct an "effective" package.json during production builds to perform a workspace-local install of my production runtime dependencies and allow electron-builder to be non-the-wiser about how dependencies are managed in the monorepo writ-large.

The monorepo for the Lemmy App also includes some projects that use Vite for builds, and these work seamlessly out of the box with my dependency management strategy, without the need to do anything with the shadow dependency concept. In fact, during development, even the desktop app uses Vite so for me the shadow dependencies only come into play when I build that one workspace for production in CI.

BTW, if you want to see the tooling I made to work with this shadow dependency concept, take another look at the “Happy App” demo code in the scripts directory at shadow.ts (or take a look on GitHub)

Outcomes and Tradeoffs

Outcomes

There is a single canonical version of every single dependency name for the entire monorepo - “The Problem” from above is not possible.
It’s easy to adopt the solution and requires no ongoing curation of your child workspace dependency versions to keep them in sync.
Guaranteed to minimize disk space usage and install time for the monorepo because there’s only 1 version of each dependency name and they all live in the root.

Tradeoffs

You now must deal with any and all version migrations/breaking changes across your entire fleet of child workspaces all at once when you upgrade a dependency.
- For my own projects, I find this to be a positive tradeoff. If this project lives for a while, I’m going to be handling those breaking changes eventually anyway, and if I just handle them all at once, it’s easier and then I can forget about them rather than have to keep the steps of the upgrade in my working memory for longer. That said, if my typical project had dozens of complex apps and doing an upgrade of React in my monorepo meant committing to an hour of changes for each of them before I can be productive again, I would definitely consider if the risks of “The Problem” as outlined above are acceptable.
You now may need to choose solutions or 3rd party dependencies more carefully because you can’t (shouldn’t) massage versions on a single workspace.
- Technically, if you really needed to, you could still have child-workspace-level 3rd party dependencies declared that could bypass the pain point of 3rd party dependencies that don’t play well together at the versions enforced by your only 1 version allowed rule. That has not come up for me, but if it did, I think I would prefer a solution that uses a different 3rd party dependency altogether (or a vendored/patch-package‘d dependency) before I introduced an exception to my only 1 version allowed rule.
For this solution to be effective, you should put safeguards in place to ensure nobody accidentally adds a 3rd party dependency to a child workspace directly.
- I wrote both a custom ESLint rule and a git pre-commit hook to guard against this at author-time.

Closing Thoughts

I do not think that the steps taken in this post are right, or even necessary, for every monorepo out there, but with how hot the topic of monorepos is right now, I do think that it’s important that potential monorepo pitfalls receive due consideration. Hopefully now, after reading, you’re slightly better prepared to deal with a potential issue in your own monorepo. Thanks for reading!

Cheers!

Building a cURL-able Developer Resume Site with Remix

Andrew Brey — Tue, 05 Apr 2022 23:13:34 GMT

A couple of weeks ago I wanted to give building something with Remix a shot, so I made a small application called WAEL for tracking my weight and exercise over time. While building WAEL I was absolutely blown away by Remix! It was an absolute delight to use and the abstractions felt so simple and intuitive. I’ve used a lot of frameworks and tools in my time as a web developer, and really, nothing has impressed me as much upon first use as Remix has so far.

BTW, maybe I’ll make a separate post talking about WAEL at some point, but in the mean time, it’s open source so if you’re interested feel free to just poke around in the repository —> https://github.com/andrewbrey/wael!

Since I finished that project my brain has been churning so much with ideas of things I could build with Remix, and yesterday I had the idea that it would be super cool to build a resume site with Remix that was as you expect when you visit it in the browser, but has a beautiful terminal output when you visit it with cURL.

🎉 Here it is! `Remix Curl Demo` 🎉

Check it out at https://remix-curl.vercel.app

You can visit this page in the browser and it will serve up a static text/html response (which you might imagine is your resume content) and you can also curl this page in your terminal and see a beautiful text/plain response that contains style ansi escape codes that your terminal can use to print a nice interface.

Of course, the code is fully open source and MIT licensed, so check that out on GitHub —> andrewbrey/remix-curl

As I expected this was honestly pretty trivial to put together and I think that really speaks to the seamlessness of the abstractions that the Remix team has built. I spent more time getting Vercel to play nicely with my use of patch-package than I did actually building the site!

I could totally imagine building a fully fledged resume that renders beautifully in both the browser and in a terminal, and, while pretty dumb and mostly pointless, who knows what hiring decision some manager might make to hire you because you impress them with your silly cURL-able resume!

Cheers!

Flōt - Keep a website always-on-top and translucent

Andrew Brey — Mon, 06 Sep 2021 16:43:17 GMT

If you’ve followed along with my blog post history or my projects in GitHub, you might already know that I tend to work best when I’m like…12% distracted by something - that’s part of why I made the Nix browser extension (see the accompanying blog post about it here), and it’s why I recently took a liking to watching tv shows in a teeny-tiny little browser window in the corner of my screen while I work.

As I have done this however, I couldn’t help wishing that the browser tab with a show in it was slightly less noticeable, and if I completely had my druthers, if it were translucent so I could see what was beneath it. I looked around the internet and really only found one app which promised to provide these features and that was the Pennywise app, but as best as I can tell it’s unmaintained (or at most minimally) and either way it didn’t work correctly on my computer which runs Linux.

I decided to take it on myself to whip up a weekend project to scratch my own itch, and from that, an app was born!

🎉 Announcing `Flōt - Keep a website always-on-top and translucent` 🎉

You can find it for download on the official website https://flot.page. It supports Mac, Windows, and Linux and works a treat.

Key features include:

Adjustable transparency
Clean and minimal interface
“Ignore clicks” mode which causes the window to ignore your mouse and allow you to click what is behind it - pretty handy!

There were some surprisingly interesting challenges to solve with this tiny little app, and one of the big ones was how to both (a) not break the adjustable transparency for any platform while (b) being able to view most sites.

The problem statement here is that transparency of apps in Electron is somewhat tricky and breakage-prone. If you Google around, you’ll find tons of issues people have made about transparency not working in their app, especially on Linux. Many app developers just say “meh, my target audience isn’t Linux, so I don’t care” but for me, since I am my target audience and I do use Linux this was a must have! It turns out that the simplest way to ensure transparency works for the embedded content is to put it in an iframe so that the child site can be styled like any other piece of DOM…but now the child is in an iframe which means you’re dealing with a whole mess of web security and sandboxing topics for the embedded content that you don’t normally have to mess with when a page is the top frame. Many sites, especially ones a user of this app might be interested in viewing, have same-site restrictions for when they are placed in an iframe and many also have very restrictive x-frame-options headers applied when their content is served. All of this lead initially to being unable to display a lot of sites like YouTube and Twitch and Vimeo within Flōt.

I could get around this by making the target site the top frame, which in Electron terms would mean giving it its own BrowserWindow object, essentially another Chromium tab, within the app but if I did this, I would only have control of transparency to the extent made possible by Electron for BrowserWindow objects, which is to say, none (on Linux anyway). For Mac and Windows, Electron can do adjustable transparency of the window, but for Linux your either are fully transparent or you are not. In the iframe world, I could just make the app window transparent for all platforms, and use the transparency of the DOM to control just how see-through the content is.

In the end, I decided to keep the initial design, using iframes since the adjustable transparency was a key feature for me, and I used request interception in the Electron main process to ignore certain security features browsers are supposed to enforce, such as x-frame-options headers and content-security-policy. This makes it so sites can be embedded (mostly, there are some things which still break it seems) and I also get to keep my transparency controls in the DOM rather than on the native window…just, at the expense of pretty reduced browsing security.

Something had to give to meet my goals for the app, and I decided that this app was a toy which wasn’t meant to be used as a regular browser - instead I use it is a “read only” window for publicly accessible pages where I don’t enter my credentials or any sensitive information into the sites I visit. For this purpose, it works quite well and it was very fun to build, so I think all-in-all it was worth the few days of diversion!

Cheers!

PWPR - The simple Playwright Pre-Render CLI

Andrew Brey — Tue, 03 Aug 2021 22:07:16 GMT

Recently, while working on automating the installation and setup of my computer I encountered an annoying situation. I was trying to script out the installation of Dropbox, and I didn’t want to hard code in an installer version - for all other tools and applications that I “manually” install with scripts I’ve been able to obtain explicit version information programmatically. For some I could get version information via dedicated webpages (e.g. golang which provides its current version here). For others, which host their release assets on GitHub I could use the GitHub Releases API (e.g. the starship shell prompt, which uses GitHub Releases). In fact, up until Dropbox, the only tool which made this even a little difficult was the aws-cli which required that I parse their CHANGELOG file to figure out the latest version.

For Dropbox though, the only good way I could programmatically discover their latest installer version was via a link present on the download page, which for me being on Linux (Pop!_OS specifically) is https://www.dropbox.com/install-linux. If you go there, you might think ”ok seems easy enough, just curl/wget the page and grab the href for the .deb file download link with grep” and that is indeed what I tried at first…then I realized that this incredibly simple and text-only page is completely javascript rendered, meaning that curl-ing it is useless as I will only get back the pre-javascript-render page shell.

As annoying as this was, I didn’t just settle for hard coding in the version for this one thing, and instead I looked around for a CLI wrapper for either Playwright or Puppeteer which are tools I’ve used extensively before to do browser automation via headless chromium, firefox, and webkit. Either of these would make it possible for me to visit the page, render the page with JS, then get the final page HTML. As it turns out, Playwright does have a first-party CLI, but amazingly to me, it’s mostly geared towards PDF creation and automation of writing Playwright procedures - not prerendering…this seems like super low hanging fruit, but ahhhh well.

Aaaanyway, I decided just to make my own wrapper for Playwright, so here it is!

🎉 Announcing `PWPR - The simple Playwright Pre-Render CLI` 🎉

You can find it on NPM at https://www.npmjs.com/package/pwpr. You don’t need to install it to use it (in fact, I don’t recommend that you do install it) - simply run:

npx pwpr --url=example.com --output=example.html

doing this will fetch https://example.com, wait for it to fully load, including running the page’s javascript, then output the rendered HTML of the page to example.html in your current working directory - it’s like the simplicity of cURL, but with the ability to run the JS of the page you’re fetching.

Of course, the source code is fully open source and MIT licensed, so check that out on GitHub —> andrewbrey/pwpr

With this, I’ve been able to keep my Dropbox install simple and programmatic, without any version hard coding. Hopefully someone else out there finds it useful!

Cheers!

NPM Global Audit

Andrew Brey — Sat, 26 Jun 2021 21:36:15 GMT

I’ve found that it’s a good idea to keep as much of the software what I use every day up to date, and that includes the handful of node modules I have installed globally on my system - this is a pretty short list with “generally useful” things like rimraf, time-tracker, live-server (plus a few others). Towards the goal of keeping things up to date, I have made running npm-check-updates against my various packages a habit, and this includes doing a daily update of all the modules I have installed globally. When you do an update against your globally installed modules, just as you get with a locally installed module, the npm cli will usually spit out some warnings about vulnerabilities in the packages you have installed and will then tell you to run npm audit to learn more…

Well, if you try and run npm audit -g or something similarly sane to attempt an audit against your globally installed packages, you learn that npm won’t do that for you. I don’t fully understand the rationale for this limitation, but whaaaatever, I’ve decided to address the limitation myself!

I have released npm-global-audit, an MIT licensed and fully open source node module which can perform an npm audit of your globally installed node modules. You don’t even need to install it to use it (in fact, I don’t recommend that you do install it) - simply run:

npx npm-global-audit

from your terminal, and away it goes! There is more detail about how it works in the README for the project, so feel free to have a poke about to learn more. Hopefully this will help someone out there who likes to stay aware of the outstanding issues present in their software, at least for javascript.

Cheers!

Nix — The Simple Noise Mixer

Andrew Brey — Sat, 06 Feb 2021 23:07:37 GMT

When I’m working at my computer, I really struggle to get into a flow and be productive if I don’t have something playing in my headphones. Usually that something is music, but lately I’ve been really into soundscape types of background noise, like the kind you get in white noise apps and websites. I used a few different sites for this, but with each, I kept finding myself wanting for the noise mixer to be:

a standalone thing I could control with keyboard shortcuts, not another website tab
usable offline for when I’m working without internet
simpler and easier to make a mixture of sounds that would be my “preferred” mix, which would be remembered when I came back

Basically I wanted to be able to turn on my favorite mixture of rain + fire crackling + thunder every time, with as few clicks as possible, even if I don’t have internet.

None of the sites I found ticked all these boxes, so I decided I would make it on my own. I decided to do so as a browser extension to take advantage of the fact that it elevates the functionality to an “always there, ready when you are” status. It also made it super simple to remember the most recent mix I had playing, and more generally, I find it really fun to make extensions for the browser!

🎉 Announcing `Nix - the simple noise mixer` 🎉

You can find it on both the Chrome Web Store and the Firefox Add-ons Store!

Of course, the extension code is fully open source and MIT licensed, so check that out on GitHub —> andrewbrey/nix

It was super fun and quick to build, and here are some highlights:

I used React for the UI. I’ve really come to enjoy working in React especially with the way that you’re are incentivized so early to make small component abstractions with almost no overhead - I really enjoy this over the ”ugh, I guess I need a whole new file for this thing, and what folder should it go into??” questions that come with Angular and Vue.
I used ActiveJS for observable state management, and it was positively delightful! I have been itching to give it a try for a while now, and I found it’s blend of rxjs observables (which I mostly liked from my time doing Angular) with the redux-like store-with-actions model to be such a nice middle ground between the different state solutions I’ve used. It you’ve never heard of ActiveJS before, I strongly recommend poking around in their docs and learning all it can do, it’s really neat.
I used Tailwind CSS - at this point, I don’t imagine I’ll ever use anything else for styling on a personal project.

I really enjoy building browser extensions, and this one didn’t break that trend - it’s quite simple, but with just enough surface area to have presented a challenge or two, mostly around picking the right architecture and message passing controls exposed to the popup by the background. Give it a try and see what you think - I find it quite nice when I’m really trying to be productive 👍

Cheers!

Announcing the Lemmy App!

Andrew Brey — Thu, 13 Aug 2020 03:24:10 GMT

Institutional Knowledge is an asset, not a liability — at least, that’s what I want to be true, but unfortunately for every company I’ve ever worked for, that hasn’t quite worked out.

In fact, I have personally only ever really observed the opposite; an engineer who was with the company at the very beginning and “knows everything” decides to take another job and out the door with them walks years of knowledge. Most of that knowledge ends up being stuff nobody even knew they had! That’s the nature of “institutional knowledge”… it becomes so second nature that it is relegated to emails, slack threads, outdated docs, stale wiki pages, or most commonly a single person’s brain. When that brain walks out the door, the hard-won lessons go with them.

The Lemmy App is a project I’ve been working on to solve this problem once and for all! I’m not ready just yet to dive into the details, but there’s a bit of teaser content over on the marketing website. When you take a look, make sure to sign up to be part of the beta program so you can be notified when the app is live!

I’ll also be blogging on the Lemmy Blog about the process of building the app, so if you want to see what it’s like to build a desktop app from scratch you can check in on there every so often to see what I’ve been up to!

Cheers!

Using TailwindCSS with Ionic 5 and Angular 10

Andrew Brey — Mon, 06 Jul 2020 02:15:33 GMT

Recently for some client work I had to dig back into an Ionic Framework mobile app code base which I had most recently edited over a year ago. In the intervening time since my last commits in that project I have seen the light of utility / functional / atomic CSS, thanks in no small part to Adam Wathan the creator of TailwindCSS and his blog post from 2017 where he talked about his evolution from so called semantic CSS to embracing utilities.

Another great discussion of the merits of utility CSS (and why you should be using it) can be found in John Polacek’s awesome presentation Rethinking CSS.

There is also a wonderful talk by Simon Vrachliotis from years ago called A Real-Life Journey Into the Opinionated World of “Utility-First” CSS which also does an excellent job of showing the logical progression going from semantic css to utility css.

As I was working I found myself over and over wishing so badly that the ergonomics of styling the prebuilt Ionic components could be as pleasant as the experience of working with Tailwind, and then inspiration struck! Why don’t you just add Tailwind to the project dummy??. I don’t know why it took such rude comments from myself to make me think of simply adding Tailwind to make the styling experience nicer but hey, whatever it takes right?

Adding TailwindCSS to a vanilla Angular CLI project

Before talking about what it takes to add Tailwind to augment the prebuilt styles of Ionic components, I’ll first touch on how I’ve been successful adding it to a plain ol’ Angular project, without having to eject the webpack configuration from control of the Angular CLI.

Since Angular doesn’t use the PostCSS loader with webpack, we can’t just tack another require onto the postcss.config.js to include Tailwind. What we can do however is simply compile our Tailwind directives ahead of time with the tailwind node binary executed from npm scripts (or a tooling of choice) before the Angular webpack build starts such that the pre-built stylesheets can then be included in the webpack build as if they were regular CSS.

If you’re using sass/scss this is even easier because you can leverage @import syntax in the scss compiler to include the tailwind.css (or whatever you name your file) in your final bundle. If you’re not using scss you can still make Angular aware of the file by pointing to it in your angular.json builder definitions for included styles.

This works quite well but isn’t as convenient as we might like - in particular one thing we might want is for file watchers on our source files to include the tailwind.config.js so that changes to our Tailwind config trigger a rebuild of our Tailwind styles and subsequently the styles Angular knows about. It’s not always straight forward to create this type of behavior but thankfully there’s a very nice little node module (isn’t there pretty much always?) called ng-tailwindcss which not only gives you the setup I described above, but also sets up file watchers for you that will rebuild your Tailwind styles whenever you change your Tailwind config and ties the whole thing up in a nice little bow!

I won’t go into detail about how to set up ng-tailwindcss as it’s well covered by the helpful README - one thing to point out is that it’s not required to use the integrated PurgeCSS from this package because Tailwind now includes a first class PurgeCSS integration out of the box.

Adding TailwindCSS to Ionic 5 with Angular

Well it probably shouldn’t come as too much of a surprise that much of what was true for a vanilla Angular project is still true for Ionic + Angular, because since Ionic 4, the framework shifted to Web Components and became decoupled from Angular. In doing so, they largely got out of the business of project structure and defer to the chosen framework to make those choices.

That said, here’s what the default Ionic 5 with Angular global.scss looks like for a project scaffolded with the blank starter:

/*
 * App Global CSS
 * ----------------------------------------------------------------------------
 * Put style rules here that you want to apply globally. These styles are for
 * the entire app and not just one component. Additionally, this file can be
 * used as an entry point to import other CSS/Sass files to be included in the
 * output CSS.
 * For more information on global stylesheets, visit the documentation:
 * https://ionicframework.com/docs/layout/global-stylesheets
 */

/* Core CSS required for Ionic components to work properly */
@import '~@ionic/angular/css/core.css';

/* Basic CSS for apps built with Ionic */
@import '~@ionic/angular/css/normalize.css';
@import '~@ionic/angular/css/structure.css';
@import '~@ionic/angular/css/typography.css';
@import '~@ionic/angular/css/display.css';

/* Optional CSS utils that can be commented out */
@import '~@ionic/angular/css/padding.css';
@import '~@ionic/angular/css/float-elements.css';
@import '~@ionic/angular/css/text-alignment.css';
@import '~@ionic/angular/css/text-transformation.css';
@import '~@ionic/angular/css/flex-utils.css';

You’ll notice that there are already some utility stylesheets included with the framework - I don’t want those since I’m going to be using Tailwind and I also need to make sure that I include Tailwind’s version of normalize because it does one or two things a reset doesn’t normally do, namely adding a default border color of gray-500 that would otherwise normally have a default border color of transparent.

If you’re curious, this is so that you can simply add border to an element instead of having to do border border-gray-500 to make a default border visible.

I also need to make sure that I don’t lose too much from the Ionic styles because there are lots of style aspects that I rely on for their Web Components. What I need on top of my normal Angular only addition of Tailwind is to be able to simulate splitting my Tailwind directives apart in the way recommended by the Tailwind docs so that I can ensure the intended specificity is applied to my Tailwind classes compared to those that come from Ionic.

I decided to accomplish this by splitting the compilation of each directive, i.e. @base, @utilities and @components into a separate prebuild step and thus import them into my global.scss in the correct relative positions compared to the Ionic imports I was keeping. Something like:

/*
 * App Global CSS
 * ----------------------------------------------------------------------------
 * Put style rules here that you want to apply globally. These styles are for
 * the entire app and not just one component. Additionally, this file can be
 * used as an entry point to import other CSS/Sass files to be included in the
 * output CSS.
 * For more information on global stylesheets, visit the documentation:
 * https://ionicframework.com/docs/layout/global-stylesheets
 */

/* Core CSS required for Ionic components to work properly */
@import '~@ionic/angular/css/core.css';

/* 
 Use the Tailwind reset instead of the one 
 from Ionic because it sets some required 
 defaults for Tailwind
*/
@import './styles/tailwind/base.css';

/* Basic CSS for apps built with Ionic */
@import '~@ionic/angular/css/structure.css';
@import '~@ionic/angular/css/typography.css';
@import '~@ionic/angular/css/display.css';

@import './styles/tailwind/components.css';

/* Optional CSS utils that can be commented out */
// @import '~@ionic/angular/css/padding.css';
// @import '~@ionic/angular/css/float-elements.css';
// @import '~@ionic/angular/css/text-alignment.css';
// @import '~@ionic/angular/css/text-transformation.css';
// @import '~@ionic/angular/css/flex-utils.css';

@import './styles/tailwind/utilities.css';

// Other normal imports...
@import './styles/abstracts/fonts';

Notice that now I don’t use the Ionic utilities and I do use all of the Tailwind goodness!

Configuration for ng-tailwindcss

Even though it’s not strictly necessary, I decided to keep ng-tailwindcss around simply because it provides a nice abstraction on top of the tailwind cli. I created the following package.json scripts to invoke it 3 separate times, once each for the Tailwind directives:

"scripts": {
  "tailwind": "yarn tw:base && yarn tw:utilities && yarn tw:components",
  "tailwind:prod": "PURGE_TW=true yarn tailwind",
  "tw:base": "ngtw b -c ng-tailwind/ng-tailwind.base.js",
  "tw:utilities": "ngtw b -c ng-tailwind/ng-tailwind.utilities.js",
  "tw:components": "ngtw b -c ng-tailwind/ng-tailwind.components.js"
}

For reasons of simply not wanting the lines to be quite long in my package.json I split the scripts apart into a little namespace tw: and then made a pair of “parent” scripts for building everything for development and production (in which purging will be invoked by the Tailwind cli).

You’ll notice that each call to ngtw in the 3 tw: namespaced scripts includes a -c flag that points to an ng-tailwindcss configuration file. Each of the files looks something like

// For example, the compilation of @base looks like this

module.exports = {
	configJS: './tailwind.config.js',
	sourceCSS: './src/styles/ng-tailwind/base.css',
	outputCSS: './src/styles/tailwind/base.css',
	sass: false,
	purge: false,
};

In turn, you’ll see that each of these configuration files for ngtw points to a css file to compile and a css file to output. I include the sourceCSS file in git as it pretty much just looks like

// ./src/styles/ng-tailwind/base.css
@tailwind base;

I then git ignore the compiled files, e.g. ./src/styles/tailwind/base.css, but it’s these compiled files which are included in my global.scss seen above. This allows me to split the Tailwind css files up just like they would be if PostCSS did the compilation, and also ensures that I don’t include a massive Tailwind development css file in my repo - woo!

Adding to the Ionic dev server and production builds

Since this is all essentially just pre-compiling the css, adding to builds (development and production) is really just a matter of invoking the script before invoking the other build commands - here’s what the full scripts key in my package.json looks like:

"scripts": {
  "start": "yarn tailwind && ionic serve --no-open --lab --external -- --proxy-config proxy.conf.json",
  "tailwind": "yarn tw:base && yarn tw:utilities && yarn tw:components",
  "tailwind:prod": "PURGE_TW=true yarn tailwind",
  "emulate": "yarn clean && yarn tailwind && env-cmd --use-shell \"ionic cordova emulate ios --buildConfig build.json -- -- --storePassword=$KEYSTORE_PASSWORD --password=$KEY_PASSWORD\"",
  "build": "yarn clean && yarn tailwind:prod && env-cmd --use-shell \"ionic cordova build ios --prod --release --device --buildConfig build.json -- -- --storePassword=$KEYSTORE_PASSWORD --password=$KEY_PASSWORD\"",
  "tw:base": "ngtw b -c ng-tailwind/ng-tailwind.base.js",
  "tw:utilities": "ngtw b -c ng-tailwind/ng-tailwind.utilities.js",
  "tw:components": "ngtw b -c ng-tailwind/ng-tailwind.components.js"
}

Unifying Tailwind and Ionic

One other consideration is that you don’t want Tailwind to stomp all over the Ionic styles (most likely). You can help this with some careful thought in your tailwind.config.js. Here’s the one I used in this project:

module.exports = {
	purge: {
		enabled: process.env.PURGE_TW === 'true',
		content: ['./src/**/*.html', './src/**/*.ts'],
	},
	theme: {
		extend: {},
		colors: {
			primary: {
				default: 'var(--ion-color-primary)',
				shade: 'var(--ion-color-primary-shade)',
				tint: 'var(--ion-color-primary-tint)',
			},
			secondary: {
				default: 'var(--ion-color-secondary)',
				shade: 'var(--ion-color-secondary-shade)',
				tint: 'var(--ion-color-secondary-tint)',
			},
			tertiary: {
				default: 'var(--ion-color-tertiary)',
				shade: 'var(--ion-color-tertiary-shade)',
				tint: 'var(--ion-color-tertiary-tint)',
			},
			light: {
				default: 'var(--ion-color-light)',
				shade: 'var(--ion-color-light-shade)',
				tint: 'var(--ion-color-light-tint)',
			},
			medium: {
				default: 'var(--ion-color-medium)',
				shade: 'var(--ion-color-medium-shade)',
				tint: 'var(--ion-color-medium-tint)',
			},
			dark: {
				default: 'var(--ion-color-dark)',
				shade: 'var(--ion-color-dark-shade)',
				tint: 'var(--ion-color-dark-tint)',
			},
			success: {
				default: 'var(--ion-color-success)',
				shade: 'var(--ion-color-success-shade)',
				tint: 'var(--ion-color-success-tint)',
			},
			warning: {
				default: 'var(--ion-color-warning)',
				shade: 'var(--ion-color-warning-shade)',
				tint: 'var(--ion-color-warning-tint)',
			},
			danger: {
				default: 'var(--ion-color-danger)',
				shade: 'var(--ion-color-danger-shade)',
				tint: 'var(--ion-color-danger-tint)',
			},
			step: {
				'50': 'var(--ion-color-step-50)',
				'100': 'var(--ion-color-step-100)',
				'150': 'var(--ion-color-step-150)',
				'200': 'var(--ion-color-step-200)',
				'250': 'var(--ion-color-step-250)',
				'300': 'var(--ion-color-step-300)',
				'350': 'var(--ion-color-step-350)',
				'400': 'var(--ion-color-step-400)',
				'450': 'var(--ion-color-step-450)',
				'500': 'var(--ion-color-step-500)',
				'550': 'var(--ion-color-step-550)',
				'600': 'var(--ion-color-step-600)',
				'650': 'var(--ion-color-step-650)',
				'700': 'var(--ion-color-step-700)',
				'750': 'var(--ion-color-step-750)',
				'800': 'var(--ion-color-step-800)',
				'850': 'var(--ion-color-step-850)',
				'900': 'var(--ion-color-step-900)',
				'950': 'var(--ion-color-step-950)',
			},
		},
	},
	variants: {},
	corePlugins: {
		textOpacity: false,
		backgroundOpacity: false,
	},
	plugins: [],
};

You’ll notice a few things, like

I use the Ionic style custom properties instead of the corresponding hex/rgb values and further, I do this while overriding the colors key from Tailwind. There was simply no need to compile all of the Tailwind colors when I had a full palette from Ionic, and I could make Ionic’s configuration there be a single source of truth.
I turned off the textOpacity and backgroundOpacity plugins - this is the one real impact of not having the process of “how styles are made” be unified. The color variables I have, due to how you configure styling in Ionic, are either hex codes that are incompatible with CSS rgba functions (which is how the color opacity utilities work), or a partial rgb value like 255, 255, 255 (but not rgb(255, 255, 255)) for white. This would work with Tailwind I think, but I decided that I didn’t want to declare these types of variables for all of my colors and variations. There’s just so many colors to specify variables for in Ionic, and I didn’t want to bother for the sake of some translucent text.

Wrapping up

Overall not that bad right? Once you make the connection that Tailwind compilation isn’t some PostCSS magic, it becomes clear that the path forward is just to add a bit of automation to the lifecycle of manually compiling the css. With this setup, I can change any css and see live reloads and I can even make changes to my tailwind.config.js and have the changes live reload too (following a Tailwind compilation which takes a few seconds in dev) since ultimately, you’re just including another css file in the scss build pipeline which Angular knows quite well how to handle!

I found the augmentation of Ionic with a sprinkle of Tailwind to be well worth the effort of figuring it out, and what’s better is I got to nuke a large amount of Ionic css from my bundle and instead include a purgable chunk of only the utilities I care about from Tailwind - give it a shot on your next Ionic Angular project :)

Cheers!

Building a CLI for file path bookmarks

Andrew Brey — Mon, 30 Mar 2020 20:13:10 GMT

A while back I had been looking for a reason to play around with the Open CLI Framework or oclif for short - it’s a nodejs framework for building command line applications and it includes a lot of really nice things out of the box like support for single and multi-command CLIs.

When inspiration struck me in the form of a convenient method of “bookmarking” file paths in my terminal so that I can quickly jump between projects and important directories I decided to give it a go.

The result was an open source npm package called file-path-bookmarks which lets you quickly and easily save bookmarks in your terminal! This is a short one but pretty much just wanted to say “hey, I tried this thing and it was pretty neat, I liked it, and I’ll probably be using it again in the future!” so, anyway…

Cheers!

Building my blog with Gatsby, TypeScript, Tailwind, and Netlify CMS

Andrew Brey — Sun, 29 Mar 2020 00:00:44 GMT

For the last several weeks as I’ve been searching for a new job I have been surprised to find that not only is React a whole lot hotter than Angular where I am in Seattle, but it’s honestly not even really close. I have spent the last 2.5 years on a weird little island doing consulting work where I could freely pick the tech stack for client projects and I tended to err on the side of what I knew best (i.e. Angular) so that my time wasn’t spent fiddling around with a unfamiliar technology.

Now that I’ve come back from my blissfully unaware island, it really feels like my corner of the web development world (i.e. the somewhere-between-a-startup-and-a-small-company world) has more or less left Angular in the past and is significantly more focused on React and to a lesser extent Vue. I have only ever worked with these two technologies a tiny bit and I felt like if I wanted to stay sharp on what is in demand, I should get a bit more experience with React under my belt - so I decided to indulge a long time I should really build a blog feeling and…well, build a blog.

You are reading this on the blog that I built with GatsbyJS, a framework for taking components written in React, filling them up with data sourced from a GraphQL layer that supports a really diverse set of data sources, and spitting out pre-rendered “static” content that is much friendlier for SEO in the same way that server rendered content is, and which can then be “re-hydrated” into a fully dynamic React application on the client - I think it’s incredibly neat technology!

Project objectives

There were a few goals I had with this project (besides the previously stated “build a blog in React”):

Build it as an open source project which serves as a demo of how to build such a blog

🎉 All source code for this blog can be found on my GitHub profile: andrewbrey/console.blog
Incorporate TailwindCSS (and possibly Tailwind UI) which has quickly become my favorite way to build and style web content
Build the React components in TypeScript instead of using PropTypes for type checking

Incidentally, I read a post on the React Native blog this week where there was discussion of moving the code base to using build-time type checking (presumably by moving to TypeScript?) rather than using run-time checking - does this imply PropTypes enforce run-time type safety? I need to look into this more…
Get it hooked up to Netlify and Netlify CMS for builds, deploys, and content management

Diving in to the details

As mentioned before, this whole project is open source and MIT licensed so feel free to skip this write up and dig in to the code if that’s more your thing, but if you’re interested in my color-commentary on the experience of building, read on!

Overview

There are a few bullet points outside of the project objectives that I think are worthy of mention that I’ll present here in a group:

All of the content for the blog is written in Markdown as part of the git repository for the blog source code. The .md files are transformed into HTML through a series of Gatsby plugins, collectively found in a file called remark.plugin.ts which is part of the overall Gatsby framework configuration. As you might guess, the plugin is called Remark and it has several accompanying plugins that do things like extract referenced images to the static asset folder upon build, calculate the reading time for a post based on the number of words, and add code syntax highlighting to code fenced snippets.
If you look at the markdown stuff just mentioned, you’ll see that I have all of the framework configuration in a “non-standard” location (at least as far as I’ve seen on other Gatsby blogs), namely a config directory. The code is also written in TypeScript and I leverage ts-node within the “normal” configuration files to allow me to import .ts code into the typical .js configs.
I’m using several other plugins for things like the rss feed, the sitemap, and the Netlify redirect/header/caching rules - many of these plugins require no manual configuration for my use case.
You’ll notice that I have a .devcontainer directory that has some Docker related content inside. This is configuration that allows me to run my local development environment inside of a container and attach my (container host) instance of the Visual Studio Code GUI to the running container instance. I get the isolation, reproducibility, and portability of containers with the nice editor experience and tools of vscode - a very nice combination! If you’re interested in learning more, check out the vscode article on developing inside containers.

Using TailwindCSS

There is a Gatsby plugin for TailwindCSS, but I opted to instead add Tailwind myself using the “preferred” installation method discussed on the Tailwind documentation site which meant using PostCSS. This worked better for me as I knew I was going to be adding other PostCSS plugins (namely PurgeCSS and Autoprefixer) and I wanted to have as little Gatsby dependency as I could for this set of configurations. After adding the PostCSS plugin, I was off to the races.

Using TypeScript

Again, this was pretty straight forward - just install the Gatsby TypeScript plugin and add a ts-config.json to the root of the project! Don’t forget to also add ts-node as a devDependency in your package.json so that you can make use of .ts and .tsx files in your Gatsby configurations.

Using Netlify and Netlify CMS

This was the goal that took the most fiddling to get working exactly right. In order to generate Netlify deployment configuration files when you run gasby build, you need only add the Netlify plugin with no extra configuration, but in order to use Netlify CMS (which is a git based static site generator CMS) I ended up having to take more manual control of the configuration than I originally wanted. To facilitate this, I grabbed the Netlify CMS plugin, which generates the admin page on your behalf, and critically, has support for customizing the runtime CMS configuration within js/ts (which I have done in src/cms/index.tsx). This allows me to run the CMS using a local git proxy server (also a Netlify offering, see the beta features documentation for details) when I’m in my local environment, and using the real Netlify git proxy API in production, all without maintaining local file modifications that I have to remember to stash before committing!

Another thing I customized in here was the blog post preview template which allowed me to make the admin editing experience produce a preview that looks just like (well, very close to) the final rendered site which is very helpful!

That’s about it for the tour - there is more code in there but a lot of this an adaptation of the Gatsby stater blog template so it should be close to familiar to anyone who has toured other Gatsby blog repositories - this just has a sprinkling of my personal preferences and technology choices on top!

Cheers!

Hello World!

Andrew Brey — Sat, 28 Mar 2020 06:50:23 GMT

Here it is, the first post of console.blog! This is a bit of an experiment for me to learn about:

React
GatsbyJS
Netlify & Netlify CMS
prefers-color-scheme
Whether or not the <marquee> element still works???

and to have an excuse to use some shiny new components from Tailwind UI which, as of this writing, is in early access. I don’t know what this blog might become, and it is very very possible that the eventual answer is:

N o t h i n g

Sweet, looks like there’s an answer for the <marquee> question—> YES! (well, as of right now, though the animation is very janky on my phone).

Anyway, Hello World!

console.blog

Your Monorepo Dependencies Are Asking for Trouble

The Problem

“Sad App” Demonstration

What Now

The Fix

Taking a Step Back Before Going Forward

Implementing a Solution

“Happy App” Demonstration

Discussion

Outcomes and Tradeoffs

Outcomes

Tradeoffs

Further Reading

Closing Thoughts

Building a cURL-able Developer Resume Site with Remix

🎉 Here it is! Remix Curl Demo 🎉

Flōt - Keep a website always-on-top and translucent

🎉 Announcing Flōt - Keep a website always-on-top and translucent 🎉

PWPR - The simple Playwright Pre-Render CLI

🎉 Announcing PWPR - The simple Playwright Pre-Render CLI 🎉

NPM Global Audit

Nix — The Simple Noise Mixer

🎉 Announcing Nix - the simple noise mixer 🎉

Announcing the Lemmy App!

Using TailwindCSS with Ionic 5 and Angular 10

Adding TailwindCSS to a vanilla Angular CLI project

Adding TailwindCSS to Ionic 5 with Angular

Configuration for ng-tailwindcss

Unifying Tailwind and Ionic

Wrapping up

Building a CLI for file path bookmarks

Building my blog with Gatsby, TypeScript, Tailwind, and Netlify CMS

Project objectives

Diving in to the details

Overview

Using TailwindCSS

Using TypeScript

Using Netlify and Netlify CMS

Hello World!

🎉 Here it is! `Remix Curl Demo` 🎉

🎉 Announcing `Flōt - Keep a website always-on-top and translucent` 🎉

🎉 Announcing `PWPR - The simple Playwright Pre-Render CLI` 🎉

🎉 Announcing `Nix - the simple noise mixer` 🎉