Retrospective on making Base 3

The first version of Base was released in late 2008. It grew into a v2 by early 2011 and kept going until the last feature release in 2020 (with a few bugfix releases after).

On the 4th July 2020, I started a new branch for Base v3.

Getting Started

In mid-2020, the codebase for Base was struggling a bit. It was mostly Objective-C (which I still love) with some tentative Swift adoption dotted here and there. Any individual unit of code wasn’t too bad, but the whole was a bit tangled and causing headaches for making larger changes.

So I decided a clean slate was needed.

I just know people will be wincing at that declaration. I do too now. Sometime in the last 20 years I had read Joel Spolsky’s blog post on how rewriting a product is the single worst strategic mistake a software company can make.

I didn’t think this really applied to me. Not for egotistical “I’m better than that” reasons, more because I didn’t think the situation was the same. Much of Spolsky’s post focuses on how reading code is more difficult than writing, on how teams need spend time figuring out how the code works. But that wouldn’t affect me, because I wrote all of it! I know why that weird edge case is there. I know what idiot (me) wrote that dodgy function.

With that in mind, I created a new app target in Xcode and made a start.

Mistakes I made

These are just the highlights. There were plenty of smaller ones that weren’t particularly notable and didn’t have much effect on the whole - except possibly in aggregate.

1. Rewriting functioning code

As we’ve already noted, I wrote the original code. I knew all the shortcuts and gnarly corners. Naturally I decided to fix all of them as I went along. The problem here turned out to be much simpler than you might expect: Time.

All of those shortcuts and gnarly corners had accumulated over 12 years. Yes, I knew why they were there and wouldn’t have to spend time rediscovering old problems, but I would still have to carefully preserve their behaviours in any new systems and ensure nothing was lost in translation. And that takes time.

2. Adopting SwiftUI too quickly

I decided to adopt SwiftUI in 2020. This was, in retrospect, optimistic at best.

SwiftUI was barely a year old, still evolving and changing with each OS release. What worked in one version would be tweaked in the next. UI that worked fine yesterday looked different today.

The learning curve was steep, and I was trying to build release-worthy software while simultaneously learning how to use the tools. And the new tools didn’t have all the features of the old way. And the tools changed between OS versions.

Over the course of building a feature I’d realise I was missing important functionality and have to back out huge chunks of work and switch back to AppKit¹.

Please don’t take this item as me blaming SwiftUI for the problems. Rather the problem was my not knowing when and when not to use it and ending up trying to force it into shapes it didn’t fit.

3. Aiming for perfection

Since this was a from-scratch rewrite, I wanted it to be done right². And I wasn’t happy with most of the progress. This led to frustratingly slow work and trying several times to get different parts right.

I’ve only recently come across a quote from Ira Glass on the taste-skill gap, describing a period of time where your skills haven’t caught up to your taste. And that’s totally where I was. I knew the app could be better, that was the reason for the rewrite! But I wasn’t moving on once it was better, because I felt it wasn’t better enough.

4. Taking the difficult path

More than once I had a choice to either accept an easy or quick solution with a compromise, or do it the hard way and try for a better result. Apparently my default is to choose the hard way - it requires active mental effort to take an easier, but “less ideal” route.

One example was the SQL editor. It’s a text view that needs syntax highlighting and autocomplete. There was an basic implementation in the existing app, which had reliability problems and did want an overhaul. There wasn’t a third party library that I could just drop in without customising, so I started making my own text view to do the job. Before long I was bogged down, trying to wrangle NSTextView to do my bidding, make the autocomplete work how it should and also keep working on the rest of the app. It never did end up the way I wanted.

This was far more work than using someone else’s existing control and customising it to my needs.

Stuff outside my control

2020 to 2025 has been an odd time to put it mildly. We’ve all seen a lot of changes. Some of those were disruptive and meant that I didn’t work on Base for longer stretches of time than I’d have liked. There’s very little I could have done to mitigate this, but it’s important to recognise that sometimes life just gets in the way of your plans.

Trying again

By 2023, almost exactly three years later, I had to acknowledge that I’d dug myself into a bit of a hole.

The original branch³, started in July 2020, had grown into something of a monster. Three years of off-and-on (mostly off) development, thousands of lines of new code, and still no useful product. The git history still contained a bunch of half finished features and experiments.

So after a bit of thinking, I started again. The git commit reads:

Add new Base3 target

I’m trying again. Again.
Back to basics this time. Try not to chase the new shiny.

What actually worked

A much simpler plan:

• Translate the app, feature by feature, from Objective-C to Swift
• Create Swift Packages for each feature and core components⁴
• Rely on some third-party libraries

No sweeping architectural changes. No adopting of the latest frameworks. Just a methodical, mechanical translation of working code into a more modern language, with some tidying as we go.

Translating without actively rewriting

Translating from Objective-C to Swift improved matters quite a lot. Partly because Swift is more terse and so the code is more compact and readable. Partly because of the hundreds (if not thousands) of micro-improvements you pick up as part of the move.

Using plenty of packages

Swift Package Manager turned out to be really useful for this sort of thing for a few reasons:

1. It enforced better separation of concerns by stopping shortcuts. Code in another package isn’t as freely available to you, so you have to consider what you’re doing more carefully. Related: Adding a dependency to a package feels like a big decision and puts a psychological barrier in the way of potential bad habits.
2. It broke the process down into clearly definable steps. I could set a goal of migrating the table editor. Then, when it was done, there was a functioning table editor package and I could delete the original code. This was also incredibly helpful to help prevent myself from feeling overwhelmed.
3. It made testing much more routine. This could possibly come under point 1 above, but is worth pointing out explicitly. Testing isolated packages is just so much nicer than testing components within an entire app. I guess it’s not technically very different, but it feels different.

Third-party libraries

The key ones are GRDB and STTextView.

GRDB is an incredibly well made and popular SQLite toolkit. Base is only using a portion of what it can do and I’d like to write more about it in the future.

STTextView is a really great text editor with plenty of room for customisation. Trying to build my own - when libraries like this exist - is not justifiable.

Scheming

Base is distributed through this website, the Mac App Store and Setapp. Each comes with a different set of requirements for updating, licensing and distribution rules. The previous version of Base was developed before the App Store and Setapp were around. Each became a separate app target within Xcode, customised to build with slightly different files and settings.

This time, I have one target with three build schemes. Each has scheme has separate build configurations in .xcconfig files which link different libraries and apply different tweaks. It’s so much easier to reason about and maintain.

---

Some observations

The decision to start a rewrite again in 2023 was less hare-brained than the first time. This time I had a lot more pieces lined up for use. I was still working inside the same Xcode project, so that parts could easily be dragged from one app target to another for re-use. Most importantly, because it wasn’t so much a rewrite as a translation.

I did actually adopt some SwiftUI in places where it made sense. And (again) in a few places it didn’t. But this time I’d learned: If you’re trying to do something with SwiftUI and hitting problems that can’t be solved quickly, give up! Several times I saw a place where I thought it could save me time, only to hit problems. Each time I tossed the code and went back to AppKit.

What I learned

• Spolsky was absolutely right and my circumstances weren’t as different as I thought⁵
• Psychological traps are everywhere. When you know your own code intimately, you see all its flaws. The urge to fix everything can be distracting and almost overwhelming.
• Code that works is valuable, even if it’s not pretty
• I’m more easily distracted than I realised
• I’m lucky that I don’t depend on the income from this project

None of this is new. Most of it has been industry dogma for years, yet we still fall into the same traps.

Base 3 eventually shipped. It took rather longer than I’d hoped.

The app is better for the rewrite - it’s faster, nicer to use, and has a few new features. Inside, it has better architecture and is more maintainable. People using it will only notice the first part, they can’t see inside - but I know.

Suggestions for others

Here I could try to lay down some rules - or at least guidelines - that I think others should follow when contemplating a rewrite of some software. To be honest, I don’t feel comfortable doing that.

If you’ve read this far, you can see the mistakes I made. You can tell yourself you’ll learn from them, or that you wouldn’t have made them in the first place. Or maybe you’ll be like I was and think that your circumstances are different. It may be worth taking a step back and checking whether that’s actually true.

Remember: If you can’t set a good example, you can at least be a horrible warning⁶.