Skip to content

Improving the SolidOS Frontend

–

I recently joined the SolidOS project to improve various aspects of the frontend such as components, styles, design systems, and accessibility. Exactly my cup of tea!

Activity

Task started

I have been involved with the SolidOS team for a while, because I used to attend every meeting and I've kept in touch with most of the people. However, I hadn't contributed any actual code 😱. But this is definitely going to change, because Timea Turdean has offered me to join the NLNet grant, and I'm really happy to finally be working on a Solid project fulltime :D.

Now, I'm not sure how long that's going to last, since I'm technically still open for work and it's not like what's left of the grant can support all of us (we're already 3 people working on it: Timea, Sharon, and me). Nevertheless, it's really cool that I can spend some time working on one of the most impactful projects in the Solid ecosystem. And as I mentioned recently, I really like the way things are going.

I've spent the first week mostly understanding how the various repositories work, and starting to make some proposals for improvements. My main takeaway so far is that the project definitely needs some conventions. The use-case the project is trying to tackle is very complex at its core, given the flexibility of Solid/RDF and its ambitious vision. But, as it often happens in these types of projects, there's also a lot of accidental complication going on (watch that video if you haven't!). Hopefully, one of the things I can contribute is a way to reduce it.

On that line, one of the first things we've been talking about is how to standardize writing UI. Right now, there are different styles mixed in the codebase, such as using Lit or creating DOM nodes with vanilla JS (document.createElement('div'), etc.). After some discussions last week, we've decided that we'll move everything towards Web Components using Lit. In particular, we're going to have 3 types of components:

  • Generic Solid Components (published under the solid-web-components package)
  • SolidOS Design System Components (published under the solid-ui package)
  • Application Components (not published anywhere, used internally to build the frontend)

I have already opened a PR with a first draft for the second type of components, check it out if you're curious.

And that's about it for now! Given that this is going to become my main gig for a while, I may be writing these updates more often than usual. But I don't think it'll be more frequent than once a week, so hopefully this won't become too noisy for people subscribed to the feed.

In case you're interested in digging in further, the SolidOS team is also working in the open, so you can check their notes and blog posts:

Hi again!

This week I've felt a lot more productive than the last. Even though we've also had a couple of meetings (links at the end), I've started to work on more elaborate components and learned a lot more about Web Components in general. If you're interested, you can see a summary of this week's work in this PR: Implement reusable primitives.

I started the week with the idea of creating an "Account" component. I have a similar component in Aerogel, and the idea is that this component can be used to show either log in/sign up links, or the user menu. In my apps this makes a lot of sense because they are local-first, but even though SolidOS isn't, it happens to have the same pattern (because you can open pages without logging in). So that's nice :).

It seems like a simple component, but it involves a couple of things that are not so trivial, such as interacting with the authentication system, invoking business logic (login, sign up, logout), and opening overlays (the dropdown user menu). Additionally, another challenge is that I wanted this component to be part of the generic Solid Components, because this is something other apps will definitely need. But after some tinkering, I decided it would make more sense to build smaller reusable components, and the "account" component would be SolidOS-specific (still reusable, though!).

Maybe the most interesting thing I've done this week is the way I've implemented the authentication and business logic interaction. Though the truth is that I haven't 😅. Let me explain. As I mentioned last week, the idea with these generic Solid Components is that they can be used in any project, not just SolidOS, and ideally we want them to be interoperable with other Web Components in the Solid ecosystem. In order to support that, I've abstracted away all the interactions with the business logic of SolidOS behind an interface. I'm still not sure how the whole ecosystem will converge on this, given that it's still an open question, but at least whenever we do, in SolidOS we'll only need to swap up this interface with a different implementation. An additional benefit here is that I've also been able to implement a Storybook implementation, which allows us to see these components in a nice UI sandbox :).

Something else relevant this week, although not Solid specific, was implementing the dialogs and overlays. This one is still a work in progress, but honestly, all the headaches I'm having show some of my initial concerns about using Web Components :(. It's not that difficult to implement a working dropdown with Javascript/CSS, but it is very difficult to make it conform with ARIA specifications and native APIs such as the Popover API. In fact, I've been looking at Web Awesome's Dropdown and it doesn't 😱. I often use this browser extension called ARIA Devtools to check whether an app is built correctly or not, and their dropdowns don't even show up :/. So yeah, I don't know... Trying to learn about this talking with some AIs and whatnot, it seems like ARIA roles aren't really "that important" (at least for dropdowns), as long as the labels and keyboard interactions are configured properly. There's also the fact that I'm not sure we should be spending that much time building these things from scratch, so this week I'll try to discuss some alternatives to use 3rd party components. We'll see how this evolves!

And that's about it for this week! Tomorrow (at the time of this writing), we're having a meeting about Jeff's Web Components, so feel free to join if you're interested to participate. The main idea of the meeting is that he explains how those components work, and how we'll integrate them in SolidOS or what'll happen with that repository moving forward (as I mentioned, ideally we'll use it to publish the generic Solid Components). But something I'll make sure to bring up as well is the interoperability between libraries, and the business logic layer I've been abstracting away. The meeting is happening on Tuesday 26th at 18:00 CEST, you can join in the SolidOS Jitsi room.

This week's meeting notes:

This week, I finally ran the entire SolidOS stack locally!

SolidOS is made up of many packages, and in fact the main SolidOS repository is nothing more than a bunch of scripts to orchestrate cloning and running them. At some point, we should probably migrate to a monorepo, but even then it's likely that some package makes more sense in its own repository. This is also a problem I've faced in my own work, for example when I am working on an app and I need to change both Soukai and Aerogel code. Recently, I even created vite-plugin-multi-root-workspace to make this easier. But SolidOS is still using webpack, so I won't be able to use it here.

Anyways, the reason why I started running this locally is that I opened my first real PR! There isn't much more to mention that I didn't say last week, other than I have been cleaning up the components and finishing the login UI. If anything, I would reiterate that working with Web Components is not being a great experience :(. Last week I mentioned the challenges with ARIA roles and element references, and this week I realized that using a native <input> inside a Web Component is not enough to make it work with forms 😱. Web Components were supposed to be the "native" way to encapsulate functionality on the web, but so far it's feeling more restrictive than Vue or React. I'm being forced to reimplement a lot of the things that worked out of the box when using plain HTML 🤷.

This week, we also discussed how to tackle AI commits in the codebase. NLNet recently introduced a policy on the use of Generative AI, and it's not completely clear what we can or cannot do in the project. Or how to even communicate it. In my own work (not SolidOS), I've been using the Co-Authored-By git convention in commits that have been AI-generated, and I also include an AI summary of the work. However, it seems like NLNet requires that we also include the entire prompt history, so that may be a problem. For now, though, I haven't been using a lot of AI in this project (at least, not to generate code). But once the foundations are laid out and we want to start using the Design System everywhere, this may become an important point of friction.

Finally, this week we also had a meeting with Jeff explaining his web components, which were pretty cool and also show what I think Solid should be all about (you can find the video recording of his demo here: solid-web-components in data-kitchen). We also released a new version with milestone 2k (I didn't do much work here, it was all Timea and Sharon!). You can find notes about these and the weekly meeting here:

Following up from last week, I kept improving the login form. In particular, this week I started looking at some of the things outside of the design. It is very common that application designs in Figma are missing many of the edge cases, specially when the task for designers was basically to redesign the entire application (or a huge part of it). But I think they are also very important, and part of what make a great UX. Usually, I would talk it over with the designers, but given that we're working with an external agency that bills by the hour, I decided to take some initiative and just do it myself. I'm not talking about anything drastic, though, simply adding some loading states to buttons when submitting, and showing errors when something goes wrong.

However, I spent most of my week migrating to Vite! On the one hand, that's great, because I do like Vite a lot more than I like webpack. On the other hand, it's not going as well as I thought 😅. It's not like the migration is going wrong per-say, but part of migrating the build system is to make sure that I don't break any of the existing consumers of the library. But I'm afraid that's not going to be so easy :(. Basically, the way the solid-ui library is being bundled and used is not ideal, and I'm not sure how much we can improve the situation without introducing breaking changes. Still, it should be "fine" to introduce breaking changes in a new major version... I'll have to discuss it in the next weekly meeting.

Finally, this week marks the first month since I joined the project. Looking back, it's been a bit different from what I expected. My intention was (and still is) to improve the UI. But so far, I've spent most of my time fighting against web components, cleaning up the architecture, and tinkering with build systems. Of course, all of that is also very important for the project and should make the UI work a lot easier. But, as I mentioned in my first note, there is a lot of accidental complication going on. Hopefully once I'm done with all of this, I can be a lot faster at working on the UI!

Here's this week's meeting notes:

We're still going back an forth about the migration to Vite, but I think we're almost done with that. Certainly, I don't think I'll have to spend a lot more time working on it. One of the reasons for the delay is that it needed to be reviewed by multiple people. One of the main topics of discussion is how much we want this to change things. You can read the weekly meeting notes at the end, but if you want a TLDR I also added a brief explanation in the README.

I am aware that this PR is very likely to break some existing dependencies. But I think that's something we have to do at some point if we want to improve the library, and now it's probably the best time to do it given that we're migrating everything to Web Components and consumers will have to rewrite their code anyways. In theory, people using the library through npm shouldn't be affected that much, because dependency versions are not updated automatically (or shouldn't). But another problem is people who's using the library through CDNs, which was documented with an unversioned url. The main problem is that the CDN used the window.UI legacy pattern to expose helpers, but that's often frowned upon for new applications (often called "Global Scope Pollution"). Instead, new applications should rely on ESM imports which are now supported by most browsers.

This week we also had a couple of private meetings. First, we talked with Jeff about his new component-interop initiative, and then we talked with Samu about the ODI's new authentication library. Both are taking a new approach at solving some of the problems we've had in the ecosystem for a while, we'll see how they pan out.

After these meetings, we reached a decision about SolidOS's Web Components: We're not going to release 2 separate libraries. A few weeks ago I mentioned we wanted to make a library with generic components, completely decoupled from the solid-ui package and design system. But it seems that wasn't the right decision, because it just caused confusion and we didn't land on a name we were happy with. So we're just going to release Web Components through solid-ui, some more reusable than others. Hopefully, this should still serve the same end goal because the library is now (or will soon be) able to expose individual components and helpers.

Finally, at the tail end of the week, I started working on my next task: to start applying the new components and design system to other panes. I started with contacts-pane because it seemed simple enough (we'll see how true that is!). I thought it could be a simple migration, and essentially it is; but I'll probably need to migrate the build system to Vite as well to support Lit's decorators and so on. I'll also take this opportunity to standardize and maybe extract some of the helpers that are duplicated in every panel repository, such as the dev sandbox. More about this next week!

This week's meeting notes:

This week, we finally merged the migration to Vite in solid-ui, so Webpack is history! At least, for that repository. We still have a few more to go 😅. And that's one of the main things I've been working on, migrating the contacts-pane to using Vite and the new design system.

But I'm not migrating all the pane. Rather, I'm using it as an example of introducing the new components progressively. In particular, I started by rewriting the "Add Contact" form into a dialog. The change itself is not too big, and the new code is a lot simpler than the old (as it should!). But this involves a bit of work upfront, such as migrating the build system to Vite.

As I mentioned last week, I've also been working on extracting a dev sandbox for panes, and I'm doing that in a new package called solidos-toolkit. The idea is that this package will include all the tooling that can be reused across the SolidOS stack. Mostly on panes, but I'll probably end up using it in solid-ui as well.

All of this work looks like yak shaving, but it's aligned with of my favorite software development mantras: Make the change easy, then make the easy change.

Other than that, we're still working on fixing the CI and our release process. Things slowed down to a crawl this week, because we decided to do a new release including the migration of the authentication library. You can find more about it in this week's meeting notes, but TLDR it seems we weren't aligned on the roadmap because I thought the next release was scheduled to include all the migration to the design system. So I had to roll back some of my UI changes in preparation for the release. Hopefully next week we'll get back on track!

This week's meeting notes:

This week marks the halfway point for this task, given that I'm going on holidays in late July and I started working on this in late May. There's still the possibility that I keep working on the project after the summer, but I'm definitely going to close this task by then.

Keeping that in mind, I started thinking about tying some loose ends, and the first one was accessibility. Of course, I always have a11y in mind from the beginning. But as I mentioned in a previous weeknote, we had to decide whether to use 3rd party components or not. We actually reached out to NLNet's accessibility partners for some feedback, but 3 weeks later we still haven't heard back from them :(. So, at least for now, we've decided to rely on Web Awesome. I still have some doubts about accessibility for Web Components, but honestly, I'm pretty sure they are ok given that a11y is one of their selling points. Unfortunately, some components are not free, such as the Combobox, and we'll need to code them ourselves (buying is not an option, given that everything in SolidOS has to be open source).

Something else I've been looking into is code reusability between components. In particular, making form inputs with Web Components requires a lot of hand-wiring (it's not enough to simply render native <input> or <select> elements under the hood). The first reaction to this may be to think of creating a parent "FormInputComponent" class, that can be extended for individual components. But that would be a mistake. Or rather, not ideal. Instead, I have created a concept of "component traits" that hook into the component lifecycle and can be composed with other traits. TLDR, composition over inheritance :).

Finally, we've continued talking about CI and dependency management. We haven't reached a final conclusion yet, but I did write most of my thoughts on this in a PR comment. We had a meeting on Friday (notes below), and it seems like we're going to configure an "ecosystem CI" that runs the entire SolidOS stack once a day to make sure everything works together. Hopefully, that will take care of some of our headaches!

This week's meeting notes:

Today I can finally say that I've completed the first task I started in SolidOS 🥳. Which is to migrate the header to the design system. Yes, that's it 😅.

When I started working on this, I definitely didn't think it would take me this long to get here... And to be honest, I can't say it's really "done" because in the process of migrating this, I may have broken a couple of things in mashlib :/. The main problem is that many things in the SolidOS stack seem to be glued together, rather than architected. And so, when you make some change, it's very likely that you break something in a completely unrelated part of the codebase (maybe not even in the repository you've made the change!).

Still, I do think this is the proper way to go for the long term. I'd much rather break things now, so that we're aware and fix them, than continue kicking the can down the road... But honestly, given the current status of the project and its prospects, I'm not so sure this is be best way to go either :(. I'm not sure if I'll continue working on this after the summer, and even if the project secures a follow-up grant from NLNet, these type of architectural issues are very difficult to fix. I'm aware that I sound very much like an architecture astronaut when I say this, but as ever, this shows me that technical debt really is more important than people think.

Anyhow, as part of migrating the header to the design system I also migrated solid-panes to Vite; and continued doing improvements to the tooling. For the last couple of weeks I have left, I think I'll probably focus on cleaning things up and leaving things as stable as possible.

It seems like we haven't got the meeting notes for this week yet, but we did get a new blog post about NLNet milestones:

One week to go before holidays, and I am mostly done with the two main NLNet milestones I wanted to tackle (3m and 3p). I'll write more about that soon, but this week I spent it mostly on improving what we already have.

First of all, I adapted profile-pane to use the new comboboxes. What makes these different from other comboboxes is that the list of options is dynamic, and you can't just render all of them at once. Instead, they have to be added as the user types into the input. Interestingly, this is someting that Web Awesome's combobox doesn't support, so I had to come up with a solution from scratch. And what I used as an inspiration was JQuery's DataTables plugin (Yes, I started programming in the jQuery era!).

However, I did it with a twist. One of the important features of solid-ui, and the reason why we chose Web Components, is that we want people to be able to use them with plain HTML, no JavaScript. But the way DataTables traditionally implement async content is with a JavaScript callback. In my case, I also did that but I'm providing a higher-level API to use this feature with plain attributes.

Here's an example:

<solid-ui-combobox
    label="Who is the best Disney character?"
    async-options-url="https://api.disneyapi.dev/character?name=%search%"
    async-options-results-field="data"
    async-options-label-field="name"
    async-options-value-field="_id"
></solid-ui-combobox>

The other thing I worked on was improving the UX of the login state. It wouldn't be a stretch to say that this is the sort of stuff I was most excited about when I started working on this project. I do like Design Systems and whatnot, but in the end I was really looking forward to improving the UX. One of the most annoying parts of SolidOS, in my opinion, is that when you are redirected back from logging in, there is a brief amount of time were the UI still shows the logged out state (maybe not so brief depending on your internet connection or device). Now, you should see a spinner indicating that the log in process is still ongoing.

This is a very small change, and the whole log in / reload experience is still a bit clunky. But hopefully this can be the start of more serious updates. It's also an example of something that an external design agency is probably not going to suggest, since they seem to be limited to creating Figma designs and may not be so familiar with Solid. But I believe these small tweaks are some of the things that can have the greatest impact in the overall UX.

Here's this week's (and last week's) meeting notes:

And so, the time has arrived for me to close this task. Tomorrow I'm leaving for holidays, and we just published an article in the SolidOS POD about the 2 milestones I've been working on. Make sure to check that out if you want a summary of my work. This update is going to be, as always, a more personal reflection.

I started working on this with a very clear goal, which is the title of the task: "improve the SolidOS frontend". And if I'm honest with myself, I don't think I've gotten even close to accomplishing it. In fact, if you try to run SolidOS locally today, you probably won't see much of a difference from 2 months ago. But the truth is not as grim as that statement makes it to be, because the work I've done has clearly moved SolidOS towards it; and I'm quite happy with it. It's just going to take a lot more time than I expected.

Now, let's backtrack a little. When I joined the project, I was offered the opportunity to work on 4 milestones (3m, 3n, 3o and 3p). The clear starting point was 3p, which was to create the components library. Also, it doesn't mention it explicitly in that document, but one of the points of doing that was to create a Design System. I've worked on Design Systems before, and you could say I've created some myself for my apps. So I thought it was just a matter of extracting existing code and "cleaning things up". However, that seemingly small task turned into a complete rewrite of the very foundations of the SolidOS stack 😅. I've addressed some of the reasons for that in the milestones article, and I also mentioned it in one of my first reviews of the project. TLDR, there was very little consistency between UI authoring styles and we needed to come up with the one true way. Which ended up being Web Components build with Lit. And then, I learned about the misery of working with Web Components 😞.

I won't delve too much into that, because I've already mentioned it in these notes and in various Matrix channels. Basically, what should have been barely a week of work to get basic components like inputs, buttons, and dialogs going, turned into almost ~3 weeks of struggle. Some issues are even still up in the air, like the a11y feedback we requested. Still, I wouldn't say it was the wrong decision given the requirements of the project. And the flexibility of Web Components has already paid off in a couple of ways. But it did mean that I spent most of my time in the project working on the fundamentals, without ever getting a chance to work much on the final UI.

Something else that slowed things down was accidental complication, which I've also alluded to in the past. Unfortunately, that is still an issue and I'm afraid it's not going away anytime soon. One of the changes I did in order to introduce the new Web Components and fix some build issues was to migrate the tooling to Vite, and rethink the bundling process. This could also have been done in a week, but the fact that SolidOS consists of multiple "independent" repositories made things a lot worse. In fact, even today, most repositories are still using Webpack and not bundling their code correctly. If I could snap my fingers and solve a single issue in the SolidOS project, it would be to migrate it to a monorepo.

Fortunately, in the last few weeks I was able to migrate some real UI and do some real improvements. But there is still a lot more work to do.

Which takes us to today. Regardless of whether I continue working on the project or not, I'm closing this task because the future is uncertain (and it's nice to close a task for once!). We're still waiting to hear back from NLNet to know if there'll be a second round of funding. But also, I'm not sure if I want to continue working on this capacity. I have enjoyed the work, and I really like being able to contribute to SolidOS, which is such an important project for the Solid ecosystem. But the truth is that I'm a bit taken aback with all the accidental complication (it reminds me too much of the reason why I left Moodle), and I'm not super happy with this model of funding based on grants. If it weren't for "the milestones that have been approved for the grant", I may have focused my efforts somewhere else; and maybe I could have influenced the real UX. Not to mention the precarity of my livelihood depending on whether we're awarded a grant or not.

So yeah, I don't know how much involved I'll be in the future. But I'm sure I'll be involved in some way, like I had been before officially joining the project.

Not to end in a sour note, let me reiterate that I'm happy with the results and I still like the project. This update may seem too negative, but that's because I preferred to leave the positive notes for the public announcement, and use this quieter place to share the rest of my thoughts. But overall, I would say it's been a good experience!

Task completed