Improving Solid Focus Task Manager

Yesterday I hammered through some UX improvements that I had wanted to do for some time.

First, I wanted to add animations when a task is added or removed from a list. I started using Vuetify transitions, and they were a good starting point to see what I wanted out of this feature. But their implementation did not achieve the fluid experience I was looking for. In particular, transitions are implemented using translate CSS transformations. The problem with this is that the computed size of the element is not animated, only its visual position. This causes an undesired jump of the content at the start or at the end of the animation. In order to achieve what I was looking for, I ended up using a custom Vue transition where I animated the height property instead of applying a transformation. I can understand the reason for Vuetify animations to be implemented that way, because in order to have a height animation working, the elements need to have a fixed height. This is not viable for a framework that aims to provide a toolbox for multiple use-cases, but for my use-case in particular I could live with that trade-off. You can see my implementation in this commit.

Another thing I worked on was increasing the size of task checkboxes. As part of the feedback I got for the first version of the app, I was told that clickable areas should be at least 32x32px on mobile, and the default size of the checkbox that comes with Vuetify is 24x24px. If you look at the repository, you'll notice that I went back and forth a couple of times with different solutions. At the end I decided to stick with a simple approach to use the default size for desktop and increase the checkbox size in a mobile layout.

I also used this opportunity to add a real implementation of a UUID generator. This made me think about how task ids are generated with the online mode of the app. I send a request to create a new document without indicating an id, so it's being generated on the server side. This could be improved generating it on the client so that I don't need to wait for a response to reflect the changes on the UI (and revert the local state in case of an error).

Last week I released a new version of the app with some UX improvements and bug fixes. You can check the details here.

I'll now start working on the data layer integration. The current implementation is quite naive, because I wanted to learn the basics and know what's going on under the hood. But it's clear that it wouldn't be scalable to continue working on the application.

I saw Ruben Verborgh's LDFlex library, which he introduced in this post and I'd recommend you to read. I want to take another look to see how it works, but I'm more inclined to use a different approach to manage models and decouple that from the UI components as much as possible. I'm used to doing that in my applications, because I've done most of my data management with Laravel through an API. So I'd like to keep a similar workflow that's worked so well for me. But I have to keep in mind that this is a different paradigm and that may not be the right approach.

The last few weeks I've been tinkering a lot with Soukai and its Solid engine. I still haven't reached a point of closure (I need to tackle model relations, for example). But it's been a while since I wrote an entry on this journal so I figured it's about time I do it.

One thing I did was setting up the js.org domain to serve the soukai documentation. It's cool because it works nicely with github pages, so anyone can host a XXX.js.org site for free. Head to js.org to learn more about it. I pondered going all in with this and have all of my projects setup with that domain, but I'll stick to pure js packages. I'll continue to use the github.io domain for other projects.

I've also had a thought about my development process. I realize that I move slow on my side-projects, certainly slower than the progress I make on my day job. And the reason for that isn't only that I dedicate less time, it's also how I approach it. In my day job, the main goal is to get things done. And of course, I also learn new things, but I try to tone down my perfectionism to be more productive and get results. But on my side-projects, I pay more attention to detail at the cost of going slower. I do it because that's how I maximize my learning and self-improvement. I recently listened to an episode of the Hurry Slowly podcast that talks about this: Creativity vs Efficiency. But the thing is, I also want to make progress on my side-projects! So I've decided to try something out. For the next few weeks, I'll alternate between "getting things done" and "perfectionism" mindsets each week. I did it for the last two weeks and so far it's been good.

Part of the perfectionism week has been dealing with repository versioning. So far, I had a single master branch and the version in package.json was the version of the latest release. This was confusing, because someone may think looking at the repo that a feature is live, when in fact it'll be available for the next release at which point the package.json version will change. After reading a while and looking at what other repositories do, I've decided to have a dev branch following the same strategy, but the master branch will only be updated for releases.

Something else from this week has been setting up Cypress to run integration tests for Solid Focus. It was specially important to add those before integrating with Soukai, so that I could be sure there weren't any regressions. It's been my first time using Cypress, and overall I'm very satisfied with it. There are some things I couldn't manage, like setting up Typescript, but it was relatively easy to set up. I've also learned about a new command: npm ci. I'd always used npm install to install my dependencies even in environments running Continuous Integration, but apparently this command was added about a year ago to ensure "getting exactly what you expect on every install". Why wouldn't this already happen with npm install is beyond my comprehension.

On getting things done week, I managed to finish the first version of soukai-solid, and I've already integrated it in Solid Focus (in the development branch). There haven't been any major drawbacks yet, so I'll get into defining relations between models and I think I'll have a first version fully migrated to Soukai soon.

Today I completed the migration to Soukai. All the data management operations to interact with a Solid POD are now delegated to soukai-solid (except authentication that's still using solid-auth-client). One example of a model defined with soukai-solid can be found here.

I believe this is an important stepping stone for me to start creating Solid applications. Using this library I can focus on app development and encapsulate Solid specifics. During the development of the library, I've been asking some members of the Linked Data community what's their opinion on this, and most of them agree that the complex relations between LD documents cannot be fully expressed with objects. That's why an ODM comes as an unorthodox approach. I still have to look at LDFlex again, because last time I checked it was lacking some features that I've been told are now available. But my opinion is that using both approaches should be compatible and even preferable. My opinion is heavily biased given my background, in particular working with Laravel's Eloquent. That's why once I have documented and cleaned up the repositories, I'll open a thread on the Solid forum to discuss both approaches and learn more about the topic. It may be my inexperience with Linked Data talking, but I see a lot of value on combining both approaches. I'll expand on this when I open the forum thread.

Also continuing with something I mentioned last time, I've updated the deployment strategy for the javascript libraries using npm. As noted on the previous comment, I created a dev branch with the new changes, but the problem was that it wasn't possible to reference that branch from CI services (because they pull from the npm registry). I was struggling to find a good solution, until I found out about npm tags. The idea is that once a new version of a package is pushed to npm, it's by default tagged as latest. But this can be overriden, and I've started using the next tag to publish development versions of the libraries. The name of the version also indicates that it isn't stable with the suffix -dev.{commit hash}. This solution is still not perfect because I'm forced to commit changes to package-lock.json frequently, but it's the best approach I could come up with. When I start doing releases I'll see how cumbersome it really is.

Now what's left of this task is to document the changes to the libraries and implement more tests for new features regarding Soukai, and I'll finish by finally implementing some new features.

I've been writing more tests with Cypress and it's awesome. I mentioned before that I wasn't able to set up Typescript for the tests, but I've finally fixed that (turns out I wasn't installing the @cypress/webpack-preprocessor npm package).

I had managed to do the first tests with relative ease, but it's true that I hadn't really got into it. I consider myself more of a bottom-up learner, so I got into the Cypress documentation (that is great, btw) and started reading for a while. I'd recommend anyone who wants to use it to read this guide because it cleared a lot of my doubts: Introduction to Cypress.

I've now finished tests covering the basic functionality, but I found a problem with the current approach. The application has two modes for data storage: Offline (local storage) and Solid. I did this at the beginning in order to isolate the functionality from Solid particulars, and I also think it could be useful for a lot of people who don't want to use Solid. But the problem with that is there are two implementations of the data management layer. It's not a problem per se, but the tests are now only testing the offline part. I could test the Solid implementation as well, but it would be pointless because the user interaction and expectations are all the same (so the tests would look the same except for the login). This could be solved if I delegate all the responsibility to Soukai instead. I could stub Soukai and make sure that it's used appropriately, then the production app would simply change between a Solid engine or a LocalStorage engine.

I'll try to tinker with that for a bit to see if I can come up with a decent approach. If I don't I'll just postpone it, because I have already planned to do this at some point to improve data synchronization (storing pending changes when the network is off, etc.).

What I mentioned on the last comment took longer than expected, but I'm very happy with the results! In trying to improve the tests, I realized the responsibility of interacting with the data layer is actually encapsulated in the Soukai library. The different implementation I was using for the different modes was only because I was not saving the models. And yet, I was storing their serialization to LocalStorage. Which got me thinking, and what makes more sense is to have a LocalStorage engine implemented in Soukai, which is what I did. In doing that I also realized the responsibilities for the SolidModel and the SolidEngine (within the soukai-solid package) were also mixed up, and I ended up doing a whole refactor in Soukai as well.

In summary, Soukai engines now know nothing about models, as it should be. Which is nice because I've removed a circular dependency that may have caused headaches on the future (for example models working only for certain engines, as it was happening with Solid). And in Solid Focus I've moved all the engines to the Auth service. Which means it is now possible to implement new Soukai engines to work with the application. I don't think I'll do any in the short term, but I am for example using the InMemoryEngine for tests which solves the problem I had a couple of weeks ago.

If you didn't understand anything of what I'm talking about, head to the Soukai docs. Depending at which point you visit them they may already have the changes I've talked about documented, since improving documentation for all three projects (Soukai, Soukai Solid & Solid Focus) is what I'll be doing next.

If you had visited the API section of the Soukai docs before today, you'd have noticed it was all empty and filled with TODOs. My first approach to documenting the interface of the library was writing it by hand, and of course I knew that would change. Well, that changed yesterday because I've started using TypeDoc.

My library is written using Typescript, so there is already a good deal of information there that can be automatically extracted. I started looking for solutions and I settled on using TypeDoc. After tinkering with it for a while I found it lacking in some areas, but good enough to use for now. The features that I found lacking are already being tracked (issues #36 and #639 on github), although it doesn't seem like they'll be added anytime soon. But it's good enough and better than my previous approach.

Basically TypeDoc will read a typescript project and generate a documentation in html. Other than Typescript primitives like types and interfaces, it'll also read documentation in the style of phpdocs or javadocs. I don't have any of that the moment, but now that I'll start documenting the recent additions to the library in the guide I may add some. When it comes to documenting code I think class and method names should be expressive enough, but sometimes there is value in adding code documentation.

You can see the current deploy of the API reference here: https://soukai.js.org/api/

In the process of updating the docs, I've had a chance to think about the licensing of my open source code.

For a while, I used MIT by default without second-guessing, given that most of the frameworks and libraries I use are licensed as such. However after getting more into privacy-focused communities and learning more about licenses, I decided to use GPL for all my subsequent projects. But as things go, I recently read DHH's keynote on railsconf this year (Open source beyond the market) and it made me ponder again. He argues against GPL in the sense that doing Open Source, nobody who uses your software should owe you anything. And that should be the true spirit of Open Source: "do whatever you want with it". I won't get into the Free Software vs Open Source debate here, but suffice to say that I mostly agree with DHH. Except for apps.

If we are talking about libraries and frameworks, I agree with the "do whatever you want" spirit, because I assume the ones using my software will be other developers, and they should know what they are doing. But when it comes to apps, the ones using the software could be end-users who know nothing about programming. And at that point I'm not ok with someone changing the app to "do whatever they want with it". They could for example be adding privacy-invading features that users are not even aware of. That's why for apps, I think GPL is actually more appropiate. It's also true that using GPL for a library could be a barrier for some people, because MIT projects cannot have GPL dependencies, while GPL projects can have MIT dependencies.

Coincidentally, I looked into other software projects and it seems like the pattern could make sense. A lot of libraries I saw use MIT (Vue, React, Angular, jQuery, Laravel, Rails, Phoenix, etc.) whilst apps use GPL (Mastodon, MoodleNet, GIMP, etc.). Of course this is not always true, but I believe it's a good rule of thumb I'll follow from now on.

Well, the updated docs are finished for now. I've put more effort in the Soukai docs than the Solid application, but I think that's fine. I consider Soukai to be a foundation for what the Solid app will become, as well as for other upcoming applications. And it's important to have strong foundations.

What I will do now to finally complete this task (4 months ongoing already!) is implement a couple of new features. It'll also be a good test to see if the foundations are really strong or not. The new features I plan to add should already be supported with the current status of the Soukai library, so let's see how it goes.

I just implemented the form to update tasks and there was a small bug with soukai-solid, but nothing dramatic.

What I spent more time on, though, was implementing the animation for the sidepanel that will contain information about the task as well as the form. I decided the best way to animate the transitions is with CSS when a task is not selected anymore. The problem is that because the task is no longer available to the Vue component, it will be blank and that's not nice UI.

In order to solve that I've created a component I'm calling "Freezable". I'm not sure if anyone else has found this use-case, but it seems to me like a good solution.

I've created a gist on github with the basic component and some examples in case you want to check it out: https://gist.github.com/NoelDeMartin/7414586bae62f79bcf9bfb9f12b13316

In that last comment I also forgot to mention that I've also started using a Vue plugin that's called portal-vue. I've wanted to use it for a while but I didn't find a decent use-case until now. I'm using it for the sidepanel and the overlays.

I've finally completed adding all the new features I wanted, which include: editing & deleting workspaces, lists & tasks; and tasks now include a description and a due date. By the way, in the process of implementing due dates, I was faced again with that ugly choice of using momentjs or not. For those reading who don't know it, it's a very good javascript library to work with dates, but it weights a lot so you should think twice before adding it to your project. Fortunately, I found a nice alternative: dayjs. I think I'll use it in most of my projects working with dates from now on.

One thing I implemented in the 100th commit 🎉 of the repository is something I've wanted to have for a while in my task managers (and arguably all my apps): built-in support for markdown! Task names and descriptions have it, and I think markdown should be supported by default in all text fields intended for text that will be displayed. I'll try to do that with my apps from now on to see if it really is a good idea.

So then, what's left now? "Not much", but of course that's what I thought last time. Basically two things: Improving the UI of the mobile layout and improving RDF types. I got some new comments in the Solid forum regarding the types, so maybe I can do something with that. But regardless of how that goes, I'll make a new release soon (for sure before going on holidays in August).

Optionally I'll also implement a script to migrate the data from the previous version, given that it's changed so much and users from the previous version will probably "lose their data" (it won't be lost since it's made with Solid, but they won't see it in the app anymore).

Finally, after 6 long months, I can close this task!

This has been in retrospective a huge task. It may not seem as much looking at the UI, given that only a bunch of things have been added (task scheduling, descriptions, editing, etc.). But the underlying code has changed a lot, just look at this diff. I've practically rewritten from scratch most of the data layer using Soukai and I've written & documented the companion libraries. Of course, this is a side-project and I wasn't working on this fulltime, but still. You can check the current status of the app with renewed documentation here: https://github.com/NoelDeMartin/solid-focus

All of this work may seem overkill, but it isn't if you keep in mind that I want to build many apps using this architecture. I also opened the forum thread I mentioned I would about working with Solid using an Active Record pattern. You can join the discussion here: Soukai: A different way to query PODs

Something I didn't quite finish is a migration script. Meaning that people (like me) who used the app in an older version, won't be able to see their data on the new version. This is not a complete disaster because using Solid the data is still accessible in the POD. But of course, for normal users this would be a problem. Stuff like this is why the app is in pre-release and I don't recommend using it for production unless you know what you're doing. The reason why I didn't complete the migration script is because of an issue I reported 5 months ago on node-solid-server hasn't been fixed yet (actually it was fixed for a short while but it's regressed now). Unfortunately, this also makes the app incompatible with the deployed version of PODs at solid community and inrupt. Which is why I'll probably install node-solid-server on my server one of these days. I'll probably document it here, so check out my other tasks to see if I did it!