What we've learnt from two years of building, testing and listening

The AI Skills Hub Launched This Week, Billed As The Biggest Innovation In Organised Learning Since Harold Wilson's Open University

At the moment, though, it's a hub of other people's courses - some out of date, some missing, some paid. It's completely overwhelming.

And as someone building in this space, I get it - getting learning right is genuinely hard. People have been trying for years. We see new innovations all the time. And we're a completely different platform than we were two years ago. For us, it's all been trial and error, and a lot of honest conversations about how people actually learn.

Here are the biggest lessons so far - and what we're doing now.

Keeping Content Up To Date Is An Endless Battle

AI moves fast. Last week, Lovable changed a button from Chat to Plan - now we need to update four bits of content.

ChatGPT and Claude are always changing their features and what's available on the free and paid plans.

Trying to keep things accurate is time-consuming, but it matters. It builds trust. And it stops learners feeling silly or overwhelmed when things don't match up to what they can see.

Proving Confidence And Competence Is Hard

It's a lot more than clicking a button, or time spent on a page. And it's definitely not passing a multiple choice quiz.

We've built a 10-point confidence scale that maps to specific activities on the platform - but we're still learning.

This week, in our AI for construction pilot, we offered people the option to either mark a page as read or chat to the chatbot. Even though we emailed people about the benefits of Learn with Dotly, and even added one final plea at the bottom of the page, a big chunk still just marked it as read.

Screen of Dotly recommending a Lovable course after user mentions Vibe Coding

That meant they missed the personalisation of the learning to their job, and we don't know if they really understood what they read.

But this is on us. We've made complete and Learn With Dotly look like equal options. They aren't - just reading text or watching a video, does not mean someone understands or can apply the learning. We're now rejigging things to have read, complete and confident as separate statuses.

What Even Is Intermediate?

If the options are beginner, intermediate and advanced, most people pick intermediate. People are bad at levelling themselves (studies show they get better at it the more they do it though).

We ran a pilot asking people to rate their data and AI skills before they started. We offered a 1-5 scale, and a text box to give some detail. First learning, 1-5 stressed people out as some wanted to pick 0.

Next - people are inconsistent. Specifically for spreadsheets, we had someone pick 2, but then in the text box tell us they could do things we considered advanced. And someone else picked 5 but didn't know what a pivot table was.

We don't ask anymore. We look at what someone needs to do, and support them through that. A chat with Dotly about skills and needs goes a really long way.

Similarly, we don't use intermediate. It's basically meaningless. And it's really hard to make intermediate education. Typically we just have multiple courses which help or feed into each other, rather than beginner, intermediate and advanced.

UX Really Matters

Don't make me think still holds true. People want to log in, and get learning with a few clicks of a button. This is the one we're still finessing as we build for more and more people.

We're really proud of some of our features (particularly how slick our profile personalisation is, or that Dotly can save useful learning outputs). But people don't want buttons shoving these in their face. They need to discover these and use them if useful. They are actually there to learn (or because their boss is making them).

We're slowly working on this. Running experiments on user flows and button placement.

We're seeing improvements, but we're always keen to make the learner experience as easy as possible. They have better things to think about.

We're Always Learning

Two years in, we're still learning. There's no perfect formula to getting this right, and it feels like it's always changing. But building something that actually helps people change how they work takes more than good content.

It takes listening, iterating, and being honest about what's not working.

We might be a small team (but mighty). We're living and breathing this - and we'd love to help do it better, at a national scale.

If anyone from Skills England is listening - or if we can help your team - get in touch.

Why Getting AI Learning Right is Genuinely Hard

Enjoyed this blog?