In one of our first pilots with APEXX - a fintech payments orchestration scaleup based in London - we asked the team to rate themselves on specific data and AI skills.
A few things became clear quickly:
- First, a scale from 1-5 upsets people. Some wanted to select zero. The scale itself was creating anxiety before anyone had answered a single question.
- Second, and more importantly, people assess their own skills very differently.
We had someone who rated themselves 5/5 on Google Sheets who could do considerably less than someone who had given themselves 3/5. Neither was necessarily wrong. Perhaps the 5 had always been able to do whatever they needed in a spreadsheet, so relative to their own experience, they were a 5. Perhaps the 3 wasn't sure how skilled they really were.
Same number. Completely different meaning.
There is some learning science behind why this happens - and it's worth understanding before you design any kind of skills assessment. But the short version is: self-assessment is a useful conversation starter. It's not evidence of capability.
If you want the long version, read our blog.
So we started thinking about whether Dotly, our AI learning companion, could do the assessment instead.
How Dotly Assesses Skills
Rather than asking someone to rate themselves, Dotly has a conversation.
It starts by understanding the person's role, their context, and what they're actually trying to do. It suggests a set of relevant skills and checks them with the learner - adding, removing, or adjusting based on what they say.
Dotly suggests relevant skills based on your role and checks them with you before the assessment begins
Then it asks questions. Not rate yourself 1-5 but how have you been using this in your work recently? and does generative AI count? - real questions that reveal real understanding.
Dotly asks questions that reveal real understanding - not a rating scale, a conversation
The output isn't a number the person assigned themselves. It's a confidence score on our 0-11 scale, based on what the conversation actually showed. Each skill gets its own score, with a radar chart so you can see the shape of someone's capability at a glance - where they're strong, where they're applying skills, where there are gaps.
The output is a confidence score on our 0-11 scale for each skill, with a radar chart showing the shape of someone's capability at a glance
The difference between a 6 (Confident) and a 7 (Apply) isn't self-reported. It's evidenced by what someone demonstrated in the conversation.
And crucially - Dotly can ask follow-up questions. It can push where something sounds thin. A survey can't do that.
It Works Across Skills And Contexts
The APEXX pilot assessed data and AI skills. But the same approach works for any skills or knowledge base you need to assess.
In our pilot with the Ministry of Justice, Dotly mapped assessments directly to 12 professional competencies for trainee probation officers. The assessment wasn't generic - it was built around the specific frameworks and standards that matter in that role.
For APEXX, we're now moving from general AI skills into specific fintech onboarding - assessing the knowledge someone actually needs to do their job, not just their familiarity with AI tools in the abstract.
The more specific the knowledge base, the more useful the assessment. Because Dotly works from your organisation's own content and frameworks, it can assess against what actually matters in your context - not a generic skills list someone else decided was relevant.
What This Means For Managers
A skills confidence assessment gives you three things a self-rating survey doesn't:
- A starting point that's based on conversation rather than gut feel
- Scores that reflect demonstrated understanding, not self-perception
- A record you can actually use - to assign work, prioritise learning, and show progress over time
It also means your team doesn't spend time on learning they don't need. If someone is already at a 7 on a skill, they don't need to start from scratch. Dotly knows where they are and responds accordingly.
If you want to understand more about how to design a skills assessment that actually tells you something useful, we've put together two guides: