In one of our first pilots with APEXX - a fintech payments orchestration scaleup based in London - we asked the team to rate themselves on specific data and AI skills.

A few things became clear quickly:

First, a scale from 1-5 upsets people. Some wanted to select zero. The scale itself was creating anxiety before anyone had answered a single question.
Second, and more importantly, people assess their own skills very differently.

We had someone who rated themselves 5/5 on Google Sheets who could do considerably less than someone who had given themselves 3/5. Neither was necessarily wrong. Perhaps the 5 had always been able to do whatever they needed in a spreadsheet, so relative to their own experience, they were a 5. Perhaps the 3 wasn't sure how skilled they really were.

Same number. Completely different meaning.

There is some learning science behind why this happens - and it's worth understanding before you design any kind of skills assessment. Read more about the learning science here. But the short version is: self-assessment is a useful conversation starter. It's not evidence of capability.

So we started thinking about whether Dotly, our AI learning companion, could do the assessment instead.

How Dotly Assesses Skills

Rather than asking someone to rate themselves, Dotly has a conversation.

It starts by understanding the person's role, their context, and what they're actually trying to do. It suggests a set of relevant skills and checks them with the learner - adding, removing, or adjusting based on what they say.

Dotly suggesting a list of eight data and AI skills for a Lead Data Scientist, asking the learner if they feel right and if anything should be added or swapped out

Dotly suggests relevant skills based on your role and checks them with you before the assessment begins

Then it asks questions. Not rate yourself 1-5 but how have you been using this in your work recently? and does generative AI count? - real questions that reveal real understanding.

A conversation between Laura and Dotly in the Skills Confidence Assessment, with Laura asking if generative AI counts and Dotly asking how she has been using it in her work

Dotly asks questions that reveal real understanding - not a rating scale, a conversation

The output isn't a number the person assigned themselves. It's a confidence score on our 0-11 scale, based on what the conversation actually showed. Each skill gets its own score, with a radar chart so you can see the shape of someone's capability at a glance - where they're strong, where they're applying skills, where there are gaps.

A Skills Confidence Assessment result for a Lead Data Scientist showing an overall score of 6.5 out of 11 rated as Confident, with a radar chart and individual confidence scores for eight skills including Python programming, AI and machine learning application, and data analysis

The output is a confidence score on our 0-11 scale for each skill, with a radar chart showing the shape of someone's capability at a glance

The difference between a 6 (Confident) and a 7 (Apply) isn't self-reported. It's evidenced by what someone demonstrated in the conversation.

And crucially - Dotly can ask follow-up questions. It can push where something sounds thin. A survey can't do that.

It Works Across Skills And Contexts

The APEXX pilot assessed data and AI skills. But the same approach works for any skills or knowledge base you need to assess.
In our pilot with the Ministry of Justice, Dotly mapped assessments directly to 12 professional competencies for trainee probation officers. The assessment wasn't generic - it was built around the specific frameworks and standards that matter in that role.

For APEXX, we're now moving from general AI skills into specific fintech onboarding - assessing the knowledge someone actually needs to do their job, not just their familiarity with AI tools in the abstract.

The more specific the knowledge base, the more useful the assessment. Because Dotly works from your organisation's own content and frameworks, it can assess against what actually matters in your context - not a generic skills list someone else decided was relevant.

What This Means For Managers

A skills confidence assessment gives you three things a self-rating survey doesn't:

A starting point that's based on conversation rather than gut feel
Scores that reflect demonstrated understanding, not self-perception
A record you can actually use - to assign work, prioritise learning, and show progress over time

It also means your team doesn't spend time on learning they don't need. If someone is already at a 7 on a skill, they don't need to start from scratch. Dotly knows where they are and responds accordingly.

If you want to understand more about how to design a skills assessment that actually tells you something useful, we've put together two guides:

Why We Stopped Asking People to Rate Their Own Skills

Want to put this into practice?