Example use of a fine-tuned RoBERTa large to classify employee opinions into meaningful topics.
The core of this method consists of a RoBERTa large, fine-tuned on a dataset of individual sentences representing employee's opinions on their company (pros and cons). Such dataset had been previously labeled using BERTopic, after splitting employees' input into individual sentences using an Anthropic API, in order to identify aspects and sentiment in a more precise way. This is due to the fact that employee often mention multiple topics within the same review.
This model is able to identify the following topics:
- miscellaneous / undefined (outlier sentences)
- culture and values
- occupational wellbeing & work/life balance
- leadership style
- career development and advancement opportunities
- group dynamics
- employer brand
- compensation and benefits
- customer relationships
- workplace perks
- training and development
- workplace location and commute
- structure, processes and tools
- overall employee experience
- performance management and decision-making
- physical work environment and conditions
- communication and meeting culture
- collaboration, team structure and work distribution
- lack of pros
- workplace atmosphere
- employee value and recognition
- workforce planning
It can also classify whether a sentence is a pro or a con.
Performance for topic identification during training are Accuracy 0.87 and F1 0.86; as for recognizing whether a sentence refers to pros vs cons, I got a .95 for both accuracy and F1.
Before using it, please make sure you download all the needed files and reference them accordingly in the code (in my case, I used google drive).
-
Sentiment (pros vs cons) classifier: https://drive.google.com/drive/folders/1XN3uzoaST2EWm5i8z05M4xDHVHXWJts8
-
Topics classifier: https://drive.google.com/drive/folders/1kRxSi5ftRNZEGtiCG43i2IuSfKQTJBjC
How to use it in an employee listening pipeline: This model can slot into any text-based employee feedback collection process, whether the source is open-ended survey questions, exit interviews, platforms like Glassdoor scrapes or engagement tools.
A typical workflow would look like this: collect raw employee feedback → split each response into individual sentences (in my example I use Claude, but feel free to use the method that suits you best) → run each sentence through the "pros vs cons" classifier to flag it as a pro or a con (alternatively: you can replace this with another sentiment analysis model, or emotional recognition model) → run it through the topic classifier to assign it to one of the 22 categories above → aggregate results by topic and sentiment to surface patterns across the workforce. The output can then feed directly into dashboards or reports, giving People teams a structured, quantitative view of qualitative feedback without the need for manual coding or traditional surveys.
This project was created in January 2025 as part of my final project for the Master in Artificial Intelligence and Data Analytics at the University of Milano - Bicocca.
Fur further information about the methods used to create the labeled dataset, please reach out to me.
This project is licensed under the CC BY-NC 4.0 License.