How To Create A Learning Data Strategy
The Experience API (xAPI) is now widely used to generate and collect learning data, not just from learning platforms, but also from workplace software, systems, websites, and mobile apps. There is no question that learning data is more available than ever before. The key challenge is deciding what data to collect and why. To convert data into value, you need a learning data strategy. Follow these 7 steps to create yours.
1. Consider The 3 Key Reasons For Collecting Learning Data
To get started, consider these three key reasons to collect learning data:
- Create a more personalized and relevant experience for each worker.
- Improve the quality, effectiveness, and convenience of your learning solutions.
- Present evidence of learning’s impact on business goals.
Your learning data strategy is likely to address all three of these reasons. Keep them in mind to help guide and focus your thinking.
2. List The Questions You Want To Answer
Learning data informs your answers to important questions. List all the questions you need to answer, who is asking, and why. What decisions might they make or actions might they take with the answer. Don’t worry about how hard you think it will be to answer a question, just list all the questions you would answer if you could.
What questions are your funders asking? They are likely to have questions about L&D’s effectiveness and efficiency. To determine effectiveness, they may question whether L&D solutions are focused on the right areas of the business, reach all the right people, and produce positive results. To determine whether L&D is operating efficiently, funders may want to know how L&D manages quality, speed, volume, and cost in creating and distributing its learning solutions.
Examples of questions from funders:
- How much learning is going on?
- What are we spending on learning?
- Is learning managed efficiently and cost-effectively?
- Does L&D's capacity match the organization's demand for learning?
- What business challenges are addressed by L&D's learning products?
- Is learning having a positive impact on productivity?
What questions are asked by performance analysts, instructional developers, and User Experience Designers? This group is likely to have questions about how L&D solutions can be improved.
Examples of questions from developers:
- How easy is it for people to find highly relevant content?
- Are the right people using our learning products?
- How can we deliver products with minimal work disruption?
- How can we make content more instantly available?
- What evidence do we have that our programs are improving performance and productivity?
What other stakeholder groups have questions? For example, the compliance department may want regulatory compliance training data for a given region or department to compare with compliance incident reports. Human resources may want learning data related to onboarding, ethics, diversity, or workplace harassment training. Each functional area of the business may have questions about how learning solutions are addressing their needs.
3. Prioritize The List Of Questions
After listing all the questions you want answered, determine their relative importance. You can place the questions into rank order from most to least important or rate the importance of each question using a scale, such as high-medium-low or critical (answer now), important (answer soon), and nice-to-have (answer eventually). This will help you establish a roadmap for implementing your learning data strategy.
4. Identify Data To Inform Your Answers
What data is needed to answer each question? Identify available data and what’s missing. Keep in mind that not all data is found in systems. People may track data in spreadsheets or other ways. Identify all systems and people that are sources of the data you need and get their permission and cooperation.
5. Assess The Reliability Of Available Data
Determine the quality of available data. Studies show that data scientists spend up to 60% of their time organizing and cleaning data. You may need to prepare data before you can use it. In some cases, you may find that, although data is available, it is not reliable enough for you to use. For each data element, determine how consistently/frequently the data is entered and maintained, evaluate its accuracy, and determine what scrubbing, organizing, or mapping is needed. If possible, work with data owners and your L&D sponsors to improve data quality.
6. Define Your Requirements For Accessing Data
Define your requirements and work with data owners to get the level of access you need. Determine the frequency with which you need data updates (e.g., annually, quarterly, monthly, weekly, daily, hourly, or real time).
Define the filters you will need to explore data. For example, you may need to explore data by date range, user group, content type, or other parameters. As you think about filtering data, keep in mind the question(s) you need to answer, who’s asking, and why.
Consider personally identifiable information (PII) policies and regulations. You are likely to need aggregate data to answer most questions, but any personally identifiable data must be handled in compliance with your organization’s PII policy and local regulations, such as Europe’s General Data Protection Regulation.
7. Specify How You Want To View The Data
Design dashboards and reports to answer each of your questions. In some cases, you may want to view learning data alongside your customer’s Key Performance Indicators or productivity metrics. Develop a straw man or mock-up of each dashboard and report and specify the data elements and sources for each mock-up. Hand off your requirements, mock-ups, and data specifications to web developers for implementation.
Summary
Learning data can be used to answer key questions from L&D’s funders, stakeholders, and staff. Data can be used to personalize the learner experience, continuously improve learning solutions, and present evidence of impact on business productivity metrics. Converting data into value requires a learning data strategy that describes the questions you need to answer, the data that can inform your answers, where it is, who owns it, how reliable it is, and how it must be collected, rendered, and reported.
