Recently, Craig Weiss published a ranking of Learning Records Stores (LRSs) on his blog – see LRS Rankings (Learning Record Store). When I first saw this, I was excited that xAPI was getting such great coverage. Craig’s rankings of eLearning technologies are well respected and his paying attention to LRSs is a great sign for the xAPI movement.
Unfortunately, Weiss missed the mark here.
I wish I could say that this error was along the lines of Steve Harvey’s Miss Universe mistake in 2016 or the Academy Award’s flub in naming Best Movie winner this year.
But Weiss’s errors in his assumptions for his rankings have to do with a fundamental misunderstanding of what an LRS is required to be under the ADL specification and a confusion between what the xAPI data specification delivers and what it might enable other tools to deliver.
LRSs are required to:
- receive and store xAPI statement that complies with the xAPI technical specification.
- reject statements which are not compliant
- consider statements received to be immutable – ie, the cannot be deleted
- provide access to data by other LRSs or 3rd party systems that request the data
That’s it. LRSs are about verifying xAPI formatted data, storing it and sharing it. Recently, ADL released a tool with which LRS vendors can verify that their LRS is conformant to ADL specification at https://lrstest.adlnet.gov/
A “by specification” LRS will likely never be seen by an end user – learners, instructors, data analysts. Some LRSs are building “back door” basic analytic interfaces, but those are not required by the specification. (More on analytics below.)
Weiss outlines several “Keys to Remember” when analyzing LRS (and by extension, xAPI). The first is:
There are some standards in terms of what data is captured that is seen in most LRSs. These include top influencers, xAPI, statement activity, most popular content, search extraction, data visualization (but all over the place from that standpoint), and connectors.
LRSs only accept, store, and share valid xAPI statement. xAPI statements, depending upon how they are constructed, may release the data needed to determine top influencers, most popular content, and search extraction information. xAPI data, like any data, can be used for data visualization. But none of these things are inherent in xAPI statements. xAPI provides access to new data to be analyzed (more on this below). As to connectors, yes, xAPI and LRSs utilize standard API protocols for connectors to other systems. This is not unique to xAPI – which is a powerful aspect of the specification.
Weiss’s second and third keys are true. Some LMS vendors are incorporating LRSs into their architecture which has advantages and disadvantages. This post by Rustici Software outlines the various configurations that can occur with an LRS. He seems confused that some of the LRSs have both an open source and a commercial version. xAPI is an open source standard. The “instructions” for building an LRS are open source,
He seems confused that some of the LRSs have both an open source and a commercial version. xAPI is an open source standard. The “instructions” for building an LRS are open source. Anyone is welcome to build their own. However, the reality, building your own LRS is not easy. (there are 1300 tests to pass for an LRS to be determined conformant by ADL.) So naturally, there will be many customers, or LMS vendors, who will be more interested in buying an already built, conformant LRS from a vendor than taking on the cost and effort to build their own.
Weiss then says that vendors have forgotten one of the “premises of LRSs”:
The premise (besides what it can do and its benefits) was that each learner has this data record and it captures everything (which it does), BUT and here is the kicker, if the learner leaves the company, school, etc., they take their data record with them.
This is and has never been a premise of the LRS nor the xAPI specifications. What he is confusing here is an aspirational goal that might be achieved if xAPI is widely implemented. The role of the LRS in this aspirational vision is that it is a vessel to hold and transfer xAPI statements which can be transferred to any other LRS with no worry about data configuration. Thus, ideally, a student’s learning activity, once recorded in xAPI statements technically becomes portable – dependent upon either the student or their new school or employer having an LRS it can be transferred to. These technologies are being developed. There are issues of data ownership and privacy that will impact this vision, but that is not relevant to this current discussion.
He then makes an accusation that is totally false.
Some vendors though have changed the premise of the data record transference. How?
They delete the record if the person leaves (regardless if they quit, fire, bolt, go the route of the school angle above, etc.)
As I stated above, a key feature of xAPI and LRSs is that statements are considered immutable once created. They can not be deleted per the specification. If an error has been made in the creation of a statement or statement, a second VOID statement can be generated to negate the first statement. But this is a very laborious procedure and, as far as I know, is generally used only to negate test statements so that they won’t appear in any analytics. Even in cases of test statements, I’ve been advised it is easier to simply create a new LRS and start over than to try to VOID all of the incorrect statements. This is a strawman on the issue of why the aspirational goal of lifelong learning records. Vendors are not deleting valid statements willy nilly.
Finally, the main basis on which Weiss founds his rankings on is the learning analytics interfaces that some of the LRS vendors have opted to add to their LRS offering. This add-on interface is not part of the ADL specification conformance. Why are the vendors doing it, then?
- Value. As has been discussed, building an LRS is free to anyone who wants to take on the task, so there is little value-add in building one – at least not enough to entirely support a re-selling business model.
- Visibility. A free standing LRS, if it were physical, is basically a box sitting there, running quietly, holding data. I don’t want the job of trying to sell that to someone! So there needs to be at least some minimal backend that shows the data is there and safe.
- Demonstration. At the same time, until very recently, no one has been paying attention to learning analytics (except many of these vendors who have been at the heart of the xAPI movement) so demo’ing the learning analytics capability of xAPI in a value in pushing both xAPI and the LRS.
Some of the vendors clearly are aiming at participating in the Business Information and Analytics Marketplace and it shows in their “dashboard”. Others as simply interested in providing some basic “control” information to show that the LRS is healthy and operating and will expect data will be drawn out to bigger beefier BI and Analytic tools.
The irony of all of this is if I had to rank the LRSs he has listed, I’d probably have the same top four. But Weiss’s assumptions underpinning his selections misrepresent the role of LRSs and ultimately do a disservice to the aspirations of the xAPI standard.
What do you think? Should LRSs be ranked? On what criteria? Please share your thoughts below in the Comments section.
Feature image by Ryan McGuire provided by Gratisography.