Things To Consider

A CONFERENCE ROOM FULL OF PEOPLE

R: Well, we've talked some about what learning objects are and how big they should be. This week I thought we could spend some time discussing when it makes sense to use learning objects.

C: When wouldn't it make sense?!? The ability to reuse materials should mean shorter development times...

V: (Cutting in) Which should mean a better bottom line. That's a key piece of our learning objects strategy...

O: (Cutting in) Or it could mean an increase in the viability of giving away free access to educational materials.

(D shakes his head unbelievingly)

R: What's wrong, D?

D: Just hillarious how everyone is ignoring the obvious problems, the hidden costs, talking about how wonderful this component-based approach is...

S: What are you talking about?

D: Well, we heard the same promises years and years ago in computer prorgamming. Object-oriented approaches would facilitate reusability of code, and this would shorten development times, and this would save money, etc., etc. Well, it's fairly well agreed that the emperor has no clothes. Sites like OOP Criticism are increasingly popular. I've been trying not to rain on the parade since that first night we started, but I'm not sure why we should expect learning objects to be any more successful than OOP.

V: (Looking at his smart phone) Well, I expect that articles with titles like "'Computer Science' is Not Science and 'Software Engineering' is Not Engineering" get people going right off the bat.

O: But there are a whole host of reasons one might consider using learning objects, right? We've all heard arguments that developing from reusable components is less expensive than writing from scratch every time. We've even heard people claim that learning objects-based approaches will be more instructionally effective.

S: (Incredulously) Cheaper I believe. But more effective?

O: As the thinking goes, since reusability means you only have to develop that algebra instruction once, you can spend some of the money you would have spent developing the next two making the first one even better.

C: Or, since you use the same instructional component several times, you may even get around to evaluating its effectiveness and revise it over time.

O: But, I think there's a problem with the effectiveness argument. As I've been thinking about the reusability paradox the last few weeks, it seems to me like the context issues should not only interfere with using learning objects in different places, but they must also interfere with different people who are themselves in different contexts learning from the materials.

V: But that can't be! Think about the disaster that would be caused if that were true! If every bit of instructional media had to be somehow tailored for each individual who used it? What would happen to the textbook industry?

D: But look where the industry is already... I mean, you don't buy a textbook, take it home, and learn calculus. You take it home, read it, only get half of it, and then go to class so that a professor *can* tailor it for you.

S: No professor I ever had did anything with my textbooks...

D: You're missing the point. When the textbook explains a principle one way, and you don't get it, you ask your professor. He then provides a *different* explanation, one he thinks more likely to speak to you. One you'll hopefully have a better chance of understanding.

C: Especially in calculus. I think the guy who wrote our text assumed eveyone taking the class already knew calculus. I think the professor and the TA both spent more time translating what the text said into understandable English than they did doing anything else.

O: Why don't people just write their textbooks so they'll be readable in the first place?

R: Let's not drift too far away from the topic, here.

O: Still, it's quite the revelation that the poster child of reusable instructional materials, I mean the textbook, has to be personalized each time it's reused. I never thought about it like that.

C: I don't want to lose O's point, though. Why don't people just make their textbooks readable in the first place, so we can understand them?

R: How would they do that?

C: I don't know. Look at the research, I guess. Figure out what works for people, and follow those principles in their teaching.

R: There's quite an argument over whether or not what you suggest is even possible. There was a series of articles in Ed Researcher exploring this very question. Have a look at Slavin's Evidence-Based Education Policies: Transforming Educational Practice and Research, Olsen's response The Triumph of Hope Over Experience in the Search for "What Works": A Response to Slavin, and Slavin's response to Olsen Education Research Can and Must Address "What Works" Questions. I'd encourage you to read all three.

V: (Scanning quickly through the pdfs on his phone) I'm not sure I'm ready to believe that educational research is basically impossible.

R: I think Olsen's point is a little more nuanced than that, V.

D: (Emphatically) Anyway, with all the criticism of OOP there has been, isn't there some criticism of the learning object way of doing things?

R: Of course. Two popular critiques are Norm Friesen's Three Objections to Learning Objects and Wiley's Learning Objects: Difficulties and Opportunities. I'd encourage you to read these, too.

V: I want to hear more about what D was saying about hidden costs.

D: Well, let's see. C - does your company have a standard process they follow for designing and developing instruction?

C: Of course!

D: Has it ever changed, or been updated?

C: Yes. About four years ago we switched from doing a lot of Authorware / Toolbook authoring for CDs, and started doing everything for the Internet. The dev team is getting into "AJAX" now, but it hasn't affected the design process a lot.

O: I get it. C, when you switched platforms from Authorware to the Internet, was it a large change? How long did it take? Tell us everything you can remember about it.

C: Oh it was painful, I can tell you that right now. I was on the team that put together the new process. We were months in preparing the process, another two weeks in retraining the designers and developers (and we actually lost a few developers who couldn't make the switch), and then everything was significantly slower for a few months. And we were short on jobs for a stretch, too, and I heard that blamed on the business people who had been writing RFP responses from the CD-ROM perspective for so long that they couldn't get used to the selling points of our new approach.

D: And there are just a few of your hidden costs. Months of time writing new process, weeks retraining people, being short-handed after losing people to the new process and the time and cost of hiring a replacement, and the very real cost of putting together a new marketing strategy and getting new types of projects. Literally hundreds of thousands of dollars I'd wager.

O: You'd wager hundreds of thousands of dollars? (Laughing) Sorry. Couldn't resist after watching Airplane! again last week.

V: Seriously, though. People never think about these sorts of costs.

C: But the costs weren't just in changing the process. The new process itself has some wild new costs associated with it. I know we're going to talk standards eventually, and I don't pretend to be an expert like S on this topic, but doing metadata for all our new materials takes *forever*. Everyday there is someone whining about how they went to school to develop cool educational materials, not fill up the card catalog like some kind of modern Marian the LIbrarian.

S: But how long does it really take to create a metadata entry for an object?

C: If we're lucky; I should say if we do things according to process, it only adds about 15 minutes per object.

V: What do you mean, if you follow the process?

C: Well, the way we do it, the metadata is supposed to be created in steps during the design and development process. Title and description information are captured during the design phase, and things like file size are captured later during development, for example. But someone invariably forgets to add their piece somewhere along the way, and the learning object gets to the end of the line incomplete, and then you have to go find the person who knows what the value of some obscure field is supposed to be, and it takes a lot longer than 15 minutes.

D: And how many objects will you create for a given project, C?

C: Well, it depends of course. We've done projects as small as 75 or so objects, and others as large as 4300 objects.

D: So even if everything goes exactly according to plan, just the metadata part of the learning objects approach adds over 1000 hours to the project. (Responding to O's stare of wonder) At 15 minutes per object, you can do four in an hour, a little over 4000 objects...

O: How does that not entirely eat up any productivity savings you have from reuse?

V: Well, you include those 1000 hours in the bid, so they're paid for - the company doesn't eat them. Still, in the early days, when you were doing this extra work without a pool of existing resources to draw on...

C: Oh, yea. It was awful! All the sourness of new designs and filling out metadata without any of the benefits of reuse, because these were the first objects we had developed. It's better now, of course. But, not being on the business side of things, I couldn't say how much better.

R: So why do companies ever expose themself to this risk of lots of extra work, lost productivity, and forecasted - but uncertain - future benefit?

C: Honestly, for us it was purely a matter of being able to stay in business. With all the RFPs coming out of Defense and Labor wanting SCORM conformance lately, what else can we do?

O: Not that any of them understand what SCORM conformance means...

V: I'm not above saying that our motivation was similar. When you're a business, either you give your customers what they want or you go under. Everybody is hot about learning objects and SCORM and things like that right now. I expect I can speak for C in saying we're all just trying to figure out how to do this stuff without going bankrupt in the meantime.

(C nods knowingly)

O: So that's it? The whole field is being tossed to and fro, and carried about with every wind of doctrine because someone decided this stuff is cool? Because 'learning objects' is the buzzword du jour? The emperor really is naked? And we're all the chamberlains walking with still greater dignity, as if we carried the train which did not exist?

R: Don't jump to that conclusion so quickly, O...

O: (Cutting in) Quickly? We've been talking for weeks now!

R: Let's think back a little. Why did you say you were interested in coming to these conversations in the first place?

O: (Caught off guard) Well, because I thought that if these things really were reusable, we might be able to provide them in an open access format for folks who really need them, and do it for free.

D: (Quietly) Forgetting the costs again...

O: Take a project like eGranery. They make agreements with individuals and organizations that build digital learning materials - learning objects by some definitions - to redistribute those materials in the developing world. The materials have already been developed for some other purpose - for profit purposes, in the case of the 278 participating publishers - and since the materials are already digital, the eGranery can make perfect copies and take those into universities in Africa at no cost to the publishers.

D: And who provides the copies to the eGranery people? Employees from the companies that could have been doing something else with their time, I expect.

O: Ok. At extremely low cost.

V: But are there not any cases of people choosing learning objects for commercial reasons other than getting the next bid?

C: I know Cisco has a large learning objects project underway. But I'm not exactly sure why.

R: For the same reason the DoD, DoL, and Boeing do. Going to all the effort of creating these reusable objects is worth it if you have either "lots" of people to train or "lots" of training to create. Cisco is always releasing new products worldwide and need a giant sales force that knows the products well enough to sell them and a support staff that knows them well enough to help people use them. It's a huge number of people, and a huge amount of training for both sales and support (on the same content, you'll notice). Boeing is the same. Thousands of different products and processes, tens of thousands or more who need the training.

S: Same for Defense. Same for Labor.

R: Right. It makes economic sense for them all to use learning objects because they're guaranteed to recover all the costs just on internal use. With a company like C or V's...

V: (Cutting in) It depends on whether we can get enough contracts in the same area to allow us to reuse materials we create...

C: (Cutting in) Which is one reason most of our companies niche very deeply in a narrow area of training.

R: Ok, I think that's about enough for tonight. At least I hope it's clear that learning objects aren't for everyone, and that they work best in very specific business contexts.

O: And humanitarian contexts.

R: Yes.

S: So tonight's big questions are?

R: Let's see...


 * 1) Why use learning objects?
 * 2) What are the costs of using learning objects that you may not immediately see?
 * 3) What are some scenarios in which the need for reuse seems to justify the cost and trouble of learning objects?

S: Not to seem to anxious to leave, but I actually have another appointment now.

C: At this time of night?

O: Must be a date!

R: Ok! That's enough! See you next week!