PROJECT
Quality Systems & Continuous Improvement

Measuring Quality So It Can Be Improved

Quality metrics are only useful when they accurately represent the work being performed and help people decide what to do next.

At LibreDigital, I inherited a quality process that calculated errors against the total number of pages in a publication. That measurement didn't account for whether those pages were actually part of the converted digital product, and the resulting data was used primarily to identify individual operator errors.

I redesigned the quality framework around a different purpose. Instead of using the data primarily to identify individual mistakes, I wanted it to show us where performance was changing and help us understand why.

That required changing both how quality was measured and what happened after a problem appeared. The resulting framework made performance more comparable, established shared definitions for errors, and created enough lead time to investigate recurring problems before contractual quality was at risk.

| The goal wasn't simply to count errors. It was to create information we could act on.

Measuring the Work That Actually Happened

The existing calculation treated every page in a publication as an opportunity for error, even when some pages, such as advertisements, were intentionally excluded from the digital product.

I changed the measurement to calculate errors against the number of content pages actually being converted.

That distinction made comparisons more meaningful. Ten errors across ten converted pages represented substantially different performance from ten errors across one hundred converted pages, even though the raw error count was identical.

Turning Publisher Requirements into Verifiable Production Standards

For eBooks, each publisher imprint had its own conversion requirements, covering everything from metadata and navigation to typography, colors, logos, and the treatment of individual book sections. Vendors needed a reliable way to apply those requirements, and our internal quality team needed a way to verify the finished work.

I introduced imprint-specific production checklists that translated those requirements into individual, actionable checks. Vendor operators used them during conversion and confirmed the steps they had completed. When internal reviewers identified an error, the completed checklist provided a shared record of what the vendor had confirmed and where the delivered EPUB differed from the requirements.

The checklists were interactive PDFs authored in Adobe FrameMaker. As requirements changed, maintaining them became a problem of its own. Every time I regenerated a PDF, I had to recreate its interactive form fields manually.

I found and implemented the TimeSaver plugin, which allowed me to define the form fields directly in the FrameMaker source. I could then revise the requirements and regenerate the interactive PDF without rebuilding the fields.

The result was a maintainable quality-control process connecting publisher requirements, vendor execution, and internal review. The checklists served as production guidance, documented confirmation of completed work, and a reference for resolving quality discrepancies.

Turning Error Definitions Into Structured Data

I created a standardized taxonomy of error types, categories, severity levels, and definitions for reviewers evaluating converted publications.

Severity reflected the impact and context of the error. Structural problems such as missing content or broken continuation links were high severity. For proofreading errors, severity also depended on context. A typo in prominent content, such as a headline or byline, carried more weight than the same kind of error within ordinary article text.

The taxonomy served two purposes. It gave operators and reviewers a shared reference for what each error meant, and the same controlled vocabulary powered data validation in the review spreadsheet so reviewers selected standardized values when reporting errors.

Creating Time to Correct Problems

I didn't want the contractual quality requirement to be the point at which we discovered that performance had deteriorated.

I introduced an internal requirement that teams maintain an average error rate of 10% or lower. This standard was intentionally more stringent than the quality requirements in our retail-channel agreements.

The difference created a proactive quality buffer. Weekly reporting could surface declining team or vendor performance while there was still time to investigate the cause and take corrective action before contractual quality was at risk.

When an “Operator Error” Was a Product Problem

Weekly quality reporting revealed a recurring pattern of errors from one external vendor. Rather than treating the pattern solely as an operator-performance problem, I investigated what was happening.

The vendor had encountered a software defect and developed a workaround to compensate for it without reporting the underlying problem.

I gathered details about the issue, documented the expected and actual system behavior, and brought the problem to Product as a fix request that was routed to Engineering.

The quality data had surfaced the symptom. Investigating the pattern revealed the actual cause.

| The appropriate intervention depended on the cause of the error, not simply the fact that an error had occurred.