Q. How do you measure what you do in editorially in a way that aligns with your mission and strategy? What metrics do you have that align with your larger goals? At our monthly association magazine, we're trying to see if there is a meaningful measurement of progress beyond the things we already measure like ad revenue, ad page counts, postage and print costs. How do you measure quality, timeliness. and engagement?
A. You're basically talking about process vs. outcome evaluation. Process is quantifiable – all those things you mentioned, such as ad revenue.Some outcome metrics can be quantifiable as well, as in data gained from surveys and web hits.Then there's the more anecdotal, perhaps through letters to the editor, or some e-mail "editor's message box” where you invite people to share their reactions to what you publish.You can make it less anecdotal and more analytic by creating a survey that asks such questions, so that the process of data collection needn't be a mutually exclusive one.When you create a survey, however, you close the metaphorical door on open-ended remarks.
Susan R. Paisner, Senior Managing Editor, American Public Transportation Association
A. I don't close the door on open-ended remarks—I pursue them. Then I do a qualitative analysis, which groups like ideas into broad categories. For example, a recent readership survey asked readers what they like best about Trends magazine. I grouped the answers into broad categories (based on the general ideas that readers expressed):
- Relevance (timely, practical, useful, interesting)
- Breadth (variety of topics, appeal to various audience segments, variety of approaches)
- Easy to read and understand (appealing, clear, concise)
- Quality (accurate, objective, complete, thorough)
Once the answers were grouped I could count the responses in each category. This is a short take on a complicated task. To get reliable results it must be done right, but it's not rocket science.
Constance Hardesty, Editor in Chief, American Animal Hospital Association
A. For a few years we have been asked to develop similar measures, pretty much along the same lines you're talking about. We still do surveys and monitor comments, but we were specifically asked to develop things that could be quantified and tracked on a monthly basis, which is difficult to do with a survey. Some of the things we have measured to determine "engagement” include:
- Number of articles e-mailed from the publication's website (Wordpress and other blog software will track this)
- Number of participants in online polls on key issues
- Number of continuing education quizzes (which we publish) taken on key issues
- Number of unique visits to specific pages of the website
None of these are perfect indicators, but they do give you some options when the assignment is quantitative, regular measurement.
Anne Zender, Vice President of Communications, American Health Information Management Association