Designing ESL Reading Comprehension Passages and Questions That Actually Test Understanding

Ask most ESL students to complete a reading comprehension quiz and watch what they actually do: skim for a phrase that matches the question, find the sentence it sits in, copy the nearest noun or number, move on. Plenty of comprehension exercises reward exactly that strategy — and a student who's good at scanning can score well on a passage they never really understood. That's the gap worth naming before building a reading comprehension exercise: locating information in a text and comprehending it aren't the same skill, and a badly designed question set tests the first while claiming to test the second.

What comprehension questions are supposed to test

A reading comprehension exercise, done properly, checks five different things: the main idea of a passage (what it's actually about, not just its topic), specific detail (a fact stated directly), inference (something true but not stated outright), vocabulary in context (what a word means here, which isn't always its dictionary definition), and the writer's purpose or tone (why this was written, and how the writer feels about it).

A question set that's all detail questions — "What time did the meeting start?", "How many people attended?" — tests memory and scanning, not comprehension. It's the easiest kind of question to write, which is exactly why it's overrepresented in quickly-assembled exercises. A stronger set mixes in at least one question a student can only answer by holding the whole passage in mind at once: what's this paragraph really arguing, not just what does it mention.

Passage length and complexity by CEFR level

Difficulty isn't just word count. A 400-word passage can be genuinely easy or genuinely hard depending on three things: sentence structure, vocabulary frequency, and how much the student has to infer versus find stated outright.

At A1–A2, that means short passages built from concrete, familiar topics, simple sentence structures (mostly single-clause or simple coordination), and high-frequency vocabulary — a student shouldn't need to guess the meaning of more than the occasional word from context. B1 can handle longer passages with a mix of concrete and slightly abstract content, more subordinate clauses, and a wider set of connectors (however, although, as a result) that the student needs to track to follow the logic. B2 passages can carry an argument or opinion piece, ask for genuine inference, and use vocabulary that isn't all high-frequency. C1–C2 is where nuance lives: implicit meaning, register shifts (formal versus conversational within the same text), longer and more complex sentences, and vocabulary a native speaker would consider unremarkable but a learner has to work for.

The practical takeaway: word count is the variable that's easiest to set and the least useful one to obsess over. A shorter passage with dense, low-frequency vocabulary can be harder than a longer one written in plain language — level the passage by ear, not just by length.

Question-writing mistakes worth checking for

A few patterns show up constantly in comprehension exercises, homemade or otherwise:

Answerable without reading. If a question can be answered from general world knowledge or from the question stem alone — "Is it good to eat healthy food?" — it's not testing the passage at all.

Distractors that are defensible. In multiple choice, a wrong option that a careful reader could reasonably argue for isn't a distractor, it's an ambiguity. If you can't point to the specific line that rules an option out, the question needs rewriting, not just a better student.

A tell in the options themselves. The correct answer is noticeably longer, more specific, or the only one that's grammatically consistent with the question stem — students learn to spot this pattern independent of the passage, and it quietly turns a comprehension test into a test-taking-strategy test.

Vocabulary tested in isolation. "What does 'address' mean?" without passage context invites the wrong (but common) definition. "What does 'address' mean in paragraph 2?" forces the student back into the text, which is the point of a vocabulary-in-context question in the first place.

Building a levelled passage and question set instead of writing one from scratch

The Reading Comprehension task type in the assignment builder is built around two different exercises, because "comprehension" doesn't mean one thing. Comprehension Questions generates a passage plus a question set — mostly multiple-choice with four options and one unambiguous correct answer, with room for one or two open/short-answer questions where the passage suits it, covering the same mix described above: main idea, detail, inference, vocabulary-in-context, and purpose or tone. Parts Removed from Text is the other mode — a Cambridge-style exercise where paragraphs or sentences are pulled from the passage for students to reinsert in the right gap. That variant is worth knowing how it actually works: the removed parts are picked deterministically, exact excerpts of the original passage rather than an AI-generated approximation, so the reconstructed text is guaranteed to match what you started with. There's no AI rewrite in that loop to introduce a mismatch.

Either mode works from one of two starting points: give the AI a topic and CEFR level and it writes an original passage, or paste or upload text you already have — a news article, a coursebook extract, anything you want your students actually reading — and it's used exactly as written, no rewriting. That second path matters if you're building around a specific text for a reason (a set text, something tied to a unit) rather than just needing a passage at the right level.

One honest limit worth knowing before you set it up: CEFR level and target word count are two separate settings, not one derived from the other. The level shapes vocabulary and sentence complexity in the generated passage; the word count is whatever you type in. Setting B1 and 800 words doesn't get corrected down to something more B1-appropriate — that combination is on you to sanity-check, the same judgment call as leveling any passage by ear, described above.

And the same review principle that applies everywhere else on this platform applies here: a generated question set is a draft, not a final exam. Worth a read-through before assigning, specifically for the two failure modes above — a distractor that's more defensible than intended, or a detail question that's technically answerable by skimming. Ten minutes checking a generated set against those two things is still faster than writing six comprehension questions from nothing, and it's the same review step you'd want to build into any leveled exercise you're setting a CEFR target for.

Where this fits with the other exercise types

Reading comprehension sits alongside gapped text, matching, and transformation drills as a different way of testing the same underlying skill — processing a text, not just producing one. Gapped text checks whether a student can track cohesion across a passage; matching checks vocabulary and collocation; transformation checks grammatical control. Comprehension questions check whether the student understood what they read, which is the one none of the others directly test — worth having in the rotation for that reason alone, not just as a change of format.

---

Generate a reading comprehension exercise at your students' level. Start Free — no credit card required.