Why ROUGE Lies About Your LLM Outlines and How Grounded Fine-Tuning Fixes It

Signup Write New Story Why ROUGE Lies About Your LLM Outlines and How Grounded Fine-Tuning Fixes It by roshankumar by roshankumar | @ roshankumar Roshan Kumar is a Senior Machine Learning Engineer at Apple with nine years of professional software and machine learning.

Roshan Kumar is a Senior Machine Learning Engineer at Apple with nine years of professional software and machine learning.

Creating a draft for an extensive unstructured document might initially seem like just summarizing the primary points as a very difficult task. However, it is essentially a structured prediction problem. Each of the elements should refer to specific sections of the text; the draft should contain everything there is in the documents with no information missing or repeated; and the headings should be short, similar in structure and content.

In this paper, we will describe a supervised fine-tuning (SFT) approach which uses paragraph IDs as a common basis for tying labels, model output, and assessment. We generate training labels based on extractive candidates and generative rewording, we reject those that do not meet relevance requirements prior to initiating the training stage, and we utilize various assessors connecting evaluation results with acquisition by the reader. Also, we present cases where n-gram metrics fail us and report on the use of prompted models.

Most teams start by prompting. Asking a general-purpose model to "generate a table of contents" produces plausible results quickly, but three problems grow with scale. Titles mention content the section does not contain.

Sections go out of order or repeat. The output format varies between each run. We see this type of issue often reported out there also at NIST they put forth confabulation as a unique generative AI risk.

Fine-tuning is what usually follows. SFT of curated sets of examples did, in fact, change model action[3, 4]; also, we see small sets of high-quality data may be enough. Training is typically the issue.

The hard part is in defining the target and the evaluator, which in turn produces a better outcome. This article looks at those two elements and puts forth a reproducible SFT pipeline for structured outline generation.

Extractive versus abstractive generation. The extractive method does well in accuracy but at times reports odd results. See et al. showed that a pointer-generator model, which can copy from the source as well as generate, reduced factual errors and repetition compared with a purely abstractive baseline on news summarization. We apply the same idea to label construction rather than to the model architecture.

Segmentation. Deciding where sections begin can be learned with supervision. In one part of an outline it is possible to use one title, whilst in an adjacent segment a completely different title is necessary.

Attribution. The AIS framework and citation benchmarks turn "is this supported by an identified source?" into a question that can be answered. Paragraph IDs make the source explicit for every outline entry.

A production outline generator has to work within the confines of what a single quality metric includes.

The model has no explicit notion of the paragraph inventory

Cross-entry ordering is not enforced by free-text decoding

A title names a topic the span never discusses

Abstractive phrasing drifts toward plausible content

Twenty one-paragraph sections, or one giant section

Granularity is a product decision, not a language skill

Models use mid-context information less reliably

Table 1: Constraints a production outline generator must satisfy.

The most critical aspect is the final line: a methodology dependent on the model's recognition of structure fails in the same lengthy documents where outlines are most beneficial.

Figure 1 presents our pipeline. At its core is the output contract. Instead of free-form headings, it has a JSON array of {title, startid, endid, level} which point to paragraphs in the source text.

Figure 1: Grounded SFT pipeline. Paragraphs which have ID’s created at the time of segmentation carry those IDs through to the

Stable paragraph IDs. Every paragraph is assigned an ID, for example, [p17], which does not change with any version of a document. Therefore, the ID represents each label; it predicts and finally gives the evaluator verdict about a certain entity.

Hybrid label builder. The section boundaries need attention first. When a paper includes headings, formatting features or numbered sections of text, they will already set them. A model in this case starts specifying the boundaries as paragraph-ID intervals. This will be done under the strictest filtering and heavy spot-checking. A title for each section will be a part of either the present heading or the best lead sentence. There is a generative pass where the draft titles will be converted into short phrases that are going to be saying the same but with different words. The extractive pass defines what the sections are, while the generative pass determines what they will look like.

One caution: The use of a model in the teacher’s rewrites communicates the teacher’s wording and blind spots directly to the student in the learning process. In order to eliminate the occurrence of such interference in the evaluating process, rotating rewrites and keeping a single human-written instance in the sessions should be done.

Grounding filter. A determinative filter first eliminates titles that contain spans that are absent, overlapping, unordered, or out-of-range, implementing the same validator as mentioned below. In the next step, an entailment verification eliminates titles whose paragraphs do not support their content. Also, it is noted that models that perform entailment better do so as reported by human assessors’ reports of content fidelity.

Turning a title into a hypothesis. “Overview” is only a title, not a claim. An NLI model therefore cannot process it as-is. A title must outperform its own component words in terms of exact match in the full title. For vague titles that pass both checks, they are sent to the rubric judge for assessment of how specific they are.

Constrained decoding, and why we still validate. Grammar-constrained decoding in [24, 25] also means that the output parses and that each field has the right type, and we use it at serving time. We did not find in the JSON Schema engines that we looked at rules like “each entry starts where the previous one ended” or “children stay inside their parent”. For those, we would require a custom decoding-time state machine, which in turn would have to be redeveloped for each serving stack. Also, we found that the validator is a requirement which we used to filter training labels and to score outputs offline. We parse the syntax against the schema; the validator alone determines whether the structure is valid.

Training. In practice, fine-tuning usually follows the standard self-supervised framework, calculating loss only for the goal tokens. The splits of train and test sets happen at the source level, thus making it impossible for leakage of nearly identical templates from a certain source into the test set. Both full fine-tuning and LoRA can be utilized. LoRA only trains a small subset of weights, which allows the retraining of the model after fixing the labels to be performed in a cost- and time-effective manner. The framework utilized can be either a PyTorch- or JAX-based stack like AXLearn.

A training example is deliberately simple:

INPUT: [p1] Overview of the onboarding process... [p2] New accounts are created by...... [p24] Frequently asked questions... TARGET: [{"title": "Onboarding overview", "start_id": 1, "end_id": 3, "level": 1}, {"title": "Creating an account", "start_id": 4, "end_id": 9, "level": 1},...] The structural validator below runs in three places: in the grounding filter on labels, as a hard gate on model output at serving time, and as the first evaluator layer.

def validate_outline(entries, n_paragraphs, max_depth=3, max_title_words=12): errors = [] stack = [] # open entries: [index, start, end, next_child_start] next_top = 1 # required start of next level-1 entry; None = resync def close_to(depth): while len(stack) > depth: idx, ps, pt, nxt = stack.pop() # children exist but stop short of the parent's end if nxt not in (ps, pt + 1, None): errors.append((idx, "children_incomplete")) for i, e in enumerate(entries): s, t, lvl = e["start_id"], e["end_id"], e["level"] title = e["title"].strip() if not title or len(title.split()) > max_title_words: errors.append((i, "bad_title_length")) max_lvl = min(max_depth, len(stack) + 1) if not isinstance(lvl, int) or not 1 <= lvl <= max_lvl: errors.append((i, "bad_level")) continue # a bad entry never becomes a parent if not (1 <= s <= t <= n_paragraphs): errors.append((i, "span_out_of_range")) close_to(lvl - 1) if lvl == 1: next_top = None else: stack[-1] = None continue # a bad span never moves the cursor close_to(lvl - 1) if lvl == 1: if next_top is not None and s!= next_top: errors.append((i, "gap_or_overlap")) next_top = t + 1 else: pidx, ps, pt, nxt = stack[-1] if not (ps <= s <= t <= pt): errors.append((i, "child_outside_parent")) elif nxt is not None and s!= nxt: errors.append((i, "gap_or_overlap")) stack[-1] = t + 1 stack.append([i, s, t, s]) close_to(0) if next_top!= n_paragraphs + 1: errors.append((-1, "incomplete_coverage")) return errors

In Level-1 we see that entries which in turn also begin where the previous left off and thus between them full coverage of each paragraph is achieved. Nested entries title their parent the same way: the first child starts at the parent's first paragraph, siblings are contiguous, and the last child ends at the parent's last paragraph. Levels cannot be skipped, and level 0 or anything deeper than max_depth is rejected.Entries that are invalid or outside the allowed range will be reported once, then ignored. They will neither produce children nor shift the intended start, and a single invalid entry will not trigger a chain of erroneous vapor overlap errors.

Instead of providing a boolean response, the function gives back its coded indications of the system error. These codes present the structure of errors in the rectification process. We see that mostly what happens is gap or overlap, also what we have is that which is of low quality does not present.

At the same time the same contract applies. When values do not pass validation we go through the process again once. If a retry fails we present an extractive outline.

An extractive outline uses the sections from the document and headers if present which at that point pass the validator. In the case the document does not have headers which is a large in most cases determination by the model we use a simplified lexical cohesion segmentation approach that is similar to TextTiling with what we term lead sentence headers. We present this as a weak solution which we separately report the results for headerless fall back.

For the great majority of readers a structurally valid out line is more useful than a fluent out line which has wrong spans and the fall back sets a floor for the fine tuned model.

ROUGE and BLEU measure n-gram overlap with a reference [16, 17], and for outlines they fail in both directions: a correct but paraphrased title is penalized, and a structurally broken outline can still score well. Onboarding overview of what we do in that process of getting started in the platform “Creating out accounts” “We will be verifying your details once you begin the process of account creation which may include a series of questions to confirm your identity for security purposes” also see how we set up your billing structure which includes various plans to fit your specific needs “How to invite your colleagues to start using the product with you in their workflows”.

Also we have the fine print that sets the rules for role assignment and what it means for the people you bring in as part of your team “And finally what happens after you finish setting things up and beginning your journey here” Candidate A has just taken the first 4 topics straight from what is provided without change, put in a different order then has combined our 5th and 6th into a single step, then has omitted the last one completely so that we as authors had no say in what is included or excluded in that final 1/3 of the onboarding process. Candidate B did an excellent job at rewriting the actual titles used and also followed the authors structure and order exactly as presented. Also note that which paragraphs in the original were not covered, that is also what is not present in the write ups by candidate B.

A (copied titles, merged section, missing conclusion)

Table 2: ROUGE evaluated over the joined and lowered versions of each candidate’s title against the reference (example put forth rather than from production data). For overlap measures which goes to A; a human reader and evaluator below go to B.

The evaluator stack therefore has layers, ordered from cheapest and most deterministic to most expensive (Figure 2):

This ‘release metric’ is in a product shape, what we may call a good outline rate which is the count of held out documents that pass all the auto layers and meet the auto rubric threshold. Per document -- all or nothing this metric brings to light what other average per entry scores may hide. If one out of twenty spans fails which in turn ruins the purpose for that particular reader; but if you average across entries that one failure brings in almost no notice at all. That is, it is this multi-metric argument applied to a single product.

Since at present there is no independent verifiable same corpus outline benchmark, Table 3 reports the relevant peer reviewed public data instead. The results stated do relate to different tasks, meaning that they cannot be considered as an assessment of the proposed system. Future head-to-head evaluations will involve prompted generation, extractive-only SFT, generative SFT, and hybrid label SFT, carrying out evaluations using the same source distinct splits, validation rules, and human audit.

Introduces a large Wikipedia-derived labeled segmentation dataset and demonstrates generalization to unseen natural text.

Public basis for learning contiguous document boundaries.

Not an evaluation of generated outline titles or hybrid SFT.

G-Eval reports Spearman correlation 0.514 with human judgments on a summarization task.

Supports using a rubric-based evaluator calibrated against human ratings.

Correlation is not an outline acceptance rate; different task and benchmark.

Public corpus of government reports with long documents and reference summaries; question-summary hierarchies are also available.

Potential public source corpus for document-structure evaluation.

No hybrid-SFT outline performance or same-set ablation reported.

Proposed hybrid outline SFT (this article)

Paragraph-ID grounded outline generation

No publicly verified comparative outcome reported in this manuscript.

Specified method and validation protocol; prospective evaluation.

Automatic pass, human audit, fallback, sample size and 95% CIs require new measurements.

Table 3: Publicly reported evidence relevant to outline segmentation and evaluation. Results are reproduced from the cited publications on their original tasks; they are NOT a held-out comparison of the proposed hybrid SFT pipeline. No original system-level acceptance rate, fallback rate, or confidence interval is claimed.

Supervised document division and the final product are done by a human. We do not see that hybrid-label SFT is better than prompted SFT, extractive-only, and generative-only approaches that use the same dataset. What is needed is a controlled evaluation against a common test corpus which includes human audits and bootstrapping methods before we can say that [7, 19].

Cost is what determines the price. Through the use of paragraph IDs, which is what makes chunking a better option, we are able to break up long texts into different sets of paragraphs in various overlapping windows, which in turn also we see to do a great job in terms of mid-context performance. The issue we are presented with is that of putting these windows back together.

The structural validator is at no cost and goes first, also expensive judge calls are for valid outputs.

The documents inputted are considered insecure, and this increases the chances of manipulation. Only the data that is validated is a title and an integer figure. We put in place a process to screen out training documents that contain sensitive issues, as once it is fine-tuned, we may be able to recover the text used for training. In the “Map”, “Measure” and “Manage” stages of the NIST AI RMF, we document these issues as well as the results of the conducted evaluations.

We log into our system every prediction that is put out, as well as the model it is based on, the prompt we used and the validation performance we get.

Lesson 1: Fix labels before tuning hyperparameters. When a label falls within the correct span but starts or ends in the wrong place, the model can learn that loose boundaries are acceptable and repeat the mistake. The proposed remedy is to correct those spans before measuring model size; as a result, no controlled model-size ablations are currently reported. Typed validator errors help diagnose the issue. If most withheld errors are vapor overlap, the labels should be examined before changing the model. This is why Figure 1 places the grounding filter before the SFT dataset.

Lesson 2: Design the fallback first. When the fine-tuned model fails a deterministic extractive fallback which in turn puts the system at a safe state, also sets a base performance that the model must outperform.

Having longer context windows will eliminate the necessity of chunking naturally, but it will not eliminate the need for grounding in a direct way. A continuing development is that of attribution as a first-class output: models that generate segments or references by conception and are assessed with attribution metrics, not based on overlap [8, 9].

Reliable outlines are a result of the model’s structure, which in turn is a result of its architecture rather than the model’s size. Use the same ID for each paragraph in every outline you create and have every outline element reference those IDs. Create labels that are extractive and very ground in the text and for the title which is to be generated, use a clean, easy-to-read format; also do a pre-training filter. Then evaluate the outlines in stages from a computer-based validator to a human review, which should mean that the better the score, the better the outline for the reader.

Z. Ji, N. Lee, R. Frieske, et al., "Survey of Hallucination in Natural Language Generation," ACM Computing Surveys, vol. 55, no. 12, 2023. [Online]. Available: https://arxiv.org/abs/2202.03629

National Institute of Standards and Technology, "Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile," NIST AI 600-1, Jul. 2024. [Online]. Available: https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf

J. Wei, M. Bosma, V. Y. Zhao, et al., "Finetuned Language Models Are Zero-Shot Learners," in Proc. Int. Conf. Learning Representations (ICLR), 2022. [Online]. Available: https://arxiv.org/abs/2109.01652

L. Ouyang, J. Wu, X. Jiang, et al., "Training language models to follow instructions with human feedback," in Advances in Neural Information Processing Systems 35 (NeurIPS), 2022. [Online]. Available: https://arxiv.org/abs/2203.02155

C. Zhou, P. Liu, P. Xu, et al., "LIMA: Less Is More for Alignment," in Advances in Neural Information Processing Systems 36 (NeurIPS), 2023. [Online]. Available: https://arxiv.org/abs/2305.11206

A. See, P. J. Liu, and C. D. Manning, "Get To The Point: Summarization with Pointer-Generator Networks," in Proc. 55th Annual Meeting of the ACL, 2017, pp. 1073–1083. [Online]. Available: https://aclanthology.org/P17-1099/

O. Koshorek, A. Cohen, N. Mor, M. Rotman, and J. Berant, "Text Segmentation as a Supervised Learning Task," in Proc. NAACL-HLT, 2018, pp. 469–473. [Online]. Available: https://aclanthology.org/N18-2075/

H. Rashkin, V. Nikolaev, M. Lamm, et al., "Measuring Attribution in Natural Language Generation Models," Computational Linguistics, vol. 49, no. 4, pp. 777–840, 2023. [Online]. Available: https://aclanthology.org/2023.cl-4.2/

T. Gao, H. Yen, J. Yu, and D. Chen, "Enabling Large Language Models to Generate Text with Citations," in Proc. EMNLP, 2023, pp. 6465–6488. [Online]. Available: https://aclanthology.org/2023.emnlp-main.398/

N. F. Liu, K. Lin, J. Hewitt, et al., "Lost in the Middle: How Language Models Use Long Contexts," Transactions of the ACL, vol. 12, pp. 157–173, 2024. [Online]. Available: https://aclanthology.org/2024.tacl-1.9/

J. Maynez, S. Narayan, B. Bohnet, and R. McDonald, "On Faithfulness and Factuality in Abstractive Summarization," in Proc. 58th Annual Meeting of the ACL, 2020, pp. 1906–1919. [Online]. Available: https://aclanthology.org/2020.acl-main.173/

W. Kryściński, B. McCann, C. Xiong, and R. Socher, "Evaluating the Factual Consistency of Abstractive Text Summarization," in Proc. EMNLP, 2020, pp. 9332–9346. [Online]. Available: https://aclanthology.org/2020.emnlp-main.750/

E. J. Hu, Y. Shen, P. Wallis, et al., "LoRA: Low-Rank Adaptation of Large Language Models," in Proc. ICLR, 2022. [Online]. Available: https://arxiv.org/abs/2106.09685

A. Paszke, S. Gross, F. Massa, et al., "PyTorch: An Imperative Style, High-Performance Deep Learning Library," in Advances in Neural Information Processing Systems 32 (NeurIPS), 2019. [Online]. Available: https://proceedings.neurips.cc/paper/2019/hash/bdbca288fee7f92f2bfa9f7012727740-Abstract.html

M. Lee, C. Lan, T. Gunter, et al., "AXLearn: Modular, Hardware-Agnostic Large Model Training," arXiv:2507.05411, 2025. [Online]. Available: https://arxiv.org/abs/2507.05411

C.-Y. Lin, "ROUGE: A Package for Automatic Evaluation of Summaries," in Text Summarization Branches Out (ACL Workshop), 2004, pp. 74–81. [Online]. Available: https://aclanthology.org/W04-1013/

K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu, "BLEU: a Method for Automatic Evaluation of Machine Translation," in Proc. 40th Annual Meeting of the ACL, 2002, pp. 311–318. [Online]. Available: https://aclanthology.org/P02-1040/

J. Novikova, O. Dušek, A. Cercas Curry, and V. Rieser, "Why We Need New Evaluation Metrics for NLG," in Proc. EMNLP, 2017, pp. 2241–2252. [Online]. Available: https://aclanthology.org/D17-1238/

Y. Liu, D. Iter, Y. Xu, et al., "G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment," in Proc. EMNLP, 2023, pp. 2511–2522. [Online]. Available: https://aclanthology.org/2023.emnlp-main.153/

L. Zheng, W.-L. Chiang, Y. Sheng, et al., "Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena," in NeurIPS Datasets and Benchmarks Track, 2023. [Online]. Available: https://arxiv.org/abs/2306.05685

P. Wang, L. Li, L. Chen, et al., "Large Language Models are not Fair Evaluators," in Proc. 62nd Annual Meeting of the ACL, 2024, pp. 9440–9450. [Online]. Available: https://aclanthology.org/2024.acl-long.511/

P. Liang, R. Bommasani, T. Lee, et al., "Holistic Evaluation of Language Models," Transactions on Machine Learning Research, 2023. [Online]. Available: https://arxiv.org/abs/2211.09110

National Institute of Standards and Technology, "Artificial Intelligence Risk Management Framework (AI RMF 1.0)," NIST AI 100-1, Jan. 2023. [Online]. Available: https://www.nist.gov/itl/ai-risk-management-framework

B. T. Willard and R. Louf, "Efficient Guided Generation for Large Language Models," arXiv:2307.09702, 2023. [Online]. Available: https://arxiv.org/abs/2307.09702

S. Geng, M. Josifoski, M. Peyrard, and R. West, "Grammar-Constrained Decoding for Structured NLP Tasks without Finetuning," in Proc. EMNLP, 2023. [Online]. Available: https://aclanthology.org/2023.emnlp-main.674/

M. A. Hearst, "TextTiling: Segmenting Text into Multi-paragraph Subtopic Passages," Computational Linguistics, vol. 23, no. 1, pp. 33–64, 1997. [Online]. Available: https://aclanthology.org/J97-1003/

L. Huang et al., “Efficient Attentions for Long Document Summarization,” 2021. GovReport dataset and documentation: https://gov-report-data.github.io/.

Roshan Kumar is a Senior Machine Learning Engineer with more than ten years of software engineering experience. His current work focuses on large language model fine-tuning, structured and grounded generation, prompt optimization and automated evaluation for search-facing applications. Before that, he spent five years building event-driven serverless backends and microservices for large-scale retail inventory replenishment and purchase-order management. He holds an MS in Computer Science from New York University.