M5.6: preserve nested annotation values in FDF exports (#1971)

* M5.6: preserve nested annotation values in FDF exports

Copy and renumber retained object graphs without expanding aliases or following
indirect cycles recursively. Refuse excessive work and propagate export errors
before the destination is written. Cover saved geometry, colour and borders,
shared graphs, streams, stale references and refusal through automation tools.

Co-Authored-By: Codex <[email protected]>

* Bound FDF annotation copying before allocation

Keep selected aliases shared, budget retained keys and values before copying, and add direct-payload and alias-count regressions. Retain copied-graph limits and document source-memory scope.

* M5.6: bound FDF cache-eviction discovery and field values

Extend the concurrent annotation-budget correction while retaining its direct-payload and alias regressions. Record annotation positions instead of resolved bodies so cache eviction cannot pin repeated parsed copies; borrow direct values and budget retained entries before cloning. Count every emitted multi-select field value before allocating its output array.

Observed pre-fix allocation-cap and field-limit failures, then fixed tests passed. Final validation: 549 scoped XFDF/engine/automation tests passed, three existing ignored; strict all-targets Clippy, WASM, fmt, layers, assets, parity and diff checks passed.

AI assistance: OpenAI Codex for @MMexicann.

* M5.6: record bounded FDF export regression evidence

Record comment cache-eviction discovery and emitted-field-value budget regressions without changing feature scores or status. Describe copied-graph limits and existing source-parser limits precisely. Final 549 scoped tests, strict Clippy, WASM, fmt, layers, assets and parity passed.

AI assistance: OpenAI Codex for @MMexicann.

* M5.6: align FDF progress with budgeted discovery

Document lazy annotation positions, cache-eviction coverage and copied-graph limits while retaining the original nested-value progress entry and all upstream notes. Estimates unchanged.

AI assistance: OpenAI Codex for @MMexicann.

* M5.6: describe FDF discovery and copied-graph limits

Align exchange documentation with the tested lazy-position discovery implementation and its copied-graph limits. Source parsing/page discovery keep existing limits.

AI assistance: OpenAI Codex for @MMexicann.

* M5.6: align milestone notes with FDF discovery limits

Retain upstream milestone history and describe the tested discovery/copy budgets without increasing estimates or claiming a total source-memory bound.

AI assistance: OpenAI Codex for @MMexicann.

* M5.6: keep nested page links out of FDF exports

Preserve graph aliases and existing budgets; exclude direct and indirect Page/Pages type names. Add saved-PDF regression and bounded type-resolution controls. Both pre-fix failures observed; 46 scoped tests, strict Clippy, WASM and formatting passed.

* M5.6: document bounded nested page exclusion

Address maintainer feedback on the existing PR. Preserve upstream work and document the tested scope and remaining validation limits.

* M5.6: record nested page-reference regression evidence

Address maintainer feedback on the existing PR. Preserve upstream work and document the tested scope and remaining validation limits.

* M5.6: retain FDF limits in milestone evidence

Address maintainer feedback on the existing PR. Preserve upstream work and document the tested scope and remaining validation limits.

* M5.6: cite nested page-reference exclusion tests

Address maintainer feedback on the existing PR. Preserve upstream work and document the tested scope and remaining validation limits.

* M5.6: respect generations while identifying page references

Address maintainer feedback on the existing PR. Preserve upstream work and document tested scope and validation limits.

* XFDF: document stale Type preservation regression

Address maintainer feedback on the existing PR. Preserve upstream work and document tested scope and validation limits.

* M5.6: record generation-aware page exclusion evidence

Address maintainer feedback on the existing PR. Preserve upstream work and document tested scope and validation limits.

* M5.6: propagate FDF export errors in usage example

Address maintainer feedback on the existing PR. Preserve upstream work and document tested scope and validation limits.

* M5.6: consolidate roadmap change and target fields

Address maintainer feedback on the existing PR. Preserve upstream work and document tested scope and validation limits.

---------

Co-authored-by: Codex <[email protected]>
Co-authored-by: skylar taylor-barrick <[email protected]>

docs/roadmap.md

@@ -1,5 +1,6 @@
 # PdfCraft roadmap: milestones and what's next
 
+> **Last reviewed:** 2026-10-18 · **Last updated:** 2026-10-18 · **Change:** minor (M5, PR #1971: bounded nested FDF export excludes source page/page-tree dictionaries; refreshed against upstream 4db6ac4a; estimates unchanged)
 > **Last reviewed:** 2026-10-15 · **Last updated:** 2026-10-15 · **Change:** minor (M14/#2383 GPU-limit candidate; estimates unchanged) · **Target:** Adobe Acrobat Pro (Acrobat DC, continuous track 26.002.21931, macOS)
 > **Last reviewed:** 2026-10-15 · **Last updated:** 2026-10-15 · **Change:** minor (M7: the Edit text caret is as tall as the row it is on; estimates unchanged) · **Target:** Adobe Acrobat Pro (Acrobat DC, continuous track 26.002.21931, macOS)
 > **Last reviewed:** 2026-10-15 · **Last updated:** 2026-10-15 · **Change:** minor (M6: CSV tab-containing headers; estimates unchanged)
@@ -131,7 +132,7 @@ total matches the parity estimate.
 | M5 | Comments (all annotation types, XFDF) | 120–200 | 88% | 12–25 | Done: notes, highlight/underline/strikeout/squiggly, text boxes, ink, lines, arrows, rectangles, ovals, stamps (dynamic, Sign Here, Standard Business), with appearance streams; replies, status, checkmarks, locking, move/resize/restyle/delete; properties dialog; panel filter and sort; hide all; summaries (comments only); XFDF/FDF import and export (partial imports preserve existing reply and review-status links); flatten; Fill & Sign with adjustable text/date size; agent tools.<br>Done since: stamps placed on a rotated page read upright (#1398); polygons, connected lines, clouds, callouts, inserted text, colour and checkmark filters; image signature pointer previews, selection handles and live image movement/resizing with proportional corners and independent edge handles; Ctrl+Enter submission for highlight comment text and replies, with Shift+Enter newlines (reconciled with upstream main for PR #1457 on 2026-10-14). Missing: custom stamps, replace-text proposals, summary layouts with the page |
 | M4 | Engine, history, save, organize | 100–180 | 95% | 8–15 | Done:<br>- command registry (menus, shortcuts and palette all use it);<br>- undo/redo; incremental, atomic and encrypted saves;<br>- autosave and crash recovery;<br>- organize, combine, extract, split and insert-from-file, with identical fonts and images stored once;<br>- CLI `edit/combine/extract/split`.<br>Done since: Combine files in its own tab (sortable, resizable columns; folders; early warnings; unlocking protected files), bookmark editing, page labels (Number pages), CLI `run`, Set Page Boxes, Crop tool, Duplicate pages, Ctrl+A / Cmd+A to select all pages in Organize, Replace pages, bookmarks from tagged headings. Missing: page transitions, recovery on the web M4: Insert-from-file preserves source bookmarks; integration tests retained with current main. |
 | M5 | Comments (all annotation types, XFDF) | 120–200 | 88% | 12–25 | Done: notes, highlight/underline/strikeout/squiggly, text boxes, ink, lines, arrows, rectangles, ovals, stamps (dynamic, Sign Here, Standard Business), with appearance streams; replies, status, checkmarks, locking, move/resize/restyle/delete; properties dialog; panel filter and sort; hide all; summaries (comments only); XFDF/FDF import and export (partial imports preserve existing reply and review-status links); flatten; Fill & Sign with adjustable text/date size; agent tools.<br>Done since: stamps placed on a rotated page read upright (#1398); polygons, connected lines, clouds, callouts, inserted text, colour and checkmark filters; image signature pointer previews, selection handles and live image movement/resizing with proportional corners and independent edge handles. Missing: custom stamps, replace-text proposals, summary layouts with the page |
-| M5 | Comments (all annotation types, XFDF) | 120–200 | 88% | 12–25 | Done: notes, highlight/underline/strikeout/squiggly, text boxes, ink, lines, arrows, rectangles, ovals, stamps (dynamic, Sign Here, Standard Business), with appearance streams; replies, status, checkmarks, locking, move/resize/restyle/delete; properties dialog; panel filter and sort; hide all; summaries (comments only); XFDF/FDF import and export (partial imports preserve existing reply and review-status links); flatten; Fill & Sign with adjustable text/date size; agent tools.<br>Done since: polygons, connected lines, clouds, callouts, inserted text, colour and checkmark filters; image signature pointer previews, selection handles and live image movement/resizing with proportional corners and independent edge handles. A refused restyle or resize of a text box another program drew leaves that comment unchanged (#1327). Make Current Properties Default on an arrow updates arrows, not plain lines (#1302). Flattening keeps a visible comment it cannot draw (#1317). Missing: custom stamps, replace-text proposals, summary layouts with the page |
+| M5 | Comments (all annotation types, XFDF) | 120–200 | 88% | 12–25 | Done: notes, highlight/underline/strikeout/squiggly, text boxes, ink, lines, arrows, rectangles, ovals, stamps (dynamic, Sign Here, Standard Business), with appearance streams; replies, status, checkmarks, locking, move/resize/restyle/delete; properties dialog; panel filter and sort; hide all; summaries (comments only); XFDF/FDF import and export (partial imports preserve existing reply and review-status links; nested FDF export uses lazy array owners and annotation positions, with retained entries budgeted before cloning under copied-graph limits and source `/Page` and `/Pages` dictionaries replaced by null at every copy depth); flatten; Fill & Sign with adjustable text/date size; agent tools.<br>Done since: polygons, connected lines, clouds, callouts, inserted text, colour and checkmark filters; image signature pointer previews, selection handles and live image movement/resizing with proportional corners and independent edge handles. A refused restyle or resize of a text box another program drew leaves that comment unchanged (#1327). Make Current Properties Default on an arrow updates arrows, not plain lines (#1302). Flattening keeps a visible comment it cannot draw (#1317). Missing: custom stamps, replace-text proposals, summary layouts with the page |
 | M6 | Forms + JavaScript | 160–320 | 50% | 40–75 | Done: filling all field types with regenerated appearances; Prepare a form (every field type, multi-select/shared properties, move/resize/delete, Field Properties General/Appearance/Position/Options/Format/Validate/Calculate); Acrobat's AF format/keystroke/validate/calculate functions and simplified field notation run natively in Acrobat's event order; tab order; push-button actions; flatten; agent tools. Form data exchange (FDF, XFDF, XML, CSV, text) done; XFDF values retain text across XML comments and processing instructions. Plain AcroForm XML imports retain all direct text around comments and processing instructions, including CDATA and character references. CSV export and spreadsheet merging quote tabs, and import distinguishes quoted tabs from TSV delimiters without changing field identities. XML/XFDF exports preserve tabs and line breaks in names and comment metadata. Refreshed against upstream `99941daa` (2026-10-15).<br>Done since: JavaScript engine (boa, sandboxed) running custom keystroke/validate/calculate/format and button scripts, document JavaScripts, console. Done since: Push-button normal labels are editable per widget through Options and form_set_props, independently of field names; rendering, undo/redo and incremental/full save/reopen are covered. (#2892). Missing: Actions tab, wider object model (annotations, layers, dialogs), XFA<br>Done since: Multi-select choice fields write ascending `/I` option indices while preserving `/V` selection order; synthetic forms/engine regressions cover reverse selection and save/reopen. CSV export preserves carriage returns in names and values. |
 | M6 | Forms + JavaScript | 160–320 | 50% | 40–75 | Done: filling all field types with regenerated appearances; Prepare a form (every field type, multi-select/shared properties, move/resize/delete, Field Properties General/Appearance/Position/Options/Format/Validate/Calculate); Acrobat's AF format/keystroke/validate/calculate functions and simplified field notation run natively in Acrobat's event order; tab order; push-button actions; flatten; agent tools. Form data exchange (FDF, XFDF, XML, CSV, text) done; XFDF values retain text across XML comments and processing instructions. Plain AcroForm XML imports retain all direct text around comments and processing instructions, including CDATA and character references.<br>Done since: JavaScript engine (boa, sandboxed) running custom keystroke/validate/calculate/format and button scripts, document JavaScripts, console. Missing: Actions tab, wider object model (annotations, layers, dialogs), XFA<br>Done since: Multi-select choice fields write ascending `/I` option indices while preserving `/V` selection order; synthetic forms/engine regressions cover reverse selection and save/reopen. CSV export preserves carriage returns in names and values. |
 | M6 | Forms + JavaScript | 160–320 | 50% | 40–75 | Done: filling all field types with regenerated appearances; Prepare a form (every field type, multi-select/shared properties, move/resize/delete, Field Properties General/Appearance/Position/Options/Format/Validate/Calculate); Acrobat's AF format/keystroke/validate/calculate functions and simplified field notation run natively in Acrobat's event order; tab order; push-button actions; flatten; agent tools. Form data exchange (FDF, XFDF, XML, CSV, text) done; XFDF values retain text across XML comments and processing instructions. XML/XFDF exports preserve tabs and line breaks in names and comment metadata. Refreshed against upstream `99941daa` (2026-10-15).<br>Done since: JavaScript engine (boa, sandboxed) running custom keystroke/validate/calculate/format and button scripts, document JavaScripts, console. Missing: Actions tab, wider object model (annotations, layers, dialogs), XFA<br>Done since: Multi-select choice fields write ascending `/I` option indices while preserving `/V` selection order; synthetic forms/engine regressions cover reverse selection and save/reopen. CSV export preserves carriage returns in names and values. |
@@ -199,6 +200,8 @@ M2 (renderer), M7 (content editing), M10 (OCR and Office), M11 (Preflight and st
 
 | Date | Change | Summary |
 |---|---|---|
+| 2026-10-18 | minor | M5, PR #1971: nested source `/Page` and `/Pages` dictionaries export as null without traversing page content or the page tree. Synthetic saved/reopened three-page regression retains unrelated metadata and aliases. FDF import semantics, copied-graph limits and estimates unchanged. |
+| 2026-10-15 | minor | M5: budget retained FDF entries before cloning; discovery keeps lazy array owners and annotation positions, with alias/cache-eviction stress fixtures. Limits cover the copied graph; source parsing/page discovery retain existing COS/parser limits, with no total source-memory bound. FDF import semantics and estimates unchanged. |
 | 2026-10-15 | minor | M7: Edit text caret height follows its row, with a kittest regression; estimates unchanged |
 | 2026-10-19 | minor | M3 #3360: retain each document’s scroll position when switching Read and the docking workspace at fixed zoom. Estimates unchanged. |
 | 2026-10-18 | minor | M4, PR #1919 review: repeated page indices apply each rotation to the latest page-local value, preserving indirect/inherited angles. Synthetic adjacent/interleaved duplicates retain siblings and page contents through incremental and full saves. Estimates unchanged. |