Connect by JBRH Open Connect

PDF text extraction

A PDF describes where marks go on a page, not what the document says. Extracting text means reconstructing reading order from geometry, which works well for a single column and badly for tables and multi-column layouts. A scanned page contains no text at all — only an image — so extraction returns nothing rather than failing.

Status
Reference What this means
Audience
both, developer
Last verified
Product version
6.3.2

What is actually in the file#

A PDF page is a sequence of drawing operations: select a font, move to a position, show a run of glyphs. There is no paragraph, no heading and no reading order, because the format's job is to make the page look identical everywhere rather than to describe its structure.

Two consequences follow immediately. Spaces frequently do not exist as characters — a gap between words is a change of position — so word boundaries are inferred from distances. And the mapping from glyph to character lives in the font: a subset-embedded font with a missing or wrong ToUnicode map produces extraction output that is visually correct on screen and meaningless as text.

Born-digital PDF
Generated from a word processor or a report tool. Has a text layer, usually extracts well
Scanned PDF
A photograph of paper wrapped in a PDF. No text layer at all
Hybrid
A scan with an OCR text layer added underneath. Extracts, with OCR's error rate
Tagged PDF
Carries real structure — headings, lists, table cells, reading order. Uncommon, and much better to extract from

Where reconstruction goes wrong#

  • Multi-column layouts read across the columns instead of down them unless the reader groups text by region first.
  • Tables lose their grid. Cells become runs of text whose relationship has to be re-inferred from alignment, and a merged cell or a wrapped line breaks the inference.
  • Headers, footers and page numbers interleave with the body and repeat every page.
  • Hyphenation at a line break leaves a word split in two.
  • Ligatures may extract as a single character that no search will match.
  • Rotated or vertical text carries a transformation the reader must apply before positions mean anything.

None of these produce an error. They produce text — plausible, readable-looking text that says something the document did not. That is the failure mode to watch for, and it is why an extracted table is worth checking against the page before anything is decided on it.

Does Connect use PDF text extraction?#

Yes, with a documented fallback. A PDF attached to a conversation or uploaded to the workspace is read for its text layer first; where there is no usable text layer, the model reads the document instead of the extractor. The practical effect is that a scanned invoice is still answerable rather than silently empty.

Parsing runs under the same rules as every other format the file workspace accepts: DOCTYPE and ENTITY declarations are refused, and so are zip bombs. A file that tries to make the parser fetch something else, or expand to an absurd size, is rejected rather than opened carefully.

Getting a better answer out of a PDF#

  1. Prefer the original format when you have it. A spreadsheet exported as PDF has lost its cells.

    Result Extraction reads structure rather than reconstructing it.

  2. Ask a question about the document rather than asking for the whole text.

    Result A targeted question tolerates imperfect extraction; a demand for verbatim text does not.

  3. For a table that matters, check two or three values against the page.

    Result You catch a column that shifted, which is the failure that reads as correct.

  4. If nothing comes back at all, assume a scan rather than a broken file.

    Result The document is an image, and the answer will come from the model reading it rather than from extraction.

Questions#

Why did a PDF produce no text?

Almost always because it is a scan: the page is an image and there is no text layer to extract. That is not a fault in the file or in the reader. Connect falls back to having the model read the document in that case, so you can still ask about it, but a verbatim copy of the text is not available.

The extracted table has values in the wrong columns. What happened?

A PDF has no grid — a table is text positioned to look like one, and column membership is inferred from alignment. A wrapped cell, a merged cell or a narrow gap between columns can shift that inference. Where the numbers matter, the original spreadsheet is a far better source than its PDF.

Is it safe to upload a PDF from someone I do not know?

The parser refuses the two file-level attacks that matter here — external entity declarations and archives that expand out of proportion — and files are held in a workspace-scoped store under row-level security. That is a statement about parsing, not about the content: a document that asks you to do something is still a document making a request, and should be read as one.