How to catch the error when reading the pdf file

[lmw0320]

I want to check plenty of pdf files, to find the invalid files, and record them.
I had tried this method to read the files, but I got the following info without any error raised. I hope I can catch these problems which would be helpful for me to record the corresponding file name.

Could you help me to solve this, thanks in advance!

[JorjMcKie]

These messages are generated by MuPDF during open. MuPDF does all it can to make a PDF accessible.
Only for hardcore errors an exception is raised.
However, you can determine whether there were issues during open even if no exception was raised:

• Document.can_be saved_incrementally is set to False if the PDF had to be repaired (and for a number of other reasons)
• Document.is_repaired is set to True if a repair has been done during open.
• There is an internal message store, which contains error and warning messages. The standard behavior is to display error messages on sys.stderr and handle warnings silently. But there are a number of global options to control all this: please consult the documentation on the Tools class.

So you could do this:

try:
    fitz.TOOLS.reset_mupdf_warnings()  # empty the message store
    doc = fitz.open(filename)
    if doc.is_repaired:
        print(f"{filename} had to be repaired!")
        print(fitz.TOOLS.mupdf_warnings())  # prints and empties the message store
except ...:
    # hardcore error - no way to repair and open this document

[lmw0320]

Thanks for your reply. The errors I shown before had been catched. But now I have more errors need to be catched. Is it possible to fix?

[lmw0320]

by the way, Is it possible to check the PDF file whether to be two_layers? (It means the words in PDF page can be selected and copied）
----Now I just use the method of Document.getTextWords()， if any return, It means that some words can be selected. I wander if any method to check the layer num of PDF files directly.

[JorjMcKie]

Is it possible to fix?

You cannot fix them. MuPDF does that for you where possible. If not, you get an exception from the function / method you were using.
You can check fitz.TOOLS.mupdf_warnings() repeatedly to see if more warnings / errors have been generated any time after open.

If you have to do with corrupt PDFs you can also clean the whole file before your actual processing - e.g. by MuPDF's own cli mutool clean ....

[JorjMcKie]

Instead of a separate batch step using mutool clean, you can do the same thing within your PyMuPDF script:

import fitz
doc = fitz.open("problem.pdf")
# do whatever preliminary checks
# if continuing makes sense, clean file and process a memory copy:
pdfbytes = doc.tobytes(garbage=3, clean=True, deflate=True)
# recycle doc object for use in your script
doc.close()
doc = fitz.open(stream=pdfbytes)
...

[JorjMcKie]

Now I just use the method of Document.getTextWords()

There is no such method, you probably mean Page.get_text(...). Why are you saying "two layers"? Did you forget to tell me you are dealing with scanned PDFs?
There are all sorts of checks you can make: presence of fields, annotations, links, images, drawings, text.

[lmw0320]

Many thanks for your reply. I will try it later.

[lmw0320]

Now I just use the method of Document.getTextWords()

There is no such method, you probably mean Page.get_text(...). Why are you saying "two layers"? Did you forget to tell me you are dealing with scanned PDFs? There are all sorts of checks you can make: presence of fields, annotations, links, images, drawings, text.

Yes, I need to check the scanned PDF files if some files were broken and some files were not scanned correctly ( We need to confirm the words in PDFs can be selected). I just check the files by the method of Page.get_text()。 I wander if there is any better method can be used. And I concern my method may not check the file correct.

[JorjMcKie]

In general, if MuPDF could open a PDF (or any other of its supported documents plethora), this dos not mean the file is error-free - even when no_ errors occurred at open.
There is no global consistency check at that point - for performance reasons, because it would mean to process the file in several ways one time through - a horrible waste of time, given that it will be obsolete almost always.

[JorjMcKie]

We need to confirm the words in PDFs can be selected

Aha! You should have mentioned that earlier!
You can use if len(page.get_text("words")) < n to confirm that a page contains at least n words.

[lmw0320]

Thank you so much for your help！

[JorjMcKie]

Depending on what you really are talking about:
There are specialized OCR checker packages, which can validate and convert OCR file formats. You may want to look at this one: https://github.com/UB-Mannheim/ocr-fileformat.