Sanitize null bytes before ingestion

[laoqiu233]

Description

When ingesting documents using Postgres some PDF documents could cause ValueError: A string literal cannot contain NUL (0x00) characters.. This PR replaces all null bytes before ingesting the documents to make sure this error doesn't happen.

Type of Change

• Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?

I ran the pgpt server locally using Ollama and Postgers as both the vector and node store. Before the fix document was not injected, and after adding the sanitization the PDF was successfully ingested and stored.

• I stared at the code and made sure it makes sense

Test Configuration:

• Hardware: MacbookPro M2Pro
• Toolchain: Ollama, Postgres

Checklist:

• My code follows the style guidelines of this project
• I have performed a self-review of my code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• Any dependent changes have been merged and published in downstream modules
• I ran make check; make test to ensure mypy and tests pass

[jaluma]

Thanks you!
Can you add a comment to explain why this is necessary?

[laoqiu233]

Thanks you! Can you add a comment to explain why this is necessary?

Done! :]

[jaluma]

LGTM!

Thank you!