Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: load model library when using pyinstaller to build executable fi… #2530

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

fix: load model library when using pyinstaller to build executable fi… #2530

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 28 additions & 6 deletions gpt4all-bindings/python/gpt4all/_pyllmodel.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
import platform
import re
import subprocess
from pathlib import Path
import sys
import threading
from enum import Enum
Expand Down Expand Up @@ -54,16 +55,37 @@


def load_llmodel_library():
"""
Loads the llmodel shared library based on the current operating system.

This function attempts to load the shared library using the appropriate file
extension for the operating system. It first tries to load the library with the
'lib' prefix (common for macOS, Linux, and MinGW on Windows). If the file is not
found and the operating system is Windows, it attempts to load the library without
the 'lib' prefix (common for MSVC on Windows).

Returns:
ctypes.CDLL: The loaded shared library.

Raises:
OSError: If the shared library cannot be found.
"""
# Determine the appropriate file extension for the shared library based on the platform
ext = {"Darwin": "dylib", "Linux": "so", "Windows": "dll"}[platform.system()]

# Define library names with and without the 'lib' prefix
library_name_with_lib_prefix = f"libllmodel.{ext}"
library_name_without_lib_prefix = "llmodel.dll"
base_path = MODEL_LIB_PATH

try:
# macOS, Linux, MinGW
lib = ctypes.CDLL(str(MODEL_LIB_PATH / f"libllmodel.{ext}"))
except FileNotFoundError:
if ext != 'dll':
# Attempt to load the shared library with the 'lib' prefix (common for macOS, Linux, and MinGW)
lib = ctypes.CDLL(str(base_path / library_name_with_lib_prefix))
except OSError: # OSError is more general and includes FileNotFoundError
if ext != "dll":
raise
# MSVC
lib = ctypes.CDLL(str(MODEL_LIB_PATH / "llmodel.dll"))
# For Windows (ext == 'dll'), attempt to load the shared library without the 'lib' prefix (common for MSVC)
lib = ctypes.CDLL(str(base_path / library_name_without_lib_prefix))
Comment on lines 57 to +88
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Documentation is great, but this is a bit much. All this change really needs to be is:

  • Replace FileNotFoundError with OSError
  • Add a comment mentioning the specific OSError that appears when using pyinstaller (it will at least have an error code, but often it is a specific subclass of OSError)
    • Example: except OSError: # e.g. FileNotFoundError, PermissionError (pyinstaller)
  • Make the quotes consistent
  • Add a concise docstring - a sentence or two is more than enough

Anything else is unnecessary and IMO too verbose.

Ideally the except clause is as narrow as possible. If there is a specific exception that applies:

try:
    ...
except (FileNotFoundError, PermissionError):  # pyinstaller raises PermissionError
    ...

Otherwise:

try:
    ...
except OSError as e:
    if e.errno not in (errno.ENOENT, errno.EPERM):  # pyinstaller fails with EPERM
        raise
    ...

And if we're going to add a return type to the docstring, I would prefer we annotate it as well (load_llmodel_library() -> ctypes.CDLL:).

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank a bunch! I will make changes later


return lib

Expand Down