-
Notifications
You must be signed in to change notification settings - Fork 368
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
problems with requiring an abandoned nmslib
for py>3.10 (doesn't build)
#1985
Comments
hey @searchivarius - what do you think? Indeed, last official release was in 2021: https://pypi.org/project/nmslib/ @stas00 thanks for bringing this up - simple solution is just to rip out all Let's discuss. |
I have been trying to find some time to fix the release pipeline, but there have always been something new. The key problem is that Travis basically defaulted on us and other OSS projects (on paper they are supportive, but they don't really give us capacity, their build servers just time out) and I have not replaced it with something like GitHub actions. So, I certainly plan to fix the failed pipeline, but I can't give any guarantees. Feel free to implement whatever simple solution you find necessary to implement. |
@searchivarius, IMHO, making a new source release is all that is needed. The binary wheels are a bonus, but if we can build from source from the desired version that's 99% fantastic. Thank you. |
The source release won't install on Windows (building anything on Windows is a separate hell). It is also going to be some trouble to install on Linux machines, b/c each installation will require building from sources, thoughts? |
Hi @stas00 I'm not opposed to (temporarily?) ripping out the Would you be willing to send a PR to do that? |
Then for a time being only the older version will be available on Windows.
There is no problem with building from source as long as it's a released source with a version. Then it gets cached and re-used on the next rebuild. The package manager knows that the latest version is in. The problem is when one builds from git+https - in that case the package manager is forced to rebuild it every time since it can't "anchor" on a specific version. So I suggest as a first step to release just the new source version that builds with higher python versions as well as the previous python versions. And then seek out how to build binary wheels as a bonus. |
@lintool, let's see if @searchivarius can make a new source release - which would immediately solve this dependency issue, even if it requires building from source. |
@stas00 sure, let me know! |
|
I'm not sure what to do with the reply above, @searchivarius Currently all users have to build from What is your objection to doing that? |
@stas00 to make sure we are on the same page: are you talking about just fixing the source or about fixing the source and uploading it to pypi? |
Making a new version and uploading to pypi, yes. And thank you! |
Ok, then we come back to my main question: wouldn't pip try to install this
new version on every platform?
…On Mon, Sep 16, 2024, 6:58 PM Stas Bekman ***@***.***> wrote:
Making a new version and uploading to pypi, yes. And thank you!
—
Reply to this email directly, view it on GitHub
<#1985 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAGJSMVD4OLJB24KPTLIUITZW5PBRAVCNFSM6AAAAABOGB64KSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNJUGE3TENRYGY>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
oh, I think I now understand what you mean. You're saying the current HEAD isn't installable on Windows, so if you make a new release, Windows users will no longer be able to install Then perhaps change the README.md's Installation section to say:
Would that work? I think that's better than nobody being able to install it. We aren't taking a way from those Windows users to install your library - it's just going to be restricted for now to |
@stas00 yes, though it's not so simple though with the README, b/c we have a lot of users. We need to ensure smooth installation. That said, I was lucky enough to fix Appveyor very quickly (unlike some prior times when it took me days, lol) and I have some second thoughts about Travis. If these thoughts are correct, I will be able to make a release very soon. |
@stas00 following up here... please close issue if you've resolved or found a workaround? |
I've been trying for two weeks to use pyserini, to replicate a paper's results for research purposes. Tried many possible workarounds mentioned on forums and nothing seems to work for my specific situation. I'm currently trying to replicate the results from "https://github.com/texttron/hyde". I was able to install nmslib from the source with the workaround mentioned on nmslib/nmslib#538 but ran into errors when importing from pyserini.search related to numpy. The message: ImportError: cannot import name 'broadcast_to' from 'numpy.lib.stride_tricks' (/home/nuno-marques/dev_env/thesis/hyde/.venv/lib/python3.10/site-packages/numpy/lib/stride_tricks.py) Any insights or help would be much appreciated. |
update: we removed the dependency on It's up to you if you would like to close the Issue or wait until the dependency issue gets resolved. I appreciate the discussion and suggestions so far. |
pyserini depends on
nmslib
which appears to be an abandoned projectThe last released version it builds with is py3.10
To use py>3.10 one has to build from git source
You can see the discussion here nmslib/nmslib#538 with the workaround proposed here nmslib/nmslib#538 (comment) - the problem is that nobody is releasing a new version (since years!) so the fix isn't available. The workaround requires building from git source and since it has to build a binary this is very problematic as it adds a huge overhead to the CI run.
The fix to the issue was merged 3 years ago in nmslib/nmslib#488 but no new releases have been made or are planned to be made.
Not sure how you can help here, but I thought I'd ask. Our project depends on
pyserini
and thisnmslib
dependency breaks our CI if we want to switch to py>3.10.Thank you.
The text was updated successfully, but these errors were encountered: