upgrade to flint3

[mezzarobba]

Upgrade to flint3.

Current Sage versions are not compatible with flint ≥ 3, and, though the diff is not huge, there are enough changes that versions including this PR will be incompatible with flint < 3.

Fixes #20003.
Closes #35993 as no longer relevant.

Related PRs in upstream projects:

• make Singular build with flint3 Singular/Singular#1177
• fix detection of ntl flintlib/flint#1408
• header-only ntl interface flintlib/flint#1489
• distribute configure, aclocal.m4 flintlib/flint#1492
• Further fixes to $DESTDIR handling flintlib/flint#1611
• Make msolve build with flint 3 alpha algebraic-solving/msolve#76
• Flint3 flatsurf/e-antic#264

Additional changes still needed for optional packages to work:

• symengine and symengine_py 0.11 #36677
• upgrade e-antic
• possibly more

Planned follow-ups:

• Autogenerate flint headers #36449
• Flint generic rings #36433

[mezzarobba]

Things look pretty good :-). Mostly a few benign output changes. I've reported the only dubious one at flintlib/flint#1410; let's wait for Fredrik's answer and the next alpha to decide how to update the tests. Hopefully it will also be possible to drop the patches at that point. We should also move the contents of libs/arb etc. to libs/flint and probably do some other cleanup of this kind. And update spkg-configure to check that flint is built with NTL support.

[mezzarobba]

See also #35993.

[videlec]

(I can not push to your branch)

diff --git a/src/sage/libs/flint/fmpz_poly.pyx b/src/sage/libs/flint/fmpz_poly.pyx
index cfcbea9090..0034494247 100644
--- a/src/sage/libs/flint/fmpz_poly.pyx
+++ b/src/sage/libs/flint/fmpz_poly.pyx
@@ -23,12 +23,15 @@ from cpython.sequence cimport *
 
 from cysignals.memory cimport sig_free
 
+from .fmpz cimport fmpz_init_set_readonly, fmpz_clear_readonly, fmpz_init, fmpz_clear, fmpz_set_mpz, fmpz_get_mpz
+
 from sage.arith.long cimport pyobject_to_long
 from sage.cpython.string cimport char_to_str, str_to_bytes
 from sage.libs.flint.fmpz cimport *
 from sage.structure.sage_object cimport SageObject
 from sage.rings.integer cimport Integer
 
 cdef class Fmpz_poly(SageObject):
 
     def __cinit__(self):

[videlec]

At 6ef8bd2 flint fails to build on my machine (linking problem with NTL)

gcc -g -g -O2 -shared -Wl,-soname,libflint.so.18  build/generic_files_merged.lo build/thread_pool_merged.lo [...] build/ca_mat_merged.lo build/interfaces/NTL-interface.lo -o libflint.so.18.0.0 -Wl,-rpath-link,/home/doctorant/sage/local/lib -L/home/doctorant/sage/local/lib -Wl,-rpath,/home/doctorant/sage/local/lib -Wl,-rpath-link,/home/doctorant/sage/local/lib -L/home/doctorant/sage/local/lib -Wl,-rpath,/home/doctorant/sage/local/lib  -pthread -lmpfr -lgmp -lm  -lstdc++ -lntl
/usr/bin/ld: _ZN3NTL9zz_pEInfoE: TLS reference in build/interfaces/NTL-interface.lo mismatches non-TLS definition in /home/doctorant/sage/local/lib/libntl.so section .bss
/usr/bin/ld: /home/doctorant/sage/local/lib/libntl.so: error adding symbols: bad value
collect2: error: ld returned 1 exit status
make[5]: *** [Makefile:377: libflint.so.18.0.0] Error 1

[mezzarobba]

+from .fmpz cimport fmpz_init_set_readonly, fmpz_clear_readonly, fmpz_init, fmpz_clear, fmpz_set_mpz, fmpz_get_mpz

This is with the version from yesterday afternoon, where I forgot the line

 from sage.libs.flint.fmpz cimport *

isn't it?

[mezzarobba]

TLS reference in build/interfaces/NTL-interface.lo mismatches non-TLS definition

Did you build sage from scratch or incrementally? Could it be that there are somehow two different versions of NTL at play here?

[videlec]

TLS reference in build/interfaces/NTL-interface.lo mismatches non-TLS definition

Did you build sage from scratch or incrementally? Could it be that there are somehow two different versions of NTL at play here?

I obtained that after a make distclean and ./configure --with-system-flint=no --with-system-giac=no --with-system-ntl=no --with-system-singular=no --with-system-symengine=no --with-system-msolve=no.

[mezzarobba]

Can you post the full build logs somewhere?

[videlec]

• config.log
• install.log.gz

[mezzarobba]

Ok, I can reproduce the issue...

(edit:) ...but with an incremental build. I see this error if I (1) first build sage with system NTL, (2) run ./configure --with-system-ntl=no (3) run make build. If however I then (4) run ./sage -f flint, then things appear to work again.

(edit:) ok, actually I see the issue when building from scratch too when I use --with-system-ntl=no.

[videlec]

Thanks. And could you see if you can reproduce the issue when building flint (with the ntl interface enabled!) separately from sage?

Flint standalone compiled. Configured with

$ ./configure --with-ntl --with-gmp --with-mpfr --with-blas --disable-static

[mezzarobba]

@videlec could you test again with this commit?

[videlec]

I encouter the same kind of linking error against ntl but now with giac instead of flint. Probably unrelated to the PR.

libtool: link: g++ -std=gnu++11  -fPIC -DPIC -shared -nostdlib /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/crti.o /usr/lib/gcc/x86_64-linux-gnu/11/crtbeginS.o  .libs/input_lexer.o .libs/sym2poly.o .libs/gausspol.o .libs/threaded.o .libs/moyal.o .libs/maple.o .libs/ti89.o .libs/mathml.o .libs/misc.o .libs/permu.o .libs/quater.o .libs/desolve.o .libs/input_parser.o .libs/symbolic.o .libs/index.o .libs/modpoly.o .libs/modfactor.o .libs/ezgcd.o .libs/derive.o .libs/solve.o .libs/intg.o .libs/intgab.o .libs/risch.o .libs/lin.o .libs/series.o .libs/subst.o .libs/vecteur.o .libs/sparse.o .libs/csturm.o .libs/tex.o .libs/global.o .libs/ifactor.o .libs/alg_ext.o .libs/gauss.o .libs/isom.o .libs/plot.o .libs/plot3d.o .libs/rpn.o .libs/prog.o .libs/pari.o .libs/cocoa.o .libs/unary.o .libs/usual.o .libs/identificateur.o .libs/gen.o .libs/tinymt32.o .libs/first.o .libs/TmpLESystemSolver.o .libs/TmpFGLM.o .libs/help.o .libs/lpsolve.o .libs/optimization.o .libs/signalprocessing.o .libs/graphe.o .libs/graphtheory.o .libs/nautywrapper.o .libs/markup.o .libs/kdisplay.o .libs/kadd.o .libs/caseval.o .libs/cutils.o .libs/graphic.o .libs/libbf.o .libs/libregexp.o .libs/libunicode.o .libs/qjsgiac.o .libs/quickjs.o .libs/quickjs-libc.o .libs/js.o   -L/home/doctorant/sage/local/lib -lntl -lpari -L/usr/lib/x86_64-linux-gnu/openblas-pthread/ -lgsl -lopenblas -lrt -lpthread -lnauty -lcliquer -lusb-1.0 -lcurl -lglpk -ldl -lecm -lmpfi -lmpfr -lgmp -L/home/doctorant/sage/local/lib/../lib -L/usr/lib/gcc/x86_64-linux-gnu/11 -L/usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu -L/usr/lib/gcc/x86_64-linux-gnu/11/../../../../lib -L/lib/x86_64-linux-gnu -L/lib/../lib -L/usr/lib/x86_64-linux-gnu -L/usr/lib/../lib -L/usr/lib/gcc/x86_64-linux-gnu/11/../../.. -lstdc++ -lm -lc -lgcc_s /usr/lib/gcc/x86_64-linux-gnu/11/crtendS.o /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/crtn.o  -g -O2 -g -O2 -Wl,-rpath-link -Wl,/home/doctorant/sage/local/lib -Wl,-rpath -Wl,/home/doctorant/sage/local/lib -Wl,-rpath-link -Wl,/home/doctorant/sage/local/lib -Wl,-rpath -Wl,/home/doctorant/sage/local/lib   -Wl,-soname -Wl,libgiac.so.0 -o .libs/libgiac.so.0.0.0
/usr/bin/ld: _ZN3NTL8ZZ_pInfoE: TLS reference in .libs/modpoly.o mismatches non-TLS definition in /home/doctorant/sage/local/lib/libntl.so section .bss
/usr/bin/ld: /home/doctorant/sage/local/lib/libntl.so: error adding symbols: bad value

[dkwo]

@mkoeppe Thanks.
@dimpase Sure, the thing is that Void linux currently does not have native builders for aarch64.
For the record, currently the blockers to cross-compilation are ntl (hence flintlib, eclib); fflas-ffpack, givaro, linbox; sympow; and maxima.

[dimpase]

you can build a native aarch64 SoC builder for under $100 - won't be fast, but will do the job.

[dkwo]

@dimpase Once again, I can of course build natively on my aarch64 machine; the point is, their build infrastructure is cross-build for that arch, so it won't be in the repositories..

[mezzarobba]

On Tuesday, 21 November 2023 17:57:43 CET dkwo wrote: Could anyone clarify whether `ntl` is still required in sage after this patch, either for building or running flint, eclib or sage itself?

It is no longer required for building flint, but still required for building sage itself.

[dkwo]

@mezzarobba Thanks a lot, this is what I thought.
What about eclib? can it also be built without ntl, as far as sage is concerned?
I guess I was a bit puzzled by this comment from @JohnCremona

for one thing it will mean that building Sage will not also involve building eclib which has flint as a dependency.

[JohnCremona]

The functionality in eclib which Sage uses does need NTL. Flint is optional. I'll look more carefully to see how vital for Sage the NTL dependency is.

[dimpase]

@dimpase Once again, I can of course build natively on my aarch64 machine; the point is, their build infrastructure is cross-build for that arch, so it won't be in the repositories..

maxima/ecl are not going to get optional any time soon. So trying to remove NTL seems to be a bit in vain.

[dkwo]

@JohnCremona Got it, thanks.

So trying to remove NTL seems to be a bit in vain.

Well, it would shorten the time needed to build sage in the above mentioned case.
Of course, the best outcome would be to make it cross buildable.

[jhpalmieri]

Are there further work issues here?

[mezzarobba]

Are there further work issues here?

None that I am aware of (at least if we are okay with temporarily breaking the e-antic and symengine optional packages).

[tornaria]

I've been testing and using this for a while, and I will be shipping 10.2 in void linux with this patch, because I cannot hold back flintlib any longer.

I'm giving this positive review so it can be merged at the start of the 10.3 cycle.

[vbraun]

There is a buildbot that tests the e-antic optional package, do you have a plan for that? Or is it just going to be silently broken?

Note that normaliz depends on it

[mkoeppe]

I think the plan is to break it and fix it soon.