Upgraded to use Python3

[hawkeye-7]

No description provided.

[sashakames]

I just noticed this checking in:

(/home/jovyan/conda-envs/esgf-pub520) jovyan@jupyter-sashakames:~/work$ esgmapfile -i config/publisher-configs/ini --project cmip6 /p/css03/esgf_publish/CMIP6/ScenarioMIP/MRI/MRI-ESM2-0/ssp245/r5i1p1f1/Oday/tossq/gn
Traceback (most recent call last):
  File "/home/jovyan/conda-envs/esgf-pub520/bin/esgmapfile", line 33, in <module>
    sys.exit(load_entry_point('esgprep', 'console_scripts', 'esgmapfile')())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jovyan/work/esgf-prepare/esgprep/esgmapfile.py", line 244, in main
    run(args)
  File "/home/jovyan/work/esgf-prepare/esgprep/mapfile/main.py", line 249, in run
    with ProcessingContext(args) as ctx:
  File "/home/jovyan/work/esgf-prepare/esgprep/mapfile/context.py", line 80, in __enter__
    dir_format=self.cfg.translate('directory_format'))
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jovyan/conda-envs/esgf-pub520/lib/python3.11/site-packages/ESGConfigParser/__init__.py", line 214, in translate
    pattern = re.sub(re.compile(r'%\((digit)\)s'), r'[\d]+', pattern)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jovyan/conda-envs/esgf-pub520/lib/python3.11/re/__init__.py", line 185, in sub
    return _compile(pattern, flags).sub(repl, string, count)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jovyan/conda-envs/esgf-pub520/lib/python3.11/re/__init__.py", line 317, in _subx
    template = _compile_repl(template, pattern)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jovyan/conda-envs/esgf-pub520/lib/python3.11/re/__init__.py", line 308, in _compile_repl
    return _parser.parse_template(repl, pattern)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jovyan/conda-envs/esgf-pub520/lib/python3.11/re/_parser.py", line 1080, in parse_template
    raise s.error('bad escape %s' % this, len(this)) from None
re.error: bad escape \d at position 1```

[tkettenba]

This answer explains what happens here. Seems to be related to python3.6 and python3.7 transition.

It also links to a patch on an unrelated project, facing the same issue.

Looks like there are options:

• add double '\\' in front of non-escape codes in the 2nd arg string.
• use regex module instead of 're' (e.g. use 'import regex as re') (probably not recommended)

[sashakames]

IMHO: both Python3.6 and 3.7 are long deprecated. We should only target supported 3.x. Unless someone has some legacy software that cannot be migrated, there is no good reason to support those.

[tkettenba]

Agreed, the forementioned python versions are rather old.

Probably the esgfconfigparser can be updated, to work with the newer python versions. I see that there is already a branch: python3.8. But I don't know how to help here. Do you have suggestions on how to proceed?

BR,
Thomas

[sashakames]

wrt esgconfigparser We have published a pre-release of a py3-compatible version for use within esg-publisher, now this is just to support the migration of config files to the proper yaml config format. I would like to deprecate the migration procedure, and settle on an EOL for the use of the .ini file format in ESGF.