Skip to content

glevava/esg-publisher

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

#*****************************************************************
# Copyright (c) 2009  Larence Livermore National Security (LLNS)
# Produced at the Lawrence Livermore National Laboratory
# All rights reserved.
#*****************************************************************
#
#   Description: Basic description of the project(s) and layout of the
#   		project.
#
#*****************************************************************

README:

ESGF Publisher

https://esgf.github.io/esg-publisher 

26 June 2018 - version 3.5.2

* REST API is default for both publish and unpublish ; support to use the ESGF CA bundle
* Switching data specs versions during operations
* esgtest_publish supports download test and skip-unpublish modes of operation
* new processing modes:
   - --skip-aggregation  
   - --commit-every  allows for more frequent database commits to aid in reducing memory footprints

Note - logging controlled now by the [DEFAULT] section of esg.ini. If you have upgraded the publisher with a previous version of the file, you may notice that the old setting is WARNING, in which case, INFO level messages may not appear.  Thus, the publisher may run successfully without any output.  



16 August 2017 - version 3.2.6

* Support for CMIP6 
- integration of PrePARE
- PID creation via RabbitMQ
- cdf2cim included
- handler for non-netCDF and skip variable scan for multiple cases

29 August 2016 - version 3.1.0

* Changes to support CMIP6
  - Configuration option to extract global attributes
  - Check for CMOR version in datafiles
  - CF checker integration
  - CMIP6-CV check integration (requires CMOR install)
* Included documentation (requires sphinx, sphinx_rtd_theme)

03 August 2016 - version 3.0.1

* Setup.py inclusion of esgprep utility - see http://esgf-prepare.readthedocs.io/ for more information
* Use of project-specific esg.ini files read from the default location
* Support for multiple mapfiles within a single directory
* support for comma or space-delimited facets


Previous versions:

06 July 2015 - version 2.13.0

 * Support for multithreaded checksumming
 * Version pick up in esgscan and use in esgpublish
 * esgfind_excludes to find files in the thredds exclude property
 * Add optional facet values with esgadd_facetvalues for published datasets
 * ACME file handler to support publication of netCDF + other formats in single data set

14 January 2015 - version 2.12.2

 * Warning message for esgscan_directory failues
 * --nodbwrite flag to enable "dry run" for fileO validation without writing to postgreSQL database
 * Additional utility scripts: 
      meta_synchro.py - metadata syncronization checks for PGSQL, Thredds and SOLR
     gen_versions.py, add_checksums_to_map.sh  - used for replica publication (at LLNL/PCMDI)
    

Packages

No packages published

Languages

  • Python 88.6%
  • Java 9.8%
  • HTML 1.1%
  • Other 0.5%