Data handling

class pavics_datacatalog.wps_processes.PavicsSearch

pavicsearch PAVICS Catalogue Search (v0.1)

Search the PAVICS database and return a catalogue of matches.

Parameters:

• facets (string, optional) – Comma separated list of facets; facets are searchable indexing terms in the database.
• shards (string, optional) – Shards to be queried
• offset (integer, optional) – Where to start in the document count of the database search.
• limit (integer, optional) – Maximum number of documents to return.
• fields (string, optional) – Comme separated list of fields to return.
• format (string, optional) – Output format.
• query (string, optional) – Direct query to the database.
• distrib (boolean, optional) – Distributed query
• type (string, optional) – One of Dataset, File, Aggregate or FileAsAggregate.
• constraints (string, optional) – Format is facet1:value1,facet2:value2,…

Returns:

• search_result (application/json, application/gml+xml) – PAVICS Catalogue Search Result
• list_result (application/json) – List of OPEnDAP urls of the search result.

class flyingpigeon.processes.FetchProcess

fetch_resources Fetch Resources (v0.10)

Fetch data resources (limited to 50GB) to the local filesystem of the birdhouse compute provider.

Parameters:resource (application/x-netcdf, application/x-tar, application/zip) – NetCDF Files or archive (tar/zip) containing netCDF files. Returns:

• output (text/plain) – File containing the local pathes to downloades files.
• output_log (text/plain) – Collected logs during process run.

References

• Documentation

class malleefowl.processes.ESGSearchProcess

esgsearch ESGF Search (v0.6)

Search ESGF datasets, files and aggreations.

Parameters:

• url (string) – URL of ESGF Search Index. Example: http://esgf-data.dkrz.de/esg-search
• distrib (boolean, optional) – If flag is set then a distributed search will be run.
• replica (boolean, optional) – If flag is set then search will include replicated datasets.
• latest (boolean, optional) – If flag is set then search will include only latest datasets.
• temporal (boolean, optional) – If flag is set then search will use temporal filter.
• search_type ({'Dataset', 'File', 'Aggregation'}, optional) – Search on Datasets, Files or Aggregations.
• constraints (string) – Constraints as list of key/value pairs.Example: project:CORDEX, time_frequency:mon, variable:tas
• query (string, optional) – Freetext query. For Example: temperatue
• start (dateTime, optional) – Startime: 2000-01-11T12:00:00Z
• end (dateTime, optional) – Endtime: 2005-12-31T12:00:00Z
• limit ({'0', '1', '2', '5', '10', '20', '50', '100', '200'}, optional) – Maximum number of datasets in search result
• offset (integer, optional) – Start search of datasets at offset.

Returns:

• output (application/json) – JSON document with search result
• summary (application/json) – JSON document with search result summary
• facet_counts (application/json) – JSON document with facet counts for constraints.

References

• Birdhouse
• User Guide

class malleefowl.processes.Download

download Download files (v0.8)

Downloads files and provides file list as json document.

Parameters:resource (string) – URL of your resource. Returns:output – Json document with list of downloaded files with file url. Return type:application/json

References

• Birdhouse
• User Guide

class malleefowl.processes.ThreddsDownload

thredds_download Download files from Thredds Catalog (v0.5)

Downloads files from Thredds Catalog and provides file list as JSON Document.

Parameters:url (string) – URL of the catalog. Returns:output – JSON document with list of downloaded files with file url. Return type:application/json

References

• Birdhouse
• User Guide

class flyingpigeon.processes.NCMergeProcess

ncmerge NetCDF merge (v0.1)

Merge NetCDF files in the time dimension.

Parameters:resource (application/x-netcdf, application/x-tar, application/zip) – NetCDF files, can be OPEnDAP urls. Returns:output – Temporally merged NetCDF files. Return type:application/x-netcdf