12. Release Notes

12.1. rasdaman v10.1 (2022-10-21)

12.1.1. High-level summary

  • Rasdaman now provides OGC WMTS support out of the box, in addition to the long present implementations of OGC WCS, WCPS, and WMS.

  • On a lower level, WCPS and rasql support sorting array slices along a given axis, as well as reversing the order of slices to e.g. flip an image along an axis.

  • Query evaluation is continuously improved to be more efficient, e.g. WCS / WCPS scale operations will now automatically use a lower-resolution pyramid coverage if one exists.

  • Last but not least, this version brings improvements wrt. user-friendliness as well. Data import is simpler to configure as some complex settings are now automatically derived, more ergonomic shorthand functions have been added in WCPS, etc.

12.1.2. New features, enhancements, bug fixes

12.1.2.1. rasdaman core

  • Support for sorting array slices along an axis by an aggregation expression (docs)

  • Support for flipping the order of array slices along an axis (docs)

  • Initialize empty areas in subsetting operations with null values

  • Fix evaluation of overlay operation on arrays with null values

12.1.2.2. OGC frontend (petascope)

  • WMTS (OGC Web Map Tile Service) support added (docs)

  • WMTS passes OGC CITE tests

  • WMS passes OGC CITE WMS raster conformance tests

  • WMS fixes invalid XML element names in GetCapabilities

  • WMS supports a non-standard random parameter to bypass Web browser cache (docs)

  • WCPS supports sort operator (docs)

  • WCPS supports flip operator (docs)

  • WCPS supports imageCrsdomain as trimming subset in scale operator

  • WCPS fixed combination of different grid domains in coverage condense

  • WCPS LET clause supports WKT expressions

  • WMS query fragment styles support combining coverages with mismatching irregular axes

  • Optimize WCPS queries with scale and subset operators to use pyramids if applicable

  • WCPS supports crsTransform() shorthand operator (docs)

  • Support importing local files to external endpoint (docs)

  • Fix WMS wrong results with rasql / wcps fragments style on virtual coverages

  • Fix scaling on sliced virtual coverages by non-XY axes

  • Optimize processing WCPS requests with multiple binary expressions

  • Support WMS GetMap requests with non XY axis labels as request parameters

  • Fix null error when parsing compound CRS

  • Fix WMS style with WCPS query fragment with fixed subset domains

12.1.2.3. wcst_import

  • Option import_order supports value “none” to import files without sorting them beforehand

  • Add timeout for analyzing input files

  • If default_null_values is set to [], then a coverage without null values is created

  • Add setting filterMessagesMatching to filter unwanted GRIB messages

  • Automate setting gridOrder in ingredients (docs)

  • Support ${bbox} expression used in after_import hook (docs)

  • Support importing local files to external endpoint (docs)

12.1.3. Upgrading

The sections below contain instructions on how to upgrade the software installation from a previous version to stable rasdaman v10.1.

12.1.3.1. From v10.0

To upgrade from the previous stable version v10.0 to stable v10.1:

  1. Stop rasdaman with sudo service rasdaman stop; make sure that the processes are really stopped: if ps aux | grep -E 'ras(mgr|server)' returns any processes, stop them with pkill -9 -f 'ras(mgr|server)'

  2. On Ubuntu update the repository metadata with sudo apt-get update

  3. Install the new rasdaman version with sudo apt-get install rasdaman (Ubuntu) or sudo yum update rasdaman (CentOS 7)

12.1.3.2. From testing

To upgrade from the latest testing version to stable v10.1:

  1. Stop rasdaman with sudo service rasdaman stop; make sure that the processes are really stopped: if ps aux | grep -E 'ras(mgr|server)' returns any processes, stop them with pkill -9 -f 'ras(mgr|server)'

  2. Remove the current rasdaman package with sudo apt-get remove rasdaman (Ubuntu) or sudo yum erase rasdaman (CentOS 7); this is required because the stable package generally has the same version but a lower iteration number than the testing one

  3. Change the testing package repository to the stable one in /etc/apt/sources.list.d/rasdaman.list (Ubuntu) or /etc/yum.repos.d/rasdaman.repo (CentOS 7)

  4. On Ubuntu update the repository metadata with sudo apt-get update

  5. Install the new rasdaman version with sudo apt-get install rasdaman (Ubuntu) or sudo yum update rasdaman (CentOS 7)

12.1.4. Patch releases

Below fixes in rasdaman v10.1.X patch releases are listed.

12.1.4.1. v10.1.4 (2023-02-02)

rasdaman core

  • rasmgr queues client connections to increase the number of concurrent clients that rasdaman can handle

OGC frontend (petascope)

  • Improve error reporting when rasdaman crashes

  • Create proper temporary collection types and delete them after a WCPS query with decode() finished

  • Log warning message when petascope failed to close a connection to rasdaman

  • WCPS fixed handling of non-numeric bounds on irregular axis subsets

12.1.4.2. v10.1.3 (2023-01-05)

rasdaman core

  • Updating a UDF library in the rasdaman UDFs directory does not require rasdaman restart

  • Support specifying variables as an array when encoding to NetCDF (docs)

  • Improved error reporting with more details

  • Improved optimization of subseting operations

  • Fixed evaluation of join of empty collections

OGC frontend (petascope)

  • WCPS fixed handling of reduce expression on a virtual coverage without subsets

  • WMS fixed handling of WCPS query fragments on a layer with pyramids

  • WCPS / WMS improved performance for irregular axis with long list of BigDecimal values

  • WCPS fixed the grid origin of XY axes after crsTransform()

  • Support creating pyramid members from a virtual base coverage

  • WCPS fixed encoding to netcdf of dynamically created multiband coverage

  • Axis labels specified in an axis iterator are incorporated in the resulting coverage

  • Fixed importing with regular time axis

  • Improve performance on coverages with very large irregular axes

  • WCPS fixed parsing of unary minus expressions

  • Encoding to netCDF preserves variable order

wcst_import

  • virtual_coverage recipe supports scale_factors / scale_levels settings

12.1.4.3. v10.1.2 (2022-11-24)

rasdaman core

  • Fixed problem in UDF evaluation with non-char array results

  • Fixed problem with evaluating induced condensers with multidimensional iterators

OGC frontend (petascope)

  • WCPS supports cellCount() to get the total number of grid pixels of a coverage expression

  • WCPS supports sum() as an alias of add() operator

  • WCPS fixes evaluation of coverage constructor with coverage-resultuing values expression

  • WCS fixes authorization for WCS GetCoverage requests

  • WMS fixes evaluation of query fragment styles with predefined non-XY slices

wcst_import

  • Fix checking if an axis is UnixTime CRS properly

  • blocking: false should show the imported files count / total files properly

federation

  • Exit rasfed when password for rasdamanUser in rasfed.properties is invalid

12.1.4.4. v10.1.1 (2022-11-04)

rasdaman core

OGC frontend (petascope)

  • WMS fixes evaluation of query fragment styles with non-XY slices

  • WCPS fixes scaling with subsets involving LET clause variables

  • WCPS supports single and multiline comments (docs)

  • Supports shorthand interpolation method values in WCS GetCoverage requests (docs)

wcst_import

  • Enhance error handling to save error stacktrace to a log file (docs)

12.2. rasdaman v10.0 (2022-03-18)

12.2.1. High-level summary

  • Rasdaman v10.0 comes with a new query processing engine bringing higher performance on modern multi-core architectures, reduced memory usage, support for fine-grain access control through privileges and triggers, more correct nodata handling with null masks, and more.

  • Virtual coverages allow building a single coverage from multiple source coverages with potentially different CRS or resolution; for example, it is possible to combine Sentinel 2 coverages over varying UTM zones into a single coverage with global CRS like EPSG:4326, such that the necessary reprojections, scaling and overlaying are done on-the-fly in the background.

  • Access can be restricted on complete datacubes, but also on spatio-temporal regions within datacubes. Access control must allow flexible definition as well as efficient enforcement on individual or group of users. Further, resource usage often needs to be restricted on a per-user and per-query basis. Rasdaman now allows to do this through a convenient web interface.

  • Rasdaman federations support flexible access control configured by mapping users between federated nodes.

  • In-situ data registration has been enhanced to support spatially overlapping files with the same semantics as non-insitu updates, so that the last registered files overlays previously registered files, but not in the areas with null values.

  • Type coercion rules have been updated and standardized for less surprising behavior in query processing, to minimize the risk for overflow as well as type inflation. Casting from larger to smaller type has also been improved to “clamp” the values outside of the valid range of the new smaller type.

12.2.2. New features, enhancements, bug fixes

12.2.2.1. rasdaman core

  • Null values have a different and more accurate semantics based on null masks, additional null-specific operators, support for per-band null values, improved handling of null when encoding data, e.g. use of “null” keyword in JSON (docs)

  • Access control is now part of rasql and much more powerful and flexible in capabilities (docs); existing users are automatically migrated and the access control mechanism in rasmgr is not used anymore

  • Insitu registration of overlapping files now matches the semantics of regular, non-insitu updates (docs)

  • Encoding of data to JSON or CSV format is now more flexible with support for controlling various elements of the output format (docs)

  • Support for marray expression with array-valued cell expression, allowing to aggregate array expressions into higher-dimensional arrays (docs)

  • Updated and formally standardized type coercion rules for less surprising behavior in query processing; they have been designed to minimize the risk for overflow as well as type inflation (docs)

  • Type management is now fully possible with rasql queries (docs); the deprecated tool rasdl has been removed

  • Axis names specified during array type creation can be used for flexible positionally-independent subsetting (docs)

  • Types complex and complexd are deprecated, use cfloat32 and cfloat64 instead (docs)

  • Casting from larger to smaller type will lead to “clamping” of values outside of the valid range of the new smaller type (docs)

  • New query syntax to list the collections available in rasdaman; local and all remote collections are listed by default, and this can be controlled by specifying a particular host (docs)

  • The dbinfo function will print the set and mdd type names and supports SVG and JSON outputs for tiling (docs)

  • The encode function supports values, interval and ramp color tables (docs)

  • The project function allows a target resolution and interpolation method to be specified (docs)

  • New virtual collection RAS_TYPES allows to select all rasdaman types (docs)

  • New script prepare_issue_report.sh for automating issue reporting (docs)

  • Support hostname:collection syntax for disambiguating collections with the same name on different nodes in a federation (docs)

  • UDF code that accepts or returns arrays of composite base type must check if the array data is pixel- or channel-interleaved (docs)

  • Starting and stopping rasdaman is more configurable and flexible, recommended to review sections on Service Control, and the scripts used underneath, start_rasdaman.sh and stop_rasdaman.sh

  • Custom script for monitoring of rasdaman services watch_rasdaman.sh

  • Multiple marray expression optimizations for a much faster evaluation of queries that contain marray expressions

  • Report user, execution time and result size in the logs for each evaluated rasql query

  • Deprecated network protocols RNP and RPC have been removed, clients compiled against these old protocols will need to be recompiled with the latest client API libs in order to be able to communicate with v10.0 rasdaman servers

  • Subsetting has to be properly within the array spatial domain, otherwise an error will be thrown (docs)

  • Logs are only readable by the rasdaman user now

  • Improved rascontrol/rasmgr help sections and error handling

  • Improved performance of tile merging on client side

  • Errors from the server have been improved to return more detailed information

  • Fix decoding and encoding of octet data with GDAL

  • Fix issues when the rasdaman data directory is stored on NFS

  • Support passing openOptions to GDAL on decode, e.g. for selecting an overview level from the input data (docs)

  • Support GDAL virtual file system filepaths starting with /vsi, e.g. /vsis3, /vsizip, etc.

  • Fix memory leak when a java client (e.g. petascope) interrupts result transfer

  • Fix forwarding of -xp parameters in rasmgr.conf to spawned rasservers

12.2.2.2. OGC frontend (petascope)

  • Support for virtual coverages which can combine multiple coverages with different CRS (e.g. UTM zones) into a single coverage with a global CRS (docs)

  • WCPS supports LET clause for defining variables (docs)

  • WCPS requires only one spatial X or Y axis to be set when scaling (docs, and does not require specification of non-spatial axes if they are not to be scaled (docs <wcps-optional-non-scaled-axes>)

  • WCPS and WCS support scaling down on irregular axes

  • WCPS and WCS scaling will use pyramid members for efficiency if they exist

  • WCPS supports decode function (docs) and positional parameters (docs)

  • WCPS supports mod function and domain function on whole coverage (docs)

  • WCPS supports describe function which returns result like WCS DescribeCoverage (docs)

  • WCPS supports min/max functions (docs)

  • WCPS supports the crsSet function to return the coverage CRS as output (docs)

  • WCPS supports binary operations on mixed geo and non-geo coverages

  • WCPS supports rasql null operations (docs)

  • WCPS supports remote coverage IDs containing ‘-’ characters

  • WCS, WCPS and WMS support setting an interpolation method on data reprojection (docs)

  • WCS supports INSPIRE coverages (docs)

  • WCS GetCapabilities supports the AcceptLanguages parameter

  • WCS GetCapabilities response adds the list of axis names for each coverage in rasdaman:axisList elements and has more precise WGS84 bboxes (docs)

  • WCS GetCoverage and WCPS encode support CIS 1.1 JSON format (docs)

  • WCS ProcessCoverages supports WCPS abstract query specified in an XML-wrapper, and the q parameter in addition to query (docs)

  • WCS-T supports importing GDAL virtual file system filepaths starting with /vsi, e.g. /vsis3, /vsizip, etc.

  • WMS supports request for deleting a layer specifically (docs)

  • WMS styles support SLD color tables (docs)

  • WMS validates the XML format of user-specified SLD styles before persisting to database

  • WMS style query fragments support multiple layer iterators (docs)

  • rasql servlet supports rasdaman user credentials in basic authentication format in the HTTP Authorization header (docs)

  • rasql servlet will try to guess and set the correct mime-type of results

  • SECORE is embedded into petascope, so that it does not get installed as a separate service (def.war) anymore; it is now available at the /rasdaman/def endpoint instead of at /def (docs)

  • Unauthenticated users will be mapped to the rasdaman_user credentials if set in petascope.properties (docs)

  • Remove petascope_admin_user and petascope_admin_pass from petascope.properties and allow rasdaman admin users to perform write requests in petascope from external IPs (docs)

  • Filter out exception stacktraces to show only lines in rasdaman code; for more details, check the full_stacktraces option in the docs

  • Improve error reporting to return correct HTTP error codes 4xx and 5xx to clients

  • Split rasql error message and rasql query in different ExceptionText elements in the returned XML exception (docs)

  • Improve startup performance and logging of requests in petascope.log

  • Log rasdaman username for each request in petascope.log if basic authentication is enabled

  • Support qualified remote coverage ids or layer names (docs)

  • Support downloading results larger than 2GB

  • Allow non-ASCII characters in coverage metadata

12.2.2.3. wcst_import

  • Only Python 3 is supported

  • Support for servers secured with basic authentication; credentials should be specified with options -u or -i (check wcst_import.sh -h)

  • Support import of individual bands in GDAL general recipe (docs)

  • Support importing overviews from TIFF / JP2000 files (docs)

  • Support importing Sentinel 1 data with a custom recipe (docs)

  • Support creating a coverage pyramid for WMS with specific factors (docs)

  • Support “python_cmd” as an alternative to “cmd” in hooks (docs)

  • Support parameter to store the imported files in the ingredients file used by after_ingestion hooks (docs)

  • Support GDAL SIGNEDBYTE metadata for properly importing signed char (octet) data

  • Support referencing the index of the current processed GRIB message (docs)

  • Support attaching local metadata to a coverage from external text files (docs)

  • Support the metadata option in map_mosaic, time_series_regular and time_series_irregular recipes

  • Support automattically extracting global metadata and colorTable from TIFF files for GDAL general_coverage recipe

  • Significantly improved performance of netcdf data import or for a large number of files

  • Validate band names in general recipe

  • Do not send WCS DescribeCoverage request for getting axis labels from coverage

  • Use non-standard request to check if a coverage exists (docs)

  • Send requests to Petascope as POST containing the built GML of the importing coverage instead of GET with a local GML file

  • Show grid domains of imported files

  • Support for importing data from Amazon S3 object stores via GDAL /vsis3 driver

  • Fixed evaluation of skip setting in general_coverage recipe

  • Fix parsing of scientific notation in null values

12.2.2.4. federation

  • Support for flexible federated access control docs

  • Support for communication between federated nodes running different versions of rasdaman

  • Fix evaluation of marray with mixed local and remote array references

  • Fix buffer overflow when reading messages with lines longer than 512 characters

  • Petascope employs exponential back-off when it fails to contact a federation peer

12.2.2.5. clients

  • A new rasql-web-console client is now available, installed in /opt/rasdaman/share/rasdaman/www/

  • wsclient

  • Removed support for login with petascope_admin_user credentials (docs)

  • Support authentication

  • Once logged in, refreshing the Web browser will preserve the session

  • Estimate and show coverage sizes

  • Updating coverage metadata requires to login as admin

  • Display all footprints button only shows the available rows in WCS/WMS GetCapabilities tabs

  • DeleteCoverage and InsertCoverage tabs are shown only when logged in as admin

  • Downscaled pyramid members management in the WMS DescribeLayer tab

  • Add text>> widget to display texts

  • raswct has improved error handling

  • rasdapy has been updated to work on Python 3 (docs); rasdapy on Python 2 is not maintained anymore, migrating to rasdapy3 is recommended

12.2.2.6. documentation

  • Guidelines on how to report problems, with a script to help preparing issue reports (docs)

  • New section with cheatsheets for WCS / WCPS / WMS / clients (docs)

  • New section for petascope configuration (docs)

    • as a result, comments in the petascope.properties file have been mostly removed

  • New section on security and reseting default passwords (docs)

  • New section with guidelines for backup of rasdaman (docs)

12.2.2.7. installation

  • On new installation, petascope will be deployed in standalone embedded mode by default, so that it does not require a system Tomcat to run (docs); if it is preferred to deploy petascope in an external Tomcat, this can be done easily by updating the configuration, and copying rasdaman.war to the webapps Tomcat directory

  • On new installation, petascope requires basic authentication with HTTP Authorization header by default

  • petascope will use the locally installed gdal-java library instead of embedding it into rasdaman.war

  • Package installation can be customized with an installation profile (docs)

  • The rasdaman service script for starting/stopping rasdaman can be customized with configuration in /etc/default/rasdaman (docs)

12.2.3. Incompatible changes

  • Ubuntu 16.04 is not supported anymore, rasdaman v10.0 can be installed only on CentOS 7, Ubuntu 18.04 / 20.04 / 22.04

12.2.3.1. rasdaman core

  • Null values have a different and more accurate semantics based on null masks, additional null-specific operators, improved handling of null when encoding data, e.g. use of “null” keyword in JSON (docs)

  • Support for access control in rasmgr has been removed; use rasql for managing rasdaman users, assigning privileges to them, or creating triggers for access and resource usage control (docs)

  • Insitu registration of overlapping files now matches the semantics of regular non-insitu updates (docs):

  • In v9.8, registering a file A which overlaps an already registered file B would cause B to be de-registered from rasdaman

  • In v10.0, both will be registered but generally in the overlapping region the data from only the last registered file (A in this case) will be available; null values are handled as well by overlaying with the less recently registered files

  • Deprecated tool rasdl has been removed, use rasql queries for type management (docs)

  • Type coercion rules which govern the type of operation results based on the input data types have significantly changed, and may result in different behavior in existing queries; it is recommended to review the docs

  • Casting from larger to smaller type will lead to “clamping” of values outside of the valid range of the new smaller type, which may result in different behavior when existing queries evaluate with v10.0 (docs)

  • Subsetting has to be within the array spatial domain now, otherwise an error will be thrown (docs); when a subset needs to extend outside or intersect the array sdom, use the extend function.

  • Existing UDF code that accepts or returns arrays of composite base type should be adapted to check if the array data is pixel- or channel-interleaved (docs)

  • If federation is enabled, listing collections will show both local and remote collections; furthermore, only collections to which the rasdaman user has access will be listed; it is recommended to review the docs

  • Deprecated network protocols RNP and RPC have been removed; clients compiled against these old protocols will need to be recompiled with the latest client API libs in order to be able to communicate with v10.0 rasdaman servers (docs)

  • rasdapy on Python 2 is not maintained anymore, migrating to rasdapy3 is recommended (docs)

12.2.3.2. OGC frontend (petascope)

  • Changed behavior of rasdaman_user in petascope.properties (docs):

    • v9.8 rasdaman_user is used if authentication is disabled with authentication_type=

    • v10.0, the old use remains, but if authentication_type=basic_header and a request is missing Basic authentication credentials:

      • if rasdaman_user=a_valid_rasdaman_user, petascope will allow the request as if it had credentials from rasdaman_user and rasdaman_pass

      • if rasdaman_user=, then petascope will throw a missing credentials exception.

  • In the GetCapabilities response, the way rasdaman adds various metadata for each coverage (sizeInBytes, axisList, hostname, endpoint and blackListed) has changed:

  • v9.8 the metadata was put under the <ows:Metadata> element for each coverage, e.g.

    <ows:Metadata>
    <rasdaman:sizeInBytes>224775000</rasdaman:sizeInBytes>
    <rasdaman:axisList>Lat,Long</rasdaman:axisList>
    ...
</ows:Metadata>

  • v10.0 the metadata is now put under an <ows:AdditionalParameters> element instead, e.g.

    <ows:AdditionalParameters>
    <ows:AdditionalParameter>
        <ows:Name>sizeInBytes</ows:Name>
        <ows:Value>224775000</ows:Value>
    </ows:AdditionalParameter>
    <ows:AdditionalParameter>
        <ows:Name>axisList</ows:Name>
        <ows:Value>Lat,Long</ows:Value>
    </ows:AdditionalParameter>
    ...
</ows:AdditionalParameters>

  • When a client sends GetMap requests, the rules below define (in conformance with the WMS 1.3 standard) how a style is applied to the requested layers:

    • If no styles are defined then rasdaman returns the data as-is, encoded in the requested format.

    • If some styles are defined, e.g. X, Y, and Z, then:

      • If the client specifies a style Y, then Y is applied.

      • If the client does not specify a style, then:

        • If the admin has set a style as default, e.g. Z, then Z is applied.

        • Otherwise, if no style has been set as default, then the first style from the list of styles, e.g. X is applied.

  • Remove use of petascope_admin_user in petascope.properties (docs):

    Version

    Behavior

    v9.8

    allows credentials configured by petascope_admin_user and petascope_admin_pass to be used for login as admin user in WSClient or execute write requests equivalently as the admin user configured by rasdaman_admin_user.

    v10.0

    petascope_admin_user and petascope_admin_pass are removed. These settings (if existing in the local system) will be copied as deprecated settings in the updated petascope.properties. Any requests using these deprecated credentials will be invalid. One has to the credentials from a valid rasdaman user with RW rights in WSClient; direct requests must include these credentials in the HTTP Authorization header in Basic authentication format.

  • Many non-OGC API supported by petascope have been redefined and consolidated under the /rasdaman/admin endpoint. Below all changes are listed (note that the server endpoint, e.g. http://localhost:8080 has been omitted from the URLs). The v9.8 URLs will not work anymore, it is required to adapt to the new syntax.

    Description

    Version

    Request example

    Create a WMS layer (docs)

    v9.8

    /rasdaman/ows?service=WMS&version=1.3.0&request=InsertWCSLayer&wcsCoverageId=MyCoverage

    v10.0

    /rasdaman/admin/layer/activate?COVERAGEID=MyCoverage

    Remove a WMS layer (docs)

    v9.8

    /rasdaman/ows?SERVICE=WMS&VERSION=1.3.0&REQUEST=DeleteLayer&LAYER=MyLayer

    v10.0

    /rasdaman/admin/layer/deactivate&COVERAGEID=MyLayer

    Create a WMS style (docs)

    v9.8

    /rasdaman/ows?SERVICE=WMS&VERSION=1.3.0&REQUEST=InsertStyle&NAME=style1&LAYER=cov1

    v10.0

    /rasdaman/admin/layer/style/add?COVERAGEID=cov1&STYLEID=style1

    Update a WMS style (docs)

    v9.8

    /rasdaman/ows?SERVICE=WMS&VERSION=1.3.0&REQUEST=UpdateStyle&NAME=style1&LAYER=cov1

    v10.0

    /rasdaman/admin/layer/style/update?COVERAGEID=cov1&STYLEID=style1&NEWSTYLEID=style2

    Remove a WMS style (docs)

    v9.8

    /rasdaman/ows?SERVICE=WMS&VERSION=1.3.0&REQUEST=DeleteStyle&LAYER=cov1&NAME=style1

    v10.0

    /rasdaman/admin/layer/style/remove?COVERAGEID=cov1&STYLEID=style1

    Check if a coverage exists (docs)

    v9.8

    /rasdaman/ows/objectExists?coverageId=cov1

    v10.0

    /rasdaman/admin/coverage/exist?COVERAGEID=cov1

    Rename a coverage (docs)

    v9.8

    /rasdaman/admin/UpdateCoverageId?COVERAGEID=cov1&NEWID=cov2

    v10.0

    /rasdaman/admin/coverage/update?COVERAGEID=cov1&NEWCOVERAGEID=cov2

    Update coverage metadata (docs)

    v9.8

    /rasdaman/admin/UpdateCoverageMetadata

    v10.0

    /rasdaman/admin/coverage/update

    Create a pyramid level (docs)

    v9.8

    /rasdaman/ows?service=WCS&request=InsertScaleLevel&version=2.0.1&coverageId=cov_3D&level=4

    v10.0

    /rasdaman/admin/coverage/pyramid/create?COVERAGEID=cov_3D&MEMBER=cov_3D_4&SCALEVECTOR=1,4,4

    Whitelist all coverages (docs)

    v9.8

    /rasdaman/admin?SERVICE=WCS&REQUEST=WhiteListAll

    v10.0

    /rasdaman/admin/wcs/whitelistall

    Blacklist all coverages (docs)

    v9.8

    /rasdaman/admin?SERVICE=WCS&REQUEST=BlackListAll

    v10.0

    /rasdaman/admin/wcs/blacklistall

    Whitelist some coverages (docs)

    v9.8

    /rasdaman/admin?SERVICE=WCS&REQUEST=WhiteList&COVERAGEID=cov1,cov2

    v10.0

    /rasdaman/admin/wcs/whitelist?COVERAGELIST=cov1,cov2

    Blacklist some coverages (docs)

    v9.8

    /rasdaman/admin?SERVICE=WCS&REQUEST=BlackList&COVERAGEID=cov1,cov2

    v10.0

    /rasdaman/admin/wcs/blacklist?COVERAGELIST=cov1,cov2

    Whitelist all layers (docs)

    v9.8

    /rasdaman/admin?SERVICE=WMS&REQUEST=WhiteListAll

    v10.0

    /rasdaman/admin/wms/whitelistall

    Blacklist all layers (docs)

    v9.8

    /rasdaman/admin?SERVICE=WMS&REQUEST=BlackListAll

    v10.0

    /rasdaman/admin/wms/blacklistall

    Whitelist some layers (docs)

    v9.8

    /rasdaman/admin?SERVICE=WMS&REQUEST=WhiteList&LAYERS=cov1,cov2

    v10.0

    /rasdaman/admin/wms/whitelist?LAYERLIST=cov1,cov2

    Blacklist some layers (docs)

    v9.8

    /rasdaman/admin?SERVICE=WMS&REQUEST=BlackList&LAYERS=cov1,cov2

    v10.0

    /rasdaman/admin/wms/blacklist?LAYERLIST=cov1,cov2

12.2.3.3. federation

  • Federation between nodes running rasdaman v9.8 and rasdaman v10.0 is not possible, older versions must be upgraded.

  • Federated requests are authenticated and authorized based on tokens in the new version (docs). In order to migrate a system from a previous version, follow these steps:

    1. Update and restart rasdaman on each machine. When the new version starts, a pair of RSA keys associated with the machine will be automatically generated and stored in /opt/rasdaman/etc/keys/local/.

    2. Copy the public key of the local machine to the /opt/rasdaman/etc/keys/federation/ directory on all other nodes in the federation (docs).

    3. Migrate the list of inpeer and outpeer definitions from /opt/rasdaman/etc/rasmgr.conf to /opt/rasdaman/etc/rasfed.properties; each peer definition must be extended with a role map (otherwise all requests will result in Access denied exceptions) (docs).

    4. Restart rasdaman on each machine.

12.2.4. Upgrading

The sections below contain instructions on how to upgrade the software installation from a previous version to stable rasdaman v10.0.

In addition, it is highly recommended to review the list of incompatible changes.

12.2.4.1. From v9.8

To upgrade from the previous stable version v9.8 to stable v10.0:

  1. Stop rasdaman with sudo service rasdaman stop; make sure that the processes are really stopped: if ps aux | grep -E 'ras(mgr|server)' returns any processes, stop them with pkill -9 -f 'ras(mgr|server)'

  2. On Ubuntu update the repository metadata with sudo apt-get update

  3. Install the new rasdaman version with sudo apt-get install rasdaman (Ubuntu) or sudo yum update rasdaman (CentOS 7)

12.2.4.2. From testing

To upgrade from the latest testing version to stable v10.0:

  1. Stop rasdaman with sudo service rasdaman stop; make sure that the processes are really stopped: if ps aux | grep -E 'ras(mgr|server)' returns any processes, stop them with pkill -9 -f 'ras(mgr|server)'

  2. Remove the current rasdaman package with sudo apt-get remove rasdaman (Ubuntu) or sudo yum erase rasdaman (CentOS 7); this is required because the stable package has the same version but a lower iteration number than the testing one

  3. Change the testing package repository to the stable one in /etc/apt/sources.list.d/rasdaman.list (Ubuntu) or /etc/yum.repos.d/rasdaman.repo (CentOS 7)

  4. On Ubuntu update the repository metadata with sudo apt-get update

  5. Install the new rasdaman version with sudo apt-get install rasdaman (Ubuntu) or sudo yum update rasdaman (CentOS 7)

12.2.5. Patch releases

Below fixes in rasdaman v10.0.X patch releases are listed.

12.2.5.1. v10.0.5 (2022-07-25)

rasdaman core

  • Fix initialization of areas outside of clipped regions to null values

OGC frontend (petascope)

  • WMS GetMap - request’s bbox doesn’t touch layer’s bbox, then, the output must be transparent

  • WCPS fixed null values for coverages created by decode() operator

12.2.5.2. v10.0.4 (2022-06-29)

rasdaman core

  • Fix import of non-consecutive messages from grib files

  • Fix ‘list collections on host’ to return only the collections on the specified peer host, instead of collections on all federated hosts

  • Return an empty tile if an insitu registered file cannot be decoded because it has been removed, instead of throwing an error; this enables smooth handling of rolling archives where registered data may be removed depending on the data retention policy

  • Fix band subsetting on nD marray constructor expressions

  • Fix type checking when setting a null mask, to correctly validate that the null mask is a boolean array matching in domain with the array to which it is set

OGC frontend (petascope)

  • Fix missing wms namespace for some elements in WMS GetCapabilities

  • Fix problem with wrong grid domains in imageCrsdomain() for clip() operator

  • Fix problem with WMS GetMap requests when zooming in with wrong results from pyramid members

  • Fix problem with WMS GetMap requests for virtual coverages with non-virtual pyramid members

  • Fix problem with WCPS domain(…).lo/hi to return geo bound for non-XY axis

  • Add column requests/second in statistics admin management page

  • Fix WMS GetLegendGraphic request to use default style if style paramater is omitted

  • Fix filtering virtual coverages by requesting user’s permission when handling WCS/WMS GetCapabilities requests

  • Fix WM GetMap request with multiple layers

  • Support mismatching time axes between pyramid members in WMS GetMap requests

  • Fix WMS scattering data at layer’s corners

  • Improve performance for WCS DescribeCoverage / GetCoverage requests

  • Support full_stacktraces in petascope.properties (besides full_stacktrace) to filter unnecessary exception logs

  • Add settings in petascope.properties for configuring the Tomcat JDBC Connection Pool

wcst_import

  • Validate if all input files have the same CRSs

  • Improve ingredients file schema validation

  • Import only messages corresponding to the specified band identifiers

wcs_client

  • Fix coverage sizes calculation for virtual coverages

  • Support renaming coverage in WCS DescribeCoverage tab

Documentation

  • Improve doc for WCPS encode format and wcst_import slicer options

12.2.5.3. v10.0.3 (2022-04-25)

rasdaman core

  • Improve handling of concurrent clients in rasmgr

OGC frontend (petascope)

  • Improve performance of WMS GetCapabilities

  • Fixed virtual layers not updated when source coverages updated

  • Embedded SECORE supports JSP to manage CRSs

  • Allow case-insensitive axis labels

  • Make WCS scaleextent and SOAP requests fully standard-compliant

wcst_import

  • Fix bug when skip is true and no input files match with the defined pattern

  • Use user’s customized band names in coverage’s global metadata for netCDF recipe

  • When wcst_import is interrupted by Ctrl + C or kill <wcst_import_pid> (SIGINT and SIGTERM signals), the resume file will be properly updated with all imported files up to that point

  • Check if user can create resume file before analyzing input files

  • Allow case-insensitive axis labels

  • Improve parsing for axes definitions in CRS

federation

  • Improve performance of WCS/WMS GetCapabilities when there are virtual coverages

  • Make local metadata collection more robust in case of failures

12.2.5.4. v10.0.2 (2022-04-05)

OGC frontend (petascope)

  • Improve performance of queries in petascopedb

  • Enhance logging of requests in petascope.log to allow distinguishing which request exactly was evaluated

wcst_import

  • Improve axis type detection from CRS

  • Make sure each file to be imported is open only once with GDAL, to improve import performance

federation

  • Fix communication between rasfed instances on federated nodes

documentation

  • Added a guide for migrating an installation from one machine to another (docs)

12.2.5.5. v10.0.1 (2022-03-30)

rasdaman core

  • More robust handling of spurious disk errors when checking if a file to be registered is readable or not

OGC frontend (petascope)

  • More robust handling of read timeouts in communication with rasfed

  • Fixed leak of large binary objects in PostgreSQL which were left in petascopedb when removing data, e.g. dropping a coverage, pyramid levels, styles, etc; these objects will be properly and automatically removed via triggers from now on.

    However, the already leaked orphan objects cannot be automatically removed, and require the following steps to address in order to reduce the size of petascopedb:

    # Stop rasdaman and embedded petascope if exists
sudo service rasdaman stop
# If petascope runs on external tomcat, then stop it
sudo service tomcat9 stop
# Rename the existing petascopedb to petascopedb_backup
sudo -u postgres psql -c "ALTER DATABASE petascopedb RENAME TO petascopedb_backup"
# Create a new empty database
sudo -u postgres createdb petascopedb
# Change in petascope.properties
metadata_url=jdbc:postgresql://localhost:5432/petascopedb_backup
# Finally, run migration script to copy only the relevant data from
# petascopedb_backup to the new petascopedb
/opt/rasdaman/bin/migrate_petascopedb.sh
# Start rasdaman (and Tomcat if needed)
sudo service rasdaman start
sudo service tomcat9 start
# Verify that services are working as expected, if all is fine the
# petascopedb_backup can be dropped from PostgreSQL (but maybe consider
# exporting it to a backup first)
sudo -u postgres dropdb petascopedb_backup

  • Improve performance of processing GetCapabilities requests in federations

  • Fix corner case of incorrectly calculating coordinates of irregular time axis when encoding to NetCDF

wcst_import

  • Fix support for per-band nullsets to match the capabilities of rasql (docs)

  • Fix validation of axis labels to be case-insensitive

federation

  • Improve error handling and network communication