3. Contributing to Rasdaman¶

We’d like to encourage everyone to contribute to the project in whatever ways they can. This is a volunteer project; all help is greatly appreciated.

We don’t have a rigid styleguide or set of rules for contributing to rasdaman; however, with the goal of making things easier for all developers, here are a few suggestions:

Get feedback from other people

This is what the Mailing Lists and this Trac project management system are for. It is always a good idea to talk to other devs on the mailing lists before submitting changes.
Use Trac tickets

This is important to track progress and activity. If you start working on an issue, accept the ticket. After you have finished, close the ticket, and add information about the changeset in the comment field. Provide also progress information when you make relevant changes as described in the UseOfTickets page
Write tests

Please write tests for any new functionality. See RasdamanTestSuites for instructions. We ask you for your understanding that patches are likely to get rejected if they do not contain adequate additions to the systemtest.
Stick to the Coding Standards

The rasdaman code guide is mandatory for all code. We ask you for your understanding that patches are likely to get rejected if they do not adhere to this guide.
Use meaningful commit messages and reference tickets

such messages help developers to understand what your goal and intent is. Further, it eases writing of release notes for new versions. Note that patches not starting with “ticket:nnn” will be automatically rejected.

3.1. Development Contributions¶

You developed a fix or some new functionality? We are grateful for your contribution. Of course we like any useful information, but best (and fastest) for inclusion is to be in sync with our development tools and processes. The following details are provided to help in this respect.

All our development is in Linux. Please consider this for your code.

We use git as a version management tool, so you may want do do that too. Check out from the repository using:

$ git clone git://rasdaman.org/rasdaman.git
$ git config --global user.name "Name Surname"
$ git config --global user.email my_email@address.xyz

rasdaman should be configured and compiled with -DENABLE_STRICT=ON to make sure that your patch doesn’t introduce new warnings.
After ensuring the tests are successful (see TODO systemtest link), stage and commit your changes (whereby NNNN indicates the number of the ticket that is fixed with this patch):
```
$ git add <file1> <file2> <dir1/> <dir2>/*.java ...
$ git commit -m "ticket:NNNN - My brief explanation of the patch"
```
Prepare your patch package through:
```
$ git format-patch -n
```

where n is the number of last commits that you want to create patch files for.

Upload your patch file (or a .tar.gz archive in case of several files) using Patch Manager. You will have to accept the Contributor Agreement. Without your stated consent we unfortunately cannot accept it, due to legal reasons.

3.2. Documentation¶

Any changes to public interfaces likely require updating the rasdaman documentation. This is a short guide on how to do this.

3.2.1. Getting started¶

Install dependencies

$ sudo pip install -U sphinx sphinx_rtd_theme

Main documentation can be found in doc/main (*.rst files).

Build the docs specifically:

$ make doc       # generate all documentation
$ make doc-html  # generate HTML documentation (requires sphinx)
$ make doc-pdf   # generate PDF documentation (requires sphinx, latexmk, texlive)
$ make doc-cpp   # generate C++ API documentation (requires doxygen)

# alternatively in doc/main/
$ ./build.sh

Make changes¶

Check the short intro below for the reST syntax
- … but it should be fairly clear from looking at the docs sources
Create a review request with arc diff before pushing changes.

3.2.2. Quick intro to reStructuredText¶

Section headers¶

In each case the underline or overline marker should be as long as the section header (use monospace font to do this correctly). From highest level to most granular section level:

# - Parts (overline and underline)
* - Chapters (overline and underline)
= - Sections (underline)
- - Subsections (underline)
^ - Subsubsections (underline)

Example from the QL guide:

####################
Query Language Guide
####################

************
Introduction
************

Multidimensional Data
=====================

Subsection
----------

Subsubsection
^^^^^^^^^^^^^

Text formatting¶

*Italics*
**Bold**
``Code``

Cannot be nested, may not start/end with whitespace, and has to be separated from surrounding text with some non-word characters.

Lists¶

* Bulleted list
* Item two

  * Nested list (note it has to have blank line before and after!)

- Bulleted list continues; you can use - instead of *

1. Numbered list
2. Item two

#. Automatically numbered list
#. Item two


term (single line)
    Definition of the term (indented on the next line)

    Definition continues with another paragraph (maintain indentation)


| Line block
| line breaks are preserved
| and appear exactly like this (without the | characters)

Option lists (e.g. the output of rasql -h) can be simply copy pasted, you just need to make sure the options and their descriptions form two columns.

Source code¶

Any source code can go as an indented text after :: (plus blank line). In the QL guide :: automatically does rasql highlighting. For example:

::

    -- example query
    select avg_cells(c) from mr2 as c

renders as

-- example query
select avg_cells(c) from mr2 as c

For different highlighting you have to use the code-block directive indicating the language, e.g. java, cpp, xml, javascript, text, ini, etc. Example for java:

.. code-block:: java

    public static void main(...) {
        ...
    }

You can see all lexers with pygmentize -L lexers; see also http://pygments.org/languages/

Images¶

If an image has no caption then use the image directive, e.g:

.. image:: media/logo_full.png
    :align: center
    :scale: 50%

If it has a caption then use the figure directive; the caption is added as an indented paragraph after a blank line:

.. _my-label:

.. figure:: media/logo_full.png
    :align: center
    :scale: 50%

    Caption for the figure.

Hyperlinks¶

To just have a URL as is nothing special needs to be done, just put as is:

http://rasdaman.org

To render the URL with alternative text, then the following form should be used:

`Link text <http://rasdaman.org>`_

Internal cross-referencing can be done by first setting up a label before a section header or a figure (see above this section Hyperlinks) and then using it to generate a link anywhere with

:ref:`my-label`

Instead of :ref: you can use :numref: to get automatic Figure number added to the link, e.g.

:numref:`my-label` -> Sec. 2

You can change the default text that :ref: generates like this:

:ref:`Custom text <my-label>`

Further information¶

Specification: http://docutils.sourceforge.net/docs/ref/rst/restructuredtext.html
Sphinx guide: http://www.sphinx-doc.org/en/master/usage/restructuredtext/index.html

3.3. Git resources¶

For extensive help on git see the online Git book.
For info on git conflicts see Handling and Avoiding Conflicts in Git or, for a quick resolve conflict by discarding any local changes, this StackOverflow answer.

Further tips:

3.3.1. Basic git for working on tickets¶

It is suggested to create a branch in your local working copy of the rasdaman git repo for each ticket/fix, so you will not mix up patches. (e.g: ticket:1450 -> branch ticket_1450, ticket:1451 -> branch ticket_1451, …)

Prerequisites

Checkout the newest source code from repository; suppose you did this in /home/rasdaman/rasdaman and you are in this directory in the terminal:
```
$ pwd
/home/rasdaman/rasdaman
```
List the branches in your local repository
```
$ git branch
```

3. Switch to branch master - as this branch is the canonical branch for the rasdaman remote repository

$ git checkout master

Pull the newest patches if possible from remote repository (rasdaman.org) to your local repository
```
$ git pull
```

Create a new branch from master branch for a particular fix or feature work:

$ git checkout -b "branch_name" # e.g: git checkout -b "ticket_1451"

# check current branch, it should be ticket_1451
$ git branch

Work and commit changes

1. You changed some files in the source code directory (e.g: file1.java, file2.cc,…) and you want to create a commit; first stage the changed files:

$ git add file1.java file2.cc ..

Warning

Avoid doing git add ., i.e. adding all changed files automatically.

Now you are ready to commit the staged files:

$ git commit -m "ticket:1451 - fix some stuff"

# see details of your commit on top
$ git log

And create a patch from the commit, i.e. a file with extension .patch created from the last commit = -1, which contains all the changes you made:
```
$ git format-patch -1
# or for code review
$ arc diff
```

3. Finish with this branch by uploading the patch to the patchmanager and switching to another ticket in a new branch, starting from master again.

Switch between pending patches

E.g you finished one ticket on ticket_1450 and uploaded to the patchmanager but the patch is rejected and needs to be updated, while you moved on to working on ticket_1460.

First, stage everything you are doing on ticket_1460; if you don’t want to create a temporary commit, you can just stash everything in current branch.
```
$ git add <file1> <file2> ...

# or stash
$ git stash
# later can be retrieved with
$ git stash pop
```
Then commit it as your pending patch on this branch
```
$ git commit -m "ticket:1460 - fixed stuff"
```

Make sure your current branch is clear

# should report: "nothing to commit, working directory clean"
$ git status

Now switch to your failure patch (e.g: ticket_1450):
```
$ git checkout ticket_1450
```
Fix the issues here and stage the newly changed files:
```
$ git add <file 1> <file 2> ...
```
Commit it without changing the ticket’s subject:
```
$ git commit --amend --no-edit
```

Create a patch from the updated commit:

$ git format-patch -1

# or for code review
$ arc diff

And upload it again to the patchmanager
Finally, you can switch back to the previous branch:
```
$ git checkout ticket_1460
```

Apply patches between branches

E.g you have 1 commit in ticket_1450 and 1 commit in ticket_1460) then you want to add this patch to ticket_1460)

Check current branch (should be ticket_1450)
```
$ git branch
```
Create a patch file (like “0001-ticket-1450-fix-some-issues.patch”) from the last commit
```
$ git format-patch -1
```
Switch to other branch
```
$ git checkout ticket_1460
```

Apply your patch from ticket_1450

$ git am -3 0001-ticket-1451-fix-some-issues.patch

Check the newest commit (if the patch is applied successfully)
```
$ git log
```

If a patch cannot be applied

You made changes on files which the patch also changes, so you have to merge it manually:

$ git am -3 0001-ticket-1450-fix-some-issues.patch
# The patch is not applied, some conflict shows here

Please follow our git conflict resolution guide, or Steps 3 to 7 of this resolving merge conflicts guide.
Once resolved, mark as such:
```
$ git am --resolved
```
Check that your patch from ticket_1450 is now the last patch in ticket_1460 branch:
```
$ git log
```

3.4. C++ Guidelines¶

The rasdaman system is implemented in C++ 11; below are some guidelines.

3.4.1. Debugging¶

The rasdaman code has facilities built in which aid debugging and benchmarking. On this page information is collected on how to use it. Target audience are experienced C++ programmers.

Important

It is best to configure rasdaman with -DCMAKE_BUILD_TYPE=Debug for debugging, and -DCMAKE_BUILD_TYPE=Release for benchmarking (and production deployment).

Debuging rasserver¶

In rasnet (the default network protocol), in order to attach to the rasserver process (with e.g. gdb -p <pid>) it is necessary to increase the values of SERVER_MANAGER_CLEANUP_INTERVAL and CLIENT_MANAGER_CLEANUP_INTERVAL in rasmgr_x/src/constants.hh to some large values; needless to say this requires recompiling and restarting rasdaman.

Once that is done, you can attach to a running rasserver process. First find the process id, second column in the output of

$ ps aux | grep rasserver

It’s best to enable only one rasserver in rasmgr.conf or with rascontrol for this purpose. Then, attach to the pid:

$ gdb -p <pid>

Debugging directql¶

When not debugging the network protocol, it’s recommended to use directql. directql has the same interface as rasql, with an important behind the scenes difference: it is a fully fledged rasserver itself actually, so it doesn’t need to go through the client protocol. This makes it ideal for running tools like gdb, valgrind, etc.

When executing directql, use the same parameters as for rasql, but add -d /opt/rasdaman/data/RASBASE (or substitute that to whatever is the -connect value in rasmgr.conf).

Example with gdb:

$ gdb --args directql -q 'query that causes a segfault' \
                      --out file -d /opt/rasdaman/data/RASBASE
...
> run
...
# show a backtrace once the segfault has happened
> bt

Memory debugging with valgrind¶

Valgrind can be used to detect uninitialized values, memory errors, and memory leaks, e.g.

$ valgrind --leak-check=full --track-origins=yes \
           directql -q 'query that causes memory problems' \
                    --out file -d /opt/rasdaman/data/RASBASE

Memory debugging with AddressSanitizer¶

AddressSanitizer can be enabled during compilation with -DENABLE_ASAN=ON. This adds -fsanitize=address to the compiler flags. Please visit the ASAN page for more details.

Enabling extra output at compile time¶

In order to effect any extra output (besides standard logging) at all, the code must be compiled with the resp. option enabled. This is not default in production operation for at least two reasons: writing an abundance of lines into log files slows down performance somewhat, and, additionally, logging has a tendency to flood file systems; however, the option is available when needed.

If you are compiling with cmake, simply use -DENABLE_DEBUG=ON before doing make. Doing this includes the above cmake flags for debugging, and it also sets two other variables to enable more-verbose logging. E.g. in your build directory

$ cmake .. -DCMAKE_INSTALL_PREFIX=$RMANHOME -DCMAKE_BUILD_TYPE=Debug -DENABLE_DEBUG=ON ...
$ make
$ make install

You may, optionally, alter settings in $RMANHOME/etc/log-client.conf and $RMANHOME/etc/log-server.conf to enable various other logging parameters, e.g. DEBUG and TRACE for extra verbose output in the logs.

3.4.2. Internal array representation¶

Internally in rasdaman, multidimensional arrays are handled as a 1-D array, linearized in row-major format. Row-major refers to matrices with rows and columns, indicating that first all cells of the first row are listed in order, then all cells of the second row, etc. Given that we are working with multidimensional arrays here, this notion needs to be generalized: the inner-most (last) axis is contiguous, and varies fastest, followed by the second last axis and so on.

For example, let’s say we have an array with sdom [5:10, -2:2, 0:5]. The 1-D internal_array (in code) corresponds to external_array (in rasql):

linear_index := 0
for i := 5..10
  for j := -2..2
    for k := 0..5
      internal_array[linear_index] == external_array[i, j, k]
      linear_index += 1

3.5. Adding Tests¶

TODO: this is somewhat outdated and incomplete.

The rasdaman source tree comes with integration tests (“systemtest” for historical reasons) and unit tests (in each component directory X there is a subdirectory X/test/). To run the integration test:

$ cd systemtest
$ make check

After your patch submission, the patchmanager will automatically run the systemtest in a sandbox; the result will be flagged in the patchmanager table for each patch submitted. Allow some time (usually 1.5 hours) until the result gets visible. Patches which do not pass systemtest will be rejected without further inspection.

make check will automatically find all tests in the four test case directories, specifically, testcases_mandatory, testcases_petascope, testcases_fixed and testcases_open.

whenever a bug is found, a corresponding test should be created in the testcases_open directory;
when the bug is fixed, the test should be moved to the testcases_fixed directory;
testcases_services holds the test cases for petascope and secore;
testcases_mandatory holds the test cases for rasql typically.

Each test should have a folder which is inside one of the above mentioned directories, by convention named test_X, e.g. test_select. The test should be executed by a shell script inside the folder; its exit code indicates whether the test passed (0) or failed (non-0). Details of the test execution should be logged in the same folder. In systemtest/util there are various bash utility functions that can be used in the test scripts, e.g. for logging, checking result, etc.

3.5.1. Add a rasql test query¶

save the test query as systemtest/test_mandatory/test_select/queries/<queryName>.rasql
save the expected query result file in systemtest/test_mandatory/test_select/oracle/<queryName>.oracle

To generate a test oracle:

if the result is a scalar, run

rasql -q  "<query>" --out string | grep Result > <queryName>.oracle

if the result is an array, run

rasql -q  "<query>" --out file --outfile <queryName>.oracle

Make sure to validate the correctness of the oracle before adding to the systemtest.

If a query is known to fail and documented by a ticket, it can be marked in the systemtest, so that the result of that query is SKIPPED, rather than FAILED. To do this create a file known_fails (if not yet existing) in the corresponding test dir (next to the test.sh) and put each query file name in a single line in this file.

3.5.2. Add a petascope test¶

The scripts for WMS, WCS and WCPS testing can be found respectively in:

rasdaman/systemtest/testcases_services/test_wcps
rasdaman/systemtest/testcases_services/test_wcs
rasdaman/systemtest/testcases_services/test_wms

To run a specific test (besides make check that runs the whole systemtest), go to the directory and execute

$ ./test.sh

Do not execute sh test.sh as the script is written for bash, and sh is often linked to a restricted version of bash like dash, or similar. Variables like Tomcat port, host, rasdaman connection details, etc. may need to be adapted before running the tests by editing rasdaman/systemtest/conf/test.cfg.

Testdata¶

The following coverages are available for the tests (see rasdaman/systemtest/testcases_services/test_wcps/README):

coverage	dim	type	pixel extent	axes	geo-boundingbox	time extension	CRS
`rgb`	2D	rgb	0:399,0:343	i/j			Index2D
`mr`	2D	char	0:255,0:210	i/j			Index2D
`eobstest`	3D	short	0:5,0:100,0:231	t/Long/Lat	25,-40 - 75,75	1950-01-01 -> 1950-01-06	Temporal + EPSG:4326
`mean_summer_ airtemp`	2D	char	0:885,0:710	Long/Lat	111.975,-44.525 156.275,-8.975		EPSG:4326
`irr_cube_1`	3D	short	0:99,0:99,0:5	i/j/k			Index3D
`irr_cube_2`	3D	float	0:62,0:35,0:3	E/N/ansi	75042.72735943, 5094865.557938- 705042.72735943, 5454865.5579385	2008-01-01 -> 2008-01-08	EPSG:32633 + ANSI

These coverages are automatically inserted if missing.

Adding tests¶

To add new tests to the test suite, simply add new WCS or WCPS queries to the queries directory. Please adhere to the naming convention, continuing from the last number:

Type	File name format
WCS KVP	`number-meaningful_name.[error.]kvp`
WCS XML	`number-meaningful_name.[error.]xml`
WCS SOAP	`number-meaningful_name.[error.]soap`
WCS REST	`number-meaningful_name.[error.]rest`
WCPS	`number-meaningful_name.[error.]test`
WCPS XML	`number-meaningful_name.[error.]xml`
rasql	`number-meaningful_name.[error.]rasql`

Note

If the test is meant to raise an exception, add a further .error suffix to the file name before its extension, for both query and oracle.

The associated oracle (.oracle) files must also be added to the oracle/ directory. The oracle can be automatically added by running the tests. In this case it can be more convenient to run the tests on the single new query by uncommenting this line in test.sh:

# uncomment for single test run
[[ "$f" == 62-* ]] || continue

and choose the proper pattern to select one or more tests.

3.5.3. Templated System Test¶

The rasdaman query templating engine rasqte (currently found in systemtest/testcases_manual/test_rasql) allows to write template queries (in Jinja2 format) that focus on the operation that should be tested; a preprocessing step expands these templates into concrete valid queries targeting various data configurations that can be evaluated in rasdaman.

In Jinja2 templates we have:

Output markup (surrounded in {{ and }} which resolves to text; this supports some basic arithmetic, string functions, etc.
Tag markup (surrounded in {% and %} which doesn’t resolve to text, and can be used for loops, conditionals, etc.
Comments - any text surrounded in {# and #}

Comprehensive documentation on Jinja2 templates can be found in the official template designer documentation.

The templating engine defines several global objects/variables that can be used in the query templates. The table below documents these objects; the Example column shows an example for 2-dimensional char data.

Important

the templating engine iterates over all dimensions in dimension_list and cell types in cell_type_name_list, and renders the template for each pair.

Variable	Description	Example	Default
Dimensionality
dimension_max	Max dimension tested	4	4
dimension_list	All tested dimensions		[1, 2, 3, 4]
dimension	Curr. dimension	2	one of dimension_list
Cell type
cell_type_name_list	All tested cell types		[boolean, octet, char, ushort, short, ulong, long, float, double, complex, complexd, char_char_char short_float]
cell_type_name	Curr. cell type	char	one of cell_type_name_list
cell_type_suffix_dic	Cell type -> constant suffix		{‘octet’: ‘o’, ‘char’: ‘c’, …}
cell_type_suffix	Curr. cell type suffix	c	cell_type_suffix_dic[cell_type_name]
cell_type_min_dic	Cell type -> min value		{‘octet’: ‘-128’, ‘char’: ‘0’, …}
cell_type_min	Min for curr. cell type	0	cell_type_min_dic[cell_type_name]
cell_type_max_dic	Cell type -> max value		{‘octet’: ‘127’, ‘char’: ‘255’, …}
cell_type_max	Max for curr. cell type	255	cell_type_max_dic[cell_type_name]
cell_type_val_dic	Cell type -> non-edge value		{‘octet’: ‘-13’, ‘char’: ‘33’, …}
cell_type_val	Value for curr. cell type	33	cell_type_val_dic[cell_type_name]
cell_type_size_dic	Cell type -> cell size (B)		{‘octet’: 1, ‘char’: 1, …}
cell_type_size	Size for curr. cell type	1	cell_type_size_dic[cell_type_name]
cell_type_signed_dic	Cell type -> is signed		{‘octet’: True, ‘char’: False, …}
cell_type_signed	Is curr. cell type signed	False	cell_type_signed_dic[cell_type_name]
cell_type_components _dic	Cell type -> comp. name/type pairs		{‘char_char_char’: [(‘b0’,’char’) (‘b1’,’char’),(‘b2’,’char’)], …}
cell_type_components	Components for curr. cell type	[]	cell_type_components_dic[cell_type_name]
Coll/Mdd type
coll_name_dic	(dim,cell type) -> coll name		{(1,’octet’): ‘test_1d_octet’, (2,’char’): ‘test_2d_char_set’, …}
coll_name	Curr. coll name	test_2d_char	‘test_’ + dimension + ‘d_’ + cell_type_name
coll_type_name_dic	(dim,cell type) -> coll type		{(1,’octet’): ‘test_1d_octet_set’, (2,’char’): ‘test_2d_char_set’, …}
coll_type_name	Curr. coll type name	test_2d_char_set	coll_name + ‘_set’
mdd_type_name_dic	(dim,cell type) -> mdd type		{(1,’octet’): ‘test_1d_octet_mdd’, (2,’char’): ‘test_2d_char_mdd’, …}
mdd_type_name	Curr. mdd type name	test_2d_char_mdd	coll_name + ‘_mdd’
MDD constants
mdd_constant_cell_count	Number of cells in mdd constants	16	16
mdd_constant_extents _list	dimension -> sdom extents		[[], [16], [4,4], [2,2,4], [2,2,2,2]]
mdd_constant_extents	dim extents for curr. dim	[4,4]	mdd_constant_extents_list[dimension]
mdd_constant_sdom_list	dimension -> sdom		[‘’, ‘[0:15]’, ‘[0:3,0:3]’, ‘[0:1,0:1,0:3]’, ‘[0:1,0:1,0:1,0:1]’]
mdd_constant_sdom	sdom for curr. dim	‘[0:3,0:3]’	mdd_constant_sdom_list[dimension]
mdd_constant_cell _values_dic	cell type -> cell values		{‘char’: [‘0c’,‘0c’,‘255c’,‘1c’, ‘99c’,‘9c’,‘109c’,‘2c’,‘5c’,‘12c’, ‘23c’,‘45c’,‘123c’,‘123c’, ‘234c’,‘250c’], …}
mdd_constant_cell_values	cell values for curr. cell type	[‘0c’,‘0c’, ‘255c’,‘1c’, ‘99c’,‘9c’, ‘109c’,‘2c’, ‘5c’,‘12c’, ‘23c’,‘45c’, ‘123c’,‘123c’, ‘234c’,‘250c’]	mdd_constant_cell_values_dic[cell_type_name]
mdd_constant_dic	(dim,cell type) -> mdd		{(2,’char’): ‘<[0:3,0:3] 0c,0c,255c,1c; 99c,9c,109c,2c;5c,12c,23c,45c; 123c,123c,234c,250c>’, …}
mdd_constant	mdd constant for curr. dim/cell type	‘<[0:3,0:3] 0c,0c,255c,1c; 99c,9c,10c,2c; 5c,12c,23c, 45c;123c,123c, 234c,250c>’	mdd_constant_dic[(dimension,cell_type_name)]
Operations
oper_induced_unary	Unary induced ops		[+,-,not]
oper_induced_unary_name	Unary induced op names		[plus,minus,not]
oper_induced_binary	Binary induced ops		[+,-,*,/,overlay,is,and,or,xor, =,<,>,<=,>=,!=,]
oper_induced_binary_name	Binary induced op names		[plus,minus,multiplication,division, overlay,is,and,or,xor, equals,less,greater,lessorequal, greaterorequal,notequal]
oper_condense_op	Condense operators		[+,*,and,or,max,min]
oper_condense_name	Condense op names		[plus,multiplication,and,or,max,min]
Functions
func_induced_unary	Unary induced functions		[sqrt,abs,exp,log,ln,sin,cos,tan, sinh,cosh,tanh,arcsin,asin, arccos,acos,arctan,atan]
func_induced_binary	Binary induced functions		[pow,power,mod,div,bit, max,min,complex]
func_condense	Condensers		[max_cells,min_cells,all_cells, some_cells,count_cells,add_cells, avg_cells,var_pop,var_samp, stddev_pop,stddev_samp]
Other
separator	Instantiation sep.		‘===’
template_name	Template file name		e.g. ‘setup’ (extension is removed)
test_id	Unique test id		template_name + “_” + dimension + “d_” + cell_type_name

In addition, the following functions can be used in the templates:

Function	Description	Example
is_atomic_cell_type(type)	return true if type is an atomic cell type	is_atomic_cell_type(‘char’) -> True
is_complex_cell_type(type)	return true if type is a complex cell type	is_complex_cell_type(‘complexd’) -> True
is_composite_cell_type(type)	return true if type is a composite cell type	is_composite_cell_type(‘short_float’) -> True

Template instantiation¶

The template instantiation engine is a script rasqte.py that takes a template file as an input and produces a concrete output file.

usage: rasqte.py [-h] [-t TEMPLATE] [-d OUTDIR] [-s SEPARATOR] [-g]

rasql query template engine takes a Jinja2 template file as an input and
renders it into a concrete output; various global variables and functions are
available in the template (see option -g and the documentation). The template
is rendered multiple times for different variable configurations; each output
is appended to the same output file in the directory specified with -d,
separated by a line with a unique separator string (=== by default).

optional arguments:
  -h, --help            show this help message and exit
  -t TEMPLATE, --template TEMPLATE
                        Template file to be rendered; the output should be
                        multiple lines of the form key:value, e.g.
                        query:SELECT version(); Consult the documentation for
                        more details.
  -d OUTDIR, --outdir OUTDIR
                        Directory for output files ('.' by default).
  -s SEPARATOR, --separator SEPARATOR
                        Separator for different renderings of the same
                        template ('===' by default).
  -g, --globals         Print all global variables/functions.

Rendered templates¶

The rendered concrete file will have many instantiations of one template. Each instantiation ends with a separator line (=== by default):

instantiated_query
===
instantiated_query
===
...
===

Each instantiated_query has this format: The id is used to compare the result of evaluating the query to an oracle file named id. id and query are mandatory, any other parameters are optional.

query: concrete rasql query (mandatory)
id: unique id (mandatory)
filter: python string or list of strings that remove matching
        lines from the output (usually output that contains random
        bits which cannot be compared to a fixed expected oracle)
disableoutfile: comment on why --out file should be removed from
                the generated rasql command (e.g. to skip comparing
                random output, or output that is really large)
timeout: set how many seconds to wait for the test to finish,
         before killing the process; 60 seconds by default
knownfail: comment on why this query currently fails; once fixed,
           a line like this should be removed
skip: comment on why this query should be completely skipped during
      the test (i.e. not evaluated at all); should be removed once fixed
...
key: value
# comment lines start with '#'

# empty lines (as above) are ignored as well
===

Concrete test evaluation¶

A systemtest script test.py then reads a concrete file and evaluates the tests, comparing to the expected oracle values.

usage: test.py [-h] [-d] [-t TESTSFILE]

rasql query systemtest evaluator; without arguments it evaluates all tests in
the queries/ directory, starting with any setup tests and ending with the
teardown tests.

optional arguments:
  -h, --help            show this help message and exit
  -d, --drop            Drop data (execute teardown queries) only and exit.
  -t TESTSFILE, --testsfile TESTSFILE
                        Execute a specific tests file (with setup before and
                        teardown after).

The following directories are used by the script:

queries - contains the rendered templates (outputs of rasqte.py)
outputs - results of evaluating tests in queries are saved in this directory. For rasql queries two file types are saved:

1. any file outputs (produced by rasql as --out file is specified) in

template_name.file* files

2. stdout, stderr, and exit code from running the program in a

template_name file
oracles - similar structure as outputs directory, it contains the expected files against which the outputs are compared.

setup and teardown tests files in queries dir are treated specially: setup is evaluated first, before any others, and teardown is evaluated at the end. This allows to ingest data for the test, and drop it at the end, for example.

Examples¶

Create collection¶

Template:

CREATE COLLECTION {{ coll_name }} {{ coll_type_name }}

Engine instantiates 4 x 13 = 52 queries (4 for dimensions [1, 2, 3, 4] and 13 for cell types [boolean, char, octet, short, unsigned short, long, unsigned long, float, double, complex, complexd, char_char_char, short_float]:

rasql -q 'CREATE COLLECTION test_1d_boolean test_1d_boolean_set' --out file
rasql -q 'CREATE COLLECTION test_1d_char test_1d_char_set' --out file
rasql -q 'CREATE COLLECTION test_1d_octet test_1d_octet_set' --out file
...
rasql -q 'CREATE COLLECTION test_4d_short_float test_4d_short_float_set' --out file

Insert literal array¶

Template:

INSERT INTO {{ coll_name }} VALUES {{ mdd_constant }}

Instantiation:

rasql -q 'INSERT INTO test_1d_boolean VALUES ...' --out file
rasql -q 'INSERT INTO test_1d_char VALUES ...' --out file
...
rasql -q 'INSERT INTO test_3d_RGBPixel VALUES ...' --out file

The mdd_constant is constructed of 16 values that contain edge values (min/max) and other “interesting” values, like 0, nan, inf, etc.

Select: sin(array)¶

Template:

select sin(c) from {{coll_name}} as c

Instantiation:

rasql -q 'select sin(c) from test_2d_char as c' --out file
...

Select: sin(scalar)¶

Template:

select sin( {{cell_max}}{{cell_type_suffix}} )

Instantiation:

rasql -q 'select sin( 255c )' --out file
...

Select: all binary induced ops¶

Template:

{%- set dimension_other = [dimension + 1] if dimension < dimension_max else [] -%}
{%- for dimension_right in [dimension] + dimension_other -%}
{%-   for cell_type_right in cell_type_name_list -%}
{%-     for op in oper_induced_binary -%}
{%-       set coll_name_right = coll_name_dic[(dimension_right, cell_type_right)] -%}
SELECT a {{ op }} b FROM {{ coll_name }} AS a, {{ coll_name_right }} AS b
{%      endfor -%}
{%-   endfor -%}
{%- endfor -%}

Instantiation:

rasql -q 'select a + b from test_2d_char as a, test_2d_char as b' --out file
rasql -q 'select a + b from test_2d_char as a, test_2d_octet as b' --out file
rasql -q 'select a + b from test_2d_char as a, test_2d_ushort as b' --out file
...

Drop data¶

Template:

drop collection {{coll_name}}

Instantiation:

rasql -q 'drop collection test_2d_char as a' --out file
...

Insert encoded data¶

TODO

Template:

---
data.file: ../testdata/data_{{dimension}}d_{{cell_type_name}}.tif
data.dimension: [2]
---
insert into {{coll_name}} values decode($1)

Instantiation:

rasql -q 'insert into test_2d_char values decode($1)' -f ../testdata/data_2d_char.tif --out file
...

3.6. rasdaman Code Guide¶

Don’t expect others to clean up your code

An open-source project is fun, but it requires a great deal of discipline to make all the code seamless that is coming from the developers worldwide. If everybody just follow their individual coding style - no matter how ingenious the code is - then the whole project will soon become unmaintainable.

To avoid this, rasdaman provides this code guide - don’t worry, it contains as few rules as possible, just enough to achieve overall coherence. Although written for C++, mutatis mutandis it applies to Java, Javascript, and even scripts.

Rules that have to be fulfilled strictly.
Recommendations which serve as suggestions for a ‘better’ coding style.
Examples to show how code should be written according to the guidelines.

Please understand that, while we always highly appreciate your contributions, we may have to reject your patch if it breaks this code guide. Your successors looking at the code will be most grateful for your efforts.

Credits: This code guide has been established by the rasdaman team based on the codeguide originally developed by Roland Ritsch who in turn has crafted it along the style guide of ELLEMTEL/Norway. Any eventual error is ours, of course.

3.6.1. Rules¶

Rule 0: Every time a rule is broken, this must be clearly documented.

—

Rule 1: Include files in C++ must have a file name extension .hh.

Rule 2: Implementation files in C++ must have a file name extension .cc.

Rule 3: Inline definition files must have a file name extension .icc.

Rule 4: Every file must include information about its purpose, contents, and copyright. For this purpose, the several standard headers are provided here. Adjust the copyright to your name / instituion as deemed adequate. All code must use a GPL header, except for files in the raslib/, rasodmg/, and rasj/ directories, which must use an LGPL header.

Rule 5: All method definitions must start with a description of their functionality using the standard method header.

Rule 6: All comments must be written in English.

—

Rule 7: Every include file must contain a mechanism that prevents multiple inclusions of the file.

Rule 8: Never use path name in #include directives. Only use relative paths and the parent path (..) is not allowed.

Rule 9: Never have indirect inclusion of a function. Collective include files are allowed.

—

Rule 10: The names of variables and functions must begin with a lowercase letter. Multiple words must be written together, and each word that follows the first starts with an uppercase letter (Camel Casing).

Rule 11: The names of constants must be all uppercase letters, words must be separated by underscores (“_”).

Rule 12: The names of abstract data types, structures, typedefs, and enumerated types must begin with an uppercase letter. Multiple words are written together and each word that follows the first is begun with an uppercase letter (Camel Casing).

—

Rule 13: The public, protected, and private sections of a class must be declared in that order (the public section is declared before the protected section which is declared before the private section). See the standard class definition for details.

Rule 14: No member functions within the class definition include file. The only exception are inline functions.

Rule 15: No public or protected member data in a class. Use public inline methods (setVariable() and getVariable()) to access private member data.

Rule 16: A member function that does not affect the state of an object (its instance variables) must be declared const.

Rule 17: If the behavior of an object is dependent on data outside the object, this data must not be modified by const member functions.

—

Rule 18: A class which uses new to allocate instances managed by the class must define a copy constructor.

Rule 19: All classes which are used as base classes and which have virtual function, must define a virtual destructor.

Rule 20: A class which uses new to allocate instances managed by the class must define an assignment operator.

Rule 21: An assignment operator which performs a destructive action must be protected from performing this action on the object upon which it is operating.

—

Rule 22: A public member function must never return a non-`const` reference or pointer to member data.

Rule 23: A public member function must never return a non-const reference or pointer to data outside an object, unless the object shares the data with other objects.

—

Rule 24: Do not use unspecified function arguments (ellipsis notation).

Rule 25: The names of formal arguments to functions must be specified and are to be the same both in the function declaration and in the function definition.

—

Rule 26: Always specify the return type of a function explicitly. If no value is returned then the return type is void.

—

Rule 27: A function must never return a reference or a pointer to a local variable.

Rule 28: Do not use the preprocessor directive #define to obtain more efficient code; instead, use inline functions.

—

Rule 29: Constants must be defined using const or enum; never use #define.

Rule 30: Do not use numeric values directly in the code; use symbolic values instead (Use constants for default values). Always document the meaning of the value.

—

Rule 31: Variables must be declared with the smallest possible scope. Do not use global variables.

Rule 32: Never declare multiple variables in the same line.

Rule 33: Every variable that is declared must be given a value before it is used.

Rule 34: Don’t use implicit type conversions.

Rule 35: Never cast an object to a virtual class.

Rule 36: Never convert a const to a non-const.

—

Rule 37: The code following a case label must always be terminated by a break statement.

Rule 38: A switch statement must always contain a default branch which handles unexpected cases.

Rule 44: Never use goto.

—

Rule 45: Do not use malloc, realloc or free, but use new and delete. In general, use C++, not C code.

Rule 47: Always provide empty brackets ([]) for delete when deallocating arrays.

Rule 48: Use C++ exception handling (try/catch) for every possible failure situation.

—

Rule 49: When submitting a patch, describe concisely in the commit message what has been accomplished in the patch. In case of a fix, include in the message the ticket# fixed and place a comment in the source file at the location the fix was done mentioning the ticket (best by its URL).

3.6.2. Recommendations¶

Recommendation 1: Optimize code only if you know that you have a performance problem. Think twice before you begin.

Recommendation 2: Eliminate all warnings generated by the compiler.

Recommendation 3: An include file should not contain more than one class declaration.

Recommendation 4: Place machine-dependent code in a special file so that it may be easily located when porting code from one machine to another.

Recommendation 5: Always give a file a name that is unique in as large a context as possible.

Recommendation 6: An include file for a class should have a file name of the form + .hh. Use all lowercase letters.

Recommendation 7: Use the directive #include “filename.hh” for user-prepared include files.

Recommendation 8: Use the directive #include for include files from system libraries.

Recommendation 9: Choose names that suggest the usage. Don’t give generic names to variables.

Recommendation 10: Encapsulate global variables and constants, enumerated types, and typedefs in a class.

Recommendation 11: Always provide the return type of a function explicitly on a separate line, together with template or inline specifiers.

Recommendation 12: When declaring functions, the leading parenthesis and the first argument (if any) are to be written on the same line as the function name. If space permits, other arguments and the closing parenthesis may also be written on the same line as the function name. Otherwise, each additional argument is to be written on a separate line (with the closing parenthesis directly after the last argument).

Recommendation 13: Always write the left parenthesis directly after a function name (no blanks). Use ‘astyle –style=allman -c -n’ for autoformatting your code.

Recommendation 14: Braces ({ }) which enclose a block are to be placed in the same column as the outer block, on separate lines directly before and after the block. Use indentation of four spaces and don’t use tab stops. Use astyle --style=allman -c -n for autoformatting your code.

Recommendation 15: The reference operator * and the address-of operator & should be directly connected with the type names in declarations and definitions. Use astyle --style=allman -c -n for autoformatting your code.

Recommendation 16: Do not use spaces around . or ->, nor between unary operators and operands. Use astyle --style=allman -c -n for autoformatting your code. Got it? ;-)

Recommendation 17: An assignment operator should return a const reference.

Recommendation 18: Use references instead of pointers whenever possible.

Recommendation 19: Use constant references (const &) instead of call-by-value, unless using a pre-defined data type or a pointer.

Recommendation 20: Avoid long and complex functions.

Recommendation 21: Avoid pointers to functions.

Recommendation 22: Pointers to pointers should be avoided whenever possible.

Recommendation 23: Use a typedef to simplify program syntax when declaring function pointers.

Recommendation 24: Always use unsigned for variables which cannot reasonably have negative values.

Recommendation 25: Always use inclusive lower limits and exclusive upper limits.

Recommendation 26: Avoid the use of continue.

Recommendation 27: Do not write logical expressions of the type if (test) or if (!test) when test is a pointer.

Recommendation 28: Use parentheses to clarify the order of evaluation for operators in expressions.

Recommendation 29: Do not allocate memory and expect that someone else will deallocate it later.

Recommendation 30: Always assign NULL to a pointer after deallocating memory.

Recommendation 31: Check the return codes from library functions even if these functions seem foolproof.

Recommendation 32: If possible, always use initialization instead of assignment. To declare a variable that has been initialized in another file, the keyword extern is always used.

Recommendation 33: Avoid implicit type conversions (casts).

Recommendation 34: Use all flavors of const as often as possible.

3.6.3. Examples¶

Standard Include Header¶

/*
* This file is part of rasdaman community.
*
* Rasdaman community is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* Rasdaman community is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with rasdaman community.  If not, see <http://www.gnu.org/licenses/>.
*
* Copyright 2003 - 2018 Peter Baumann / rasdaman GmbH.
*
* For more information please see <http://www.rasdaman.org>
* or contact Peter Baumann via <baumann@rasdaman.com>.
*/
/*************************************************************
 *
 * PURPOSE:
 *
 * COMMENTS:
 *
 * BUGS:
 *
 ************************************************************/

Standard Include Header (LGPL)¶

/*
* This file is part of rasdaman community.
*
* Rasdaman community is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* Rasdaman community is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with rasdaman community.  If not, see <http://www.gnu.org/licenses/>.
*
* Copyright 2003 - 2018 Peter Baumann / rasdaman GmbH.
*
* For more information please see <http://www.rasdaman.org>
* or contact Peter Baumann via <baumann@rasdaman.com>.
*/
/*************************************************************
 *
 * PURPOSE:
 *
 * COMMENTS:
 *
 * BUGS:
 *
 ************************************************************/

Standard Source Headers¶

/*
* This file is part of rasdaman community.
*
* Rasdaman community is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* Rasdaman community is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with rasdaman community.  If not, see <http://www.gnu.org/licenses/>.
*
* Copyright 2003 - 2018 Peter Baumann / rasdaman GmbH.
*
* For more information please see <http://www.rasdaman.org>
* or contact Peter Baumann via <baumann@rasdaman.com>.
*/
/*************************************************************
 *
 * PURPOSE:
 *
 * COMMENTS:
 *
 * BUGS:
 *
 ************************************************************/

Standard Source Header (LGPL)¶

/*
* This file is part of rasdaman community.
*
* Rasdaman community is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* Rasdaman community is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with rasdaman community.  If not, see <http://www.gnu.org/licenses/>.
*
* Copyright 2003 - 2018 Peter Baumann / rasdaman GmbH.
*
* For more information please see <http://www.rasdaman.org>
* or contact Peter Baumann via <baumann@rasdaman.com>.
*/
/*************************************************************
 *
 * PURPOSE:
 *
 * COMMENTS:
 *
 * BUGS:
 *
 ************************************************************/

Standard Inline Header¶

/*
* This file is part of rasdaman community.
*
* Rasdaman community is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* Rasdaman community is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with rasdaman community.  If not, see <http://www.gnu.org/licenses/>.
*
* Copyright 2003 - 2018 Peter Baumann / rasdaman GmbH.
*
* For more information please see <http://www.rasdaman.org>
* or contact Peter Baumann via <baumann@rasdaman.com>.
/
/**
 * INLINE SOURCE:
 *
 * MODULE:
 * CLASS:
 *
 * COMMENTS:
 *
*/

Standard Script / Make Header¶

#
# MAKEFILE FOR:
#
# This file is part of rasdaman community.
#
# Rasdaman community is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# Rasdaman community is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with rasdaman community.  If not, see <http://www.gnu.org/licenses/>.
#
# Copyright 2003 - 2018 Peter Baumann / rasdaman GmbH.
#
# For more information please see <http://www.rasdaman.org>
# or contact Peter Baumann via <baumann@rasdaman.com>.
# Top Level makefile. This points to the various modules that have to be build
# and/or deployed
#
#
# COMMENTS:
#
##################################################################

Standard Script / Make Header (LGPL)¶

#
# MAKEFILE FOR:
#
# This file is part of rasdaman community.
#
# Rasdaman community is free software: you can redistribute it and/or modify
# it under the terms of the GNU Lesser General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# Rasdaman community is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU Lesser General Public License for more details.
#
# You should have received a copy of the GNU Lesser General Public License
# along with rasdaman community.  If not, see <http://www.gnu.org/licenses/>.
#
# Copyright 2003 - 2018 Peter Baumann / rasdaman GmbH.
#
# For more information please see <http://www.rasdaman.org>
# or contact Peter Baumann via <baumann@rasdaman.com>.
#
#
# COMMENTS:
#
##################################################################

Recomendation 12¶

Correct:

inline int
getLenght()
{
    ...
}

Wrong:

inline int getLenght()
{
    ...
}

Macros vs inline functions¶

Wrong:

#define SQUARE(x) ((x)*(x))         // wrong
int a = 2
int b = SQUARE(a++)                 // a == 6

Right:

inline int
square( int x );                    // right
{
  return (x*x)
}
int c = 2;
int d = square(c++);                // d == 4

Constants vs Standalone Values¶

Wrong:

if (iterations <= 0)
    iterations = 5;

Correct:

// Default number of iterations in units
const int defaultIterationsNumber = 5;

...

if (iterations <= 0)
    iterations = defaultIterationsNumber;

Macros vs const variables¶

#define BUFSIZE 7            // no type checking

const int bufSize = 7        // type checking takes place

enum  size { BufSize = 7 };  // type checking takes place

Standard Method Declaration¶

/**
* Description of addNumbers
* @param n1 the first argument.
* @param n2 the second argument.
* @return The return value
*/
template <class P>
int
addNumbers(int n1, int n2)
{
    ...
}

Case statement¶

switch(tag)
{
  case A:
    // do something
    // break is missing and foo() is also called in case A    // wrong

  case B:
    foo();
    // do something else
    break;

  default:
    // if no match in above cases, this is executed
    break;
}

Dynamic array allocation and deallocation¶

int n = 7
T* myT = new T[n];  // T is type with defined constructors and destructors

//........

delete myT;         // No! Destructor only called for first object in array a.
delete [10] myT ;   // No! Destructor called on memory out of bounds in array a.
delete [] myT ;     // OK, and always safe.

Standard Class Definition¶

Example class definitions in accordance with the style rules

class String : private Object
{
public:
    String();
    String(const String&);
    unsigned getLenght() const;
    inline Encoding getEncoding() const;
    inline void setEncoding(Encoding newEncoding);

protected:
    int checkIndex( unsigned index ) const;

private:
    unsigned noOfChars;
    Encoding encoding;

};

Wrong:

class String
{
  public:
    int getLength() const // No !!
    {
      return length;
    };

  private:
    int length;
};

Correct:

class String
{
  public:
    int getLength() const;

  private:
    int length;
};

inline int
String::getLength() const
{
  return len ;
}

Classes with dynamic member data¶

Declaration examples of the assignment operator:

MySpezialClass&
MySpezialClass::operator= (const MySpezialClass msp);     // no

void
MySpezialClass::operator= (const MySpezialClass msp);     // well

const MySpezialClass&
MySpezialClass::operator= (const MySpezialClass msp);     // recommanded

Class definition

class DangerousBlob
{
  public:
    const DangerousBlob& operator=(const DangerousBlob& dbr);

  private:
    char* cp;
};

Definition of assignment operator:

const DangerousBlob&
DangerousBlob::operator=(const Dangerous& dbr)
{
  if ( this != &dbr )          // Guard against assigning to the "this" pointer
  {
    // ...
    delete cp;                 // Disastrous if this == &dbr
    // ...
  }
}

Constant references as return types:

class Account
{
  public:
    Account ( int myMoney ): moneyAmount(myMoney) { };
    const int& getSafemoney()  const { return moneyAmount;};
    int&       getRiskyMoney() const { return moneyAmount;};  // no

  private:
     int moneyAmount;
};

Account myAcc(10);
myAcc.getSafeMoney()  += 100000;  // compilation error: assignment to constant
myAcc.getRiskyMoney() += 1000000; // myAcc::moneyAmount = 1000010 !!

Note

Method definition within the class definition is forbidden by rule.

Parameter declaration¶

int setPoint( int, int )     // wrong
int setPoint( int x, int y )

int
setPoint( int x, int y )
{
  //....
}

Return type¶

int
calculate ( int j )
{
  return 2*j;
}

void
noReturnType( char* xData, char* yFile)
{
  //....
}

Include directive¶

// file is PrintData.cc

#include "PrintData.hh"    // user include file

#include <iostream.h>      // include file of the system library

Avoid global data¶

class globale
{
  public:
    //........

  protected:
    const char* functionTitle = "good style";

    int   constGlobal;
    char* varGlobal;
}

Formating of functions¶

void foo (); // no
void foo();  // better

// right
int
myComplicateFunction( unsigned unsignedValue,
                      int intValue
                      char* charPointerValue );

// wrong
int myComplicateFunction (unsigned unsignedValue, int intValue char* charPointerValue);

Formating of pointer and reference types¶

char*
object::asString()
{
  // something
};

char* userName = 0;
int   sfBlock  = 42;
int&  anIntRef = sfBlock;

Assignment operator¶

MySpezialClass&
MySpezialClass::operator=( const MySpezialClass& msp ); // no

const MySpezialClass&
MySpezialClass::operator=( const MySpezialClass& msp ); // recommended

Reference vs pointer¶

// Unnecessarily complicated use of pointers
void addOneComplicated ( int* integerPointer )
{
  *integerPointer += 1:
}
addOneComplicated (&j)


// Write this way instead
void addOneEasy ( int& integerReference )
{
  integerReference +=1:
}
addOneEasy(i);

Call-by-value vs call-by-constant-reference¶

// this may lead to very inefficient code.
void foo( string s );
string a;
foo(a)               // call-by-value

// the actual argumment is used by the function
// but it connot be modified by the function.
void foo( const string& s );
string c;
foo(c);              // call-by-constant-reference

Avoid continue¶

while ( /* something */ )
{
  if (/* something */)
  {
    // do something
    continue;                // Wrong!
  }
  // do something
}

// By using an extern 'else' clause, continue is avoided and the code
// is easier to understand

while ( /* something */ )
{
  if (/* something */)
  {
    // do something
  }
  else
  {
    // do something
  }
}

Parentheses¶

// Interpreted as (a<b)<c, not (a<b) && (b<c)
if (a<b<c)
{
  //...
}

// Interpreted as a & (b<8), (a&b) <8
if (a & b<8)
{
  //..
}

// when parentheses are recommended
int i = a>=b && c < d && e+f <= g+h;        // no
int j = (a>=b)&&(c<d) && (( e+f) <= (g+h)); // better

Include Files¶

Include file for the class PackableString:

#ifndef PACKABLESTRING_HH
#define PACKABLESTRING_HH

#include "string.hh".
#include "packable.hh".

/**
 * A test class with elaborate description.
/*

class Buffer:public String:public Packable
{
  public:
    class PackableString (const String& s);
    class Buffer* put (class Buffer* outbuffer);
    //.......
};

#endif

Implementation file for the class PackableString:

// PackableString.cc
// not recommanded <../include/iostream.h> Wrong

#include <iostream.h> // Right
#include "PackableString.hh"
// to be able to use Buffer instances, buffer.hh must be included.
#include "buffer.hh"

Buffer*
PackableString::put(Buffer* outbuffer)
{
    //......
}

3.7. Geo services¶

3.7.1. Petascope Developer’s Documentation¶

Introduction¶

This page serves as an introduction to the petascope component from a developer’s perspective (see also Geo Services Guide).

Petascope is built on the Spring Boot Framework with Hibernate as object relational mapping data model for backend-communication with petascopedb; Implements support for the Coverage Schema Implementation (CIS version 1.0: GridCoverage, RectifiedGridCoverage and ReferenceableGridCoverage and CIS version 1.1: GeneralGridCoverage which is the unified class for coverage types in CIS 1.0).

Petascope can be deployed on more backend DBMS beside PostgreSQL like HSQLDB, H2, etc. Postgresql is still the most stable database for deploying petascope, but the user can switch to other databases by changing the configuration in petascope.properties.

The Spring Boot Framework provides many utilities that aid in quicker development of petascope. Petascope can now start as an embedded web application with an internal embedded Tomcat (i.e: no need to deploy to external Tomcat).

Code¶

Petascope is divided in 3 applications:

core contains the classes to generate petascopedb’s tables by Hibernate with Liquibase and other utilities classes. This is the core library used by other petascope’s applications.
main contains the classes to handle WCS, WCPS, WMS, WCST-T requests and generates rasql queries for rasdaman. This is the rasdaman.war application to be deployed to external Tomcat or started in embedded mode with java -jar rasdaman.war.
migration handles petascopedb migration (must need when updating from v9.4 to v9.5+) using Liquibase; it can also migrates petascopedb from Postgresql to another DBMS like H2 or HSQLDB.

Database migration¶

To support different kinds of databases, we use **Liquibase**, which creates the changes for each update in XML and uses that to generate the SQL statements for the target database (e.g: Postgresql, HSQLDB, H2, etc). To further understand how Liquibase works to populate database tables, see comments in the liquibase.properties config file.

CRS management¶

Petascope relies on a SECORE Coordinate Reference System (CRS) resolver that can provide proper metadata on a coverage’s native CRS. One can either deploy a local SECORE instance, or use the official OGC SECORE resolver at http://www.opengis.net/def/.

It currently keeps a few internal caches, especially for SECORE CRS resources and responses: the gain is both on performance and on robustness against network latencies. Caching information about CRSs is safe as CRSs can be considered static resources - normally they do not change (and with the CRS versioning recently introduced by OGC a particular CRS version never will change indeed).

It is suggested to run a WCS GetCapabilities after a fresh new deployment, so that the CRS definitions of all the offered coverages are cached: after that single request, mainly almost all the CRS-related information has already been cached.

The CrsUtil class serves several purposes:

CRS definitions: the relevant information parsed from a GML CRS definition is stored as a CrsDefinition object. This includes both spatial and temporal reference systems;
CRS equivalence tests: thanks to the /equal endpoint of SECORE, effective equivalence (no simple string comparison) between two reference systems can be verified. This operation is required when checking if a CRS has been cached or not: as an example, KVP notation of a CRS URI is independent of the order of key/value pairs, so that http://www.opengis.net/def/crs?authority=EPSG&version=0&code=32633 and http://www.opengis.net/def/crs?version=0&authority=EPSG&code=32633 are equivalent despite their different URI identifier.

Testing¶

The systemtest/testcase_services covers all the possible cases for WCS, WCPS, WMS and WCS-T. The easiest way to understand how Petascope works is by running some tests and debug it with your IDE (e.g: NetBeans, IntelliJ IDEA,…).

For instance: send this request in Web Browser with deployed petascope in Tomcat: http://localhost:8080/rasdaman/ows?service=WCS&version=2.0.1&request=GetCapabilities. Then you can set a debug in class petascope.controller.PetascopeController of petascope-main application, then follow all following classes when debugging to understand how the request is handled inside petascope.

Warnings¶

Don’t create BigDecimal directly from a double variable, rather from double.toString(). E.g. BigDecimal a = new BigDecimal(0.2356d) will result with random fraction numbers after the real value of double (0.235653485834584395929090423904902349023904290349023904); subsequently this would lead to wrong coefficient calculation in petascope.

3.7.2. WSClient Developer’s Documentation¶

Introduction¶

WSClient is a frontend Web application which facilitates interactions from users to petascope (OGC WCS/WCS-T/WCPS/WMS standards implementation). It it built based on AngularJS framework version 1.4 with other libraries like CSS Bootstrap and WebWorldWind to make a single page application.

When building petascope, WSClient is added to rasdaman.war. This is then deployed to Tomcat. Example of deployed WSClient folder in external Tomcat:

/var/lib/tomcat/webapps/rasdaman/WEB-INF/classes/public/WSClient/

Code¶

WSClient uses TypeScript language rather Javascript directly. To compile WSClient, developers need to install some dependencies:

npm - Node package manger. Example:

sudo yum install npm

bower - Used for managing dependencies. Example:

sudo npm install -g bower

Typescript - Used for compiling .ts files to .js. Example:

sudo npm install -g tsc

TSD - Used for retrieving typings. Example:

sudo npm install -g tsd

Once all dependencies are installed, in the source folder of WSClient (application/wcs-client) run these commands once:

npm install
tsd install
bower install

Then, everytime a new feature/fix is added, one needs to compile from TypeScript to Javascript files to work in Web Browsers with the following command in WSClient source folder:

tsc

After that, 2 important files in application/WSClient/app folder main.js and main.js.map are generated which need to be included in the patch besides other added/updated files.

3.7.3. SECORE Developer’s Documentation¶

Introduction¶

SECORE (Semantic Cordinate Reference System Resolver) is a server which resolves CRS URLs into full CRS definitions represented in GML 3.2.1. Offical SECORE of rasdaman is hosted at: http://www.opengis.net/def.

Same as Petascope, SECORE builds on Spring framework. However, as it is an XML database resolver (mainly all CRSs are occupied from EPSG releases), hence it does not rely on any relational database as petascopedb.

Code¶

SECORE stores and queries XML data in a BaseX XML database. On the disk this database is stored in $CATALINA_HOME/webapps/secoredb (e.g: /var/lib/tomcat/webapps), this is the directory where external Tomcat process will typically have write access. The database is created and maintained automatically, so no action by the user is required regarding this.

In SECORE, there are 2 types of GML Database (UserDictionary.xml and GmlDictionary.xml). User will add/update/delete CRSs only in UserDictionary.xml when GmlDictionary.xml comming from EPSG releases are intact.

SECORE database tree can be viewed and (upon login) modified via graphical web interface at “http://your.server/def/index.jsp”.

More generally, any folder and definition can turn to EDIT mode by appending a /browse.jsp to its URI: e.g. “http://your.server/def/uom/EPSG/0/9001/browse.jsp” will let you view/edit EPSG:9001 unit of measure, whereas “http://your.server/def/uom/EPSG/0/browse.jsp” will let you either remove EPSG UoM definitions or add a new one, not necessarily under the EPSG branch: the gml:identifier of the new definition will determine its position in the tree.

As explained in the `related publication <http://link.springer.com/chapter/10.1007%2F978-3-642-29247-7_5>`_, SECORE supports parametrization of CRSs as well: with this regard, you should mind that relative XPaths are not allowed (either start with / or // when selecting nodes); non-numeric parameters must be embraced by single or double quotes both when setting optional default values in the definition or when setting custom values in the URI.

Update new EPSG version¶

When EPSG announces a new release, one can download the new GML dictionary file from this link: http://www.epsg-registry.org.

From the downloaded .zip file, extract GmlDictionary.xml file inside and add it to SECORE secore database under a folder with version name (e.g: 9.4.2/GmlDictionary.xml).

After that, build SECORE normally to have a new web application def.war and redeploy it to Tomcat server. Finally, check if a new EPSG version is added from http://your.server/def/EPSG/. Example:

<identifiers xmlns="http://www.opengis.net/crs-nts/1.0"
   xmlns:gco="http://www.isotc211.org/2005/gco"
   xmlns:gmd="http://www.isotc211.org/2005/gmd"
   at="http://localhost:8080/def/crs/EPSG/">
   <identifier>http://localhost:8080/def/crs/EPSG/0</identifier>
   <identifier>http://localhost:8080/def/crs/EPSG/8.5</identifier>
   <identifier>http://localhost:8080/def/crs/EPSG/8.9.2</identifier>
   <identifier>http://localhost:8080/def/crs/EPSG/9.4.2</identifier>
</identifiers>