Advanced Usage (Script)

After the installation, the next sections describe how to get started. The sections configuration and logging describe the general methods we use, this is helpful to understand how you can change model parameters and similar.

Advanced Installation (Linux)

This is a quick guide on how to install AvaFrame and the required dependencies on your machine. AvaFrame is developed on Linux machines (Ubuntu/Manjaro/Arch) with recent Python versions > 3.8. These instructions assume you are familiar with working in a terminal. This guide is described for Linux. For Windows, see Advanced Installation (Windows).

Requirements

Install git and python, we suggest to work with miniconda/anaconda. For installation see miniconda or anaconda. Some operating systems might require the python headers (e.g python-dev on ubuntu) or other supporting libraries/packages (e.g. Visual Studio on Windows needs the c++ compiler components).

Setup AvaFrame

Create a new conda environment for AvaFrame, activate it and install pip, numpy and cython in this environment:

conda create --name avaframe_env
conda activate avaframe_env
conda install pip numpy cython

Clone the AvaFrame repository (in a directory of your choice: [YOURDIR]) and change into it:

cd [YOURDIR]
git clone https://github.com/avaframe/AvaFrame.git
cd AvaFrame

Compile the cython com1DFA part. You might also have to install a c-compiler (gcc or similar) through your systems package manager:

python setup.py build_ext --inplace

Warning

You will have to do this compilation every time something changes in the cython code. We also suggest to do this everytime updates from the repositories are pulled.

Install avaframe and its requirements by either doing:

pip install -e .

or if this fails (see github issue 986), do:

python setup.py develop

This installs avaframe in editable mode, so every time you import avaframe the current (local) version will be used.

Test it by starting python and do an import avaframe. If no error comes up, you are good to go.

Head over to First run for the next steps.

Advanced Installation (Windows)

This is a quick guide on how to install AvaFrame and the required dependencies on your machine. AvaFrame is developed on Linux machines (Ubuntu/Manjaro/Arch) with recent Python versions > 3.8. These instructions assume you are familiar with working in a terminal. This guide is described for Windows. For Linux, see Advanced Installation (Linux).

Requirements

Install git and python, we suggest to work with miniconda/anaconda. For installation see miniconda or anaconda.

Install Microsoft C++ compiler. Follow the installation steps for the version corresponding to the installed python version.

Setup AvaFrame

Open conda shell and create a new conda environment for AvaFrame, activate it and install pip, numpy and cython in this environment:

conda create --name avaframe_env
conda activate avaframe_env
conda install pip numpy cython

Clone the AvaFrame repository (in a directory of your choice: [YOURDIR]) and change into it:

cd [YOURDIR]
git clone https://github.com/avaframe/AvaFrame.git
cd AvaFrame

Compile the cython com1DFA part:

python setup.py build_ext --inplace

Warning

You will have to do this compilation every time something changes in the cython code. We also suggest to do this everytime updates from the repositories are pulled.

Before compilation in Windows, make sure to delete AvaFrame/build directory, in addition to any .pyd, .c, and .pycache files in AvaFrame/avaframe/com1DFA

Install avaframe and its requirements by either doing:

pip install -e .

or if this fails (see github issue 986), do:

python setup.py develop

This installs avaframe in editable mode, so every time you import avaframe the current (local) version will be used.

Test it by starting python and do an import avaframe. If no error comes up, you are good to go.

If wanted: Integrate conda environment into editor of your choice (i.e. PyCharm, Spyder or any other)

Head over to First run for the next steps.

Using QGIS from Conda

It is possible to have the script installation and the plugin installed on your machine at the same time. However, be aware: Depending on your mode of QGis installation (direct installer, conda, OSGeo4W…) your script AvaFrame installation might be overruled.

The steps below install QGis and the Connector plugin in a separate conda environment. If the steps above have been followed, this method will result in two script installations where one will be used by QGis and the other can be worked on separately, preventing any overlap.

If you have another QGis installation somewhere, make sure that the AvaFrame plugin is not installed. To create and activate a new environment, which will contain your installation of QGis open your conda terminal and run:

conda create -n qgis_latest
conda activate qgis_latest

To install the latest QGIS version, use:

conda install qgis --channel conda-forge

Note

Conda will always try to install the latest version. If you want to use another version, you need to specify it, e.g.:

conda install qgis=3.34.11 --channel conda-forge

Now install the avaframe requirements:

conda install pip numpy cython

Now find the directory where your environment is located. The path should be something like: C:\Users\USER\miniconda3\envs\qgis_latest. Change into it:

cd \miniconda3\envs\qgis_latest

Clone the AvaFrame repository and change into it:

git clone https://github.com/avaframe/AvaFrame.git
cd AvaFrame

Note

This will pull the latest developer version of AvaFrame, if you want the current release version for the plugin, you need to specify the version, e.g.:

https://github.com/avaframe/AvaFrame/tree/1.8.3

Compile the cython com1DFA part and create an editable avaframe version in your environment:

python setup.py build_ext --inplace
pip install -e .

Run QGIS:

qgis

Now you can install the AvaFrameConnector plugin via QGIS as per usual (Operational Installation).

AvaFrame should now be installed, and both installations should work and be editable separately. To display the installed version and installation location, use pip show avaframe in the respective environment, or use the getVersion function in the plugin. Note that the script versions are separate, and changes in one installation will not affect the other.

Updating QGIS

To update QGIS to the most recent version, you need to run the following command with the respective environment active:

conda update qgis -c conda-forge

First run

Follow these steps to run your first simulation:

change into your AvaFrame directory (replace [YOURDIR] with your path from the installation steps):
```
cd [YOURDIR]/AvaFrame/avaframe
```
run:
```
python runCom1DFA.py
```

a similar output should show up:

logUtils - INFO -  Started logging at: 03.11.2020 22:42:04
logUtils - INFO -  Also logging to: data/avaParabola/runCom1DFA.log
runCom1DFA - INFO -  MAIN SCRIPT
runCom1DFA - INFO -  Current avalanche: data/avaParabola
...

This will perform a dense flow avalanche simulation using the com1DFA module. The results are saved to data/avaParabola/Outputs/com1DFA. For a first look at the results, got to the folder reports, there you can find a markdown report of the simulations performed including some plots of the results.

To display markdown files in a nice way use a markdown viewer of your choice. Some other options are:

Use the Atom editor with a markdown plugin
If you have pandoc installed use this to convert it to pdf/html
Some browsers have markdown extensions you can install easily

Workflow example

The following example should make it easier for you to find your way in AvaFrame and setup your own AvaFrame workflow after you did the full setup. There is also a directory with examples for different workflows, see more here: Example runscripts.

Make sure you change to your AvaFrame directory by:

cd [YOURDIR]/AvaFrame

Replace [YOURDIR] with the directory from your installation step.

Initialize project

To create the folder where the input data lies and where the output results will be saved, specify the full path to the folder in the local_avaframeCfg.ini (which is a copy of avaframeCfg.ini that you need to create). So:

cd avaframe
cp avaframeCfg.ini local_avaframeCfg.ini

and edit local_avaframeCfg.ini with your favorite text editor and adjust the variable avalancheDir.

Then run

python runScripts/runInitializeProject.py

This will create a new directory with the input required by AvaFrame structured as described in Initialize Project.

Input data

Check the input data required by the different modules you want to use and fill the Inputs/ inside the [avalancheDir] folder from the initialize step accordingly.

For example the com1DFA module needs input as described in Input. You can also have a look at the default setting for the module you want to use (for example com1DFACfg.ini for module com1DFA). If you want to use different settings, create a local_ copy of the .ini file and modify the desired parameters.

More information about the configuration can be found here: Configuration

Building your run script

Create your own workflow by taking the runOperational.py script as template.

We suggest you copy it and adjust it to your liking. There are annotations in the code that should help you to understand the structure.

A lot more examples can be found in the runScripts directory (see also Example runscripts).

Configuration

In order to set the configurations required by all the modules within Avaframe, the python module configparser is used.

This is done in two steps. The first step fetches the main settings:

from avaframe.in3Utils import cfgUtils
# Load avalanche directory from general configuration file
cfgMain = cfgUtils.getGeneralConfig()
avalancheDir = cfgMain['MAIN']['avalancheDir']

In the second step the specific settings to a given module are imported:

from avaframe.tmp1Ex import tmp1Ex
# Load all input Parameters from config file
# get the configuration of an already imported module
# Write config to log file
cfg = cfgUtils.getModuleConfig(tmp1Ex)

The in3Utils.cfgUtils.getModuleConfig() function reads the settings from a configuration file (tmpEx.ini in our example) and writes these settings to the log file. The default settings can be found in the configuration file provided within each module.

It is possible to modify these settings, there are two options:

provide the path to your own configuration file when calling cfgUtils.getModuleConfig(moduleName, path to config file)
create a copy of the module configuration file called local_ followed by the name of the original configuration file and set the desired values of the individual parameters.
see Override configuration for additional options to modify configuration

So the order is as follows:

if there is a path provided, configuration is read from this file.
if there is no path provided, the local_... configuration file is read if it exists.
if there is no local_..., the getModuleConfig function reads the settings from the default configuration file with the default settings.

In the configuration file itself, there are multiple options to vary a parameter:

replace the default parameter value with desired value
provide a number of parameter values separated by | (e.g. relTh=1.|2.|3.)
provide a number of parameter values using start:stop:numberOfSteps (e.g. relTh=1.:3.:3)) - a single value can be added by appending &4.0 for example

Override configuration

If tools of one module, let’s call this module A for now, are called from another module B, there is the option to include an collectionName_A_override section in the configuration file of module B. In this case, the default configuration of module A is read and the parameters in the A_override section in the module B configuration file are used to update the configuration settings of module A. This has the advantage of gathering all the configuration parameters used for one task in one configuration file. An example of this usage can be found in ana1Tests/energyLineTestCfg.ini.

Logging

In order to generate simulation logs and to control what is prompted to the terminal, we use the python module logging.

Let’s have a look at the simple example in runScripts.runTmp1Ex and tmp1Ex.tmp1Ex() on how this is used within AvaFrame.

In your main script call:

from avaframe.in3Utils import logUtils
# log file name; leave empty to use default runLog.log
logName = 'runTmp1Ex'
# specify the working directory
avalancheDir = './'
# ---------------------------------------------
# Start logging
log = logUtils.initiateLogger(avalancheDir, logName)

This will configure the logging (it sets the console output as well as the log file). In your modules/subscripts add:

import logging
log = logging.getLogger(__name__)

So you can use:

log.debug('Should be here')
log.info('DEM : %s',variable)

To get output that looks like this in your console:

tmp1Ex:DEBUG - Should be here
tmp1Ex:INFO - DEM : /path/to/DEM

and something similar in the .log file which is saved in ./runTmp1Ex.log in this example. The logging configuration is set in AvaFrame/avaframe/in3Utils/logging.conf.

You can modify this logging.conf file to modify the levels or format of the messages to display (python doc will help you).

Example runscripts

In runScripts we provide ready-to-use scripts for different applications of the modules provided within AvaFrame.

Derive input data

runScripts.runComputeDist

Create a new project

runScripts.runInitializeProject

Generate idealized/generic topography data

runScripts.runGenerateTopo
runScripts.runGenProjTopoRelease

Postprocessing

runScripts.runAna3AIMEC
runScripts.runAna3AIMECCompMods
runScripts.runAna3AIMECIndi
runScripts.runStatsExample
runScripts.runProbAna

Visualisation

runScripts.runQuickPlotSimple
runScripts.runQuickPlotOne
runScripts.runPlotTopo
runScripts.runExportToCsv

Testing

runScripts.runDamBreak
runScripts.runSimilaritySol
runScripts.runTestFP
runScripts.runStandardTestsCom1DFAOrig
runScripts.runComparisonModules
runScripts.runFetchBench
runScripts.runWriteDesDict

Update AvaFrame

To update go to your avaframe repository [YOURDIR]/Avaframe, pull the latest changes via:

git pull

and repeat the compilation step from above:

python setup.py build_ext --inplace

If there are updates on the requirements inside setup.py, it might be necessary to run:

pip install -e .

again to get the additional requirements installed.