Command line (qgis_process)

Similar to QGIS processing algorithms you can also run EnMAP-Box algorithms directly from the command line using the QGIS Processing Executor , i.e., the qgis_process command. This can be especially useful to automate repetitive tasks (batch processing) and/or when deploying EnMAP-Box workflows on server environments.

Note

Depending on your installation the command might also be named qgis_process-qgis or qgis_process-qgis-ltr

Basics

On systems without window manager (e.g. servers) set the environment variable QT_QPA_PLATFORM before using qgis_process

export QT_QPA_PLATFORM=offscreen

Make sure to activate the EnMAP-Box plugin first via

qgis_process plugins enable enmapboxplugin

To show all available processing algorithms, run

qgis_process list

For information on a specific algorithm and its parameters, use the help command with the corresponding algorithm name, e.g.:

qgis_process help enmapbox:ImportEnmapL2AProduct

Finally, to run an algorithm, use the run command followed by the algorithm name and its parameters, e.g.:

qgis_process run enmapbox:ImportEnmapL2AProduct --file="ENMAP01-____L2A-DT0000001280_20220627T104500Z_001_V010301_20230517T020623Z-METADATA.XML" --outputEnmapL2ARaster="output.vrt"

Example 1: Bulk import of L2A scenes

In this example we import multiple EnMAP L2A scenes (all located inside a folder $data_dir) and save them as a GeoTiff to another folder $output_dir. The ImportEnmapL2AProduct algorithm only outputs virtual rasters (.vrt), so we add the SaveRasterLayerAs algorithm as a second step and handle intermediate outputs as temporary data.

#!/bin/bash

# Define input and output directories
data_dir="somepath/data"
output_dir="somepath/output"

# Iterate over files in the data directory
find "$data_dir" -name "*METADATA.XML" -print0 | while IFS= read -r -d '' f; do

    # Generate output filename with the desired extension
    output=$(basename "$f" | sed "s/\-METADATA.XML/.tif/")

    # Create a temporary directory
    tmp_dir=$(mktemp -d)

    # Run QGIS processing commands
    qgis_process run enmapbox:ImportEnmapL2AProduct \
        --file="$f" \
        --outputEnmapL2ARaster="$tmp_dir/temp.vrt"

    qgis_process run enmapbox:SaveRasterLayerAs \
        --raster="$tmp_dir/temp.vrt" \
        --creationProfile="GTiff INTERLEAVE=BAND COMPRESS=LZW PREDICTOR=2 BIGTIFF=YES" \
        --outputRaster="$output_dir/$output"

    # Clean up temporary directory
    rm -r "$tmp_dir"
done

Or run in parallel using GNU parallel:

#!/bin/bash

# Define input and output directories
data_dir="somepath/data"
output_dir="somepath/output"

# Function to process a single file
process_file() {
    local f="$1"
    local output_dir="$2"

    # Generate output filename with the desired extension
    local output=$(basename "$f" | sed "s/\-METADATA.XML/.tif/")

    # Create a temporary directory
    local tmp_dir=$(mktemp -d)

    # Run QGIS processing commands
    qgis_process run enmapbox:ImportEnmapL2AProduct \
        --file="$f" \
        --outputEnmapL2ARaster="$tmp_dir/temp.vrt"

    qgis_process run enmapbox:SaveRasterLayerAs \
        --raster="$tmp_dir/temp.vrt" \
        --creationProfile="GTiff INTERLEAVE=BAND COMPRESS=LZW PREDICTOR=2 BIGTIFF=YES" \
        --outputRaster="$output_dir/$output"

    # Clean up temporary directory
    rm -r "$tmp_dir"
}

# Export the function so that it can be used with parallel
export -f process_file

# Use GNU Parallel to process files in parallel (4 CPUs)
find "$data_dir" -name "*METADATA.XML" -print0 | parallel -j4 -0 process_file {} $output_dir

Tip

Instead of enmapbox:SaveRasterLayerAs you could also use the more elaborate enmapbox:TranslateRasterLayer where you can, among other things, make spatial and/or spectral subsets of the input raster.

Example 2: Classification

The EnMAP-Box image classification process involves three main steps:

Creating a classification dataset using a feature raster and a layer which holds information on classes (in this example a point vector layer)
This dataset with the pixel values (features) and the categories from the vector layer (target) is then used to train a machine learning model
The resulting model is then applied to an image for classification

Using qgis_process, a typical classification workflow could look like this:

#!/bin/bash

data_dir="$HOME/.local/share/QGIS/QGIS3/profiles/default/python/plugins/enmapboxplugin/enmapbox/exampledata"
output_dir="somepath/output"

qgis_process run enmapbox:CreateClassificationDatasetFromCategorizedVectorLayerAndFeatureRaster \
  --featureRaster="$data_dir/enmap_potsdam.tif" \
  --excludeBadBands=1 \
  --categorizedVector="$data_dir/landcover_potsdam_point.gpkg" \
  --categoryField="level_2" \
  --outputClassificationDataset="$output_dir/dataset.pkl"

qgis_process run enmapbox:FitRandomforestclassifier \
  --dataset="$output_dir/dataset.pkl" \
  --classifier="from sklearn.ensemble import RandomForestClassifier; classifier = RandomForestClassifier(n_estimators=100, oob_score=True)" \
  --outputClassifier="$output_dir/rfc_fit.pkl"

qgis_process run enmapbox:PredictClassificationLayer \
  --raster="$data_dir/enmap_potsdam.tif" \
  --classifier="$output_dir/rfc_fit.pkl" \
  --matchByName=1 \
  --outputClassification="$output_dir/classification.tif"