Command-Line Interface
======================

.. contents:: Table of Contents

.. _CLI:

The SPBench framework works with a command-line interface (CLI).

The CLI offers some commands. You can check them running:

.. code-block:: bash

   ./spbench --help

For each command there is also a "help" guide. You can access it using:

.. code-block:: bash

   ./spbench [command] -h


Install
-------

``./spbench install -h``

**Description:**
   It install the dependencies for the SPBench benchmarks.
   See :doc:`installation` for more information.

**Optional arguments:**
   - ``-app <some_application>``
      Install dependencies only for the seleted application.

---------

Update
--------

**Description:**
   Use it for setting up the paths and for generating new makefiles for the benchamarks benchmarks.

   You must run it everytime you change the SPBench path or after running the 'configure' command.

**Usage examples:**
   
   For all applications:

   - ``./spbench update``
   
   For a specific benchmark:
   
   - ``./spbench update -bench <benchmark_name>``
   
   For all benchmarks based on a specific application:
   
   - ``./spbench update -app <application_name>``
   
   For all benchmarks implemented with a specific PPI:
   
   - ``./spbench update -ppi <ppi_name>``
   
   For more info:
   
   - ``./spbench update -h``

---------

New
---

``./spbench new -h``

**Description:**
   It allows for creating a new benchmark based on a SPBench application.
   This command will create a fresh copy of the selected application and add it to the database. 

**Required arguments:**
   - ``-bench <new_benchmark_id>``
      Set a name for your new benchmark.

   - ``-app <application_name>``
      You must insert an application ID. See (**applications page link**) for more details.

   - ``-ppi <any_ppi>``
      We recommend inserting a name for the PPI you will use here. But you are free to insert any identification word.

**Optional arguments:**
   - ``-from <other_benchmark_id>`` (Optional)
      Use this option to create your new benchmark as a copy of an existent benchmark. Insert the name of the benchmark you want to copy from

   - ``-nsources``
      Use this option to create a new benchmark with support for multiple sources

**Usage example:**

   - ``./spbench new -bench <new_bench_name> -app <application_name> -ppi <ppi_name>``

---------

Edit
----

``./spbench edit -h``

**Description:**
   It opens a choosen benchmark for editing the source code.

**Required arguments:**
   - ``-bench <benchmark_id>``

**Optional arguments:**
   - ``-editor <editor_name>`` (Default: nano)
      You can select a different text editor available in your machine.

**Usage example:**
   - ``./spbench edit -bench <benchmark_name> -editor <text_editor_name>``

---------

Configure
---------

``./spbench configure -h``

**Description:**
   It allows users to add compiling dependencies for a benchmark.
   It opens a JSON configuration file for users to write the required dependencies.

**Required arguments:**
   - ``-bench <benchmark_id>``

**Optional arguments:**
   - ``-editor <editor_name>`` (Default: nano)
      You can select a different text editor available in your machine.

**Usage example:**
   - ``./spbench configure -bench <benchmark_name> -editor <text_editor_name>``

---------

Compile
-------

``./spbench compile -h``

**Description:**
   It allows users to compile the benchmarks.

**Required arguments:**
   - ``-bench <benchmark_id>`` or ``-ppi <ppi_id>`` or ``-app <application_id>``
      Name ID of the benchmark(s) to compile. These arguments can be combined. Example:

      - ``-bench my_awesome_bench`` compiles the benchmark called 'my_awesome_bench'.

      - ``-bench all`` compiles all benchmarks.

      - ``-app ferret`` compiles all ferret benchmarks.

      - ``-ppi tbb`` compiles all benchmarks parallelized with TBB.

      - ``-app ferret -ppi tbb`` compiles all ferret benchmarks parallelized with TBB.

**Optional arguments:**
   - ``-editor <editor_name>`` (Default: nano)
      You can select a different text editor available in your machine.

   - ``-clean``
      Removes all the object files that had been created for the selected benchmark(s) before compile it.

**Usage example:**
   - ``./spbench configure -bench <benchmark_id> -editor <text_editor_name>``

---------

Exec
----

``./spbench exec -h``

**Description:**
   This command runs the given benchmarks.

**Required arguments:**
   - ``-bench <benchmark_id>`` or ``-ppi <ppi_id>`` or ``-app <application_id>``
      Name ID of the benchmark(s) to execute. These arguments can be combined. Example:

      - ``-bench my_awesome_bench`` executes the benchmark called 'my_awesome_bench'.

      - ``-bench all`` executes all benchmarks.

      - ``-app ferret`` executes all ferret benchmarks.

      - ``-ppi tbb`` executes all benchmarks parallelized with TBB.

      - ``-app ferret -ppi tbb`` executes all ferret benchmarks parallelized with TBB.

   - ``-input <input_id>``
      Insert an input ID (run ``./spbench list-inputs``). 
      You can also insert multiple inputs for multiple source benchmarks.

**Optional arguments:**

   - ``-batch <batch_size>``
      It can be used to change the batch size (default: 1).

   - ``-batch-interval <time_interval_sec>``
      It can be used to change the batch time interval. Default = 0 (disabled).

   - ``-frequency <items_per_second>`` 
      A limited number of items will be available for the source per second.
      By default, no limit is aplied, which is equivalent to maximum frequency.
      The maximum frequency is defined by your system's memory or disk access speed.

   - ``-freq-pattern <frequency_pattern>`` 
      (pattern,period,max,min). 

    This argument overwrites the -frequency command and sets a varying frequency for the input stream. 
    For this argument users need to supply a tuple string: <pattern,period,max,min>, where pattern can be "wave", "spike", "binary", "increasing" or "decreasing". 
    Period is the duration in seconds of one cycle of the pattern. 
    And min is the minimum frequency and max the maximum frequency, defined in items per second. 
    For the spike pattern users can also set the duration of the spikes as percentage of the period (0-100) (e.g. <pattern,period,max,min,spike>). 
    The default spike duration is 10 percent of the period. 
    Usage example: ``-freq-pattern wave,10,5,80``

   - ``-nthreads <number_of_threads>``
      A global variable that can be used to change the number of threads for the parallel benchmarks.

   - ``-in-memory``
      It runs the application in in-memory mode. 

   - ``-latency``
      Print average latency results at the end of the execution.

   - ``-throughput``
      Print the average throughput at the end of the execution.

   - ``-resource-usage``
      It prints the global memory and CPU usage at the end of the execution.
      To use it is required to run the benchmark as root or adjust paranoid value.

   - ``-monitor <time_interval>`` (millisecond)
      Use this option to monitor latency, throughput, CPU and memory usage.
      You must indicate a monitoring time interval in milliseconds (e.g. 250).

   - ``-latency-monitor``
      It monitors latency per stage and per item and generates a log file.

   - ``-user-arg <user_custom_argument>``
      User custom argument. Allow users to pass one or more arguments to use inside the benchmarks. 
      Single argument: [-user-arg single_argument] or [-user-arg "single argument"]. 
      Multiple arguments: [-user-arg two arguments] or [-user-arg two -user-args arguments].

   - ``-test-result``
      This option will run a correctness test on the output (when available) after the benchmark execution.

   - ``-repeat <number_of_repetitions>``
      Use this argument to repeat this execution n times.

   - ``-d`` (Bzip2 only)
      Enables decompress mode for Bzip2 benchmarks.

**Usage example:**

   - ``./spbench exec -bench ferret_tbb_farm -input small -nthreads 8``
      Simple run.

   - ``./spbench exec -bench lane_sequential -input medium -latency -throughput -resource-usage -test-result``
      Running with basic metrics and result testing.

   - ``./spbench exec -bench bzip2_ff_farm -d -input large -nth 6 -latency-monitor -monitor 300``
      Running Bzip2 in decompress mode with monitoring metrics enabled.

   - ``./spbench exec -bench person_seq_ns -input large -input my_custom_input -in-memory``
      Running multiple source Person Recognition benchmark with two inputs and in-memory enabled.

   - ``./spbench exec -bench my_custom_bench -input small -batch 4 -frequency 50 -user-arg "My awesome argument"``
      Running a custom benchmark with batch size = 4 and maximum of 50 input items per second, plus a custom argument.

   - ``./spbench exec -bench my_custom_bench -input huge -batch-time 1.2 -freq-patt binary,7,10.5,70.8``
      Running a custom benchmark with the workload class huge, batch time interval = 1.2 sec., and a binary frequency pattern that oscilates the frequency between 10.5 and 70.8 every 7 seconds.
   
---------

List
----

``./spbench list -h``

**Description:**
   This command lists all SPBench benchmarks, including user's custom benchmarks.

**Optional arguments:**
   - ``-app <application_name>``
      Lists only benchmarks based on the given application.

**Usage example:**
   - ``./spbench list``

---------

Delete
------

``./spbench delete -h``

**Description:**
   It completely deletes a given benchmark. 

**Required arguments:**
   - ``-bench <benchmark_id>``
      Name of the benchmark to delete.

**Usage example:**
   - ``./spbench delete -bench my_not_so_awesome_bench``

---------

Rename
------

``./spbench rename -h``

**Description:**
   This command can be used to change the name of a given benchmark inside SPBench.

**Required arguments:**
   - ``-old-name <old_bench_id>``
      Insert the old and existent benchmark ID.

   - ``-new-name <new_bench_id>``
      Insert the new ID.

**Usage example:**
   - ``./spbench -old my_certainly_working_bench -new my_doubtfully_working_bench``

---------

New-input
---------

.. _CLI-new-input:

``./spbench new-input -h``

**Description:**
   It allows for adding an alias for a custom input.
   The chosen alias will be registered for future uses.
   You must use this option to run the benchmarks using other workloads than the ones provided by SPBench.

**Required arguments:**
   - ``-id <new_input_id>``
      Set a name for your new input (existent entries will be replaced)

   - ``-app <application_name>``  
      Insert an application from the SPBench suite.                   

   - ``-input "<input_string>"``
      You must insert the input string (e.g., "/home/user/my_awesome_input_file ...")

**Optional arguments:**
   - ``-md5 <md5_hash_for_result_testing>`` 
      SPBench uses the associated md5 hash to check the correctness of the result for this input (Optional). 
      ATTENTION: do not insert the input file resulting md5 hash here.
      You must insert the md5 hash regarding the resulting output file.
      We strongly recommend that you run your new input with a sequential benchmark to compute this hash of the resulting output file.

**Usage example:**
   - ``./spbench new-input -id immensely_huge -app bzip2 -input "/home/me/awesome_songs/never_gonna_give_you_up-remix.mp3"``

-----------

List-inputs
-----------

``./spbench list-inputs -h``

**Description:**
   This command lists all application's inputs provided by SPBench, including users custom ones.

**Optional arguments:**
   - ``-app <application_name>``
      Lists only inputs based on the given application.

**Usage example:**
   - ``./spbench list-inputs``

---------

Delete-input
------------

``./spbench delete-input -h``

**Description:**
   Delete a given input ID from the SPBench inputs registry.
   It does not delete the associated files.

**Required arguments:**
   - ``-id <new_input_id>``
      Insert the ID of the input to delete.

   - ``-app <application_name>``  
      Insert the associated app for the given input.  

**Usage example:**
   - ``./spbench delete-input -id my_super_extra_medium_input -app lane_detection``

---------

Download-inputs
---------------

``./spbench download-inputs -h``

**Description:**
   Download the inputs for the SPBench benchmarks.

**Optional arguments:**
   - ``-app <application_name>``
      Use this option for downloading inputs for a specific application.

   - ``-class <workload_class>`` (It applies only for Bzip2)
      You can insert the class of workload you want to download (leave it empty to download all classes).
      Possible options: all, test, small, medium, large, and huge.

   - ``-force``
      Use this option to delete any respective input file and download it again.


**Usage example:**
   - ``./spbench download-inputs`` (recommended)
      It will download everything.
   
   - ``./spbench download-inputs -app bzip2 -class small -force``
      Bzip2 has some large files to download, so users can use this option to avoid downloading everything.
---------

Edit-operator
-------------

``./spbench edit-op -h``

**Description:**
   Open an operator of a given benchmark to edit its source code.

**Required arguments:**
   - ``-benchmark <benchmark_id>``
      Enter the name of an application benchmark to edit (mandatory)

   - ``-operator <operator_id>``  
      Enter the name of the benchmark's operator to edit (mandatory)

**Optional arguments:**
   - ``-editor <editor_name>``  
      Enter the name of a text editor available in your machine, e.g. "vim", "nano", etc. (default: nano)

**Usage example:**
   - ``./spbench edit-op -bench ferret_sequential -op segmentation -editor vi``

---------

List-operators
--------------

``./spbench edit-op -h``

**Description:**
   List all available operators for a given benchmark.

**Required arguments:**
   - ``-benchmark <benchmark_id>``
      You must give the name of the respective benchmark you want to see the operators list (mandatory).

**Usage example:**
   - ``./spbench list-op -bench my_awesome_bench``

---------

Reset-operators
---------------

``./spbench reset-op -h``

**Description:**
   Reset to default the source code of all operators for a given benchmark 
   (it still keeps the old operators as backup until this command is executed again).

**Required arguments (at least one of them must be used):**
   - ``-benchmark <benchmark_id>``
      Insert the name of a benchmark to reset its operators. Caution: It will remove any current modifications on the operators. You can use [-bench all] to reset the operators for all benchmarks.

   - ``-app <app_id>``  
      Reset operators for all benchmarks based on a specific given application.

   - ``-ppi <ppi_id>``  
      Reset operators for all benchmarks implemented using a specific given PPI.

**Usage example:**
   - ``./spbench reset-op -bench my_awesome_bench``

---------

Clean
-----

``./spbench clean -h``

**Description:**
   Clean a benchmark (similar to make clean).
   It also removes old operators.

**Required arguments (at least one of them must be used):**
   - ``-benchmark <benchmark_id>``
      Insert the name of a benchmark to clean (make clean) (optional). You can use 'all' to clean all benchmarks.

   - ``-app <app_id>``  
      Clean all benchmarks based on a specific given application.

   - ``-ppi <ppi_id>``  
      Clean all benchmarks implemented using a specific given PPI.

**Usage example:**
   - ``./spbench clean -bench all``

---------
