diff --git a/docs/source/CLI.rst b/docs/source/CLI.rst index 009b1307c3c33390dab239f3644796ba2623cea4..4bb70a211641b3390daac5d3e377be4fca22d8e2 100644 --- a/docs/source/CLI.rst +++ b/docs/source/CLI.rst @@ -19,15 +19,14 @@ For each command there is also a "help" guide. You can access it using: ./spbench [command] -h - -Install -------- +SPBench install +--------------- ``./spbench install -h`` **Description:** It install the dependencies for the SPBench benchmarks. - See :doc:`source/installation` for more information. + See :doc:`installation` for more information. **Optional arguments:** - ``-app `` @@ -35,8 +34,31 @@ Install --------- -Update ------- +Global configuration file +------------------------- + +``./spbench global-config -h`` + +**Description:** + It opens a JSON configuration file for adding global compiling dependencies. + The information added to keys in the global configuration file will override the information present in the individual configuration file of each benchmark when generating makefiles. + The 'EXTRA' keys do not override the original information, but append to it instead. + See :doc:`_global-configuration` for more details. + +**Optional arguments:** + - ``-editor `` (Default: nano, vim, or vi) + You can select a different text editor available in your machine. + +**Usage example:** + - ``./spbench global-config -editor `` + +--------- + +Operations with benchmarks +-------------------------- + +Update benchmarks +~~~~~~~~~~~~~~~~~ **Description:** @@ -72,8 +94,8 @@ Update --------- -New ---- +New benchmark +~~~~~~~~~~~~~ ``./spbench new -h`` @@ -86,13 +108,13 @@ New Set a name for your new benchmark. - ``-app `` - You must insert an application ID. See (**applications page link**) for more details. + You must insert an application ID. See :doc:`applications` for more details or run [./spbench list-apps] to see the available applications. +**Optional arguments:** - ``-ppi `` - We recommend inserting a name for the PPI you will use here. But you are free to insert any identification word. + We recommend inserting a name for the PPI you will use here. But you are free to insert any identification word. Default is 'sequential'. -**Optional arguments:** - - ``-from `` (Optional) + - ``-from `` Use this option to create your new benchmark as a copy of an existent benchmark. Insert the name of the benchmark you want to copy from - ``-nsources`` @@ -104,8 +126,8 @@ New --------- -Edit ----- +Edit benchmark +~~~~~~~~~~~~~~ ``./spbench edit -h`` @@ -124,14 +146,15 @@ Edit --------- -Configure ---------- +Configure benchmark +~~~~~~~~~~~~~~~~~~~ ``./spbench configure -h`` **Description:** It allows users to add compiling dependencies for a benchmark. It opens a JSON configuration file for users to write the required dependencies. + See :doc:`configuration` for more details. **Required arguments:** - ``-bench `` @@ -145,8 +168,8 @@ Configure --------- -Compile -------- +Compile benchmark +~~~~~~~~~~~~~~~~~ ``./spbench compile -h`` @@ -179,8 +202,8 @@ Compile --------- -Exec ----- +Execute benchmarks +~~~~~~~~~~~~~~~~~~ ``./spbench exec -h`` @@ -212,7 +235,6 @@ Exec - ``-debug`` By default, SPBench captures the benchmark output while it is running and prints all the output at once at the end of the execution. SPBench does this so that it can compute some additional metrics based on the output of the benchmarks. If you want it to print the output on-the-fly (e.g. to print debug info added by you, or any print message), you must use the -debug argument in the execution command. Notice that, if this option is enabled, some advanced SPBench features may not work properly. It will not show the standard deviation of the metrics or store the result in the general log file, for instance. - - ``-nthreads `` Number of threads. It can be used inside the benchmarks as a global variable. You can insert a single number or a range using the ':' character (e.g. ``-nthreads 1:10``). You can also define iteration step of the range (e.g. ``-nthreads 4:4:16`` would run the benchmarks with 4, 8, 12, and 16 threads). Default value: 1. @@ -318,19 +340,23 @@ Exec If you use repetitions (``-repeat ``) and ``-nthreads `` in range mode together in the exec command, a specific resulting log will be automatically generated inside the log folder. This log file will store average latency, throughput, and exec. time, with their respective std. deviation side by side. It may be handy for running experiments and saving time gathering resulting data. The resulting log file is ready for users to draw charts over the results. +.. Note:: + + See :doc:`how_to_run`, :doc:`management_options`, and :doc:`metrics` for more helpful information. + **Usage example:** - ``./spbench exec -bench ferret_ff_farm -input test -latency -throughput -nthreads 1:2:10 -repeat 3`` You can find the resulting log file for this usage case in the ``log/ferret_ff_farm_1-2-10.dat`` file. --------- -List ----- +List benchmarks +~~~~~~~~~~~~~~~ ``./spbench list -h`` **Description:** - This command lists all SPBench benchmarks, including user's custom benchmarks. + This command lists all SPBench benchmarks, including the user's custom benchmarks. **Optional arguments:** - ``-app `` @@ -341,8 +367,8 @@ List --------- -Delete ------- +Delete a benchmark +~~~~~~~~~~~~~~~~~~ ``./spbench delete -h`` @@ -358,8 +384,8 @@ Delete --------- -Rename ------- +Rename a benchmark +~~~~~~~~~~~~~~~~~~ ``./spbench rename -h`` @@ -378,9 +404,96 @@ Rename --------- -New-input +Clean a benchmark +~~~~~~~~~~~~~~~~~ + +``./spbench clean -h`` + +**Description:** + Clean a benchmark (similar to make clean). + It also removes old operators. + +**Required arguments (at least one of them must be used):** + - ``-benchmark `` + Insert the name of a benchmark to clean (similar to make clean). You can use 'all' to clean all benchmarks. + + - ``-app `` + Clean all benchmarks based on a specific given application. + + - ``-ppi `` + Clean all benchmarks implemented using a specific given PPI. + + - ``-logs `` + Delete all execution logs of SPBench. + + - ``-outputs `` + Delete all the outputs generated by the benchmarks + +**Usage example:** + - ``./spbench clean -bench all`` + Clean all benchmarks. + + - ``./spbench clean -app lane_detection -logs`` + Clean all lane detection benchmarks and all SPBench logs + + - ``./spbench clean -ppi -outputs`` + Clean all benchmarks using a specific PPI and the outputs from all benchmarks. + + - ``./spbench clean -logs -outputs`` + Delete all SPBench logs and the outputs from all benchmarks. + + +Operations with input workloads +------------------------------- + +Download-inputs +~~~~~~~~~~~~~~~ + +``./spbench download-inputs -h`` + +**Description:** + Download the inputs for the SPBench benchmarks. + +**Optional arguments:** + - ``-app `` + Use this option for downloading inputs for a specific application. + + - ``-class `` + You can insert the class of workload you want to download (leave it empty to download all classes). + Possible options: all, test, small, medium, large, and huge. + + - ``-force`` + Use this option to delete any respective input file and download it again. + +**Usage example:** + - ``./spbench download-inputs`` (recommended) + It will download everything. + + - ``./spbench download-inputs -app bzip2 -class small -force`` + Bzip2 has some large files to download, so users can use this option to avoid downloading everything. + --------- +List-inputs +~~~~~~~~~~~ + +``./spbench list-inputs -h`` + +**Description:** + This command lists all application's inputs provided by SPBench, including users custom ones. + +**Optional arguments:** + - ``-app `` + Lists only inputs based on the given application. + +**Usage example:** + - ``./spbench list-inputs`` + +--------- + +New-input +~~~~~~~~~ + .. _CLI-new-input: ``./spbench new-input -h`` @@ -412,25 +525,8 @@ New-input ----------- -List-inputs ------------ - -``./spbench list-inputs -h`` - -**Description:** - This command lists all application's inputs provided by SPBench, including users custom ones. - -**Optional arguments:** - - ``-app `` - Lists only inputs based on the given application. - -**Usage example:** - - ``./spbench list-inputs`` - ---------- - Delete-input ------------- +~~~~~~~~~~~~ ``./spbench delete-input -h`` @@ -450,60 +546,78 @@ Delete-input --------- -Download-inputs ---------------- - -``./spbench download-inputs -h`` - -**Description:** - Download the inputs for the SPBench benchmarks. +Operations with applications +---------------------------- -**Optional arguments:** - - ``-app `` - Use this option for downloading inputs for a specific application. +List applications +~~~~~~~~~~~~~~~~~ - - ``-class `` - You can insert the class of workload you want to download (leave it empty to download all classes). - Possible options: all, test, small, medium, large, and huge. +``./spbench list-apps -h`` - - ``-force`` - Use this option to delete any respective input file and download it again. +**Description:** + List the information of all available applications in SPBench. **Usage example:** - - ``./spbench download-inputs`` (recommended) - It will download everything. - - - ``./spbench download-inputs -app bzip2 -class small -force`` - Bzip2 has some large files to download, so users can use this option to avoid downloading everything. + - ``./spbench list-apps`` --------- -Edit-operator -------------- +New application +~~~~~~~~~~~~~~~ -``./spbench edit-op -h`` +``./spbench -new-app -h`` **Description:** - Open an operator of a given benchmark to edit its source code. + This command can be used for adding a new application to SPBench. + This command will create a toy application and add it to the database. + Users can use this toy application as a template to write their onw application or import/translate into SPBench an existing one. + Please, check :doc:`applications` **Required arguments:** - - ``-benchmark `` - Enter the name of an application benchmark to edit (mandatory) + - ``-app `` + You must insert an ID for your new application. - - ``-operator `` - Enter the name of the benchmark's operator to edit (mandatory) + - ``-operators [list of operators]`` + You must insert a list with the names of the operators of your new application (in order). Source and Sink operators are automatically included. You should not include them. **Optional arguments:** - - ``-editor `` - Enter the name of a text editor available in your machine, e.g. "vim", "nano", etc. (default: nano) + - ``-name <"application name">`` + By default, the application's name would be the same as the application ID. However, you can define a different name for it using this argument. + + - ``-description <"description">`` + A description of your application. + + - ``-notes <"notes">`` + Any notes, observations, or usage instructions for your application. **Usage example:** - - ``./spbench edit-op -bench ferret_sequential -op segmentation -editor vi`` + + - ``./spbench new-app -app my_new_app -name "Full Name of My App" -op anOperator anotherOp lastOp -notes "Item order must be guaranteed on the output"`` --------- +Delete an application +~~~~~~~~~~~~~~~~~~~~~ + +``./spbench delete-app -h`` + +**Description:** + It completely deletes a given application. + +**Required arguments:** + - ``-app `` + Name of the application to delete. + +**Usage example:** + - ``./spbench delete-app -app my_test_application`` + +-------- + +Operations with operators +------------------------- + List-operators --------------- +~~~~~~~~~~~~~~ ``./spbench edit-op -h`` @@ -519,65 +633,50 @@ List-operators --------- -Reset-operators ---------------- +Edit-operator +~~~~~~~~~~~~~ -``./spbench reset-op -h`` +``./spbench edit-op -h`` **Description:** - Reset to default the source code of all operators for a given benchmark - (it still keeps the old operators as backup until this command is executed again). + Open an operator of a given benchmark to edit its source code. -**Required arguments (at least one of them must be used):** +**Required arguments:** - ``-benchmark `` - Insert the name of a benchmark to reset its operators. Caution: It will remove any current modifications on the operators. You can use [-bench all] to reset the operators for all benchmarks. + Enter the name of an application benchmark to edit (mandatory) - - ``-app `` - Reset operators for all benchmarks based on a specific given application. + - ``-operator `` + Enter the name of the benchmark's operator to edit (mandatory) - - ``-ppi `` - Reset operators for all benchmarks implemented using a specific given PPI. +**Optional arguments:** + - ``-editor `` + Enter the name of a text editor available in your machine, e.g. "vim", "nano", etc. (default: nano) **Usage example:** - - ``./spbench reset-op -bench my_awesome_bench`` + - ``./spbench edit-op -bench ferret_sequential -op segmentation -editor vi`` --------- -Clean ------ +Reset-operators +~~~~~~~~~~~~~~~ -``./spbench clean -h`` +``./spbench reset-op -h`` **Description:** - Clean a benchmark (similar to make clean). - It also removes old operators. + Reset to default the source code of all operators for a given benchmark + (it still keeps the old operators as backup until this command is executed again). **Required arguments (at least one of them must be used):** - ``-benchmark `` - Insert the name of a benchmark to clean (similar to make clean). You can use 'all' to clean all benchmarks. + Insert the name of a benchmark to reset its operators. Caution: It will remove any current modifications on the operators. You can use [-bench all] to reset the operators for all benchmarks. - - ``-app `` - Clean all benchmarks based on a specific given application. + - ``-app `` + Reset operators for all benchmarks based on a specific given application. - - ``-ppi `` - Clean all benchmarks implemented using a specific given PPI. - - - ``-logs `` - Delete all execution logs of SPBench. - - - ``-outputs `` - Delete all the outputs generated by the benchmarks + - ``-ppi `` + Reset operators for all benchmarks implemented using a specific given PPI. **Usage example:** - - ``./spbench clean -bench all`` - Clean all benchmarks. - - - ``./spbench clean -app lane_detection -logs`` - Clean all lane detection benchmarks and all SPBench logs - - - ``./spbench clean -ppi -outputs`` - Clean all benchmarks using a specific PPI and the outputs from all benchmarks. - - - ``./spbench clean -logs -outputs`` - Delete all SPBench logs and the outputs from all benchmarks. + - ``./spbench reset-op -bench my_awesome_bench`` +--------- diff --git a/docs/source/SPBench_applications.rst b/docs/source/SPBench_applications.rst index 970b910b94c3a191112e913b2c827bac51326af2..9fe28f28ddbdd732910e4deec23adaa8b95af9db 100644 --- a/docs/source/SPBench_applications.rst +++ b/docs/source/SPBench_applications.rst @@ -122,3 +122,50 @@ This application can be divided into a three-stage Pipeline (source, compress/de .. rst-class::_space_here_clear-both +Adding New Applications +======================= + +.. _new-applications: + +Adding new applications to SPBench can be challenging, even small test application. +The main issue is that in SPBench the source codes of each application are in different places in the SPBench file system, making it harder to visualize the applications as a whole. +The representation below shows how a simpliest structure an application can have inside SPBench. + +.. code-block:: shell + + # File structure of a simple application: + SPBench + |__ benchmarks + | |__ app_id + | |__ PPI_id + | |__ bench_id + | |__ operators + | | |__ include + | | | |__ operator1_op.hpp + | | | |__ operatorN_op.hpp + | | |__ src + | | | |__ operator1_op.cpp + | | | |__ operatorN_op.cpp + | | |__ operators.json + | |__ config.json + | |__ bench_id.cpp + | |__ app_id.hpp + | |__ Makefile + |__ sys + |__ apps + |__ app_id + |__ templates + | |__ operators + | | |__ include + | | | |__ operator1_op.hpp + | | | |__ operatorN_op.hpp + | | |__ src + | | | |__ operator1_op.cpp + | | | |__ operatorN_op.cpp + | | |__ operators.json + | |__ app_id.hpp + | |__ app_id.cpp + | |__ config.json + |__ app_id_utils.hpp + |__ app_id_utils.cpp + diff --git a/docs/source/conf.py b/docs/source/conf.py index 35a20be5af8a7ddd3cdb4726b97e0db7fbb5f8b8..d5a2fca113c8fca3e33a8f0a38b9ce209614ab75 100644 --- a/docs/source/conf.py +++ b/docs/source/conf.py @@ -19,6 +19,7 @@ extensions = [ 'sphinx.ext.intersphinx', 'sphinx_rtd_theme', 'sphinxcontrib.slide', + 'sphinx.ext.autosectionlabel', ] intersphinx_mapping = { diff --git a/docs/source/configuration.rst b/docs/source/configuration.rst index 69364ba548cb0dc5d06c551d1396f09ee98b6e43..91a6871bde9181fa544581fd8b46ed922b22bbd8 100644 --- a/docs/source/configuration.rst +++ b/docs/source/configuration.rst @@ -1,7 +1,7 @@ Compiling Configuration ======================= -.. _configuration +.. _configuration: .. contents:: Table of Contents @@ -101,6 +101,8 @@ You must not add the encapsulation of the command, only the parameters. Global Compiling Configuration ------------------------------ +.. _global-configuration: + The global JSON configuration file is similar to the benchmark-specific one, but it adds some extra keys. The basic rule is that the JSON keys in the global configuration file override the local ones. diff --git a/docs/source/index.rst b/docs/source/index.rst index 93fa225fd141777457502547b3ac62a18d073268..f527740aba772143ca7555543e3c039109a3eb71 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -38,7 +38,7 @@ The SPBench suite comprises the following stream processing applications and wil - Bzip2 - Face Recognizer (Person Recognition) -See more details in :doc:`applications`. +See more details in :ref:`applications`. For each of them, there are parallel benchmarks implemented using the following parallel programming interfaces and will be more in the future: diff --git a/docs/source/management_options.rst b/docs/source/management_options.rst index 82d2601a321812b7719737467ead1a97ab139c05..cf4c3f15b0572cd9981df3666d9cee3e3054b474 100644 --- a/docs/source/management_options.rst +++ b/docs/source/management_options.rst @@ -1,7 +1,7 @@ Management Options ================== -.. _management +.. _management: .. contents:: Table of Contents diff --git a/docs/source/moving.rst b/docs/source/moving.rst index 6056515850e203956e4246e1293eee47b0a3b448..1a6080bfd272029d413d71128a33fa394660c0ca 100644 --- a/docs/source/moving.rst +++ b/docs/source/moving.rst @@ -1,7 +1,7 @@ Moving SPBench ============== -.. _moving +.. _moving: Here we discuss how to move SPBench somewhere else, which could be another directory or another computer.