basic-usage.RmdThe purpose of the mrgvalidate package is to generate 7 specific documents that are necessary for the software validation process at Metrum Research Group. Those documents are:
This vignette demonstrates how to generate these with a few simple library calls.
library(mrgvalidate)By default, these calls (create_package_docs() and create_metworx_docs()) will write .docx files for all seven documents into your working directory. Parameterized .Rmd files are first copied over, but are removed at the end (you can override this with the argument cleanup_rmd = FALSE). If all input data conforms to what is described in ?mrgvalidate::input_formats, you should only have to write a single a line.
The spec_df object and test directories below are created by helper functions in the mrgvalprep package. See the function documentation for more details.
create_package_docs(
product_name = "Fake Product",
version = "vFake",
repo_url = "git@github.com:metrumresearchgroup/mrgvalidate.git",
specs = spec_df, # a tibble containing stories and requirements to validate
release_notes_file = "NEWS.md", # file path to a formatted markdown doc of release notes.
auto_test_dir = "some_dir", # directory containing automated test results
style_dir = "style_ref_dir" # Directory that has style references for the generated docx files
)See ?create_package_docs for other arguments to tweak how and where the documents are rendered. The contents of the release_notes_file file should be filtered to just the current version via mrgvalprep.
create_metworx_docs(
product_name = "Fake Product",
version = "vFake",
specs = spec_df, # a tibble containing stories and requirements to validate
release_notes_file = "NEWS.md", # file path to a formatted markdown doc of release notes.
auto_test_dir = "some_dir", # directory containing automated test results
man_test_dir = "some_dir", # directory containing automated test results
style_dir = "style_ref_dir", # Directory that has style references for the generated docx files
output_dir = "output_dir"
)See ?create_metworx_docs for other arguments to tweak how and where the documents are rendered.
If the documents produced by create_package_docs() or create_metworx_docs() don’t look right, or you need to debug for any reason, you can run the pieces separately. For the demonstration below, we will use the following constants:
TEST_INPUTS_DIR <- system.file("test-inputs", package = "mrgvalidate")
output_dir <- file.path("mrgvalidate-create-validation-docs")
product_name <- "example"
# Filter to automatic tests only
specs <- readRDS(file.path(TEST_INPUTS_DIR, "specs.RDS")) %>% dplyr::filter(!grepl("MAN", TestIds))
res_df <-
mrgvalidate::create_package_docs(
product_name = product_name,
version = "vFAKE",
repo_url = "git@github.com:metrumresearchgroup/mrgvalidate.git",
specs = specs,
release_notes_file = file.path(TEST_INPUTS_DIR, "release_notes_sample.md"),
auto_test_dir = file.path(TEST_INPUTS_DIR, "validation-results-sample"),
output_dir = output_dir
)
list.files(output_dir)
#> [1] "release-notes-example.docx"
#> [2] "requirements-specification-example.docx"
#> [3] "testing-plan-example.docx"
#> [4] "testing-results-example.docx"
#> [5] "traceability-matrix-example.docx"
#> [6] "validation-plan-example.docx"
#> [7] "validation-summary-example.docx"Packages do not have manual tests, but for demonstration purposes lets include the previously filtered out manual tests. By default, any missing required links will cause the docs to not be rendered. Additionally, a message displaying the missing links will be returned.
res_df <-
mrgvalidate::create_package_docs(
product_name = product_name,
version = "vFAKE",
repo_url = "git@github.com:metrumresearchgroup/mrgvalidate.git",
specs = specs,
release_notes_file = file.path(TEST_INPUTS_DIR, "release_notes_sample.md"),
auto_test_dir = file.path(TEST_INPUTS_DIR, "validation-results-sample"),
output_dir = output_dir
)
#> Warning in check_input(test_data$dd): Required links between tests and/
#> or requirements are missing. Returning missing information. Docs will not be
#> generated
#> $find_reqs_with_missing_tests
#> # A tibble: 2 × 3
#> RequirementId RequirementDescription TestId
#> <chr> <chr> <chr>
#> 1 AMD-R001 Desktop should have a mountain on it MAN-FAKE-001
#> 2 AMD-R001 Desktop should have the ocean on it MAN-FAKE-002You can also call find_missing() to return all missing links, though unlinked tests are entirely acceptable.
mrgvalidate::find_missing(res_df)
#> $find_tests_without_reqs
#> # A tibble: 0 × 2
#> # … with 2 variables: TestId <chr>, TestName <chr>
#>
#> $find_reqs_with_missing_tests
#> # A tibble: 2 × 3
#> RequirementId RequirementDescription TestId
#> <chr> <chr> <chr>
#> 1 AMD-R001 Desktop should have a mountain on it MAN-FAKE-001
#> 2 AMD-R001 Desktop should have the ocean on it MAN-FAKE-002
#>
#> $find_reqs_without_stories
#> # A tibble: 0 × 2
#> # … with 2 variables: RequirementId <chr>, RequirementDescription <chr>You can easily return the combined dataframe, without generating any of the validation docs, with join_specs_to_tests(), which can be useful for quick debugging:
res_df <- mrgvalidate::join_specs_to_tests(
specs = specs,
auto_test_dir = file.path(TEST_INPUTS_DIR, "validation-results-sample")
)| StoryId | StoryName | StoryDescription | ProductRisk | RequirementId | RequirementDescription | date | test_type | TestId | TestName | passed | failed | man_test_content | result_file |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| JUL-S001 | Julia | As a User, I want to have access to the latest version of Julia. | medium | JUL-R001 | On the current Metworx platform, users should be able to use the current version of Julia | 2021-07-27 13:30:40 EDT | automatic | MXI-JUL-001 | Julia installed in /opt/julia and found in the path | 1 | 0 | NA | vscode-julia-results |
| JUL-S001 | Julia | As a User, I want to have access to the latest version of Julia. | medium | JUL-R001 | On the current Metworx platform, users should be able to use the current version of Julia | 2021-07-27 13:30:40 EDT | automatic | MXI-JUL-002 | Julia version is 1.6.1 | 1 | 0 | NA | vscode-julia-results |
| JUL-S001 | Julia | As a User, I want to have access to the latest version of Julia. | medium | JUL-R001 | On the current Metworx platform, users should be able to use the current version of Julia | 2021-07-27 13:30:40 EDT | automatic | MXI-JUL-003 | Julia can run base test LinearAlgebra/eigen to confirm base library operation | 0 | 1 | NA | vscode-julia-results |
| UTL-S001 | Linux Utilities | As a User, I want to use “basic” Linux utilities (e.g. curl/wget/less/jq/etc). | low | UTL-R001 | Users should have access to the basic tools and utilities relevant to general-use within a Linux environment. | 2021-07-29 12:14:00 | automatic | MXI-UTL-001 | dpkg: packages are installed with correct versions | 10 | 0 | NA | util-results |
| UTL-S001 | Linux Utilities | As a User, I want to use “basic” Linux utilities (e.g. curl/wget/less/jq/etc). | low | UTL-R001 | Users should have access to the basic tools and utilities relevant to general-use within a Linux environment. | 2021-07-29 12:14:00 | automatic | MXI-UTL-002 | required system packages are installed | 20 | 0 | NA | util-results |
| VSC-S001 | VSCode | As a user, I want to have access to Visual Studio, and that a pre-selected list of extensions are available to me. | medium | VSC-R001 | VSCode (Visual Studio) Desktop is available on the remote desktop to users. | 2021-07-27 13:30:40 EDT | automatic | MXI-VSC-001 | something about vscode | 1 | 0 | NA | vscode-julia-results |
| VSC-S001 | VSCode | As a user, I want to have access to Visual Studio, and that a pre-selected list of extensions are available to me. | medium | VSC-R002 | VSCode Desktop should have the following pre-selected extensions successfully connected… | 2021-07-27 13:30:40 EDT | automatic | MXI-VSC-002 | something else about vscode | 1 | 0 | NA | vscode-julia-results |
| ADM-S001 | Pleasant UX | As an Administrator, I want to ensure that the user has an attractive desktop. | Low | AMD-R001 | Desktop should have a mountain on it | NA | NA | MAN-FAKE-001 | NA | NA | NA | NA | NA |
| ADM-S001 | Pleasant UX | As an Administrator, I want to ensure that the user has an attractive desktop. | Low | AMD-R001 | Desktop should have the ocean on it | NA | NA | MAN-FAKE-002 | NA | NA | NA | NA | NA |