Quick Start Guide

This guide helps you quickly set up Qualityfolio, generate a queryable SQLite database from test artifacts, and launch a Test Management Dashboard using Spry, Surveilr, and SQLPage.

Welcome to Qualityfolio - a Markdown-native Test Management system backed by Spry’s ontology engine and Surveilr’s data ingestion pipeline. Qualityfolio follows a Test Management as Code (TMaC) approach, where test artifacts, execution evidence, and dashboards are all derived from structured files under version control.

Objectives

Qualityfolio is designed to:

Generate an SQLite database from test artifact and execution evidence files.
Generate a Test Management Dashboard for test metrics, traceability, and execution insights.
HTML UI Generation (Live Preview) with real-time ontology parsing, live reload, and structural validation.

Prerequisites

Ensure the following tools and files are available on your system before proceeding:

Deno – runtime required by Spry
Homebrew (brew) – package manager for installing dependencies
Surveilr – ingestion and transformation engine
Spry – runbook and SQLPage orchestration engine
SQLPage – UI layer for the Test Management Dashboard
qualityfolio-json-etl.sql – SQL-based ETL script for Qualityfolio data
qualityfolio.md – Markdown file containing database configuration and SQLPage queries
sqlpage.json – SQLPage runtime configuration file (under ./sqlpage/)

Installation

To work with Spry and the Surveilr-based Qualityfolio framework, you need to install Deno, Homebrew, Spry, Surveilr, and SQLPage, and ensure the required Qualityfolio files are available in your workspace.

Installation Steps

Open your terminal and switch to your workspace directory:

cd $HOME/workspaces

1. Install Homebrew (if not already installed)

Homebrew is required to install Spry.

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

Verify Homebrew installation:

brew --version

2. Install Deno

Install Deno using the following command:

curl -fsSL https://deno.com/install.sh | sh

Verify your Deno installation:

deno --version

Upgrade Deno to the latest version:

deno upgrade

3. Install Spry

If you are a first-time Spry user:

brew install programmablemd/packages/spry

If Spry is already installed:

brew uninstall spry
brew untap programmablemd/packages
brew install programmablemd/packages/spry

Verify your Spry installation:

spry --version

For more details on Spry installation and usage, refer to the official Spry documentation.

4. Install Surveilr

Check if Surveilr already exists on your system:

which surveilr

If an existing version is found, remove it:

sudo rm -rf <surveilr_path_with_file_name>

Download Surveilr version 3.10.0 or above (the following example uses version 3.10.0):

wget https://github.com/surveilr/packages/releases/download/3.10.0/surveilr_3.10.0_x86_64-unknown-linux-gnu.tar.gz

Extract the archive:

tar -xzf surveilr_3.10.0_x86_64-unknown-linux-gnu.tar.gz

Move Surveilr to /usr/local/bin/:

sudo mv surveilr /usr/local/bin/

Verify Surveilr installation and ensure the version is v3.10.0 or above:

surveilr --version
sudo surveilr upgrade -v 3.10.0

For more details on Surveilr installation and usage, refer to the official Surveilr documentation.

5. Install SQLPage

Install SQLPage using Homebrew:

brew install sqlpage

Verify SQLPage installation:

sqlpage --version

Folder Structure

Qualityfolio expects the following directory structure:

ASSURANCE-PRIME/
├── support/
│   └── assurance/
│       └── qualityfolio/
│           ├── evidence/
│           │   ├── TC-GLUE-001/
│           │   │   └── 1.1/
│           │   │       ├── result.auto.json
│           │   │       └── run.auto.md
│           │   └── TC-GLUE-002/
│           │       └── 1.1/
│           │           ├── loginButtonClick.png
│           │           ├── result.auto.json
│           │           └── run.auto.md
│           ├── sqlpage/
│           │   └── sqlpage.json                  # runtime configuration file for SQLPage
│           ├── test-artifacts/
│           │   └── example-artifact.md
│           ├── qualityfolio-json-etl.sql         # SQL ETL script for Qualityfolio data
│           ├── qualityfolio.md                   # SQLPage Markdown page (DB config + queries)
│           └── resource-surveillance.sqlite.db   # Database generated

Key Directories and Files

test-artifacts/ – Authoring location for test cases, plans, suites, and metadata
evidence/ – Auto-generated test execution evidence (results, runs, attachments)
qualityfolio.md – Core logic file that defines database creation and SQLPage queries
qualityfolio-json-etl.sql – ETL script that transforms ingested JSON into relational tables
sqlpage/sqlpage.json – SQLPage runtime configuration
resource-surveillance.sqlite.db – Generated SQLite database containing test data

Authoring Tests in Markdown

Qualityfolio tests follow a structured Markdown hierarchy to ensure full traceability and machine-readable execution. Qualityfolio projects can range from simple to highly complex structures. You can explore ready-made examples based on real-world implementations.

An example test artifact file (example-artifact.md) for the project, OWASP - GLUE UP (with Qualityfolio’s Small pattern) is available here.

This example demonstrate:

Header-based classification
Hierarchical test design
Evidence block patterns

Step 1: Generate SQLite Database from Test Artifacts

To generate a queryable SQLite database from your test artifacts and execution evidence, run the following command from the qualityfolio directory:

spry rb run qualityfolio.md

What This Does

Parses test artifacts under test-artifacts/
Ingests execution evidence under evidence/
Applies the ETL logic defined in qualityfolio.md and qualityfolio-json-etl.sql
Produces a clean, ready-to-use SQLite database: resource-surveillance.sqlite.db

You can open this database using any SQLite-compatible SQL editor for ad-hoc analysis or reporting.

Step 2: Generate Test Management Dashboard

The Test Management Dashboard provides a live, browser-based view of Qualityfolio metrics.

2.1 Prepare Test Artifacts

Ensure your test cases and related metadata are authored as Markdown files under: test-artifacts/

2.2 Run Spry + SQLPage Pipeline

Execute the following commands in a terminal from the qualityfolio directory:

spry rb run qualityfolio.md
spry sp spc --fs dev-src.auto --destroy-first --conf sqlpage/sqlpage.json --md qualityfolio.md
spry sp spc --fs dev-src.auto --destroy-first --conf sqlpage/sqlpage.json --md qualityfolio.md --watch

This pipeline:

Rebuilds the database from artifacts and evidence
Generates sqlpage.json in the sqlpage folder and configures SQLPage using sqlpage.json
Loads SQLPage queries defined in qualityfolio.md
Watches for file changes and auto-refreshes the data

2.3 Start SQLPage Server

In a separate terminal, from the qualityfolio directory, start the SQLPage server:

sqlpage

2.4 Access the Test Management Dashboard

Open your browser and navigate to: http://localhost:9227/

The dashboard includes:

Test metrics and summaries
Requirement Traceability Matrix (RTM)
Test cycle-wise execution status
Case-level execution details with linked evidence

Step 3: HTML UI Generation (Live Preview)

You can launch the server using the following command in a terminal from the qualityfolio directory:

spry axiom web-ui test-artifacts/example-artifact.md

Running this command loads the parsed test artifact(s) in your browser with:

Real-time ontology parsing
Automatic live reload whenever you save a file
A structured HTML UI showing headings, roles, metadata, hierarchy, cases, and evidence
Built-in validation of classification rules and document structure

This feature allows you to visually inspect and validate your Qualityfolio pattern directly from HTML UI.

Next Steps

Once your database and dashboard are running, you can:

Query resource-surveillance.sqlite.db for custom analytics
Extend SQLPage views by updating qualityfolio.md
Add new test artifacts and execution evidence
Integrate Qualityfolio into CI/CD pipelines