Spec: Test results structure, documentation and automation

[taylor]

Structure and documentation for test results

Goal:

• easy to find test results
• easy to understand test results
• allow automation of running and saving selected set of tests

Planned changes

• Make the test framework (NFVbench) web-based
  • Control through HTTP (REST/json). Maybe make use of Python API
  • Results available through Kibana/Grafana (Elasticsearch)
• Configure Fluentd to handle data between NFVbench and Elasticsearch
• Create dashboards for Kibana/Grafana

Tasks:

• Containerized deployment of Elasticsearch
• Containerized deployment of Kibana/Grafana
• Reconfigure NFVbench for CNF Testbed
  • Use web API for control
  • Push data to Fluentd
• Set up web-based control of NFVbench
  • (maybe) create web-page for controlling NFVbench
  • (alternative) provide tools for easier use of REST API
• Set up Kibana/Grafana to parse/display test results
  • Set up web interface
  • Create dashboard(s)
• Create documentation for how to use

[taylor]

Quick ideas:

• New folder test_reults with subfolders for different tests (probably associated with use cases/examples)
• Each test folder would have a dated test run subfolder for a test run
• Each test folder would have a README documenting what is being tested and the result data (eg. maximum receive rate)

Nice to have:

• summary of results in each test run subfolder
• link to latest summary of results from test folder

[michaelspedersen]

Updated task with more details. Started looking at implementation

Not sure if this should be included in spec, or in a separate task

[michaelspedersen]

(WIP) Data sent from NFVbench though Fluentd to Elasticsearch. Visualized using Kibana dashboard

[michaelspedersen]

Documentation has been added for the visualization part