Rx - RIExerciser - Home Networking Testing Tool

Rx Project Management:

Current Project Overall Status:

Summary of Rx Project Phase 1

OcapAppDriver – The OcapAppDriver library is phase 1 complete except for the following areas:
RiScriptlet – The RiScriptlet is phase 1 complete. Current known issues and additional requested functionality is as follows:
Ri Bean Shell Test Scripts – The following table lists the current status of the tests from the spreadsheet which were to be completed for Phase 1. The list of known issues with test scripts follows this table.
1. OCORI-4057 - OcapAppDriverTest.bsh should be completed and have tests to cover all OcapAppDriver functions

OCORI-4056 - runRx.sh should perform "-setup" option to ensure scripts are in place

OCORI-4054 - OcapAppDriverTest.bsh become the default test for RiScriptlet

OCORI-4053 - runRx.sh should suppress errors

OCORI-4050 - Add ability to retrieve HTTP HEAD/GET response strings from RI telnet interface to facilitate testing

OCORI-4049 - RiExerciser should read in the value for "OCAP.hn.server.name"

OCORI-3776 - DvrExerciser should support an HN only build

OCORI-2177 - DVRExerciser - need to resolve directory used when publishing vs. directory used for streaming

OCORI-4046 - RxScriptlet has exception at startup which does not impact running test but should figure out why it happens

OCORI-4040 - HN Playback fails intermittently in RiExerciser

OCORI-4036 - RiExerciser needs to implement JMF playback

OCORI-4032 - Convenience features missing from RiExerciser GUI

OCORI-4037 - RiExerciser needs resource contention handling

OCORI-4034 - bean shell scripts not able to run from level2

OCORI-4033 - shutting down RiScriptlet telnet session thru telnet quit command causes telnet thread to spin

OCORI-4029 - RiExerciser needs to add a playback indicator for local DVR playback and remote playback

OCORI-4028 - RiExerciser needs the ability to log use of CDS content item

OCORI-4027 - RiExerciser needs ability to log use of CDS recording content item

OCORI-4026 - RiExerciser needs ability to log use of local recording item

OCORI-4008 - Live Streaming playback on the same machine returns error code from AppDriver

Agenda for Monday, Feb. 6th

Past Meeting Agenda & Minutes - Meeting Notes

Rx Project Overview:

The RxProject is an effort to take the current DvrExerciser OCAP Xlet and refactor it into a frame work which provides the following:

A library of functionality which aggregates OCAP API calls into functional building blocks provided by RI Stack & PC Platform.
A scripting interface which allows automation of testing of these functional building blocks.
A "Guide" Type OCAP Xlet which is referred to as the RxExerciser which supports functional/application level testing and development through a GUI.

Rx Project Purpose:

QUESTION: We already have a bunch of existing test frameworks (CTP, Swak, DvrTest, etc.), why are we inventing another one?

ANSWER:

DvrExerciser is our current "guide type application" and widely used by RI developer. It is also used to support DLNA CTT and UPnP CTT tests. As part of the rewrite, we needed ability to verify its functionality which will be done through this framework. What better way to test RiExerciser?
RiExerciser which is an ad hoc tool which is currently used by RI developers. Developers put the effort in to adding additional functionality to DvrExerciser so features can be tested at a functional level. Why not leverage the results of developer testing? This framework allows for automation without much additional effort. An example is the need for automated integration type tests for HN Streaming and Resource Contention Handling.
RxProject will be part of the open source so outside contributors will be able to write bean shell scripts to illustrate issues and also demonstrate fixes (can't do this with the CTP framework).
Currently have a multiple of Xlets in the QA directory. Each are required to implement Xlet and there is much redundant code. If we used the Rx framework instead, could either utilize common utilities already found in OcapAppDriver or add to the common functionality (Rx framework is better than writing individual xlet and duplicating existing code).
For HN, we need a framework where tests can be run to verify robustness in a network environment with many devices. HN CTP tests assume an isolated network.

QUESTION: Because of the limitations placed on the OcapAppDriver interface, no callbacks, only simple type calling parameters, minimal thread support, it will never meet the needs of RI developers for testing, can it?
ANSWER:

If the framework needs additional concurrency support, we could leverage some existing code such as the Concurrency Utils Scott D recommended, DvrTest or Swak framework classes, etc.
The purpose of this framework is to meet the majority of our integration type test needs. We will adhere to our existing rules and keep the framework simple to meet the needs of the majority of our tests. Test cases which are so complex that can't be performed in this framework, will need to be performed elsewhere.

Rx Project Phases:

The Rx Project has been divided into the following phases along with proposed dates for completion:

Phase 0: Minimal Functionality required to demonstrate the POC UC (Completion Date: Monday, Dec. 5, 2011):