<?xml version="1.0"?>
<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.1.2//EN"
               "http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd" [
<!ENTITY % version-entities SYSTEM "version.entities">
%version-entities;
]>
<refentry id="gst-validate-launcher">
  <refentryinfo>
    <author>
      <othername>The GstValidate team</othername>
      <personblurb>see http://cgit.freedesktop.org/gstreamer/gst-devtools/</personblurb>
    </author>
    <productname>gst-validate</productname>
  </refentryinfo>

  <refmeta>
    <refentrytitle>gst-validate-launcher</refentrytitle>
    <manvolnum>1</manvolnum>
    <refmiscinfo class="source">GstValidate</refmiscinfo>
    <refmiscinfo class="version">&GST_API_VERSION;</refmiscinfo>
    <refmiscinfo class="manual">GstValidate Manual Pages</refmiscinfo>
  </refmeta>

  <refnamediv>
    <refname>gst-validate-launcher</refname>
    <refpurpose>Tool to launch GstValidate testsuites</refpurpose>
  </refnamediv>

  <refsynopsisdiv>
    <cmdsynopsis>
      <command>gst-validate-launcher</command>
      <arg choice="opt" rep="repeat">options</arg>
      <arg choice="opt" rep="repeat">TESTSUITE</arg>
    </cmdsynopsis>
  </refsynopsisdiv>

  <refsect1>
    <title>Description</title>
    <para><command>gst-validate-launcher</command> is an application to create
      full testsuites on top of the GstValidate tools, testing behaviour with
      dynamic pipelines and user actions (seeking, changing the pipeline
      state, etc.) as described by the <link linkend="scenarios">scenario</link> format.
    </para>
    <refsect2 id="gst-validate-launcher--default-testsuite">
      <title>Run the GstValidate default testsuite</title>
      <para>
      GstValidate comes with a default testsuite to be executed on a default set of media samples.
      Those media samples are stored with <command>git-annex</command> so you
      will need it to be able to launch the default testsuite.
      </para>
      <para>
        The first time you launch the testsuite, you will need to make sure that the media samples are
        downloaded. To do so and launch the testsuite you can simply do:
      </para>
      <informalexample>
          <programlisting>gst-validate-launcher validate --sync</programlisting>
      </informalexample>
      <para>
        This will only launch the GstValidate tests and not other applications that might be supported
        (currently <command>ges-launch</command> is also supported and has its own default testsuite).
      </para>
      <para>
        Launching the default testsuite will open/close many windows, you might want to mute it
        so you can keep using your computer:
      </para>
      <informalexample>
        <programlisting>gst-validate-launcher validate --sync --mute</programlisting>
      </informalexample>
    </refsect2>
    <refsect2 id="gst-validate-launcher--implement-testsuite">
      <title>Example of a testsuite implementation</title>
      <para>
        To implement a testsuite, you will have to write some simple python code that defines
        the tests to be launched by <command>gst-validate-launcher</command>.
      </para>
      <para>
        In this example, we will assume that you want to write a whole new testsuite based on
        your own media samples and <link linkend="scenarios">scenarios</link>.
        The set of media files and the testsuite implementation file will be structured as follow:
      </para>
      <literallayout>
testsuite_folder/
  |-> testsuite.py
  |-> sample_files/
      |-> file.mp4
      |-> file1.mkv
      |-> file2.ogv
  |-> scenarios
      |-> scenario.scenario
      |-> scenario1.scenario
      </literallayout>
      <para>
          You should generate the <filename>.media_info</filename> files. To generate them for local files,
          you can use:
      </para>
      <informalexample>
        <programlisting>gst-validate-launcher --medias-paths /path/to/sample_files/ --generate-media-info</programlisting>
      </informalexample>
      <para>
            For remote streams, you should use <command>gst-validate-media-check-&GST_API_VERSION;</command>. For an http stream you can for example do:
      </para>
      <informalexample>
            <programlisting>gst-validate-media-check-&GST_API_VERSION; http://someonlinestream.com/thestream \
              --output-file /path/to/testsuite_folder/sample_files/thestream.stream_info</programlisting>
      </informalexample>
      <para>
        The <command>gst-validate-launcher</command> will use the generated
        <filename>.media_info</filename> and <filename>.stream_info</filename>
        files to validate the tests as those contain the necessary information.
      </para>
      <para>
        Then you will need to write the <filename>testsuite.py</filename> file. You can for example implement the following testsuite:
      </para>
      <informalexample>
        <programlisting language="python">
import os

# Make sure gst-validate-launcher uses our media files
options.paths = os.path.dirname(os.path.realpath(__file__))

# Make sure GstValidate is able to use our scenarios
# from the testsuite_folder/scenarios folder
os.environ["GST_VALIDATE_SCENARIOS_PATH"] = \
    os.path.join(os.path.dirname(os.path.realpath(__file__)), "scenarios")

# You can activate the following if you only care about critical issues in
# the report:
# os.environ["GST_VALIDATE"] = "print_criticals"

# Make gst-validate use our scenarios
validate.add_scenarios(["scenario", "scenario1"])


# Now add "Theora and Vorbis in OGG container" as a wanted transcoding format. That means
# that conversion to this format will be tested on all the media files/streams.
validate.add_encoding_formats([MediaFormatCombination("ogg", "vorbis", "theora")])

# Use the GstValidatePlaybinTestsGenerator to generate tests that will use playbin
# and GstValidateTranscodingTestsGenerator to create media transcoding tests that
# will use all the media format added with validate.add_encoding_formats
validate.add_generators([validate.GstValidatePlaybinTestsGenerator(validate),
                         GstValidateTranscodingTestsGenerator(self)])

# Blacklist some tests that are known to fail because a feature is not supported
# or due to any other reason.
# The tuple defining those tests is of the form:
# ("regex defining the test name", "Reason why the test should be disabled")
validate.set_default_blacklist([
        ("validate.*.scenario1.*ogv$"
         "oggdemux does not support some action executed in scenario1")]
        )
        </programlisting>
      </informalexample>

      <para>
        Once this is done, you've got a testsuite that will:
      </para>
      <itemizedlist>
        <listitem>
          <para>
          Run playbin pipelines on <filename>file.mp4</filename>, <filename>file1.mkv</filename> and
          <filename>file2.ogv</filename>> executing <literal>scenario</literal> and <literal>scenario1</literal> scenarios
          </para>
        </listitem>
        <listitem>
          <para>
          Transcode <filename>file.mp4,</filename> <filename>file1.mkv</filename> and <filename>file2.ogv</filename>
          to Theora and Vorbis in a OGG container
          </para>
        </listitem>
      </itemizedlist>
      <para>
        The only thing to do to run the testsuite is:
      </para>
      <informalexample>
        <programlisting>gst-validate-launcher --config /path/to/testsuite_folder/testsuite.py</programlisting>
      </informalexample>
    </refsect2>
  </refsect1>

  <refsect1><title>Invocation</title>
    <para>
        You can find detailed information about the launcher by launching it:
    </para>
    <informalexample>
      <programlisting>gst-validate-launcher --help</programlisting>
    </informalexample>
  </refsect1>
</refentry>
