Testing

Changes

The test environment is significantly changed. This had to be dones due to the required support of automated distributed combined tests on various platforms [platformids] with various implementations [pythonids].

This required some patches for the implementations, and in addition the possibility of simple distribution and execution of the complete stack of data processing on multiple HOSTs and VMs. The distribution is hereby designed as a user-land developer distribution either based on git or rsync via SSH.

There for the packaging is changed by separation of runtime components for the production system, and the test data with the test procedures including the complete sources as the SDK package.

The most significant changes for the testing are:

• changed all imports to absolute/relative to PYTHONPATH - see also “Google Python Style Guide” [pystylegoogle]

• splitted code and tests:

  • production package with runtime only including runtime sources and data

  • SDK package, including complete sources, document sources, and test data and test cases

  • changed file locations for test data:

    • from : xmlschema/tests/test_cases

    • to: testdata/xmlschema

  • changed file locations for tests:

    • from : xmlschema/tests

    • to: tests

  • decomposed test cases to smaller more speific subdirectory structures:

    • from : xmlschema/tests/*

    • to: tests/<case-tree>

• Changes to the original setup.py of xmlschema:

  • Official support for Python-3.4 dropped due to platform components which may require Python-3.5+, but may still work.

  • added support for implementations, and though tested - see also datafusion, xmllathe, xmldata, and pythonids

    • IPython - some minor differences to CPython

    • IronPython

    • Jython

    • PyPy

  • added support for all mayor OS, and though tested - see also datafusion, xmllathe, xmldata, and platformids

    • Linux

    • BSD

    • OS-X

    • Windows

    • UNIX - Solaris10 + Solaris11

  • added command classes for setuplib

Call Interface

The new directory layout provides for:

• Simple extension and reduction of the actual performed test cases, by commandline use of inittest as well as by integrated application via the PyDev plugin of Eclipse.

• The generic and reusable parts of the test scripts are separated into the tests.xmlschema_acue_tests.testools subdirectory in order to be easily applied by arbitrary and slim drop-in test cases.

• The test data is separated into the directory testdata.xmlschema, though supports eased shared usage for arbitrary test cases.

• The test cases are supported by the setplib with the command testx for simplified regression testst - via console access and batch scripts.

• The test cases are split into smaller groups and theme parts.

Added some hotfixes for the support of Jython version 2.7.0 and 2.7.1. Prepared structure to serve partially as a pattern for other projects.

• changes of the runtime API in the runtime package is 100% compatible, while the test-data and the cases are now contained in the source package only

• renamed package temporarily to xmlschema_acue for avoidance of name collisions in public repos - PyPi.org

Basic Transformation of Test-Scripts

The application of unittest remains unchanged, and is expanded for closer integration of TDD into PyDev by the enhancement of the drop-in use of arbitrary test cases.

The following basic changes and decomposition is made, for the actual details refer to the directories:

original	changed to	remarks
xmlschema.tests.test_all.py	tests.xmlschema	Simply call the unittest in the root directory.
xmlschema.tests.test_helpers.py	tests.xmlschema_acue_tests.basics.xsd	Tests for some used basic helper functions and classes
xmlschema.tests.test_meta.py	tests.xmlschema_acue_tests.libs.meta	Tests for the XSD meta-schema and XSD builtins
xmlschema.tests.test_models.py	tests.xmlschema_acue_tests.libs.models	Tests concerning model groups validation
xmlschema.tests.test_package.py	tests.xmlschema_acue_tests.basics.etree	Tests regarding ElementTree import and code packaging
xmlschema.tests.test_regex.py	tests.xmlschema_acue_tests.libs.regex	Tests about XSD regular expressions
xmlschema.tests.test_resources.py	tests.xmlschema_acue_tests.libs.resources	Tests about XML/XSD resources access
xmlschema.tests.test_schemas.py	tests.xmlschema_acue_tests.libs.schemas	Tests about parsing of XSD Schemas
xmlschema.tests.test_validators.py	tests.xmlschema_acue_tests.libs.validators	Tests regarding XML data validation/decoding/encoding
xmlschema.tests.test_xpath.py	tests.xmlschema_acue_tests.libs.xpath	Tests for XPath parsing and selectors
various contained shared components	tests.xmlschema_acue_tests.testtools	Shared base classesses and generic functions
xmlschema.tests.test_cases	testdata.xmlschema	XML data used for tests
xmlschema.tests.test_w3c_suite	tests.xmlschema_acue_tests.standards.w3c	Tests on official reference data from the W3C - w3c.org
–	tests.xmlschema_acue_tests.standards.ietf	Tests on official reference data from the IETF - ietf.org
–	tests.xmlschema_acue_tests.standards.oasis	Tests on official reference data from the OASIS - oasis.org

Test cases based on files

The tests based on files by the standard library fileinput remain unchanged, the call interface related to the filesystem location is adapted to the new directory structure.

The call interface is changed to parameterless calls, thus the commandline arguments are dropped and separated in favour of the static definition of automated unittests. For example in case of Elementtree and lxml.

Compliance Tests for Standards

The compliance tests are separated into the directory tests.xmlschema_acue_tests.standards with subdirectories for each organization. This comprises official conformance test suites as well as extracts from the standards texts for the verification of standard and special cases.

W3C

Testing with the W3C XML Schema 1.1 test suite

From release v1.0.11, using the script test_w3c_suite.py, you can run also tests based on the W3C XML Schema 1.1 test suite. To run these tests, actually limited to XSD 1.0 schema tests, clone the W3C repo on the project’s parent directory and than run the script:

git clone https://github.com/w3c/xsdtests.git
python tests/xmlschema/standards/w3c/w3c_suite_11/CallCase.py

OASIS

IETF