Skip to content

Latest commit

 

History

History
543 lines (370 loc) · 18 KB

HACKING.rst

File metadata and controls

543 lines (370 loc) · 18 KB
Raw

GENERAL DEVELOPMENT NOTES

Project's sources are accessible from a Mercurial version control repository hosted at BitBucket.

Project development should be tracked in the TODO.txt file.

  • Exact formatting is not important as long as its content is kept formatted consistently.
  • Done tasks should be marked as such and not deleted.

Testing:

  • pytest testing framework needed to run unit tests.
  • To run the tests using Python 3 first process them and the rest of the library sources using the Python py2to3 conversion tool.
  • For more detailed information see the DEVELOPMENT & TESTING ENVIRONMENT section below.

Reproducing problematic use cases:

  • Failing web service processing examples can be easily packaged as reproducible test cases using the suds library 'message & reply injection' technique.
  • Some things you can achieve using this technique (for examples, see existing project unit tests):
    • Create a client object based on a fixed WSDL string.
    • Have a client object send a fixed request string without having it construct one based on the loaded WSDL schema and received arguments.
    • Have a client object process a fixed reply string without having it send a request to an actual external web service.

Base sources should remain Python 2.x compatible. Since the original project states aiming for Python 2.4 compatibility we should do so as well.

Python features that need to be avoided for compatibility with older Python versions:

  • Features introduced in Python 2.5.

    • any & all functions.

    • with statement.

    • try/except/finally blocks.

      • Prior to this Python release, code like the following:

        try:
    A
except XXX:
    B
finally:
    C

        was considered illegal and needed to be written using nested try blocks as in:

        try:
    try:
        A
    except XXX:
        B
finally:
    C

    • yield expression inside a try block with a finally clause.

      • Prior to this Python release, code like the following:

        try:
    yield x
finally:
    do_something()

        is considered illegal, but can be replaced with legal code similar to the following:

        try:
    yield x
except:
    do_something()
    raise
do_something()

  • Features introduced in Python 2.6.

    • bytes type.
    • Byte literals, e.g. b"quack".
    • String format() method.

  • Features introduced in Python 2.7.

    • Dictionary & set comprehensions.
    • Set literals.

Handling Python 3 related patches applicable to the original suds development project.

  • Originally synchronized with the Mercurial patch queue maintained by Bernhard Leiner.
  • Used to be kept synchronized with the originating queue but no longer. It became more and more difficult to keep them synchronized as our work progressed and both the original suds project and this patch queue seem dead otherwise.
  • Used to be committed first to the 'Python 3 support' branch and then merged back to the trunk from there.

External documentation:

For additional design, research & development project notes see the project's notes/ folder.

TOP-LEVEL FILES & FOLDERS

.hg/
.hgignore
.hgtags
  • Mercurial version control related data.
build/
dist/
suds_jurko.egg-info/
  • Folders created during project setup procedure (build/install).
notes/
  • Internal project design, research & development notes.
suds/
  • Basic project source code.
tests/
  • Project test code.
MANIFEST.in
  • Build system configuration file listing the files to be included in the project's source distribution packages in addition to those automatically added to those packages by the used package preparation system.
HACKING.rst
LICENSE.txt
README.txt
TODO.txt
  • Internal project documentation.
run_all_tests.cmd
  • Basic development script for running the full project test suite using multiple Python interpreter versions on a Windows development machine.
setup.cfg
  • Basic project Python configuration.
setup.py

  • Standard Python project setup script.

  • Usage examples:

    setup.py --help

    show detailed usage information

    setup.py build

    build the project

    setup.py develop

    prepare the development environment (add the project folder to the Python module search path)

    setup.py install

    build & install the project

    setup.py register

    register a project release at PyPI

    setup.py sdist

    prepare a source distribution

    setup.py test

    run the project's test suite (requires pytest)

    setup.py upload

    upload prepared packages to PyPI

RELEASE PROCEDURE

  1. Document the release correctly in README.rst.
  2. Test the project build with the latest available setuptools project and update the ez_setup.py setuptools installation script as needed.
  • Chosen setuptools version needs to support all the Python interpreter versions supported by our project.
  • setuptools version 2.0 dropped support for Python 2.4 & 2.5.
  1. Version identification.
  • Remove the (development) suffix for official release builds.
  1. Tag in Hg.
  • Name the tag like release-<version-info>, e.g. release-0.5.
  1. Prepare official releases based only on tagged commits.
  • Official releases should always be prepared based on tagged revisions with no local changes in the used sandbox.
  • Prepare source distribution packages (both .zip & .tar.bz2 formats), register the new release at PyPI and upload the prepared source packages.
    • Run setup.py sdist register upload.
  • Upload the prepared source package to the project site.
    • Use the BitBucket project web interface.
  1. Next development version identification.
  • Bump up the forked project version counter.
  • Add back the (development) suffix, e.g. as in 0.5 (development).
  1. Notify whomever the new release might concern.

DEVELOPMENT & TESTING ENVIRONMENT

In all command-line examples below pyX, pyXY & pyXYZ represent a Python interpreter executable for a specific Python version X, X.Y & X.Y.Z respectively.

Testing environment is generally set up as follows:

  1. Install Python.
  2. Install setuptools (using setup_ez.py or from the source distribution).
  3. Install pip using setuptools (optional).
  4. Install pytest using pip or setuptools.

This should hold for all Python releases except some older ones explicitly listed below.

To run all of the project unit tests with a specific interpreter without additional configuration options run the project's setup.py script with the 'test' parameter and an appropriate Python interpreter. E.g. run any of the following from the top level project folder:

py243 setup.py test
py27 setup.py test
py3 setup.py test

To have more control over the test suite run it from the top level project folder using pytest, e.g.

  • Using a Python 2.x interpreter:

    py27 -m pytest

  • Using a Python 3.x interpreter:

    py33 setup.py build & py33 -m pytest build

This way you can specify additional pytest options on the command-line.

In both cases, tests run using Python interpreter version 3.x will be run in the build folder constructed by the setup.py script running the py2to3 tool on the project's sources. You might need to manually remove the build folder in order to have sources in it regenerated when wanting to run the test suite using a different Python 3.x interpreter version, as those sources are regenerated based solely on the original & processed source file timestamp information and not the Python version used to process them.

See the pytest documentation for a detailed list of available command-line options. Some interesting ones:

-l show local variable state in tracebacks
--tb=short shorter traceback information for each failure
-x stop on first failure

On Windows you might have a problem setting up multiple parallel Python interpreter versions in case they match their major and minor version numbers, e.g. Python 2.4.3 & 2.4.4. In those cases, standard Windows installer will automatically remove the previous installation instead of simply adding a new one. In order to achieve such parallel setup we suggest the following steps:

  1. Install the first version in a dummy folder, and do so for the current user only.
  2. Copy the dummy target folder to the desired folder for the first installation, e.g. Python243.
  3. Uninstall the original version.
  4. Set up a shortcut or a batch script (e.g. py243.cmd) for running this interpreter without having to have it added to the system path.
  5. Repeat the steps for the second installation.

Installing Python for the current user only is necessary in order to make Python install all of its files into the target folder and not move some of them into shared system folders.

Note that this will leave you without start menu or registry entries for these Python installations. Registry entries should be needed only if you want to run some external Python package installation tool requiring those entries in order to determine where to install its package data. In that case you can set those entries manually, e.g. by using a script similar to the one found at http://nedbatchelder.com/blog/201007/installing_python_packages_from_windows_installers_into.html.

Notes on setting up specific Python versions

Python 2.4.3

  • Does not work with HTTPS links so you can not use the Python package index directly, since it, at some point, switched to using HTTPS links only.

    • You could potentially work around this problem by somehow mapping its https: links to http: ones or download its link page manually, locally modify it to contain http: links and then use that download link page instead of the default downloaded one.

    • An alternative and tested solution is to install into Python 2.4.4 and then copy all the related site-packages entries from that installation into this one.

      • For pytest 2.4.1 with py library version 1.4.15 the following data was copied.

        • Folders:

          _pytest
argparse-1.2.1-py2.4.egg-info
py
py-1.4.15-py2.4.egg-info
pytest-2.4.1-py2.4.egg-info

        • Files:

          argparse.py
pytest.py

Python 2.4.x

  • Can not run pip using python.exe -m pip. Workaround is to use one of the pip startup scripts found in the Python installation's Scripts folder or to use the following invocation:

    py244 -c "import pip;pip.main()" <regular-pip-options>

  • pip

    • 1.1 - last version supporting Python 2.4.

      • Install using:

        py244 -m easy_install pip==1.1

    • Can not be run using python.exe -m pip.

      • Workaround is to use one of the pip startup scripts found in the Python installations Scripts folder or the following invocation:

        py244 -c "import pip;pip.main()" <regular-pip-options>

  • pytest

    • 2.4.1 - last version supporting Python 2.4.

      • Install using:

        py244 -c "import pip;pip.main()" install pytest==2.4.1

    • Depends on the py package library version >= 1.4.16. However those versions fail to install with Python 2.4 (tested up to and including 1.4.18).

      • May be worked around by forcing pytest to use an older py package library version:

        1. Run the pytest installation using pip. It will fail but it will install everything needed except the py package library.

        2. Install the py package library version 1.4.15 using:

          py244 -c "import pip;pip.main()" install py==1.4.15

      • If worked around by using the py 1.4.15 library version, pytest's startup scripts will not work (as they explicitly check pytest's package dependencies), but pytest can still be run using:

        py244 -m pytest <regular-pytest-options>

STANDARDS CONFORMANCE

There seems to be no complete standards conformance overview for the suds project. This section contains just some related notes, taken down while hacking on this project. As more related information is uncovered, it should be added here as well, and eventually this whole section should be moved to the project's user documentation.

Interpreting message parts defined by a WSDL schema

  • Each message part is interpreted as a single parameter.
    • What we refer to here as a 'parameter' may not necessarily correspond 1-1 to a Python function argument passed when using the suds library's Python function interface for invoking web service operations. In some cases suds may attempt to make the Python function interfaces more intuitive to the user by automatically unwrapping a parameter as defined inside a WSDL schema into multiple Python function arguments.
  • In order to achieve interoperability with existing software 'in the wild', suds does not fully conform to the WSDL 1.1 specification with regard as to how message parts are mapped to input data contained in SOAP XML web service operation invocation request documents.
    • WSDL 1.1 standard states:
      • 2.3.1 Message Parts.
        • A message may have message parts referencing either an element or a type defined in the WSDL's XSD schema.
        • If a message has a message part referencing a type defined in the WSDL's XSD schema, then that must be its only message part.
      • 3.5 soap:body.
        • If using document/literal binding and a message has a message part referencing a type defined in the WSDL's XSD schema then that part becomes the schema type of the enclosing SOAP envelope Body element.
    • Suds supports multiple message parts, each of which may be related either to an element or a type.
    • Suds uses message parts related to types, as if they were related to an element, using the message part name as the representing XML element name in the constructed related SOAP XML web service operation invocation request document.
    • WS-I Basic Profile 1.1 standard explicitly avoids the issue by stating the following:
      • R2204 - A document/literal binding in a DESCRIPTION MUST refer, in each of its soapbind:body element(s), only to wsdl:part element(s) that have been defined using the element attribute.
    • Rationale.
      • No other software has been encountered implementing the exact functionality specified in the WSDL 1.1 standard.
      • Already done in the original suds implementation.
      • Example software whose implementation matches our own.

WSDL XSD schema interpretation

  • minOccurs/maxOccurs attributes on all, choice & sequence schema elements are ignored.
    • Rationale.
      • Already done in the original suds implementation.
    • Extra notes.
      • SoapUI (tested with version 4.6.1).
        • For all, choice & sequence schema elements with their minOccurs attribute set to "0", does not explicitly mark elements found in such containers as optional.
  • Supports sending multiple same-named web service operation parameters, but only if they are specified next to each other in the constructed web service operation invocation request document.
    • Done by passing a list or tuple of such values to the suds constructed Python function representing the web service operation in question.
    • Rationale.
      • Already done in the original suds implementation.
    • Extra notes.
      • Such same-named values break other web service related tools as well, e.g. WSDL analyzer & invoker at http://www.validwsdl.com.