Functional Tests ================ So far all the testing we've done has been *unit* testing, where we carefully isolate some piece of code and make assertions about how it (and only it) works. It's easy to get yourself into a situation where you mock yourself into placation, and the code doesn't actually work anymore when you glue the pieces together even though all the unit tests pass. To avoid this situation, there are functional (or "system") tests. Adding a UI to Our Portfolio ---------------------------- Our portfolio code is, right now, essentially a *library*. It is a bit of code that is meant to be imported and used by other people. Let's give it a UI and turn it in to something that normal people might see. In other words, let's write an application that uses our Portfolio object. Install Pyramid, pyramid_chameleon, waitress, and selenium: .. code-block:: bash $ pip install pyramid pyramid_chameleon waitress selenium Add a ``views.py`` to our project (next to portfolio.py): .. code-block:: python from pyramid.view import view_config from .portfolio import Portfolio @view_config(renderer='index.pt') def index(request): p = Portfolio() p.buy('IBM', 100, 176.48) p.buy('HPQ', 100, 36.15) return {'portfolio':p} Add an ``index.pt`` to our project (next to views.py): .. code-block:: html Change our ``__init__.py`` so that it looks like this: .. code-block:: python from waitress import serve from pyramid.config import Configurator def main(): """Entry point for the application script""" config = Configurator() config.include('pyramid_chameleon') config.scan() app = config.make_wsgi_app() serve(app) If we rerun ``setup.py develop``, and run ``sample``, we'll see something like: .. code-block:: text $ sample serving on http://0.0.0.0:8080 We have now turned our library into an application. The application shows each stock's name, the number of shares we own, its buying price, and its current value, one per line. We can make assertions about this output using ``selenium``, a functional testing system. Adding Functional Tests ----------------------- Add a directory named ``functests`` to the ``sample`` project (next to ``tests``). Add a file named ``__init__.py`` within it. Give it these contents: .. code-block:: python # -*- coding: utf-8 -*- import unittest browser = None # we create a global browser object for speed reasons. # not ideal.. it breaks test isolation. def setUpModule(): from selenium.webdriver import Firefox global browser browser = Firefox() return browser def tearDownModule(): browser.quit() class PageTests(unittest.TestCase): def test_render_default(self): browser.get('http://localhost:8080/') self.assertTrue('IBM 100 176.48' in browser.page_source) Run ``nosetests sample.functests``: .. code-block:: bash $ nosetests sample.functests . ---------------------------------------------------------------------- Ran 1 test in 4.400s OK When you run these tests you will see the Firefox browser launched on your system briefly. Selenium is a system that executes browsers to ensure that web applications are operating properly. We are using the ``nosetests`` script that is installed with ``nose`` for convenience. We could have changed things around such that ``setup.py nosetests`` executed the functional tests too, but we explicitly don't want them to run when we run unit tests because when lots of test cases are added, they're too slow, and people don't run slow test suites. Improving the Application ------------------------- The template we wrote demonstrates that our Portfolio API is a little lame. Let's improve it and evolve the template along with it, as well as adding a few more tests. A Real World Test Suite ----------------------- Let's take a look at the Deformdemo application to get a sense of a real-world test suite for an application. https://github.com/Pylons/deformdemo Follow the instructions in its README.rst to be able to run its test suite. Do this is a *new shell*, unrelated to the portfolio demo. (Where it says ``virtualenv2.7`` instead use ``virtualenv -ppy27``). Take a look at the tests defined in ``deformdemo/test.py`` (https://github.com/Pylons/deformdemo/blob/master/deformdemo/test.py). Note that you can: - Click buttons. - Fill in form fields. - Make assertions about the contents of HTML elements. - Find specific elements using XPath, CSS, or by unique HTML id. - Send keystrokes. - Test the result of executing JavaScript (such as drag and drop). Selenium can be driven using Python, and I tend to use it that way. It can also be driven via a UI (Selenium-UI), or by bindings for most other dynamic languages. .. sidebar:: What About Integration Tests? Integration tests, to me, are the worst of both worlds. They have all the fragility of unit tests and all of the slowness of system tests. I avoid them where possible. Sometimes they're required such as when you test code that integrates with something that, for practical reasons, cannot be mocked, such as a database. But otherwise, they are not very useful, IMO.