Functional Tests
================
So far all the testing we've done has been *unit* testing, where we carefully
isolate some piece of code and make assertions about how it (and only it)
works.
It's easy to get yourself into a situation where you mock yourself into
placation, and the code doesn't actually work anymore when you glue the pieces
together even though all the unit tests pass. To avoid this situation, there
are functional (or "system") tests.
Adding a UI to Our Portfolio
----------------------------
Our portfolio code is, right now, essentially a *library*. It is a bit of code
that is meant to be imported and used by other people. Let's give it a UI and
turn it in to something that normal people might see. In other words, let's
write an application that uses our Portfolio object.
Install Pyramid, pyramid_chameleon, waitress, and selenium:
.. code-block:: bash
$ pip install pyramid pyramid_chameleon waitress selenium
Add a ``views.py`` to our project (next to portfolio.py):
.. code-block:: python
from pyramid.view import view_config
from .portfolio import Portfolio
@view_config(renderer='index.pt')
def index(request):
p = Portfolio()
p.buy('IBM', 100, 176.48)
p.buy('HPQ', 100, 36.15)
return {'portfolio':p}
Add an ``index.pt`` to our project (next to views.py):
.. code-block:: html
-
${stock[0]} ${stock[1]} ${stock[2]} ${current[stock[0]]}
Change our ``__init__.py`` so that it looks like this:
.. code-block:: python
from waitress import serve
from pyramid.config import Configurator
def main():
"""Entry point for the application script"""
config = Configurator()
config.include('pyramid_chameleon')
config.scan()
app = config.make_wsgi_app()
serve(app)
If we rerun ``setup.py develop``, and run ``sample``, we'll see something like:
.. code-block:: text
$ sample
serving on http://0.0.0.0:8080
We have now turned our library into an application. The application shows each
stock's name, the number of shares we own, its buying price, and its current
value, one per line. We can make assertions about this output using
``selenium``, a functional testing system.
Adding Functional Tests
-----------------------
Add a directory named ``functests`` to the ``sample`` project (next to
``tests``). Add a file named ``__init__.py`` within it. Give it these
contents:
.. code-block:: python
# -*- coding: utf-8 -*-
import unittest
browser = None
# we create a global browser object for speed reasons.
# not ideal.. it breaks test isolation.
def setUpModule():
from selenium.webdriver import Firefox
global browser
browser = Firefox()
return browser
def tearDownModule():
browser.quit()
class PageTests(unittest.TestCase):
def test_render_default(self):
browser.get('http://localhost:8080/')
self.assertTrue('IBM 100 176.48' in browser.page_source)
Run ``nosetests sample.functests``:
.. code-block:: bash
$ nosetests sample.functests
.
----------------------------------------------------------------------
Ran 1 test in 4.400s
OK
When you run these tests you will see the Firefox browser launched on your
system briefly. Selenium is a system that executes browsers to ensure that web
applications are operating properly.
We are using the ``nosetests`` script that is installed with ``nose`` for
convenience. We could have changed things around such that ``setup.py
nosetests`` executed the functional tests too, but we explicitly don't want
them to run when we run unit tests because when lots of test cases are added,
they're too slow, and people don't run slow test suites.
Improving the Application
-------------------------
The template we wrote demonstrates that our Portfolio API is a little lame.
Let's improve it and evolve the template along with it, as well as adding a few
more tests.
A Real World Test Suite
-----------------------
Let's take a look at the Deformdemo application to get a sense of a real-world
test suite for an application. https://github.com/Pylons/deformdemo
Follow the instructions in its README.rst to be able to run its test suite. Do
this is a *new shell*, unrelated to the portfolio demo.
(Where it says ``virtualenv2.7`` instead use ``virtualenv -ppy27``).
Take a look at the tests defined in ``deformdemo/test.py``
(https://github.com/Pylons/deformdemo/blob/master/deformdemo/test.py).
Note that you can:
- Click buttons.
- Fill in form fields.
- Make assertions about the contents of HTML elements.
- Find specific elements using XPath, CSS, or by unique HTML id.
- Send keystrokes.
- Test the result of executing JavaScript (such as drag and drop).
Selenium can be driven using Python, and I tend to use it that way. It can
also be driven via a UI (Selenium-UI), or by bindings for most other dynamic
languages.
.. sidebar:: What About Integration Tests?
Integration tests, to me, are the worst of both worlds. They have all the
fragility of unit tests and all of the slowness of system tests. I avoid
them where possible. Sometimes they're required such as when you test code
that integrates with something that, for practical reasons, cannot be
mocked, such as a database. But otherwise, they are not very useful, IMO.