Here's a tip for testers: you can use a machine to run tests for you. If you know introductory Python, you'll be able to create sets of instructions that a machine will carry out for you. How does this happen? Thank Sugarbot.
Some history: In 2008, Zach Riggle created Sugarbot during Google Summer of Code. His project was mentored by the editor of http://agiletesting.blogspot.org. That set him on a very good foundation. Sugarbot basically works as an activity withinside Sugar, it just happens to play with other Acitivities.
Further (only slightly technical) details after the jump.
Let's have a look at a sample of what it would take to test a generic activity, that we'll call Cookbook
#! /usr/bin/env python2.6
# Testing Cookbook
# derived from http://bit.ly/9G2CnP
sugarActivityName = "Cookbook"
def sugarbot_main(widget):
assert widget['entryBox'].text == 'Recipe'
widget['entryBox'].text = 'New Recipe'
assert widgets['Share with:'].selected == "Private"
widgets['Share with:'].selected = "My Neighborhood"
assert widgets['Share with:'].selected == "My Neighborhood"
What does this code do?
Setting things up is first.
sugarActivityName = "Cookbook"
def sugarbot_main(widgets):
sugarActivityName and sugarbot_main are both required in every Sugarbot script. sugarActivityName tells the bot which activity to open. sugarbot_main is a function that processes every widget available to the bot. The widgets argument is a dictionary (Python's standard implementation of an associative array[http://en.wikipedia.org/wiki/Associative_array]) with
Next comes our first test. :
assert widget['entryBox'].text == 'Recipe'
widget['entryBox'].text = 'New Recipe'
A test is an assert statement with a comparison operator. Actually, a test is an assert statement with something that evaluates to True or False. Therefore, the easiest way to create a test is to use the equality operator (==). These two lines of code check that "Some text" is the text of the widget named entryBox. It then updates that text to 'New Recipe'.
assert widgets['Share with:'].selected == "Private"
widgets['Share with:'].selected = "My Neighborhood"
assert widgets['Share with:'].selected == "My Neighborhood"
These three lines share the Cookbook Activity with the Neighborhood. We first test that the Activity defaults to private, then we test that . Note, to actually test that the activity was tested, we would need unit testing to test the models independently of the Sugarbot process.
Localisation
You'll notice that the tests are currently testing a single locale, en. Perhaps you could take a look at how hard it would be to integrate gettext to make the testing process compatible with international deployments?
Coding style
Experienced computer programmmers will notice the style of coding of the sugarbot_main() method as procedural programming. This is verbose and generally unfashionable. However, this example script is very simple. As you increase the complexity of the scripts, to meet the needs of individual Activities, you should call functions (perhaps you can help create a library of helper functions to make testing easier on everyone). As it stands, procedural programming does indicat the programme's flow well. This makes it easy to see what the bot is doing.
One disadvantage of using assert statements is that tests are not clearly demarcated from one another. This pleaces the burdon on the tester to make sure that she's being clear to the next reader of her script, which could be her in a few months time. I recommend placing empty lines in appropriate places to maintain the readability of the code.
Widget naming
How do testers know the names of the widgets that they are going to test? This is actually where communication with Sugar developers comes in. They are able to set the name of every widget they use with the .name_widget() method. Seem simple! Unfortunately, things are a little more complicated. Because Sugarbot is still not being fully utilised by the Sugar community, there are unnamed widgets in Activities.
Sugarbot does its best with unnamed widgets. Following the rules outlined in the Sugarbot wiki (see resources, below), every widget is given a unique identifier. The burden is then placed on the test script developer to track down that identifier so that things can be tested meaningfully.
The downside
Unfortunately for testers like us, the fully automated testing has not been fully ingrained into the Sugar development process. This has probably led to Zack, who developed the software, to become somewhat fustrated that his effort is stagnating. There have been very few (if any) commits to Sugarbot since late 2008. This means that if we as part of the Sugar community wish to utilise this work, we will probably need to spend some time getting the software up-to-date.
Resources
Summing up
Sugarbot is an underutilised piece of software. I recommend that people who are looking into developing experience in Python & automated testing take a look. There was a significant ammount of effort put in by Zack in 2008, and I'm sure its worth looking into.
tags: technical, software development, testing
