Testing

Automated game testing From rails Jump to: navigation, search Contents

1 Usage Instructions
    1.1 Running the tests
    1.2 Adding test(s)
    1.3 Resetting of test(s)
2 Implementation Details
    2.1 Limitations
    2.2 Avoid pitfalls
    2.3 More Details
        2.3.1 Sequence of CashHolders
        2.3.2 Static variables
        2.3.3 ReportBuffer
        2.3.4 Local Text
3 List of games
    3.1 Real games
    3.2 Test Games
    3.3 Bug Fix Games

Usage Instructions Running the tests

The tests are built dynamically from the TestGameBuilder class. It parameterizes the TestGame class by traversing the test data directory and searching for rails game files.

Via Eclipse/TestRunner: Select the TestGameBuilder class in the package explorer and run as JUnit Test. Results are shown in the JUnit Pane.

Via Ant; Select build.xml and run as ant build (Dialog). Select the target TestGames and add the Junit jar to the ant classpath. The html report is generated in directory test/report. See the attachment for an example. Adding test(s)

Every (valid) rails save file can create an additional test. Simply add the game file (with the extension .rails) in the path below test/data. The directory structure there implies the nested structure of the TestRunner test suites and is reflected in the test names of the report generated with ant. Resetting of test(s)

Sometimes a code change might change the game reports of some or all tests. If you are sure that this is the only reason why those tests fail, you can recreate new game reports for each test by running the main method of the TestGameBuilder. A file dialog will allow to select individual files or directories. In the latter case all tests in and below that directory are re-initialized. Implementation Details Limitations

What the tests will never detect:

Changes to the GUI classes. 

Changes to the undo mechanism. 

Actions which do not generate any entry in the game report. 

Avoid pitfalls

Important for future Rails coding:

Avoid static class variables. 

Always define a unique ordering for messages added to the ReportBuffer. Otherwise tests will randomly fail. 

More Details Sequence of CashHolders

To allow testing all items of the game report require a defined sequence. The revenue distribution to cashholders deviated from this. Thus I had to define an ordering for all classes that implement the cashholder interface and then define a unique sequence for the combined set of all cashholders.

Company class: Implements Comparable, ordering based first on the companytype name, then on the name itself.

Player: Already sorted by wealth, added names as tie breakers.

Class SequenceUtil defines the ordering Players, Companies, Bank in method sortCashHolders Static variables

There are some classes in Rails that have static variables that are in reality game specific. I changed those to instance variables.

A list of those:

ComponentManager directories
OperatingRound step, OperatingRound_1856 step
TileManager directories 

Two classes (SpecialProperty and Token) keep track of instance ids in static variables. Those have to be reinitialized at the start of each game, this is done from Game.setup(). One class (CashCorrectionManager) uses the Singleton pattern, but keeps gameManager reference. Updates reference after change of gameManager instance. Remark: All of the cases above have to be refactored to allow rails running several games simultaneously. ReportBuffer

The reportBuffer is implemented as List instead as a (long) String. As the game report file is read line-by-line, no message send to the reportbuffer should contain a manual line feed "\n". Instead use several messages. Local Text

An special local code "te_st0" is added that does not resolve to localization text, but uses the keys in the program code instead. Arguments are simply added in sequence, separated by commas. Thus changes to the text localizations will never trigger failed tests. List of games Real games Game Id Version State #Players Round Notes Date 1830 A 1.4.2 out of money 5 OR 8.3 Coalfields&Reading 28/07/2011 1856 A 1.4.1 out of money 5 OR 8.3 21/05/2011 1889 A 1.2 in progress 3 OR 6.2 17/03/2010 1889 B 1.4.2 out of money 4 OR7.3 17/07/2011 1889 C 1.4.1+ out of money 3 OR 9.3 12/03/2011 18AL A 1.1.3 bankrupt 4 OR 6.2 18/03/2010 18GA A 1.4.1+ out of money 4 OR 7.1 05/10/2011 18TN A 1.5.4 bankrupt 4 OR 7.1 01/03/2012 Test Games Game Name Version Scenario Test Owner Notes Date Bug Fix Games Game Name Version Bug Case But Reported Patch by Commit Notes Date 1856 tokens 1.5 THB home token cannot be layed Bill Rosgen Stefan Frey 4ebc90 29/09/2011 1830 sellPresidency 1.6.0 in specific cases presidency cannot be sold Bill Probst Erik Vos a26c98 bug was forwarded 30/01/2012