mirror of
https://github.com/love2d/love.git
synced 2026-08-16 08:11:02 +02:00
ell
137 lines
6.5 KiB
Markdown
137 lines
6.5 KiB
Markdown
# löve.test
|
|
Basic testing suite for the löve APIs, based off of [this issue](https://github.com/love2d/love/issues/1745)
|
|
|
|
Currently written for löve 12
|
|
|
|
---
|
|
|
|
## Primary Goals
|
|
- [x] Simple pass/fail tests in Lua with minimal setup
|
|
- [x] Ability to run all tests with a simple command.
|
|
- [x] Ability to see how many tests are passing/failing
|
|
- [x] No platform-specific dependencies / scripts
|
|
- [x] Ability to run a subset of tests
|
|
- [x] Ability to easily run an individual test.
|
|
|
|
---
|
|
|
|
## Running Tests
|
|
The initial pass is to keep things as simple as possible, and just run all the tests inside Löve to match how they'd be used by developers in-engine.
|
|
To run the tests, download the repo and then run the main.lua as you would a löve game, i.e:
|
|
|
|
WINDOWS: `& 'c:\Program Files\LOVE\love.exe' PATH_TO_TESTING_FOLDER --console`
|
|
MACOS: `/Applications/love.app/Contents/MacOS/love PATH_TO_TESTING_FOLDER`
|
|
|
|
By default all tests will be run for all modules.
|
|
|
|
If you want to specify a module you can add:
|
|
`--runSpecificModules filesystem`
|
|
For multiple modules, provide a comma seperate list:
|
|
`--runSpecificModules filesystem,audio,data"`
|
|
|
|
If you want to specify only 1 specific method only you can use:
|
|
`--runSpecificMethod filesystem write`
|
|
|
|
All results will be printed in the console per method as PASS, FAIL, or SKIP with total assertions met on a module level and overall level.
|
|
|
|
An `XML` file in the style of [JUnit XML](https://www.ibm.com/docs/en/developer-for-zos/14.1?topic=formats-junit-xml-format) will be generated in the `/output` directory, along with a `HTML` file with a summary of all tests (including visuals for love.graphics tests) - you will need to make sure the command is run with read/write permissions for the source directory.
|
|
> Note that this can only be viewed properly locally as the generated images are written to the save directory.
|
|
> An example of both types of output can be found in the `/examples` folder
|
|
|
|
---
|
|
|
|
## Architecture
|
|
Each method has it's own test method written in `/tests` under the matching module name.
|
|
|
|
When you run the tests, a single TestSuite object is created which handles the progress + totals for all the tests.
|
|
Each module has a TestModule object created, and each test method has a TestMethod object created which keeps track of assertions for that method. You can currently do the following assertions:
|
|
- **assertNotNil**(value)
|
|
- **assertEquals**(expected, actual)
|
|
- **assertNotEquals**(expected, actual)
|
|
- **assertRange**(actual, min, max)
|
|
- **assertMatch**({option1, option2, option3 ...}, actual)
|
|
- **assertGreaterEqual**(expected, actual)
|
|
- **assertLessEqual**(expected, actual)
|
|
- **assertObject**(table)
|
|
|
|
Example test method:
|
|
```lua
|
|
-- love.filesystem.read test method
|
|
-- all methods should be put under love.test.MODULE.METHOD, matching the API
|
|
love.test.filesystem.read = function(test)
|
|
-- setup any data needed then run any asserts using the passed test object
|
|
local content, size = love.filesystem.read('resources/test.txt')
|
|
test:assertNotNil(content)
|
|
test:assertEquals('helloworld', content, 'check content match')
|
|
test:assertEquals(10, size, 'check size match')
|
|
content, size = love.filesystem.read('resources/test.txt', 5)
|
|
test:assertNotNil(content)
|
|
test:assertEquals('hello', content, 'check content match')
|
|
test:assertEquals(5, size, 'check size match')
|
|
-- no need to return anything or cleanup, GCC is called after each method
|
|
end
|
|
```
|
|
|
|
After each test method is ran, the assertions are totalled up, printed, and we move onto the next method! Once all methods in the suite are run a total pass/fail/skip is given for that module and we move onto the next module (if any)
|
|
|
|
For sanity-checking, if it's currently not covered or we're not sure how to test yet we can set the test to be skipped with `test:skipTest(reason)` - this way we still see the method listed in the tests without it affected the pass/fail totals
|
|
|
|
---
|
|
|
|
## Coverage
|
|
This is the status of all module tests currently.
|
|
"objects" is a special module to cover any object specific tests, i.e. testing a File object functions as expected
|
|
```lua
|
|
-- [x] audio 26 PASSED | 0 FAILED | 0 SKIPPED
|
|
-- [x] data 7 PASSED | 0 FAILED | 3 SKIPPED [SEE BELOW]
|
|
-- [x] event 4 PASSED | 0 FAILED | 2 SKIPPED [SEE BELOW]
|
|
-- [x] filesystem 27 PASSED | 0 FAILED | 2 SKIPPED
|
|
-- [x] font 4 PASSED | 0 FAILED | 1 SKIPPED [SEE BELOW]
|
|
-- [ ] graphics 65 PASSED | 0 FAILED | 31 SKIPPED [SEE BELOW]
|
|
-- [x] image 3 PASSED | 0 FAILED | 0 SKIPPED
|
|
-- [x] math 17 PASSED | 0 FAILED | 0 SKIPPED
|
|
-- [x] physics 22 PASSED | 0 FAILED | 0 SKIPPED
|
|
-- [x] sound 2 PASSED | 0 FAILED | 0 SKIPPED
|
|
-- [x] system 6 PASSED | 0 FAILED | 2 SKIPPED
|
|
-- [x] thread 3 PASSED | 0 FAILED | 0 SKIPPED
|
|
-- [x] timer 6 PASSED | 0 FAILED | 0 SKIPPED
|
|
-- [x] video 1 PASSED | 0 FAILED | 0 SKIPPED
|
|
-- [x] window 32 PASSED | 2 FAILED | 2 SKIPPED [SEE BELOW]
|
|
|
|
-- [ ] objects STILL TO BE DONE
|
|
--------------------------------------------------------------------------------
|
|
-- [x] totals 226 PASSED | 4 FAILED | 43 SKIPPED
|
|
```
|
|
|
|
The following modules are not covered as we can't really emulate input nicely:
|
|
`joystick`, `keyboard`, `mouse`, and `touch`
|
|
|
|
---
|
|
|
|
## Todo / Skipped
|
|
Modules with some small bits needed or needing sense checking:
|
|
- **love.data** - packing methods need writing cos i dont really get what they are
|
|
- **love.event** - love.event.wait or love.event.pump need writing if possible I dunno how to check
|
|
- **love.font** - newBMFontRasterizer() wiki entry is wrong so not sure whats expected
|
|
- **love.graphics** - still need to do tests for the drawing and state methods
|
|
- **love.image** - ideally isCompressed should have an example of all compressed files love can take
|
|
- **love.math** - linearToGamma + gammaToLinear using direct formulas don't get same value back
|
|
- **love.objects** - not started properly yet
|
|
|
|
---
|
|
|
|
## Failures
|
|
- **love.window.isMaximized()** - returns false after calling love.window.maximize?
|
|
- **love.window.maximize()** - same as above
|
|
|
|
---
|
|
|
|
## Stretch Goals
|
|
- [ ] Tests can compare visual results to a reference image
|
|
- [ ] Ability to see all visual results at a glance
|
|
- [ ] Automatic testing that happens after every commit
|
|
- [ ] Ability to test loading different combinations of modules
|
|
- [ ] Performance tests
|
|
|
|
There is some unused code in the Test.lua class to add preview vs actual images to the HTML output
|