mirror of
https://github.com/love2d/love.git
synced 2026-08-14 10:13:46 +02:00
ell
137 lines
6.6 KiB
Markdown
137 lines
6.6 KiB
Markdown
# Lövetest
|
|
Basic testing suite for the [Löve](https://github.com/love2d/love) APIs, based off of [this issue](https://github.com/love2d/love/issues/1745).
|
|
|
|
Currently written for [Löve 12](https://github.com/love2d/love/tree/12.0-development), which is still in development.
|
|
|
|
---
|
|
|
|
## Features
|
|
- [x] Simple pass/fail tests in Lua with minimal setup
|
|
- [x] Ability to run all tests with a simple command
|
|
- [x] Ability to see how many tests are passing/failing
|
|
- [x] Ability to run a subset of tests
|
|
- [x] Ability to easily run an individual test
|
|
- [x] Ability to see all visual results at a glance
|
|
- [x] Automatic testing that happens after every commit
|
|
- [x] No platform-specific dependencies / scripts
|
|
|
|
---
|
|
|
|
## Coverage
|
|
This is the status of all module tests currently.
|
|
| Module | Done | Todo | Skip |
|
|
| ----------------- | ---- | ---- | ---- |
|
|
| 🟢 audio | 28 | 0 | 0 |
|
|
| 🟢 data | 12 | 0 | 0 |
|
|
| 🟡 event | 4 | 1 | 1 |
|
|
| 🟢 filesystem | 28 | 0 | 2 |
|
|
| 🟢 font | 7 | 0 | 0 |
|
|
| 🟡 graphics | 93 | 14 | 1 |
|
|
| 🟢 image | 5 | 0 | 0 |
|
|
| 🟢 math | 20 | 0 | 0 |
|
|
| 🟡 physics | 22 | 6 | 0 |
|
|
| 🟢 sound | 4 | 0 | 0 |
|
|
| 🟢 system | 6 | 0 | 2 |
|
|
| 🟢 thread | 5 | 0 | 0 |
|
|
| 🟢 timer | 6 | 0 | 0 |
|
|
| 🟢 video | 2 | 0 | 0 |
|
|
| 🟢 window | 34 | 0 | 2 |
|
|
|
|
> The following modules are not covered as we can't really emulate input nicely:
|
|
> `joystick`, `keyboard`, `mouse`, and `touch`
|
|
|
|
---
|
|
|
|
## Running Tests
|
|
The testsuite aims to keep things as simple as possible, and just runs all the tests inside Löve to match how they'd be used by developers in-engine.
|
|
To run the tests, download the repo and then run the main.lua as you would a Löve game, i.e:
|
|
|
|
WINDOWS: `& 'c:\Program Files\LOVE\love.exe' PATH_TO_TESTING_FOLDER --console`
|
|
MACOS: `/Applications/love.app/Contents/MacOS/love PATH_TO_TESTING_FOLDER`
|
|
LINUX: `./love.AppImage PATH_TO_TESTING_FOLDER`
|
|
|
|
By default all tests will be run for all modules.
|
|
If you want to specify a module/s you can use:
|
|
`--runSpecificModules filesystem,audio`
|
|
If you want to specify only 1 specific method only you can use:
|
|
`--runSpecificMethod filesystem write`
|
|
|
|
All results will be printed in the console per method as PASS, FAIL, or SKIP with total assertions met on a module level and overall level.
|
|
|
|
When finished, the following files will be generated in the `/output` directory with a summary of the test results:
|
|
- an `XML` file in the style of [JUnit XML](https://www.ibm.com/docs/en/developer-for-zos/14.1?topic=formats-junit-xml-format)
|
|
- a `HTML` file that shows any visual test results
|
|
- a `Markdown` file for use with [this github action](https://github.com/ellraiser/love-test-report)
|
|
> An example of all types of output can be found in the `/examples`
|
|
> The visual results of any graphic tests can be found in `/output/actual`
|
|
|
|
---
|
|
|
|
## Architecture
|
|
Each method and object has it's own test method written in `/tests` under the matching module name.
|
|
|
|
When you run the tests, a single TestSuite object is created which handles the progress + totals for all the tests.
|
|
Each module has a TestModule object created, and each test method has a TestMethod object created which keeps track of assertions for that method. You can currently do the following assertions:
|
|
- **assertNotNil**(value)
|
|
- **assertEquals**(expected, actual, label)
|
|
- **assertNotEquals**(expected, actual, label)
|
|
- **assertRange**(actual, min, max, label)
|
|
- **assertMatch**({option1, option2, option3 ...}, actual, label)
|
|
- **assertGreaterEqual**(expected, actual, label)
|
|
- **assertLessEqual**(expected, actual, label)
|
|
- **assertObject**(table)
|
|
- **assertPixels**(imgdata, pixeltable, label)
|
|
- **assertCoords**(expected, actual, label)
|
|
|
|
Example test method:
|
|
```lua
|
|
-- love.filesystem.read test method
|
|
-- all methods should be put under love.test.MODULE.METHOD, matching the API
|
|
love.test.filesystem.read = function(test)
|
|
-- setup any data needed then run any asserts using the passed test object
|
|
local content, size = love.filesystem.read('resources/test.txt')
|
|
test:assertNotNil(content)
|
|
test:assertEquals('helloworld', content, 'check content match')
|
|
test:assertEquals(10, size, 'check size match')
|
|
content, size = love.filesystem.read('resources/test.txt', 5)
|
|
test:assertNotNil(content)
|
|
test:assertEquals('hello', content, 'check content match')
|
|
test:assertEquals(5, size, 'check size match')
|
|
-- no need to return anything or cleanup, GCC is called after each method
|
|
end
|
|
```
|
|
|
|
Each test is run inside it's own coroutine - you can use `test:waitFrames(frames)` to pause the test for a small period if you need to check things that won't happen for a few seconds.
|
|
|
|
After each test method is ran, the assertions are totalled up, printed, and we move onto the next method! Once all methods in the suite are run a total pass/fail/skip is given for that module and we move onto the next module (if any)
|
|
|
|
For sanity-checking, if it's currently not covered or it's not possible to test the method we can set the test to be skipped with `test:skipTest(reason)` - this way we still see the method listed in the test output without it affected the pass/fail totals
|
|
|
|
---
|
|
|
|
## Todo
|
|
Things still left to do:
|
|
- [ ] physics.Body, physics.Contact, physics.Fixture,
|
|
physics.Joint, physics.Shape, physics.World
|
|
- [ ] graphics.Canvas, graphics.Font, graphics.Image, graphics.Mesh,
|
|
graphics.ParticleSystem, graphics.Quad, graphics.Shader,
|
|
graphics.SpriteBatch, graphics.Text, graphics.Texture, graphics.Video
|
|
- [ ] event.wait
|
|
- [ ] graphics.present
|
|
- [ ] graphics.drawInstanced
|
|
- [ ] graphics.setDepthMode (needs actual graphical comparison if possible)
|
|
- [ ] graphics.setFrontFaceWinding (needs actual graphical comparison if possible)
|
|
- [ ] graphics.setMeshCullMode (needs actual graphical comparison if possible)
|
|
- [ ] @deprecated setStencilTest (use setStencilMode)
|
|
- [ ] @deprecated physics methods (sasha changes)
|
|
- [ ] check 12.0 wiki page for new methods
|
|
- [ ] need a platform: format table somewhere for compressed formats (i.e. DXT not supported)
|
|
- [ ] ideally graphics.isCompressed should have an example of all compressed files love can take
|
|
|
|
---
|
|
|
|
## Future Goals
|
|
- [ ] Tests can compare visual results to a reference image (partially done)
|
|
- [ ] Ability to test loading different combinations of modules
|
|
- [ ] Performance tests
|