diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index ee0121242..787bf5b4a 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -234,7 +234,7 @@ jobs: name: love-macos path: love-macos.zip - name: Run All Tests - run: love-macos/love.app testing --runAllTests + run: love-macos/love.app/Contents/MacOS/love testing --runAllTests - name: Test Report uses: dorny/test-reporter@v1 with: diff --git a/.gitignore b/.gitignore index a8b316a2d..5bc5b9401 100644 --- a/.gitignore +++ b/.gitignore @@ -68,4 +68,6 @@ stamp-h1 /src/love /src/tags .vs/ -.vscode/ \ No newline at end of file +.vscode/ +/testing/output/*.xml +/testing/output/*.html diff --git a/testing/classes/TestSuite.lua b/testing/classes/TestSuite.lua index 9552b5aee..012947771 100644 --- a/testing/classes/TestSuite.lua +++ b/testing/classes/TestSuite.lua @@ -144,9 +144,11 @@ TestSuite = { '
  • ' .. finaltime .. 's


    • ' -- @TODO use mountFullPath to write output to src? - love.filesystem.createDirectory('output') - love.filesystem.write('output/' .. self.output .. '.xml', xml .. self.xml .. '') - love.filesystem.write('output/' .. self.output .. '.html', html .. self.html .. '') + love.filesystem.mountFullPath(love.filesystem.getSource() .. "/output", "tempoutput", "readwrite") + love.filesystem.remove('tempoutput/' .. self.output .. '.xml') + love.filesystem.remove('tempoutput/' .. self.output .. '.html') + love.filesystem.write('tempoutput/' .. self.output .. '.xml', xml .. self.xml .. '') + love.filesystem.write('tempoutput/' .. self.output .. '.html', html .. self.html .. '') self.module:log('grey', '\nFINISHED - ' .. finaltime .. 's\n') local failedcol = '\27[31m' diff --git a/testing/output/readme.md b/testing/output/readme.md new file mode 100644 index 000000000..1a1059a7b --- /dev/null +++ b/testing/output/readme.md @@ -0,0 +1,2 @@ +# Testing Output +Any tests run will output an XML and HTML file here, assuming the tests are run with readwrite permissions for this repo \ No newline at end of file diff --git a/testing/readme.md b/testing/readme.md index 4269e4447..a156d2675 100644 --- a/testing/readme.md +++ b/testing/readme.md @@ -34,9 +34,9 @@ If you want to specify only 1 specific method only you can use: All results will be printed in the console per method as PASS, FAIL, or SKIP with total assertions met on a module level and overall level. -An `XML` file in the style of [JUnit XML](https://www.ibm.com/docs/en/developer-for-zos/14.1?topic=formats-junit-xml-format) will be generated in your save directory, along with a `HTML` file with a summary of all tests (including visuals for love.graphics tests). +An `XML` file in the style of [JUnit XML](https://www.ibm.com/docs/en/developer-for-zos/14.1?topic=formats-junit-xml-format) will be generated in the `/output` directory, along with a `HTML` file with a summary of all tests (including visuals for love.graphics tests) - you will need to make sure the command is run with read/write permissions for the source directory. > Note that this can only be viewed properly locally as the generated images are written to the save directory. -> An example of both types of output can be found in the `/output` folder +> An example of both types of output can be found in the `/examples` folder --- @@ -45,6 +45,7 @@ Each method has it's own test method written in `/tests` under the matching modu When you run the tests, a single TestSuite object is created which handles the progress + totals for all the tests. Each module has a TestModule object created, and each test method has a TestMethod object created which keeps track of assertions for that method. You can currently do the following assertions: +- **assertNotNil**(value) - **assertEquals**(expected, actual) - **assertNotEquals**(expected, actual) - **assertRange**(actual, min, max) @@ -60,14 +61,14 @@ Example test method: love.test.filesystem.read = function(test) -- setup any data needed then run any asserts using the passed test object local content, size = love.filesystem.read('resources/test.txt') - test:assertNotEquals(nil, content, 'check not nil') + test:assertNotNil(content) test:assertEquals('helloworld', content, 'check content match') test:assertEquals(10, size, 'check size match') content, size = love.filesystem.read('resources/test.txt', 5) - test:assertNotEquals(nil, content, 'check not nil') + test:assertNotNil(content) test:assertEquals('hello', content, 'check content match') test:assertEquals(5, size, 'check size match') - -- no need to return anything just cleanup any objs if needed + -- no need to return anything or cleanup, GCC is called after each method end ``` diff --git a/testing/todo.md b/testing/todo.md index 969cbf547..86fc2b105 100644 --- a/testing/todo.md +++ b/testing/todo.md @@ -2,25 +2,24 @@ # v0.2 -## Changed -- Added tests for all obj creation, transformation, window + system info graphics methods -- Added half the state methods for graphics + added placeholders for missing drawing methods -- Added TestMethod:assertNotNil() for quick nil checking -- Added time total to the end of each module summary in console log to match file output - -- Removed a bunch of unessecary nil checks -- Removed :release() from test methods, collectgarbage("collect") is called between methods instead - -- Renamed /output to /examples to avoid confusion - -- Replaced love.filesystem.newFile with love.filesystem.openFile -- Replaced love.math.noise with love.math.perlinNoise / love.math.simplexNoise - -- Fixed newGearJoint throwing an error in 12 as body needs to be dynamic not static now - -- Some general cleanup, incl. better comments and time format in file output - ## Todo -- graphics state methods -- graphics drawing methods +- finish graphics state methods +- start graphics drawing methods +- start object methods +- look into XML reader github actions to display results in readme? + dorny/test-reporter@v1.6.0 seems to do it + would need to run the tests first then could use it like: + + - name: Run Tests + - run: PATH_TO_BUILT_APP ./testing --runAllTests + - name: Test Report + uses: dorny/test-reporter@v1 + with: + name: Test Output + path: output/*.xml + reporter: jest-junit + + and d. check format: https://github.com/testmoapp/junitxml + + - need a platform: format table somewhere for compressed formats (i.e. DXT not supported)