Update readme: Testing section#282
Conversation
jtojnar
left a comment
There was a problem hiding this comment.
Here is a bit nitpicky dump of a heat wave liquefied brain, sorry.
| The php-mf2 tests are in `/tests/Mf2` and contain many parsing examples, internal parser tests, and regression tests for specific issues over php-mf2’s history. If you do not have a live internet connection, you can exclude tests that depend on it by running `composer tests-no-internet` | ||
|
|
||
| Currently php-mf2 passes the majority of it’s own test case, and a good percentage of the cross-platform tests. Contributors should ALWAYS test against the PHPUnit suite to ensure any changes don’t negatively impact php-mf2, and SHOULD run the cross-platform suite, especially if you’re changing parsing behaviour. | ||
| The other test suite is the cross-platform [microformats/tests](https://github.com/microformats/tests). This has backwards-compatible tests for microformats1, some of which php-mf2 will fail, so it does not automatically run them currently. To run these tests, first run `composer install`, then run `composer microformats-test-mf1`. There are additional composer scripts listed below for running other parts of this test suite. |
There was a problem hiding this comment.
Running composer install is required for all tests since it installs phpunit.
| The php-mf2 tests are in `/tests/Mf2` and contain many parsing examples, internal parser tests, and regression tests for specific issues over php-mf2’s history. If you do not have a live internet connection, you can exclude tests that depend on it by running `composer tests-no-internet` | ||
|
|
||
| Currently php-mf2 passes the majority of it’s own test case, and a good percentage of the cross-platform tests. Contributors should ALWAYS test against the PHPUnit suite to ensure any changes don’t negatively impact php-mf2, and SHOULD run the cross-platform suite, especially if you’re changing parsing behaviour. | ||
| The other test suite is the cross-platform [microformats/tests](https://github.com/microformats/tests). This has backwards-compatible tests for microformats1, some of which php-mf2 will fail, so it does not automatically run them currently. To run these tests, first run `composer install`, then run `composer microformats-test-mf1`. There are additional composer scripts listed below for running other parts of this test suite. |
There was a problem hiding this comment.
I think this makes it sound too much like the mf1 is the only thing this repo contains.
|
|
||
| Currently php-mf2 passes its own test cases and a good percentage of the cross-platform tests. Contributors MUST always test against the PHPUnit suite to ensure any changes don’t negatively impact php-mf2, and SHOULD run the cross-platform suite, especially if you’re changing parsing behaviour. | ||
|
|
||
| **Composer scripts** |
There was a problem hiding this comment.
We might want to use script-descriptions in composer.json instead:
https://getcomposer.org/doc/articles/scripts.md#custom-descriptions
| Currently php-mf2 passes the majority of it’s own test case, and a good percentage of the cross-platform tests. Contributors should ALWAYS test against the PHPUnit suite to ensure any changes don’t negatively impact php-mf2, and SHOULD run the cross-platform suite, especially if you’re changing parsing behaviour. | ||
| The other test suite is the cross-platform [microformats/tests](https://github.com/microformats/tests). This has backwards-compatible tests for microformats1, some of which php-mf2 will fail, so it does not automatically run them currently. To run these tests, first run `composer install`, then run `composer microformats-test-mf1`. There are additional composer scripts listed below for running other parts of this test suite. | ||
|
|
||
| Currently php-mf2 passes its own test cases and a good percentage of the cross-platform tests. Contributors MUST always test against the PHPUnit suite to ensure any changes don’t negatively impact php-mf2, and SHOULD run the cross-platform suite, especially if you’re changing parsing behaviour. |
There was a problem hiding this comment.
It passes all cross-platform tests except for about half (17/39) of the mf1 ones. “Good percentage” undersells it IMO.
| Currently php-mf2 passes the majority of it’s own test case, and a good percentage of the cross-platform tests. Contributors should ALWAYS test against the PHPUnit suite to ensure any changes don’t negatively impact php-mf2, and SHOULD run the cross-platform suite, especially if you’re changing parsing behaviour. | ||
| The other test suite is the cross-platform [microformats/tests](https://github.com/microformats/tests). This has backwards-compatible tests for microformats1, some of which php-mf2 will fail, so it does not automatically run them currently. To run these tests, first run `composer install`, then run `composer microformats-test-mf1`. There are additional composer scripts listed below for running other parts of this test suite. | ||
|
|
||
| Currently php-mf2 passes its own test cases and a good percentage of the cross-platform tests. Contributors MUST always test against the PHPUnit suite to ensure any changes don’t negatively impact php-mf2, and SHOULD run the cross-platform suite, especially if you’re changing parsing behaviour. |
There was a problem hiding this comment.
I do not like that we now have two places with instructions. It feels redundant, especially when CI enforces it.
There was a problem hiding this comment.
I'm not sure what you mean, where's the second place?
| There are two test suites described in more detail below. At a minimum, you should run `composer check-and-tests`, which will run `phpcs`, php-mf2 tests, and microformats/tests (except mf1). | ||
|
|
||
| The other, in `tests/test-suite`, is a custom test harness which hooks up php-mf2 to the cross-platform [microformats test suite](https://github.com/microformats/tests). To run these tests you must first install the tests with `./composer.phar install`. Each test consists of a HTML file and a corresponding JSON file, and the suite can be run with `php ./tests/test-suite/test-suite.php`. | ||
| The php-mf2 tests are in `/tests/Mf2` and contain many parsing examples, internal parser tests, and regression tests for specific issues over php-mf2’s history. If you do not have a live internet connection, you can exclude tests that depend on it by running `composer tests-no-internet` |
There was a problem hiding this comment.
tests are in
/tests/Mf2
Turns out the cross-platform ones have entry point there as well. Not sure how to reformulate it, though.
There was a problem hiding this comment.
I believe it's the only entry point. microformats/tests is a set of HTML and expected JSON outputs, not a set of unit tests, so php-mf2 has to build the unit tests in MicroformatsTestSuiteTest.php
| - same, but exclude tests that require internet: `composer tests-no-internet` | ||
| - microformats/tests suite: | ||
| - run mf1 tests with `composer microformats-tests-mf1` | ||
| - run mf2 tests with `composer microformats-tests-mf2` |
There was a problem hiding this comment.
Actually, thinking about it some more, do we even need an explicit command to run all these different subsets?
But when working on the codebase, a developer probably either wants to run all tests to check they did not introduce a regression, or a specific one, for which we will not be able to have a composer script anyway and they will have to use something like composer tests -- --filter hcard (but that's what PHPUnit docs are for).
This matches what we had previously – we only introduced scripts as needed due to external circumstances like network not being available or stuff not being implemented.
I would even go as far and drop the separate tests-mf1 script to simplify this as much as possible: #284
There was a problem hiding this comment.
Heh, I added those two because you suggested them. :)
It is a goal for php-mf2 to pass all of the backcompat tests in microformats/tests. When we introduced that test suite, it failed a large portion of them, so we chose to exclude them from the default test run. We still want a way to run them, thus the extra script.
| * All content apart from the 'html' key in dictionaries produced by parsing an `e-*` property is not HTML-escaped. For example, `<span class="p-name"><code></span>` will result in `"name": ["<code>"]`. At the very least, HTML-escape all properties before echoing them out in HTML | ||
| * If you’re using the raw HTML content under the 'html' key of dictionaries produced by parsing `e-*` properties, you SHOULD purify the HTML before displaying it to prevent injection of arbitrary code. For PHP we recommend using [HTML Purifier](http://htmlpurifier.org) | ||
|
|
||
| ## Contributing |
There was a problem hiding this comment.
A bit unrelated, I also wonder if we should not move this to CONTRIBUTING.md:
Co-authored-by: Jan Tojnar <jtojnar@gmail.com>
Current instructions based on composer.json scripts and #163