test generator and ui

[afknapping]

(continuing from twitter)

here is how i imagine it:

after initial set up (which probably needs dev support) a designer can run a generator script against a css file.

this generates tests for all selectors in the file.

a ui lets you pick the tests you want to keep (in the way you already mark failed tests as accepted).

this approach would remove the need for the generator to be intelligent in any way (eg recognize components, as you mentioned).

then again, the script maybe could also be run directly against a selector. that already would be a huge help in setting up initial tests before refactoring.

[afknapping]

just some more ideas since i just found this in the docs:

Adds a test case to be executed, with testCase being an object of key/value pairs with
url link to the test case HTML document,
hover a selector matching a single element on the page that should receive a :hover effect,
active a selector matching a single element on the page that should receive a :active effect,
width the viewport width, default: 800 (added in upcoming 0.4.0),
height the viewport height, default: 100 (added in upcoming 0.4.0).
Optionally the url can be passed in directly if no other options are needed.

I guess, what I want to do is to give a http adress as the url. since same origin stuff, i would need to open the testrunner from the same adress?

since it is possible to give selectors to hover and active, i could also give selectors without state for testing.

viewport could be set to rendered height and width of selector.

maybe a .csscriticrc file like this (yaml-like, not sure if this valid yaml):

title: landing page
    url: http://0.0.0.0:8888
    selectors: main.css

title: about page
    url: http://0.0.0.0:8888/about
    selectors: main.css

As there is a TestRunner.html, there is also a TestGenerator.html. It matches selectors between url markup and cssfile and offers the ui to keep or discard the tests with previews of the selectors. states are also read from the css. plus there is the standard list of elements which can have those states. this would lead to all elements being tested if showing state at all.

thinking about it, the css file to match could be extracted from the <link> tags.

To avoid long-running tests on inital run, in the api there could be a threshold of maximum area which is to be previewied (default could be really small). Larger tests are not renderd yet. again there is an ui to trigger them and then keep the test or discard.

maybe adding http url and plain selectors to the api would be smallest step? in terms of value, the follwing would already enable me to set up many tests in estimatable time:

window.onload = function() {
            csscritic.addReporter(csscritic.BasicHTMLReporter());
            csscritic.add({
                'title': 'landing page',
                'url': 'http://0.0.0.0:8888/',
                'selectors': [
                    '#teaser',
                    '.call-to-action:hover:active',
                    'aside.left #sidebar-navigation .nav-point'
                ]

            )};
            csscritic.add({
                'title': 'about page',
                'url': 'http://0.0.0.0:8888/about',
                'selectorSource': [ 'selectors-about.yaml' ]
            )};
            csscritic.execute();
        };

where selectors-about.yaml is just a flat list:

#teaser
.call-to-action:hover:active
aside.left #sidebar-navigation .nav-point

I'm very aware though that imagining all this stuff is cheap... :/

[cburgmer]

Thanks for the detailed description (and sorry for the late answer). I'd like to think about it further. There are a few initial concerns:

1. Combinatory complexity: We probably don't want to match any given selector with all others. Having a typical sass/less hierarchy in mind - one could detect parent/child relashionships and construct matching dom.
2. HTML element choices and browser presets: Good CSS, they say, does not select on HTML elements. So which elements should be generated, if browser presets are relevant (box-sizing for <button>, margins for other elements)?
3. Rendering dependencies outside of CSS: the actual code might have text content or images, that are relevant to the test case.

As you say, this is a rather general approach that will provide false positives (and negatives). So we would need to verify this idea with a proof of concept, would we?