How to make acceptance tests more stable? (using Chimp && Webdriver)

testing
1

am I the only one wondering, how to make the acceptance tests more stable??

Strange things are happening - on my local development machine and also on CircleCi:

  1. browser.waitForExist() && browser.waitForVisible() act really unstable. The give false negatives, telling me that an element is NOT visible/existing, but it is.
  2. browser.setValue is really unstable and likes to mess up keys, p.e. inserts ‘„est Question?T”’ instead of ‘„Test Question?”’
  3. modal-windows do NOT open, if the browser does NOT have focus (or for whatever reason - this also happens on CI)
  4. … and so on…

I am starting to really really hate it, because trying to fix things is soo time-intense and I still did NOT have any successfull-build on CircleCi There are always tests failing.

Are there some tricks that I am missing?
Anyone out there to share some tears? :smile:

For beginners, I started wrapping webdriver functionality into a wrapper, p.e. this guy here helps from time to time:

/**
 * More stable version of waitForVisible.
 */
waitForVisible(element) {
  Stabilize.waitForExist(element)
  Logger.log(`waiting for element to be visible "${element}"`)
  const isVisible = Stabilize.isVisible(element)
  if (!isVisible) {
    Logger.log('element exists, but is NOT visible. Let`s scroll to it', 1)
    browser.scroll(element)
  }
  Logger.log('waiting for element to be visible', 1)
  try {
    browser.waitForVisible(element)
  } catch (e) {
    // I don't know why, but it really happens, that right NOW the element
    //  IS visible. So there seems to be a bug in webdriver.
    //  Use try-catch as a workaround and do NOT throw
    browser.isExisting(element)  // KEEP it - it might force the modal to show
    if (!browser.isVisible(element)) {
      Logger.log('element is NOT visible after timeout! This is some data for debug', 1)
      console.log('browser.isExisting(element)', browser.isExisting(element))
      console.log('browser.isVisible(element)', browser.isVisible(element))
      throw e
    } else {
      Logger.log('we just faked webdriver.waitForVisible! The element IS visible(!!)', 1)
    }
  }
  Logger.log(`element is finally visible: ${element}`, 1)
}
1 Like
2

Updates:

point 1) I managed to get it fixed by setting different value for waitforInterval. Tipp: Keep it small - my mistake before was that I had it set at a much too high interval

  // - - - - WEBDRIVER-IO  - - - -
  webdriverio: {
    waitforTimeout: 10000, // time to wait until element appears
    waitforInterval: 250,  // KEEP SMALL (!!!) this is the INTERVAL in which waitFor* is looped for checks
  },

point 2) I could not get modal-stability fixed. Within my wrapper I tried out this, but it did NOT help. Any other ideas on this?

  /**
   * Set a value to an input-element
   */
  static setValue(selector, value) {
    Stabilize.waitForVisible(selector)
    browser.setValue(selector, '')  // trying to prevent error that happens from time to time,
                                    //  where letters are messed up, p.e.
                                    //  expected '„est Question?T”' to equal '„Test Question?”'
    browser.pause(500)
    browser.setValue(selector, value)
  }

point 3) I found out that bootstraps modal itself simply does NOT appear as expected from time to time. Webdriver clicks the button, but the modal simply does not fire up reliably. I am using “theduke:meteor-bootstrap-modal-prompt” to show it and have created an issue on github https://github.com/theduke/meteor-bootstrap-modal-prompt/issues/9.

point 4) Grabbing some screenshots and browser-console-logs from CircleCi could help finding the issue - does anyone know how to do this? I created a seperate topic for this How to make CircleCi save SCREENSHOTS and LOGS of the browsers console?

It would be really cool, if someone could give some feedback on this.
@sam Any suggestions?