Consider next steps to improve applications QA

We completed a survey of the tor-qa mailing list in tpo/ux/research#67 (closed), which didn't receive much engagement unfortunately. Either way, it's a sign that we should move on. Here's a list of the next steps as I see them:

Tests

  • Produce a new manual test script based on the old Quality Assurance wiki page
  • Put in place some process to review the list of manual tests before each major release
  • Test and fix all current automated tests (that we want to continue using): see applications/tor-browser-bundle-testsuite
  • Review and update the testsuite wiki/documentation if necessary
  • Put in place some process to regularly run and maintain the testsuite

Recruitment campaign

  • Discuss the timing of running a mini-campaign on social media, the forum and email to recruit to Alpha testers
  • Create graphics and messaging to support the campaign: https://gitlab.torproject.org/tpo/operations/communications/-/issues/64
  • Review and improve the tor-qa landing page: tpo/web/community#269 (closed)
  • Write a template for "what to test" section to go in Alpha release posts
  • Write a blog post for the campaign: https://gitlab.torproject.org/tpo/operations/communications/-/issues/65
  • Prepare tweets and other social media messaging
  • Create forum badges for Alpha testers: tpo/community/support#40076 (closed) & tpo/community/support#40075 (closed)
  • Update the sticky post in the Alpha feedback category
  • Figure out what to do with release posts (e.g. cross-post them into the Alpha feedback category): tpo/community/support#40077 (moved)
  • Select some (or all) of the above for translation into RU
  • Launch the campaign and onboard new Alpha testers

QA party

  • Schedule an internal QA party during an all-hands approx 1-1.5 months before stable release
  • Work with User Support to create a format for this session
  • Host the QA party and translate the results into tickets

Other

  • Include a test-banner on about:tor on Nightly and Alpha releases: tor-browser#40886 (closed)
  • Begin collecting metrics on the platforms we're receiving feedback on to identify gaps in our QA
  • Review our Nightly and Alpha /download pages, and potentially combine these
Edited Dec 08, 2022 by donuts
Assignee Loading
Time tracking Loading