Docs/feedback

feedback/about-page-feedback.md

@@ -0,0 +1,88 @@
+---
+altLangPage: https://conception.canada.ca/retroaction/fonctionnement.html
+breadcrumbs:
+  - title: About Canada.ca
+    link: https://www.canada.ca/en/government/about.html
+  - title: Analytics and feedback
+    link: https://www.canada.ca/en/analytics.html
+  - title: Page feedback
+    link: https://design.canada.ca/feedback/index.html
+dateModified: 2023-02-28
+layout: default
+title: How the page feedback tool works
+---
+
+<div class="gc-stp-stp">
+  <div class="row">
+    <ul class="toc lst-spcd col-md-12">
+      <li class="col-md-4 col-sm-6"><a class="list-group-item active" href="about-page-feedback.html">How the tool works</a></li>
+      <li class="col-md-4 col-sm-6"><a class="list-group-item" href="when.html">When to use page feedback</a></li>
+      <li class="col-md-4 col-sm-6"><a class="list-group-item" href="benefits.html">How feedback can help</a></li>
+    </ul>
+  </div>
+</div>
+
+## On this page
+
+* [Why collect user feedback](#why-collect-user-feedback)
+* [Feedback tool design](#feedback-tool-design)
+* [Protecting personal information](#protecting-personal-information)
+
+The page feedback tool is still in a pilot phase. This means that you can use the pattern, but there may be changes to how it looks or is coded in the future.
+
+## Why collect user feedback
+
+The primary purpose of collecting feedback is to use it as a **research tool** to improve content.
+
+> Client feedback is a critical input into ensuring that services meet the needs of clients and to support continual improvement.
+>
+> Source: [Guideline on Service and Digital](https://www.canada.ca/en/government/system/digital-government/guideline-service-digital.html#ToC2_2)
+
+First person feedback captured “in the moment” of failure gives us a lens into the issues affecting a page of content, task or service.
+
+Combining feedback insights with other quantitative data is an important step towards building a service culture that is human-centred and evidence-driven to continuously improve our services in a timely manner for Canadians.
+
+## Feedback tool design
+
+The feedback tool is an **optional** pattern. It can be added to the bottom of a content page after the page content and before the date modified. It replaces the “Report a problem” pattern while actively collecting user feedback.
+
+The feedback tool invites visitors to:
+
+* indicate if they found what they were looking for (yes / no)
+* offer their feedback describing the problem
+
+<details>
+  <summary>Image of the Canada.ca footer with the feedback tool</summary>
+  <figure class="mrgn-tp-lg">
+    <img class="img-responsive border" alt="Image of the footer, with the feedback tool placed after the page content and before the Date modified" src="images/footer-feedback-en.png" />
+  </figure>
+</details>
+
+<details>
+  <summary>Image of the form after clicking "No"</summary>
+  <figure class="mrgn-tp-lg">
+    <img class="img-responsive border" alt="A long description can be found after the image." src="images/description-en.jpg" />
+    <details>
+      <summary>Feedback tool</summary>
+      <p>A heading labelled "Please provide more details". Followed by the text "You will not receive a reply. Don't include personal information (telephone, email, SIN, financial, medical, or work details). Maximum 300 characters", and a text field to provide more details.</p>
+    </details>
+  </figure>
+</details>
+
+## Protecting personal information
+
+Filters are in place to remove these common types of personal information and profanity if submitted:
+
+* phone numbers
+* email addresses
+* social insurance numbers
+* passport numbers
+* postal codes
+* curse words
+
+When personal information is automatically scrubbed, it is replaced with hashtags (###).
+<nav role="navigation" class="mrgn-bttm-lg">
+  <ul class="pager">
+    <li class="next"><a href="when.html" rel="next">Next: When to use page feedback</a></li>
+  </ul>
+</nav>

feedback/access-feedback.md

@@ -0,0 +1,59 @@
+---
+altLangPage: https://conception.canada.ca/retroaction/access-feedback.html
+breadcrumbs:
+  - title: About Canada.ca
+    link: https://www.canada.ca/en/government/about.html
+  - title: Analytics and feedback
+    link: https://www.canada.ca/en/analytics.html
+  - title: Page feedback
+    link: https://design.canada.ca/feedback/index.html
+dateModified: 2023-02-28
+layout: default
+title: How to access page feedback
+---
+<div class="gc-stp-stp">
+  <div class="row">
+    <ul class="toc lst-spcd col-md-12">
+      <li class="col-md-4 col-sm-6"><a class="list-group-item active" href="access-feedback.html">Access feedback</a></li>
+      <li class="col-md-4 col-sm-6"><a class="list-group-item" href="analyze-feedback.html">Analyzing feedback</a></li>
+      <li class="col-md-4 col-sm-6"><a class="list-group-item" href="insights.html">Sharing insights</a></li>
+      <li class="col-md-4 col-sm-6"><a class="list-group-item" href="prioritize.html">Deciding what to improve</a></li>
+    </ul>
+  </div>
+</div>
+
+## On this page
+* [Where feedback is stored](#where-feedback-is-stored)
+* [How to access feedback](#how-to-access-feedback)
+
+## Where feedback is stored
+
+When a user submits a comment, it is sent and stored in a database managed by the Digital Transformation Office at TBS.
+
+## How to access feedback
+
+Feedback can be viewed and downloaded as CSV or Excel files at any time using the Feedback Viewer website.
+
+The Feedback Viewer is a password protected environment available to all page feedback participants. It is only available from within the GC network. Contact the DTO to access to the site.
+
+Email: [[email protected]](mailto:[email protected])
+
+### Feedback Viewer filters
+
+The Feedback Viewer includes filters so that you can access and download specific feedback, including:
+* Date range
+* Institution, Theme, and section of content
+* URL
+* Language
+* Comment field search
+
+<details>
+  <summary>Image of the Feedback Viewer</summary>
+  <p><img src="images/feedback-viewer.png" alt="screenshot of the Feedback viewer showing the filter categories" class="img-responsive"/></p>
+</details>
+
+<nav role="navigation" class="mrgn-bttm-lg">
+  <ul class="pager">
+    <li class="next"><a href="analyze-feedback.html" rel="next">Next: Methods for analyzing feedback</a></li>
+  </ul>
+</nav>

feedback/analyze-feedback.md

@@ -0,0 +1,223 @@
+---
+altLangPage: https://conception.canada.ca/retroaction/methodes.html
+breadcrumbs:
+  - title: About Canada.ca
+    link: https://www.canada.ca/en/government/about.html
+  - title: Analytics and feedback
+    link: https://www.canada.ca/en/analytics.html
+  - title: Page feedback
+    link: https://design.canada.ca/feedback/index.html
+dateModified: 2023-02-28
+layout: default
+title: Methods for analyzing feedback
+---
+
+<div class="gc-stp-stp">
+  <div class="row">
+    <ul class="toc lst-spcd col-md-12">
+      <li class="col-md-4 col-sm-6"><a class="list-group-item" href="access-feedback.html">Access feedback</a></li>
+      <li class="col-md-4 col-sm-6"><a class="list-group-item active">Analyzing feedback</a></li>
+      <li class="col-md-4 col-sm-6"><a class="list-group-item" href="insights.html">Sharing insights</a></li>
+      <li class="col-md-4 col-sm-6"><a class="list-group-item" href="prioritize.html">Deciding what to improve</a></li>
+    </ul>
+  </div>
+</div>
+
+## On this page
+
+* [Who should analyze feedback](#who-should-analyze-feedback)
+* [How much feedback is needed](#how-much-feedback-is-needed)
+* [Research questions shape analysis](#research-questions-shape-analysis)
+* [Manual analysis methods](#manual-analysis-methods)
+* [Tools for analysis](#tools-for-analysis)
+* [Machine learning pilot](#machine-learning-pilot)
+* [Include other supporting data sources](#include-other-supporting-data-sources)
+
+## Who should analyze feedback?
+
+This should be done by people who know the subject quite well.
+
+People should be adept at spotting patterns and themes in data.
+
+It’s best to have someone who is bilingual.
+
+If more than one person is sharing the task of reading feedback, having a shared understanding of the issues (and how you will group feedback) is very important.
+
+It’s good to get into the habit of looking at user feedback regularly to identify any emerging or persistent issues affecting task success.
+
+## How much feedback is needed
+
+There is no magic number for how many comments you need.
+
+With feedback, you are looking for enough comments to sufficiently describe an issue or answer a research question. There is a point of diminishing returns when collecting more feedback does not lead to additional insights. This is called “saturation”.
+
+If feedback shows that something is broken, you don’t need hundreds of comments to determine if you should take action.
+
+When identifying issues, don’t rely on volume of feedback alone to prioritize improvements. Feedback submitted may be from people who face cultural, linguistic, geographical, disability, technological, socioeconomic, or other barriers to uptake.
+
+You can seek confirmation of feedback insights using other data sources such as: web analytics, call volumes, social media trends, and GC Task Success Survey results.
+
+While it is best to read the full dataset of feedback, sampling a smaller set of feedback can help identify trends when dealing with more feedback than you can read. If you continue to receive more feedback than you can read, it is best to remove the tool until you have completed your updates.
+
+## Research questions shape analysis
+
+Start by thinking in terms of research questions and who you will be sharing your findings with. Doing this in advance can help clarify how to group feedback when doing manual analysis.
+
+Common research questions:
+
+* What are the most common issues being reported?
+* Are there specific reasons for failure or specific suggestions to improve the experience?
+* What pages are receiving the most feedback?
+* Has feedback increased or decreased after a page update?
+* What types of issues were the most common (findability, comprehension, technical)?
+
+## Manual analysis methods
+
+Grouping feedback with similar issues together with tags is useful for both small and large datasets. It helps you be more efficient with analysis by having smaller sets of data to analyse.
+
+A small dataset may only need a few tags to make sense of the feedback. A large dataset may require two levels of tags to understand specific content issues.
+
+### Best practices for choosing how to group and tag feedback
+
+<details>
+  <summary>Familiarize yourself with your data</summary>
+  <p>Read through a sample of feedback data and try to spot recurring patterns. Jot them down to get a rough overview of WHAT tasks, topics, or issues people are talking about.</p>
+  <p>Not every comment will be useful - sometimes it will be too unclear to use or be completely about another topic.</p>
+</details>
+
+<details>
+  <summary>Consider tags based on a task or issue</summary>
+  <p><strong>Task-based tags</strong> are recommended when analyzing feedback for a group of pages where there are multiple user tasks.</p>
+  <p>To identify tasks, ask yourself why the user came to the site. What were they trying to do, or what question were they trying to answer? </p>
+  <p><strong>Issue-based tags</strong> may be a better strategy when gathering feedback on a single page, single topic, or where a single task dominates your feedback.</p>
+  <p><strong>For large datasets</strong> you may find a second level of tags is needed to add precision. This can be done at the same time you tag the feedback OR when you are ready to analyze a smaller set of feedback.</p>
+
+  <h3>Example of some tags used for feedback on vaccine pages</h3>
+  <table class="provisional gc-table table table-striped" id="myTable1">
+    <caption class="wb-inv">Example feedback tagging model </caption>
+    <thead>
+      <tr>
+        <th scope="col">Tag</th>
+        <th scope="col">User task or issue</th>
+        <th scope="col">Topics included</th>
+      </tr>
+    </thead>
+    <tbody>
+      <tr>
+        <td data-label="Tag"><span class="text-left">Vaccine safety</span></td>
+        <td data-label="User task"><span class="text-left">Is the vaccine safe for me?</span></td>
+        <td data-label="Topics included"><span class="text-left">Pre-existing conditions, ingredients/allergies, side effects</span></td>
+      </tr>
+      <tr>
+        <td data-label="Tag"><span class="text-left">Getting vaccinated</span></td>
+        <td data-label="User task"><span class="text-left">How do I get vaccinated?</span></td>
+        <td data-label="Topics included"><span class="text-left">Eligibility, when, where, how to register</span></td>
+      </tr>
+      <tr>
+        <td data-label="Tag"><span class="text-left">Proof of vaccination</span></td>
+        <td data-label="User task"><span class="text-left">How do I get a copy of my vaccine record?</span></td>
+        <td data-label="Topics included"><span class="text-left">Vaccine records, provincial apps, federal vaccine proof</span></td>
+      </tr>
+    </tbody>
+  </table>
+</details>
+
+<details>
+  <summary>Limit the number of tags being used</summary>
+  <p>Start with broad tags and only include those for which you have multiple examples. Your goal with this first review is to succinctly group recurring topics/issues.</p>
+  <p>Aim to keep your set of tags to under 15 for the task.  Limiting the number of tags will help surface the issues that need the most assistance.</p>
+  <p>“Other” is a tag too!  Tag one-offs or low-frequency comments as “Other” until there are enough for them to graduate into having a tag of their own.</p>
+</details>
+
+<details>
+  <summary>Avoid using overlapping or ambiguous tags </summary>
+  <p>Make sure each tag is clearly differentiated from the others. Your aim is to reduce doubt about which tag a comment should get.</p>
+</details>
+
+<details>
+  <summary>Be prepared to tweak your choice of tags</summary>
+  <p>As you read more of your dataset, review your initial tag choices. Are they clear and unambiguous? Does one tag alone cover the majority of feedback? Do you need to divide it into separate tags?</p>
+  <p>There’s no one-size-fits-all strategy. As you collect more feedback, you may find you need to adjust your choice of tags.</p>
+</details>
+
+<details>
+  <summary>Document and test your tagging strategy</summary>
+  <p>Document your choice of tags with examples.   This is especially useful if more than one person will share the responsibility for reviewing feedback.</p>
+  <p>Ask others to review your tag choices to make sure that the tags are clear to other people.  This is especially critical if more than one person will be helping to analyze feedback.  Agreeing on a common set of tags in the beginning (and when adjusting tags) avoids feedback being tagged poorly between people.</p>
+
+  <h3>Example of some tags used for feedback on vaccine pages</h3>
+  <table class="provisional gc-table table table-striped" id="myTable1">
+    <caption class="wb-inv">Example feedback tagging model </caption>
+    <thead>
+      <tr>
+        <th scope="col">Tag</th>
+        <th scope="col">User task or issue</th>
+        <th scope="col">Topics included</th>
+      </tr>
+    </thead>
+    <tbody>
+      <tr>
+        <td data-label="Tag"><span class="text-left">Vaccine safety</span></td>
+        <td data-label="User task"><span class="text-left">Is the vaccine safe for me?</span></td>
+        <td data-label="Topics included"><span class="text-left">Pre-existing conditions, ingredients/allergies, side effects</span></td>
+      </tr>
+      <tr>
+        <td data-label="Tag"><span class="text-left">Getting vaccinated</span></td>
+        <td data-label="User task"><span class="text-left">How do I get vaccinated?</span></td>
+        <td data-label="Topics included"><span class="text-left">Eligibility, when, where, how to register</span></td>
+      </tr>
+      <tr>
+        <td data-label="Tag"><span class="text-left">Proof of vaccination</span></td>
+        <td data-label="User task"><span class="text-left">How do I get a copy of my vaccine record?</span></td>
+        <td data-label="Topics included"><span class="text-left">Vaccine records, provincial apps, federal vaccine proof</span></td>
+      </tr>
+    </tbody>
+  </table>
+  <a class="btn btn-primary" href="images/feedback-tagging-template.xlsx" role="button"><span class="fa fa-download" aria-hidden="true"></span> Download a tagging strategy template (Excel, 61KB)</a>
+</details>
+
+<h3>What to avoid when tagging feedback</h3>
+<details>
+  <summary>Mixing types of tags</summary>
+  <p>If you want to add additional ways to analyse your dataset, it’s best to create new columns in your spreadsheet to note different kinds of facets. For example, adding a status or specifying a particular sub-issue.</p>
+</details>
+
+<details>
+  <summary>Trying to be overly-precise</summary>
+  <p>The purpose of tagging is to help you identify user priorities and group feedback into smaller datasets to analyze. A “good enough” approach to defining and assigning tags will do.</p>
+  <p>If you have more feedback than you can manage to review, classify and analyze, adjust your strategy: choose a specific task or time frame to focus on.</p>
+</details>
+
+## Tools for analysis
+
+For small datasets, any spreadsheet software should be adequate to group and sort feedback (Excel, Google Sheets, etc.).
+
+For larger datasets, it’s helpful to use a tool that has more advanced functionality to sort, filter, and tag. If you have a data science specialist, they may prefer or have access to more specialized tools.
+
+* [Download a tagging strategy template](images/feedback-tagging-template.xlsx) (Excel, 61KB)
+* [Download a template to analyze page feedback and/or GC TSS feedback](https://docs.google.com/spreadsheets/d/1pcQgee6lN6P30EIMRb3o6RxcTPBiUFtsZAmbeVNpDW4/edit?usp=sharing) (Google Sheets)
+
+## Machine learning pilot
+
+For institutions that receive high volumes of feedback, we are piloting alternative methods to access and analyze feedback using data science tools.
+
+Contact the Digital Transformation Office if you are receiving more feedback that you can manage through manual analysis.
+
+Email: [[email protected]](mailto:[email protected])
+
+## Include other supporting data sources
+
+Include other data sources in your reporting to build a more complete picture, confirm your insights, or add urgency from sources such as:
+
+* GC Task Success Survey results and feedback
+* analytics
+* call centre volumes
+* search trends
+* usability study results
+* questions received through social media
+
+<nav role="navigation" class="mrgn-bttm-lg">
+  <ul class="pager">
+    <li class="next"><a href="insights.html" rel="next">Next: Sharing insights</a></li>
+  </ul>
+</nav>