Sprint 07 Christmas tree ePermit research results January 24, 2018

Hypotheses tested

• Do people understand the information presented on the 'Buy a Christmas tree now' page? YES!
• Do people think the website is targeted towards first-time or repeat users? (SEE BELOW)
• What do people expect to see when visiting the website during a closed Christmas tree season? (SEE BELOW)
• Do people know how to properly use their Christmas tree permit? YES!
• What do people expect to see when they visit the website in the off season? (SEE BELOW)
• Do people understand how to get help on an error message they may receive on the website? (SEE BELOW)

Participants

• Four people were unfamiliar with the Christmas tree permit application and were recruited from friends and family of the project.
• Two people had participated in a previous usability test.

Do people understand the information presented on the 'Buy a Christmas tree now' page?

They really like it! All people interviewed completely understood the overall purpose of the page and the reason for selecting a forest. People also interpreted the page language to mean that not all forests offered a Christmas tree permit to be purchased online. Four people said it did not matter if the word "pilot" was on the page, while one person thought the wording of "pilot" on the page would help them to understand the website was still under development.

• "Yes, I can clearly tell this is for Christmas tree permits. It also says that all forests don't sell online permits."
• "That all of the National Forests might have different rules about ..and prices for cutting down Christmas trees and that some forests may not offer permits online."
• "Basically it says that, I have an option to buy a Christmas tree on...in some National Forests, otherwise I need to go to a ranger station."
• "Ok..so I think its saying that each National Forest is different and so you will have to check which one you plan to cut your tree in to see if it has permit sales online. You can use this website to see if the one you are planning to go to offers an online permit...but it may not be available depending on which forest you are going to."

Do people think the website is targeted more towards first-time or repeat users?

Most people thought the website was targeted more towards the first-time user but felt that the experienced users would not be put off by the amount of content provided.

• "Let's see..I think it's kind of a balance of ..I think it's hard for me to say since I have some experience with it, so some of it does feel like people who have done it before sort of know the limitations and the windows of when you can do it...so there is some familiarity...it doesn't feel like it's giving you too much information. It would be annoying to someone who has already done this..so I think maybe both."
• "I think it's fine for the first-time user."
• "I would lean more towards the first-time user. There is a lot of useful data here."
• "Yeah I think it's pretty easy, I mean first-time users......it's no different from anything else you buy online."

What do people expect to see when visiting the website during a closed Christmas tree season?

People expect to see basic information, along with language that explains permits are unavailable for purchase. Three people wanted the 'permits unavailable' language on the landing page. People also still wanted the ability to browse the guidelines pages for the national forests.

• -"That...some kind of header when you first logon that says permits unavailable at this time...come back at XXXX time."_
• "Basically I would want to see everything but an active link to buy a permit...so like all the restrictions. I would expect to see the windows for purchasing that year or the previous year and dates. [referring to the selected forest's page] I would probably want to see that on the home page as well."
• "Some kind of notification that says permits aren't for sale currently; it's the off season and a date when they would become available."
• "Probably a short notice on when the season would start.....maybe on the front page [referring to the landing page] so it is one of the first things you see."
• "I would expect it to say you either you can't get a permit until a certain date, or you can get a permit...but it wouldn't be valid until a certain date."

Do people know how to properly use their Christmas tree permit?

Yes they do! People understood the important messages, ie. a digital copy of the permit is invalid and the permit must be in clear view on the dashboard.

• "To be valid in the forest, I would have to print it and put it in my vehicle....on plain white paper."
• "I would fill in the dates for the trees I cut down...with a black felt tip pen of course."
• "Permits are non-transferable. It says in the guidelines, permit holder must be present at time of tree cutting..so I would assume it can't be transferred."

Do people understand how to get help on an error message they may receive on the website?

Yes. People understood what to do if they received an error message for something they couldn't fix on their own. Two people expected to see the errors on the forest's webpage itself, instead of being redirected to an general error message page.

• "Well, I would go back to buy a permit I guess and try it again, but I would expect that if I were buying a permit and tried to purchase and it didn't work, that it would show me which field has an error."
• "I'd assume I did something wrong [when receiving an error] and I would go right back to the buy a permit page and try to fill it in again. If I received it again...then I would call the 1-800 number."
• "I would go back, I would first press back [referring to the browser back button] and then I would just go to buy a permit and try it over again. If that didn't work, I would call the 1-800-number...I wouldn't email..but I would definitely call."

Issues on the site in general

Four notable issues appeared during usability testing and should be addressed.

• Pre-permit purchase tree selection
• Site map
• Cutting area times and dates
• Review and submit button on pay.gov mock form

First, the issue of people thinking they have to select a tree before purchasing a permit appeared again. One person stated

"I would probably choose a Douglas Fir since that is what I usually get. [when talking through purchasing a permit]..and I would probably choose 3 of those.."

Second, most people did not know what a site map was or how they would use it to navigate the website.

Third, people who visited the Arapaho-Roosevelt National Forest guidelines page were confused by the cutting area times and dates. They believed that the dates and times were mistakenly repeated, instead of being truly different for various districts in the forest. After examining the site more closely, one person said

"Oh, I did not catch that..."

Lastly, two people expected to be able to review their information before submission on the pay.gov page. The confusion stemmed from the button, which says 'review and submit.' People expected a separate webpage that would allow them to review the information before submitting.

What problems were most severe

Cleaner content

• Simpler presentation of cutting dates and times The cutting dates and times listed on the forest guidelines pages need a simpler and cleaner presentation so people can more easily scan and digest the time sensitive information. Arapaho-Roosevelt National Forest causes more confusion than others, largely because it has multiple districts with multiple cutting times that do not overlap.