Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add default email template #1898

Open
wants to merge 15 commits into
base: main
Choose a base branch
from
Open

Add default email template #1898

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
868435f
Create default_email_html.txt
milescalabresi Nov 22, 2015
d584bf0
use default HTML email template
milescalabresi Nov 22, 2015
eff3d7b
fixes missing comma
milescalabresi Nov 22, 2015
594b670
add instructions for admins to use new template
milescalabresi Nov 22, 2015
3284a76
fixes missing import in commmodule.py
milescalabresi Nov 22, 2015
eda9ad5
add (curently inactive) radio buttons for text vs HTML
milescalabresi Aug 5, 2016
53a614c
add DIY unsubscribe message at bottom
milescalabresi Mar 31, 2017
1db5e2d
initial crude JS validator to see if HTML present
milescalabresi Apr 20, 2018
b06685e
finish JS validator for HTML vs plaintext
milescalabresi May 4, 2018
06a548b
add explanation for 'your email' and move JS to separate script file
milescalabresi May 10, 2018
f68ef1d
Merge remote-tracking branch 'origin/main' into add-default-email-tem…
milescalabresi May 29, 2018
e50ec11
improve handling of user's selected language
milescalabresi May 31, 2018
0b4ec30
pass correct html vs plaintxt variables
milescalabresi May 31, 2018
121b465
pass correct template and escape HTML
milescalabresi May 31, 2018
d8599f4
rename email template wrapper for consistency
milescalabresi May 31, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion esp/esp/program/modules/handlers/commmodule.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@
from django.template import Context as DjangoContext
from esp.middleware.threadlocalrequest import AutoRequestContext as Context
from esp.middleware import ESPError
from django.template import loader

class CommModule(ProgramModuleObj):
""" Want to email all ESP students within a 60 mile radius of NYC?
Expand Down Expand Up @@ -109,6 +110,11 @@ def commprev(self, request, tl, one, two, module, extra, prog):
esp_firstuser = ESPUser(firstuser)
contextdict = {'user' : ActionHandler(esp_firstuser, esp_firstuser),
'program': ActionHandler(self.program, esp_firstuser) }

htmlbody = unicode(loader.get_template('email/default_email_html.txt').render(DjangoContext({'msgbdy': body,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the more standard thing is to use render_to_string() rather than get_template() and unicode(), and in that case you don't need to call the Context instance yourself, you just pass a dict.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, I think you want htmlbody instead of body here, otherwise lines 105-108 are meaningless. Also also, passing the user and program in here is unnecessary.

Edit: Oh, I see why you might want to. I guess it's fine, although it might be good to have a warning somewhere (not sure where) that if you modify the template you should make sure not to use any user vars (ideally not even any program vars) inside the {% autoescape off %} block.

'user': ActionHandler(esp_firstuser, esp_firstuser),
'program': ActionHandler(self.program, esp_firstuser)})))

renderedtext = Template(htmlbody).render(DjangoContext(contextdict))

return render_to_response(self.baseDir()+'preview.html', request,
Expand Down Expand Up @@ -175,7 +181,10 @@ def commfinal(self, request, tl, one, two, module, extra, prog):
sendto_fn_name = sendto_fn_name,
sender = fromemail,
creator = request.user,
msgtext = body,
msgtext = unicode(loader.get_template('email/default_email_html.txt').render(DjangoContext(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Likewise here I think render_to_string is simpler. Also, nit: can this either be on one line or be wrapped to a reasonable length?

{'msgbdy': body,
'user': request.user,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This causes the template to render user variables in the template itself (not message body) with information about the sender. Not a problem with the default since there's no use of the user, but this is what's causing Yale's bug. Since the template is going to get rendered into one copy for all users, which then gets further transformed later into the individual copies, user information shouldn't be passed here at all.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a way to add user information into the template? It would be very useful to put, for instance, the username the email was sent to.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, you can still put user information in the template; it will just get interpreted on the second render pass, when we add user variables.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I don't quite understand what the difference is / what needs to be done in order to make sure the right thing happens instead of the variables being rendered in the template?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This template should get rendered twice. First, here, we render msgbdy into default_email_html.txt to get msgtext. This happens once, and therefore should not include any user data, because we don't yet know what user the email will be sent to. Then, later, below, we the output of that (msgtext) into the actual email, once for each user, injecting their information into the template. Any variables in msgtext will get interpreted on the second pass. When you include request.user the first time, it means that any user variables in default_email_html.txt get rendered according to the user sending the email, because that's who request.user is; instead you want to render them the second time with the email's recipient as the user.

I wrote too quickly above and was a bit incorrect: any template variables that you put in default_email_html.txt that should get rendered on the second pass, i.e., any user variables, need to have their template tags excaped, using {% templatetag openvariable %} instead of {{ and {% templatetag closevariable %} instead of }}. This way, the first rendering pass will convert those to {{ and }} and then the second rendering pass will see them as normal variables, and interpolate the correct user's information.

'program': self.program }))),
special_headers_dict
= { 'Reply-To': replytoemail, }, )

Expand Down
5 changes: 5 additions & 0 deletions esp/templates/email/default_email_html.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
<html>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is redundant -- we tell users to include their own <html> if they want HTML.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, this file should either be called default_email.html, if it's html, default_email.txt, if it's text, or default_email if it could be either.

{% autoescape off %}
{{ msgbdy }}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If msgbdy might have HTML, you'll need to use {{ msgbdy|safe }} so django won't escape the HTML. (When doing this you should make sure that only admins are passing msgbdy, or that it's escaped somewhere else; in this case the former is true.)

{% endautoescape %}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if the default template should include an unsubscribe footer that links to the disable account page. (If so, the text should definitely say so.)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 👍 👍 and maybe some phone-compatibility code? If we include unsubscribe footers (I have a good one of those that I can add) and other stuff, then does that tie us into making them use HTML or Markdown?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we're allowing both HTML and plain text, we probably actually want to have two templates; the plain-text one can just include the URL.

What do you mean by phone-compatibility code?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, if we want, we could consider including a per-user unsubscribe link that doesn't require logging in, since making people log in to unsubscribe is really annoying. It would require a bit more code so perhaps it's better to do that separately.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean code that makes emails display more nicely on phones (there are sometimes issues with rotating screens and font sizing that you can throw enough HTML at to make them vanish).

The per-user link would be nice if that's possible. We can also include an "email us if you want to unsubscribe" if the additional code wouldn't be worth the hassle.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Huh, ok, cool. If by "email us if you want to unsubscribe" you intend for a human to handle it, that's fine; we can't really do an automated thing since the from/reply-to could be an arbitrary address controlled by the chapter.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, sorry. I mean for humans to handle the request if logging in is annoying or they forgot their password.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we have a nice template, making a 1-click unsubscribe wouldn't be too hard -- there might even be a library that will do most of the work for us. But in any case, we can do that after this is merged.

</html>
3 changes: 2 additions & 1 deletion esp/templates/program/modules/commmodule/step2.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,8 @@ <h2>Step 2:</h2>
Please write your email/message now. Note that you can use <tt>{{ parameter }}</tt> to enter a parameter
into the text of the message&mdash;the parameters are listed below.

<p><b>Tip:</b> If you would like to send your message in HTML format, wrap your entire message body in &lt;html&gt; and &lt;/html&gt; tags.</p>
<p><b>Tip:</b> The default {{ settings.ORGANIZATION_SHORT_NAME }} template (which you can modify <a href="/admin/utils/templateoverride/">here</a>) is automatically wrapped around your message below.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can actually probably link straight to /admin/utils/templateoverride/?name=default_email_html.txt or something to automagically do the search/filter for the right TO (if it exists).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a great idea! Thanks! I forgot you could search by name. I'll add that in once we settle on a good name for file, which will probably depend on what we decide for this guy.

Don't forget to write the body in HTML, but no need for &lt;html&gt; and &lt;/html&gt; tags.</p>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should still allow writing plain-text emails, because it's easier and if you don't need any formatting it's better. (Ideally we may want to switch to Markdown for emails, because that's both easier and can be used to make pretty things.)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, my thought was to include a default HTML email that did something basic (such as replacing newlines with paragraph tags, adding some kind of standard unsubscribe message, and maybe some phone-compatibility things) but benign so that it will automatically format it a bit for them. Most of the chapter board was very enthusiastic about this idea, and it seems nice to include some baseline level of professionalism (it's actually illegal to send mass emails like ours without some unsubscribe instructions, for instance, and unformatted plain text doesn't look very nice). Of course, programs could modify this file as they wish (that's the whole point), but having a default with some easy prettify-ing HTML in there seems like a nice way to add a little style to the emails. But of course I'm open to discussion on this point

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we want to do that, we should actually do that, probably by converting markdown (perhaps optionally). You can look at how we do it for QSD as a guide. If we do that I'm fine not allowing plaintext, but as I understand it, as-is this will just eat the newlines and such, and I do think for short emails unformatted text looks fine.

Incidentally, I believe CAN-SPAM doesn't actually apply to our use case since we are non-commercial and all users have signed up for our mailing list by registering an account, although we certainly should include unsubscribe links.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, while we're considering this, we may want to think about whether we should always be attaching a plain-text version of HTML emails, since there are still some email clients that will be sad if you don't. If we do markdown and people don't include raw HTML tags, this is fairly easy; in the more general case it could be a lot trickier. So we may not want to bother, or if we do, may want to do it in a separate pull request.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be nice to do the full-blown version. I'll take a look at the markdown conversion. Good call on including the plain-text version though. That seems worth doing. And maybe a line to the effect of "click here to see this email in your browser" if we can manage that without too much trouble?

I'm kind of tempted to force admins to write in something that looks nice. We could include an optional if-statement to catch emails under 50 words with no formatting tags (or whatever) that would just wrap them in <p> tags and add the default headers/footers and for emails over 50 words, require admins to use HTML. Thoughts on that?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Click here to see this email in your browser" is fairly reasonable; we'd just have to render the same template into a page. The only problem is that we'd either have to always make people log in for it, or somehow make it user-specific without doing that, which is a little tricky to do correctly (i.e. without leaking user data). But it's probably doable.

Markdown seems better (and probably easier) than trying to autowrap <p> tags ourselves, and should do that for us.

Incidentally, we'll want to make sure the release notes of whatever release this goes in include a mention of the fact that the defaults have changed, since some of these are breaking changes.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, cool. I don't know how to safely render emails without making them log in, but I can search around a bit.

Sounds good. I'm not super familiar with how Markdown works, but I'll check out the QSD-conversion code and see if I can figure out how to adapt it.

Good call on the release notes. At this point, I think I should open a new pull request after making some changes to this one.

To summarize, next steps are:

  1. Look into making a more general code that will permit admins to enter plain or marked-up text from the comm panel and merge their message with the default HTML template.
  2. Expand the current default template to include some "invisible" compatibility code (for phones, and maybe a "view in browser" link) as well as a generic unsubscribe footer and easy slots for admins to add/edit their own headers and footers.
  3. Update step2.html (include a link directly to the template override) and release notes with new instructions for admins. This should probably include some warnings about what not to touch (autoescape tags and where not to mix HTML/Markdown)
  4. Update Travis if necessary (depending on how steps 1 and 2 work out).
  5. Fix style in the Python files (among other things, use render_to_string() or find out what went wrong last time).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Basically we'd want to do it similar to an unsubscribe link: give each user a unique unguessable link, which the website can decode to get their username and therefore render the correct email for them. I'm not sure what the best way to do that in practice is, but there may be some package out there that will do the crypto for you.

For release notes, if you want you can just add a blank release notes file now, and put your stuff in it, or you can do it when we actually get around to cutting the release. It would probably be nice if we did more of the former, but since we haven't in the past it's fine if you want to do the latter.

Those steps sound good!


<form action="/manage/{{program.getUrlBase}}/commprev" method="post" name="comm2">
<table border="0" cellspacing="0" width="100%">
Expand Down