Migrating the blog to a static web site with Lektor
We have been having issues with the blog for a long time. The template we are using was developed for another purpose and never really finished. We also spend a lot of money on Drupal that could rather be spent somewhere else.
I propose to migrate the blog to a static website with lektor and have comments running from discourse.org. I have actually been in contact with discourse and they have offered to run a forum for us for free (legacy/trac#33105 (moved)). Addittionally they would run an .onion and a torproject.org subdomain.
Moderation on discourse is much easier than on drupal comments (another pain point for the blog), and we would get a forum that we could use for other purposes too.
This is how the blog can be migrated.
- We will do mockups and approve them with all the parties involved.
- We will start migrating content and have it on staging for testing.
- Once we are happy we will archive the current blog into static pages and move the static bloc to blog.torproject.org.
What we will lose:
- Version history on posts.
- Old comments. I see no value in migrating old blog comments to discourse to be honest. It would be a lot of effort and the old comments will be archived anyways in the blog archive.
What we will gain:
- Once the blog has been migrated creating posts will be no different than editing anyone of our websites. It would actually be a bit easier since there are a few different template and pages in our website and the blog will just have posts.
- Anyone that is currently updating the websites will be able to help people with blog posts, therefore we will gain a lot more of blog admins.
- The blog will run from our static www rotation, so we will not have to run expensive services or 2 cache VMs in front of it to avoid paing a lot to a third party for page views.
- We will not have to update a service every now and again. The blog will run out of static HTML.
- We will be able to update the templates according to our styleguide.
- Functionalities that we will build won't break because of a drupal update.
Designs
- tpo/tpa/team #40297Launch support's Forum and Blog migration
- #40004Launch support's Forum and Blog migration
Activity
Trac:
Parent: legacy/trac#33105 (moved) to N/A
Description: We have been having issues with the blog for a long time. The template we are using was developed for another purpose and never really finished. We also spend a lot of money on Drupal that could rather be spent somewhere else.I propose to migrate the blog to a static website with lektor and have comments running from discourse.org. I have actually been in contact with discourse and they have offered to run a forum for us for free (<ticket#>). Addittionally they would run an .onion and a torproject.org subdomain.
Moderation on discourse is much easier than on drupal comments (another pain point for the blog), and we would get a forum that we could use for other purposes too.
This is how the blog can be migrated.
- We will do mockups and approve them with all the parties involved.
- We will start migrating content and have it on staging for testing.
- Once we are happy we will archive the current blog into static pages and move the static bloc to blog.torproject.org.
What we will lose:
- Version history on posts.
- Old comments. I see no value in migrating old blog comments to discourse to be honest. It would be a lot of effort and the old comments will be archived anyways in the blog archive.
What we will gain:
- Once the blog has been migrated creating posts will be no different than editing anyone of our websites. It would actually be a bit easier since there are a few different template and pages in our website and the blog will just have posts.
- Anyone that is currently updating the websites will be able to help people with blog posts, therefore we will gain a lot more of blog admins.
- The blog will run from our static www rotation, so we will not have to run expensive services or 2 cache VMs in front of it to avoid paing a lot to a third party for page views.
- We will not have to update a service every now and again. The blog will run out of static HTML.
- We will be able to update the templates according to our styleguide.
- Functionalities that we will build won't break because of a drupal update.
to
We have been having issues with the blog for a long time. The template we are using was developed for another purpose and never really finished. We also spend a lot of money on Drupal that could rather be spent somewhere else.
I propose to migrate the blog to a static website with lektor and have comments running from discourse.org. I have actually been in contact with discourse and they have offered to run a forum for us for free (legacy/trac#33105 (moved)). Addittionally they would run an .onion and a torproject.org subdomain.
Moderation on discourse is much easier than on drupal comments (another pain point for the blog), and we would get a forum that we could use for other purposes too.
This is how the blog can be migrated.
- We will do mockups and approve them with all the parties involved.
- We will start migrating content and have it on staging for testing.
- Once we are happy we will archive the current blog into static pages and move the static bloc to blog.torproject.org.
What we will lose:
- Version history on posts.
- Old comments. I see no value in migrating old blog comments to discourse to be honest. It would be a lot of effort and the old comments will be archived anyways in the blog archive.
What we will gain:
- Once the blog has been migrated creating posts will be no different than editing anyone of our websites. It would actually be a bit easier since there are a few different template and pages in our website and the blog will just have posts.
- Anyone that is currently updating the websites will be able to help people with blog posts, therefore we will gain a lot more of blog admins.
- The blog will run from our static www rotation, so we will not have to run expensive services or 2 cache VMs in front of it to avoid paing a lot to a third party for page views.
- We will not have to update a service every now and again. The blog will run out of static HTML.
- We will be able to update the templates according to our styleguide.
- Functionalities that we will build won't break because of a drupal update.
is it part of the spec here to allow users unfamiliar with git to edit the website? i'm wondering if we've given any thought to which tools could be used for that purpose...
i wonder if tools like this could be useful for us:
https://gohugo.io/tools/frontends/
those are hugo frontends, but they could easily apply to lektor as well....
I would like to highlight the need for an events/calendar plugin or similar.
Also, we might want to keep the following notes on this from the last dev meeting: https://trac.torproject.org/projects/tor/wiki/org/meetings/2019Stockholm/Notes/Blog
Sounds like a great plan. Less work, less risk, and lower cost.
Replying to pili:
I would like to highlight the need for an events/calendar plugin or similar.
I think the calendar is the one big missing piece.
We also have "Recent Updates" on the blog, are we keeping that? It seems easy to implement using a template.
I also see tags on posts, they should be easy with a template.
And I think it's ok to lose comments and old versions of posts. Discourse is a great platform, it's really well designed for healthier conversations.
What's our plan for migrating media?
What's our plan for converting links in old posts? Can links between old blog posts point to the new blog, once the old posts have been converted?
Is there anything else we're going to lose, or any other changes we don't know about?
Is there a proof of concept somewhere, that we could look at? Or are we not at that stage yet?
Replying to hiro:
I propose to migrate the blog to a static website with lektor and have comments running from discourse.org.
I like this plan in theory. I agree that our current situation is not really sustainable.
I have two concerns that I want to make sure we address well enough with the new plan:
Moderation on discourse is much easier than on drupal comments (another pain point for the blog), and we would get a forum that we could use for other purposes too.
Do we have anybody on our side with real in-depth experience in managing discourse communities? I ask because with both the stackexchange case and the reddit case, we had plans for real Tor people who actually understand Tor things to engage, and it started out working that way, but then those people got distracted or burnt out or otherwise didn't keep up, and the platforms (for better or for worse) have a mechanism for random people to become highly trusted if they just stick with it -- so the result was that all the moderators are now random people whose primary property is that they have free time and they're stubborn, rather than that they are people we know and have some relationship with.
I no longer engage much with Tor's stackexchange, because some of the recent times I did, some random person stepped in and told me I was wrong about my answer and changed it to an answer I didn't want it to be. The original plan there was to pick a better platform for building our FAQ... but if we can't choose the answers on our FAQ, that's not so helpful to us.
So: do we have a plan for how this time will be different? Delegating our community moderation to a third party platform means giving the control to steer topics to...whom exactly?
At least with the current blog case, it fails closed when we spend a while not giving it attention, rather than failing open by going off in some surprising and unhelpful direction.
What we will lose:
- Old comments. I see no value in migrating old blog comments to discourse to be honest. It would be a lot of effort and the old comments will be archived anyways in the blog archive.
I agree that migrating old blog comments to discourse seems like a bad step. But are the only two choices 'migrate to discourse' or 'drop'? For example, another option might be to migrate them to the static html blog entries.
Many of our old blog posts have real content that would be a shame to lose. For two examples, https://blog.torproject.org/comment/224125#comment-224125 and https://blog.torproject.org/comment/78918#comment-78918 but there are many many more.
-
Replying to teor:
What's our plan for converting links in old posts? Can links between old blog posts point to the new blog, once the old posts have been converted?
That's always the tricky part in a conversion. I've done Drupal "fossilizations" before, which consists of turning a dynamic website into a static one. It's an involved process, but it can be done, with enough time and tweaking. The result is a static archive of the Drupal site, in raw HTML.
The tricky part is where to put that archive. If we want to keep the
blog.torproject.orgdomain active with the new lektor site, we'll have to do some dance like we did with the main website (move the old website to 2019.www.tpo, specifically). I'm not sure that worked very well, because it broke a lot of links and we didn't have good mechanisms to redirect to the older site on the new one.Could we consider creating the new site on a new domain, say
news.torproject.org?Is there a proof of concept somewhere, that we could look at? Or are we not at that stage yet?
... but yeah, maybe we're getting ahead of ourselves a little here. As far as I know, we haven't resolved those questions yet and there's no proof of concept. But those are excellent questions!
Replying to arma:
Do we have anybody on our side with real in-depth experience in managing discourse communities? [... notes about the limitations of stack exchange and reddit ...] do we have a plan for how this time will be different? Delegating our community moderation to a third party platform means giving the control to steer topics to...whom exactly?
The key difference between Reddit/Stackexchange and "our own discourse instance" (whether it's self-hosted or hosted at discoursehosting.net, as we plan to do) is that we would be admins of the instance, which is not the case of the other communities. We would have the final say in what's in and what's out, the same way we do right now with the blog comments. Although...
At least with the current blog case, it fails closed when we spend a while not giving it attention, rather than failing open by going off in some surprising and unhelpful direction.
... I am not familiar with how blog moderation works now. If it's "a priori moderation" (ie. that a comment does not show up before it's approved), that might be difficult to replicate in Discourse. But I'm not too worried about this: Discourse is specifically built to help manage a more healthy forum community, and I'm hopefully we can build one with existing staff resources (who currently moderate the blog) and possibly even community people (who would be promoted within the normal Discourse process)....
-
Replying to anarcat:
is it part of the spec here to allow users unfamiliar with git to edit the website? i'm wondering if we've given any thought to which tools could be used for that purpose...
i wonder if tools like this could be useful for us:
https://gohugo.io/tools/frontends/
those are hugo frontends, but they could easily apply to lektor as well....
Lektor has a frontend that can be used as a CMS. When you start the lektor server on your machine you can also access it on your localhost and create and/or edit pages from your browser.
-
I would like to migrate the old post to the "new" blog without the comments. Also I would like to keep an archive of the blog as static html pages exported from drupal. The archive will have the comments. I understand that in this plan there is a bit of redundancy but I also understand this is a way not to lose links to all our posts, which are valuable. In the new blog old posts can have a link at the bottom to the archive page where people can read the archived comments.
Lektor can define models that can then be edited as markdown. The system is quite flexible. We will not need a "plugin" for events or calendar like in drupal. We will just need to define a model for "events" and display all the events on html pages that look like a calendar. This will just be defined in html and css.
-
I would like to migrate the old post to the "new" blog without the comments. Also I would like to keep an archive of the blog as static html pages exported from drupal. The archive will have the comments. I understand that in this plan there is a bit of redundancy but I also understand this is a way not to lose links to all our posts, which are valuable. In the new blog old posts can have a link at the bottom to the archive page where people can read the archived comments.
Sounds like a great compromise to me! Cool. :)
moved from legacy/trac#33115 (moved)
added Bug label and removed 1 deleted label
-
Closing this issue. Future discussion about migrating the blog will be tracked on: https://gitlab.torproject.org/tpo/web/blog/-/issues/7
marked this issue as related to tpo/tpa/team#40297 (closed)
mentioned in issue #40004 (closed)
marked this issue as related to #40004 (closed)
changed milestone to %Launch support's Forum and Blog migration
