Migrate the blog out of Drupal and into a static site generator with a comment service

added Backlog label

marked this issue as related to #33115 (closed)

added Next label and removed Backlog label

added Backlog label and removed Next label

added Doing label and removed Backlog label

changed the description

marked the checklist item Import events from Drupal as completed

marked the checklist item Configure RSS/Atom feeds (posts + events) as completed

marked the checklist item Setup 2-column responsive layout as completed

marked this issue as related to #40006 (closed)

marked this issue as related to #15032 (closed)

marked the checklist item Mirror header and footer contents from other Tor websites as completed

I moved over the main repository to this project space. The website staging area is now: https://tpo.pages.torproject.net/web/blog

marked the checklist item Import comment archive as completed

I was able to deal with archiving comments in a satisfactory manner. The old Drupal comments will be integrated in the blog posts within Lektor as HTML. This markup is styled appropriately for the new version of the site. Using redirects, we'll avoid breaking comment permalinks.

The caveat with maintaining permalinks for comments is that in building the site, Lektor will need to generate several thousand small HTML files under /comment. This is likely to impact the time it takes to build the site, but how much remains to be seen.

Next step: test a larger import (100 blog posts).

changed milestone to %Launch support's Forum and Blog migration

changed the description

I was attempting to test the Embedding feature on our staging Lektor site (for comments), however the forum still being private at the moment is blocking:

Embedding comments from a private site

For private Discourse instances, if Discourse is on a subdomain of the blog’s domain, comments will be displayed for users who are logged into Discourse. Users who are not logged into Discourse will see a ‘refused to connect’ message. If Discourse and the blog are on completely separate domains, no comments will be displayed for private forums.

(from https://meta.discourse.org/t/embedding-discourse-comments-via-javascript/31963#embedding-comments-from-a-private-site-9)

We're on completely different domains, (torproject.net vs torproject.org) so nothing is displayed, rather I get a Firefox error message in the iframe resulting from a 400 Bad Request reply on Discourse's side.

marked the checklist item Test draft post feature as completed

changed the description

There's not built-in "Draft" feature in Lektor but one can set _discoverable: no in a contents.lr file to hide the page from the index, sidebar and feeds. If we use a merge-request style workflow, however, we probably won't need it.

is there a place where we can see a preview of the new static site?

Yes, it's here: https://tpo.pages.torproject.net/web/blog

The link is shown under Settings => Pages.

That looks amazing, congratulations!

changed the description

Hi! @gus asked the comms team to review / test Proposed web-based workflow for Lektor.

I finally did that, and it worked great for me. Thanks for the clear instructions. A handful of things:

I need help when it comes to step 10, as I don't know how to upload / add an asset. Just not something I have done before.

Upload any images or attachments as needed in assets/static/images/blog

Will we be able to add tags to posts? If so, how?

I need help when it comes to step 10, as I don't know how to upload / add an asset. Just not something I have done before.

Here is my best bet:

When you are in your project, navigate to the assets/static/images/blog folder. Click on the + icon which has an arrow pointing down next to it (see the image), then click on the option to upload a file.

Thanks @HackerNCoder, however the interface isn't quite the same when using the GitLab Web IDE as I show in the proposed workflow.

@smith I've updated step 10 with some extra info, let me know if it's still unclear

Will we be able to add tags to posts? If so, how?

Yes. I purposely haven't documented that yet because it depends on the outcome of #40008 (closed)

I've updated step 10 with some extra info, let me know if it's still unclear

Very clear, thank you.

Yes. I purposely haven't documented that yet because it depends on the outcome of #40008 (closed)

Got it!

marked this issue as related to #40008 (closed)

Making a note here that an automated site rebuild will be required at least once per day, otherwise the contents of "Upcoming events" eventually becomes outdated.

that can be automated in gitlab ci, right?

...

On 2021-10-04 14:19:27, Jérôme Charaoui (@lavamind) wrote:

Jérôme Charaoui commented:

Making a note here that an automated site rebuild will be required at least once per day, otherwise the contents of "Upcoming events" eventually becomes outdated.

-- Antoine Beaupré torproject.org system administration

Yes.

changed due date to October 30, 2021

changed due date to November 01, 2021

mentioned in issue tpo/tpa/team#40183 (closed)

marked this issue as related to tpo/tpa/team#40183 (closed)

Lately I've been testing embedding blog posts into the forum for the comment area. This works, but the challenge is making the posts on the forum look good. I've opened a channel with the Discourse team to find ways to improve this and will report my findings here.

marked the checklist item Test Discourse comments embedding (depends on tpo/tpa/team#40183 (closed)) as completed

With the help of the Discourse team I've been able to work it out. When a new blogpost will be added to the blog, the first visitor will trigger the creation of the topic on the forum. The post will be imported in full and appear immediately in the topic list on the forum as a new topic. Example: https://forum.torproject.net/t/global-encryption-day-maketheswitch/232

Important to note, posts imported from Drupal will not trigger the creation of a forum topic, and a read-only copy of the old comment section will be displayed under the post.

changed the description

changed due date to November 17, 2021

This will not be ready this week. We'll aim for Nov. 17th when the next State of the Onion livestream will happen.

changed the description

marked the checklist item Implement categories to replace tags (depends on #40008 (closed)) as completed

changed the description

marked this issue as related to #40011 (closed)

marked the checklist item Ensure functioning newsletter image URL redirections (example) as completed

marked the checklist item Plan/implement publishing moratorium and comments section closure on Drupal as completed

marked the checklist item Full, final content import from Drupal as completed

marked the checklist item Setup static web mirrors for new site as completed

changed the description

We're on track to deploy the new blog this Wednesday. A working copy of the site is already available on the static mirror system, responding to queries for https://blog.torproject.org.

There's still a few issues to iron out, mainly related to implementing Review Apps to support the publishing workflow.

see, specifically tpo/tpa/gitlab#114 (closed) and tpo/tpa/team#40515 (closed)

...

On 2021-11-16 01:54:01, Jérôme Charaoui (@lavamind) wrote:

There's still a few issues to iron out, mainly related to implementing Review Apps to support the publishing workflow.

-- Antoine Beaupré torproject.org system administration

I had it on the schedule to make a blog post tomorrow, but the blog transition is happening, which I imagine means no new posts for the day, at least. When do you expect we should be able to make new posts to the blog?

@smith While @anarcat and I set up the new publishing workflow, you can open an issue in this project which includes your markdown-formatted text and any accompanying assets, and I'll make the commit(s). You'll be able to review the result on the staging site before we push to prod.

thanks!

mentioned in issue #40024 (closed)

marked the checklist item Fossilize old Drupal site (#40011 (closed)) as completed

marked the checklist item Transition blog.torproject.org address as completed

The DNS entry is now pointing to the new Lektor blog hosted on TPO infrastructure.

marked the checklist item Implement Review Apps for easy MR previews as completed

ey there:

It seems the community portal depends on an "https://blog.torproject.org/events.xml" file to display next events at https://community.torproject.org/outreach/ . Is there a new rss feed or something we can pull now?

see:

https://gitlab.torproject.org/tpo/web/community/-/blob/main/templates/outreach.html#L38

ok, i have opened #40030 (closed) for that because i saw that lavamind was trying to make it work in dd0df946

marked the checklist item Document final publishing workflow of the new blog as completed

I've updated the documentation for the new blog: https://gitlab.torproject.org/tpo/tpa/team/-/wikis/service/blog

It will surely see some fine-tuning but the essential parts should be there.

it would be great to see the document follow the template a little more closely.

otherwise thanks for writing that documentation! :)

...

On 2021-11-24 23:36:05, Jérôme Charaoui (@lavamind) wrote:

I've updated the documentation for the new blog: https://gitlab.torproject.org/tpo/tpa/team/-/wikis/service/blog

It will surely see some fine-tuning but the essential parts should be there.

-- Antoine Beaupré torproject.org system administration

marked this issue as related to tpo/tpa/team#40526 (closed)

marked this issue as related to tpo/tpa/team#40527 (closed)

changed the description

Closing this issue as resolved.

I created two new issues to track the wrap up phase, namely winding down our Pantheon subscription and retiring the two dedicated caching proxy machines.

Thanks everyone!

closed

mentioned in issue tpo/ux/research#46 (closed)

Migrate the blog out of Drupal and into a static site generator with a comment service

Content

Layout

Deployment

Workflow

Designs

Child items ...

Activity