Migrate the docs SSG to Docusaurus

[Jwaegebaert]

Issues

High Prio

• Setup Docusaurs work branch #4535
• Docusaurs: Script the old syntaxes to the new form #4574 Ready to go!
• Docusaurs: Update the footer UI & mobile menus #4611
• Docusaurs: Resolve 'Meet the team' page #4587
• Docusaurs: Add index page to sample-scripts #4640
• Docusaurs: Adjust --help print to accommodate different syntaxes #4673 Ready to go! Still need to write the tests 😅
• Docusaurs: Update sidebar.js to manually order cmd folder content #4743
• Docusaurs: Modify GitHub actions to build & test #4754
• Docusaurs: Integrate Mendable in docs #4765
• Add Google Analytics to Docusaurus docs #4792

High Prio After Deploy

• Docusaurs: Setup the Algolia search engine #4741

Low Prio

• Docusaurs: Add more white space to the pages for readability #4575
• Docusaurs: Check & adjust the accessibility #4742
• Docusaurs: Validate the double npm package approach #4744
• Bug report: Docusaurus go to top button is hidden by mendable button #4806
• Docusaurs: Add GitHub stats counter to the navbar #4880
• Docusaurus: Fix syntax highlighting in code blocks #4883
• Docusaurs: Include Discord announcement bar #4931

Info:

For our docs, we will be changing the current Python-based SSG to a JavaScript one, Docusaurus to be more specific. This has been concluded from our recent discussion #4283 (comment).

Docusaurus comes with a bunch of out-of-box features that align well with our current docs setup. Pages are written in MarkDown, layered navigation, table of content for each page, and much more. We currently have a PoC running where you can look at the new UI/UX.

PoC

Regarding the PoC, an example of our docs is available at https://jwaegebaert.github.io/cli-microsoft365-docusaurus/
I've built it with version 6.1 and the source code is available in the repo https://github.com/Jwaegebaert/cli-microsoft365-docusaurus.

There are a few things to keep in mind when looking through the PoC.

• The search engine isn't enabled because we'll look into making our search engine with Azure Cognitive Search.
• The admonition syntax is different from MKDocs so they won't be rendered correctly.
• The same goes for the tabs we use with the command outputs but I've refactored all the Planner commands so be sure to take a peak there. planner bucket add
• Navigation has been modified a bit with landing pages for the categories. planner
• Navigation in mobile view doesn't look super clean yet, the same goes for the footer

ToDo

• Set up a search engine with Azure Cognitive Search. Ref: Migrate the docs to a JS based Static Site Generator #4283 (reply in thread)
• Resolve the admonition syntaxes. The admonition syntax is different from MKDocs so they won't be rendered correctly.
• Resolve the command output syntaxes. Take a look at planner bucket add to see how this has been achieved.
• Resolve the embeddings used for global options to the new syntax
• Add more white space to the pages for readability. Ref: Migrate the docs to a JS based Static Site Generator #4283 (reply in thread)
• Modify the footer UI
• Modify the mobile navigation
• Remove the category landing pages. Ref: Migrate the docs to a JS based Static Site Generator #4283 (reply in thread)
• Integrate with our Google Analytics (final step before going live)
• Update GitHub workflow to publish docs using Docusaurus
• Update the option descriptions for all the commands as they don't look right

[Jwaegebaert]

@pnp/cli-for-microsoft-365-maintainers What would be the best way to start working on this?
Shall we continue working on the PoC until it's ready to be swapped with our current docs, or create a docsBeta folder that we work out of?

[waldekmastykarz]

@pnp/cli-for-microsoft-365-maintainers What would be the best way to start working on this? Shall we continue working on the PoC until it's ready to be swapped with our current docs, or create a docsBeta folder that we work out of?

Are we looking into splitting the work or will one person do it all? If the latter, then we treat it as any other PR. If the former, then we need to think about the best way forward to be able to integrate everyone's work and preview in the meantime. While we could create a separate branch in the upstream, where multiple folks would submit PRs to, we'd need to pay attention to sync it regularly with main to avoid conflicts, and also adjust the GitHub workflow so that it doesn't build docs from it when merging PRs.

[waldekmastykarz]

Btw, I just found out that MkDocs uses Lunr for its search. Seems like there's a Docusaurus plugin as well https://www.npmjs.com/package/docusaurus-lunr-search which we could consider using instead of standing up Azure Cognitive Search.

[Jwaegebaert]

It will be best to split it up I think. Mainly, because it's a lot of work that needs to be done. We can look into automating several tasks but some tasks will be achieved quicker by doing it manually than by creating an automation process. My main concern at the moment is keeping our new docs up to date with our current ones during this process. With this I mean, when somebody updates the docs how we'll reflect this change in our current work? Anyway could be to create an issue for this specific task and that we, the maintainers, update it after we have merged some new PRs with related changes to the docs.

Btw, I just found out that MkDocs uses Lunr for its search. Seems like there's a Docusaurus plugin as well https://www.npmjs.com/package/docusaurus-lunr-search which we could consider using instead of standing up Azure Cognitive Search.

Isn't this a bit similar to Algolia DocSearch where we were worried about the terms of service for a third-party service?

[waldekmastykarz]

My main concern at the moment is keeping our new docs up to date with our current ones during this process. With this I mean, when somebody updates the docs how we'll reflect this change in our current work?

I think this is no different than working on a new major version and each time we merge something to main, we need to merge it to the next major version branch too and resolve any conflicts. The benefit here is that we can do this as fast as we can without any heads up because our docs are not an API.

Isn't this a bit similar to Algolia DocSearch where we were worried about the terms of service for a third-party service?

Not quite. When using Lunr, the search index is compiled into a JSON file that's deployed along the docs. When searching for things, the search is executed against the index without any third party service. It might be not as advanced as Azure Cognitive Search, but it's certainly 0-maintenance which in our case is a big pre.

[Jwaegebaert]

I think this is no different than working on a new major version and each time we merge something to main, we need to merge it to the next major version branch too and resolve any conflicts. The benefit here is that we can do this as fast as we can without any heads up because our docs are not an API.

That's a good plan, one thing we lose with this approach is the ability to preview our changes live. Then we just have to pull the changes from the branch and work from there, which shouldn't be a problem. Then, if it's alright, I'll set up a branch where we can work on Docusaurus. I think it will be best to protect this branch so we can review the PRs coming in.

Not quite. When using Lunr, the search index is compiled into a JSON file that's deployed along the docs. When searching for things, the search is executed against the index without any third party service. It might be not as advanced as Azure Cognitive Search, but it's certainly 0-maintenance which in our case is a big pre.

Well, that's perfect for our use case. We should definitely start with Lunr then!

---

I'll look into creating a script for all the docs pages. This script should cover the following:

• Resolve the admonition syntaxes. The admonition syntax is different from MKDocs so they won't be rendered correctly.
• Resolve the command output syntaxes. Take a look at planner bucket add to see how this has been achieved.
• Resolve the embeddings used for global options to the new syntax

These points will be the biggest tasks to do manually, so looking into scripting will save us a lot of time here. I'll create a separate issue for this task. Here we can discuss what tasks need to be done by the script to each '*.md' file.

[milanholemans]

We should also look for the indent we use right now for our option descriptions. I kinda like it that way.

[waldekmastykarz]

We should also look for the indent we use right now for our option descriptions. I kinda like it that way.

yeah, using the dt/dd syntax is very useful, especially when rendering in the console. If we can keep the formatting for the web, it would be awesome

[Jwaegebaert]

@pnp/cli-for-microsoft-365-maintainers, I've been tinkering with sorting out which tasks to focus on first. The idea is that once we finish everything that falls under the High Prio category, we can finally launch. How do you feel about how the tasks are distributed?

[milanholemans]

Shouldn't the Lunr search engine be listed under high prio?

[Jwaegebaert]

I've put it into low prio because we're looking into working with Mendable. It could be a bit of waste if we put in all the work to implement Lunr, and then a few weeks later we throw it all out.

[milanholemans]

Added an extra high prio:

• Docusaurs: Integrate Mendable in docs #4765

[milanholemans]

Moved search engine task to Low prio because we can only obtain an Algolia license when our docs are live.

[milanholemans]

Added another high prio

• Add Google Analytics to Docusaurus docs #4792

If I'm not mistaken this hasn't been implemented yet right @Jwaegebaert?

[milanholemans]

Is SEO also something we should configure? https://docusaurus.io/docs/seo

[Jwaegebaert]

Good catch. GA should definitely be implemented. Maybe we should also look to update to GA4 because they are looking into deprecating the version we use.

The SEO out-of-the-box should be fine. But it wouldn't hurt to do some research into this 😄

[Jwaegebaert]

As this has already been migrated it should be safe to close this epic.