This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

News About WikiTraccs

M365 Authentication, API Requests, and Blocked Endpoints

This post dives into API requests related to Microsoft 365 authentication, what happens when those are blocked, and how to test for that.

A recent customer support case required deeper insights into the API requests originating from WikiTraccs. A list of endpoints was required to whitelisting those at the proxy level.

While the WikiTraccs Endpoint Reference already covers endpoints, they weren’t detailed enough.

How to get more details?

I’ll describe how to use Microsoft’s Dev Proxy tool to get a list of endpoints.

Using Dev Proxy to Log API Requests

Dev Proxy is a small command line tool that creates a local proxy that can be used to log and change API requests of applications. It’s provided by Microsoft and free to use.

We’ll use Dev Proxy to log all API requests that WikiTraccs makes.

Once installed, we configure it. Configuration is done via a JSON file.

The following configuration instructs Dev Proxy to log all requests to the console:

{
    "$schema": "https://raw.githubusercontent.com/dotnet/dev-proxy/main/schemas/v0.24.0/rc.schema.json",
    "plugins": [
        {
            "name": "UrlDiscoveryPlugin",
            "enabled": true,
            "pluginPath": "~appFolder/plugins/dev-proxy-plugins.dll",
            "urlsToWatch": [
                "https://*/*"
            ]
        }
    ],
    "logLevel": "information",
    "newVersionNotification": "stable",
    "showSkipMessages": true,
    "showTimestamps": true
}

Save above text to devproxy.json.

Now we start Dev Proxy and instruct it to monitor WikiTraccs-related processes. Run the following command in a Terminal:

devproxy --config-file "C:\path\to\file\here\devproxy.json" --watch-process-names WikiTraccs.GUI WikiTraccs.Console conhost chromedriver chrome

Dev Proxy will now log all API requests done by WikiTraccs or the Chrome browser.

API Request for Authenticating with Microsoft 365

Since the customer that triggered this investigation reported problems when authenticating with Microsoft 365, let’s have a look at the API calls that go over the wire when doing that.

When hitting the Test SharePoint Connection button in the blue WikiTraccs.GUI window, the following endpoints are called:

  1. https://login.microsoftonline.com/common/discovery/instance?api-version=1.1&authorization_endpoint=https://login.microsoftonline.com/421cd8a4-daf6-434c-8eae-a685c9af1808/oauth2/v2.0/authorize
  2. https://login.microsoftonline.com/421cd8a4-daf6-434c-8eae-a685c9af1808/oauth2/v2.0/authorize?scope=https://contoso.sharepoint.com/.default+openid+profile+offline_access&response_type=code&client_id=b05e893b-866e-40d3-be10-75e44e5c38c2&redirect_uri=http://localhost:64189&client-request-id=bbc47ed0-c9e4-44e5-b1df-72c6589892e8&x-client-SKU=MSAL.NetCore&x-client-Ver=4.61.3.0&x-client-OS=Microsoft+Windows+10.0.22000&prompt=select_account&code_challenge=snip&code_challenge_method=S256&state=snip&client_info=1
  3. https://login.microsoftonline.com/421cd8a4-daf6-434c-8eae-a685c9af1808/reprocess?ctx=snip&sessionid=a11f9fda-ae88-4a93-8d8b-68a3dd193e7b
  4. https://login.microsoftonline.com/421cd8a4-daf6-434c-8eae-a685c9af1808/oauth2/v2.0/token
  5. https://contoso.sharepoint.com/sites/testing/_api/Web?$select=Id,Url,RegionalSettings/*,RegionalSettings/DateFormat&$expand=RegionalSettings,RegionalSettings/TimeZone
  6. … more calls of the SharePoint API

Note that

  • 421cd8a4-daf6-434c-8eae-a685c9af1808 is the SharePoint tenant ID
  • b05e893b-866e-40d3-be10-75e44e5c38c2 is Entra ID application client ID
  • https://contoso.sharepoint.com is the SharePoint tenant URL

Unblocking above endpoints should make the authentication succeed.

Simulating Blocked Connections

Keep reading if you want to know how to simulate slow network connections or incomplete proxy configuration.

This is one of the value propositions of Dev Proxy: you can see how applications behave in unexpected circumstances.

I’d like to emulate a proxy that delays responses of certain endpoints related to Microsoft 365 authentication.

Let’s extend the configuration file for Dev Proxy as follows:

{
    "$schema": "https://raw.githubusercontent.com/dotnet/dev-proxy/main/schemas/v0.24.0/rc.schema.json",
    "plugins": [
        {
            "name": "UrlDiscoveryPlugin",
            "enabled": true,
            "pluginPath": "~appFolder/plugins/dev-proxy-plugins.dll",
            "urlsToWatch": [
                "https://*/*"
            ]
        },
        {
            "name": "LatencyPlugin",
            "enabled": true,
            "pluginPath": "~appFolder/plugins/dev-proxy-plugins.dll",
            "configSection": "latencyPlugin1",
            "urlsToWatch": [
                "https://login.microsoftonline.com/common/discovery/instance*"
            ]
        },
        {
            "name": "LatencyPlugin",
            "enabled": true,
            "pluginPath": "~appFolder/plugins/dev-proxy-plugins.dll",
            "configSection": "latencyPlugin2",
            "urlsToWatch": [
                "https://login.microsoftonline.com/421cd8a4-daf6-434c-8eae-a685c9af1808/oauth2/v2.0/token*"
            ]
        }
    ],
    "latencyPlugin1": {
        "minMs": 180000,
        "maxMs": 180000
    },
    "latencyPlugin2": {
        "minMs": 86400000,
        "maxMs": 86400000
    },
    "logLevel": "information",
    "newVersionNotification": "stable",
    "showSkipMessages": true,
    "showTimestamps": true
}

This configuration file adds the LatencyPlugin that slows down responses for endpoints. It is added two times, for two different endpoints, to set two different delays.

Endpoint one is https://login.microsoftonline.com/common/discovery/instance - this gets a response delay of 3 minutes (180000 ms).

Endpoint two is https://login.microsoftonline.com/421cd8a4-daf6-434c-8eae-a685c9af1808/oauth2/v2.0/token - this gets such a high response delay that it is equivalent with the proxy stalling forever.

Running Dev Proxy with above configuration has the following effects on WikiTraccs, both when testing the SharePoint connection and when starting a migration:

  • the browser tab for logging in to SharePoint Online will only appear after a 3-minute delay
  • after (seemingly) successful authenticating with SharePoint Online, WikiTraccs will wait for 100 seconds and run into a timeout, never completing the authentication

While waiting, WikiTraccs will appear stuck, not giving any feedback at all. This is something that needs to be addressed in a future update.

PnP.PowerShell

A quick aside about PnP.PowerShell which you might use in Microsoft 365-related projects.

Under the hood, WikiTraccs uses the same authentication mechanism as PnP.PowerShell.

This is why the Dev Proxy configuration from the last section will also make PnP.PowerShell fail when connecting.

Here’s the PnP.PowerShell command that corresponds to what WikiTraccs does when connecting to SharePoint Online:

Connect-PnPOnline -Url https://contoso.sharepoint.com/sites/testing -LaunchBrowser -ClientId b05e893b-866e-40d3-be10-75e44e5c38c2 -Interactive -Tenant 421cd8a4-daf6-434c-8eae-a685c9af1808

In this command:

  • -Url https://contoso.sharepoint.com/sites/testing is the SharePoint site to connect to
  • -LaunchBrowser tells it to open a tab in a browser where you can re-use an existing login
  • -ClientId b05e893b-866e-40d3-be10-75e44e5c38c2 is the Entra ID application client ID; this can be the same you use for WikiTraccs
  • -Interactive says that you’ll sign in with account credentials
  • -Tenant 421cd8a4-daf6-434c-8eae-a685c9af1808 is the tenant ID to use; you can copy that from the blue WikiTraccs.GUI window

So, if you ever want to test how your PowerShell scripts behave in “special” circumstances - you now know how.

Wrap

In this post we looked at the API endpoints that WikiTraccs uses when logging in to Microsoft 365 and how to assemble this list using Microsoft’s Dev Proxy tool.

As an aside we also looked at how to use Dev Proxy to simulate “special” network conditions and hint at how to test PowerShell scripts under similar conditions.

Migrating Confluence Cloud Whiteboards

This post provides information about whiteboard migrations and how they are expected to look like.

Whiteboard migration can be enabled in the WikiTraccs settings:

This will export Confluence whiteboards as image to SharePoint.

It’s a Gray Setting which means it might not be fully supported by a vendor.

Whiteboard Export is Not Supported by Atlassian

Let’s get that out of the way. Exporting whiteboards in a programmatic, automated manner is not possible at the moment.

The only way to export a whiteboard as image is to click the right buttons in the browser.

Here is one of the many whiteboard-related issues in the Atlassian Jira: Ability to export/import whiteboards (CONFCLOUF-76634). Please vote.

So…

How Does WikiTraccs Export Whiteboards?

WikiTraccs can export whiteboards as images.

It does it exactly like any user would do: by clicking the right buttons.

That means, WikiTraccs knows how the current Confluence Cloud user interface looks, how to find the right buttons, and how to click them, to start the image export.

After clicking the export button, it grabs the exported image from the download folder and that’s it.

Sounds rickety? It kind of is.

The export works, as long as the user interface does not change.

It can break if Atlassian changes the user interface of Confluence Cloud.

This video shows how it looks like, when WikiTraccs exports a whiteboard (32 seconds):

The following happens in this video:

  • WikiTraccs discoveres a whiteboard to migrate
  • WikiTraccs opens a remote-controlled browser and navigates to the whiteboard (note: I had to move the browser window over from the second monitor)
  • WikiTraccs clicks all the right buttons to export the whiteboard
  • Chrome saves the downloaded image as Whiteboard.png
  • WikiTraccs migrates Whiteboard.png to SharePoint, as page attachment

Everything that happens inside the browser is done by WikiTraccs, there was no user interaction (apart from moving the browser into view).

The resulting SharePoint page shows the exported image:

(Note: the whiteboard image can be huge, depending on the size of the original whiteboard. Clicking the image in SharePoint will expand it. It can also be downloaded to view it in full size.)

What Could Go Wrong?

As long as Atlassian doesn’t change the Confluence Cloud UI in a way that changes the buttons needed for the export, the process is pretty stable (although quite slow). So far, it broke about once in three months.

What sometimes happens is that remote-controlling the Chrome browser window fails for some reason. Might be a Chrome update, might be some Windows shenanigans. If that is the case, close WikiTraccs and all browser windows and restart the migration.

What sometimes happens is that Atlassian is showing some advertisement for a new feature in the browser. Often, this ad is an overlay that is being shown until you manually dismiss it by clicking the X or Cancel. This might interfer with the export.

If there is an overlay or dialog being shown that blocks the export, simply close it. The whiteboard migration might continue, but restarting might be necessary as well.

If the export breaks, WikiTraccs probably will require an update.

Under the Hood

To find the right button to click in the Confluence Cloud user interface, WikiTraccs uses CSS and XPath selectors.

You find those in a TOML file that is located at WikiTraccs.Console\Templates\BrowserAutomation\export-whiteboard.toml.

It contains selectors like this one:

[selectors.three-dot-menu]
description = "Three-dot menu in the upper right corner of a page"
language = "css"
query = "#more-actions-trigger"

This selector describes how to find the three dot menu button for a Confluence page.

There is a chance that, if whiteboard export breaks due to user interface changes, adjusting those selectors can fix the issue.

This approach to user interface automation is often used when programmatically testing user interfaces of web applications. WikiTraccs makes use of that approach to export whiteboards.

Authentication

Since WikiTraccs needs to open the whiteboard in the browser, this approach only works with Interactive Login where an account is logged in to Confluence to look at the whiteboard. It won’t work with Personal Access Tokens.

Wrap

In this blog post we looked at the current state of whiteboard exportability in Confluence Cloud, at how WikiTraccs does it and how it could break.

We all agree that this is a temporary solution and we hope that Atlassian provides an official export API in the near future (although I personally don’t believe that).

Harnessing the New SharePoint Page Format

This post hightlights the changes Microsoft did to SharePoint pages over in 2024 and how WikiTraccs adapts to that.

In 2024, Microsoft upgraded the internal page format for SharePoint modern pages. The new format comes with an updated page editor and new formatting capabilities.

This change rolled out over the course of 2024 and now all new SharePoint pages use the new format.

But what about older pages?

How Microsoft Upgrades Older Pages

When a SharePoint page is edited in the browser, a (Microsoft-owned) script processes the page and - if required - upgrades it to the new page format.

This page upgrade happens only when a page is edited in the browser, so you can see the result immediately. During the upgrade, the styling of page elements might change.

What Microsoft doesn’t do is bulk-upgrading pages in the SharePoint backend. The upgrade only happens on a page-per-page basis when editing a page.

Here’s a sample of how the page upgrade looks:

This is what happens in the video:

  1. a SharePoint page is open in the browser
  2. the user clicks the Edit button to edit the page
  3. the content of the text web part briefly disappears
  4. the content of the text web part reappears, now upgraded

This video also illustrates one of the older (and now resolved) issues that could happen during page upgrade (the table ends up way too narrow).

Nowadays SharePoint also shows a progress message:

Page Upgrade Issues

There were several issues with the way Microsoft upgraded pages that were created in the pre-2024 format, e.g. with tables looking awful after the upgrade (as can be seen in above video), or lists that couldn’t contain images anymore. Most issues seem resolved.

One remaining issue affects larger pages with images.

When upgrading a SharePoint page, the upgrade script converts embedded images to a new format. Unfortunately, this process sometimes fails.

What has usually helped so far is to wait a bit and avoid scrolling while the page upgrade is in progress. Alternatively, trying a different browser or closing and reopening the browser before editing a long page can help (I suspect memory might be an issue during page upgrade).

Essentially, keep trying until it works.

Before each new try, always revert the page to the previous page version. Reverting to the previous version makes the images reappear. After that, the page can be edited again, and hopefully, the upgrade script will complete successfully.

A Microsoft support ticket might also help, but I’ve had to do this a few times with regard to SharePoint pages, and it’s usually a slow process with moderate success. So far repeated attempts always resolved the issue.

What’s New in the New Pages?

In my view, the Text web part is the one that gets the most out of this page upgrade.

It gained the following new capabilities:

  • line height settings
  • more versatile spacing settings (space before, space after)
  • new color options
  • different styles for citations
  • … and more

Overall, the new pages appear visually more balanced and appealing than their older counterparts.

How Does WikiTraccs Handle Old and New Pages?

WikiTraccs up until version v1.23 creates pages in the older format.

Going forward, since the new SharePoint page format seems to have stabilized, WikiTraccs will create pages in the new format.

Creating pages in the new format is opt-in.

How to Opt-in to New Pages?

Use WikiTraccs release v1.24.0 or newer and make sure to uncheck Create SharePoint pages with pre-2024 type in the WikiTraccs Settings:

While creating pages in the new format is currently an opt-in setting, it will be the default in a future WikiTraccs release.

For now, you have to explicitly choose to have new pages created.

Wrap

Creating SharePoint pages in the new page format is a big change for WikiTraccs and allows using the full set of features that those pages bring.

Please report any issues that you see with the new pages.

Splitting a Space

This post explains how to split a space and migrate the parts to different target sites.

Assume the following use case where a space needs to be split into multiple parts:

“We have a Confluence space that contains information contributed by seven different teams. Each team’s content needs to be migrated to a separate SharePoint site.”

Let’s assume that all pages belonging to one team have a common parent page. That’s 7 parent pages for 7 teams.

How to migrate each of the 7 parent pages and all their decendants from Confluence to 7 different target sites in SharePoint?

With WikiTraccs, there are two approaches to selecting parts of a space for a migration:

  • the Content ID selector
  • the CQL selector

We’ll look at the Content ID selector in depth.

Use the Content ID Selector

With the Content ID selector you give WikiTraccs a list of page IDs to migrate.

How you assemble the list of page IDs is up to you. With Confluence Server and Data Center a common approach is via database query, in the Cloud other approaches are warranted.

In this section we are looking at:

  • a sample tree to illustrate migrating a subtree of pages
  • how to assemble a list of page IDs from the Confluence database
  • how to properly format those page IDs
  • how to create a Content ID selector entry in the Confluence Space Inventory list
  • how the Space Inventory looks after configuration

One way to get page IDs is from the Confluence database using SQL queries.

Consider the following page tree. How to get all page IDs of the Special Content page and all its descendants (because we want to migrate those)?

First we need the starting point, the page ID of the Special Content page, which is the root of the subtree.

To get the page ID for any page, open the Page Information page:

Then look at the browser bar that should show something like this: https://contoso.com/confluence/pages/viewinfo.action?pageId=65604.

The subtree root page ID in our case is 65604.

Now we can run the following SQL query on the Confluence database to get the list of descendants:

SELECT c.contentid,
       c.title
FROM content c
FULL OUTER JOIN confancestors ca ON ca.descendentid = c.contentid
WHERE ca.ancestorid = '65604'
    AND c.content_status = 'current'
    AND c.prevver IS NULL;

The result might look like this, but could also look different, depending on the tool that is being used:

WikiTraccs expects those IDs to be in a specific format, though. We use an adjusted SQL query to get the list of IDs in the expected format:

SELECT count(*), string_agg(contentId || ';#' || contenttype, ',') AS contentIdSelectorValue
FROM (
	SELECT c.contentid,
		   LOWER(contenttype) AS contenttype,
		   c.title
	FROM content c
	FULL OUTER JOIN confancestors ca ON ca.descendentid = c.contentid
	WHERE ca.ancestorid = '65604'
		AND c.content_status = 'current'
		AND c.prevver IS NULL
	ORDER BY contenttype, contentId
) subquery;	

The result now looks different:

This result contains the number of pages (16) and the list of IDs, including their type, in the expected format:

65608;#page,65611;#page,65635;#page,65637;#page,65640;#page,65642;#page,65644;#page,65646;#page,65648;#page,65650;#page,65652;#page,1179651;#page,5472257;#page,5472259;#page,5472261;#page,17924097;#page

We can now create the content ID selector in the Confluence Space Inventory.

Click Add new item to invoke the respective dialog:

You need to fill the following fields:

FieldMeaningSample Value
TitleNot used, but it’s a mandatory field; enter anythingdummy
WT_In_CfSiteIdConfluence base address; look at the other entries in the space inventory list to see what to enterhttps://contoso.com/confluence
WT_Setting_RequestTransformationCheck this box to migrate this selector
WT_Setting_ContentSelectorValueThe list of IDs in the expected format65608;#page,65611;#page,​65635;#page,65637;#page,​65640;#page,…
WT_Setting_TargetSiteRootUrlSharePoint target site for migrated pageshttps:​//contoso.sharepoint.com/​sites/ContentIDSelectorDemoTargetONE

All other values can be left empty, as those are not required for the Content ID selector.

The space inventory list, after adding two Content ID selectors, might look like this:

Two things are important as well:

  • Don’t forget to add the ancestor page to the ID list of the selector as it is NOT part of the SQL result above.
  • Remove the entry for the space your are splitting from the space inventory. Otherwise link translation might get confused and link to the wrong target site, as pages would be included in two selectors at the same time.

Now start a migration.

WikiTraccs should migrate the pages indicated by each Content ID selector to the respective target page. Links will be transformed as well, based on the target site URL configuration.

Use the CQL Selector to Choose Pages for Migration

The CQL selector works similar to the Content ID selector, but instead of giving WikiTraccs a list of page IDs you give it a CQL query - a Confluence search query.

Using a CQL selector works just like the Content ID selector:

  1. create a new item in the space inventory list
  2. enter field values as described in the last section; there’s only one difference
  3. instead of the list of IDs you enter the CQL query into the WT_Setting_ContentSelectorValue field

Using a CQL query, you can refer to Confluence content by criteria you define. Let’s create a simple CQL query that refers to all pages that have the label migration set:

The CQL query that finds those pages is label = migration.

You should always test that your CQL query works, before using it as a selector for WikiTraccs. You’ll spot errors in the query much faster.

Testing CQL queries in Confluence is a bit cumbersome as you have to use the REST API. Assuming your Confluence base address is https://contoso.com/confluence, the address for testing above CQL query is https://contoso.com/confluence/rest/api/content/search?cql=label = migration. Navigate to this address and the result should show a list of pages in JSON format:

Have a look at the CQL selector documentation to learn more details and read about some caveats associated with this selector type.

Configuring the Page Tree in SharePoint

If you choose to migrate a subtree of pages and you opted to add the WikiPakk page tree to the game, it will work well.

After migrating a subtree, open the migrated root page. Then open the page tree panel and click Move to tree root.

The subtree root will now always be shown when opening the page tree panel.

Note that this only works when migrating a subtree as this will also migrate the page hierarchy within the subtree. Migrating a list of arbitrarily picked pages won’t result in a proper tree in SharePoint.

Testing WikiTraccs for Markdown

This post documents the current state of WikiTraccs for Markdown, highlights a sample repository, and describes how to test publishing

The last blog post provided some background on how WikiTraccs for Markdown came to be.

This blog post shows how to publish Markdown files to SharePoint Online.

Sample Repository

We use a sample knowledge repository to test publishing Markdown to SharePoint Online. There is one available in our library: AI for Beginners Sample Repo.

This repository is a copy of one of Microsoft’s repositories about AI and contains lots of Markdown files, images, and folders:

This is not an ideal structure, but taking something “from the wild” seems like a good test. WikiTraccs will make educated guesses about IDs, parent-child relationships and so on.

For testing, download the whole library repository as zip file and extract it. Then use the ai-for-beginners-sample-repo folder as base address for your test.

Publishing Markdown to SharePoint Online

Starting with release v1.23.6, WikiTraccs supports file pathes as “Confluence” base address.

Enter the path to the ai-for-beginners-sample-repo folder:

Make sure to activate the Skip Connection Checks setting as those are not required:

Then proceed as if you would migrate from Confluence:

  1. Update the space inventory
  2. Choose sources in the space inventory
  3. Start the migration

WikiTraccs should now start publishing Markdown to SharePoint Online.

Limitations and Work Items

This is a non-exhaustive list of things that need to be done before WikiTraccs for Markdown is ready for production use:

  • auto-disable connection checks for Markdown source
  • don’t use the folder path as source “Site ID”, as this makes migrating from different migration machines impossible
  • optimize speed for larger repositories; currently it’s rather slow
  • the page hierarchy is not yet properly migrated for the sample repo
  • adjust the user interface to be “Confluence-free”
  • figure out where to put images and files that are referenced by multiple pages; images currently are handled as “external” and duplicates are created per referencing page
  • provide documentation on the expected source repository format and possible configuration options
  • figure out a good format for IDs (numbers? GUIDs?) and how to make maintenance easy (e.g. preventing duplicates)

And probably more.

Finding the Confluence Version

How to find the version of a Confluence instance or site.

Time and again the question comes up: which Confluence version am I running on?

One way of finding the Confluence version is as easy as scrolling down on any Confluence page. The version might be written there:

If you don’t see any version information in the page footer you might be using a theme that hides that information.

You can always resort to the /rest/applinks/1.0/manifest endpoint that shows the Confluene version as well:

  • if your Confluence is available at https://confluence.contoso.com the address to navigate to would behttps://confluence.contoso.com/rest/applinks/1.0/manifest
  • if your Confluence is available at https://contoso.com/confluence the address to navigate to would behttps://contoso.com/confluence/rest/applinks/1.0/manifest

Navigate to the address in a browser where you are logged-in to Confluence and the page that opens shows the Confluence version:

Here the Confluence version would be 8.7.1.

This also works in Confluence Cloud. The version number for Confluence Cloud always start with 1000.

WikiTraccs for Markdown

This post introduces WikiTraccs for Markdown

When demoing Confluence to SharePoint migrations with WikiTraccs, I sometimes get approached with something different in the context of a broader strategy shift.

Client are consolidating their knowledge repositories to cut costs, reduce complexity, and to be AI-ready.

In-house CMS systems are being sundowned, Confluence content is being moved to SharePoint Online (but only for so long).

One goal is to break free of any specific system for documentation and knowledge management, while still having the choice to publish to any service to make information available to users.

Clients are switching to Markdown.

So far, WikiTraccs transforms Confluence pages to SharePoint pages.

Here’s the question that I get asked: Can WikiTraccs also transform Markdown to SharePoint pages?

Soon, the answer will be Yes.

Publish Markdown to SharePoint Online

WikiTraccs for Markdown will be your tool to publish from Markdown to SharePoint.

Many other (free) tools are available to create, edit, and publish Markdown. WikiTraccs will help you with publishing to SharePoint Online.

I’m currently gathering information on how a Markdown-based knowldge base repository can compete with third-party services like a Confluence Enterprise wiki. Read more about that here: Strategy Shift to Text Files for Knowledge Repositories.

Proof of Concept

WikiTraccs for Markdown is currently in a proof of concept phase.

It can already pick up Markdown files from the files system and publish those to SharePoint Online, including page attachments and images.

WikiTraccs can already publish this Microsoft AI starter repository to SharePoint: AI for Beginners.

Here’s a page from that GitHub repository, as shown on GitHub:

And here’s the SharePoint page published by WikiTraccs:

Both pages are based on this Markdown file: README.md.

The Proof of Concept already raised some questions, among those are:

  • What’s a common layout for the Markdown-based knowledge repository?
  • How many pages and files might such a repository contain?
  • How are attachments and pages related; how common is image-reuse?
  • Is page attachments still the way to go in SharePoint, or should attachments be stored in separate Document Libraries?
  • While Markdown files are perfect for being stored in a version control system, how about large attachment files?

Get in touch if you’d like to provide your thoughts or want to learn more.

Making SharePoint tables look pretty

This post shows how to use colored and merged table cells in SharePoint Online tables, and how to enable this for WikiTraccs.

You cannot create tables in SharePoint Online that have colored or merged cells.

(The blog post could end here.)

But this is not entirely true. What is true is that, when creating a new SharePoint page, there are no tools that you could click in the text web part’s tool bar. There is no “merge cells” button and no color picker for cell backgrounds.

Nevertheless, it is possible to copy a table from Microsoft Word and paste it to a SharePoint page. Now, if that is officially supported (and I don’t know if it is), WikiTraccs could use the underlying mechanism as well.

This is why WikiTraccs introduces the Use non-standard table transformation setting, and we’ll look at why it’s named like that further down.

Copying tables from Word to SharePoint preserves styles

Take this table for example, created in Word Online:

This table can be copied to the clipboard:

And from the clipboard, the table can now be pasted into a SharePoint page, preserving its style:

Apparently copying and pasting a table from Word carries over cell colors and merged cells. And even more important: those styles stay in place when editing and saving the SharePoint page.

I had multiple clients reach out to me asking for this to be added to WikiTraccs.

WikiTraccs adds support for cell colors and merged cells

Note: this is available as of WikiTraccs v1.22.5.

WikiTraccs can now transform Confluence tables such that cell colors and merged cells are preserved.

This feature is opt-in, you have to activate it in the WikiTraccs settings.

Here’s how to activate the feature:

  1. in the blue WikiTraccs.GUI window, click Settings in the menu bar to open the Settings dialog
  2. in the Settings dialog, in the Migration tab, check the option Use non-standard table transformation

Is this supported by Microsoft?

I don’t know if those copied-over tables are officially supported, as end users can not create them in the text web part editor. Copy and pasting seems to be the only way.

At the same time, I heard from clients that they use this functionality and also that this seemed to be advertised by Microsoft as a feature some time ago.

Why am I cautious?

I’m cautious because page content that cannot be created in the browser editor, by the user, by clicking buttons, might cease to work with a future update (by Microsoft) of the text web part. This might also affect how pages behave when being edited; styles might change or content even disappear.

Will that be the case in the future? I don’t know. I hope not.

Is there a way out if something happens in the future? There certainly is, by modifying page contents via PowerShell.

As long as there is this uncertainty, the setting is placed in the Gray Settings area of the Settings dialog, and it carries the non-standard hint in its name.

Visual comparison

Here’s a colorful Confluence table that already appeared in a blog post:

Table in Confluence

Here’s how that looks in SharePoint with the new setting enabled:

Table in SharePoint, now with colored cells

And here’s how the table would normally look, with the new setting disabled:

Table in SharePoint, new setting disabled

Another Confluence table that uses merged cells:

Table in Confluence

Here’s how that looks in SharePoint with the new setting enabled:

Table in SharePoint, now with merged cells

Here’s how the table would normally look, with the new setting disabled:

Table in SharePoint, new setting disabled

Note that in above image the presence and the direction of cell merging is hinted at by arrows.

Overall, real merged cells are a lot more pleasant to look at.

Wrap

In this blog post we looked at a way to create tables with colored and merged cells in SharePoint Online modern pages, by copying and pasting styled tables from Word.

We also looked at a new WikiTraccs setting that can leverages this behavior.

There is no documentation by Microsoft (that I know of) that describes this as an official feature and words of caution have been written.

If somebody can find official documentation by Microsoft that this is an actual feature that is officially supported, please let me know.

How much time will a Confluence to SharePoint migration take?

This post looks into one of the most asked questions: how long will the migration take?

The migration time depends heavily on the source content in Confluence, and the network speed, both for downloads and uploads.

We’ll do some test migrations to have real-world data to look at.

Summary (TL;DR)

Here are the average migration times per page, taken from test migrations detailed further down this post:

  • 07 seconds (~2h per 1000 pages) (note: few attachments)
  • 11 seconds (~3h per 1000 pages)
  • 12 seconds (~3h per 1000 pages)
  • 12 seconds again
  • 13 seconds (~4h per 1000 pages)
  • 13 seconds again
  • 15 seconds (~4h per 1000 pages)
  • 20 seconds (~6h per 1000 pages)
  • 35 seconds (~9h per 1000 pages) (note: lots of attachments)

Read further to get details.

How does the source content influence migration speed?

The following list shows migration activities that are always present and can take quite some time:

  • creating a page in SharePoint and setting up the metadata around it
  • uploading files to SharePoint Online (mainly on slow connections)
  • downloading files from Confluence (mainly on slow connections)

Here are some activities that also take time, but are not always needed, depending on the source content:

  • resolving Confluence links - WikiTraccs will check every link it finds to know where the link links to, to properly transform the link
  • resolving @-mentions - this involves looking up user information, writing this to the user mapping table, and trying to find the user in SharePoint
  • transforming Jira macros - this involves reaching out to Jira; Confluence’s Jira integration is rather slow

If you migrate pages with only text content and no attachments they’ll migrate in no time.

If you migrate pages with 500 attachments each, it might take minutes for each page to be migrated.

Using live migration metrics to get insights

With release v1.22.1, WikiTraccs introduced live metrics that show how long migrating pages take (those are also written to the log file). Those metrics also provide insights into which of the mentioned activities - downloading and uploading content, resolving links, etc. - happen, and how much time they take.

You can read more about those live metrics here: Live Progress Indicators.

Since the migration times depend on the source content, we need some source content to perform actual migrations, to get actual data to look at.

Thankfully, there are plenty of open Confluence sites available to run some measurements; some run by hobbyists, many by universities, some by non-profits and NGOs.

Testing migrations using open Confluence sites

We’ll pick some of those open Confluence sites, not based on the meaning of their content, but on the amount and type of their content. I’m not affiliated with any of those sites. They just provide a great way to test one thing or another.

Note: over time the accessibility of those sites will change. Some will close, some will move. The number of Confluence on-prem instances decreases, the number of Confluence cloud sites increases.

Again: those sites are purely chosen by googling for site:atlassian.net and clicking through sites until finding some that have a decent amount of spaces and pages.

Migration machines used for test migrations

Test migrations were performed on the following migration machines:

“Mig VM”

An Azure VM with the following characteristics:

  • Operating System: Windows 10
  • Size: B2s
  • vCPUs: 2
  • RAM: 4 GB
  • Max IOPS: 1280
  • Cost per month: €32.80
  • Download speed: 1200 Mbps
  • Upload speed: 1800 Mbps

“Mig Laptop”

A laptop:

  • Operating System: Debian Linux
  • vCPUs: 6
  • RAM: 40 GB
  • Download speed: 93 Mbps
  • Upload speed: 38 Mbps

Download and upload speed were determined using one of the many free internet sites that measure those values.

Confluence Cloud Site: University of Texas at Austin

The UT is providing us with an site that has over 300 spaces, many small, but two bigger ones.

Migration Metrics for Space 1

  • Migration Machine: Mig VM
  • Migrated number of pages: 801
  • Migration duration: 92 minutes

Here are the metrics (as logged into the common log files) 1 hour into the migration, based on data of this hour, covering 410 pages:

Median time (IQR)    : 4 s/item
Median time          : 4 s/item
75th percentile (IQR): 7 s/item
75th percentile      : 9 s/item
Mean time (IQR)      : 5 s/item
Mean time            : 8 s/item
Avg links transformed: 2/item
Have files           : 22% of items
File count avg       : 2.3 per item that has files
File size avg        : 1649 KB/file
Download speed       : ~485 KB/s
Upload speed         : ~373 KB/s
File size sum down   : 1299.6 MB
File size sum up     : 1293.5 MB

Top activities were:

(SharePoint, Page, Content)                                 : 28,52 min
(SharePoint, WikiTraccs, Prerequisites)                     : 14,98 min
(Confluence, Page, Link_Soft)                               : 14,69 min

There are only a few attachments in this space, only every fifth page has about 2 attachments. This is probably the reason why file upload and download are not in the top activities.

Most present are activities that require a lot of calls to APIs, like checking prerequisites, prepping the page, setting its content, and setting metadata for page and attachment folder.

In the end, looking at the overall migration duration and number of pages, the average migration duration was 7 seconds per page.

Migration Metrics for Space 2

  • Migration Machine: Mig VM
  • Migrated number of pages: 1233
  • Migration duration: 12h 10m

Here are the metrics (as logged into the common log files) 1 hour into the migration, based on data of this hour, covering 140 pages:

Median time (IQR)    : 11 s/item
Median time          : 13 s/item
75th percentile (IQR): 27 s/item
75th percentile      : 31 s/item
Mean time (IQR)      : 17 s/item
Mean time            : 25 s/item
Avg links transformed: 15/item
Have files           : 56% of items
File count avg       : 13.6 per item that has files
File size avg        : 1235 KB/file
Download speed       : ~297 KB/s
Upload speed         : ~218 KB/s
File size sum down   : 425.6 MB
File size sum up     : 427.5 MB

Top activities were:

(SharePoint, File, Content)                                 : 17.19 min
(SharePoint, Page, Content)                                 : 10.21 min
(Confluence, File, Content)                                 : 9.86 min

Compared with the first space, pages in this space have a lot of attachments. Half of the pages have an average of 13 attachments, each about 1,5 MB in size. That’s why the download and upload activities take more time in this space. All page metrics that measure time are about triple of what they were for the first space.

Remember, those values are measured over a period of 60 minutes. Let’s look at how the values changed after 4 hours of migration, covering 89 page-like contents:

Median time (IQR)    : 29 s/item
Median time          : 29 s/item
75th percentile (IQR): 47 s/item
75th percentile      : 50 s/item
Mean time (IQR)      : 35 s/item
Mean time            : 40 s/item
Avg links transformed: 3/item
Have files           : 91% of items
File count avg       : 19.6 per item that has files
File size avg        : 499 KB/file
Download speed       : ~495 KB/s
Upload speed         : ~357 KB/s
File size sum down   : 919.3 MB
File size sum up     : 957.0 MB

Top activities were:

(SharePoint, File, Content)                                 : 24.83 min
(Confluence, File, Content)                                 : 14.17 min
(SharePoint, Page, Content)                                 : 7.61 min

All times went up. The number of pages having attachments went from 56% to 91% - those file downloads and uploads take time.

In the end, looking at the overall migration duration and number of pages, the average migration duration was 35 seconds per page.

Confluence Cloud Site: Geocaching.com Wiki

This site had one space.

Migration Metrics for Space

  • Migration Machine: Mig Laptop
  • Migrated number of pages: 262 (whole site)
  • Migration duration: 53 minutes

Metrics for all 262 pages:

Median time (IQR)    : 5 s/item
Median time          : 5 s/item
75th percentile (IQR): 12 s/item
75th percentile      : 14 s/item
Mean time (IQR)      : 8 s/item
Mean time            : 12 s/item
Avg links transformed: 4/item
Have files           : 35% of items
File count avg       : 2.5 per item that has files
File size avg        : 382 KB/file
Download speed       : ~173 KB/s
Upload speed         : ~96 KB/s
File size sum down   : 234.7 MB
File size sum up     : 234.7 MB

Top activities were:

(SharePoint, Page, Content)                                 : 22.15 min
(Confluence, Page, Link_Soft)                               : 12.91 min
(Confluence, Page, Transformation_Macro_Other)              : 12.12 min

In the end, looking at the overall migration duration and number of pages, the average migration duration was 12 seconds per page.

Confluence Cloud Site: Xilinx Wiki

This site had one space.

Migration Metrics for Space

  • Migration Machine: Mig VM
  • Migrated number of pages: 1284 (whole site)
  • Migration duration: 7h 22m

Here are the metrics (as logged into the common log files) 1 hour into the migration, based on data of this hour, covering 221 pages:

Median time (IQR)    : 10 s/item
Median time          : 10 s/item
75th percentile (IQR): 18 s/item
75th percentile      : 21 s/item
Mean time (IQR)      : 14 s/item
Mean time            : 15 s/item
Avg links transformed: 7/item
Have files           : 61% of items
File count avg       : 6.7 per item that has files
File size avg        : 1419 KB/file
Download speed       : ~724 KB/s
Upload speed         : ~434 KB/s
File size sum down   : 1264.3 MB
File size sum up     : 1264.3 MB

Top activities were:

(SharePoint, File, Content)                                 : 22.25 min
(Confluence, Page, Transformation_Macro_Other)              : 20.55 min
(SharePoint, Page, Content)                                 : 15.97 min

In the end, looking at the overall migration duration and number of pages, the average migration duration was 20 seconds per page.

Confluence Cloud Site: Duke Universtiy

This site had 120 spaces.

Migration Metrics for whole Site

  • Migration Machine: Mig Laptop
  • Migrated Migrated number of pages: 3117 (whole site)
  • Migration duration: 12h 46m

Here are the metrics (as logged into the common log files) 1 hour into the migration, based on data of this hour, covering 330 pages:

Median time (IQR)    : 5 s/item
Median time          : 5 s/item
75th percentile (IQR): 11 s/item
75th percentile      : 13 s/item
Mean time (IQR)      : 8 s/item
Mean time            : 10 s/item
Avg links transformed: 2/item
Have files           : 43% of items
File count avg       : 2.6 per item that has files
File size avg        : 421 KB/file
Download speed       : ~239 KB/s
Upload speed         : ~104 KB/s
File size sum down   : 463.5 MB
File size sum up     : 463.5 MB

Top activities were:

(SharePoint, Page, Content)                                 : 23.05 min
(SharePoint, File, Content)                                 : 15.38 min
(SharePoint, WikiTraccs, Prerequisites)                     : 11.04 min

Let’s look at how the values changed after 4 hours of migration, covering 110 page-like contents:

Median time (IQR)    : 14 s/item
Median time          : 14 s/item
75th percentile (IQR): 59 s/item
75th percentile      : 59 s/item
Mean time (IQR)      : 32 s/item
Mean time            : 32 s/item
Avg links transformed: 0/item
Have files           : 65% of items
File count avg       : 17.8 per item that has files
File size avg        : 399 KB/file
Download speed       : ~444 KB/s
Upload speed         : ~212 KB/s
File size sum down   : 567.1 MB
File size sum up     : 587.2 MB

Top activities were:

(SharePoint, File, Content)                                 : 35.10 min
(Confluence, File, Content)                                 : 15.23 min
(SharePoint, Page, Content)                                 : 10.87 min

In the end, looking at the overall migration duration and number of pages, the average migration duration was 15 seconds per page.

Confluence Cloud Site: Harvard University

This site had 46 spaces.

Migration Metrics for whole Site

  • Migration Machine: Mig VM
  • Migrated Migrated number of pages: 6647 (whole site)
  • Migration duration: 20h 23m

Here are the metrics (as logged into the common log files) 1 hour into the migration, based on data of this hour, covering 211 pages:

Median time (IQR)    : 7 s/item
Median time          : 9 s/item
75th percentile (IQR): 15 s/item
75th percentile      : 17 s/item
Mean time (IQR)      : 10 s/item
Mean time            : 17 s/item
Avg links transformed: 6/item
Have files           : 50% of items
File count avg       : 4.2 per item that has files
File size avg        : 822 KB/file
Download speed       : ~371 KB/s
Upload speed         : ~199 KB/s
File size sum down   : 1200.9 MB
File size sum up     : 1200.9 MB

Top activities were:

(SharePoint, Page, Content)                                 : 14.74 min
(SharePoint, File, Content)                                 : 13.92 min
(Confluence, Page, Link_Soft)                               : 10.44 min

Let’s look at how the values changed after 4 hours of migration, covering 325 page-like contents:

Median time (IQR)    : 4 s/item
Median time          : 4 s/item
75th percentile (IQR): 9 s/item
75th percentile      : 11 s/item
Mean time (IQR)      : 7 s/item
Mean time            : 11 s/item
Avg links transformed: 2/item
Have files           : 35% of items
File count avg       : 2.6 per item that has files
File size avg        : 558 KB/file
Download speed       : ~165 KB/s
Upload speed         : ~105 KB/s
File size sum down   : 340.8 MB
File size sum up     : 340.7 MB

Top activities were:

(SharePoint, Page, Content)                                 : 22.18 min
(SharePoint, File, Content)                                 : 12.34 min
(SharePoint, WikiTraccs, Prerequisites)                     : 11.77 min

In the end, looking at the overall migration duration and number of pages, the average migration duration was 11 seconds per page.

Confluence Cloud Site: Hyperledger Foundation

This site had 74 spaces.

Migration Metrics for whole Site

  • Migration Machine: Mig VM
  • Migrated Migrated number of pages: 9238 (whole site)
  • Migration duration: 32h 40m

Here are the metrics (as logged into the common log files) 1 hour into the migration, based on data of this hour, covering 390 pages:

Median time (IQR)    : 7 s/item
Median time          : 7 s/item
75th percentile (IQR): 9 s/item
75th percentile      : 9 s/item
Mean time (IQR)      : 7 s/item
Mean time            : 9 s/item
Items per hour       : 400-514 (based on IQR Median and IQR 75th percentile)
Avg links transformed: 3/item
Have files           : 62% of items
File count avg       : 1.1 per item that has files
File size avg        : 294 KB/file
Download speed       : ~679 KB/s
Upload speed         : ~94 KB/s
File size sum down   : 1618.1 MB
File size sum up     : 1619.3 MB

Top activities were:

(Confluence, Page, Link_Soft)                               : 53.22 min
(SharePoint, Page, Content)                                 : 29.33 min
(Confluence, Page, Transformation_UserMention)              : 17.99 min

Let’s look at how the values changed after 4 hours of migration, covering 239 page-like contents:

Median time (IQR)    : 14 s/item
Median time          : 14 s/item
75th percentile (IQR): 18 s/item
75th percentile      : 18 s/item
Mean time (IQR)      : 14 s/item
Mean time            : 15 s/item
Items per hour       : 200-257 (based on IQR Median and IQR 75th percentile)
Avg links transformed: 4/item
Have files           : 53% of items
File count avg       : 1.6 per item that has files
File size avg        : 370 KB/file
Download speed       : ~102 KB/s
Upload speed         : ~81 KB/s
File size sum down   : 93.1 MB
File size sum up     : 93.1 MB

Top activities were:

(SharePoint, Page, Content)                                 : 24.94 min
(SharePoint, File, Content)                                 : 13.86 min
(Confluence, Page, Link_Soft)                               : 13.01 min

In the end, looking at the overall migration duration and number of pages, the average migration duration was 13 seconds per page.

Confluence Cloud Site: logica.atlassian.net

This site had 19 spaces.

Migration Metrics for whole Site

  • Migration Machine: Mig VM
  • Migrated Migrated number of pages: 1494 (whole site)
  • Migration duration: 5h 9m

Here are the metrics (as logged into the common log files) 1 hour into the migration, based on data of this hour, covering 489 pages:

Median time (IQR)    : 4 s/item
Median time          : 4 s/item
75th percentile (IQR): 4 s/item
75th percentile      : 6 s/item
Mean time (IQR)      : 4 s/item
Mean time            : 7 s/item
Items per hour       : 900-900 (based on IQR Median and IQR 75th percentile)
Avg links transformed: 1/item
Have files           : 13% of items
File count avg       : 1.5 per item that has files
File size avg        : 372 KB/file
Download speed       : ~127 KB/s
Upload speed         : ~99 KB/s
File size sum down   : 294.0 MB
File size sum up     : 294.2 MB

Top activities were:

(SharePoint, Page, Content)                                 : 30.43 min
(Confluence, Page, Link_Soft)                               : 17.85 min
(SharePoint, WikiTraccs, Prerequisites)                     : 17.11 min

Let’s look at how the values changed after 4 hours of migration, covering 255 page-like contents:

Median time (IQR)    : 4 s/item
Median time          : 4 s/item
75th percentile (IQR): 7 s/item
75th percentile      : 11 s/item
Mean time (IQR)      : 6 s/item
Mean time            : 14 s/item
Items per hour       : 514-600 (based on IQR Median and IQR 75th percentile)
Avg links transformed: 1/item
Have files           : 30% of items
File count avg       : 2.9 per item that has files
File size avg        : 2069 KB/file
Download speed       : ~705 KB/s
Upload speed         : ~451 KB/s
File size sum down   : 13192.5 MB
File size sum up     : 13828.6 MB

Top activities were:

(SharePoint, Page, Content)                                 : 14.34 min
(SharePoint, WikiTraccs, Prerequisites)                     : 8.29 min
(SharePoint, File, Content)                                 : 7.54 min

In the end, looking at the overall migration duration and number of pages, the average migration duration was 12 seconds per page.

Confluence Cloud Site: vzvz.atlassian.net

This site had 26 spaces.

Migration Metrics for whole Site

  • Migration Machine: Mig VM
  • Migrated Migrated number of pages: 8541 (whole site)
  • Migration duration: 31h 45m

Here are the metrics (as logged into the common log files) 1 hour into the migration, based on data of this hour, covering 369 pages:

Median time (IQR)    : 6 s/item
Median time          : 6 s/item
75th percentile (IQR): 13 s/item
75th percentile      : 13 s/item
Mean time (IQR)      : 8 s/item
Mean time            : 9 s/item
Items per hour       : 276-450 (based on IQR Median and IQR 75th percentile)
Avg links transformed: 3/item
Have files           : 47% of items
File count avg       : 3.4 per item that has files
File size avg        : 271 KB/file
Download speed       : ~219 KB/s
Upload speed         : ~103 KB/s
File size sum down   : 173.9 MB
File size sum up     : 174.7 MB

Top activities were:

(SharePoint, Page, Content)                                 : 27.74 min
(Confluence, Page, Link_Soft)                               : 22.21 min
(SharePoint, File, Content)                                 : 20.38 min

Let’s look at how the values changed after 4 hours of migration, covering 255 page-like contents:

Median time (IQR)    : 10 s/item
Median time          : 10 s/item
75th percentile (IQR): 13 s/item
75th percentile      : 15 s/item
Mean time (IQR)      : 11 s/item
Mean time            : 13 s/item
Items per hour       : 276-327 (based on IQR Median and IQR 75th percentile)
Avg links transformed: 1/item
Have files           : 26% of items
File count avg       : 1.9 per item that has files
File size avg        : 533 KB/file
Download speed       : ~134 KB/s
Upload speed         : ~84 KB/s
File size sum down   : 68.5 MB
File size sum up     : 67.4 MB

Top activities were:

(SharePoint, Page, Content)                                 : 30.62 min
(SharePoint, WikiTraccs, Prerequisites)                     : 12.94 min
(Confluence, Page, Link_Soft)                               : 11.94 min

In the end, looking at the overall migration duration and number of pages, the average migration duration was 13 seconds per page.

Wrap

In this blog post we looked at some metrics from real test migrations to get a feeling about how long a migration can take.

Live metrics can only predict the future if we assume that pages are comparable in their characteristics (mainly number of attachments), which might not be the case. Thus, the metrics will change over the course of the migration as metrics are always calculated based on data from the last 60 minutes.

So far it seems that the metrics Mean Time (IQR) and 75th percentile (IQR) could be predictors for the overall migration time within a space.

Over time, I’ll add more samples to this blog post and might also update the values, as the metrics calculation evolves.

Confluence Quirks

This post is a collection of quirky stuff that Confluence throws at WikiTraccs.

When speaking about page migrations failing, I mean the things on this page. All those tripped up page migration at some point, those are now known and worked around by WikiTraccs (where possible).

This list is more like a “fun” documentation of things, I never thought could happen.

  • page with empty title; how is that even possible?
  • duplicate attachments:
  • attachment name that looks like a path; can’t be downloaded:
  • attachments that differ only by Unicode Normalization Form; names look the same, but are differently encoded; depending on the application that handles those files, they are treated as duplicate - or not
  • attachments that are listed in the attachment list, but are missing when trying to download them
  • attachments that are listed in the attachment list, but give an error when trying to download them:
    • note that in above case the image preview for this image was missing as well, but clicking this “broken image” placeholder did show the image:
  • attachments names with special characters that are valid on Linux, but not on Windows
  • attachment file sizes reported by Confluence are wrong

This list will be extended as more quirks are uncovered.

Good Practices for your Confluence to SharePoint Migration

This post is about good practices that I picked up from clients.

Often I get asked about best practices around Confluence to SharePoint migrations and WikiTraccs.

Please note that I am not directly involved in migration projects. If there is a perfect project where the client doesn’t need my help because WikiTraccs does its job, I’ll never hear from them.

What I hear about are issues. Issues with tooling, but also non-technical issues.

Recommendations

Here’s a collection of things I recommend.

Prepare users for SharePoint

I often hear phrases like “Leadership has decided” to proceed with moving content to SharePoint. However, some users may not have had any prior experience with SharePoint. Familiarize them with SharePoint or, at the very least, help them anticipate that SharePoint will be different from Confluence.

Test the migration

Perform a test migration using WikiTraccs. Evaluate the results. Purchase only, if the results match your expectations. Depending on how well your stakeholders know SharePoint, expectations vary wildly.

Expect layout changes. SharePoint is no Confluence. SharePoint doesn’t offer the same formatting and layouting features as Confluence. You’ll see that for example with images, where SharePoint is not capable of showing them side by side 😔.

Prepare for macros being gone

Most of the Confluence macros don’t exist in SharePoint. Identify key use cases that exist in Confluence and rely on specific macros or metadata. Manually try to rebuild those cases in SharePoint. WikiTraccs might help you with that, but you will also have to think about using extensions like PnP Search web parts. You’ll probably also use services outside SharePoint to re-implement those use cases, like the Power Platform, Loop, Teams, or others.

Prepare your service desk for hypercare questions

After the migration users will ask questions. Provide them with a channel to get those questions answered.

Example questions:

  • Why does the table of contents doesn’t update in SharePoint?
    • Answer: because the list of links is static and the table of contents macro is gone.
  • Why are child pages not showing up on the page?
    • Answer: because the children macro is gone.

Don’t do a permission migration

Confluence and SharePoint behave differently when it comes to permissions. Confluence can have page hierarchies. In Confluence, you can restrict pages to a more narrow circle of users with each hierarchy level. In SharePoint, there is just one level. Multiple levels of Confluence page restrictions need to be combined to just one SharePoint page.

For a more complex hierarchy of restrictions in Confluence it is nearly impossible to map that 1:1 to SharePoint.

That being said, WikiTraccs supports migrating permissions, to the extend it is possible. But a permission migration might give you a headache.

Archive older content to one site

Identify old Confluence content to archive and migrate this content to one SharePoint archival site.

Skip personal Confluence spaces

Prepare users to manually move content they need from their personal spaces.

Don’t change networking infrastructure during the migration

One client upgraded Confluence to TLS 1.3 during the migration. This caused issues in combination with Windows 10. Everything can be solved, but this can cause interruptions that should be prevented during a migration.

Plan all target sites before the migration

You should know which Confluence content will be migrated to which target SharePoint site. Enter target site URLs in the Space Inventory list. This is required for the link transformation to be successful throughout the migration as WikiTraccs will look up those sites when transforming cross-space links.

Note that WikiTraccs doesn’t create those sites, you create them, or choose from existing ones.

Check the WikiTraccs settings

The blue WikiTraccs window has a menu bar with a Settings option. Click that and click through the tabs in the Settings window to familiarize yourself with the settings. Some choices are to be made like whether blogposts should be migrated or not.

Use migration waves for larger space counts

You can define migration waves. If you have hundreds of spaces, it might be beneficial to not choose all at once for migration, but to give them wave numbers and migrate each wave separately, or even (some) in parallel.

Bring the page tree to SharePoint Online

The number one missing productivity feature in SharePoint with regard to pages is the page tree. WikiPakk finally brings the page tree to SharePoint.

Repeat migration runs

During longer migration runs there can be connection issues that prevent single pages to migrate. Re-run the migration to get those pages over as well.

With WikiTraccs, you can re-run the migration multiple times. Before each run, WikiTraccs checks which pages are missing in SharePoint and only migrates those that are missing. This proved beneficial especially in environments with an unstable internet connection.

Only when a migration run doesn’t seem to migrate any more pages you go ahead and look at the progress log files to check the final result.

Check the playbook

The Migration Playbook provides guidance about configuring WikiTraccs and evaluating migration results.

Additional topics

Some additional topics that often come up.

How many SharePoint sites do we need?

I cannot answer that as this is up to your information architecture.

What I often see is a 1:1 relationship between spaces and sites. So, all pages of a Confluence space are migrated to a target SharePoint site.

How long will the migration take?

This is hard to answer as it depends on a number of factors. The FAQ has some thoughts about that: How much time does it take to migrate?

Confluence Cloud Specialties

This post explains how certain content can only be migrated using workarounds and also lists the prerequisites for those workarounds to work.

To migrate content from Confluence Cloud to SharePoint Online, WikiTraccs needs to access content in Confluence Cloud to download it. Content means page content, attachments, metadata, macro details, new content types like whiteboards, and more.

There are two ways to get content from Confluence Cloud: using a programming interface that is officially supported by Atlassian, and using internal interfaces.

Ideally, only officially supported interfaces are used; for Confluence Cloud, that would be the REST API.

Since this post exists, you might already guess where this is leading. Not every content is accessible using official ways.

Content that can only be migrated via workarounds

There are Confluence elements which cannot be accessed using officially documented interfaces (as the Confluence Cloud REST API would be one).

WikiTraccs tries to work around those limitations.

Here’s the list of (currently known) elements that need a workaround to be migrated:

  1. Jira issue lists
  2. Whiteboards

Let’s look at each of those.

Jira issue lists

WikiTraccs takes static snapshots of Jira issue lists so that the migrated SharePoint page contains a static issue table as well.

In Confluence Cloud the endpoint to do that conversion is not officially accessible (WikiTraccs issue #123).

WikiTraccs works around this limitation by using the same endpoint that Confluence uses when you work in the browser.

Prerequisite: This workaround is only available to WikiTraccs when using Interactive Login. It is not supported when using API tokens or Anonymous access.

Whiteboards

Read this blog post to learn how WikiTraccs works around the missing Confluence Cloud export functionality at the ramifications: Migrating Confluence Cloud Whiteboards

Closing notes

WikiTraccs uses workarounds to migrate content that otherwise is not accessible to apps via official Atlassian interfaces.

Those workarounds are based on internal interfaces and they might break at any time since Atlassian is not required to announce changes. So, it might happen that a workaround stops working temporarily, might require a WikiTraccs update, or might stop working at all. There is nothing that can be done about that, apart from lobbying for Atlassian to provide proper interfaces for exporting all content.

WikiTraccs uses workarounds exclusively for content that otherwise would not be accessible at all and would thus be missing in migrated pages.

WikiTraccs switches to Confluence Cloud v2 REST API

This blog post outlines a bigger change in how WikiTraccs works with Confluence Cloud.

Background

Confluence Cloud so far provided programming interfaces (APIs) that allowed applications like WikiTraccs to treat it as no different that Confluence Server or Confluence Data Center.

Those days are over.

Atlassian is removing critical parts of the v1 REST API (that provided access compatible with the on-prem Confluence versions). Instead, applications are now forced to use the v2 REST API, starting December 2, 2024.

Here is the announcement by Atlassian:

What does this mean for users of WikiTraccs?

Ideally there should be no noticable difference. But the v2 API, in my view, is a step back in terms of developer convenience. More work has to be done to retrieve the same data as before.

From now on it takes both more time and more transmitted data to get the same work done as before.

WikiTraccs Update Required by December 2, 2024

Starting with version v1.21.3 WikiTraccs uses the new v2 REST API instead of deprecated v1 endpoints when talking to Confluence Cloud.

Starting December 2, 2024, using this or a newer WikiTraccs version is mandatory for Confluence Cloud to SharePoint migrations (unless Atlassian extends the deadline).

Until December 2, 2024, switching back to v1 API is possible. You can use the Force use of v1 REST API setting to switch back to v1:

If there are unforseeable issues, this setting can be used to check if those are caused by the switch to v2. It shouldn’t be necessary.

If you find issues, use any of the support channels to get in touch.

Migrated SharePoint Page Names Explained

This post goes into the naming scheme WikiTraccs applies to migrated pages.

SharePoint pages migrated by WikiTraccs all follow the same naming scheme. We’ll take a look at how those names look and why this scheme is applied.

How WikiTraccs names migrated pages

In contrast to Confluence, SharePoint pages are files and those files have a file name and a title. When migrating a Confluence page to SharePoint, WikiTraccs has to set both the file name and the title.

We’ll focus on the file name (also referred to as page name or just name).

The SharePoint page file name will be derived from

  • the Confluence space key (soon: space alias)
  • the Confluence page title, and
  • the Confluence page ID.

For a space key of HR, a title of Onboarding, and a page ID of 123456789, the SharePoint page name will be HR-Onboarding-123456789.aspx.

This scheme WikiTraccs applies is fixed and cannot be changed.

Why WikiTraccs names pages like it does

The naming scheme presented in the previous section is applied so that WikiTraccs can create SharePoint links between pages, without having to actually migrate all pages that link to each other. It can infer page names from page metadata.

Let’s look at an example of two pages (space key, page title, page ID):

  • HR, Onboarding, 123456789
  • IT, Welcome Package, 987654321

The HR Onboarding page links to the IT Welcome Package page.

Now you task WikiTraccs to migrate all Confluence pages of space HR to SharePoint.

When migrating the HR Onboarding page, WikiTraccs needs to transform the (Confluence) link to the IT Welcome Package page to a SharePoint link.

In general, SharePoint page links look like https://yourcompany.sharepoint.com/sites/<SITENAME>/SitePages/<PAGENAME>.

For each link to a Confluence page, attachment, or space, WikiTraccs needs to create the corresponding SharePoint link.

So, WikiTraccs has to figure out the SITENAME and PAGENAME parts of the each SharePoint link it creates.

For the PAGENAME part of the SharePoint link, WikiTraccs applies the naming scheme that is the focus of this post. It knows that the IT Welcome Package page, once migrated, will get the page name IT-Welcome Package-987654321.aspx.

The SITENAME part of the link is looked up in the Space Inventory.

Having both PAGENAME and SITENAME, the final SharePoint link might look like this:

https://yourcompany.sharepoint.com/sites/ITS/SitePages/IT-Welcome Package-987654321.aspx

How to name pages differently?

Currently there is no way to change the SharePoint page file names that WikiTraccs creates during the migration as it relies on those names to creates links between pages.

Can I rename pages later?

After finishing all migration tasks, you might choose to rename pages via PowerShell. Since I did not conduct large-scale tests in this direction, I cannot say how well SharePoint handles renamed pages and especially linking to those renamed pages from other pages and web parts.

One sample of a working rename operation can be found in the WikiTraccs discussions.

Wrap

In this post we looked at how SharePoint pages are being named by WikiTraccs and how the naming scheme looks. We also looked into how the naming helps to create links between pages.

How to migrate only some pages?

This post show how to migrate some selected pages from Confluence to SharePoint, instead of a whole Confluence space.

With WikiTraccs, selecting pages for migration is always done in the Confluence Space Inventory list (short: space inventory).

The space inventory contains source selectors that tell WikiTraccs which pages to migrate.

In this blog post we’ll look at one specific source selector type that allows to select single pages for migration: the Content ID selector.

How to tell WikiTraccs which pages to migrate, by content ID

Everything in Confluence - pages, blog posts, attachments, comments, you name it - has an ID: the content ID.

For now, let’s assume you already have a list of content IDs you want to migrate (We’ll look at how to get those IDs in the next section). You would mark them for migration by creating a new source selector in the space inventory, like this:

  1. open the space inventory (note: this is a SharePoint list and can be used as such)
  2. add a new item
  3. enter the following data for the newly created item:
FieldSample ValueRemarks
Titlenotusedthe title is ignored, but is a mandatory field; enter anything here
WT_In_​SiteId​https://CHANGEME.atlassian.net​/wikienter your Confluence base URL; look at the other items in the space inventory and copy it from there
WT_Setting_​RequestTransformationYesmark the selector for migration
WT_Setting_​TargetSiteRootUrl​https://contoso.sharepoint.com​/sites/targetenter the target SharePoint site here
WT_Setting_​ContentSelectorValue123456789;#page,​987654321;#blogpostthis list of ID-type combinations tells WikiTraccs what to migrate; each value consists of the content ID (like 123456789) followed by the content type (page, blogpost, whiteboard, database, …), separated by ;# (like so: ID;#TYPE); multiple values are separated by comma (like so: ID;#TYPE,ID;#TYPE,ID;#TYPE)
  1. the other fields are optional and can be left blank

When starting the next migration, WikiTraccs should process this selector and migrate the pages.

How to get content IDs and types?

Depending on your migration team’s level of access there are different ways to get content IDs and content types.

How to manually look up the content ID and type for a page

Any user that can view Confluence pages in the browser can do this.

Confluence Server and Data Center

In the browser, open the Confluence page you need the content ID for.

Open the Page Information for that page:

The browser will now show the page ID in its address bar; here it is 10387457:

The Page Information view and address bar will look similar for pages and blog posts. You can infer that you are looking at a blog post by looking at the breadcrumb, which contains a date:

Note: The Page Information view is also available in Confluence Cloud.

Confluence Cloud

In Confluence Cloud the content ID and type are always shown in the address bar of your browser when navigating pages.

Having open a page the address bar will look like this:

https://contoso.atlassian.net/wiki/spaces/SPACE/pages/123456789/Page+Title

The /pages/ part tells you that this is a page, and its ID is 123456789.

For a blog post this looks similar:

https://contoso.atlassian.net/wiki/spaces/SPACE/blog/2024/10/24/​987654321/Blog+Title

The /blog/ part tells you that this is a blog post, and its ID is ​987654321.

This works similarly for other content types like whiteboards.

How to use a space backup to look up content IDs and types

As Confluence system administrator, you can create a space backup. You’ll see content IDs and types in this backup.

Each space backup is a zip file that contains an entities.xml file.

You’ll find information about all pages (of the backed-up space) in this xml file:

The same for blog posts:

How to use SQL queries to look up content IDs and types (on-premises only)

This option is the most flexible one, but can only be applied with administrative access to the database server backing Confluence.

Have a look at this recipe on how to use SQL to get a list of pages for a space: Getting a list of pages per space [on-premises].

Use cases for the Content ID selector

The Content ID selector has several benefits:

The drawback of course is that you manually have to assemble the list of IDs. Using the space selector to migrate whole spaces is much more convenient.

Wrap

In this post we created a Content ID selector to migrate two selected pages. We also learned how to get the content IDs for pages. Depending on the permissions of the migration team, there are different ways to do that.

Confluence vs. SharePoint - Part 1: Overall structure

In this post we look at how content is structured in both Confluence and SharePoint.

Confluence

When looking at a Confluence page the basic structure looks like this:

A Confluence page

The following parts make up a Confluence page:

  • Page Content, including
    • Versions
    • Inline Comments
    • Inline Tasks
    • @-Mentions
    • Macros
  • Attachments, including
    • Versions
    • Comments
  • Footer Comments
    • like page content…
  • Restrictions
  • Metadata, including
    • Title
    • Creator, Author
    • Creation Date, Modification Date
    • Likes
    • Labels

Each page lives in a Confluence space where each space can contain hundreds, thousands, or even tens of thousands of pages.

Pages within a space usually form a hierarchy. This means there are parent pages, child pages, and leaves, like a tree:

A Confluence space contains many pages

Access to pages can be restricted at each level of the tree, which also affects all child pages. If a user is not allowed to access a parent page, they cannot access any of the child pages as well.

Confluence can have many spaces. A typical instance has dozens or a couple of hundred space, but there’s also instances with a couple of thousand spaces.

The following parts make up a Confluence space:

  • Pages
  • Metadata
    • Title
    • Description
    • Archived or not
  • Restrictions

Spaces have no hierarchy:

Confluence organizes content in spaces

SharePoint

While Confluence consists of spaces, SharePoint consists of sites, which serve a similar purpose.

The following parts make up a SharePoint site:

  • Lists (note: those are like a table)
    • List Items
      • Item Data in List Columns
      • List Item Permissions
    • List Permissions
  • Document Libraries (note: those are like a drive or folder)
    • Files
      • File Metadata in List Columns
      • File Permissions
    • Document Library Permissions
  • Site Permissions
  • Apps

A SharePoint site

Some SharePoint document libraries are “special” in that they serve a specific purpose. With regard to pages, there’s the Site Pages library and the Site Assets library.

All SharePoint pages are stored in the Site Pages library. For SharePoint, pages are just files with some metadata. There is no page hierarchy and all pages are stored flat in the Site Pages library.

Page attachments are stored in the Site Assets library, in a folder that belongs to the page. Each page has its own folder.

When setting the permissions for a SharePoint page, SharePoint takes care of setting those permissions on the page’s folder as well.

How does it map?

Often Confluence spaces are compared to SharePoint sites.

I’d say this is a fair comparison, when focusing on wiki functionality.

Confluence spaces and SharePoint sites have some things in common. Both:

  • have a title and description
  • have owners
  • can be access restricted to users and groups
  • contain wiki pages and files
  • have their content indexed by search
  • have no strict technical hierarchy (bear with me on this one)
  • have their permissions inherited by pages, but pages can also have their own

Confluence pages and SharePoint pages also have some things in common. Both:

  • support the basic text formatting capabilities, like bold, headings, tables, images, etc.
  • can be created from templates
  • can have sections to structure page content
  • can contain “blocks of functionality” - macros in Confluence, web parts in SharePoint

Here are the areas where Confluence and SharePoint differ:

ConfluenceSharePoint Online
pages form a hierarchy (with parent and child pages)pages are flat, without a hierarchy
page breadcrumb navigation above pagesno page breadcrumb navigation
pages have attachmentspages have associated files in a special folder (we can make it look like attachments using a web part)
attachments (files) are bound to pagesfiles are stored in document libraries and linked to by pages
pages seem a bit better suited for documenting knowledgepages seem a bit too much focused on presenting content nicely
macros can be nestedweb parts cannot be nested
page restrictions have a hierarchyitem level permissions for pages have no hierarchy
rich app marketplacemarketplaces are not Microsoft’s strength
pages support @-mentioning other usersno @-mention support on pages
comments under pages with rich formatting, deeply nestedplain text comments under pages, 2 levels
inline comments on pagesno inline comments
inline tasks on pagesno inline tasks
integration with Jirano integration with Jira
pages can easily be moved to other Confluence spaces (due to their self-contained nature)pages cannot easily be moved to other SharePoint sites (at least not without breaking links or attached metadata)

Confluence Cloud introduces some new restrictions compared to Confluence Server and Data Center, for example with regard to formatting content and nesting macros.

Confluence Link Types Explained

This post shows the different link types in Confluence and highlights the difference between soft links and hard links. It covers the relevance of knowing link types and variations for migration tools.

Migration tools like WikiTraccs needs to take care of Confluence links, because otherwise those are broken after a migration.

WikiTraccs will translate links within Confluence to proper SharePoint links. This includes page links, space links, and attachment links.

There is one challenge for migration tools when analyzing Confluence links and that is properly detecting both the two link types (soft links and hard links), and the different hard link variations.

I get that asked quite a lot. From an end user perspective the difference normally is not visible.

Here is an example of one soft link and one hard link, both linking to another Confluence page, as seen by an end user:

Can you spot the difference?

The difference can only be seen when looking at the page’s storage format, which looks like this:

<ac:link>
   <ri:page ri:space-key="linktarget" ri:content-title="Get the most out of your team space"
      ri:version-at-save="1" />
   <ac:link-body>Get the most out of your team space</ac:link-body>
</ac:link>
<a
   href="//wikitransformationproject.atlassian.net/wiki/spaces/linktarget/pages/60162329/Get+the+most+out+of+your+team+space">Get
   the most out of your team space
</a>

This is how tools like WikiTraccs “see” the page when migrating it to SharePoint Online, or processing it in any other way.

The first link - a soft link - is represented as ac:link element that has metadata attached that allows WikiTraccs to look up information about this page.

The soft link contains readily available metadata, like the target space key (linktarget) and the title of the page (Get the most out of your team space). This is easy to handle for a tool like WikiTraccs. And most importantly, Confluence is able to keep this link up-to-date even if the target page changes.

The second link - a hard link - is a plain HTML link. It also points to the target page, but uses the absolute page address to do so.

The hard link doesn’t contain easily accessible page metadata. If you look closely you’ll see information like the space key, page ID, and so on; those certainly help, but there are many variations of those links, and we look at those further down. Confluence will not keep hard links up-to-date, so, if the target page changes, the link might be broken.

Let’s create one.

Ideally, there shouldn’t be hard links at all. When pasting a link onto a Confluence page, Confluence analyzes the link, detects the content it links to, and creates a proper soft link.

Nevertheless, hard links are often present in more complex content that has been copy and pasted into pages. Confluence’s hard link conversion does not always kick in.

Here’s a sample of how to fool the hard link detection and create a soft link and a hard link:

Above animation shows the following:

  1. we copy a link to a Confluence Cloud page to the clipboard
  2. we paste the link as plain text
  3. Confluence recognizes that this is a hard link and converts it to a soft link
  4. we change the settings of that soft link to look like a regular link
  5. we paste the link a second time
  6. we edit the second link to remove the https: part; the result is still a proper and valid hard link, but it fools Confluence’s logic and it leaves the hard link in place
  7. we save the page

The result is a page containing one soft link and one hard link.

It is relevant if you migrate content from one Confluence environment into another environment.

So, it is relevant for those use cases:

  • Confluence Cloud site to site migrations
  • Confluence Data Center to Confluence Cloud migrations
  • Confluence to SharePoint Online migrations

In all those cases, links between migrated pages will be broken, as well as links from external systems, like Jira descriptions and comments.

To fix those broken links, you need to:

  1. identify the links
  2. modify the links to point to the new location (or redirect)

Knowing that hard links and soft links exist is crucial for identifying the links in the first place.

Ideally, migration tools cover that, so you don’t have to care. WikiTraccs covers this.

We used a very simple hard link example so far (from Confluence Cloud):

wiki/spaces/linktarget/pages/60162329/Get+the+most+out+of+your+team+space

This link contains lots of metadata to look up the target content:

  • content type (page)
  • content ID (60162329)
  • space key or alias (linktarget)

Here’s a list of other hard link variations:

  • wiki/display/Test (space link)
  • wiki/spaces/Test (space link, cloud)
  • wiki/pages/viewpage.action?pageId=10162329 (page link)
  • wiki/pages/viewpage.action?pageId=10162329&pageVersion=1 (link to page version)
  • wiki/x/xYBtAQ (tiny page link)
  • wiki/display/Test/news (page link)
  • wiki/spaces/Test/blog/2024/10/24/181542831/news (blog post link, cloud)
  • wiki/spaces/Test/blog/181542831 (blog post link, cloud)
  • wiki/spaces/Test/pages/edit-v2/2123259301 (page edit link, cloud)
  • wiki/pages/viewpageattachments.action?pageId=60162247&preview=/60162247/174891441/image.png (attachment view link)
  • wiki/download/attachments/10162247/image.png (attachment download link)
  • wiki/spaces/Test/overview?homepageId=3020326337 (page link)

Wrap

In this post we learned about soft link and hard links, and looked at different hard link variations.

When moving content from one Confluence environment to another environment (Confluence, SharePoint Online, …) links to pages, attachments, and spaces will break. Those links need to be taken care of, either by tools like WikiTraccs, or proxy solutions that recognize those links and redirect to the new location.

Mapping user accounts from Confluence to SharePoint

This blog post shows a basic user mapping.

In this blog post we’ll create a Confluence page, look at author and editor, migrate the page, configure user mappings, look at the resulting SharePoint page, and update the page’s metadata - to demonstrate how to map users from Confluence to SharePoint.

We start with at a single Confluence source page.

Confluence Source Page

This Confluence page was created by Admin, and last edited by Parzival.

Here are those users in the Confluence administration:

Those users are local Confluence users, so not synchronized with Entra ID or other external directories.

However, note that the email address of Parzival corresponds to the email address of their Entra ID user. We’ll come back to that fact at a later point.

Migrating the Source Page to SharePoint

We now migrate the page to SharePoint.

When migrating pages, WikiTraccs looks at each page and “collects” user accounts from metadata fields and page restrictions.

Information about those user accounts will be stored in the Confluence User and Group Mapping (WikiTraccs) table, in the WikiTraccs site in SharePoint:

This list contains information like:

  • Principal Type (user or group)
  • Display Name
  • Internal Name
  • Key
  • Email Address (note: only if one could be gotten)
  • Principal Source (note: the source is either onprem for Confluence Server and Data Center, or cloud for Confluence Cloud; it does not refer to specific user directories or their location)

The column WT_Setting_MapForDataAndMentions is where the corresponding Entra ID account has to be entered.

Filling in the WT_Setting_MapForDataAndMentions column is a manual task, but can also be automated via PowerShell. Clients also had success filling that via Excel.

There is an automapping feature built into WikiTraccs, though. WikiTraccs tries to find a matching Entra ID account based on the Confluence account’s email address.

That is why (in above screenshot) the mapping for Parzival is already filled in; this account has the same email address in Confluence and Entra ID.

For all other accounts the mapping has to be filled in manually (by you).

Looking at the Migration Result

What we’ve got so far is:

  • one page migrated from Confluence to SharePoint
  • the page author in Confluence is Admin, page editor is Parzival
  • the user mapping for Parzival has been automapped by WikiTraccs in the Confluence User and Group Mapping (WikiTraccs) table
  • the user mapping for Admin is yet to be done
  • the migration account used to log in to SharePoint was WikiTraccs

The metadata of the resulting SharePoint page will look like this:

The creator of the SharePoint page is the Entra ID account WikiTraccs, not Admin. That is because there is no mapping configured for Admin, yet. So, the creator is set to the account doing the migration.

The editor of the SharePoint page is set to Parzi Val, which is the Entra ID account that matches (via email address) the Confluence account Parzival.

We now want to update the mapping for Admin.

Updating Mappings, Updating Pages

So far everything is going as planned.

We migrated some content and got a mapping to do.

Let’s configure the mapping for the Confluence Admin account which we map to the Heinrich Entra ID account:

But how to update the SharePoint page?

WikiTraccs has an update mode built-in for this specific case. In the blue WikiTraccs window, choose it in the Settings dialog:

Update mode that updates page metadata based on configured account mappings

Leave all other settings in place and start the migration again.

WikiTraccs will now run over all migrated pages where a mapping was missing and update the page metadata.

How does WikiTraccs know where a mapping was missing? Those pages have been marked via the Check Principal Mapping column, in the Site Pages library:

After successfully updating author and editor metadata of a page (and only then), WikiTraccs removes this check mark. (Note: If you ever want to force an update of author and editor, set the check mark again before running the migration in update mode.)

After WikiTraccs is finished with the update mode run the SharePoint page should now look like this:

Creator and author are properly set to what was configured via the Confluence User and Group Mapping (WikiTraccs) table.

Confluence Authentication Overview

This post summarizes the different Confluence authentication options that WikiTraccs supports, with a focus on navigating some harder cases.

When starting a Confluence to SharePoint migration with WikiTraccs, you need to authenticate with Confluence so that WikiTraccs can access content.

You’ll need a Confluence user account for the migration

You’ll need a Confluence user account to log in with (“migration account”). WikiTraccs’ access to Confluence happens as this user.

WikiTraccs does not support “application permissions” where it would access Confluence as “itself”. WikiTraccs will always access Confluence in the context of a user account.

Anonymous access

There is one exception to the rule that you always need to use a migration account, and that is a Confluence instance with anonymous access enabled.

Anonymous access is good for testing or demo purposes. The quick start tutorial also uses that with a publicly available Confluence instance.

Anonymous access is not recommended for production migrations as certain metadata cannot be retrieved, like information about users (name, email address) and permission configuration.

Interactive login

Interactive Login is the easiest and most compatible authentication type.

WikiTraccs opens a browser window that you’ll use to log in to Confluence with the migration account. WikiTraccs takes over the user session and all requests to Confluence will be made in the context of the migration user account.

Here is more information: Cookie-based authentication / Interactive login.

The session cookies that WikiTraccs uses to take over the user session are:

  • JSESSIONID
  • seraph.confluence
  • tenant.session.token (Confluence Cloud)
  • _shibsession* (Shibboleth identity provider)

In case your identity provider needs additional cookies to identify a logged in user, click the Advanced Settings button and enter those cookie names:

If there are issues with interactive login, like WikiTraccs not being able to start the Chrome browser for interactive login, refer to the troubleshooting article dedicated to getting around those issues: Confluence Authentication Issues.

Personal Token

The Personal Token authentication type uses a special kind of password (“token” or “key”) to log in. This password is created in Confluence in the context of a Confluence user account.

Please refer to this article for configuration details: Personal Access Token.

Confluence Server and Data Center just need the token

Confluence Cloud needs the name as well

The Personal Token is also referred to as Personal Access Token, API Token, or API Key.

Confluence 7.9 started supporting this authentication type.

In Confluence Cloud some transformations may not avilable with this authentication type (Jira Issue List).

The harder cases…

What are hard cases with regard to Confluence authentication?

Here are some hallmarks of a hard case:

  • Interactive Login authentication does not work immediately
  • Personal Token authentication is not available because Confluence is older than version 7.9 or because of multi-factor or other requirements
  • the cookies.txt workaround to provide session cookies manually for Interactive Login does not work

Over time, I encountered the following hard cases:

  • Kerberos prevented WikiTraccs from taking over the session cookies from Interactive Login; only connections made from the Chrome browser were considered valid by the Confluence backend
  • WikiTraccs once could not detect a successful login to Confluence and was thus not able to get the session cookies for Interactive Login; cookies.txt also did not work; reason unknown
  • Confluence could only be connected to using TLS 1.3-secured connections; Windows and the Confluence backend could not negotiate a mutual TLS 1.3 cipher to use; this issue does not affect the Chrome browser as Chrome brings its own cryptographic stack that operates independently of Windows

Note that those cases were the exception and not the rule.

Nevertheless, those cases needed to be handled as some of them were introduced by configuration changes that were made while migrating and thus blockers. So, WikiTraccs got something called the proxy mode.

Proxy Mode

The proxy mode is a workaround for all of the above issues with the Interactive Login authentication type.

Normally, connection and authentication issues tend to be absent when a user logs in to Confluence in the browser, but start appearing once WikiTraccs starts talking to Confluence directly.

If those issues cannot be resolved, there is only one way left: go back to using the browser to connect to Confluence.

That’s what the proxy mode does. WikiTraccs will keep an automated Chrome browser window open and route all requests to Confluence right through that browser.

This comes with some drawbacks as well. Routing everything through the browser is slower. Also, the proxy mode is not perfectly polished, as it is only thought as a workaround. So WikiTraccs might get confused at times with regard to automating the Chrome browser window, which can be resolved by closing and reopening all windows.

But overall the proxy mode is like the silver bullet to authentication issues.

You activate the proxy mode in WikiTraccs Settings -> Misc -> Proxy Confluence API calls through browser:

Summary

When it comes to authenticating with Confluence those are the steps to take:

In WikiTraccs:

  1. Choose the Interactive Login authentication type and test connecting; if that doesn’t work immediately:
  2. Choose the Personal Token authentication type and test connecting; if that isn’t available:
  3. Use the cookies.txt workaround and test connecting using the Interactive Login;
    • if that does work: good; find the missing cookie names and enter them in the Advanced Settings dialog
    • if that doesn’t work:
  4. Use the Proxy Mode in combination with Interactive Login

If your environment blocks connections to certain endpoints, it might also be necessary to provide the Chrome Driver manually. WikiTracccs uses the Chrome Driver to automate the Chrome browser.

General considerations:

  • check if the migration account can be enabled for Personal Token authentication
  • make sure all endpoints can be reached by WikiTraccs, e.g. to download the Chrome driver needed to automate the Chrome browser for Interactive Login
  • make sure the Confluence backend and Windows can work together with regard to supported TLS versions and ciphers

Usually, none of the workarounds are necessary. But now and then they are needed. That’s why they are available.

And if nothing works - get in touch and we’ll have a look.

Confluence might misreport space contents

This post highlights a challenge some Confluence instances pose - they misreport the content of spaces.

There is one specific issue that customers have now and then: not all pages of a space are being migrated. Some pages will be missing in SharePoint.

Why is that?

While there can be multiple reasons, one is Confluence itself misreporting the contents of spaces.

Let’s look at the root issue and how WikiTraccs tries to work around it.

The root issue: Confluence lies about its space contents

If the Confluence issue is present, it affects the most convenient type of WikiTraccs’ source content selectors, the Space Selector.

When WikiTraccs starts migrating a space it asks Confluence about the contents of this space, like “give me all page IDs in this space”.

Confluence will happily answer and the list of page IDs it returns might look like this: 00001, 00002, 00003, 00004, 00005, 00006. This would be 6 pages to migrate. That’s what we expect.

But sometimes the result looks different, although all 6 pages are definitely there. Confluence might report the list of page IDs like this: 00001, 00002, 00002, 00002, 00005, 00006.

Notice the difference? Page ID 00002 is listed three times, while 00003 and 00004 are missing.

This is a problem. Why is Confluence lying to us? I don’t know.

WikiTraccs tries to work around this issue

The latest release of WikiTraccs contains a workaround for this Confluence issue.

WikiTraccs detects duplicate page IDs and will take that as a hint that page IDs will be missing as well. It will then use a different method to retrieve the page IDs for a space.

One caveat of this workaround is that it’s significantly slower than just getting the list of page IDs handed by Confluence. But at least it should retrieve a complete list.

How to verify that all pages have been migrated?

When Confluence is not lying to us, WikiTraccs’ progress log files are the way to go. The __30-aggregated-info file shows a summary of a space’s migration progress. This is the happy path.

If you have a hunch (or see it in the logs) that Confluence might be lying about the contents of a space, your only chance is to look at and compare with the Confluence database.

Here’s how to get a list of page IDs from the Confluence database, for a given space: Getting a list of pages per space.

Compare the list of page IDs you got from the database with the list of pages WikiTraccs got handed by Confluence. The pages WikiTraccs knows about for a space can be seen in the __25-update-state-of-migrated-pages progress log file.

Hints about duplicates in the WikiTraccs log file

The WikiTraccs common log files contain information about duplicate page detection and applied workarounds.

You want to find this message for each space that has been selected for migration:

No duplicate IDs found for selector (Type=ConfluenceSpaceKey; Query=GOOD)

Above log message says that all is fine for space with key GOOD, as there were no duplicate page IDs. Also, WikiTracccs takes this as a hint that there won’t be page IDs missing (note: this is an assumption that is yet to be proven wrong).

To the contrary, the following log message indicates that duplicates where found for space with key DUPE:

Duplicate content IDs found for selector (Type=ConfluenceSpaceKey; Query=DUPE)

Searching for the text [DUPLICATES] in the WikiTraccs log files will surface further details about the affected spaces, like which pages are affected and which pages could only be retrieved via the built-in workaround.

Working around the issue with the Content ID Selector

The issue only affects Space Selectors as for those WikiTraccs will ask Confluence about space’s contents. Consequently Confluence might choose to lie to us.

To prevent this kind of issue, you might choose Content ID Selectors instead to tell WikiTraccs the page IDs it should migrate. With this type of selector you take the “page ID retrieval” part in your own hands. Have a look at the documentation about the details.

In general, Space Selectors are easier to handle than Content ID Selectors. So in an ideal world, there would be no need to choose one over the other to work around Confluence issues.

How to fix this Confluence issue?

I don’t know. Let me know if you find a solution.

Can feature 'XYZ' be added to WikiTraccs?

This post outlines how I add new features to WikiTraccs and how we can help each other.

It depends.

Adding XYZ to WikiTraccs vs. project-specific solutions

The development of WikiTraccs is mostly driven by customer demand. That, and changes by Microsoft and Atlassian to their services.

Every environment and migration project is different, but there are similar challenges. If customers (or I) face the same challenge over and over again, it’s usually time to think about extending WikiTraccs to support the underlying use cases.

New WikiTraccs feature?

I’m open to extending WikiTraccs if the new feature helps with an often-faced challenge that is not isolated to a specific migration project.

New WikiTraccs features must:

  1. ✅ work in a standard SharePoint Online environment, without having to install third-party apps
  2. ✅ solve a challenge for multiple clients
  3. ✅ have a technical solution
  4. ✅ take reasonable time to implement

If some of those boxes cannot be checked then the solution is usually project-specific, although we should definitely talk this through.

Project-specific solution?

Project-specific solutions solve challenges for a specific migration project or a specific environment.

Modifying the page header to comply with CI/CD? Turn off the comments section on pages? Add page metadata for HR? Those are unique requirements within a project.

Project-specific solutions usually involve a combination of configuration (search configuration, content types, managed metadata), low-code solutions (Power Automate workflows), and scripting (PowerShell, JavaScript).

Missing feature XYZ is a blocker for me - can you add it to WikiTraccs?

I’d like to learn more. Does the feature check all of above boxes? That’s a good start.

The less time it takes to implement a feature, the more likely it is that I’ll add it for you. The more time it takes, the more useful it has to be in a broader sense.

I consider anything below ~2 days of end-to-end development effort as relatively low effort. Adding transformations for yet-unknown but simple-to-handle macros fall into that category. Or adjustments to text placeholders.

Note that I cannot give a guarantee that or when a feature will be added.

That being said - I’m always open for feature proposals as this is the way for me to learn about real-world challenges.

If we haven’t been already in touch via a demo or email or GitHub, please get in touch!

Help me help you

When looking at your challenge, I need to understand it first. Please help me with that.

Screenshots help illustrate your case

Screenshots are a good way to start diagnosing migration results.

Something looks unexpected after migrating pages to SharePoint? Take a screenshot of the source Confluence page and a screenshot of the migrated page in SharePoint. Highlight your expectations in the screenshots and write some lines about what you expect.

Create the expected outcome manually in SharePoint

Help me see the SharePoint page as you envision it to look after the migration.

Create a new modern SharePoint page and edit it. Manually add all the necessary elements to make it resemble the source Confluence page as closely as possible. Use only out-of-the-box SharePoint web parts - no PnP, no third-party tools (I nevertheless like to hear about those as well! Maybe it’s the exception to the rule).

Send me a screenshot of both the source Confluence page and target SharePoint page.

This ensures that it is technically possible to achieve what you’d like to see on the SharePoint side and serves as a mockup for me to understand your requirements.

Furthermore, It would help a great deal if you could send me the SharePoint storage format of the page. Here’s how to get that: Get the SharePoint Storage Format

Send me the Confluence page’s storage format XML

There is a Confluence page that contains something that you’d like to see migrated differently? Like a macro that ends up being a placeholder in SharePoint?

Send me a screenshot of the Confluence page.

I also need to see the page how WikiTraccs sees it to estimate how much effort it takes to support new transformations.

Send me the storage format XML. Here’s how to get that: Get the Confluence Storage Format.

The storage format XML contains a page’s text content, structure, formatting, and most importantly information about all macros, including their parameters.

Closing note

Please review the existing feature proposals in the GitHub issue list and vote for those you’d like to see gain traction. Alternatively, you can create a new feature proposal or start a discussion.

Converting Gliffy and draw.io to SVG

This blog post shows how to convert popular diagramming solutions to SVG.

Motivation

The motivation behind this blog post is a customer request:

When migrating from Confluence to SharePoint, how to migrate Gliffy diagrams as SVG files?

Unfortunately, WikiTraccs cannot do much in this regard. WikiTraccs migrates Confluence page attachments, which include the diagram file (from Gliffy or draw.io) and the diagram’s preview image. Read the previous blog post for details on that.

At least you end up having the diagram file in SharePoint Online.

Having the diagram file in SharePoint is a good thing, because it gives us the option to work with those diagrams even when Confluence is not around anymore.

How to export diagrams as SVG files

Let’s look at our options to export diagrams as SVG.

Gliffy => SVG

When Confluence is still around, you can export from Confluence:

  1. Open Confluence page
  2. Edit Gliffy diagram
  3. File… > Export > SVG

After having migrated to SharePoint Online, you need to work with the diagram file instead:

  1. Open a migrated SharePoint page
  2. Download the migrated Gliffy diagram file from the page’s attachments
  3. Open app.diagrams.net (a free online diagram software) in browser
  1. Drag the diagram file into the browser
  2. File > Export as > SVG…

draw.io => SVG

When Confluence is still around, you can export from Confluence:

  1. Open Confluence page
  2. Edit draw.io diagram
  3. File > Export as > SVG…

After having migrated to SharePoint Online, you need to work with the diagram file instead:

  1. Open migrated SharePoint page
  2. Download migrated draw.io diagram file from the page’s attachments
  3. Open app.diagrams.net?offline=1 in browser (a free online diagram software), in offline mode
  4. Drag the diagram file into the browser
  5. File > Export as > SVG…

Note that this doesn’t seem to upload the diagram file as it already has the draw.io format.

Privacy Considerations

When Confluence is not around anymore and you end up using app.diagrams.net (note: which is the only free tool I know of that can convert Gliffy to draw.io), there is one fundamental difference between Gliffy and draw.io files.

The app.diagrams.net online editor needs to convert Gliffy diagrams to draw.io diagrams first. This is done at a remote location. This means your diagram files will be uploaded to a third party.

This is contrary to draw.io diagram files. When loading those into the app.diagrams.net online editor, no upload seems to happen.

draw.io Desktop (free)

There is an alternative to app.diagrams.net and that is the draw.io Desktop application. You can download it at drawio.com.

draw.io Desktop seems to have the same capabilities as app.diagram.net. There is one important exception: the desktop version cannot convert Gliffy files to the draw.io format. Which is unfortunate when you only have Gliffy files that you want to convert to SVG.

There is no free offline tool available that I know of that can convert from Gliffy to draw.io.

Gliffy-to-draw.io Mass Conversion (paid-for)

There is a paid option available to convert Gliffy to draw.io when Confluence is still around.

The draw.io Confluence plugin can convert Gliffy to draw.io.

According to its documentation, it can do mass conversions of all Gliffy diagrams in a Confluence instance:

Also according to the documentation, it doesn’t require a license to do the actual conversion. But it will convert all Gliffy macros to draw.io macros as well, so having no license doesn’t seem to make sense.

Mass-converting Gliffy to draw.io before migrating to SharePoint seems like an option to have those draw.io files ready in SharePoint, in case users want to work with those diagrams using either app.diagrams.net or the draw.io Desktop app.

Migrating Gliffy and draw.io macros to SharePoint Online

WikiTraccs migrates Gliffy and Draw.io macros. This article has insights.

We first look at how Gliffy and draw.io work in Confluence, then we look at a migration result.

How are the diagrams stored in Confluence?

This is a sample Confluence page (in edit mode) containing both the Gliffy and draw.io macro:

Both macros create two page attachments when you create a diagram.

One attachment file is the diagram file, containing the digram definition in a macro-specific format. The other attachment is the preview image that is shown on the page.

Here’s how the page attachmens for above sample page look. It’s two attachments per macro:

How do Gliffy and draw.io look in SharePoint Online?

First of all, WikiTraccs migrates the page content and page attachments as usual. So, both the diagram file and the preview image will be available in SharePoint as well.

When migrating page contents, WikiTraccs looks out for Gliffy and draw.io, and replaces those macros with a SharePoint image web part. The image web part is configured to show the respective macro preview image:

Looking at the SharePoint page’s attachments we can see the same files as in Confluence:

This is the result you can expect after migrating Confluence pages with Gliffy or draw.io macros to SharePoint Online using Wikitraccs.

Transforming even more colors from Confluence to SharePoint!

This post describes the latest additions and changes with regard to text colors, highlight colors, and table cell colors.

The latest release of WikiTraccs contains multiple improvements with regard to colors.

Text colors, the newly introduced text highlight colors (Confluence Cloud) and table cell colors got some attention.

What has SharePoint Online to offer in terms of colors?

The SharePoint Online color palette for text looks like this:

Note that we cannot really use the theme colors as they change, depending on the theme. That leaves us with 11 text colors.

The SharePoint Online color palette for highlights looks like this:

That’s a whopping 15 colors.

What about table cell colors? SharePoint does not have those. We’ll see how WikiTraccs handles that further down.

Text Color Transformation

Those are the 21 text colors available (at least at the moment) in Confluence Cloud:

Text Colors in Confluence Cloud

WikiTraccs maps those colors to the visually nearest available text color in SharePoint Online:

Text Colors in SharePoint Online

Here’s another example from Confluence Server, which has 40 colors to offer:

Text Colors in Confluence Server 6

This is how it’s transformed to SharePoint:

Text Colors in SharePoint Online

Highlight Color Transformation (new)

Now for the new highlight colors in Confluence Cloud:

Highlights in Confluence Cloud

Those map pretty well to SharePoint Online in terms of available colors:

Highlights in SharePoint Online

SharePoint’s highlight colors are way darker than in Confluence which makes black text hard to read for the teal and purple highlights.

Table Cell Color Transformation

Here’s how table cell colors look in Confluence Cloud:

Table cell colors in Confluence Cloud

SharePoint does not support setting colors for table cells. Maybe that’s coming in the future, but so far I did not spot anything related on the roadmap. (Let me know if you see or hear something!)

WikiTraccs is using a workaround to at least indicate in SharePoint, that the table cell was colored in Confluence.

"Table cell colors" in SharePoint Online

Here’s another sample from Confluence Server 6, which has less cell colors to offer:

Table cell colors in Confluence Server 6

This is how it’s transformed to SharePoint:

Table cell colors in Confluence Server 6

In both instances, WikiTraccs adds a block of colored spaces to each formerly colored table cell, serving as a “color marker”. And again, it is restricted to colors that SharePoint has to offer.

What did change in the latest WikiTraccs release?

The latest release handles text highlight colors that have been introduced by Atlassian in June 2024.

Furthermore, the color markers for table cells now support the full range of text colors available in SharePoint. Previously, WikiTraccs used colored emoji squares like 🟩 as marker - but those only come in very few colors.

And last but not least: the algorithm to calculate the visual distance between colors has been improved. This maps Confluence colors even more closely to their SharePoint equivalents.

How to run parallel WikiTraccs migrations?

You might want to run multiple WikiTraccs migrations in parallel, to speed up things. Let’s take a look on how to do this and what to look after.

You can run multiple WikiTraccs instances in parallel to speed up the overall migration. You’ll need a separate machine or VM per WikiTraccs instance as only one instance is allowed to run on the same machine.

Some definitions

The computer or virtual machine (VM) WikiTraccs is running on is the machine or migration machine.

We’ll call the running WikiTraccs program a WikiTraccs instance. When two WikiTraccs programs are running, that would be two instances.

The central SharePoint site that WikiTraccs needs and where the Space Inventory list (and more) is located is called WikiTraccs site.

One WikiTraccs instance is allowed to run on one machine.

How to run multiple WikiTraccs instances in parallel?

Looking at client’s projects I see different approaches.

Some create cloud-based virtual machines in Azure or AWS. Some use spare laptops.

Install WikiTraccs to each of those machines and your are good to go.

Configuring WikiTraccs when parallelizing the migration

There is local and shared configuration.

Configure locally on all machines

Make sure you configure the WikiTraccs settings on all machines. This affects things like comments migration, macro ignore list, migration mode etc. They can be different on different machines, but usually are the same.

Locally stored settings are:

  • everything you enter into the blue WikiTraccs.GUI window
  • settings available in the blue WikiTraccs.GUI window via the Settings menu
  • any settings configured via appsettings.json

Note that the Space Inventory that contains the source-to-target mapping is NOT a local setting, but stored in SharePoint, in the WikiTraccs site. But the link to the WikiTraccs site (that contains the Space Inventory) is part of the local settings.

Be aware of shared configuration via WikiTraccs site

One WikiTraccs site can be used by multiple WikiTraccs instances, or each instance can use their own WikiTraccs site.

With the same WikiTraccs site, multiple WikiTraccs instances share the same mappings.

There is one caveat of sharing one WikiTraccs site: they will also want to migrate all the same content.

We’ll look at two two scenarios now: one with shared WikiTraccs site, one without.

Parallelization Scenario - Shared WikiTraccs Site

Multiple WikiTraccs instances share the same WikiTraccs site.

Two WikiTraccs instances sharing a single WikiTraccs site.

To achieve that, simply configure the same WikiTracc site address for all WikiTraccs instances.

Pro:

  • you need to configure the mapping in the Space Inventory only once
  • you need to configure the user and group mapping only once
  • easy, as there is only one WikiTraccs site

Contra:

  • (note: before WikiTraccs v1.18) you need to change the WT_Setting_RequestTransformation value in the Space Inventory before starting each WikiTraccs instance as otherwise multiple instances migrate the same pages (!); one approach is to use PnP.PowerShell to manipulate the space inventory

Parallelization Scenario - Separate WikiTraccs Sites

Two WikiTraccs instances each using their own WikiTraccs site.

To achieve that, use a different WikiTracc site address for each WikiTraccs instance.

Pro:

  • the WT_Setting_RequestTransformation value in the Space Inventory can be configured per WikiTraccs instance, as each instance has its own Space Inventory
  • the snapshots that WikiTraccs takes of pages (in the Confluence Content Snapshots library), that are stored in the WikiTraccs site, are now scattered across multiple WikiTraccs sites, which can be an advantage permission-wise (when different teams do the migration that are not allowed to see each other’s content)

Contra:

  • the mapping in the Space Inventory needs to be configured in all WikiTraccs sites, for all spaces and CQL selectors (!)
  • the user and group mapping needs to be configured in all WikiTraccs sites

General parallelization notes

Multiple instances can migrate to the same target SharePoint site, as long as they don’t migrate the same content at the same time to the same target.

Parallel migration is covered by the WikiTraccs license, without limits.

You shouldn’t go too crazy with the parallel instances. There are limits in place set by Microsoft on how fast an application can access the Microsoft 365 services. Microsoft will throttle WikiTraccs, if it goes to fast. This means: WikiTraccs will have to wait some time (a couple of seconds, or even minutes) until it is again allowed to send content. Those limits are set by Microsoft, adjusted to overall cloud load, and dependent on the number of licenses in the tenant (among other factors).

Fixing image positioning in SharePoint

How to cope with the new way SharePoint handles images that are placed next to each other.

Summary of what happened images in SharePoint

Up until about mid-2023 SharePoint text web parts were able to display inline images next to each other. Then Microsoft started upgrading the editor component that powers the text web part, while at the same time introducing changes to the overall page style. This broke adjacent image positioning.

Specifially, when editing a SharePoint page, its text web parts are auto-upgraded to the new editor version which positions once-adjacent images now vertically, not horizontally:

So what are our options to fix this?

How to fix image positioning? And who?

The SharePoint text web part, at the time of this writing, does not support putting multiple images next to each other. Neither in the browser editor, nor via programmatic means.

Here’s a list of parties that could influence image positioning in text web parts, in principle:

  1. Microsoft - but they say the current behavior is by design, so they won’t help
  2. WikiTraccs
  3. You

Number one is out, let’s look at the other two options.

What can WikiTraccs do?

Short version:

At the moment, nothing.

Longer version:

When migrating Confluence pages to SharePoint, WikiTraccs creates pages that still show adjacent images. That’s because it’s generating pages in a version that still supports this. SharePoint supports displaying pages that use the (now) older version.

As soon as you edit such a page it is upgraded to a newer version that doesn’t support adjacent images anymore.

The only thing that WikiTraccs could do is starting to generate pages in such a way that images won’t be adjacent anymore, to begin with. You’d then see the non-adjacent version right away, without having to edit the page.

I’ll look into such options as soon as the new way to display pages is confirmed as generally available by Microsoft. I don’t have such a confirmation, yet.

That means that WikiTraccs cannot do anything at the moment.

What can you do?

You have two options:

  1. Re-arrange images and content when you edit a page.
  2. Roll back the layout change that Microsoft did to screw up image positioning.

Option 1 is straightforward: you edit a page, adjacent images are not next to each other anymore, you have to clean up.

Option 2 involves rolling back a single layout change that Microsoft introduces when upgrading the text web part.

The Wiki Transformation Project provides a tool to achieve option 2: the WikiTraccs Page Companion.

Introducing the WikiTraccs Page Companion

For option 2 to work, we need to change pages as they are displayed in the browser. It’s like duct taping a small piece of paper over a bad part of the page, using duct tape that can be removed without traces.

Fortunately Microsoft provides a supported means of integrating with SharePoint pages: SharePoint Framework (SPFx) Application Customizer. Using such a customizer, we can hook into each page as it is being displayed in the browser.

Regrettable, the change we need to make to how pages are being displayed is not officially supported by Microsoft. We need to accept that if we want to proceed with this duct tape option.

Here’s a demo video of the WikiTraccs Page Companion in action. About 11 seconds into the video, note the little box reading Legacy image positioning (by WikiTraccs) in the lower right corner, that’s the extension:

(Note, because I get asked: the page tree you see in the video is WikiPakk, the SharePoint page tree experience.)

The toggle button can be used to toggle the image positioning fix on and off on a per-page basis. On means, the duct tape is applied. Off means, the duct tape is being removed, again. It is opt-in, so not enabled by default.

As an author, you would check the page you are editing, and toggle the button as needed. The fix affects the page both when being edited and being viewed.

The WikiTraccs Page Companion is available as pre-compiled download (to be deployed to a SharePoint app catalog), and also as source code for you to build yourself. It’s all on GitHub: WikiTraccs Page Companion.

Note that the non-standard nature of this option means that it might stop working when Microsoft makes changes to pages. If that happens, the adjustments that need to be done need to be re-evaluated.

I’m putting the source code out there, so that you are in control as well.

Get in touch if you have any questions.

Broken inline image positioning in SharePoint

A recent update to the text editor web part seems to break positioning for adjacent images.

In general, SharePoint is behind Confluence when it comes to positioning images in relation to each other and to surrounding content.

But at least putting images next to each other is possible. Is? Was. s Those two blog posts took a deep dive into the topic of images:

Unfortunately a recent update to the SharePoint text editor web part breaks adjacent images. They aren’t adjacent anymore when editing a page.

Microsoft is upgrading the editor version of SharePoint pages - and the trouble starts

The text editor web part is powered by the CKEditor editor component. It can be licensed by any third party to build rich editor experiences in the browser. Microsoft did that.

The following things define how the text editor web part behaves:

  1. the features that CKEditor provides - like formatting highlighted text, adding tables and so on
  2. add-ons to the editor that Microsoft develops
  3. internal settings set by Microsoft defining which features are available to users
  4. surrounding styles (CSS) that define how things look (e.g. font size)
  5. page contents

The most recent version of CKEditor is version 5 (v5). So far SharePoint used version 4 (v4) of CKEditor.

Microsoft now upgrades pages to use CKEditor v5.

Why is this important for content creators? Because the content of SharePoint pages is somewhat coupled to the editor version of the text editor web part. Microsoft cannot just upgrade from CKEditor v4 to v5 without breaking old pages. Therefore Microsoft auto-upgrade pages when they are being edited. And that modifies the page content.

When you start editing a page and its content briefly disappears and then comes back - this might have been an auto-upgrade.

And somewhere along the way of this auto-upgrade the handling of inline images changed.

How does the page upgrade affect adjacent inline images?

Here are three visuals showing how pages used to behave, and how they behave now.

How it used to be…

This is a page with three adjacent images in view mode, using CKEditor v4:

This is the same page in edit mode, using CKEditor v4:

How it’s now…

And here is a video that shows what happens when the auto-upgrade to v5 for this page kicks in:

Note: I used the browser’s developer tools to slow down the internet connection, which allows us to watch everything in slow motion.

What can be seen in above video:

  1. the page enters edit mode and the three inline images can be seen next to each other for a brief moment
  2. then all web part content vanishes; the auto-upgrade happens where SharePoint converts the content from CKEditor v4 to CKEditor v5
  3. the web part content reappears; the images are placed on top of each other (NOT GOOD)
  4. bonus: all images are gone! (this was a first when recording this video) (NOT GOOD)

Editing this page worked perfectly fine for at least the last 12 months. Images stayed in place, no problems.

Now broken.

What does this mean for pages that WikiTraccs migrated from Confluence to SharePoint?

Pages migrated by WikiTraccs use CKEditor v4. Adjacent images look fine when viewing those pages.

When editing a page with adjacent images SharePoint might decide to skip upgrading this page to v5. The page will look fine in view and edit mode.

When editing a page with adjacent images SharePoint might decide to upgrade this page to v5. Adjacent images won’t be adjacent anymore in edit mode and need to be re-arranged manually. This looks like a newly introduced technical limitation of SharePoint. Note that currently there is no way to put images next to each other via the browser editor.

Note: the same applies to pages transformed by other tools, e.g. when transforming classic SharePoint pages to modern SharePoint pages using the PnP Modernization Tooling.

Resources

I’m keeping track of topics related to page upgrades here.

Adjacent image positioning issues

Issues with adjacent image positioning are reported by the first WikiTraccs client in mid-September.

I created a Microsoft Tech Community post to get input: Modern pages with text web parts upgraded from CKEditor v4 to v5 lose image positioning.

I also created a minimal PowerShell script that can be used to create a SharePoint page with adjacent images to test the page upgrade: GitHub repository. To showcase and pinpoint the issue.

Furthermore I opened a case with Microsoft support, which is closed now. The case ID is 2310071420000151. According to them this is expected behavior.

Other known issues

There were issues with tables when upgrading pages from CKEditor v4 to v5:

There are issues with the new spacing between paragraphs:

Wrapping up

It’s not clear what exactly breaks image positioning on modern pages after upgrading the editor from v4 to v5.

It might be the content upgrade for those pages. But it might also be the changed surrounding styles that come with the upgrade.

I really hope that there will be a proper solution to putting images next to each other on a MODERN SharePoint page. Nothing more. Just some images. Next to each other.

How to migrate rich Confluence tables to limited SharePoint tables?

This blog post covers table migrations and how SharePoint differs from Confluence in this regard.

What are different table capabilities of Confluence and SharePoint?

The following table formatting options are available in Confluence:

  • setting the background color for table cells
  • having multiple header rows or columns
  • nesting tables
  • merging table cells

All of the above is not possible in SharePoint at the time of this writing.

Let’s see how WikiTraccs works around that.

How does WikiTraccs adjust for table formatting limitations in SharePoint?

WikiTraccs has to get creative when it comes to mimicking the full range of Confluence table formatting options.

Table cell background color is transformed to colored emojis. The color is moved from the cell background to the cell content, as there are no colored table cells in SharePoint.

Confluence table with colored cells

Table after migrating to SharePoint, with color markers

Only a limited set of rectangular colored emojis is available to do this, so the color often will be off.

Multiple header rows or columns are represented as normal table rows and columns in SharePoint where the text content is formatted bold.

Confluence table with multiple header rows and columns

Table after migrating to SharePoint, with bold text simulating the headers

Nested tables are de-nested. This approach is analoguous to how the the Microsoft SharePoint Migrations tool does it for SharePoint on-prem to SharePoint Online migrations.

Merged table cells cannot be created in SharePoint Online. But WikiTraccs marks those cells that once were merged (as of WikiTraccsrelease v1.3.11).

Here is an example of how it looks after migrating a table with merged cells from Confluence to SharePoint:

Confluence table with merged table cells

Table after migrating to SharePoint, without marking former merged cells (note: default behavior before WikiTraccs v1.3.11)

Table after migrating to SharePoint, with marking former merged cells (note: default behavior as of WikiTraccs v1.3.11)

The little arrow in former merged cells points in the direction of the cell it had been merged with before having been migrated.

Marking former merged table cells is standard as of WikiTraccs v1.3.11.

Wrap-up

WikiTraccs needs to get creative when it comes to transforming Confluence tables to SharePoint tables. The results depend on the complexity of the tables that are migrated from Confluence to SharePoint and a manual content review does not hurt.

Let’s hope that SharePoint keeps maturing to a point where formatting tables is on a par with what’s possible in Confluence.

Migrating large Confluence spaces to SharePoint

This post is about migrating spaces with 10000+ pages which can be challenging due to long page retrieval times.

How does WikiTraccs determine which pages to migrate for a space?

When WikiTraccs starts to migrate a space to SharePoint it retrieves the full list of pages for this space.

Say a space contains 20000 pages. WikiTraccs retrieves basic information about those 20000 pages and adds them to a migration queue. (Later, when migrating each page, WikiTraccs retrieves the contents of this page and creates the corresponding SharePoint page.)

Confluence doesn’t allow retrieving information of more than 200 pages at once. So to retrieve information about 20000 pages WikiTraccs needs to request 100 batches of 200 pages, as that is supported.

This paged retrieval is where things get interesting for larger spaces.

Why can large Confluence spaces pose a challenge?

The more pages a space contains the longer it takes to retrieve a batch of pages.

Here are numbers from a Confluence 6 test migration of a space with about 23000 pages:

  • retrieving batch 1 takes < 1 second
  • retrieving batch 10 takes about 2 seconds
  • retrieving batch 25 takes about 4 seconds
  • retrieving batch 50 takes about 6 seconds
  • retrieving batch 75 takes about 10 seconds
  • retrieving batch 100 takes about 14 seconds

Assuming retrieving each batch of 200 pages takes a mean of 8 seconds, retrieving 20000 space pages would take 100*8=800 seconds, which is about 14 minutes.

Exporting this space (with empty dummy pages) via the Confluence space export function takes about 16 minutes:

Successful page migrations

Unfortunately there is not much that can be done about the time it takes to retrieve the list of pages. With release v1.6.8 WikiTraccs started to migrate pages while still retrieving the list of pages for a space. This at least allows the waiting times to be used to migrate the first pages.

Confluence 7.18 introduced an API that allows for faster page retrieval, but so far there was no demand I am aware of as migrated environments were older. If you are migrating a Confluence version 7.18 or newer then please comment here to make this demand visible: User faster page retrieval API (Confluence 7.18 and up).

Conclusion

Large Confluence spaces (more than 10000 pages) can add significantly to the migration time.

When WikiTraccs starts migrating a space it retrieves a list of all space pages, which can take several minutes for those large spaces. The list of pages is retrieved multiple times, for example after migrating the space to check if all pages were migrated.

Those times can add up.

So it’s good to know the spaces with large page numbers beforehand, or to learn about them during a test migration.

Ignoring macros when migrating pages from Confluence to SharePoint

This post describes the new macro ignore list feature of WikiTraccs.

Why do I need a macro ignore list?

Certain Confluence macros are only visible when editing a wiki page. Often WikiTraccs can see those macros and migrates them to the SharePoint page - where they will be visible for visitors as well.

One sample is the Excerpt macro with the hidden option set.

WikiTraccs offers a macro ignore list that can be used to exclude such macros from the migration.

How to use the macro ignore list?

This series of screenshots shows how to identify macros to ignore and how to configure the ignore list.

Here’s a Confluence page with one visible macro, as shown for readers:

The Confluence page in edit mode reveals the second hidden Excerpt macro:

Without a configured Macro Ignore List WikiTraccs will migrate the hidden macro to SharePoint, making it visible. Here’s a screenshot of such SharePoint page:

To solve this, open the Confluence page, choose View Storage Format for the page and copy the offending macro XML:

In WikiTraccs.GUI, in the menu bar, choose Settings > Configure Transformation. Paste the macro XML to the Macro Ignore List:

In the Macro Ignore List, remove every macro parameter except hidden:

Delete previously migrated pages as needed after changing the Macro Ignore List, and start the migration.

The macro is now removed by WikiTraccs. Here’s the migrated page again as it looks in SharePoint:

Any macro can be added to the Macro Ignore List. Macros matching the Macro Ignore List will be removed from all pages in the Confluence to SharePoint migration.

During migration, WikiTraccs looks at the macro name and its parameters. If it sees a macro that has the same name, and all of the parameters and parameter values as a macro in the Macro Ignore List it will skip this macro and won’t add it to the SharePoint page.

How to monitor if the filter works as expected?

WikiTraccs logs removed macros while migrating pages from Confluence to SharePoint.

Search for the phrase Removed macro via config by matching template in the WikiTraccs log file to see which macros were removed and which template from the Macro Ignore List did match.

In addition, the Site Pages column WT: Transformation Log contains this information as well.

Wrap-up

The Macro Ignore List allows to specify which macros should be skipped in any Confluence to SharePoint page transformation.

It is one feature that was born directly from customer feedback. And it’s another step in the direction of a more configurable transformation process.

Get your migration started today!

Give WikiTraccs a try and check out its transformation capabilities!

Quick start guide >>

Start today with WikiTraccs’ free Trial Version:

Download FREE WikiTraccs now

Or get in touch via email if you are interested in a demo. Give it 45 minutes and you’ll be up to speed on how WikiTraccs can help you.

Registering WikiTraccs as app in Azure AD

Azure AD is now Entra ID - please refer to the updated documentation: Registering WikiTraccs as app in Entra ID.

Registering WikiTraccs as app in Entra ID

This post shows how to get that Client ID needed by WikiTraccs.GUI.

I recently got the following inquiry about WikiTraccs:

Please explain again briefly what the Tenant ID and Azure Client IDs are and how to acquire them.

This request is not the first of its kind and totally understandable. It refers to the following configuration in WikiTraccs.GUI:

Unless you are a Microsoft 365 developer or admin you normally never have to care about those IDs.

Why do I have to enter Tenant ID and Client ID in WikiTraccs? Where do those come from?

When WikiTraccs creates your migrated pages in SharePoint it needs to access APIs provided by Microsoft. Those are the SharePoint API and the Microsoft Graph API.

Accessing SharePoint in any form requires authentication. Open <company>.sharepoint.com (replace ‘<company>’ with the actual value of your SharePoint) in a private browser tab and it will ask for your credentials like email address and password.

Authentication is also required when WikiTraccs starts the migration. It will open a browser window for you to sign in.

Besides users, applications like WikiTraccs have to authenticate as well to access SharePoint.

To allow an application to authenticate, it has to be registered in Entra ID (formerly known as Azure Active Directory, or Azure AD). This is a manual configuration that needs to be done once.

A registered application has an ID - the Client ID. And the Entra ID tenant it is created in also has an ID - the Tenant ID. Those are the IDs needed by WikiTraccs.GUI.

How to register WikiTraccs as Application in Entra ID?

In a browser, navigate to https://entra.microsoft.com.

Sign in with an account that has the Application Developer role in Entra ID.

The Microsoft Entra admin center (former Azure Active Directory portal) opens.

In the left menu, under Applications, select App registrations, then New registration:

The Register an application blade opens:

Configure the following:

  • under Name enter WikiTraccs Migration Tool (note: or any other name of your choosing)
  • under Supported account types select Accounts in this organizational directory only
  • under Redirect URI (optional) choose Public client/native (mobile & desktop) and type http://localhost in the edit field

Select Register to confirm the app registration.

The application’s settings are now open:

In the left menu, under Manage, choose API permissions.

Choose Add a permission to add the following permissions:

  • Microsoft Graph > Sites.FullControl.All (delegated)
  • SharePoint > AllSites.FullControl (delegated)

Make sure to select the Delegated permissions (not application permissions). Here’s a sample screenshot:

The configured permissions need to be consented by the tenant admin.

As a tenant admin, choose Grant admin consent for… and confirm:

In the left menu choose Overview and make note of both the Tenant ID (aka Directory ID) and Client ID (aka Application ID):

Those IDs need to be entered in WikiTraccs.GUI as Tenant ID and Azure AD Application Client ID.

What did I just configure? Can WikiTraccs now access all content in SharePoint?

NO, WikiTraccs can NOT access all content in SharePoint.

Let me explain.

When starting a migration with WikiTraccs you have to log in with a Microsoft 365 account.

Now the permissions WikiTraccs gets are the intersection of two things:

  • the delegated permissions you configured above (that is Sites.FullControl.All, AllSites.FullControl) AND
  • the permissions of the account you logged in with

This is the magic behind delegated permissions. WikiTraccs can only access as much of SharePoint as the logged-in account.

If you log in with an account that is owner in all sites - sure, WikiTraccs could now access those sites.

But if you log in with a dedicated migration account that has only access to some sites - WikiTraccs can now only access those, nothing more.

Are there alternatives to those permissions?

I hear from customers who have trouble to get the tenant admin to consent permissions.

Use less permissions, get less things migrated

If you can’t get admin consent for FullControl permissions you can try using the following Manage permissions instead:

  • Microsoft Graph > Sites.Manage.All (delegated)
  • SharePoint > AllSites.Manage (delegated)

Those allow a content migration as well.

But the functionality of WikiTraccs will be limited somewhat:

  • page permissions cannot be configured, as WikiTraccs won’t be allowed to do so
  • out-of-the-box SharePoint page and file metadata Created By, Created (Date), Modified By, Modified (Date) cannot be set, as this requires the same permissions as configuring permissions

Note that an admin can also grant admin consent to those permissions.

Use an existing application

If you have access to another Entra ID application that has the required permissions configured you can use the Client ID of this existing app.

That’s why there is the Use M365 PnP Client ID button in WikiTraccs.GUI. It enters the well-known ID of the application used by PnP.PowerShell. This only works, if you have access to and the needed permissions are configured for the application (this is not always the case!). Note: The multitenant PnP.PowerShell enterprise application has been retired by Microsoft.

Troubleshooting

Check the platform

There is one common oversight when configuring the Entra ID application: selecting the right platform.

You might see the following error when testing the connection to SharePoint or starting the migration if the platform is not correct:

“A configuration issue is preventing authentication - check the error message from the server for details. You can modify the configuration in the application registration portal. See https://aka.ms/msal-net-invalid-client for details. Original exception: AADSTS7000218: The request body must contain the following parameter: ‘client_assertion’ or ‘client_secret’.”

This error happens when the platform of the Entra ID application has been set to Web instead of Public client/native (mobile & desktop).

Change the platform to Public client/native (mobile & desktop), which then shows up as Mobile and desktop applications here:

The error should now be gone.

Check the redirect address after authenticating

After logging in to Microsoft 365 via the browser, a redirect to http://localhost happens in the browser address bar.

This has to look like this:

Normally WikiTraccs takes over from this point. Get in touch if there are issues.

Test connecting with PnP.PowerShell

You can use PnP.PowerShell to check if an authentication issue is related to your Entra ID application configuration, or related to WikiTraccs doing something wrong.

Start PowerShell 7 and install the PnP.PowerShell module.

Configure and run the following script:

# make sure to use PowerShell 7 and install the PnP.PowerShell module; use the following command for that
# Install-Module PnP.PowerShell -Scope CurrentUser

# enter your SharePoint site to connect to here (the same as in WikiTraccs, either WikiTraccs site or default target site):
$siteUrl = "https://contoso.sharepoint.com/sites/somesite"
# enter your Entra ID application client id (application id) here (the same as in WikiTraccs):
$clientId = "017d043a-df74-4ab0-a411-430104faeaa4"

# authenticate with app, as user, and open login dialog in new browser window
Connect-PnPOnline -Url $siteUrl -ClientId $clientId -Interactive -LaunchBrowser:$true
# this should print the title of the site we connected to, if the connection was successful
Get-PnPWeb

This PowerShell script should open the Microsoft 365 login experience in the browser, just like WikiTraccs does. After logging in, it should print the title of the site you connected to. Otherwise, there will be an error that might help diagnose the app configuration issue.

Check that the migration user has access to the target SharePoint sites

If the migration user has no access to the SharePoint site, the following error might be shown:

Error while testing connection to target site...
HttpResponseCode: 403
Code: System.UnauthorizedAccessException
Message: Attempted to perform an unauthorized operation

Make sure the migration user account is site collection administrator of all target sites and the WikiTraccs site.

Check that the migration user is site collection administrator of the target SharePoint sites

If the migration user has only limited access to the SharePoint site, the following error might be shown:

Successfully connected to target site '2024-11-05-permission-test-deleteme2' but permissions are missing - refer to the docs for details 
(view list items: YES, view pages: YES, allowed to approve items: NO, manage lists: NO, add and customize pages: NO, manage the site: NO).

Please give permissions, site admin works well. See log for details. Cannot continue for now.

This happens if the migration user is only visitor, member, or contributor in the site. Those limited permission levels don’t allow WikiTraccs to properly migrate pages.

Make sure the migration user account is site collection administrator of all target sites and the WikiTraccs site.

Check the log files for details

Note: this log output is avilable as of WikiTraccs v1.20.40.

WikiTraccs logs information about both the Entra ID authentication result and the SharePoint site access to the common log files.

When authenticating with Entra ID, WikiTraccs receives an access token. The claims of those access token can be seen in the log files:

[ 15:59:14 INF] #1 === START Claims of Access Token === | 
[ 15:59:14 INF] #1 aud=00000003-0000-0ff1-ce00-000000000000 | 
[ 15:59:14 INF] #1 amr=pwd | 
[ 15:59:14 INF] #1 amr=mfa | 
[ 15:59:14 INF] #1 app_displayname=Confluence Migration | 
[ 15:59:14 INF] #1 appid=abcae113-5a68-4df9-b0f9-70243943beef | 
[ 15:59:14 INF] #1 family_name=Vance | 
[ 15:59:14 INF] #1 given_name=Adele | 
...
[ 15:59:14 INF] #1 scp=AllSites.FullControl Sites.FullControl.All | 
[ 15:59:14 INF] #1 tid=1acdc182-0353-4ee1-b9fa-225ce104406b | 
[ 15:59:14 INF] #1 === END Claims of Access Token === | 

Above claims show that user Adele Vance authenticated with app Confluence Migration, using interactive login, being granted delegated permissions AllSites.FullControl and Sites.FullControl.All. You can use that to double-check your configuration.

Then, when accessing SharePoint, WikiTraccs logs information about the migration user’s access to the SharePoint site:

[ 15:59:14 INF] Logging SharePoint site 'Test Site' (/sites/test-site) permission details (for diagnosing access-related issues): | 
[ 15:59:14 INF] PermissionKind."ViewListItems": true | 
[ 15:59:14 INF] PermissionKind."AddListItems": false | 
[ 15:59:14 INF] PermissionKind."EditListItems": false | 
[ 15:59:14 INF] PermissionKind."ManageLists": false | 
...
[ 15:59:14 INF] PermissionKind."ManageWeb": false | 
[ 15:59:14 INF] PermissionKind."FullMask": false | 

Here, the migration user appears to have read-only access because they are allowed to view list items, but are not allowed to add list items, or manage lists.

Make sure the migration user account is site collection administrator of all target sites and the WikiTraccs site.

Incremental consent might be required, if, after successful authentication, WikiTraccs shows (and logs) messages that look like those:

No permission granted to SharePoint, whatsoever.

Some permission to SharePoint has been granted via other permission scopes, but not enough to even read content.

WikiTraccs asks for the special .default permission scope when the user authenticates with Entra ID. The resulting access token contains all permission scopes that have been consented to by an admin on behalf of the organization (note: this is required for all FullControl scopes), and scopes that have been already consented to by the migration account.

The latter will cause issues when working with Sites.Manage.All and AllSites.Manage scopes that don’t require admin consent. Those scopes might be missing from the access token. Refer to the log files to check the presence of those scopes.

Specifically, when Sites.Manage.All or AllSites.Manage are missing in WikiTraccs’ access token, you have to invoke the consent dialog for those scopes for the migration account with an individual consent URL that looks like this:

https://login.microsoftonline.com/00000000-0000-0000-0000-000000000000/oauth2/v2.0/authorize
?client_id=11111111-0000-0000-0000-000000000000
&response_type=code
&redirect_uri=http://localhost
&response_mode=query
&prompt=select_account
&scope=https://graph.microsoft.com/Sites.Manage.All+https://contoso.sharepoint.com/AllSites.Manage

Modify above URL template as follows:

  • Instead of 00000000-0000-0000-0000-000000000000, enter your SharePoint Tenant ID (note: you can see the tenant ID in the blue WikiTraccs window)
  • Instead of 11111111-0000-0000-0000-000000000000, enter the client ID of your Entra ID application
  • In https://contoso.sharepoint.com/AllSites.Manage replace contoso.sharepoint.com with your SharePoint host

So, the URL might look like this:

https://login.microsoftonline.com/6dbba70a-8629-4c31-9991-e2cbcd6f6d90/oauth2/v2.0/authorize?client_id=1df1b835-b821-413b-9c42-c67313e84c35&response_type=code&redirect_uri=http://localhost&response_mode=query&prompt=select_account&scope=https://graph.microsoft.com/Sites.Manage.All+https://adventureworks.sharepoint.com/AllSites.Manage

Open a browser where the migration account is authenticated to SharePoint, copy the final URL to the browser address bar and press the Return key, to navigate to this address. Follow the login flow.

Eventually, a consent dialog like the following should open in the browser:

Click Accept to give consent. This unblocks the migration.

Repeat the connection test in WikiTraccs. The error message should change, telling you that you don’t have full control access, but that migration should still work. Also, when starting the migration, it should now create pages in SharePoint.

Wrap-up

WikiTraccs needs an Entra ID application registration to access SharePoint. This is true for all applications integrating with Microsoft 365 services.

A common challenge is getting the right people to consent the configuration. The purpose of the app needs to be communicated clearly.

Technically, registering the app registration is quickly done following the steps in this post.

And last but not least, the migration user needs proper access to all target SharePoint sites.

Try WikiTraccs!

Give WikiTraccs a try and check out its transformation capabilities!

Quick start guide >>

Start today with WikiTraccs’ free Trial Version:

Download FREE WikiTraccs now

Or get in touch via email if you are interested in a demo. Give it 45 minutes and you’ll be up to speed on how WikiTraccs can help you.

The art of positioning images - part 2 of 2

How are images handled when migrating pages? In this blog post we look at actual migration results.

This blog post is the second of a two-parter about images in the context of a Confluence to SharePoint migration:

In the last post we looked at image positioning options in Confluence and SharePoint. How can images be formatted in Confluence? How in SharePoint?

Now we look at actual migration results.

The easy ones: images in their own paragraph

An image in a separate paragraph:

ConfluenceSharePoint

Apart from SharePoint taking more space in general the layout looks comparable.

Now a centered, floating image:

ConfluenceSharePoint

SharePoint can do that, good.

Off to something SharePoint cannot do well.

The problematic ones: images as “characters” in the text flow

Here is something that SharePoint cannot do: putting an image at the end of a line:

ConfluenceSharePoint

How about putting text to the right of an image:

ConfluenceSharePoint

This looks better, although WikiTraccs has to cheat here to make it look like this. This image is converted to a floating one, with the text flowing to the right. More on floating images in the next section.

By the way, putting images next to each other for Confluence is like putting characters next to each other. WikiTraccs lets those images float as well, which can look pretty decent in SharePoint:

ConfluenceSharePoint

And something that is just not possible in SharePoint: putting an image right inside the text, like a “giant character”:

ConfluenceSharePoint

Now we have a look at floating images.

The compatible ones: floating images

Text can flow around floating images on the left or on the right side.

Here is a left-floated image, with text flowing around it on the right side:

ConfluenceSharePoint

And here is a right-floated image, with text flowing around it on the left side:

ConfluenceSharePoint

A special case…

A common pattern that I could observe on Confluence pages is to put some explanatory text next to an image like so:

ConfluenceSharePoint

There is a trick WikiTracks needs to play to make this work at all.

WikiTraccs did insert a one-pixel “floatbreaker” image to wrap the text at the right spot, after the sentence “One lone sentence next to the image”.

Here you see the SharePoint page in edit mode, on the left with floatbreaker (selected in the editor), on the right after deleting it.

SharePoint with “floatbreaker”SharePoint without “floatbreaker”

The floatbreaker is another tiny, invisible image in the SharePoint page with positioning set to center. That breaks the text flow where it’s needed. And it’s the only way I found to stop the other text from flowing around as well.

Kind of hacky, but it works and it survives entering the page edit mode. Of course page authors are free to remove the dummy image and change the formatting.

Wrap-up

Migrating images from Confluence to SharePoint is no easy task for WikiTraccs. Confluence provides more formatting options than SharePoint.

And don’t mention Confluence pages that were created by importing from Word files! Somebody will start crying.

But WikiTraccs will make sense of the most common cases it finds. This should make those image-laden instruction pages readable in SharePoint as well.

Try WikiTraccs!

Give WikiTraccs a try and check out its transformation capabilities!

Quick start guide >>

Start today with WikiTraccs’ free Trial Version:

Download FREE WikiTraccs now

Or get in touch via email if you are interested in a demo. Give it 45 minutes and you’ll be up to speed on how WikiTraccs can help you.

The art of positioning images - part 1 of 2

How are images handled when migrating pages? In this blog post we look at what is possible in Confluence and SharePoint.

This blog post is the first of a two-parter about images in the context of a Confluence to SharePoint migration:

Off we go.

Why is image positioning important?

Hey ChatGPT, why is image positioning important in a Confluence wiki page?

Sounds about right.

Let’s look at what Confluence has up its sleeve for images.

How can images be positioned in Confluence?

Images as a “giant characters”

You can upload images and add them anywhere on a wiki page.

When inserting an image into text it behaves like one big character:

The image cut the word in half and sits there as if it were text, while taking its space.

Spoiler alert: SharePoint cannot do this. This will become relevant when looking at how WikiTraccs migrates such images to SharePoint.

Here is another sample, looking at it from different angles.

First there is the image, without text around it.

Then a line break on the right edge is removed, hoisting text up next to the image.

Then the image itself is hoisted up, now having text on its left edge.

And then the image is again put right into the text, and when making it smaller it starts to fit in.

This mode of positioning images works especially well for putting multiple images next to each other:

Now let’s look at another image positioning option.

Floating Images

There is a different way of positioning Confluence images known as floating.

Text can flow or wrap around such images.

Here is a sample on how to do this in Confluence Server 8:

And that’s it. Now we look at SharePoint.

How can images be positioned in SharePoint Online?

SharePoint does only support the second way of positioning images - floating them.

Here’s a demo of how inserting an image into text works in SharePoint:

The text is split into two parts above and below the image, the image is centered.

Selecting the image shows buttons that control image positioning:

This behavior looks like in Confluence.

Content is either wrapping on the left or right of the image, or the image is centered with text above and below it.

Now I tried to put two images next to each other.

I tried copy and paste. I tried to insert multiple at once. I tried moving them around.

Here’s me failing to position two images next to each other:

And that sums up the state of inline images in SharePoint pretty well. I was not able to put images next to each other.

Luckily WikiTraccs can control the content on a finer level than me with the mouse, while still creating content that adheres to SharePoint standard. More on that in the next post.

Try WikiTraccs!

Give WikiTraccs a try and check out its transformation capabilities!

Quick start guide >>

Start today with WikiTraccs’ free Trial Version:

Download FREE WikiTraccs now

Or get in touch via email if you are interested in a demo. Give it 45 minutes and you’ll be up to speed on how WikiTraccs can help you.

Confluence Page Tree in SharePoint

This post demos hierarchical page navigation in SharePoint.

How to create a dynamic page tree in SharePoint?

That is a question I hear time and time again.

Pages in SharePoint have no hierarchy (in contrast to Confluence). All modern pages are basically stored at the same level in the Site Pages Library.

Ever since starting working with SharePoint over 10 years ago I helped customers build trees of different sorts:

  • Search-driven page trees
  • Static page trees (deeper that the standard allows)
  • Taxonomy-driven page trees
  • Metadata-driven page trees

There is still no ready-made solution we can take and use to make navigating migrated pages easier. At least none that is free. (If you know one get in touch and I’ll update the post!)

Let’s effin build a page tree!

You’ll probably need a page tree in SharePoint if you are using WikiTraccs to migrate pages from Confluence. Users might be lost without it, depending on the number of pages.

Here is a demo of how such a page tree can look in SharePoint:

What are we seeing here?

The animation starts with a view of the Site Pages library. WikiTraccs was used to migrate the shown pages from Confluence to SharePoint.

Then a page is clicked, opening this page.

Above the page you can see a new bread crumb navigation and a page tree icon.

Clicking the page tree icon opens a panel on the right side that shows a dynamic page tree. The current page is highlighted. Any other page can be opened by selecting it in the tree.

Note: the page trees that are shown as part of the page content are static views of the Confluence page tree macro. This could be the next step: replacing those static placeholders with dynamic web parts. But that is something for the future.

How was it done?

The bread crumb and page tree above are metadata-driven.

When migrating pages to SharePoint WikiTraccs takes care of migrating page tree metadata as well:

  • Page ID
  • Parent page ID
  • Page order

Every migrated page carries this information in the Site Pages library.

The bread crumb and page tree are part of a SharePoint Framework (SPFx) solution.

The bread crumb is inserted above every page as application customizer SPFx extension.

Data is read from SharePoint using PnPjs.

And finally the user interface is a combination of Fluent UI and Material UI.

No additional backend code is involved, this is pure client-side code.

I want such a page tree!

Ok you can have it 😇.

Although it’s more of a demo to see if a page tree in this form proves useful.

Some things could certainly be done feature-wise. It also needs polishing in non-migrated-page contexts.

You could also build one on your own - the metadata is there!

Anyway, let me know if you want to try it. There is no official release, yet.

I’m trying to get the SPFx solution into Microsofts store AppSource. But I heard that this can be pretty complicated, so not sure if this will succeed. Fingers crossed.

As always: Try WikiTraccs!

Give WikiTraccs a try and check out its transformation capabilities!

Quick start guide >>

Start today with WikiTraccs’ free Trial Version:

Download FREE WikiTraccs now

Or get in touch via email if you are interested in a demo. Give it 45 minutes and you’ll be up to speed on how WikiTraccs can help you.

What about those Confluence Macros?

When referring to content migration it’s good to know which content is handled how, and macros are a big part of that.

Let’s start with something that has different meanings depending on context.

Content

Page content for space owners might encompass everything on and around a Confluence page: text, macros, attachments, metadata and maybe also comments and permissions.

All those things would ideally be migrated 1:1 to SharePoint. Migrate, turn off Confluence, done.

As you might expect: it’s not that easy.

There are different angles to approach this topic and the different types of content one might expect to be migrated.

For today I want to highlight one type of content that needs awareness and preparation: macros.

What’s in a macro?

Confluence macros can be used to display different types of content.

Macros can aggregate content from other pages. Or they can show search results. They also allow entering unique content, like the Code or Info macro.

When WikiTraccs encounters macros it has different strategies to handle them:

  • replace by something similar in SharePoint
  • replace by a static representation (e.g. Page Attachments and Page Tree macros)
  • replace by its rich content (e.g. Expand macro)
  • replace by a mere text placeholder

Let’s look at an example page in edit mode:

Above Confluence page features text and three macros:

  1. the Info macro, containing text that I entered
  2. the Favorite Pages macro that shows my favorite pages
  3. the Lorem Ipsum macro that generates a random paragraph of text

Here’s how this page looks after having been migrated to SharePoint:

This looks pretty similar.

WikiTraccs, when migrating the page, retrieved text, the Info macro plus content and markers for the other two macros. And because Favorite Pages and Lorem Ipsum web parts don’t exist in SharePoint they are represented by a text placeholder in the resulting page.

Let’s now look at the page in Confluence without being in edit mode:

Confluence knows what Favorite Pages and Lorem Ipsum macros are supposed to do and shows additional, dynamically generated, content.

This, of course, is not possible to replicate in SharePoint.

Now what?

There’s not much WikiTraccs can do about macros not being available in SharePoint.

WikiTraccs already has special handling built-in for a handful of frequently used macros, like the Page Attachments and Page Tree macros.

For many macros you’ll see a placeholder in the migrated page.

Let’s zoom out for a moment.

It’s best to see this as a chance rather than an obstacle. SharePoint is different from Confluence. And as part of Microsoft 365 there is a good chance that something powerful can be built on the Microsoft-side of things.

And zooming in again here are some points for the migration-planning check list:

  • create an inventory of spaces and check which can be excluded from the migration
  • create an inventory of macros that are in use on Confluence pages in to-be-migrated spaces
  • evaluate how removing macros might affect legibility and understandability of pages
  • take a look at SharePoint standard web parts as well as community solutions like PnP Samples
  • think about how the content will be used in SharePoint; is it like an archive or will the content be worked on?
  • identify use cases that are affected by macros not being available in SharePoint anymore
  • re-think use cases to SharePoint / Microsoft 365 where needed (don’t forget about Microsoft Teams, Viva, the Power Platform etc.)

Something wholistic like a transformation workshop or down-to-earth requirements analysis might be in place. (My colleagues at Communardo can support with those.)

With WikiTraccs it’s free and easy to just test the migration. Looking at the results can spark meaningful discussions as well.

Get started today!

Give WikiTraccs a try and check out its transformation capabilities!

Quick start guide >>

Start today with WikiTraccs’ free Trial Version:

Download FREE WikiTraccs now

Or get in touch via email if you are interested in a demo. Give it 45 minutes and you’ll be up to speed on how WikiTraccs can help you.

What to expect from WikiTraccs?

So what can you expect from the migration tool WikiTraccs? Or even more general: What to expect from any migration tool?

I often demo WikiTraccs to potential customers. Those demos spark discussions around the future structure, navigation patterns, usage scenarios, and more.

These are all topics that need to be worked on in the context of a migration project.

WikiTraccs can support your migration efforts.

It can migrate content from spaces to sites. It can migrate permissions and metadata. It has certain limitations. And it has distinct capabilities.

But there are things that WikiTraccs is not meant to do.

So let’s turn the headline around.

What to not expect from WikiTraccs

The following topics need to be worked on in the context of a transformation workshop, a migration project, or whatever format you choose to generate answers:

  • choose what to migrate - content only? metadata? permissions?
  • make decisions about which content goes from where to where - which spaces need to be migrated? which archived? which deleted? which sites do we migrate to? how many are there? how are they created? use new ones or existing ones?
  • know the amount of content to migrate (at least not in a demo context) - 10, 100 or 1000 spaces? 1000, 10000 or 100000 pages? 10, 100 or 1000 GB of attachments?
  • plan an access scheme for the SharePoint sites - users, groups, permissions given at which level in the SharePoint hierarchy of elements?
  • decide which macros are important for users, and which not
  • choose a strategy for your migration - big bang or in waves? who needs to be informed? how? when?
  • choose migration environments - need test environments? how many? where do those come from? how exact do they need resemble the production environment?
  • …and more, driven by your use cases

WikiTraccs cannot do this for you.

So what to expect from WikiTraccs?

WikiTraccs will help you realize the plans you made.

Let’s assume the following simplified migration plan:

  • 📄 migrate all Confluence pages, from all spaces (10000 pages overall; 20000 attachments, mostly images)
  • 🏗️ one target site for each source space
  • 👥 keep page author and editor, no permissions
  • 📝 accept that macros will mostly be lost and plan to educate users accordingly
  • ⚙️ run multiple test migrations
  • 📅 choose a weekend for the final migration

Here’s how WikiTraccs has got your back:

  • 📄 WikiTraccs migrates content and metadata
  • 🏗️ WikiTraccs migrates according to your space-to-site mapping
  • 👥 WikiTraccs replaces Confluence user accounts by Azure AD accounts according to your mapping
  • 📝 WikiTraccs handles macros according to the known macros list
  • ⚙️ you test-migrate as often as you need (the license is not bound to volumes of data)
  • 📅 using the experience from multiple test migrations verify a migration time of about 2 days

A demo is a good way to see this in action.

Testing yourself is even better. You can quickly put above steps into action in your environment.

Give it a try!

Give WikiTraccs a try and check out its transformation capabilities!

Quick start guide >>

Start today with WikiTraccs’ free Trial Version:

Download FREE WikiTraccs now

Or get in touch via email if you are interested in a demo. Give it 45 minutes and you’ll be up to speed on how WikiTraccs can help you.

New attachments macro transformation

What’s new regarding page attachments discoverability in SharePoint and the new attachments macro transformation?

WikiTraccs migrates a snapshot of the attachments macro

The Confluence attachments macro shows a list of page attachments, which looks like this:

A Confluence page with attachments macro.

WikiTraccs now transforms this macro to a table in the target SharePoint page. The result looks like this:

A SharePoint page with attachment links in a table.

This feels a lot better than the placeholder that has been generated before.

And do you see the Page Attachments section at the bottom? Let’s talk about that.

Attachments are also available via SharePoint web part

WikiTraccs creates a Page Attachments section on each SharePoint page with attachments. This allows for easy access of page attachments even on pages that had no page attachments macro.

The section is collapsed by default. Click the section title to expand it:

SharePoint list web part showing page attachments.

This section contains a standard SharePoint list web part. It shows all attachments of the page.

Where are those page attachments even stored??

SharePoint puts files that belong to pages in the Site Assets library. Each page has its own folder. And this is where WikiTraccs puts page attachments when migrating a page from Confluence to SharePoint.

The Site Assets library contains all page attachmetns.

The web part in a page’s Page Attachments section shows the content of the page’s folder.

Summary

Both the page attachment macro transformation and the list web part that shows page attachments should make working with page attachments a lot smoother. They are an addition to the latest release of WikiTraccs.

Give it a try

Give WikiTraccs a try and check out its transformation capabilities!

Quick start guide >>

Start today with WikiTraccs’ free Trial Version:

Download FREE WikiTraccs now

Or get in touch via email if you are interested in a demo. Give it 45 minutes and you’ll be up to speed on how WikiTraccs can help you.

Mapping principals and migrating permissions

This blog post covers new features allowing to migrate permissions and map users and groups from Confluence to SharePoint.

Terminology

We need to define some terms before starting into the topic.

The term permissions is used broadly to describe which audiences can access which content in Confluence or SharePoint, and to which extent. Those audiences can be users and groups. Users and groups are collectively referred to as principals.

When explicitly talking about permissions for Confluence pages the term page restrictions will be used as this is the correct Confluence terminology.

A sample permission migration

We’ll do a sample permission migration. Screenshots from this migration will be shown throughout the blog post.

For the sake of this sample migration, let’s assume we want to migrate the following Confluence pages:

Confluence Pages

Each of those pages has a different permission configuration. Some pages have view and edit restrictions set. Some for users, some for groups, and some for both. And some pages form a hierarchy to illustrate the challenges associated with that.

How to migrate permissions with WikiTraccs

WikiTraccs can migrate page restrictions from Confluence to SharePoint (within certain limits, more on that later). Migrating page restrictions is done in three steps.

STEP 1: Migrate content and configure mappings for users and groups

First, you migrate page contents. WikiTraccs stores Confluence page restriction and principal information in SharePoint lists, along the way.

When a page is migrated, WikiTraccs also receives page restrictions for this page as well as information about users and groups involved in those restrictions.

No permission configuration is applied to SharePoint pages, yet.

After successfully migrating the Confluence pages shown earlier (only content!) this is how the Pages Library of the migration target site looks:

Migrated Confluence Pages in SharePoint Pages Library

In addition, the document library Confluence Page Permission Snapshots (located in the WikiTraccs site) now contains permission snapshots for those pages, waiting to be applied by WikiTraccs:

Confluence Page Permission Snapshots

Those permission snapshots contain information about a page’s restrictions.

Then there’s a third list that WikiTraccs populates, the Confluence User and Group Mappings list (also located in the WikiTraccs site):

Confluence Users and Groups

You now configure the user and group mapping in this list. WikiTraccs needs to know which Confluence principal maps to which SharePoint principal.

STEP 2: Mapping Confluence users and groups to SharePoint

One factor for a successful permission migration is the mapping of Confluence users and groups to their counterparts in SharePoint. You configure this mapping manually.

Let’s configure the mapping like this, in the Confluence User and Group Mappings table (located in the WikiTraccs site):

Mapping Confluence to SharePoint Principals

The highlighted column WT_Setting_MapForPermissions contains accounts from Entra ID and SharePoint groups.

Those two mappings shall be highlighted in particular, as we’ll revisit them in STEP 3:

  • the Admin Heinrich (admin) Confluence account is mapped to two target accounts Wiki SpaceAdmin and Heinrich Admin Ulbricht
  • the Wiki Reader (wiki.reader) Confluence user is mapped to the Wiki Reader AD account

When you are done with the mapping, proceed with the next step.

STEP 3: Apply permissions to SharePoint pages

In this last step WikiTraccs applies the stored permission information to the already migrated SharePoint pages. This step has to be started manually and separately from the first step.

After choosing permission migration mode, start the migration as usual. WikiTraccs now applies the already migrated permission information to the already migrated pages in SharePoint.

We now have a look at a page permission migration result in depth.

In Confluence, the restrictions for page Page with view-only restriction for Wiki Reader look like this:

Restrictions of Page "Page with view-only restriction for Wiki Reader" in Confluence

Here’s the resulting permissions for the migrated SharePoint page:

Permissions of the Migrated SharePoint Page

Looking at the principals and permission levels this is what happened:

  • Confluence account “wiki.reader” was mapped to the “Wiki Reader” AD account and got Read permissions which corresponds to the Confluence view restriction
  • Confluence account “admin” was mapped to both the “Heinrich Admin Ulbricht” and the “Wiki SpaceAdmin” accounts and got Contribute permissions which corresponds to the Confluence edit permission
  • “WikiTraccs Admin” is the migration account and got Full Control permissions (the migration account is always added)

The basis for this permission configuration had been layed in step 2 when you configured the principal mappings.

Tracking permission migration progress

WikiTraccs logs the result of each page permission migration in the Page Transactions list, that is located in each migration target site in SharePoint.

For our sample migration this looks like this:

Permission Migration Results

There are several reasons why migrating Confluence page restrictions to SharePoint might fail.

The good thing is that a range of permission migration failures can be fixed easily. The permission migration can be run multiple times and will retry failed migrations. This can be repeated until all resolvable errors have been resolved.

Going back to our sample migration you might have noticed that (in the last screenshot) two permission migrations where not successful. For two pages the result is cannot merge hierarchical group restrictions when calculating merged page restrictions which means that for those pages restrictions could not be migrated.

The next section takes a deeper look at what WikiTraccs currently can and cannot do when it comes to migrating permissions.

Capabilities and limitations with regard to migrating permissions

WikiTraccs currently has two types of limitations when it comes to permission migrations: not-implemented features and nearly-impossible-to-add features.

But first, here is what WikiTraccs will do:

  • apply Confluence user and group page restrictions to SharePoint modern pages, when no hierarchy of page restrictions is involved
  • apply hierarchical page restrictions from Confluence to SharePoint modern pages in certain narrow circumstances (e.g. only users are involved)
  • provide a mapping opportunity for each Confluence principal to one or more Entra ID accounts and SharePoint groups
  • map the Confluence view and edit restrictions to corresponding SharePoint permission levels via a hard-wired logic

Features that might be useful in the future, but are not currently implemented:

  • using other group types than SharePoint groups as mapping target
  • provide fallbacks for when a mapping is missing or empty
  • configurable behavior for failed permission migrations (should the SharePoint page be restricted or not?)
  • more control over the permission migration scope (single pages, spaces, …)
  • migrating space and global permissions (those are currently not migrated and need to be set manually on the SharePoint site or sub site level)
  • allowing to configure SharePoint permission levels to be used in the target page (currently the out-of-the-box permission levels are used)
  • regularly updating SharePoint permission based on a configurable schedule

Features that will be hard or impossible to add:

  • migrating arbitrary hierarchies of page restrictions

Re-apply permissions after changing principal mappings

You can change the principal mapping that you configured in STEP 2, even during or after a permission migration.

To do so, proceed as follows:

  1. in the WikiTraccs SharePoint site, change principal mappings via the Confluence User and Group Mappings list
  2. in each migration target SharePoint site, delete entries from the Page Transactions table for pages that should have their permission configuration updated; otherwise WikiTraccs won’t update those pages
  3. in WikiTraccs, choose Permission Migration mode and start the migration

WikiTraccs will now apply page permissions again, using the already migrated information about Confluence page restrictions, and the updated principal mapping.

Summary

With the next release WikiTraccs adds the ability to migrate page restrictions from Confluence to SharePoint.

You’ll be able to configure which Confluence users and groups correspond to which SharePoint users and groups. WikiTraccs uses this information when applying unique permissions to SharePoint pages.

This release marks a milestone from which further development will be driven by your feedback. WikiTraccs could support several additional scenarios and development effort will be directed to areas with the most demand.

Furthermore, user and group mappings will be the basis for the migration of page metadata like author and editor.

Give it a try

Give WikiTraccs a try and check out its transformation capabilities!

Quick start guide >>

Start today with WikiTraccs’ free Trial Version:

Download FREE WikiTraccs now

Or get in touch via email if you are interested in a demo. Give it 45 minutes and you’ll be up to speed on how WikiTraccs can help you.

WikiTraccs Quick Start Video out now!

You can have your first migration running in less than 30 minutes.

This quick start video is all you need:

Download FREE WikiTraccs now

New release - UI overhaul

This covers the latest release 0.0.222 and its changes.

The user interface got some additions

The WikiTraccs.GUI user interface has been changed.

The settings have been grouped into steps: First configure the source, then configure the target, then test the configuration, and then start the migration.

Some configuration options are gone. You now use a SharePoint list to configure the Confluence spaces to migrate. This list shows basic information about all Confluence spaces and is filled by WikiTraccs.

Note: The idea behind using SharePoint lists for configuration has been described in more detail here: A new approach to configuring WikiTraccs

A big improvement are the progress indicators. You no longer have to rely on the console output of WikiTraccs to see progress. Instead, you get a progress bar for overall page progress and the progress of each migrating Confluence space:

Migration progress

Laying the foundation for user and group mapping, and permission migration

Starting with the current WikiTraccs release 0.0.222, you need to create a SharePoint site dedicated to WikiTraccs. This is the “WikiTraccs site”. You use this site to configure WikiTraccs, like choosing the spaces to migrate. And WikiTraccs uses the site to store metadata about the migration.

This site also holds two SharePoint lists, Confluence User and Group Mapping and Confluence Permission Snapshots. You will notice that those will be filled while migrating.

The current release does not yet support user or group mappings, or permission migration. But a future release might make use of this data.

Fixes and improvements

When migrating pages with large file attachments WikiTraccs could run into timeouts. This has been fixed.

Parallel file downloads have been added. WikiTraccs now loads two attachment files from Confluence at the same time, optimizing for large and small files.

The list of known edge cases in Confluence page content transformation has been expanded.

The retrieval of Jira issue information has been improved for pages with a large amount of linked Jira issues.

Summary

This release is a big step and lays the foundation to add two big missing features: user and group mapping, and permission migration. I’m excited for the next releases as they will bring WikiTraccs further to what one would expect from a migration tool.

Give WikiTraccs a try and check out its transformation capabilities!

Quick start guide >>

Start today with WikiTraccs’ free Trial Version:

Download FREE WikiTraccs now

I’m also eager to get feedback, so get in touch when you try it: Contact.

Measuring page migration success

This post is about the metrics that WikiTraccs provides about migrated Confluence pages.

Which metrics measure page migration success?

Each Confluence page consists of content, formatting, layout, attachments, links, mentions and more that has to be handled when transforming the page to SharePoint.

WikiTraccs calculates the following metrics for each migrated page:

  • the percentage of characters in the page that was migrated
  • the number of unknown macros
  • the number of links that could not be resolved
  • the number of migrated images
  • the number of user references in the page that could not be resolved
  • the number of transformation errors

Those numbers are saved as page metadata.

A SharePoint view for the Site Pages library can be used to visualize those numbers. Here is a migration result indicating success:

Successful page migrations

Source content issues

Below image shows the result for a page where not everything could be migrated. Not all images could be transferred. And there were two links to inaccessible pages:

Missing images and inaccessible pages

Errors like missing images and inaccessible content can often be solved by fixing permissions or broken content in the source Confluence system.

Also note the “3” in the “Crystal Ball Transformations” column. This means there were 3 macros that WikiTraccs doesn’t know yet. It used a generic transformation approach for those macros.

Transformation issues

Sometimes WikiTraccs cannot handle content properly (for whatever reason):

Migration errors

Above image shows that 12 elements in a page could not be transformed and only 51% of the page’s text content made it to the SharePoint page.

Those issues can be caused by faulty Confluence pages triggering transformation errors. There are already dozens of special cases and quirks that WikiTraccs works around. But there will always be cases that are new and need to be looked into.

For such a page, click on “View Entries” to see the content that cause the transformation issue. And ultimately a good thing to do might be opening an issue on GitHub.

Summary

In this post we looked at the metrics WikiTraccs uses to highlight migration success and migration issues. Everything being green is a good indication of a successful migration.

WikiTraccs counts content characters in each Confluence page and again counts characters in the target SharePoint pages. The result will be reported as “Text Transferred Percent”. This percentage is a good indicator if content is missing or not.

A new approach to configuring WikiTraccs

This post is about configuring WikiTraccs with SharePoint lists, a different approach to the GUI-based one planned so far.

So far my plan was to establish WikiTraccs.GUI as a visual frontend for configuring WikiTraccs.Console.

But I decided to change that approach a bit. WikiTraccs.GUI will still be application you start transformations with. But the configuration will mainly be done in SharePoint.

How is SharePoint used to configure WikiTraccs?

WikiTraccs stores information about the migration into SharePoint lists. You will create a site collection for WikiTraccs to use. This could be called “WikiTraccs Store” and holds data during the migration. It can be deleted after the migration.

When starting WikiTraccs it will create several lists there. One is the Confluence Space Inventory list. This list stores information about all spaces in Confluence, like space name and space key. But it also allows you to select the spaces to be migrated:

Confluence Space Inventory List in SharePoint

Above image shows the Confluence Space Inventory list that shows some spaces from a publicly available Confluence instance.

In the image on the right side, marked with a red rectangle, are two columns that allow you to:

  1. mark a space for migration via Yes/No column
  2. specify the target site for the migration via a text column to put the target site URL in

More columns might follow as needed. (Note: the column names are still technical and are yet to be named properly.)

Why use SharePoint for configuration?

One benefit of using SharePoint lists for configuration is that they already provide a way to display tabular data, like the space inventory. This data can be sorted and filtered out of the box.

List data can be exported to and imported from Excel files for others to review. So the list of spaces could be given to stakeholders to decide which spaces should be migrated. Then they are imported again and WikiTraccs migrates only the selected spaces.

Using SharePoint as a user interface also means there is less potential for programming errors in WikiTraccs since the interface is owned by Microsft.

SharePoint lists provide a familiar user interface, if users are already used to working with SharePoint. There is no new UI that has to be learned.

The data can be shared with select stakeholders by using SharePoint list or list item permissions.

You are free to run WikiTraccs on any machine, even multiple ones, and change the machine at will. Every instance of WikiTraccs will operate on the same configuration.

And last but not least you have the SharePoint Search at your disposal to locate information you need.

Wrap up

That’s it for the latest update. What do you think about using SharePoint lists for configuring a migration? Get in contact and let me know!

There are more lists that are relevant to WikiTraccs that will be covered in future blog posts:

  • Confluence User and Group Mapping
  • Confluence Permission Snapshots

As the names suggest those are relevant for mapping Confluence users and groups to SharePoint and to ensure the permissions and metadata can be rebuilt properly.

Announcing WikiTraccs

WikiTraccs migrates content from Confluence to SharePoint Online

I’m delighted to announce the first public release of WikiTraccs.

What is WikiTraccs?

WikiTraccs is a migration tool for Confluence to SharePoint Online migrations.

Which problem does it solve and who needs it?

WikiTraccs makes migrating content from Atlassian Confluence to SharePoint Online easier. It contains hundreds of transformation rules for a wide range of Confluence formattings, structures and macros.

A migration from Confluence might be beneficial for:

  • users of Confluence Server, since Atlassian will eventually phase out Server, urging users to migrate to Confluence Data Center or Confluence Cloud
  • users of Confluence Data Center, because it only buys some time before migrating to the Atlassian cloud might be inevitable
  • users of Confluence Cloud, because of license costs

Often Microsoft 365 licenses are available already, covering the use of SharePoint Online modern pages to display wiki content. In this case SharePoint Online is a potential target for the migration.

Is it ready for production?

No, not yet. The current release is an early access version. This means the basics work but a lot is still to be done.

The focus of this first release is to migrate content and structure of Confluence pages as reliable as possible. This includes:

  • Text
  • Formatting like bold, italics, underline etc.
  • Text colors and background colors
  • Headings
  • Task lists
  • Emoticons
  • Complex tables
  • Images, internal and external
  • Mentions
  • Page and attachment links, on the same page and to different pages and spaces
  • Layouts with multiple columns and rows
  • Macros - a wide range is supported out of the box, for unknown ones generic rules are applied

It has been tested on thousands of Confluence pages and dozens of gigabytes of images and attachments. Since every Confluence is unique with regard to its content there will be cases not covered. But those will be covered over time as feedback comes in.

WikiTraccs keeps track of every word that is present in a source Confluence page and checks if it is present in the target SharePoint page as well. It highlights pages where this is not the case. Thus pages with lacking transformation success can be identified easily and immediately.

The Feature Overview page will eventually cover all supported and not yet supported types of Confluence content and metadata.

You can also read more about how WikiTraccs works in How does WikiTraccs work?.

Which Confluence versions are supported?

WikiTraccs supports Confluence 6 and up, including Confluence Cloud. Confluence 5 and older are not supported.

How do I get started?

Have a look at How to get started?

If you want to get in touch have a look at the options you have for logging issues, start discussions or just give feedback.

What’s the meaning of “WikiTraccs”?

“WikiTraccs” is short for “Wiki Transformation Accelerators”. WikiTraccs for Confluence should indeed accelerate things compared to a manual migration.

There are a lot of interesting topics to be covered which will be done in a future posts!