How much time will a Confluence to SharePoint migration take?
The migration time depends heavily on the source content in Confluence, and the network speed, both for downloads and uploads.
We’ll do some test migrations to have real-world data to look at.
Summary (TL;DR)
Here are the average migration times per page, taken from test migrations detailed further down this post:
- 07 seconds (~2h per 1000 pages) (note: few attachments)
- 11 seconds (~3h per 1000 pages)
- 12 seconds (~3h per 1000 pages)
- 12 seconds again
- 13 seconds (~4h per 1000 pages)
- 15 seconds (~4h per 1000 pages)
- 20 seconds (~6h per 1000 pages)
- 35 seconds (~9h per 1000 pages) (note: lots of attachments)
Read further to get details.
How does the source content influence migration speed?
The following list shows migration activities that are always present and can take quite some time:
- creating a page in SharePoint and setting up the metadata around it
- uploading files to SharePoint Online (mainly on slow connections)
- downloading files from Confluence (mainly on slow connections)
Here are some activities that also take time, but are not always needed, depending on the source content:
- resolving Confluence links - WikiTraccs will check every link it finds to know where the link links to, to properly transform the link
- resolving @-mentions - this involves looking up user information, writing this to the user mapping table, and trying to find the user in SharePoint
- transforming Jira macros - this involves reaching out to Jira; Confluence’s Jira integration is rather slow
If you migrate pages with only text content and no attachments they’ll migrate in no time.
If you migrate pages with 500 attachments each, it might take minutes for each page to be migrated.
Using live migration metrics to get insights
Note
Those metrics are only available in Migrate content mode.With release v1.22.1, WikiTraccs introduced live metrics that show how long migrating pages take (those are also written to the log file). Those metrics also provide insights into which of the mentioned activities - downloading and uploading content, resolving links, etc. - happen, and how much time they take.
You can read more about those live metrics here: Live Progress Indicators.
Since the migration times depend on the source content, we need some source content to perform actual migrations, to get actual data to look at.
Thankfully, there are plenty of open Confluence sites available to run some measurements; some run by hobbyists, many by universities, some by non-profits and NGOs.
Testing migrations using open Confluence sites
We’ll pick some of those open Confluence sites, not based on the meaning of their content, but on the amount and type of their content. I’m not affiliated with any of those sites. They just provide a great way to test one thing or another.
Note: over time the accessibility of those sites will change. Some will close, some will move. The number of Confluence on-prem instances decreases, the number of Confluence cloud sites increases.
Note
“Open site” means a Confluence site on the internet that has anonymous access enabled, so that content is freely accessible by everyone visiting the site, and also indexed and discoverable by search engines like Google.Again: those sites are purely chosen by googling for site:atlassian.net and clicking through sites until finding some that have a decent amount of spaces and pages.
Migration machines used for test migrations
Test migrations were performed on the following migration machines:
“Mig VM”
An Azure VM with the following characteristics:
- Operating System: Windows 10
- Size: B2s
- vCPUs: 2
- RAM: 4 GB
- Max IOPS: 1280
- Cost per month: €32.80
- Download speed: 1200 Mbps
- Upload speed: 1800 Mbps
“Mig Laptop”
A laptop:
- Operating System: Debian Linux
- vCPUs: 6
- RAM: 40 GB
- Download speed: 93 Mbps
- Upload speed: 38 Mbps
Download and upload speed were determined using one of the many free internet sites that measure those values.
Confluence Cloud Site: University of Texas at Austin
The UT is providing us with an site that has over 300 spaces, many small, but two bigger ones.
Migration Metrics for Space 1
- Migration Machine: Mig VM
- Migrated number of pages: 801
- Migration duration: 92 minutes
Here are the metrics (as logged into the common log files) 1 hour into the migration, based on data of this hour, covering 410 pages:
Median time (IQR) : 4 s/item
Median time : 4 s/item
75th percentile (IQR): 7 s/item
75th percentile : 9 s/item
Mean time (IQR) : 5 s/item
Mean time : 8 s/item
Avg links transformed: 2/item
Have files : 22% of items
File count avg : 2.3 per item that has files
File size avg : 1649 KB/file
Download speed : ~485 KB/s
Upload speed : ~373 KB/s
File size sum down : 1299.6 MB
File size sum up : 1293.5 MB
Top activities were:
(SharePoint, Page, Content) : 28,52 min
(SharePoint, WikiTraccs, Prerequisites) : 14,98 min
(Confluence, Page, Link_Soft) : 14,69 min
There are only a few attachments in this space, only every fifth page has about 2 attachments. This is probably the reason why file upload and download are not in the top activities.
Most present are activities that require a lot of calls to APIs, like checking prerequisites, prepping the page, setting its content, and setting metadata for page and attachment folder.
In the end, looking at the overall migration duration and number of pages, the average migration duration was 7 seconds per page.
Migration Metrics for Space 2
- Migration Machine: Mig VM
- Migrated number of pages: 1233
- Migration duration: 12h 10m
Here are the metrics (as logged into the common log files) 1 hour into the migration, based on data of this hour, covering 140 pages:
Median time (IQR) : 11 s/item
Median time : 13 s/item
75th percentile (IQR): 27 s/item
75th percentile : 31 s/item
Mean time (IQR) : 17 s/item
Mean time : 25 s/item
Avg links transformed: 15/item
Have files : 56% of items
File count avg : 13.6 per item that has files
File size avg : 1235 KB/file
Download speed : ~297 KB/s
Upload speed : ~218 KB/s
File size sum down : 425.6 MB
File size sum up : 427.5 MB
Top activities were:
(SharePoint, File, Content) : 17.19 min
(SharePoint, Page, Content) : 10.21 min
(Confluence, File, Content) : 9.86 min
Compared with the first space, pages in this space have a lot of attachments. Half of the pages have an average of 13 attachments, each about 1,5 MB in size. That’s why the download and upload activities take more time in this space. All page metrics that measure time are about triple of what they were for the first space.
Remember, those values are measured over a period of 60 minutes. Let’s look at how the values changed after 4 hours of migration, covering 89 page-like contents:
Median time (IQR) : 29 s/item
Median time : 29 s/item
75th percentile (IQR): 47 s/item
75th percentile : 50 s/item
Mean time (IQR) : 35 s/item
Mean time : 40 s/item
Avg links transformed: 3/item
Have files : 91% of items
File count avg : 19.6 per item that has files
File size avg : 499 KB/file
Download speed : ~495 KB/s
Upload speed : ~357 KB/s
File size sum down : 919.3 MB
File size sum up : 957.0 MB
Top activities were:
(SharePoint, File, Content) : 24.83 min
(Confluence, File, Content) : 14.17 min
(SharePoint, Page, Content) : 7.61 min
All times went up. The number of pages having attachments went from 56% to 91% - those file downloads and uploads take time.
In the end, looking at the overall migration duration and number of pages, the average migration duration was 35 seconds per page.
Confluence Cloud Site: Geocaching.com Wiki
This site had one space.
Migration Metrics for Space
- Migration Machine: Mig Laptop
- Migrated number of pages: 262 (whole site)
- Migration duration: 53 minutes
Metrics for all 262 pages:
Median time (IQR) : 5 s/item
Median time : 5 s/item
75th percentile (IQR): 12 s/item
75th percentile : 14 s/item
Mean time (IQR) : 8 s/item
Mean time : 12 s/item
Avg links transformed: 4/item
Have files : 35% of items
File count avg : 2.5 per item that has files
File size avg : 382 KB/file
Download speed : ~173 KB/s
Upload speed : ~96 KB/s
File size sum down : 234.7 MB
File size sum up : 234.7 MB
Top activities were:
(SharePoint, Page, Content) : 22.15 min
(Confluence, Page, Link_Soft) : 12.91 min
(Confluence, Page, Transformation_Macro_Other) : 12.12 min
In the end, looking at the overall migration duration and number of pages, the average migration duration was 12 seconds per page.
Confluence Cloud Site: Xilinx Wiki
This site had one space.
Migration Metrics for Space
- Migration Machine: Mig VM
- Migrated number of pages: 1284 (whole site)
- Migration duration: 7h 22m
Here are the metrics (as logged into the common log files) 1 hour into the migration, based on data of this hour, covering 221 pages:
Median time (IQR) : 10 s/item
Median time : 10 s/item
75th percentile (IQR): 18 s/item
75th percentile : 21 s/item
Mean time (IQR) : 14 s/item
Mean time : 15 s/item
Avg links transformed: 7/item
Have files : 61% of items
File count avg : 6.7 per item that has files
File size avg : 1419 KB/file
Download speed : ~724 KB/s
Upload speed : ~434 KB/s
File size sum down : 1264.3 MB
File size sum up : 1264.3 MB
Top activities were:
(SharePoint, File, Content) : 22.25 min
(Confluence, Page, Transformation_Macro_Other) : 20.55 min
(SharePoint, Page, Content) : 15.97 min
In the end, looking at the overall migration duration and number of pages, the average migration duration was 20 seconds per page.
Confluence Cloud Site: Duke Universtiy
This site had 120 spaces.
Migration Metrics for whole Site
- Migration Machine: Mig Laptop
- Migrated Migrated number of pages: 3117 (whole site)
- Migration duration: 12h 46m
Here are the metrics (as logged into the common log files) 1 hour into the migration, based on data of this hour, covering 330 pages:
Median time (IQR) : 5 s/item
Median time : 5 s/item
75th percentile (IQR): 11 s/item
75th percentile : 13 s/item
Mean time (IQR) : 8 s/item
Mean time : 10 s/item
Avg links transformed: 2/item
Have files : 43% of items
File count avg : 2.6 per item that has files
File size avg : 421 KB/file
Download speed : ~239 KB/s
Upload speed : ~104 KB/s
File size sum down : 463.5 MB
File size sum up : 463.5 MB
Top activities were:
(SharePoint, Page, Content) : 23.05 min
(SharePoint, File, Content) : 15.38 min
(SharePoint, WikiTraccs, Prerequisites) : 11.04 min
Let’s look at how the values changed after 4 hours of migration, covering 110 page-like contents:
Median time (IQR) : 14 s/item
Median time : 14 s/item
75th percentile (IQR): 59 s/item
75th percentile : 59 s/item
Mean time (IQR) : 32 s/item
Mean time : 32 s/item
Avg links transformed: 0/item
Have files : 65% of items
File count avg : 17.8 per item that has files
File size avg : 399 KB/file
Download speed : ~444 KB/s
Upload speed : ~212 KB/s
File size sum down : 567.1 MB
File size sum up : 587.2 MB
Top activities were:
(SharePoint, File, Content) : 35.10 min
(Confluence, File, Content) : 15.23 min
(SharePoint, Page, Content) : 10.87 min
In the end, looking at the overall migration duration and number of pages, the average migration duration was 15 seconds per page.
Confluence Cloud Site: Harvard University
This site had 46 spaces.
Migration Metrics for whole Site
- Migration Machine: Mig VM
- Migrated Migrated number of pages: 6647 (whole site)
- Migration duration: 20h 23m
Here are the metrics (as logged into the common log files) 1 hour into the migration, based on data of this hour, covering 211 pages:
Median time (IQR) : 7 s/item
Median time : 9 s/item
75th percentile (IQR): 15 s/item
75th percentile : 17 s/item
Mean time (IQR) : 10 s/item
Mean time : 17 s/item
Avg links transformed: 6/item
Have files : 50% of items
File count avg : 4.2 per item that has files
File size avg : 822 KB/file
Download speed : ~371 KB/s
Upload speed : ~199 KB/s
File size sum down : 1200.9 MB
File size sum up : 1200.9 MB
Top activities were:
(SharePoint, Page, Content) : 14.74 min
(SharePoint, File, Content) : 13.92 min
(Confluence, Page, Link_Soft) : 10.44 min
Let’s look at how the values changed after 4 hours of migration, covering 325 page-like contents:
Median time (IQR) : 4 s/item
Median time : 4 s/item
75th percentile (IQR): 9 s/item
75th percentile : 11 s/item
Mean time (IQR) : 7 s/item
Mean time : 11 s/item
Avg links transformed: 2/item
Have files : 35% of items
File count avg : 2.6 per item that has files
File size avg : 558 KB/file
Download speed : ~165 KB/s
Upload speed : ~105 KB/s
File size sum down : 340.8 MB
File size sum up : 340.7 MB
Top activities were:
(SharePoint, Page, Content) : 22.18 min
(SharePoint, File, Content) : 12.34 min
(SharePoint, WikiTraccs, Prerequisites) : 11.77 min
In the end, looking at the overall migration duration and number of pages, the average migration duration was 11 seconds per page.
Confluence Cloud Site: Hyperledger Foundation
This site had 74 spaces.
Migration Metrics for whole Site
- Migration Machine: Mig VM
- Migrated Migrated number of pages: 9238 (whole site)
- Migration duration: 32h 40m
Here are the metrics (as logged into the common log files) 1 hour into the migration, based on data of this hour, covering 390 pages:
Median time (IQR) : 7 s/item
Median time : 7 s/item
75th percentile (IQR): 9 s/item
75th percentile : 9 s/item
Mean time (IQR) : 7 s/item
Mean time : 9 s/item
Items per hour : 400-514 (based on IQR Median and IQR 75th percentile)
Avg links transformed: 3/item
Have files : 62% of items
File count avg : 1.1 per item that has files
File size avg : 294 KB/file
Download speed : ~679 KB/s
Upload speed : ~94 KB/s
File size sum down : 1618.1 MB
File size sum up : 1619.3 MB
Top activities were:
(Confluence, Page, Link_Soft) : 53.22 min
(SharePoint, Page, Content) : 29.33 min
(Confluence, Page, Transformation_UserMention) : 17.99 min
Let’s look at how the values changed after 4 hours of migration, covering 239 page-like contents:
Median time (IQR) : 14 s/item
Median time : 14 s/item
75th percentile (IQR): 18 s/item
75th percentile : 18 s/item
Mean time (IQR) : 14 s/item
Mean time : 15 s/item
Items per hour : 200-257 (based on IQR Median and IQR 75th percentile)
Avg links transformed: 4/item
Have files : 53% of items
File count avg : 1.6 per item that has files
File size avg : 370 KB/file
Download speed : ~102 KB/s
Upload speed : ~81 KB/s
File size sum down : 93.1 MB
File size sum up : 93.1 MB
Top activities were:
(SharePoint, Page, Content) : 24.94 min
(SharePoint, File, Content) : 13.86 min
(Confluence, Page, Link_Soft) : 13.01 min
In the end, looking at the overall migration duration and number of pages, the average migration duration was 13 seconds per page.
Confluence Cloud Site: logica.atlassian.net
This site had 19 spaces.
Migration Metrics for whole Site
- Migration Machine: Mig VM
- Migrated Migrated number of pages: 1494 (whole site)
- Migration duration: 5h 9m
Here are the metrics (as logged into the common log files) 1 hour into the migration, based on data of this hour, covering 489 pages:
Median time (IQR) : 4 s/item
Median time : 4 s/item
75th percentile (IQR): 4 s/item
75th percentile : 6 s/item
Mean time (IQR) : 4 s/item
Mean time : 7 s/item
Items per hour : 900-900 (based on IQR Median and IQR 75th percentile)
Avg links transformed: 1/item
Have files : 13% of items
File count avg : 1.5 per item that has files
File size avg : 372 KB/file
Download speed : ~127 KB/s
Upload speed : ~99 KB/s
File size sum down : 294.0 MB
File size sum up : 294.2 MB
Top activities were:
(SharePoint, Page, Content) : 30.43 min
(Confluence, Page, Link_Soft) : 17.85 min
(SharePoint, WikiTraccs, Prerequisites) : 17.11 min
Let’s look at how the values changed after 4 hours of migration, covering 255 page-like contents:
Median time (IQR) : 4 s/item
Median time : 4 s/item
75th percentile (IQR): 7 s/item
75th percentile : 11 s/item
Mean time (IQR) : 6 s/item
Mean time : 14 s/item
Items per hour : 514-600 (based on IQR Median and IQR 75th percentile)
Avg links transformed: 1/item
Have files : 30% of items
File count avg : 2.9 per item that has files
File size avg : 2069 KB/file
Download speed : ~705 KB/s
Upload speed : ~451 KB/s
File size sum down : 13192.5 MB
File size sum up : 13828.6 MB
Top activities were:
(SharePoint, Page, Content) : 14.34 min
(SharePoint, WikiTraccs, Prerequisites) : 8.29 min
(SharePoint, File, Content) : 7.54 min
In the end, looking at the overall migration duration and number of pages, the average migration duration was 12 seconds per page.
Wrap
In this blog post we looked at some metrics from real test migrations to get a feeling about how long a migration can take.
Live metrics can only predict the future if we assume that pages are comparable in their characteristics (mainly number of attachments), which might not be the case. Thus, the metrics will change over the course of the migration as metrics are always calculated based on data from the last 60 minutes.
So far it seems that the metrics Mean Time (IQR) and 75th percentile (IQR) could be predictors for the overall migration time within a space.
Over time, I’ll add more samples to this blog post and might also update the values, as the metrics calculation evolves.