Monitoring Per Selector Progress

This article explains how to get higher-level insights into the migration progress, by looking at each selector and how many pages have been migrated, and how many are still missing.

Terminology note: What is a “page”?

Note

Selectors are used to tell WikiTraccs which content to migrate. Have a look at the selector documentation to learn more.

Using progress log files to get insights

Note

Depending on whether you are running WikiTraccs.GUI or WikiTraccs.Console log files will be stored at different locations.

When using WikiTraccs.GUI, look at the WikiTraccs.GUI\logs folder. When using WikiTraccs.Console, look at the WikiTraccs.Console\logs folder.

Progress log files are stored in a sub folder that has the current date as its name, e.g. logs\2023-05-19.

As of release v1.1.0 WikiTraccs provides information about the migration progress in different progress-related log files:

Log files containing information about the Confluence to SharePoint migration progress.

The information you can gather from these files is:

How many pages are scheduled to be migrated?
Which pages have already been migrated to SharePoint?
Which pages are yet to be migrated?
Which pages have been migrated but need an update?

See below for a quick rundown of the log files and their content.

The values in those files are separated by tabulator. So it’s nearly CSV, but with tab instead of comma. The file name contains a timestamp, the Confluence site ID as specified in the configuration, the space key, and the last part of the target SharePoint site URL.

Progress log file documentation

Note

Multiple progress log files will be created with each migration run.

When starting the migration from Confluence to SharePoint WikiTraccs creates progress log files for each analyzed space. Those reflect the state before the migration of a space.

After migrating all scheduled spaces WikiTraccs creates a second batch of progress log files for each space that had pages migrated for. This will be done when going into idle mode. You may interrupt this if you don’t need those files.

When pressing Ctrl+C in WikiTraccs.Console (to cancel the running operation) it is not guaranteed that the final batch of progress log files will be generated. To get the current state just start the migration again and the files will be generated for each scheduled space.

Keep an eye on the file name and timestamp when you search for the most recent progress log files.

Note

The content of these files is subject to change as customer feedback keeps coming in, pointing to additional information that would be valuable.

xxx__10-not-yet-migrated-pages.txt

This file contains information about Confluence pages that are yet to be migrated.

Sample content:

CASIG	78022359	CA2SIG - Meeting November 29	/display/CASIG/CA2SIG+-+Meeting+November+29
CASIG	78022377	2022-11-22 Standard WG	/display/CASIG/2022-11-22+Standard+WG
CASIG	80773916	2022 12 06 Standards WG	/display/CASIG/2022+12+06+Standards+WG

The tab-separated columns in this file are:

Confluence space key
Confluence page ID
Confluence page title
Confluence page URL

xxx__20-migrated-pages

This file contains information about migrated pages. It contains information about existing Confluence pages where a corresponding SharePoint page exists as well.

Sample content:

CASIG	24781062	Climate Action and Accounting SIG Home	/display/CASIG/Climate+Action+and+Accounting+SIG+Home	97652
CASIG	24781122	Meetings	/display/CASIG/Meetings	97653
CASIG	24781124	Member Directory	/display/CASIG/Member+Directory	97654

The tab-separated columns in this file are:

Confluence space key
Confluence page ID
Confluence page title
Confluence page URL
SharePoint page ID (in the Site Pages library)

xxx__25-update-state-of-migrated-pages

This file contains information about the freshness of migrated pages.

Note

The file format changed with WikiTraccs release v1.7.

Sample content:

+++
SourceTenantId = "https://confluence.contoso.com/"
PageSelectorType = "ConfluenceSpace"
PageSelector = "CASIG"
CreationDateUtc = 2023-01-21T15:30:07.1691711
SchemaVersion = 2
+++
CASIG	Page	78022313	2022-11-21 Peer Programming Call	/display/CASIG/2022-11-21+Peer+Programming+Call	2022-11-30T22:17:28	2022-11-29T23:17:28 2022-11-29T23:17:28	needsupdate
CASIG	Page	78022335	CA2 SIG - Meeting November 15	/display/CASIG/CA2+SIG+-+Meeting+November+15	2022-11-24T06:52:40	2022-11-24T06:52:40 2022-11-29T23:17:28	uptodate
CASIG	Page	78022347	CA2 SIG - Meeting December 13	/display/CASIG/CA2+SIG+-+Meeting+December+13	2022-12-14T01:46:57	2022-12-14T01:46:57 2022-11-29T23:17:28	uptodate

The file contains a header that is enclosed with +++.

Following the header, the list of page update states starts.

The tab-separated columns are:

Confluence space key
Confluence content type: Page or Blogpost
Confluence page ID
Confluence page title
Confluence page URL
Modification date of the Confluence page
Stored modification date of the migrated page in SharePoint (this is the Confluence page modification date at the time of migration)
Modification date of the SharePoint page (note: added in release v1.12.24)
State of the SharePoint page
- uptodate: this page is up to date
- needsupdate: this page has been updated in Confluence since its migration
- cannotdetermine: metadata in SharePoint is missing, cannot determine if update is needed

Note: WikiTraccs will output additional state values in a verification run.

Tip

This file can be used to update outdated pages. See Updating previously migrated pages for details.

xxx__30-aggregated-info

This file contains information that could be gathered from the other files, but already aggregated:

Sample content:

Source Confluence Site: https://wiki.hyperledger.org
Target SharePoint Site: https://contoso.sharepoint.com/sites/migration-target
Space Key: CASIG
Blog posts included in migration and calculation: no
Confluence page count for space space CASIG: 292
Migrated SharePoint pages that correspond to found Confluence pages in space CASIG: 259
Migrated SharePoint pages overall for space CASIG: 259
Pages yet to be migrated for space CASIG: 33

If Migrated SharePoint pages overall for space is larger than Migrated SharePoint pages that correspond to found Confluence pages in space then pages turned inaccessible in Confluence (deleted? permission denied?) but the once-migrated pages in SharePoint still exist.

Progress log file cadence

WikiTraccs creates progress log files at specific times:

when starting a migration - log files for each scheduled space will be created
when stopping a running migration by pressing Ctrl+C in the console window of WikiTraccs.Console - log files for each space handled so far will be created
when the migration is done - log files for each space handled so far will be created

This means that multiple progress log files per space will be created. Look at the ones with the most recent timestamp to see the latest progress information.

Note

WikiTraccs does not create a file if there is nothing to log. So some progress log files can be missing and that’s expected.

You can delete or archive old progress log files. Every new migration run will create new files.

Troubleshooting missing pages

Have a look at the troubleshooting recipe: Find out why some pages won’t migrate.

Last modified December 11, 2024