SharePoint Online PnPProvisionning include document library file contents and file Versions !

Last year I wrote an article about using the PnP Provisioning engine to add references to documents in the Template file generated.

A couple of days a friend on linked in asked me if it were possible to include also all the documents versions in the Template file generated. So though about it and came up with a solution.

Basically the idea is of course to get all the versions of the document then download them and inject their reference in the Template pnp provisioning file.

First you have to know that when you get a reference to a file with

$folder = Get-PnPFolder -RelativeUrl $folderUrl
For ($i = 0; $i -lt $total; $i++) {
        $file = $folder.Files[$i]

The $file.Versions will return an error as the collection has not being initialised. This is because the collection property has not been specified. So in order to load the .Version collection you have to specifically load it with

$total = $folder.Files.Count
$ctx = Get-PnPContext
	
For ($i = 0; $i -lt $total; $i++) {
        $file = $folder.Files[$i]
        
        $ctx.load($file.Versions)
        $ctx.ExecuteQuery()

Now we can iterate through the Version collection and get the Url property to download the file.
Again I am not sure why if I used the Get-PnpFile -Url $file.Url it was returning me an error 404 not found so I used the WebClient object to download the file with

$webClient = New-Object System.Net.WebClient 
$webClient.Credentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($userName, $sPassword)
$webClient.Headers.Add("X-FORMS_BASED_AUTH_ACCEPTED", "f")

$webclient.DownloadFile($ServerFileLocation,$DownloadPath)

If we put it all together my ProcessFolder function becomes

function ProcessFolder($folderUrl, $destinationFolder) {
	Write-Output "Folder URL " $folderUrl  " destinationFolder " $destinationFolder
    $folder = Get-PnPFolder -RelativeUrl $folderUrl
    $tempfiles = Get-PnPProperty -ClientObject $folder -Property Files
   
    if (!(Test-Path -path $destinationfolder )) {
        $dest = New-Item $destinationfolder -type directory 
    }

    $total = $folder.Files.Count
	$ctx = Get-PnPContext
	
    For ($i = 0; $i -lt $total; $i++) {
        $file = $folder.Files[$i]
        
		$ctx.load($file.Versions)
        $ctx.ExecuteQuery()

		foreach($version in $file.Versions)
		{
			$filesplit = $file.Name.split(".") 
			$fullname = $filesplit[0] 
			$fileext = $filesplit[1] 
			$FullFileName = $fullname+"\"+$version.VersionLabel+"\"+$file.Name         

			$fileURL = $destination+"/"+$version.Url


			$DownloadPath = $FullFileName

			if (!(Test-Path ($destinationfolder + "\" + $fullname + "\" + $version.VersionLabel)))
			{
				New-Item ($destinationfolder + "\" + $fullname + "\" + $version.VersionLabel) -type directory -Force
			}

			HTTPDownloadFile "$fileURL" ($destinationfolder + "\" + $fullname + "\" + $version.VersionLabel + "\" + $file.Name)
			
			$versionSourceFolder =  "./" + $siteTitle + "/" + $folder.Name + "/" + $fullname + "/" + $version.VersionLabel + "/" + $file.Name
			Add-PnPFileToProvisioningTemplate -Path ($saveDir + "Template.xml") -Source $versionSourceFolder -Folder $folderUrl -FileLevel Published
		}
		
        Get-PnPFile -ServerRelativeUrl $file.ServerRelativeUrl -Path $destinationfolder -FileName $file.Name -AsFile -Force	

		Add-PnPFileToProvisioningTemplate -Path ($saveDir + "Template.xml") -Source ($destinationfolder + "\" + $file.Name) -Folder $folderUrl -FileLevel Published
		
    }
	
}

with the HTTPDownloadFile function

function HTTPDownloadFile($ServerFileLocation, $DownloadPath)
{
	$userName = "LOGIN_NAME"
	$password = "PASSWORD"

	#create secure password
	$sPassword = $password | ConvertTo-SecureString -AsPlainText -Force

	$webClient = New-Object System.Net.WebClient 
	$webClient.Credentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($userName, $sPassword)
	$webClient.Headers.Add("X-FORMS_BASED_AUTH_ACCEPTED", "f")

    $webclient.DownloadFile($ServerFileLocation,$DownloadPath)
}

The complete script is available on github at https://github.com/alaabitar/provisioning/blob/master/scriptUpgraded.ps1

Business card reader with PowerApps

AI Builder (still in preview) for PowerApps comes with a control called Business card reader and it does just its name is :)

The control exposes several properties from Company Name to Job Title, Phones etc... so basically you just click on the control select take a picture of load a picture from your gallery or hard drive and you just relax and let the control do what it does best. It takes a couples of seconds to load the different properties.

I have tested it with several different kind of business cards I can say that I had an 80% of success.

I have upload a small PowerApps app available at https://github.com/alaabitar/powerapps/blob/master/BusinessCardReader_20190625085751.zip

It reads the information from the Business card reader control and a Save button uploads the contact to a SharePoint lists named Contacts.

Here is the PnP PowerShell script to create the Contacts list :

Connect-PnPOnline -url URL_OF_YOUR_SITE

New-PnPList -Title "Contacts" -Template GenericList

Add-PnPField -List "Contacts" -DisplayName "CompanyName" -InternalName "CompanyName" -Type Text -AddToDefaultView
Add-PnPField -List "Contacts" -DisplayName "Department" -InternalName "Department" -Type Text -AddToDefaultView
Add-PnPField -List "Contacts" -DisplayName "Email" -InternalName "Email" -Type Text -AddToDefaultView
Add-PnPField -List "Contacts" -DisplayName "FirstName" -InternalName "FirstName" -Type Text -AddToDefaultView
Add-PnPField -List "Contacts" -DisplayName "FullAddress" -InternalName "FullAddress" -Type Text -AddToDefaultView
Add-PnPField -List "Contacts" -DisplayName "FullName" -InternalName "FullName" -Type Text -AddToDefaultView
Add-PnPField -List "Contacts" -DisplayName "JobTitle" -InternalName "JobTitle" -Type Text -AddToDefaultView
Add-PnPField -List "Contacts" -DisplayName "LastName" -InternalName "LastName" -Type Text -AddToDefaultView
Add-PnPField -List "Contacts" -DisplayName "Phone1" -InternalName "Phone1" -Type Text -AddToDefaultView
Add-PnPField -List "Contacts" -DisplayName "Phone2" -InternalName "Phone2" -Type Text -AddToDefaultView
Add-PnPField -List "Contacts" -DisplayName "Phone3" -InternalName "Phone3" -Type Text -AddToDefaultView
Add-PnPField -List "Contacts" -DisplayName "Website" -InternalName "Website" -Type Text -AddToDefaultView

On save my primary key for a Contact is its email address so the OnSelect of my Save Button is :

Set(
    _CurrentContact,
    LookUp(
        Contacts,
        Email = TextInput1_2.Text
    )
);
If(
    IsBlank(_CurrentContact),
    Patch(Contacts,Defaults(Contacts),
        {
            Title: "Contact saved with Scan business card",
            CompanyName: TextInput1.Text,
            Department: TextInput1_1.Text,
            Email: TextInput1_2.Text,
            FirstName: TextInput1_3.Text,
            FullAddress: TextInput1_4.Text,
            FullName: TextInput1_5.Text,
            JobTitle: TextInput1_6.Text,
            LastName: TextInput1_7.Text,
            Phone1: TextInput1_8.Text,
            Phone2: TextInput1_9.Text,
            Phone3: TextInput1_10.Text,
            Website: TextInput1_11.Text
        }
    ),
Patch(Contacts,
    _CurrentContact,
    {
        Title: "Contact updated with Scan business card",
        CompanyName: TextInput1.Text,
        Department: TextInput1_1.Text,
        Email: TextInput1_2.Text,
        FirstName: TextInput1_3.Text,
        FullAddress: TextInput1_4.Text,
        FullName: TextInput1_5.Text,
        JobTitle: TextInput1_6.Text,
        LastName: TextInput1_7.Text,
        Phone1: TextInput1_8.Text,
        Phone2: TextInput1_9.Text,
        Phone3: TextInput1_10.Text,
        Website: TextInput1_11.Text
    }
)
);
Notify("Contact " & _CurrentContact.FirstName & " " & _CurrentContact.LastName & " saved")

I know I have been lazy I did not rename the TextInput control but hey why don't you do better than me :)

Tell me what do you think about this new great feature!

SharePoint Online PnPProvisionning include document library file contents

For those of you who are familliar with PnP Provisioning you might have noticed that out of the box when you save a template and you want to apply your template to another site the document library contents is not included.

I will show you how to add some subroutines to include document library content in your template.xml file.

First we will connect to the target site and initialize some variable

Connect-PnPOnline -Url $siteURL -Credentials $credential; 
Write-Output "Connected!"

$web = Get-PnPWeb

$siteTitle = $web.Title

$saveDir = "C:\Template\" + $siteTitle + "\"

Then just trigger the Get-PnPProvisioningTemplate function 

Get-PnPProvisioningTemplate -Out $($saveDir + "Template.xml") -Force -PersistBrandingFiles -PersistPublishingFiles -IncludeNativePublishingFiles -Handlers Navigation, Lists,PageContents, Pages, Files

I am specifying only fice handlers but you can go with the defaults ones

This will create a Template.xml file but no references to files in any document library.

Now the fun begins

Using the Add-PnPFileToProvisioningTemplate command we will add entry in the Template.xml file to reference a file we want to include in our template. The idea her is to create a subroutine that will parse all the document libray in the site and read all the files and include a reference to those files in the Template.xml

This is how it can be done

First two subroutine to loop through all the folder and subfolder of a document libray

function ProcessFolder($folderUrl, $destinationFolder) {
	Write-Output "Folder URL " $folderUrl  " destinationFolder " $destinationFolder
    $folder = Get-PnPFolder -RelativeUrl $folderUrl
    $tempfiles = Get-PnPProperty -ClientObject $folder -Property Files
   
    if (!(Test-Path -path $destinationfolder )) {
        $dest = New-Item $destinationfolder -type directory 
    }

    $total = $folder.Files.Count
    For ($i = 0; $i -lt $total; $i++) {
        $file = $folder.Files[$i]
        
        Get-PnPFile -ServerRelativeUrl $file.ServerRelativeUrl -Path $destinationfolder -FileName $file.Name -AsFile -Force	

		Add-PnPFileToProvisioningTemplate -Path ($saveDir + "Template.xml") -Source ($destinationfolder + "\" + $file.Name) -Folder $folderUrl -FileLevel Published
		
    }
	
}

function ProcessSubFolders($folders, $currentPath) {
    foreach ($folder in $folders) {
        $tempurls = Get-PnPProperty -ClientObject $folder -Property ServerRelativeUrl    
        #Avoid Forms folders
        if ($folder.Name -ne "Forms") {
            $targetFolder = $currentPath +"\"+ $folder.Name;
            ProcessFolder $folder.ServerRelativeUrl.Substring($web.ServerRelativeUrl.Length) $targetFolder 
            $tempfolders = Get-PnPProperty -ClientObject $folder -Property Folders
            ProcessSubFolders $tempfolders $targetFolder
        }
    }
}

Note here the

Get-PnPFile -ServerRelativeUrl $file.ServerRelativeUrl -Path $destinationfolder -FileName $file.Name -AsFile -Force

This will download the file and save it in the destination folder locally. Then with 

Add-PnPFileToProvisioningTemplate -Path ($saveDir + "Template.xml") -Source ($destinationfolder + "\" + $file.Name) -Folder $folderUrl -FileLevel Published

it will add an entry to the Template.xml file referencing the file we just downloaded.

What is left now is to loop through all  the document libraries and call our subroutines

$docLibs = Get-PNPList | Where-Object{$_.BaseTemplate -eq 101}

    Write-Output "getting doc list"

    foreach( $doc in $docLibs ){

            if( $doc.Title -ne "Site Assets"){
                #Download root files
                ProcessFolder $doc.Title ($saveDir + $doc.Title)
                
                #Download files in folders
                $tempfolders = Get-PnPProperty -ClientObject $doc.RootFolder -Property Folders
                ProcessSubFolders $tempfolders $($saveDir + $doc.Title) + "\"
            }
    }

Here the $_.BaseTemplate -eq 101 is to get only document libraries and the $doc.Title -ne "Site Assets" is just to skip the Site Assets library.

Now you have a beautifull Template.xml file with all your document library files references. You just need to apply it to another site with

Connect-PnPOnline -url $destination -Credentials $credential;

Apply-PnPProvisioningTemplate -path ($saveDir + "Template.xml") -Handlers Navigation, Lists, Pages, Files -ClearNavigation

The complete script is available at https://github.com/alaabitar/provisioning/blob/master/script.ps1