The SharePoint Trip to Modern – Are We There Yet? Part 2

On the trip to SharePoint “Modern”…  I’m having a serious case of deja vu. Some macro-patterns I saw eons ago as a Microsoft Windows Developer I see again in SharePoint “Modern”… especially with SharePoint “Modern” forms.

SharePoint forms – “classic” and “Modern” – fall into three broad categories: list and library forms, survey forms, and “forms over data” applications (aka InfoPath). Classic forms in each category are functional… but hard to customize, and definitely not beautiful. So where are we, exactly, on the trip to modern forms, and what are these “macro-patterns” I see?

Let’s break it down.

Microsoft Forms is the “Modern” replacement for SharePoint classic surveys, PowerApps is the “Modern” replacement for InfoPath, and Modern list forms …we’ll discuss those momentarily.

I never cared much for SharePoint “classic”  surveys.  They seem too toy-like for anything but the simplest of surveys, and look ugly (to me). In fact, beautifying them requires custom development… typically with React or Angular, and supporting documentation for users to use it, site owners to configure it, and developers to support it. All for very little functionality. Which is why I was happy to see Microsoft Forms…  the “Modern” answer to SharePoint “classic” surveys.

Actually, I thought Microsoft Forms (which sort of sneaked up on me last year) was also too toy-like at first. But with constant improvements over the last year…  I think Microsoft has a winner here. The surveys are easy to design, look nice, are easily embedded in a SharePoint page, and easy to configure. Microsoft Forms plays well with others too – particularly with Microsoft Flow. There is even a curious integration with Excel (someone please tell me if this is useful). If Microsoft Forms stays on its current trajectory, it’ll be the SurveyMonkey® for corporate intranets.

PowerApps is out of the toy-stage, and positioned to replace InfoPath.  I expect PowerApps will do for SharePoint what Visual Basic did for Microsoft Windows development… and become the most popular tool for SharePoint “Modern” power users and developers.  A word to the wise for “I’m too cool for PowerApps… I do React/Angular/Vue” developers out there….  Visual Basic supplanted C++/MFC for corporate Windows development – because it was easier to learn, and quicker for creating “Forms over Data” applications. Will history repeat here, and relegate SPFx development to a smaller – but important – niche for high-end customizations? Maybe.  Regardless, I predict the PowerApps tsunami is coming. Are you ready?

Lastly, consider SharePoint List forms. Specifically, the classic EditForm.aspx, NewForm.aspx and listview.aspx forms, and their Modern alternatives. Many sophisticated SharePoint workflows depend on highly customized versions of these forms to view, create and update SharePoint list items.

SharePoint List/Library forms can be quite complex mini-applications in service to SharePoint workflows. Form data must be loaded, and fields labelled, disabled/enabled, validated, hidden or revealed – depending on the workflow state and who is looking. Error handling must be friendly and helpful.  Extra points for beauty.

Clients often don’t realize the effort required to achieve all this…  so it’s important to set expectations, and decide if functional – albeit ugly – forms are sufficient.  Or is beauty important too?

The “Microsoft approved” way to customize SharePoint “classic” forms is via JS Link and Client Side Rendering (CSR)  – you add customizations with client-side JavaScript/TypeScript, using Microsoft-provided hooks at key points in the form life-cycle. The coding patterns take some getting used to. And to make the forms visually appealing, you’re on your own. Many SharePoint developers have crashed and burned attempting to wrangle the ridiculously daunting CSS in SharePoint “classic” forms. Which brings me to SharePoint “Modern” form options.

We’re still screwed…  but less so. Beauty is within reach – with Microsoft Office UI Fabric React-based components and SPFx. But a few things are missing for developers.

When I built my first SharePoint “Modern” modal dialog box as an SPFx web part with these components, I was reminded of the early days of Microsoft Windows development, when Microsoft provided a Windows Software Development Kit (SDK) without a GUI-based dialog editor. Windows developers resorted to “imagining” their dialog layouts while coding them with a text editor. It was highly inefficient….  much like the situation today with SharePoint “Modern” modal dialogs. There is no GUI-based form Editor… the React-based field controls are “laid out” using JSX in a code editor. I hope Microsoft will get around to providing a GUI-based editor for list-item forms with Office UI Fabric React components (yes… I know Modern lists can be customized with PowerApps – but it’s not a viable option – not yet – for a SharePoint Modern modal dialog box containing Office UI Fabric components).

What else is missing? For starters, the Office UI Fabric React components lack “SharePoint awareness”. So you’ll need an abstraction layer over them. You could add your own abstraction layer… but smarter to use the SharePoint Patterns and Practices SPFx React Controls. These extend the Office UI Fabric React controls… adding SharePoint awareness.  But even here… we see signs of immaturity. Take for instance the oh-so-innocent looking People Picker control.

A SharePoint People Picker control is one of the nastiest, most difficult form controls to write yourself. And this is from personal experience. It’s the “iceberg” of SharePoint field controls, because so much is hidden beneath the surface. So I was very happy to see it offered in the Office UI Fabric React component library… that is, until I discovered it was neutered.  That is, the darn thing isn’t connected to SharePoint (or rather, the Azure AD instance attached to a SharePoint tenant).

But the PnP People Picker control fixes this.  You add it within your Office UI Fabric Dialog component…  and it just works…. almost. At the time of this writing, it lacks a property to initialize the people picker with users and/or SharePoint Groups (I’ve requested an enhancement to rectify this… so stay tuned). This means that if you have an existing list item containing a people field, and that people field is not empty… you cannot initialize the people picker control with that person.  While we’re waiting for a fix, you can do what I did and create your own people picker abstraction layer, starting from this React people picker control.

My beloved Knockout library is useless with these Office UI Fabric controls (and the PnP offshoots)… because you cannot add the Knockout bindings to the HTML attributes within the controls.. you don’t have access to the innards. Knowing how much Knockout reduces “classic” form complexity,  I worried that Modern form complexity would be unwieldy. But my fears were unfounded, since the controls have sufficient hooks to capture mouse and keyboard events, and properties to control the enabling, disabling, and show/hide behavior. At this point, it’s probably better to show you, rather than tell you about it.

But let’s pull over at the next rest stop….   I need a short break before showing you some code. And no… stop asking… we’re still not there yet… on our SharePoint trip to Modern. But almost.

Here’s our stop… want some Ice cream?

-bob

 

 

 

 

 

 

 

 

The SharePoint Trip to Modern – Are We There Yet? Part 1

Is time travel possible? It sure feels like it with the SharePoint Modern experience. Leave it to the Microsoft SharePoint team to turn me into that little boy on a car trip, asking “Are we there yet?”.

I’m referring to the current state of SharePoint, and the trip to “Modern”. I’m loving the sights along the way…  but when is this journey going to end?

As a SharePoint architect, I weigh many aspects of the platform (and the client’s needs and capabilities) before formulating a solution. This has been – ahem – especially challenging as Microsoft sloooowly releases its “Modern” vision piecemeal.

Hub sites, Communication sites, modern team sites, modern lists, O365 Groups, Flow, PowerApps, SPFx web parts and extensions, Microsoft Teams….  and the tools to regulate, control and govern everything….  all in different states of readiness and maturity.

This drip, drip, drip of wonderfulness puts SharePoint architects and developers in a weird spot… titillating us with ooh-la-la “modern” features…  but making us wait… drip.. for feature gaps to close… drip… before using them.  Like you, I am reluctant to deploy yet another “classic” team site, create yet another custom MasterPage, design yet another SharePoint 2013 Workflow, as the “new” taunts me….  dares me… to “go modern”. But are we there yet?  No. Not yet. But almost.

Of course, it’s not all or nothing…  some modern goodness is ready now…  Microsoft Teams has “blown up” (popularity-wise, in a good way)…  and the “low code, no code” ways to extend it are terrific.  At Slalom (where I work as a Solution Principal in the Content & Collaboration practice in NYC), we’ve published a Slack vs Teams whitepaper that is extremely popular among our clients evaluating both.  And while Teams has caught fire in the last year, the recently released governance & compliance controls are only now allowing highly regulated industries to realistically consider adopting Teams.

Microsoft Flow is terrific…. I’ve used it for over a year now and use it whenever I can….  but there are many things it still cannot do (such as elevated permissions not tied down to a specific user, or long-running state-based workflows, to name two features Flow currently lacks), and I find myself “enhancing Flow” with the occasional SharePoint 2013 Designer workflow.

Modern SharePoint publishing disappoints me – in its current state – as I wait for the modern equivalent to classic page layouts, Master Pages, item scheduling, and a better way to control article/news metadata than the way explained here. And while the recently announced “News Approval” workflow is very important to some organizations, in practice, I found many organizations preferred to keep it simple and forgo the classic “approval workflow”, instead using just check-in, check-out and Major/Minor drafts with versioning.

I’d like to see a better way to schedule articles. In classic, the way this worked is that a background timer job changes an article from draft to published, or visa-versa – depending on the schedule. But to create an article archive for “no-longer published articles” that visitors could search for and re-read, the classic scheduling approach is bad, because when an article is in “draft mode”, it cannot be read by visitors. I’d prefer a “modern” approach that avoids this technique, so visitors can search and view an archive of old articles… OOTB. But I digress.

Microsoft Communication sites bring the “wow”. Just the other day, we created one for a client unaware of them. The client was resigned to getting a “classic” experience.  So we surprised them (not always a good idea, but OK in this case). In a short demo, I heard a half-dozen “awesomes” and several “wows”… all for only a few days of configuration (and some snazzy images). I want all my engagements to delight clients like that.

I think Communication sites make ideal Hub sites. The “hubiness” idea is not new with Modern SharePoint. Classic SharePoint has offered the “Portal Site Connection” feature going back at least to SharePoint 2007 MOSS.  But Modern Hubs are so much better.  We’ll get to that later.

Two more “classic features” awaiting modernization are Document Sets and Search. I don’t use Document Sets that often, but when I need them, they are terrific. I find them to be most useful in “lawyer-like” scenarios, where a “matter” is best handled with several documents that are related.  Document Sets allow all documents in a set to be auto-tagged with the same meta-data. And provide a logical grouping that is natural to “lawyer-like” scenarios.  Microsoft has promised to modernize the Document Set experience – BUT I’M STILL WAITING.

Search is the “SharePoint killer app” (MS Teams not withstanding)… and has been since SharePoint 2013. But in my experience over the years, many clients have not realized the value.  One reason is lack of what it can do, when properly configured. It requires expertise in several areas –  such as defining a metadata taxonomy and associated site columns and content types, configuring the search web parts and customizing Display Templates. These are all do-able, repeatable and don’t require a PhD.  At present, there are no modern search web part equivalents to the “classic” ones. I anxiously await their arrival. Until then, I’ll continue to build the “classic” search experience with a “classic” publishing site collection and “classic” search web parts. (By the way, this is not in lieu of using modern “delve-like” search experiences based on Microsoft Graph. The graph-based modern search experience fills a different niche – highly personalized and useful for discovering new – often unanticipated – search results.  The “classic” search experience will remain extremely valuable for “Knowledge Management” search experiences, where content is authoritative and curated, and users have a general idea of what they’re looking for, and seek authoritative results.

In upcoming articles, I’ll take a deeper look at many modern capabilities, highlight the gaps, and suggest work-arounds where possible.

So sit back and enjoy the ride.  And together we’ll answer  “Are we there yet?”

-bob

Modern SharePoint Information Architecture

pnp-hub sites

I found this slide (from Vesa Juvonen’s March 2018 PnP video on SharePoint Hub Sites video presentation)  to be a good illustration of the new, modern direction for SharePoint site IA. It stresses a “flat-earth” approach for modern site collections… dispensing of site collection sub-sites in the name of O365 performance, and the flexibility and ease of making and breaking site associations between hubs and root-site site collections.

Notice the black-pentagons in the diagram…  for those scenarios where “classic” site collections need sub-sites.  I do think there remain scenarios for these until modern sites fill the gaps….  scenarios where search sites, record centers and departmental IA concerns (e.g. site collection specific site columns, Content Types, search, document sets, etc) are required.

Interestingly….  2 hours after I published this, Microsoft released a Hub Planning Guide.  After a quick read, I think many (but not all) of the ideas in the guide are reasonable. I’ll have more to say about this soon.

Want Simplicity? Use the SharePoint PnP JavaScript Core Library

If you’re a SharePoint developer and not using the SharePoint PnP JavaScript Core Library… you’re working too hard.

In an earlier blog post, I looked at Typescript’s asynch/await feature as a way to simplify asynchronous SharePoint code. In retrospect, the SharePoint PnP JavaScript Core Library is a better, simpler, more elegant solution.

We’ve all been here…  deciding to drop what works… in favor of risking something better. For SharePoint client-side asynchronous code… this was an easy decision.  Here’s a short snippet to illustrate the point:

static GetRoles ()
{
let deferred = $.Deferred();
// Get the groups the current user belongs to.
//
$.ajax({
url: _spPageContextInfo.webAbsoluteUrl + 
"/_api/web/currentuser/Groups?$select=Title",
method: "GET",
headers: {"accept": "application/json;odata=verbose"},
success: function (resultsData)
   {
   deferred.resolve(resultsData.d.results);
   },
error: function (jqXHR, textStatus, errorThrown)
   {
   window.console.log('error: loggedInUser.GetRoles returned an error');
   deferred.reject();
   }
});
return deferred.promise();
}

:
:

let promise1 = GetRoles();
:
$.when (promise1).done(function(data1)
   {
   processRoles (data1);

The GetRoles function make an asynch call into SharePoint, returning a promise, and the results are processed later. The GetRoles function is loaded with (typical) ugly asynch artifacts.

Now compare that version to this PnP version:
$pnp.sp.web.siteUsers.getById(_spPageContextInfo.userId).groups.select("Title").get()
.then ( groups=>
{
for (let group of groups)
   {
   // process groups...

Wow…  the GetRoles function is reduced to a one-liner…  with low-level asynch plumbing out of sight. I found that stringing multiple asynch calls together in parallel or serial fashion was easy, too.

I took the PnP code snippet above from a “traditional CSR” SharePoint list form…  where the PnP library was “included” via this JSLink expression:

https://cdnjs.cloudflare.com/ajax/libs/sp-pnp-js/3.0.2/pnp.min.js

I used the dollar sign, like this: “$pnp” to reference the PnP library in my (non-React) TypeScript code.

But in my React components, I import the PnP library like this:

import pnp from "sp-pnp-js";
 and reference the PnP library without the $,  like this:

pnp.sp.web.siteUsers.getById(loggedInUserId).get().then ( result=>
It’s all explained in the PnP Core Library github site.
I hope you find the SharePoint PnP JavaScript Core Library as enjoyable as I do.

Microsoft Ignite 2017 – Day 1 Highlights

Keynote

Another year, another huge crowd at the annual Microsoft Ignite conference – hosted in another warm place (Orlando). Satya Nadella (Microsoft CEO… for the 3 people on the planet who didn’t know that) gave the Keynote – and I enjoyed it (unlike the sales-oriented keynote by Scott Guthrie last year).

Satya’s keynote featured a nifty Mixed Reality demo by Ford Motor engineers wearing HoloLens headsets. It was a clever demo, with engineers in separate physical locations – collaborating on car grille and side-view mirror designs using a 3-D car image overlaid with virtual text, menus, and car parts. (Incidentally, this was my first time hearing “Mixed Reality”…  shouldn’t it be called “Augmented Reality”?)

The virtual scene rendering appeared (to me) to be sluggish – what kind of processing power and network bandwidth is required to pull this off? And speaking of processing power… we got a glimpse of quantum computing.

Satya brought on stage 4 PhDs – a famous mathematician (well.. his credentials sounded impressive, anyway), two physicists and a computer scientist – and they took their seats on bar stools, facing Satya, in what looked like a TED-Talk. My friends, this was a whole new level of geekiness….  and I – along with 20,000 fellow geeks – sat in quiet, curious anticipation.

The mathematician was the first up, describing the mathematics of quantum computing. I took a lot of advanced math and physics in college, but I couldn’t understand 95% of what the mathematician said – as if “the topology of spatial chronological whooshwixers” was common lingo. Next up were the physicists – injecting “this is hard stuff” into every second sentence, followed by the computer scientist explaining she had to invent an entirely new computer language with a data type called a qbit. And oh yes, the entire shebang needed to run at 0.1 degrees Kelvin… well beyond the comfort zone of the typical home computer (and my refrigerator). Satya looked like a pleased father encouraging his children…  and heck, can you blame him? Quantum computing could change the world… and these enthusiastic, smiling PhDs are on the bleeding edge… and they know it.

And then…  some quantum hardware was revealed. One of the physicists handed Satya a Quantum chip, which Satya accepted with the reverence of being handed the world’s first transistor.  Was it for real?  How could I know?

Next, the curtain was raised to reveal a Quantum computer that appeared to come from the 1960 H.G Wells ‘The Time Machine” movie.  No mention was made about whether this thing worked… but someone gets extra credit for it’s very cool retro sci-fi movie design.

The other sessions today were rather pedestrian….  I attended the talk on “What’s new with Search”.. or something like that.  Not much, apparently, but the presenters were pumped. Ok, to be fair, there were a few nice innovations, like “automatic OCR” for images dropped into SharePoint. OCR is performed automatically, creating an image with searchable text. This is akin to scanning a physical image and having the scanner perform OCR and output a dual-layer searchable PDF file – one layer for the scanned image, another for the text. SharePoint does this “scanning” for you now. Nice.  And further down the road, the presenters promised SharePoint will automatically extract meta-data contained in documents and images and auto-categorize them. That seems ambitious to me…  given the difficulty of categorizing documents even when you know what’s in them.  So I’m keeping an eye on that one.

Another claimed innovation is how SharePoint search is able to extract location information and show it on a Bing map. Am I mistaken, or is this old hat for Google search?

I finished the day attending a “What’s new with AD FS”. Security is a particularly interesting topic for me…  and underlies the entire Microsoft universe. AD FS is enormously important for secure Enterprise computing. But….  I just… couldn’t … stay focused watching variations of login sessions. But I am looking forward to exciting sessions later this week.

Stay tuned.

 

Was waiting for async/await worth it?

I read once that async and await were coming to ES2015 (ES6) and thought “hmm…   that’s nice – call me when it works in ES5.” Time passed. Then Microsoft announced TypeScript 1.7 support for async/await –  but only for ES2015 or above. More time passed. Then recently, Hell froze over and ES5 support arrived with TypeScript 2.1.

Prior to async/await, I used jQuery “Deferreds” to manage asynchronous function calls. My needs were simple – typically requiring one asynchronous function call. Sometimes, two – called sequentially or in parallel. In all cases, jQuery “Deferreds” met my needs.

But I could never remember the intricacies of these jQuery “Deferreds”. Anytime I needed to work with asynchronous functions, I had to review – once more – how Promises worked, how to chain together two function calls, how to handle errors.

I wondered, would async/await make asynchronous programming easier? The logic clearer? The error-handling better? I investigated. And here is what I found.

Yes, async/await does make asynchronous function call error-handling easier to understand, and easier to implement. And I found that async/await makes it easier to know which code fragments will “run now” versus those that will “run later”.

But async/await doesn’t remove your need to know what promises are, and how to use them. And I did not experience a significant reduction in lines of code after refactoring some legacy code to use async/await. And you can abandon all hope of trying to understand the ES5 code generated from the async/await statements, unless you’re a masochist (or work on the TypeScript team). The Typescript transpiler literally vomits bizarre ES5 code… bearing no semblance to the original TypeScript source. So you absolutely need to use Source Maps for debugging, and trust that Microsoft got the transpilation right. But I trust they did and I like debugging with Source Maps (as long as the output isn’t minified) so that I can follow the TypeScript code I wrote and not the JavaScript code emitted.

(Incidentally, you will benefit greatly if you understand how JavaScript uses a single threaded event loop in conjunction with a message queue to process “run now” and “run later” code fragments. There are many good articles explaining the details… just search on ‘JavaScript single threaded event loop’).

If you recall from my last blog post, I inquired if async/await reduced the number of lines of code in my jQuery “Deferred” code (and as mentioned above, the answer is “yes”). But there are additional benefits – better syntax, better error-handling – which make async/await a must going forward.

Let me show you two simple examples I used to explore the goodness of async/await. In the first example, I used a classic “Promises-only” approach (well… classic for me, anyway). The second example uses async/await.

Example 1 – Calling two asynchronous functions using Promises

Here is the first example (the VS Code project files are here). It consists of two asynchronous functions, asynch1 and asynch2, and a function that calls them sequentially and another function that calls them in parallel.

The getAsyncDataSequentially function calls asynch1 and waits for it to return before calling asynch2, and then waits for asynch2 to return before proceeding. This is the classic “sequential” pattern.

The getAsyncDataInParallel function calls asynch1 and then – without waiting – calls asynch2 allowing both asynchronous functions to execute in parallel, and waits for both to return before proceeding. This is the classic “parallel” pattern.

Note that asynch1 and asynch2 are identical, except for their name, and the duration of their setTimeout delays. They simulate asynchronous ajax functions that take different amounts of time to complete.

fig01

I’m not going to delve into the mechanics of how all this works, since I expect you’ve already used these patterns yourself. But you may not be familiar with the import statement at the top

import "es6-promise"
This is needed to bring in the ES5 “Promise” polyfill (and described nicely here).

With a “Promises only” approach, you use .then to resolve a promise, and .catch to handle errors. These add considerable complexity to the getAsyncDataSequentially function, due to the need to nest them. The getAsyncDataInParallel function is less complex because we don’t need to nest .then and .catch.

Example 2 – Calling two asynchronous functions using async/await

Example 2 (the VS Code project files are here) refactors Example 1 to use async/await. I was expecting syntactic miracles with async/await… hoping they’d remove the need for Promises. But it was magical thinking on my part. Promises are still needed in the asynchronous functions. So asynch1 and asynch2 don’t get refactored.  The refactoring is with getAsyncDataSequentially and getAsyncDataInParallel – the functions that call asynch1 and asynch2. This was a surprise.

Here is a side-by-side comparison of getAsyncDataSequentially – before and after using async/await… so you can see for yourself:

fig02.png

To my eyes…  async/await (on the right) cleans up the code (on the left) beautifully. In the refactored code, error-handling is exclusively try/catch. And 10 lines of code were eliminated. These benefits – cleaner code, better error-handling – convinced me to use async/await going forward.

Here’s getAsyncDataInParallel – before and after:

fig03

The refactoring didn’t reduce the code… but I still like the improved try/catch error-handling. I also like the cleaner semantics.  In the example on the left, the statement on line 84:
console.log("3") ;

executes before the .then block. But not the async/await code on the right. It will pause at the await statement, and proceed only after the asynchronous calls complete (with or without an error).  I found this much more intuitive.

Feel free to try out the examples yourself (I’ve configured a launch.json file so you can execute & debug them from within VS Code).

In my last blog post, I promised to refactor my Client Side Rendering example to see if async/await improved the code.  It did…  consistent with the examples above. The VS Code files are here.

Here is a comparison of using jQuery Deferreds vs ES2015 Promises in the getRoles asynchronous function:

fig04

I replaced the jQuery “Deferred” statements with “real Promises”. Other than that, the code remained the same.

The bigger change was in the postProcessEditForm function, which calls two asynchronous functions using a parallel pattern:

fig05

Both of the postProcessEditForm functions above call two asynchronous functions – GetRoles and GetCTMemberValues – “in parallel”,  pausing until both return. The jQuery “Deferred” version uses $.when to pause, the async/await version uses await.

I didn’t use error-handling in the jQuery Deferred version (no reason not to, I merely left it out of the example), but I used try/catch with the async/await version, and I find it easy to understand.

In Conclusion…

I heartily recommend using async/await and real “ES2015 Promises” in place of jQuery “Deferreds”. They make your code cleaner, and the error-handling better. And you will get the benefit of less code if you need to chain several asynchronous functions sequentially.

You’ll continue to need Promises… async/await requires them.

In my next blog post, I’ll see if the latest PnP JS Core JavaScript library  can simplify my code more than async/await did. From what I heard, it has functions for calling the SharePoint REST API without the need for Promises or async/await.

Stay tuned.