xCP 2 Designer : connecting the dots

In november EMC introduced the new product xCP 2.0 It is a complete overhaul of their Case Management solution. With xCP 2 comes the xCP Designer. The Designer is the new all-in-one configuration tool that can be used to create xCP applications. xCP Designer is a great step forward from the days of FormsBuilder, ProcessBuilder, Taskspace, Composer and several other tools that were needed to create xCP 1 applications.

xCP Designer makes application creation much easier than before, but there is a learning curve. It is a powerful tool that can be used to create an object model, events, business processes, queries and the user interface. This can be a lot at first and since so much has changed from the previous version, the question arises of What goes Where? And more importantly: How do you combine all these elements into a usable application?

One thing I learned while doing a xCP project is that when information needs to go from one component to another, the Designer only allows you to specify inputs. You always need to go to the Destination component and specify where it gets its input from. It is an information pull model. Once you get your head around this, it becomes easier to connect the dots.

For instance, if you want to create a page to update account details for a customer, you may want to perform some validation on the inputs before allowing the user to press the Save button. Let's say we want to check that a valid bank account nr was entered. In xCP Designer this can be configured using a Stateless Process. The process would take a customer ID as input, perform some activities that check the account nr in the bank's CRM system and produce a validation message as output.
The process could be as simple as this:

A few quick tips:

• xCP Pages can only set and read the variables of a process. Packages cannot be used.
• Don't forget to set the 'Stateless Service settings' of your input variable to 'Use for input only' and for your output variable to 'Use for output only'.

Now that we have the process, how can a user interface Page make use of it?
First of all we need to add the Stateless Process as a Data Service Instance to the Page.

• Open the Data Service Instances panel, click on the green + sign and add the process to the page
• Select 'Refresh when input value changes' on the General tab of the process
• On the Input tab, specify the widget that we wish to validate the value of, in this case the Account nr field.
• Notice there is no Output tab, even though the process produces an output message

So where does the output message go?

• Add a Value Display field to the form that will show the output message
• For the value of the field you can select the output message of the Stateless Process

As an extra you can also configure that the Save button is only enabled when the validation message is empty. You can do that on the Behaviour tab of the Save button.

As you can see, when you want to connect the dots in xCP Designer you always specify on the Destination object where it gets its inputs.

I hope this post was useful to you. You can leave a comment if you have anything to add.

First impressions of xCP 2.0

Last week I attended the Momentum Developer Conference.It was the first one to be held in 6 years and it was a great succes.With all the new products that EMC is releasing there was a lot to be learned. And learn we did, in keynote sessions, presentations and hands-on labs.

One of my main interests was in xCP 2.0, a completely new product that can be used to create Documentum applications with a fresh, modern UI.The slides from Momentum in Vienna looked great, so I was keen to find out more. And so I did, in some great sessions, a full day of hands-on labs and a Hack-a-thon where we built our own xCP application.

My first impressions of xCP 2.0 are very positive. Having had experience with Documentum since Workspace 3.0 i can compare it with most of their products. The first thing I noticed is the overall level of enthousiasm. Dan Ciruli and Ahson Ahmad did a great co-presentation about what xCP can do and after that people really wanted to get their hands on and never let go. There is so much excitement in the developer community. They WANT to start building with this new tool. That is very different from the old applications like WebTop, where the feeling was more that you HAD to build with WDK, like it or not (and I won't even go into RightSite here).

What can we expect from xCP2.0?

It is platform for building applications on top of Documentum 7.

It has a development tool called xCP Designer, that integrates and replaces all of the tools you needed for xCP 1 applications, like Composer, FormsBuilder and ProcessBuilder.

The Designer is used to configure all components of an application. Designer projects can be version controlled using SVN or some other version control system. Projects can be aggragated into libraries that can be re-used in other projects.

WDK has been replaced by a modern user interface based on HTML, CSS and extJS.

It supports all of the new D7 features, such as Business Events, Relationships and Stateless Processes.

Instead of FormsBuilder there is now a Page editor that can be used to build an application's user interface. There is a Master Page and you can add your own pages. Pages are made up of widgets. The options for inter widget communication are very powerfull. Widgets can be linked so that a widget is updated when another widget's content changes for instance. Combining that with the new expressions model and stateless processes opens up some great opportunities.
Data Services are operations on objects in the system. Data Services can be used to add, change, or delete information in your xCP application. Data Services are generated automatically for the object types you model in xCP Designer, so if you add a Employee, CaseFolder, or HrDocument type to your application, you will be able to use Data Servies to work with these objects in your application's UI widgets, or processes. Data Services can also be created from Queries (DQL-, BAM- and full text queries).

Business Events are the replacement of the old TBO and give you the ability to configure many of the things that you needed to program in Java in D6. With a Business Event you can start a process, or add data to the BAM database when things happen to objects in the system, such as its properties change, it is linked, its is related to another object, etc.

Relationships aren't actually new; they have been in the object model for years, but they have been enhanced so that the relation can have it's own custom properties. xCP Designer supports releationships everywhere and really makes them shine. This really changes the way be build Documentum applications.

xCP leverages the xMS deployment mechanism. It takes a bit of setting up, but after that, deploying your application can be done by simply clicking the green 'play' button in xCP Designer. Very nice!

So, what is my impression, having worked with it for a day?
I think it's great. This is the sort of thing that Documentum needed, the sort of thing that can change a customer's mind set from 'Why do I want this?' to 'I really want this!' The sort of thing that will finally stop the old timers from thinking back warmly to the good old days of WorkSpace.
The new xCP model offers so much power and flexibilty that we spent a full day in labs with a room full of developers and none of them complained that we did not write any code at all.
The only things that can bar xCP2 from being a success now are product bugs and frustrations in run-time debugging. As for the former, we will see in the coming months but it looked pretty stable in the labs. As for the latter, the xCP product team is working on an end-to-end debugger in xCP Designer.

Enough said, I am off to the PowerLink download site. D7 and xCP2 are available there now!
 

Meanwhile at Momentum Developer Conference 2012

Last week Vienna was the place for the annual Momentum conference, the place where the European Documentum community gathers to exchange news and views and to get the latest on what EMC has been doing and is planning for the IIG products. In Vienna a whole list of product announcements where made, most notably the release of Documentum 7.0, xCP 2.0 and Captiva 7.0.
Now this week nerds techies people from around the world gather in Pleasanton for the Momentum Developer Conference, right next to the EMC IIG headquarters. This is where a lot of technical details of all the new products are going to be revealed and where developers can actually get their hands on in lab sessions.
The first 2 days of keynotes and presentations where very interesting. Now I am ready to get my hands dirty tomorrow.

I had 2 main questions for this conference:

1. What are the differentiators between D2 and xCP2. When should my clients use the former and when the latter?
2. What are the migration options to the new products?

Here is what I gathered so far:

When should you use D2 and when xCP2 ?

 D2 4.0 is the User Interface that focuses on configuration in stead of coding. Everything plus your grandma's rocking chair can be modeled using the D2 configuration interface. The downside for some clients is that D2 cannot be extended with custom code. You have to make do with what is in the D2 box.
There are simple consequences. For instance eRoom or CenterStage like collaboration functionality is not possible in D2 4.0, as is Records Management and Brava! viewer integration will be added in 4.1. Also in 4.1 or 4.2 all 'specialized interfaces' such as CenterStage, eRoom and Media Workspace will be incorporated into D2.

Then there is xCP 2.0. You may not gather this from the name, but this is a completely new product that goes way beyond anything that was offered with xCP 1.x. It offers a completely new way of building applications on Documentum and ties into the new D7 features, such as the Business Objects, the event model, stateless processes and xMS deployment. xCP Designer is a new unified developer IDE that replaces all 6 tools that were needed to build someting for xCP 1.
xCP2 also focuses on configuration in stead of coding. The difference being that xCP has build-time configuration and D2 has run-time configuration.
 xCP2 also has well defined extension points, that you can use if you need customization.

As far as the UI goes, both D2 and xCP2 produce modern looking flexible user experiences, based on HTML5 and javascript. Both offer UI widgets that can be used to compose role based user interfaces. The functionality offered in D2 is more geared towards Content centric applications and the focus of xCP2 is more toward Process or Case centric applications, but that is not a great differentiator.
Going forward, the choice will become even more difficult, as there is talk of supporting D2 configuration in xCP and vise versa and there is even talk of the 2 UI's merging completely into 1 unified UI.

So what to advice to clients that are looking for an improved UI? There are clear cases where D2 is not appropriate. The word is still out on the other cases. If D2 can fully support your use case, why not use it? But then again, why not use xCP?

What are migration options ?

Migration is my other area of interest this week. It turns out that there is a fairly clear migration strategy that most customers can follow. It requires a phased approach.

1. Upgrade your client UI to version 6.7SP2. This is needed because 6.7SP2 clients are certified against D7 server as well as older servers (6.5, 6.6 and 6.7). This will give you a working system with a 6.7SP2 client working with the server that is still on the version that you were on.
2. Now you can upgrade the server to D7. The 6.7SP2 clients will work with that server. It is adviced to do the upgrade by creating a clone environment first and then performing the upgrade on the clone. This gives you the opportunity to upgrade the server HW and OS and has the advantage that the old server can keep running while you do the upgrade.
3. Now you can upgrade the clients to D2, xCP, or WebTop 7.0. This may not need to be a big-bang conversion. You can go over 1 application at a time and may even select a different UI for each application that you have, though using more than 1 product may present a challenge on the license front. Webtop is still there but is not being developed. When you do go over to D2 or xCP2, you should redesign your whole application because the interfaces are so diferrent and you would just miss out on most of the fun if you try to rebuild your Webtop as-is in one of the new UI's.

More to come, stay tuned...

Documentum Big Data import/export

I've been away from my blog for a while, busy on projects for clients.
I learned something on one of these projects that I thought worth sharing.

In a nutshell
Importing BIG files to or exporting them from Documentum is a challenge, but you can get around the out-of-the-box limits

Here is what happened

I was asked by a client with an existing Documentum system to help them with document import/export. They were unhappy with the solution that the previous contractor had built, using Taskspace and UCF. They complained that import often failed. They also wanted to add the ability for external systems to automatically import and export documents.
I asked about the kinds of documents they are storing and they turned out to be somewhat a-typical for a Documentum system. I my experience most Documentum systems are filled with documents of kilobytes to megabytes in size, with 1Gb being considered very big. For my customer, most files were between 10 and 50 Gb, with some as big as 500 Gb. That's BIG.

Documentum has no problem storing files of that size. The challenge is in getting the files from the client to the server and back.
Since they asking for import/export functionality for interactive clients as well as back-end integration with other systems, I proposed to create a webservice using the Documentum DFS (webservices framework).

Now DFS has serveral options for content transfer:

• BASE64: This will include the content as part of the reply message to the webservice client. This is the easiest, but also the most restrictive. Only advisable for very small content files.
• UCF: This is Documentum's proprietary content transfer method. It has many cool features for xml files and virtual documents and such, but it had proven unreliable in my customer's environment with the BIG files they have
• MTOM: The Message Transmission Optimization Mechanism is a W3C standard especially meant to reliably send binary data in SOAP webservice calls.

MTOM looked promising but I had run into boundaries using MTOM for big files in a previous project. When exporting several big files simultaneously, the App Server running the web services would run into Java memory issues. That previous project had considered 10Mb big, so we were sure to run into the same boundaries here.

I solved this by cutting the content transfer up in pieces.
Exporting a file now goes like this:

• The web service client starts an export by specifying which file it wants to receive. The web service returns an export token (a unique ID for this export request).
• The webservice client the calls the web service again, supplying the token and the maximum number of bytes he wishes to receive (the default being 1Mb). The web service returns part of the content file using MTOM.
• The web service client keeps calling until the full content file is transferred.

This very simple protocol turned out to work like a charm, even when simultaneously transferring files of many Gb . We did advice the client to use a separate DFS server machine, so the Documentum content server is not congested with all the disk- and network traffic the big files are causing and TaskSpace can keep running smoothly for the users.

TaskSpace
For the interactive clients we did one more trick so they can use the new export/import webservice.
normally you would have a component on the TaskSpace application server that acts as a web service client, but that would mean that the content would be sent to the application server and the application server would then send it to the user's browser. That would mean that the big files are sent over the network twice, causing unnecessary delays.
Documentum has a feature called Accelarated Content Services (ACS), but we could not use that in this project.

We did find a way to get the content from the DFS server directly to the user's browser:
We added a little javascript to the export page that calls the export web service and combines all the parts of the file into 1 BIG file.

It works, it performs, 1 solution for both interactive and integration use, I am happy !

Let me know what you think

How the internet is changing economics

This post is a response to a article of the same name by Kas Thomas.

Kas noticed that in 'the old days' most markets are dominated by 2, or 3 brands. For instance, Coca Cola and Pepsi dominate the soft drinks market. with the advent of the internet, some new markets have emerged and many existing markets are changing drastically. Kas noticed that the internet based markets seem to be dominated by just 1 big player. Google dominates the Search market, Amazon dominates the on-line bookstore market, FaceBook dominates the social network market, etc.
Kas predicts that major shake-ups are due, as markets move to the internet. Where there was room for several big players before, there will be just 1 left after the transition to an on-line market is complete.

To me, this sounds like a horror scenario. Like Kas, I am no economy major, but having just 1 dominant market player means a practical monopoly. Monopolies mean high prices, or less than top-quality service.

Though this scenario will probably come true in some markets, I think there is hope. Though the internet is apperently a good place to create a monopoly, the opposite is also true. The internet can be a great unifier that provides chances to smaller componies. In the automobile industry for instance, more and more cars are sold through the internet. There are some great search engines that will let you search for your prefered make, model, etc. The result list will show cars offered by big companies, small firms and private cars. This way of working prevents a monopoly and promotes competition, leading to lower prices and higher quality service.
The same goes for electronic equipment for instance. There are several sites that will help you find the right equipment, show specs and reviews and a list of stores that sell the product, ordered cheapest-first.

I also think that the monopoly scenario may be more relevant in the US than in other countries. Amazon for instance is a popular site in Holland, but the local company bol.com is doing just as well. The local company has an advantage, as it knows the local market, culture and language.

Finally, I don´t think there will ever be just 1 operating system for our devices. Microsoft managed to create a monopoly 30 years ago when the digital age was just starting, but no one will be able to repeat that now. With Apple, Google and Microsoft there is just too much money and power to let that happen. There is a fashion in OSs. One year iOS will be in vogue and the next year Android will be hip, or the newest Windows version will be the new must-have. I don´t see a monopoly in SmartPhone or Tablet OS any time soon.

DCTM Tip: job polling

Something has been annoying me and I finally took some time to look it up. It thought I'd spare you all the hassle and share what I found here.

Problem at hand: whenever you start a DCTM job in a repository at a client, it will take 5 to 10 minutes before the job is actually started. That is annoying, especially when you are testing a custom job. I looked around in the usual places, but found nothing. Then I found the following in the DCTM Server Admin guide:

Setting the polling interval

The agent exec process runs continuously, polling the repository at specified intervals

for jobs to execute.

To change the polling interval add the

-override_sleep_duration argument with the desired value to the agent_exec_method

command line. Use Documentum Administrator to add the argument to the command

line. For example:

.\dm_agent_exec -override_sleep_duration 120

The polling interval value is expressed in seconds (120 is 2 minutes expressed as

seconds). The minimum value is 1 second.

Bonus: you can also set the number of jobs that will be run in 1 polling interval:

Setting the number of jobs in a polling cycle

By default, the agent exec executes up to three jobs in a polling cycle. To change the

maximum number of jobs that can run in a polling cycle, add the -max_concurrent_jobs

argument with the desired value to the agent_exec_method method command line.

For example:

.\dm_agent_exec -max_concurrent_jobs 5

Use Documentum Administrator to modify the command line.

Can CMIS cope in the world of real ECM applications ?

I was inspired by an FME Group post comparing CMIS to DFC. In neatly describes the pros and cons of using Documentum Foundation Classes vs. the Content Management Interoperability Services. For documentum customer this can be used to determine the right interface for your application: Documentum specific, or international standard.

I think the comparison can show a few things about the CMIS standard and where it stands in its current 1.0 version. CMIS is aiming to provide a product independant interface to ECM systems, to allow software companies to write ECM applications that work, no matter which ECM system is used to store the documents in. Check the CMIS page on Wikipedia for more information.

So, how does the current version of CMIS live up to that idea?
First of all, CMIS offers a full set of essential content management operations, to create ,import, or find documents and their metadata. It has folder structures and access control. It supports user authentication. All very good stuff. It also has its limits.

CMIS allows only single object operations. A logical choice imo, though it does mean that batch updates and deletes are not supported. To me this is no big deal, since batch-wise updates and stuff like that is mostly in the domain of the Super User, or Application Maintenance people and they will most probably need a proprietary tool for their work any way.
CMIS has a query language that can be used to find documents or folders. Full-text search can be used, when supported by the underlying ECM system. INSERT, UPDATE, or DELETE queries are not supported however. JOIN queries are only partly supported by some implementations (currently only by Documentum and Nuxeo)

CMIS has no support for Transactions. So if you need to combine several updates into 1 logical operation, your application will need to provide its own transaction logic.

CMIS supports permissions on objects, yet only, read, write, or 'all' rights can be specified for a user or group. There is no function to request the permissions for a given user on a given object. Only the permissions for the current user can be requested.


More advanced ECM functionality, such as Renditions, are not fully supported yet. Only a few vendors offer read-only access to renditions through CMIS, none offer functionality to add a rendition to a document. Even more advances functions, such as workflows, are not supported at all.

As you can see, CMIS supports most basic ECM functionality. It is suitable for writing applications that only need basic ECM functions, such as DropBox-like browser apps, or those sexy mobile applications that are mostly used for consuming information. Applications like that using CMIS will be usable with most major ECM products. This is great news!
More advanced applications will run into its limitations, with no support for transactions, renditions and such. Let's hope CMIS 1.1 brings us some of those features.