Waiting for engine...
Skip to main content

Including additional information and metadata in documents

Developers building a connector using the Connector SDK can use the connector descriptor file (connector-descriptor.xml) and include additional information or metadata (for example, Message Id) in documents (referred to in as document properties).

Document properties​

By using document properties to add extra information that is not part of the document payload, you provide greater flexibility and tracking for users as a document progresses through the process steps. In there are two types of document properties:

  • Standard document properties — Pre-defined properties declared by connectors and selectable by users.
  • Dynamic document properties — Free-form text names defined by users. In the Connector SDK, these are referred to as "user-defined" properties.

When building a connector, a best practice is to declare your own standard document properties rather than using dynamic document properties. The advantages of this practice are making the available options for selection by the user clearer, and allowing for document tracking in Process Reporting.

Document property subtypes (dynamicProperty and trackedProperty)​

In the Connector SDK, standard document properties are identified by two subtypes:

  • dynamicProperty — An inbound value set by the user and passed into a Connector shape to augment or override the default connector component (typically the Operation component).
  • trackedProperty — An outbound value set by the connector and returned to the process to provide the user with additional information or metadata about the document. These values are captured and displayed as part of document tracking in Process Reporting.

Implementing a dynamicProperty​

To get additional properties from the user, define an SDK dynamic property in the connector descriptor like the following sample:

<dynamicProperty type="string" id="MessageId" label="Message Id"/>

The dynamicProperty element supports these attributes:

  • @id — defines the name of the field that is passed as the property key to the connector.
  • @label — an optional text string displayed in for the field. If this attribute is not defined, the id is used.
  • @type — the type of property value passed to the connector. This attribute restricts the values that can be entered in to “string”, “password”, “boolean”, and “integer”.

Code sample​

In the code, the property value is retrieved using the getDynamicProperties() method of a TrackedData instance like the following sample:


public static String getMessageId( TrackedData data )
{
    Map<String, String> dynamicProperties = data.getDynamicProperties();
    String propertyValue = dynamicProperties.get("MessageId");
    return propertyValue;

User impact​

The new property appears in the list of standard connector document properties the user can select in the Set Properties shape.

The user can also set the new property from the Set Document Property map function and from a Groovy script within a custom scripting step.

Implementing a trackedProperty​

In addition to the content of the document returned by a connector, you may also want to provide metadata values that users can track or use in a process. To implement this fuctionality, define an SDK tracked property in the connector descriptor like the following sample:

<trackedProperty id="MessageId" label="Message Id"/>

The trackedProperty element supports these attributes:

  • @id — defines the name of the field that is passed as the property key to the connector.
  • @label — an optional text string displayed in for the field. If this attribute is not defined, the id is used.

Code sample​

In the code, the tracked property value is set as metadata to the payload returned for each document like the following sample:


public static void sendResponse( TrackedData data, 
                                 OperationResponse response,
                                 String output, String messageId )
{
    PayloadMetadata metadata = response.createMetadata();
    metadata.setTrackedProperty("MessageId", messageId);

    response.addResult(data, OperationStatus.SUCCESS, "0", "OK",
                        PayloadUtil.toPayload(output, metadata));
}

User impact​

The new property appears in the list of standard connector document properties the user can select in the parameter value list for most process steps.

The user can also get the new property from the Get Document Property map function or from a Groovy script within a custom scripting step. The values also show up in Process Reporting like other tracked properties.

Using a dynamic property and tracked property for the same document value​

To have a document value be set dynamically and also tracked in Process Reporting, define both a dynamicProperty and trackedProperty in the connector descriptor. This effectively allows the user to see a single document property that can be set before an operation and read after the connector executes.

Implementing a userDefinedProperty​

When building a connector, a best practice is to declare your own standard document properties rather than using dynamic document properties. The advantages of this practice are making the available options for selection by the user clearer, and allowing for document tracking in Process Reporting.

To get the dynamic document properties a user sets on a document, you do not need to define anything in the connector. Instead, get the property values directly using the getUserDefinedProperties() method of a TrackedData instance like the following code sample:


public static String getUserDocumentProperty( TrackedData data, 
                                              String propertyName )
{
    Map<String, String> userProperties = data.getUserDefinedProperties();
    String propertyValue = userProperties.get(propertyName);
    return propertyValue;
}

Document property use cases​

Connectors can typically handle all the operation inputs and outputs using the document profile. However, there are certain situations where additional properties are needed.

Raw binary documents​

When a document is binary data, properties need to be attached to it. By using document properties, you can have the user set and get properties attached to the binary data document that the connector can use. Here are two examples:

  • Disk connector — The connector reads and saves files from the file system, and the document data is the binary content of the file. The connector has a File Name document property that the user can set to define the name the file will have when the connector saves it, or the name of the file the connector read from the file system.
  • Microsoft Azure Service Bus connector — The connector sends and receives binary messages from and to a queue. The connector has a Message Id document property (both a dynamicProperty and a trackedProperty) so the user can set or get the unique identifier for a message. It also defines a Content Type, Label read/write properties, and Sequence Number and Lock Token read-only properties (SDK tracked property).

Override the operation component's field​

You may have configuration values that are configured in the connection or operation, but may find it useful to allow the user to override the configuration on a document level. In this case, you typically do not want to have the value added to the document profile because it is something that is not directly related to the document's content but instead to the document's processing. You can use document properties to attach the configuration to a document when you need to override one of the values and make the connector use them accordingly when setting up the values internally.

Display values in Process Reporting​

If you want certain values displayed in Process Reporting as part of document tracking, you can use trackedProperty document properties.

Reading custom properties​

When the connector needs to support custom properties that cannot define when the profile is imported or created, using userDefinedProperty values is appropriate. One example is the Google Pub/Sub connector that sends and receives binary messages from and to a queue, allowing the user to set any metadata to be sent along with the document set as dynamic document properties (user-defined properties).

On this Page