Waiting for engine...
Skip to main content

Implementing the Query operation

The Query operation provides many benefits, making it easy to build queries with field selection, filter/search criteria, and sorting. 

Benefits​

If the service provides a query grammar for selecting fields, creating boolean search criteria, and specifying sorting, consider implementing this operation because it will enhance the usability and power of your connector. The operation configuration is flexible enough to specify the following:

  • Single or multiple sort parameters.
  • Descending and ascending sort orders.
  • Simple single level or complex nested boolean expressions.
  • AND or both AND and OR boolean expressions.

Limitations​

Query does not allow any input parameters (no input profile) so it cannot be used in RESTful cases where path parameters are required. For example, a URL such as /{customerID}/orders may support searching and sorting all orders for a customer. However, since the customerID must be specified for the operation, Query will not work for this path type. It may be possible to support this by allowing the user to set the entire path with a custom dynamic document property or a required filter parameter. However, that complexity may undermine the usability and advantages of the Query user interface. 

You would need to implement your query operation as a custom operation. Instead of having the benefit of the Query user interface, you would have to manually specify the input profile fields or custom dynamic document properties in the low-level service-specific grammar to implement the query parameters.

<operation types="EXECUTE" customTypeId="QUERY" customTypeLabel="QUERY"/>

Code sample​

Here is a simple example of the Query operation illustrating how to get the filter field and parameter.

Example: Query operation
public class ExampleQueryOperation extends BaseQueryOperation
{
    public ExampleQueryOperation(ExampleConnection conn)
    {
        super(conn);
    }

    @Override
    protected void executeQuery(QueryRequest request, OperationResponse response)
    {
        // grab request information
        FilterData requestData = request.getFilter();

        QueryFilter filter = requestData.getFilter();

        List<Payload> results = new ArrayList<Payload>();

        List<String> selectedFields = getContext().getSelectedFields();
        for (String field : selectedFields) {
            //  format query with requested fields to select
        }

        List<Sort> sortFields = filter.getSort();
        for (Sort s : sortFields) {
            String sortProperty = s.getProperty();
            String sortOrder = s.getSortOrder();
            //  format query with proper sort statement
        }

        if((filter == null) || (filter.getExpression() == null)) {

            // .. execute a QUERY "all" request, populate results ...

        } else {

            // grab base expression (which may be simple or grouped)
            Expression expression = filter.getExpression();

            if (expression instanceof SimpleExpression) {

                // handle simple expression
                SimpleExpression simpleExpr = (SimpleExpression)expression;
                String operator = simpleExpr.getOperator();
                String property = simpleExpr.getProperty();
                List<String> arguments = simpleExpr.getArguments();

                // ... convert simple expression to implementation request ...

            } else {

                // handle grouping expression
                GroupingExpression groupExpr = (GroupingExpression)expression;
                GroupingOperator operator = groupExpr.getOperator();
                List<Expression> nestedExprs = groupExpr.getNestedExpressions();

                // ... convert grouping expression to implementation request ...

            }

            // ... execute QUERY operation using generated filter, populate results ...

        }

        // (if success) return all the results for a single input (requestData)
        ResponseUtil.addSplitSuccess(response, requestData, code, results);
    }

    @Override
    public ExampleConnection getConnection() 
    {
        return (ExampleConnection) super.getConnection();
    }

}

The QueryRequest data provides you with a getFilter() method. From this method, you get the QueryFilter object which has a getExpression() method. A null filter or a filter with no expression means to query all objects of the relevant type. The expression returned from this method is one of the following:

  • GroupingExpression — an AND or OR.

  • SimpleExpression — a true/false condition such as:

    a EQUALS b or c is not null

This structure supports as much nesting of filters as your application requires. The first part of the previous example shows how to handle a simple expression. From the SimpleExpression, you can get the property (which is the filter field name), the operator (EQUALS, GREATER_THAN, etc), and the arguments (the parameter value) assigned through the Parameter tab. The second part of the previous example shows how to handle a grouping expression.

The QueryFilter object also has a getSort() method, which returns a List of Sort objects. You can use this list to format the sort statement in your query.

The OperationContext object has a getSelectedFields() method, which returns the List of fields you selected to be returned.

Query pagination​

A benefit of implementing a connector with a Query operation is that the service returns large amounts of records requiring pagination techniques. Pagination is challenging with the generic HTTP Client connector because you have to add orchestrations and decision steps to loop through pages of connectors. Instead, you can configure the operation to support pagination. When the operation executes, it can request pages sequentially until all the records are received. A page size specified by the user would determine how many records to include per page. All the records will exit the Connector shape on the right side of the canvas as a large stream of documents.

Often, the native format of the service output includes high-level elements to control pagination parameters and a child array for results. For Query operations, it is best to simplify the presentation of the output profile by only including the type for a single response array item and excluding “control” type fields from the user. This results in enhancing the usability of the connector.

On this Page