How to implement paging in the Custom REST Configuration Editor.
There is a demonstration video that explains three different pagination templates available in the Custom REST Configuration Editor. This demonstration video only shows a brief overview that the templates are available and how they can be used to implement common pagination types. The demonstration video can be found on the REST documentation --> Click the Connection tab --> Click "Demonstration" on the right side menu --> choose the demonstration video named "Reading with pagination" and press the play button.
While the Custom REST Configuration Editor is able to include the code for the <Iterator>, it cannot predict the correct implementation for the Parameters that need to be included. Therefore, the parameters have to be defined by the person creating the package in the configuration because the engine cannot automatically include them. Those parameters can be part of the query string but they could also participate in the body. That's why they have to be defined by the person creating the package based on the requirements from the documentation from the service they are connecting to.
In the demonstration video, the 3rd example given for the "Cursor based" template, gives this further instruction that parameters also need to be created in order to make the pagination work. More specifically, instructions are given to create the parameters to associate the internal COZYROC iterator and batchSize to the required resources parameter names which are "limit" and "offset" in this case. These parameters need to be created for the READ endpoint for the resource where pagination is to occur.
<Read Url="/events" Result="{{=response.data}}">
<Parameters>
<Parameter Name="limit" Value="{{=parameters.batchSize}}" />
<Parameter Name="offset" Value="{{=parameters.iterator}}" />
</Parameters>
<Iterator>
<Next Value="{{=parseInt(parameters.iterator || 0) + parameters.batchSize}}" />
</Iterator>
</Read>
In the demonstration video, the 2nd example for "Offset & BatchSize" does not go into detail about creating these necessary parameters, however they do also need to be created as well for this pagination type. The pagination will not work until the author of the package also creates the appropriate parameters similar as shown in the "Cursor based" example in the demonstration video.
Book a Presales Call
Are you new to COZYROC and evaluating our solutions?
Book a Presales Call with us now and get 5% off SSIS+.
Book a Presales Call with us now and get 5% off SSIS+.
Related Articles
How to set up Fiddler for testing a REST configuration
This article (<-- Click the link) describes how to install Fiddler to trace traffic as well as decrypt https traffic and how to set up filters if you know what to filter on. Since the REST authentication will use https, it can be helpful to close any ...How to use an XML configuration in a REST Connection Manager
Follow these steps to use the configuration Right Click Connection Managers. Choose New Connection. Select Connection manager for REST connections. In the Configuration drop down choose {Custom}. A wrench appears to the right of the Configuration ...What is an experimental REST configuration and where can I find the documentation for the experimental REST configurations?
Experimental REST configurations are a starting point for configurations to REST services that COZYROC has provided in order to give you a jump start to using the Custom REST Configuration Editor which will allow you to continue building out any of ...How to set up authentication for a REST application using COZYROC's REST Wizard UI
If you want to use COZYROC's REST wizard UI to define the authentication required to connect to a custom REST application, please use the Custom REST Starter Series on this page. It goes through 101 steps to set up and test authentication using the ...I see a response returned from the web service, but the preview in REST Source does not show any data.
CozyRoc Rest Source Component allow you to Read data from REST API. Developer can quickly extract response data from desire endpoint using HTTP Method (GET, POST) and Output into column data. In this article we will walk through some examples that ...