Skip to content

Commit e639a37

Browse files
committed
add about tip pageSize modifiedSince
1 parent 0a0ae18 commit e639a37

4 files changed

Lines changed: 104 additions & 5 deletions

File tree

‎Article.md‎

Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -261,6 +261,32 @@ Result:
261261
"packageType":"bulk"
262262
}
263263
```
264+
265+
### More on /file-store/v1/packages parameters
266+
267+
Beside the ```packageName``` query, the ```/file-store/v1/packages``` endpoint supports the following optional parameters:
268+
- *packageType*: Return all packages that match the specified package type.
269+
- *bucketName*:Return all packages that are associated with the specified bucket name.
270+
- *page*: Filter results by a specific pagination index (If client has already specified this query parameter, the
271+
skipToken logic will be excluded)
272+
- *includedTotalResult*: The total search result will be counting and added to the first response message.
273+
- *skipToken*: A token to retrieve the next set of result that exceeds page size.
274+
- *pageSize*: The number of packages that will be shown on one page. Default value is 25.
275+
- *includedEntitilementResult*: CFS will perform a permission check on each package against the client
276+
permission.
277+
278+
**Tips**
279+
280+
It is recommended to call the endpoint with ```pageSize=100``` parameter as follows:
281+
282+
``` HTTP
283+
GET /file-store/v1/packages?bucketName={bucket-name}&pageSize=100 HTTP/1.1
284+
Host: api.refinitiv.com
285+
Authorization: Bearer <Access Token>
286+
```
287+
288+
Please find more detail on the [CFS API User Guide](https://developers.lseg.com/en/api-catalog/refinitiv-data-platform/refinitiv-data-platform-apis/documentation#cfs-api-user-guide) document.
289+
264290
## Step 3: Listing the FileSets using the Bucket Name and Package ID
265291

266292
Now we come to getting the FileSets information. The application needs to send an HTTP ```GET``` request to the RDP ```/file-store/v1/file-sets?bucket={bucket-name}&packageId={packageId}``` endpoint to list all FileSets under the input ```bucket-name``` and ```packageId```.
@@ -328,6 +354,46 @@ The FileSets response message from the API is as follows:
328354

329355
The File ID is in the ```files``` array above. I am demonstrating with the ```4c35-1775-c1a590ea-8376-ac6c1546b908``` file id.
330356

357+
### More on /file-store/v1/file-sets?bucket parameters
358+
359+
Beside the ```bucket``` and ```packageId```queries, the ```/file-store/v1/file-sets?bucket``` endpoint supports the following optional parameters:
360+
361+
- *name*: The name of the file-set. Only exactly matched results are returned.
362+
- *packageId*: Package ID
363+
- *status*: Filter file-set by status (Ready/Pending)
364+
- *availableFrom*: Return all file-sets that become visible to permissioned users after the specified Datetime.
365+
- *availableTo*: Return all file-sets that is no longer visible to permissioned user after the specified Datetime.
366+
- *contentFrom*: Filter results by the age of the content within the file-set.
367+
- *contentTo*: Filter results by the age of the content within the file-set.
368+
- *createdSince*: Return all file-sets that have a created date after the specified Datetime
369+
- *modifiedSince*: Return all file-sets that have a modified date after the specified Datetime.
370+
- *attributes*: Return a list of publisher-defined attributes of the file-sets.
371+
- *pageSize*: The number of file-sets that will be shown on one page. Default value is 25.
372+
- *skipToken*: A token to retrieve the next set of file-set result that exceeds page size.
373+
374+
The ```modifiedSince``` parameter can help an application to limit the returned File-Set only for the File-Set that has been modified after a specified time.
375+
376+
**Tips**
377+
378+
It is recommended to call the endpoint with ```pageSize=100``` and ```modifiedSince``` parameters as follows:
379+
380+
``` HTTP
381+
GET /file-store/v1/file-sets?bucket={bucket-name}&packageId={packageId}&modifiedSince={datetime}&pageSize=100 HTTP/1.1
382+
Host: api.refinitiv.com
383+
Authorization: Bearer <Access Token>
384+
```
385+
386+
Example:
387+
388+
``` HTTP
389+
GET /file-store/v1/file-sets?bucket=bulk-ESG&packageId=4288-ebb6-93372235-acb2-89882a826af1&pageSize=100&modifiedSince=2022-01-26T00:00:00Z HTTP/1.1
390+
Host: api.refinitiv.com
391+
Authorization: Bearer <Access Token>
392+
```
393+
394+
Please find more detail on the [CFS API User Guide](https://developers.lseg.com/en/api-catalog/refinitiv-data-platform/refinitiv-data-platform-apis/documentation#cfs-api-user-guide) document.
395+
396+
331397
### Step 3.1: Listing the packageId using the Bucket Name - Paging
332398

333399
My next point is the paging feature. By default, the ```/file-store/v1/file-sets?bucket={bucket-name}``` endpoint always returns 25 results per request. You can adjust the number of return results via the ```pageSize``` query parameter, the maximum number is **100**.
@@ -519,6 +585,17 @@ Result:
519585
'https://a206464-bulk-esg.s3.amazonaws.com/Bulk-ESG-Global-Symbology-Organization-v1/2023/11/26/Bulk-ESG-Global-Symbology-Organization-v1-Init-2023-11-26T16%3A04%3A11.525Z.jsonl.gz?x-request-Id=e7658630-f8c6-4bd3-9443-4d87efa20b5c&x-package-id=4037-e79c-96b73648-a42a-6b65ef8ccbd1&x-client-app-id=b4842f3904fb4a1fa18234796368799086c63541&x-file-name=Bulk-ESG-Global-Symbology-Organization-v1-Init-2023-11-26T16%3A04%3A11.525Z.jsonl.gz&x-fileset-id=4646-6302-b810e622-8808-85367d798021&x-bucket-name=bulk-ESG&x-uuid=GESG1-178570&x-file-Id=4de0-ceda-25b5a1f1-9b7e-35c10b384078&x-fileset-name=Bulk-ESG-Global-Symbology-Organization-v1-Jsonl-Init-2023-11-26T16%3A04%3A11.525Z&x-event-external-name=cfs-claimCheck-download&X-Amz-Security-Token=IQoJb3JpZ2luX2VjEFAaCXVzLWVhc3QtMSJHMEUCIEzV%2BjWQVpI6MyZSaZ8SDQHlLPSsv8n50rxgWVDO6l%2F6AiEAu6f00kJgGFokGZxSWXicGqPbiL2X1SwEI16MlBgrMQwqowIIuP%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FARAEGgw2NDIxNTcxODEzMjYiDBu3pKa9FpE%2Fi14VMCr3AQDhdsrjQAR4YmsEBme6RolP2AlZSYOhk8cH5xRqqus1fYhG0jIxx5Rj0t5n7%2Fcy5fq7TX9ygoR4JJDjRKpHhS4weeTn2oqcEPEyGlegGJuktEjmWrRFqANR3vSzFQQbUECxDSC%2FnHAuIUz2X130j30SC31aNihaF1XNWJEGcxGYVNWKPslvVe3Ohg1euVup4kvH3YpIhfAGHnPHhyAHoK7M8K417rAMqkuSP05XGyf%2BD%2BuPSiS9n2EM66XnHZUthf5nkm70bk1%2B%2FpA%2BC5opQIRfhE7kHMg29qKWpPiyJJCtvmj9N79AeEwnfm%2FD%2BXlV5ZM%2BVx8cHGkw5YPLqwY6nQFj%2BbHSvw088XgcKLEde1RJdKZ9E1t39X%2FT3m765zf8I94m8skLdLd9tSJalzFnQbq%2BWx4zABO5VKm1xJ9Z%2FFyRklKzTX8B6fmV4ioEqbvauMn0OT4Lcn2BQLDoLefsZc0WaUQd5p1N8UVSfgVzcv3yNR4%2FHCFmXhKPMT%2F8MNWjlwG438YlrNRVCrWZpl2Eogx7shza%2Bl99v0rLdtF4&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20231208T071237Z&X-Amz-SignedHeaders=host&X-Amz-Expires=21600&X-Amz-Credential=ASIAZLA4M7GHBJJLJVV4%2F20231208%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=8f67fbf45259c2830f7e2578d03e7e546832ef34fba37b769b951e071ad08175'
520586
```
521587

588+
### More on /file-store/v1/files/ parameters
589+
590+
Beside the ```file_id``` query, the ```/file-store/v1/files/``` endpoint supports the following optional parameters:
591+
592+
- *createdSince*: Return all files that have a created date after the specified Datetime.
593+
- *modifiedSince*: Return all files that have a modified date after the specified Datetime.
594+
- *pageSize*: The number of files that will be shown on one page. Default value is 25.
595+
- *skipToken*: A token to retrieve the next set of file result that exceeds page size
596+
597+
Please find more detail on the [CFS API User Guide](https://developers.lseg.com/en/api-catalog/refinitiv-data-platform/refinitiv-data-platform-apis/documentation#cfs-api-user-guide) document.
598+
522599
### Step 5: Downloading the file
523600

524601
That brings us to download the file. You can download the bulk file using that S3 URL (**as is**). **Do not alter or make any changes to the URL text string**. It will cause unable to download or signature mismatch error.

‎RDP-Generic-CFS.html‎

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7917,6 +7917,12 @@ <h3 id="More-on-/file-store/v1/packages-parameters">More on /file-store/v1/packa
79177917
permission.</li>
79187918
</ul>
79197919
<p>Please find more detail on the <a href="https://developers.lseg.com/en/api-catalog/refinitiv-data-platform/refinitiv-data-platform-apis/documentation#cfs-api-user-guide">CFS API User Guide</a> document.</p>
7920+
<p><strong>Tips</strong></p>
7921+
<p>It is recommended to call the endpoint with <code>pageSize=100</code> parameter as follows:</p>
7922+
<div class="highlight"><pre><span></span><span class="nf">GET</span> <span class="nn">/file-store/v1/packages?bucketName={bucket-name}&amp;pageSize=100</span> <span class="kr">HTTP</span><span class="o">/</span><span class="m">1.1</span>
7923+
<span class="na">Host</span><span class="o">:</span> <span class="l">api.refinitiv.com</span>
7924+
<span class="na">Authorization</span><span class="o">:</span> <span class="l">Bearer &lt;Access Token&gt;</span>
7925+
</pre></div>
79207926
</div>
79217927
</div>
79227928
</div>
@@ -8134,12 +8140,14 @@ <h3 id="More-on-/file-store/v1/file-sets?bucket-parameters">More on /file-store/
81348140
<li><em>skipToken</em>: A token to retrieve the next set of file-set result that exceeds page size.</li>
81358141
</ul>
81368142
<p>The <code>modifiedSince</code> parameter can help an application to limit the returned File-Set only for the File-Set that has been modified after a specified time.</p>
8137-
<div class="highlight"><pre><span></span><span class="nf">GET</span> <span class="nn">/file-store/v1/file-sets?bucket={bucket-name}&amp;packageId={packageId}&amp;modifiedSince={datetime}</span> <span class="kr">HTTP</span><span class="o">/</span><span class="m">1.1</span>
8143+
<p><strong>Tips</strong></p>
8144+
<p>It is recommended to call the endpoint with <code>pageSize=100</code> and <code>modifiedSince</code> parameters as follows:</p>
8145+
<div class="highlight"><pre><span></span><span class="nf">GET</span> <span class="nn">/file-store/v1/file-sets?bucket={bucket-name}&amp;packageId={packageId}&amp;modifiedSince={datetime}&amp;pageSize=100</span> <span class="kr">HTTP</span><span class="o">/</span><span class="m">1.1</span>
81388146
<span class="na">Host</span><span class="o">:</span> <span class="l">api.refinitiv.com</span>
81398147
<span class="na">Authorization</span><span class="o">:</span> <span class="l">Bearer &lt;Access Token&gt;</span>
81408148
</pre></div>
81418149
<p>Example:</p>
8142-
<div class="highlight"><pre><span></span><span class="nf">GET</span> <span class="nn">/file-store/v1/file-sets?bucket=bulk-ESG&amp;packageId=4288-ebb6-93372235-acb2-89882a826af1&amp;modifiedSince=2022-01-26T00:00:00Z</span> <span class="kr">HTTP</span><span class="o">/</span><span class="m">1.1</span>
8150+
<div class="highlight"><pre><span></span><span class="nf">GET</span> <span class="nn">/file-store/v1/file-sets?bucket=bulk-ESG&amp;packageId=4288-ebb6-93372235-acb2-89882a826af1&amp;pageSize=100&amp;modifiedSince=2022-01-26T00:00:00Z</span> <span class="kr">HTTP</span><span class="o">/</span><span class="m">1.1</span>
81438151
<span class="na">Host</span><span class="o">:</span> <span class="l">api.refinitiv.com</span>
81448152
<span class="na">Authorization</span><span class="o">:</span> <span class="l">Bearer &lt;Access Token&gt;</span>
81458153
</pre></div>

‎RDP-Generic-CFS.ipynb‎

Lines changed: 17 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -318,7 +318,17 @@
318318
"- *includedEntitilementResult*: CFS will perform a permission check on each package against the client \n",
319319
"permission.\n",
320320
"\n",
321-
"Please find more detail on the [CFS API User Guide](https://developers.lseg.com/en/api-catalog/refinitiv-data-platform/refinitiv-data-platform-apis/documentation#cfs-api-user-guide) document."
321+
"Please find more detail on the [CFS API User Guide](https://developers.lseg.com/en/api-catalog/refinitiv-data-platform/refinitiv-data-platform-apis/documentation#cfs-api-user-guide) document.\n",
322+
"\n",
323+
"**Tips**\n",
324+
"\n",
325+
"It is recommended to call the endpoint with ```pageSize=100``` parameter as follows:\n",
326+
"\n",
327+
"``` HTTP\n",
328+
"GET /file-store/v1/packages?bucketName={bucket-name}&pageSize=100 HTTP/1.1\n",
329+
"Host: api.refinitiv.com\n",
330+
"Authorization: Bearer <Access Token>\n",
331+
"```"
322332
]
323333
},
324334
{
@@ -502,16 +512,20 @@
502512
"\n",
503513
"The ```modifiedSince``` parameter can help an application to limit the returned File-Set only for the File-Set that has been modified after a specified time. \n",
504514
"\n",
515+
"**Tips**\n",
516+
"\n",
517+
"It is recommended to call the endpoint with ```pageSize=100``` and ```modifiedSince``` parameters as follows:\n",
518+
"\n",
505519
"``` HTTP\n",
506-
"GET /file-store/v1/file-sets?bucket={bucket-name}&packageId={packageId}&modifiedSince={datetime} HTTP/1.1\n",
520+
"GET /file-store/v1/file-sets?bucket={bucket-name}&packageId={packageId}&modifiedSince={datetime}&pageSize=100 HTTP/1.1\n",
507521
"Host: api.refinitiv.com\n",
508522
"Authorization: Bearer <Access Token>\n",
509523
"```\n",
510524
"\n",
511525
"Example:\n",
512526
"\n",
513527
"``` HTTP\n",
514-
"GET /file-store/v1/file-sets?bucket=bulk-ESG&packageId=4288-ebb6-93372235-acb2-89882a826af1&modifiedSince=2022-01-26T00:00:00Z HTTP/1.1\n",
528+
"GET /file-store/v1/file-sets?bucket=bulk-ESG&packageId=4288-ebb6-93372235-acb2-89882a826af1&pageSize=100&modifiedSince=2022-01-26T00:00:00Z HTTP/1.1\n",
515529
"Host: api.refinitiv.com\n",
516530
"Authorization: Bearer <Access Token>\n",
517531
"```\n",

‎RDP-Generic-CFS.pdf‎

4.66 KB
Binary file not shown.

0 commit comments

Comments
 (0)