Reading More Than 5000 Items Using REST API In SharePoint Framework (SPFx) With Angular

SharePoint always comes with some restriction to boost up the performance. One of the restrictions is that a threshold limit to read data from list is 5000, which means we cannot read more than 5000 items from a list for an API call. Here we are going to see how to read all the items in a SharePoint List with more than 5000 items.

The approach we are talking here will fit any type of technology you use to contact SharePoint through Rest API with some slight changes in the headers. Let me explain this for SPFx now.

We could do this with a pagination concept by crawling the pages/items in a list. But the first requirement to go with this approach is we need to know the total item count in the list.

Below is the REST Api url to get item count in a list using spHttpClient GET method

{YourSiteCollectionUrl}/_api/web/Lists/GetByTitle('YourListTitle')/ItemCount

Below is the REST Api Url to read the paged items list using spHttpClient GET method

{YourSiteCollectionUrl}/_api/web/Lists/GetByTitle('{YourListTitle}')/items?%24skiptoken=Paged%3DTRUE%26p_ID%3D{ItemToStartsWith}&%24top={TopCount}&$select=Attachments,AttachmentFiles,*&$expand=AttachmentFiles

Let me explain about this in some depth, the core concept is looping through the items. Still here the SharePoint threshold limit of 5000 items per API call applies so your page size should not exceed 5000 items.

Below is a simple overview of how you will get the 15000 items in a list with just 3 API calls.

First API call

ItemsToStartWith=1, TopCount=5000, Rest API Url will be formatted like below

{YourSiteCollectionUrl}/_api/web/Lists/GetByTitle('{YourListTitle}')/items?%24skiptoken=Paged%3DTRUE%26p_ID%3D1&%24top=5000&$select=Attachments,AttachmentFiles,*&$expand=AttachmentFiles

Second API call

ItemsToStartWith=5001, TopCount=5000, Rest API Url will be formatted like below

{YourSiteCollectionUrl}/_api/web/Lists/GetByTitle('{YourListTitle}')/items?%24skiptoken=Paged%3DTRUE%26p_ID%3D5001&%24top=5000&$select=Attachments,AttachmentFiles,*&$expand=AttachmentFiles

Third API call

ItemsToStartWith=10001, TopCount=5000, Rest API Url will be formatted like below

{YourSiteCollectionUrl}/_api/web/Lists/GetByTitle('{YourListTitle}')/items?%24skiptoken=Paged%3DTRUE%26p_ID%3D10001&%24top=5000&$select=Attachments,AttachmentFiles,*&$expand=AttachmentFiles

That’s it we have read all the 15,000 items in a list with 3 API calls.

Below is the basic code snippet to call SharePoint REST Api call using GET method

this.context.spHttpClient.get(requestUrl, SPHttpClient.configurations.v1).then((response: SPHttpClientResponse) => {
    if (response.ok) {
        response.json().then((responseJSON) => {
            console.log(responseJSON);
        });
    }
});