POST, PUT, DELETE Limits

The kintone REST API has limits on the number of records that can be operated on at one time for GET, POST, PUT, and DELETE.

The per-call limits for record operation APIs are as follows (check the cybozu developer network for the latest specifications):

Since it is cumbersome to input kintone.api… every time you use the REST API and create error handling, I think there are not a few people who define generic functions or classes, but the problem that tends to occur at such times is reaching this limit.

For GET, the cybozu developer network publishes code that retrieves all records using the cursor API, but there is no equivalent for POST, PUT, or DELETE.

Reference: Retrieve All Records

This time, we will introduce a function that allows you to create records in bulk without worrying about the above record limits.

Although we will only introduce the POST pattern this time, it can also be adapted to PUT and DELETE by changing the API method.

Source Code

Easily Describe Repeated Single Requests

Here is the basic code. If the number of records you want to execute exceeds the limit, the API is executed in chunks of the limit from the beginning, and the remainder is executed at the end.

/** REST API endpoint */
const END_POINT = '/k/v1/';

/** Record limit for a single POST */
const LIMIT_POST = 100;

/** Split an array into chunks of the given size */
const chunk = (arr, size) =>
  Array.from({ length: Math.ceil(arr.length / size) }, (_, i) =>
    arr.slice(i * size, i * size + size)
  );

const postAllRecords = async (app, records) => {
  for (const chunked of chunk(records, LIMIT_POST)) {
    await kintone.api(kintone.api.url(`${END_POINT}records`, true), 'POST', {
      app: app,
      records: chunked,
    });
  }
};

When Using bulkRequest

In the pattern using only the POST API mentioned above, if an error occurs, it can only be rolled back in units of 100.

If you want to maximize the number of records that can be rolled back, use the following code that utilizes bulkRequest.

/** REST API endpoint */
const END_POINT = '/k/v1/';

/** Record limit for a single POST */
const LIMIT_POST = 100;

/** Request limit for simultaneous processing */
const LIMIT_BULK_REQUEST = 20;

/** Split an array into chunks of the given size */
const chunk = (arr, size) =>
  Array.from({ length: Math.ceil(arr.length / size) }, (_, i) =>
    arr.slice(i * size, i * size + size)
  );

const bulkRequest = async (requests) => {
  const responses = [];

  for (const chunked of chunk(requests, LIMIT_BULK_REQUEST)) {
    responses.push(
      await kintone.api(kintone.api.url(`${END_POINT}bulkRequest`, true), 'POST', {
        requests: chunked,
      })
    );
  }

  return responses;
};

const postAllRecords = async (app, records) => {
  const payloads = chunk(records, LIMIT_POST).map((records) => ({ app, records }));

  const requests = payloads.map((payload) => ({
    method: 'POST',
    api: `${END_POINT}records.json`,
    payload,
  }));

  return bulkRequest(requests);
};

The result of executing bulkRequest is an object, and the results of the normal API execution are stored in an array within results.

To avoid making it obvious that bulkRequest is being used, it is necessary to flatten the return value. For example, combine the results of each response into one array:

// At the end of the bulkRequest function (instead of `return responses;`)
return responses.flatMap(({ results }) => results);

For PUT and DELETE

Although we introduced the POST pattern in the sample, it can also be adapted to PUT and DELETE by changing method and matching the payload format.

One thing to note is that the content passed as records and the request payload format change for POST, PUT, and DELETE.

In the case of POST, it was an array of kintone record data, but in the case of PUT, it becomes an array of objects containing the record ID and kintone record as follows (the payload is { app, records }).

In the case of DELETE, it becomes an array of record IDs, and the payload is { app, ids }.

// PUT
records = [
  { id: 1, record: kintoneRecord1 },
  { id: 2, record: kintoneRecord2 },
  //...
];

// DELETE
records = [
  1,
  2,
  3,
  4,
  5, //...
];

Things to Note

Beyond the scope of a single API call, successfully processed records are not rolled back automatically if a later step fails.

bulkRequest is transactional within a single call: either all of its requests succeed or all are cancelled. One call can handle at most

POST record limit × bulkRequest request limit = 100 × 20 = 2,000 records.

For example, if you register 5,000 records, bulkRequest is split into three calls of 2,000, 2,000, and 1,000 records. If an error occurs at the 3,000th record, the second call (records 2,001–4,000) is cancelled, but the first call (records 1–2,000) has already succeeded and remains registered.

This cannot be solved with the current API, so for safer execution, check the data consistency in advance. Logging how far the registration got also makes it easier to re-run after a failure.