Batch Poster Guide

Batch Poster Guide#

Batch Poster is a tool for posting Inventory records (Instances, Holdings, Items) to FOLIO using the batch synchronous storage APIs.

Overview#

Batch Poster:

Posts records in configurable batches to FOLIO’s storage APIs
Supports upsert operations (create or update)
Provides field preservation options during updates
Tracks progress with real-time feedback
Captures failed records for retry

Supported Record Types#

Batch Poster works exclusively with FOLIO Inventory storage records:

Object Type	API Endpoint	Description
`Instances`	`/instance-storage/batch/synchronous`	Bibliographic records
`Holdings`	`/holdings-storage/batch/synchronous`	Holdings records
`Items`	`/item-storage/batch/synchronous`	Item records
`ShadowInstances`	`/instance-storage/batch/synchronous`	Consortium shadow instances (ECS)

Note

For other data types, use the appropriate tool:

Users: Use the users command
MARC Records: Use the marc command

Input Format#

Input files must be JSON Lines format (.jsonl) with one record per line:

{"id": "550e8400-e29b-41d4-a716-446655440000", "title": "The Great Gatsby", "instanceTypeId": "6312d172-f0cf-40f6-b27d-9fa8feaf332f", "source": "FOLIO"}
{"id": "660e8400-e29b-41d4-a716-446655440001", "title": "1984", "instanceTypeId": "6312d172-f0cf-40f6-b27d-9fa8feaf332f", "source": "FOLIO"}

Each record must include an id field (UUID). For holdings records, also include sourceId.

Basic Usage#

Command Line#

folio-data-import batch-poster \
  --gateway-url https://folio-snapshot-okapi.dev.folio.org \
  --tenant-id diku \
  --username diku_admin \
  --password admin \
  --object-type Instances \
  --file-path instances.jsonl

CLI Parameters#

FOLIO Connection Parameters#

Parameter	Environment Variable	Description
`--gateway-url`	`FOLIO_GATEWAY_URL`	FOLIO API gateway URL
`--tenant-id`	`FOLIO_TENANT_ID`	FOLIO tenant identifier
`--username`	`FOLIO_USERNAME`	Username for authentication
`--password`	`FOLIO_PASSWORD`	User password
`--member-tenant-id`	`FOLIO_MEMBER_TENANT_ID`	ECS member tenant (if applicable)

Job Configuration Parameters#

Parameter	Default	Description
`--object-type`	(required)	Record type: `Instances`, `Holdings`, `Items`, or `ShadowInstances`
`--file-path`	(required)	Path(s) to JSONL file(s). Accepts multiple values and glob patterns
`--batch-size`	100	Number of records per batch (1-1000)
`--upsert`	false	Enable upsert mode to update existing records
`--failed-records-file`	none	Path to file for writing failed records
`--rerun-failed-records`	false	After main run, reprocess failed records one at a time
`--no-progress`	false	Disable progress bar display

Upsert Options#

These options only apply when --upsert is enabled:

Parameter	Default	Description
`--preserve-statistical-codes`	false	Keep existing statistical codes (merged with new)
`--preserve-administrative-notes`	false	Keep existing administrative notes (merged with new)
`--preserve-temporary-locations`	false	Keep existing temporary location (Items only)
`--preserve-temporary-loan-types`	false	Keep existing temporary loan type (Items only)
`--overwrite-item-status`	false	Allow item status changes (status preserved by default)
`--patch-existing-records`	false	Enable selective field patching
`--patch-paths`	none	Comma-separated list of fields to patch

Configuration File#

For complex configurations, use a JSON configuration file:

folio-data-import batch-poster config.json \
  --gateway-url https://folio-snapshot-okapi.dev.folio.org \
  --tenant-id diku \
  --username diku_admin \
  --password admin

config.json:

{
  "object_type": "Items",
  "file_paths": ["items1.jsonl", "items2.jsonl"],
  "batch_size": 100,
  "upsert": true,
  "preserve_statistical_codes": true,
  "preserve_administrative_notes": true,
  "preserve_temporary_locations": true,
  "preserve_temporary_loan_types": true
}

Configuration uses snake_case keys. CLI parameters override config file values.

Upsert Mode#

Basic Upsert#

Enable upsert to create new records or update existing ones:

folio-data-import batch-poster \
  --gateway-url https://folio-snapshot-okapi.dev.folio.org \
  --tenant-id diku \
  --username diku_admin \
  --password admin \
  --object-type Items \
  --file-path items.jsonl \
  --upsert

Records are matched by their id field. Existing records are fetched to obtain their _version for optimistic locking.

Preservation Options#

When updating existing records, you can preserve specific data:

folio-data-import batch-poster \
  --gateway-url https://folio-snapshot-okapi.dev.folio.org \
  --tenant-id diku \
  --username diku_admin \
  --password admin \
  --object-type Items \
  --file-path items.jsonl \
  --upsert \
  --preserve-statistical-codes \
  --preserve-administrative-notes \
  --preserve-temporary-locations \
  --preserve-temporary-loan-types

How preservation works:

Statistical codes: Existing codes are merged with new codes (duplicates removed)
Administrative notes: Existing notes are merged with new notes (duplicates removed)
Temporary locations: Existing temporary location is kept (Items only)
Temporary loan types: Existing temporary loan type is kept (Items only)
Item status: Preserved by default; use --overwrite-item-status to change

Selective Field Patching#

For fine-grained updates, use --patch-existing-records with --patch-paths:

folio-data-import batch-poster \
  --gateway-url https://folio-snapshot-okapi.dev.folio.org \
  --tenant-id diku \
  --username diku_admin \
  --password admin \
  --object-type Items \
  --file-path items.jsonl \
  --upsert \
  --patch-existing-records \
  --patch-paths "barcode,materialTypeId"

This updates only barcode and materialTypeId from your input file while preserving all other fields from the existing record.

Protected Fields#

Always-Preserved Fields#

Certain fields are always preserved from existing records during upsert, regardless of configuration:

Field	Applies To	Reason
`hrid`	All record types	Human-readable ID - changing it breaks external references
`lastCheckIn`	Items	Circulation data - should not be overwritten by migrations

These fields cannot be overwritten through upsert operations. If you need to change an hrid, you must delete and recreate the record.

MARC-Sourced Instance Protection#

When updating Instance records that have a MARC source (i.e., source contains “MARC”), Batch Poster automatically restricts which fields can be patched. This protects MARC-managed fields from being overwritten, as they would be reverted on the next SRS update anyway.

Allowed fields for MARC-sourced Instances:

Field	Purpose
`discoverySuppress`	Discovery suppression flag
`staffSuppress`	Staff suppression flag
`deleted`	Deletion flag
`statisticalCodeIds`	Statistical codes (merged with existing)
`administrativeNotes`	Administrative notes (merged with existing)
`instanceStatusId`	Instance status

Example: If you try to update the title of a MARC-sourced Instance, that change will be ignored to protect the MARC-managed data.

Note

This protection is automatic. You don’t need to configure anything - MARC-sourced records are detected and handled appropriately.

Multiple Files#

Process multiple files in one run:

folio-data-import batch-poster \
  --gateway-url https://folio-snapshot-okapi.dev.folio.org \
  --tenant-id diku \
  --username diku_admin \
  --password admin \
  --object-type Instances \
  --file-path file1.jsonl \
  --file-path file2.jsonl \
  --file-path file3.jsonl

Or use glob patterns:

folio-data-import batch-poster \
  --gateway-url https://folio-snapshot-okapi.dev.folio.org \
  --tenant-id diku \
  --username diku_admin \
  --password admin \
  --object-type Instances \
  --file-path "data/*.jsonl"

Error Handling#

Failed Records File#

Capture failed records for later review or retry:

folio-data-import batch-poster \
  --gateway-url https://folio-snapshot-okapi.dev.folio.org \
  --tenant-id diku \
  --username diku_admin \
  --password admin \
  --object-type Items \
  --file-path items.jsonl \
  --failed-records-file failed_items.jsonl

Failed records are written in JSON Lines format, allowing you to fix issues and re-run.

Error Types#

Common errors and solutions:

Error	Cause	Solution
422 Unprocessable Entity	Invalid record data	Check required fields, valid UUIDs
409 Conflict	Duplicate record without upsert	Enable `--upsert` or remove duplicates
400 Bad Request	Malformed JSON	Validate JSON Lines format
Timeout	Batch too large	Reduce `--batch-size`

Progress Tracking#

By default, a progress bar shows:

Total records to process
Records completed
Success/failure counts
Processing rate

Disable for CI/CD or scripting:

--no-progress

Common Workflows#

Initial Load (No Upsert)#

For loading new records into an empty system:

folio-data-import batch-poster \
  --gateway-url https://folio-snapshot-okapi.dev.folio.org \
  --tenant-id diku \
  --username diku_admin \
  --password admin \
  --object-type Instances \
  --file-path instances.jsonl \
  --batch-size 100

Bulk Update with Preservation#

Update existing records while preserving administrative data:

folio-data-import batch-poster \
  --gateway-url https://folio-snapshot-okapi.dev.folio.org \
  --tenant-id diku \
  --username diku_admin \
  --password admin \
  --object-type Items \
  --file-path items.jsonl \
  --upsert \
  --preserve-statistical-codes \
  --preserve-administrative-notes \
  --failed-records-file failed_items.jsonl

Update Specific Fields Only#

Update only barcodes without touching other fields:

folio-data-import batch-poster \
  --gateway-url https://folio-snapshot-okapi.dev.folio.org \
  --tenant-id diku \
  --username diku_admin \
  --password admin \
  --object-type Items \
  --file-path barcode_updates.jsonl \
  --upsert \
  --patch-existing-records \
  --patch-paths "barcode"

Rerun Failed Records#

When a batch fails, some records in that batch may have succeeded individually. The --rerun-failed-records flag automatically reprocesses failed records one at a time after the main run completes, giving each record a second chance:

folio-data-import batch-poster \
  --object-type Items \
  --file-path items.jsonl \
  --upsert \
  --failed-records-file failed_items.jsonl \
  --rerun-failed-records

This will:

Process all records in batches (normal operation)
If any batches fail, reprocess those failed records individually
Write still-failing records to a new file with _rerun suffix (e.g., failed_items_rerun.jsonl)

The original failed records file is preserved, and the rerun processes records in a streaming fashion without loading them all into memory.

Note

--rerun-failed-records requires --failed-records-file to be set.

Consortium Shadow Instances (ECS)#

For FOLIO ECS (consortium) environments, use ShadowInstances to post shadow copies to member tenants. This automatically converts the source field:

MARC → CONSORTIUM-MARC
FOLIO → CONSORTIUM-FOLIO

folio-data-import batch-poster \
  --gateway-url https://folio-snapshot-okapi.dev.folio.org \
  --tenant-id central \
  --member-tenant-id member1 \
  --username admin \
  --password admin \
  --object-type ShadowInstances \
  --file-path instances.jsonl \
  --upsert

Environment Variables#

Set connection parameters as environment variables to simplify commands:

export FOLIO_GATEWAY_URL="https://folio-snapshot-okapi.dev.folio.org"
export FOLIO_TENANT_ID="diku"
export FOLIO_USERNAME="diku_admin"
export FOLIO_PASSWORD="admin"

# Then run with just job parameters
folio-data-import batch-poster \
  --object-type Instances \
  --file-path instances.jsonl \
  --upsert

Batch Poster Guide

Contents

Batch Poster Guide#

Overview#

Supported Record Types#

Input Format#

Basic Usage#

Command Line#

CLI Parameters#

FOLIO Connection Parameters#

Job Configuration Parameters#

Upsert Options#

Configuration File#

Upsert Mode#

Basic Upsert#

Preservation Options#

Selective Field Patching#

Protected Fields#

Always-Preserved Fields#

MARC-Sourced Instance Protection#

Multiple Files#

Error Handling#

Failed Records File#

Error Types#

Progress Tracking#

Common Workflows#

Initial Load (No Upsert)#

Bulk Update with Preservation#

Update Specific Fields Only#

Rerun Failed Records#

Consortium Shadow Instances (ECS)#

Environment Variables#

See Also#