Bulkload data from an XBRL repository, Part 3: Configure Local File System Repository
Estimated reading time: 1 minute- 1: Introduction and Installation
- 2: Metadata and Storage
- 3: Configure Local File System Repository
- 4: Configure AWS S3 Repository
- 5: Run CellStore ETL Tool
Part 3: Configure Local File System Repository
The CellStore ETL Tool can be configured to bulkload XBRL archives from local file system.
The configuration for the Tool needs to be added to the reportix.properties file:
ETL_<myConfigName>_SOURCE_TYPE=FILE
ETL_<myConfigName>_FILE_ROOT=D:/data/archive
ETL_<myConfigName>_LOG_TYPE=FILE
ETL_<myConfigName>_LOG_PATH=log/etl_TEST2_%tF.log
Because there can be several configurations in the same reportix.properties each configuration needs to be given a custom name (replacing placeholder: <myConfigName>).
| Config Entry | Description |
|---|---|
ETL_<myConfigName>_SOURCE_TYPE |
The source type (where the imported filings are fetched from) |
ETL_<myConfigName>_FILE_ROOT |
The root directory in which the filings are stored |
ETL_<myConfigName>_LOG_TYPE |
How to log the results of each filing procession (Currently the only value and default is “FILE”) |
ETL_<myConfigName>_LOG_PATH |
Where to log the results of each filing processed. Adds one log line for each filing. The default is log/etl_%tF.log (which stores the logs in the server home dir) |
For example we can create one source with name MYLOCALDIR:
ETL_MYLOCALDIR_SOURCE_TYPE=FILE
ETL_MYLOCALDIR_FILE_ROOT=D:/data/archive
ETL_MYLOCALDIR_LOG_TYPE=FILE
ETL_MYLOCALDIR_LOG_PATH=log/etl_TEST2_%tF.log