Tiered Ingest allows you to group all of the files corresponding to a simple asset's datastreams (including archival files, publication files, other derivatives created outside of Islandora, with the exception of RELS-EXT) into a sub-directory.
| Warning |
|---|
When ingesting derivative files, with this method, they may be overwritten by the DAMS software. |
| Note |
|---|
This tiered batch ingest method is NOT suitable for paged content (complex/compound assets with children). See Batch ingest complex assets (paged content) for instructions on how to ingest assets comprised of multiple pages. The tiered ingest allows you to store additional files with a digital asset, and you can use this method to ingest externally created derivative datastreams (e.g. for streaming audio). See Content models for a breakdown of the expected datastreams per content model, and for information which datastreams can be published to e.g. the Collections Portal. |
| Multiexcerpt include | ||||
|---|---|---|---|---|
| ||||
| Multiexcerpt include | ||||
| MultiExcerptName | datastreams generator script | PageWithExcerpt | DAMS datastreams.txt generator
Staging folder structure
- All of the files you are ingesting as part of one asset will be staged in one directory per asset, as a sub-directory of
...
- a batch job folder.
...
- Each sub-directory corresponds to one asset and must
...
- contain at least a manifest file for the
...
- key datastreams
...
- (datastreams.txt).
...
- The batch job folder can contain just one asset folder, but would still need the extra nesting
Sample folder structure
| Code Block |
|---|
eid1234_example-batch-submission/ (batch job folder)
├── asset1/
│ ├── datastreams.txt
│ ├── modsfile.xml
│ ├── primaryfile.tif
│ ├── anyarbitraryderivativefile.ext
│ ├── anyarbitrarycomponentfile.ext
│ └── anymediaphotographfile.ext
├── asset2_audio_example/
│ ├── datastreams.txt
│ ├── modsfile.xml
│ ├── audiofile.wav
│ ├── derivative_audiofile_for_streaming.mp4 (e.g. for creating PROXY_MP4 datastream, which is required for streaming audio)
│ └── audio_transcript.txt
└── asset3_video_example/
├── datastreams.txt
├── modsfile.xml
├── videofile.mp4
├── video_captions.vtt
└── video_transcript.txt
└── page02_custom_ocr.txt |
Step 2: Create datastreams.txt manifest
Subdirectories in the batch job folder MUST each contain a manifest file named datastreams.txt. The manifest file specifies the intended structure of the DAMS asset, for instance pointing to the MODS XML containing the metadata for the asset, or specifying which additional datastreams should be created from staged files.
Each line of the manifest file contains an argument-value pair in the following format:
<ARGUMENT>==<VALUE>
Use 2 (two) equal signs to separate arguments and values.
| Multiexcerpt include | ||||
|---|---|---|---|---|
|
| Code Block | ||||
|---|---|---|---|---|
| ||||
OBJ==primaryfile.ext MODS==metadata.xml # optional, if no MODS file is included, minimal metadata is automatically generated during ingest PDF==custom.pdf # optional ARCHIVAL_FILE==originalversionof_primaryfile.ext # optional, use for archival file (e.g. uncropped scan) COMPONENT1==componentfile1.ext # optional, can for instance be used in cases where a primary image is stitched from multiple component images; increment for additional files in same directory # DO NOT use for complex objects that can be modeled as paged content or Islandora component assets! MEDIAPHOTOGRAPH1==anymediaphotographfile.ext # optional, can be used for images documenting physical media, cases, covers, etc.; increment for additional files in same directory DERIVATIVE1==anyarbitraryderivativefile.ext # optional, use for derivative files with direct descendant relationship from file designated OBJ; increment for additional in same directory # CAUTION, do not duplicate derivative files that are automatically generated by the DAMS |
Manifest Arguments
The manifest may contain instructions to create the following datastreams. Refer to Anatomy of DAMS digital assets and Content models for a list of allowed/expected datastreams per content model.
| Warning |
|---|
DO NOT use any of the Restricted Datastream IDs. |
| <ARGUMENT> | Value Associated | Purpose | Accepted File Types | Additional Notes | ||
|---|---|---|---|---|---|---|
| MODS | MODS XML file name | provide MODS metadata for an asset | xml | Can be used for publication/series-level assets, book and issue-level assets. | ||
| TN | thumbnail image file name | provide a thumbnail picture for an asset | png, jpg, jpeg | Can be used for publication/series-level assets, book and issue-level assets. If no thumbnail is provided during batch ingest, the DAMS will copy the thumbnail image of the first page of the asset to the book/issue level asset. | ||
| FULL_TEXT_CUSTOM | name of text file with externally created full text (text extracted from PDF) | allows you to provide your own FULL_TEXT datastream for a book/issue | txt | Can be used for book/issue-level assets.
| ||
| name of your pdf file | PDF for resource | Can be used for book/issue-level assets. Use to add an externally created PDF document to an asset.
|
OBJ==primaryfile.ext [designation of primary file is at digital stewardship staff discretion, in consultation with requesting content holder]
...
Sample folder structure
| Code Block |
|---|
eid1234_example-batch-submission/ (batch job folder)
├── asset1/
│ ├── datastreams.txt
│ ├── modsfile.xml
│ ├── primaryfile.tif
│ ├── anyarbitraryderivativefile.ext
│ ├── anyarbitrarycomponentfile.ext
│ └── anymediaphotographfile.ext
├── asset2_audio_example/
│ ├── datastreams.txt
│ ├── modsfile.xml
│ ├── audiofile.wav
│ ├── derivative_audiofile_for_streaming.mp4 (e.g. for creating PROXY_MP4 datastream, which is required for streaming audio)
│ └── audio_transcript.txt
└── asset3_video_example/
├── datastreams.txt
├── modsfile.xml
├── videofile.mp4
├── video_captions.vtt
└── video_transcript.txt
└── page02_custom_ocr.txt |
Notes:
...