Skip to content

Commit

Permalink
Merge pull request #91 from NYPL/new-transfer
Browse files Browse the repository at this point in the history
New Transfer Pages
  • Loading branch information
dreyjo committed May 2, 2024
2 parents c90fa6f + 6a1fb1c commit fe0d54e
Show file tree
Hide file tree
Showing 19 changed files with 236 additions and 53 deletions.
61 changes: 61 additions & 0 deletions transfers/Completing-transfer.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
---
title: Completing Transfer
layout: default
nav_order: 8.1
parent: Transfers
---
# Completing Transfer <!-- (digarch issue 87) -->
This page outlines the steps to take after transferring media.
* Verifying all carriers were transferred
* Verifying all packages meet requirements
* Verifying all SPEC records were updated
* Storing the carriers and updating Location

## Verifying all carriers were transferred
* Note when carrier transfer fails
* Note when carriers are identified as AMI
* Update objects in SPEC
* Send carriers to Audio and Moving Image Description

(Refer to transfer instructions for the format you are transferring)

## Verifying all packages meet requirements
Under Construction
{: .label .label-yellow}
* Run the packaging script

## Updating records in SPEC
* Navigate to objects records for an acquisition in SPEC (see [verifying acquisition](preparing-for-transfer.md#verifying-acquisition-record-exists-in-spec))
* Select an object record to update
* Select no status indicated under MIGRATION/DIGITIZATION STATUS on the left

<details>
<summary>Select to view image</summary>
<img alt="MIGRATION/DIGITIZATION STATUS in object record"
src="completing-transfer/SPEC-migration-status.png" width="70%"/>
</details>

* Select Add Migration Status

<details>
<summary>Select to view image</summary>
<img alt="add migration status button"
src="completing-transfer/SPEC-add-migration.png" width="70%"/>
</details>

* Select a status from the dropdown

<details>
<summary>Select to view image</summary>
<img alt="migration status dropdown"
src="completing-transfer/SPEC-migration-status-dropdown.png" width="70%"/>
</details>

* Select a date

## Storing the carriers
* Return carriers to shelving by the entrance door
* Place carriers on `02.**` or `03.**` shelves as space allows
* Update the carriers' location in Trello using the Location Custom Field
* Update the carriers' location in SPEC

Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
167 changes: 167 additions & 0 deletions transfers/preparing-for-transfer.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,167 @@
---
title: Preparing for Transfer
layout: default
nav_order: 1
parent: Transfers
---
# Preparing for Transfer <!-- (digarch issue 79) -->
This page outlines the steps to take before transferring media objects.
* Locating and verifying carriers against SPEC records
* Removing non-digital archives carriers
* Commercial software
* AMI (addressed in [completing transfer](completing-transfer.md))
* Choosing transfer methods
* Setting up space to store files on RAID

## Locating Carriers
* Carriers are located on shelving by the entrance door
* Carriers to be transferred are likely on `02.**` shelves
*
**Before a media object can be imaged it first must be recorded SPEC.**

## Verifying SPEC records
An acquisition record should exist in SPEC and SPEC object records should exist
for each carrier being transferred.

### Verifying acquisition record exists in SPEC
* Use the Filemaker Pro desktop app or or [web app](https://fm.nypl.org/fmi/webd/CollectionInfo) to access SPEC
* Log in with your credentials
* Navigate to the bottom left of the screen
* Select Find/propose an acquisition under Acquisitions heading

<details>
<summary>Select to view image</summary>
<img alt="SPEC find acquisition screen" src="preparing-for-transfer/SPEC-find-acq.png" width="25%"/>
</details>

* Enter acquisition number in the ID field or name in the Acquisition field

<details>
<summary>Select to view image</summary>
<img alt="SPEC acquisition search" src="preparing-for-transfer/SPEC-acq-search.png" />
</details>

* Select search

**Contact Collection Management if you cannot find the acquisition**

### Verifying object records exist in SPEC
* Select the acquisition from the list

<details>
<summary>Select to view image</summary>
<img alt="SPEC acquisition search results"
src="preparing-for-transfer/SPEC-acq-search-result.png" />
</details>
* Note if there are any associated collections

<details>
<summary>Select to view image</summary>
<img alt="SPEC associated collections"
src="preparing-for-transfer/SPEC-associated-collections.png" />
</details>
* Navigate to the left side of the screen
* Note the number of items listed for Digital media under Extent Summary

<details>
<summary>Select to view image</summary>
<img alt="SPEC extent summary" src="preparing-for-transfer/SPEC-extent-summary.png" width="40%"/>
</details>
* Select format summary next to Extent Summary, the right most button

* Select digital media, mouseover text reads go to object records

<details>
<summary>Select to view image</summary>
<img alt="SPEC format summary"
src="preparing-for-transfer/SPEC-object-format-count.png" />
</details>
* Confirm the number of carriers Digital Archives has match the object records

<details>
<summary>Select to view image</summary>
<img alt="SPEC object records list"
src="preparing-for-transfer/SPEC-object-records.png" />
</details>
* Confirm the number of items listed for Digital media matches the object records
* Report mismatch of digital media summary and object records to Systems and Operations

**Contact Collection Management if the number of carriers and records don't match**

### Verifying additional carriers from Archival Processing
When archivists survey a collection they may find additional carriers in boxes.
Archivists should create object records for any carriers they find and notify
Digital Archives of their discovery.
* [Verify](#verifying-object-records-exist-in-spec) objects records exist in SPEC

**Contact the processing archivist if the number of carriers and records don't match**

### Handling Commercial Software
**These instructions are for Digital Preservation staff (including fellows) only.**
* Label commercial software with a simple acquisition name.
* `Crouch` for Stanley Crouch papers.
* Image commercial software only when specifically instructed to do so.
* Commercial software media don't receive objects records.

**When objects records were created for commercial software create an issue.**

* Select issues from the object record menu in SPEC
<details>
<summary>Select to view image</summary>
<img alt=SPEC object record issues
src="preparing-for-transfer/SPEC-object-record-issue.png" />
</details>
* Select Add Issue

<details>
<summary>Select to view image</summary>
<img alt="SPEC Issue Add button" src="preparing-for-transfer/SPEC-issue-list-add.png" />
</details>

* Select Type: Digital Media
* Enter Issue: Commercial Software

<details>
<summary>Select to view image</summary>
<img alt="SPEC Issue input" src="preparing-for-transfer/SPEC-issue-type.png" />
</details>

## Choosing Transfer Methods
File transfer is the most common method of migrating files to more stable storage.
Disk imaging is done when an image is required to access files, usually for emulation
of legacy file types.

File transfer methods include:
* Rsync
* Bagit
* Mass Optical Disc File Transfer with Nimbie (creates bags and other transfer metadata)
* Cloud Transfer - Rclone
* FTK Imager File Export
* IsoBuster File Export

Disk Imaging methods include:
* Kryoflux Imaging
* FTK Imager
* dd

### Choosing Transfer Method by Carrier

Digital Archives has a standard transfer method for most of the carrier types we
encounter. Generally, carriers will be transferred with the standard method unless
standard methods fail.


| Carrier | Transfer |
|:-------------------:|:--------------------------------:|
| External Hard Drive | File Transfer |
| Internal Hard Drive | File Transfer |
| Thumb Drive | File Transfer |
| SD Card | File Transfer |
| Optical Media | Mass Optical Disc Transfer |
| Floppy Disk | Floppy Disk Imaging - Kryoflux |
| Iomega Disk | Iomega Disk Imaging - FTK Imager |
| Google Drive | Cloud File Transfer - Rclone |

## Verifying Storage Space
Before beginning transfer make sure Digital Archives Diskstation (DADS) has enough
space to store the transfers.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
25 changes: 7 additions & 18 deletions transfers/transfers.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,11 @@ has_children: true
### &nbsp;
{: .no_toc .text-delta }

1. TOC
{:toc}
<!-- # Transfers Parent Page (digarch issue 78)
* A generic description of the purpose of transfers
* What can be transferred
* accessioned collections (which will then be processed)
* special projects (which will be discarded) -->

Born digital collection material can be acquired through file transfer or forensic imaging.
The workflows vary based on media types and file types encountered but the general steps are as follows:
Expand All @@ -21,17 +24,12 @@ The workflows vary based on media types and file types encountered but the gener
2. Acquiring basic metadata about the media and files.
3. Preparing the files for later archival processing.

**Before a media object can be imaged it first must be recorded SPEC.**

### File Transfers

During the file transfer process, some or all of the files are transferred from the original media using standard file copying tools.
Processing archivists may assist in determining which files to transfer.
The most common method of transfer uses a Bagit script via command line.

* Hard drives, USB drives, and other mass storage devices are transferred using a [BagIt script](file-transfers.html).
* CDs, DVDs and other optical media are transferred using a [Nimbie Robot](nimbie-transfers.html)

### Forensic Imaging

Forensic imaging is reserved for cases where imaging facilitates viewing or accessing material.
Expand All @@ -40,14 +38,5 @@ Imaging this material allows it to be viewed in an emulator.
Floppy disks and Iomega disks are usually imaged.
Forensic imaging may also be used as a fallback method when file transfers are unsuccessful.

* [Floppy disk images](floppy-disk-imaging.html) are created using the Kryoflux Disk System.
* [Optical media](optical-media-imaging.html) (CD-R/DVD-R) are imaged using the dd command.
* Hard drives, and removable media such as USB thumbdrives, and [Iomega disks](iomega-disk-imaging.html) are imaged with FTK Imager.

### Vendor Transfers

When the lab does not have the proper drive to access a type of media the media may be sent to a vendor for transfer.

### Network Transfers

In some cases born-digital material may be transferred over a network. Accessioning procedures for these types of transfers are still in progress.
### Transfer Pages
Pages in this section describe [preparing for transfer](preparing-for-transfer.md), [using write blockers](using-tableaus.md), transfer methods, and [completing transfer](completing-transfer.md).
35 changes: 0 additions & 35 deletions transfers/verify-inventory.md

This file was deleted.

1 change: 1 addition & 0 deletions using/using-lab-equipment.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ To schedule a time
* Select a date and time
* Change the calendar from your personal calendar to Digital Archives Lab

![event with Digital Archives Lab calendar selected](<using-lab-equipment/DAL-calendar.png>)
If there are any issues, please contact lab staff.

# Labeling
Expand Down
Binary file added using/using-lab-equipment/DAL-calendar.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit fe0d54e

Please sign in to comment.