Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ACQ-ID New Floppy and Iomega tranfer pages #92

Merged
merged 15 commits into from
Aug 8, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions _config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,3 +16,34 @@ heading_anchors: true

ga_tracking: G-4GBB05KF5M
ga_tracking_anonymize_ip: true

callouts_level: quiet # or loud
callouts:
# use for documentation still not fully complete, but partially launched
construction:
title: Under Construction
color: yellow
# use after launching new documentation for external staff, remove after some period time
new:
title: New
color: green
# use for legacy instruction that will eventually be removed, remove when documentation is removed
deprecated:
title: Deprecated
color: red
# use to callout requirements before starting process
important:
title: Important
color: yellow
# use for context to instructions
note:
title: Note
color: purple
# use for situations where no further work should be completed until situation is resolved
warning:
title: Warning
color: red
# use for final instruction or failure remediation
highlight:
color: blue

1 change: 1 addition & 0 deletions _data/working.csv
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
name,desc,use
makesips (SIPdir.sh),This bash script is used by Digital Archives staff to create consecutive numbers of submission information packages for material transferred from digital carriers., Navigate to collection directory in diskImages and run ```makesips``` in terminal. Follow script instructions to select collection number type and enter collection number and submission information package range.
movekflux (kryofluxmove.sh),This bash script is used by Digital Archives staff to move floppy disk images transferred using kryoflux to submission information packages.,Navigate to kryofluxOutput and run ```movekflux``` in terminal. Follow the script instructions to enter collection number and move floppy disk images.
package_images.py, This python script creates archival information packages based on a given acquisition id and moves related files from a transfer diretory into the associated packages in a destination directory. This script is primarily used in the imaging process for floppy and Iomega zip disks., The script takes three arguments: ```--acqid``` for acquisiton id in the syntax ACQ-####. ```--source``` for the path to directory holding images to be packaged. ```--dest``` for the path to destination directory for package.
report_ftk_extents.py,This python script transforms XML reports exported from FTK after Processing Archivists have completed bookmarking into JSON files for import into ASpace.,The script takes two arguments: ```-f``` or ```--file``` followed by the path to the XML report to be transformed and ```-o``` or ```--output``` followed by the path the destination directory for JSON output.
report_HDD_extents.py,This python script collects the extents of finding aid components created by a Processing Archivist when processing work is done at a workstation with a hard drive. The script then produces an extents JSON file for import into ASpace.,The script takes one argument: ```-d``` or ```--dir``` for the finding aid components directory on a hard drive.<br> Run the script following the syntax: ```python3 path/to/report_ftk_extents.py -d /path/to/collection/er/directory```
6 changes: 4 additions & 2 deletions transfers/Completing-transfer.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,10 @@ This page outlines the steps to take after transferring media.
(Refer to transfer instructions for the format you are transferring)

## Verifying all packages meet requirements
Under Construction
{: .label .label-yellow}

{: .construction }
packaging instructions to follow

* Run the packaging script

## Updating records in SPEC
Expand Down
45 changes: 33 additions & 12 deletions transfers/floppy-disk-imaging.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,23 +5,29 @@ nav_order: 5
parent: Transfers
---

# Floppy Disks
# Floppy Disks

**Before a media object can be imaged it first must be recorded SPEC.**
This page describes instructions for imaging floppy disks using the KryoFlux microcontroller and software. For more detailed installation and use information see the [Kryoflux documentation](https://nypl.github.io/digarch/tools/kryoflux) in our Tools section.

## Getting set up
### Before Imaging
{: .important }
Before any media can be imaged **it first must have an object record in SPEC**.
* Verify a tracking card exists on Trello for the associated acquisition being imaged
* Create a card if it does not already exist
* See our [Preparing for Transfer](https://nypl.github.io/digarch/transfers/preparing-for-transfer.html) page for detailed pre-imaging and transfer instructions.

### Connect the KryoFlux

* Ensure the floppy drive is correctly connected to the Kryoflux.
* Attach the KryoFlux USB cable from the forensic workstation to the Kryoflux board.
* See a green power light glow on the Kryoflux board.
* Attach the AC power cable to the power supply.
* See the green LED on the power supply light up.
* Listen for a faint click from the disk drive.

**See the [KryoFlux page in the Tools section for more](https://nypl.github.io/digarch/tools/kryoflux).**
* Listen for a faint click from the disk drive.


### Calibrate the KryoFlux

* Open the KryoFlux software from the desktop "kryoflux-ui.jar".
* Click on the drive menu and ensure that ‘Drive 0’ is selected.
* Select ‘calibrate’ from the drive menu, then ‘yes’ from the pop-up window.
Expand All @@ -30,7 +36,8 @@ parent: Transfers
* Calibrate only once when you begin a session, unless calibration fails.

### Resolve communication with the drive.
Follow these steps if the KryoFlux is unable to communicate with the drive.

Follow these steps if the KryoFlux is unable to communicate with the drive:
* Close the application.
* Disconnect the power source from the drive.
* Disconnect the USB cable from the KryoFlux board.
Expand All @@ -55,10 +62,12 @@ Follow these steps if the KryoFlux is unable to communicate with the drive.
* Select “Multiple” from the drop-down below the "Enter name ..." text field when you have determined the correct sector format.
* Select “KryoFlux stream image” and, while holding down the control key, select the correct sector format from the dropdown.
* Select ‘ok’ to close the popup.
* Click the start button to begin the imaging process.
The KryoFlux will now create a folder with the name of the id of the media that contains the stream image files and a sector image (again with the name of the media plus an “.001” file extension). Both the stream directory and the sector image file will be written to the kryofluxOutput directory on Staging.
* Click the start button to begin the imaging process.

The KryoFlux will now create a folder with the name of the id of the media that contains the stream image files and a sector image (again with the name of the media plus an “.001” file extension). Both the stream directory and the sector image file will be written to the kryofluxOutput directory on Staging.

### Tracking

* Navigate to the [Tracking](https://drive.google.com/drive/folders/1tv4nr9Nq_c8wkqPpz_eQX7NKRRrlEisp?usp=share_link){:target="_blank"} folder in Google Drive.
* Find the spreadsheet for the acquisition that you will be working with.
* Copy [Tracking_TEMPLATE](https://docs.google.com/spreadsheets/d/1TwWMsrCf2hf5LzdtA6EG-2wcgFW_Uz750x-PZtFop90/edit?usp=sharing) to create a spreadsheet if one doesn't exist.
Expand Down Expand Up @@ -106,10 +115,23 @@ Follow these steps if the KryoFlux is unable to communicate with the drive.
| -- |
| Y |

### Package Images with package_images.py
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Align Package Images instructions for Floppy and Iomega


The python script package_images.py identifies digital carrier images in an input directory, creates a archival information package, and moves image files into the package. This section describes packaging with package_images.py for floppy and Iomega disks. For detailed information on installation and use see the [package_image.py](https://nypl.github.io/digarch/tools/working-scripts.html#package_imagespy) listing in the Working Scripts section of our documentation.

Use package_images.py with the followings steps:
* Open Terminal
* Enter the following command with the appropriate paths: ```python3 package_images.py –-acqid (M#### or ACQ_####) -–source path/to/images -–dest path/to/diskImages```
* Navigate to destination directory to verify packages have been created and image files have been moved
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* Navigate to destination directory to verify packages have been created and image files have been moved
* Navigate to destination directory to verify packages were created and image files were moved


### Completing the imaging process

* Put the media back in the collection’s box and move or remove the pink “To image” flag as necessary if you are working on a large collection.
* Move the media to the “Small collections complete” box if you are working on a small collection without a box.
* Return media to the collection’s box or move media to the "Small Collections Transferred" box if you are working on a small collection without a box.
* Update object record migration status in SPEC. See [Updating records in SPEC](https://nypl.github.io/digarch/transfers/Completing-transfer.html#updating-records-in-spec) section on Completing a Transfer page for detailed instructions


{: .deprecated}
The following sections describe imaging and packaging processes no longer regularly used. Consult these sections when working with legacy collections requires understanding previously done work.

### Using Digital Archives scripts
Make sure the setup instructions for Digital Archives [scripts](https://github.com/NYPL/digarch_scripts){:target="_blank"} have been completed before running the scripts in the next section.
Expand Down Expand Up @@ -175,7 +197,6 @@ Or
* /objects

### Package images

On Mac:

* Open Terminal.
Expand Down
178 changes: 90 additions & 88 deletions transfers/iomega-disk-imaging.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,78 +6,14 @@ parent: Transfers
---

## Iomega Disks
This page covers Iomega disk imaging with the FTK Imager software. Iomega disks are usually imaged with FTK Imager. Use FTK Imager when creating a disk image of a hard drive or removable media.

**Before a media object can be imaged it first must be recorded SPEC.**

This section covers imaging with FTK Imager. Iomega disks are usually imaged with FTK Imager. Use FTK Imager when creating a disk image of a hard drive or removable media.

### Using Digital Archives scripts
Make sure the setup instructions for Digital Archives [scripts](https://github.com/NYPL/digarch_scripts){:target="_blank"} have been completed before running the scripts in the next section.

### Create destination directories

**These instructions show you how to create destination directories for a number of consecutive disks. Consider using a one-line command to create directories if the disks you are packaging do not have consecutive MediaID numbers.**

On Mac:

* Open Terminal.

* Navigate to DigArchDiskStation.

* Change into diskImages directory.
```$ cd /Volumes/DigArchDiskStation/Staging/ingest/diskImages```

* Create a directory for your collection if it does not exist.
```$ mkdir ACQ_acqID```

* Change into your collection directory.
```$ cd ACQ_acqID```

* Run [makesips script](https://nypl.github.io/digarch/tools/working-scripts.html#makesips-sipdirsh){:target="_blank"} to create a consecutive number of submission information packages for material from digital media.

Or

* Change to diskImages directory.

```$ cd /Volumes/DigArchDiskStation/Staging/ingest/diskImages```
* Enter ```mkdir``` command.
```mkdir -p ACQ_acqID/ACQ_acqID_specObjectID/{metadata,objects}```

On Windows via WSL:

* Open WSL.
* If you do not see the Y:\ drive in /mnt or /mnt/y appears to be empty then it must be re-mounted by:
* Changing to the top level directory by entering ```cd ../```
* Entering the command ```sudo mount drvfs Y: /mnt/y```
* Change into diskImages directory.
```$ cd /mnt/y/Staging/ingest/diskImages```

* Create a directory for the acquisition if it does not exist.
```$ mkdir ACQ_acqID```

* Change into the acquisition directory.
```$ cd ACQ_acqID```

* Run [makesips script](https://nypl.github.io/digarch/tools/working-scripts.html#makesips-sipdirsh){:target="_blank"} to create a consecutive number of submission information packages for material from digital media.

Or

* Change to diskImages directory.

```$ cd /mnt/y/Staging/ingest/diskImages```
* Enter ```mkdir``` command.
```mkdir -p ACQ_acqID/ACQ_acqID_specObjectID/{metadata,objects}```

#### Directory structure

* /M2319-0021
* /metadata
*
* /objects


<!-- ![](media/media/image29.png){width="6.069444444444445in"
height="2.3472222222222223in"} -->
### Before Imaging
{: .important }
Before any media can be imaged **it first must have an object record in SPEC**.
* Verify a tracking card exists on Trello for the associated acquisition being imaged
* Create a card if it does not already exist
* See our [Preparing for Transfer](https://nypl.github.io/digarch/transfers/preparing-for-transfer.html) page for detailed pre-imaging and transfer instructions.

### Image disks

Expand Down Expand Up @@ -185,22 +121,88 @@ height="4.55in"} -->
| -- |
| Y |

### Package Images with package_images.py
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Align Package Images instructions for Floppy and Iomega


The python script package_images.py identifies digital carrier images in an input directory, creates a archival information package, and moves image files into the package. This section describes packaging with package_images.py for floppy and Iomega disks. For detailed information on installation and use see the [package_image.py](https://nypl.github.io/digarch/tools/working-scripts.html#package_imagespy) listing in the Working Scripts section of our documentation.

Use package_images.py with the followings steps:
* Open Terminal
* Enter the following command with the appropriate paths: ```python3 package_images.py –-acqid (M#### or ACQ_####) -–source path/to/images -–dest path/to/diskImages```
* Navigate to destination directory to verify packages have been created and image files have been moved

### Completing the imaging process

* Put the media back in the collection’s box and move or remove the pink “To image” flag as necessary if you are working on a large collection.
* Move the media to the “Small collections complete” box if you are working on a small collection without a box.

### Metadata
Deprecated
{: .label .label-red }
* Open Cygwin and enter the following commands:
* Connect to ARCHV Mac.
```$ ssh archv```
* Change to the diskImages directory and change directory to the objects folder for the media object.
`$ diskimages`
`$cd ACQ_acqID/ACQ_acqID_specObjectID/objects`
* Run disktype on the disk image to get the file system metadata.
`$ disktype ACQ_acqID_specObjectID.E01`
* Copy the file system information data into the media objects CMS record.
* Run program to move metadata files to metadata directory.
`$ movemetadata`
* Return media to the collection’s box or move media to the "Small Collections Transferred" box if you are working on a small collection without a box.
* Update object record migration status in SPEC. See [Updating records in SPEC](https://nypl.github.io/digarch/transfers/Completing-transfer.html#updating-records-in-spec) section on Completing a Transfer page for detailed instructions


{: .deprecated }
The following sections describe imaging and packaging processes no longer regularly used. Consult these sections when working with legacy collections requires understanding previously done work.

### Using Digital Archives scripts
Make sure the setup instructions for Digital Archives [scripts](https://github.com/NYPL/digarch_scripts){:target="_blank"} have been completed before running the scripts in the next section.

### Create destination directories

**These instructions show you how to create destination directories for a number of consecutive disks. Consider using a one-line command to create directories if the disks you are packaging do not have consecutive MediaID numbers.**

On Mac:

* Open Terminal.

* Navigate to DigArchDiskStation.

* Change into diskImages directory.
```$ cd /Volumes/DigArchDiskStation/Staging/ingest/diskImages```

* Create a directory for your collection if it does not exist.
```$ mkdir ACQ_acqID```

* Change into your collection directory.
```$ cd ACQ_acqID```

* Run [makesips script](https://nypl.github.io/digarch/tools/working-scripts.html#makesips-sipdirsh){:target="_blank"} to create a consecutive number of submission information packages for material from digital media.

Or

* Change to diskImages directory.

```$ cd /Volumes/DigArchDiskStation/Staging/ingest/diskImages```
* Enter ```mkdir``` command.
```mkdir -p ACQ_acqID/ACQ_acqID_specObjectID/{metadata,objects}```

On Windows via WSL:

* Open WSL.
* If you do not see the Y:\ drive in /mnt or /mnt/y appears to be empty then it must be re-mounted by:
* Changing to the top level directory by entering ```cd ../```
* Entering the command ```sudo mount drvfs Y: /mnt/y```
* Change into diskImages directory.
```$ cd /mnt/y/Staging/ingest/diskImages```

* Create a directory for the acquisition if it does not exist.
```$ mkdir ACQ_acqID```

* Change into the acquisition directory.
```$ cd ACQ_acqID```

* Run [makesips script](https://nypl.github.io/digarch/tools/working-scripts.html#makesips-sipdirsh){:target="_blank"} to create a consecutive number of submission information packages for material from digital media.

Or

* Change to diskImages directory.

```$ cd /mnt/y/Staging/ingest/diskImages```
* Enter ```mkdir``` command.
```mkdir -p ACQ_acqID/ACQ_acqID_specObjectID/{metadata,objects}```

#### Directory structure

* /M2319-0021
* /metadata
*
* /objects


<!-- ![](media/media/image29.png){width="6.069444444444445in"
height="2.3472222222222223in"} -->
21 changes: 14 additions & 7 deletions transfers/preparing-for-transfer.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,9 @@ This page outlines the steps to take before transferring media objects.
## Locating Carriers
* Carriers are located on shelving by the entrance door
* Carriers to be transferred are likely on `02.**` shelves
*
**Before a media object can be imaged it first must be recorded SPEC.**

{: .important }
Before a media object can be imaged it first must be recorded SPEC.

## Verifying SPEC records
An acquisition record should exist in SPEC and SPEC object records should exist
Expand All @@ -43,7 +44,8 @@ for each carrier being transferred.

* Select search

**Contact Collection Management if you cannot find the acquisition**
{: .warning }
Contact Collection Management if you cannot find the acquisition

### Verifying object records exist in SPEC
* Select the acquisition from the list
Expand Down Expand Up @@ -86,24 +88,29 @@ src="preparing-for-transfer/SPEC-object-records.png" />
* Confirm the number of items listed for Digital media matches the object records
* Report mismatch of digital media summary and object records to Systems and Operations

**Contact Collection Management if the number of carriers and records don't match**
{: .highlight }
Contact Collection Management if the number of carriers and records don't match

### Verifying additional carriers from Archival Processing
When archivists survey a collection they may find additional carriers in boxes.
Archivists should create object records for any carriers they find and notify
Digital Archives of their discovery.
* [Verify](#verifying-object-records-exist-in-spec) objects records exist in SPEC

**Contact the processing archivist if the number of carriers and records don't match**
{: .highlight }
Contact the processing archivist if the number of carriers and records don't match

### Handling Commercial Software
**These instructions are for Digital Preservation staff (including fellows) only.**

{: .note }
These instructions are for Digital Preservation staff (including fellows) only.
* Label commercial software with a simple acquisition name.
* `Crouch` for Stanley Crouch papers.
* Image commercial software only when specifically instructed to do so.
* Commercial software media don't receive objects records.

**When objects records were created for commercial software create an issue.**
{: .highlight }
When objects records were created for commercial software create an issue.

* Select issues from the object record menu in SPEC
<details>
Expand Down