Skip to content

Commit

Permalink
Merge pull request #89 from NYPL/revert-88-acq-bash-fix
Browse files Browse the repository at this point in the history
Revert "Acq bash fix"
  • Loading branch information
arhonemus committed Feb 16, 2024
2 parents 0a0b7db + d986e7e commit 6c47d3f
Show file tree
Hide file tree
Showing 27 changed files with 398 additions and 424 deletions.
6 changes: 3 additions & 3 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,12 @@ _site/
.sass-cache/
.jekyll-cache/
.jekyll-metadata

_config.yml
# Ruby
.bundle/
.ruby-gemset
.ruby-version

*.gem
Gemfile.lock

# Mac/Windows System Files
Expand All @@ -19,4 +19,4 @@ Icon

Thumbs.db
_config.yaml

_config.yml
2 changes: 0 additions & 2 deletions Gemfile
Original file line number Diff line number Diff line change
Expand Up @@ -28,5 +28,3 @@ end
# Performance-booster for watching directories on Windows
gem "wdm", "~> 0.1.1", :platforms => [:mingw, :x64_mingw, :mswin]

#adding plugin for redirects:
gem 'jekyll-redirect-from'
3 changes: 0 additions & 3 deletions _config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,3 @@ heading_anchors: true

ga_tracking: G-4GBB05KF5M
ga_tracking_anonymize_ip: true

plugins:
- jekyll-redirect-from
40 changes: 40 additions & 0 deletions _data/program.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
name,desc,inst,conf,use,docs
7-zip,7-zip is used in the lab to compress evidence and cases generated in Forensic Toolkit. It is also occasionally used to unpack collection material that is not supported by FTK.,[Download 7-zip](https://www.7-zip.org/),
apple3rtr,Apple /// Ready-to-Run (apple3rtr) is a software bundle used for MAME Apple emulation. It has been used by Digital Archives staff to view Apple II files.,For detailed installation instructions visit the [apple3rtr github repo](https://github.com/datajerk/apple3rtr).,,,https://github.com/datajerk/apple3rtr
AWS, Amazon Web Services (AWS) is a cloud computing platform offered by Amazon. AWS S3 features are used to store copies of born-digital material both before and after the collection has been processed by archivists.,,,,https://docs.aws.amazon.com/
Bagger, Bagger is an applciation created by the U.S. Library of Congress to produce files packages following the Bagit specification. It has been used in the Digital Archives lab for transferring born-digital material.,,,,https://github.com/LibraryOfCongress/bagger
Bagit.py, Bagit.py is a command line tool used produce files packages following the Bagit specification, Check python version by entering ```python --version``` or ```python3 --version``` into terminal. Then install using pip by entering ```pip install bagit``` or ```pip3 install bagit```. Use the link below the main download on the VirtualBox download site to install the Extension Pack on your host.,,,https://github.com/LibraryOfCongress/bagit-python
CiderPress, CiderPress is a Windows utility for managing Apple II disk images. It has been used in the Digital Archives Lab for viewing Apple file formats.,,,,https://a2ciderpress.com
dbpoweramp, dbpoweramp is a program used in conjunction with [Iromlab](#Iromlab) for transferring born-digital material from optical media.,
dd, dd is a command line utility used in the Digital Archives Lab to create disk images of optical media. In most cases transfers are made of optical media instead of disk images.,,,,https://linux.die.net/man/1/dd
Disk Utility, Disk Utility is a MacOS utility for disk volume management. Disk Utility used by Digital Archives staff for mounting/unmounting remvable media.,,,enter ```list disk``` or ```list volume``` to confirm media is mounted.,https://support.apple.com/guide/disk-utility/welcome/mac
diskpart, diskpart is Windows command line utility for disk partitioning. diskpart is used by Digital Archives staff for confimring removable media are properly mounted.,,,,https://learn.microsoft.com/en-us/windows-server/administration/windows-commands/diskpart
disktype, disktype is a command line tool used by Digital Archives staff for extracting filesystem metadata like sector format for floppy disks, For Mac OS:<br> In terminal use homebrew to install by entering ```brew install disktype```,, In terminal navigate to the directory containing floppy disk image(s) and enter ```disktype a/disk/image```.,https://disktype.sourceforge.net/doc/
DosBox, DosBox is an open-source emulator for the MS-DOS operating system. DosBox has been used by Digital Archives staff to view DOS executables and word processing files.,,,,https://www.dosbox.com/DOSBoxManual.html
DROID, DROID is file format identification tool developed by the UK National Archives.,,,,https://cdn.nationalarchives.gov.uk/documents/information-management/droid-user-guide.pdf
Emailchemy, Emailchemy has been used in the Digital Archives lab for migrating email fomats.,,,,https://www.weirdkid.com/products/emailchemy/doc/Emailchemy_User_Manual.pdf
ePADD, ePADD is an open-source program developed by Stanford University's Special Collections and University Archives. The software ha been used for appraisal and arrangement of email.,,,,https://github.com/ePADD/epadd/releases
Forensic Toolkit, Forensic Toolkit is a computer forensics software used in the Digital Archives Lab to appraise and arrange born-digital material.
FTK Imager, FTK Imager is a disk imaging program associated with Forensic Toolkit. The software has been used in the Digital Archives lab to image and extract files from digital carriers.,
gtar, Gtar is a command line program used on Digital Archives MacOS workstations to compress Kryoflux stream files., For MacOS: Install via Homebrew by entering ```brew install gnu-tar``` in terminal.,,,https://www.gnu.org/software/tar/manual/tar.html
HFS Explorer, HFS Explorer is a program usd to view HFS formatted disks on Windows operating systems.,,,,https://catacombae.org/hfsexplorer/
Iromlab, Iromlab is software for automated imaging of optical media using [Nimbie](#tools/nimbie) disc robots.,,,,https://catacombae.org/hfsexplorer/
IsoBuster, IsoBuster is data recovery software that has been used in the Digital Archives lab to image and extract files from optical media.,
Java, Java is a software package which has been a dependency for some Digital Archives tools like DROID and Bagger.,,, Check for the version of Java installed with commands ```which java``` or ```java -version``` in terminal on Mac or WSL on Windows.,
MediaInfo, MediaInfo is an open-source program for extracting technical information from media files. The program has been used in the Digital Archives Lab to extract metadata from born-digital video.,
Mediaconch, MediaConch is an open source software project developed by MediaArea and utilizing MediaInfo. It is used by Digital Archives staff to develop programatic policies for checking born-digital audiovisual material against preservation standards.,
Mini Vmac, Mini vMac is an emulator for running early Macintosh software.
openssl, OpenSSL is a software library used to ssh.,,,,https://www.openssl.org/docs
Oracle VM VirtualBox, Virtualbox is open soure virtualization software. It has been used by Digital Archives staff to run teh BitCurator environment as a virtual machine., Download and install the [current VirtualBox release and Extension Pack](https://www.virtualbox.org/wiki/Downloads).
pip, Pip is package management software used for installing and upgrading Python modules,,,Pip should be installed with python3. Call pip by entering ```pip``` or ```pip3``` in terminal or command line.,
Python3, Python3 is the latest version of the Python programming language, For Windows: Check for python versions by entering ```python --version``` on command line. Installers are avilable at python.org.<br> For MacOS: Check for python versions by entering ```python --version``` in terminal. Install your selected version of python with homebrew by entering ```brew install [email protected]```.,
QuickTime Player, QuickTime is used in teh Digital Archives Lab to appraise Mcintosh formatted born-digital video.,
QuickView Plus, QuickView Plus has been used in the Digital Archives Lab to view a range of legacy formats like word processing and photo formats.,
rclone,rclone is a command line tool for managing and moving files between remote and local storage locations,For detailed installation and configuration instructions [see our dedicated rclone page](rclone.html).,,,https://rclone.org/docs/
Sha1deep, Sha1deep is a command line tool for computing and cryptographic hashes. (M5deep | sha1deep | sha256deep are a set of programs to compute MD5 | SHA-1 | SHA-256 message digests on files.) Sha1deep is used by Digital Archives Staff to identify duplicate material across digital media in a collection by comparing checksums., For MacOS: Install via Homebrew by entering ```brew install m5deep```,,,https://linux.die.net/man/1/sha1deep
Sheepshaver, Sheepshaver is an Apple Macintosh emulator. It has been used in the Digital Archives Lab to view legacy Mac files.,
Siegfried, Siegfried is a command line file format identification tool. Siegfried is used in the Digital Archives Lab to identify and validate file formats. Siegfried output also includes checksums., For Windows: Download [the latest Siegfried binary](https://github.com/richardlehane/siegfried/releases/download/v1.10.1/siegfried_1-10-1_win64.zip).<br> For MacOS: Install via homebrew by entering ```brew install richardlehane/digipres/siegfried``` into terminal,Current installations of siegfried use the config file to store preferred command options. Enter ```sf -setconf -csv -hash sha1 -z``` in terminal to set the preferred command options., Enter ```sf PATH > MediaID``` in terminal to create a report of file formats. Enter ```sf -help``` for a complete list of sf commands.,https://www.itforarchivists.com/siegfried/
tree, tree is command line program for recursive directory listing and has been used in the Digital Archives Lab to identify contents of media including number of folders and files, For MacOS: Install via homebrew by entering ```brew install tree``` into terminal.,
VLC, VLC is an open source media player used by Digital Archives staff to view and appraise born-digital audio and video., Download [the latest release of VLC Media Player](https://www.videolan.org/vlc/).,
WSL, Windows Subsystem for Linux (WSL) is a Windows feature used to run Linux environments on Windows operating systems.
WinUAE, WinUAE is an Amiga emulator and has been used in the Digital Archives Lab to view Amiga files.,
4 changes: 2 additions & 2 deletions _data/working.csv
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
name,desc,use
movekflux (kryofluxmove.sh),This bash script is used by Digital Archives staff to move floppy disk images transferred using kryoflux to submission information packages.,Navigate to kryofluxOutput and run ```movekflux``` in terminal. Follow the script instructions to enter collection number and move floppy disk images.
makesips (SIPdir.sh),This bash script is used by Digital Archives staff to create consecutive numbers of submission information packages for material transferred from digital carriers., Navigate to collection directory in diskImages and run ```makesips``` in terminal. Follow script instructions to select collection number type and enter collection number and submission information package range.
kryofluxmove.sh,This bash script is used by Digital Archives staff to move floppy disk images transferred using kryoflux to folders with legacy CMS media IDs of the associated digital carrier.,
SIPdir.sh,This bash script is used by Digital Archives staff to create consecutive numbers of submission information packages for material transferred from digital carriers.,
report_ftk_extents.py,This python script transforms XML reports exported from FTK after Processing Archivists have completed bookmarking into JSON files for import into ASpace.,The script takes two arguments: ```-f``` or ```--file``` followed by the path to the XML report to be transformed and ```-o``` or ```--output``` followed by the path the destination directory for JSON output.
report_HDD_extents.py,This python script collects the extents of finding aid components created by a Processing Archivist when processing work is done at a workstation with a hard drive. The script then produces an extents JSON file for import into ASpace.,The script takes one argument: ```-d``` or ```--dir``` for the finding aid components directory on a hard drive.<br> Run the script following the syntax: ```python3 path/to/report_ftk_extents.py -d /path/to/collection/er/directory```
6 changes: 5 additions & 1 deletion accessioning/accessioning.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,4 +5,8 @@ nav_order: 2
has_children: true
---
# Accessioning
Accessioning consists of taking legal, administrative, and physical custody of digital material to be transferred to the lab. Accessioning born-digital material includes verifying that the acquisition has been properly documented, validating the material’s completeness and fixity, and the creating an inventory in CMS to support collection management and project planning.
Accessioning consists of taking legal, administrative, and physical custody of digital material to be transferred to the lab. Accessioning born-digital material includes verifying that the acquisition has been properly documented, validating the material’s completeness and fixity, and then creating an object record in SPEC to support collection management and project planning.

Accessioning of digital material is usually done by Collection Management.

Processing archivists create SPEC object records for additional carriers found during processing.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
3 changes: 0 additions & 3 deletions accessioning/inventorying-digital-media-accession.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,8 @@
---
title: Inventorying Digital Carriers (Accessioning)
title: Inventorying Digital Carriers (Accessioning)
layout: default
nav_order: 1
parent: Accessioning
redirect_from:
- https://nypl.github.io/digarch/accessioning/digitalmediaseparation.html
---

# Digital Carrier Separation (Accessioning)
Expand Down
33 changes: 21 additions & 12 deletions staging/Archivist-Workstation-Processing.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,8 @@ Archivists processing electronic records without using FTK or ePADD will receive

The goal of the arrangement and description phase is to approximate the
process in which an archivist works with a physical collection. By using
their workstation the archivist will be able to appraise records as they are contained
on their original media, create a set of intellectual components
their workstation the archivist will be able to appraise a copy of records with the same
structure as contained on their original media, create a set of intellectual components
(arrangement), summarize the logical extents (size) and date ranges of
the components, and enter them into ArchivesSpace.

Expand All @@ -38,13 +38,17 @@ the components, and enter them into ArchivesSpace.
* Double-click the drive icon to display the drive folders in Windows Explorer.
* The folders will have the following structure:
* \M12345_workingfiles
* \M12345-0001
* \metadata
* \objects
* \ACQ_12345_54321
* \metadata
* \objects
* \data (optional)

* Note the metadata folder may contain metadata created during the transfer of the files.
* Note a file directory listing of the files in .csv format, if present.
* Navigate to the objects folder to view the files to be arranged.

OR

* Navigate to the data folder in the objects folder to view the files to be arranged.
<!--* Note manifest.txt files contain a directory listing. -->

Expand All @@ -58,13 +62,18 @@ the components, and enter them into ArchivesSpace.

* Create a new folder on the drive you received at the same folder level as \M12345_workingfiles.
* Use the naming convention ```CollectionID_FAcomponents``` to create a collection folder.
* Name the folder /M12345_FAcomponents, for example.
* Create a folder for each FA Component using ```CollectionID_ER_1```, ```CollectionID_ER_2```.
* Create an objects folder within each ER_# folder. ```CollectionID_ER_1\objects```
* Move the files into the new FA Component objects folders as your process the files.
* Calculate the size of each FA Component by right clicking on the FA Component folder.
* Select Properties from the drop down menu.
* Note the file count and byte extent, when you are done processing.
* Name the folder \M12345_FAcomponents, for example.
* Create a folder for each heading represented in the electronic records. ```Correspondence```
* Create a folder for each FA Component using ```ER # Title, dates```.
* Create an objects folder within each ER_# folder. ```ER 1 Files, 2012\objects```
* Move the files into the new FA Component objects folders as your process the files.

### Finishing up
* Email [Digital Archives](mailto:[email protected]) when you are finished processing.
* Digital Archives will arrange to pick up the hard drive.
* Digital Archives will email you a JSON file for import into ASpace.
* Import the JSON file received from Digital Archives into ASpace.
* Email [Digital Archives](mailto:[email protected]) when your arrangement is approved.

### Entering your collection in ArchivesSpace
After arrangement is complete Processing Archivists notify Digital Archives and deliver the external hard drive containing FA Components to Digital Archives for review. Digital Archives staff then create a JSON file containing extents for import into ASpace. For detailed instruction on importing the extents JSON to ASpace review the [Importing FTK ERs in ArchiveSpace](https://docs.google.com/document/d/1BVMaDOzdcPFIht5yN5V16zmsnjMTwG_3e73HPTdVzSg/edit) guide.
Expand Down
13 changes: 12 additions & 1 deletion staging/FTK-Export.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,18 @@ When a finding aid has been approved and the Bookmarks have been reviewed the co
* Open the case for the collection and navigate to the Bookmark
tab.
* Note how many finding aid components you have in the Bookmarks.
* Windows subsystem for Linux and enter the following commands to build your folders:
* Open the WSL terminal and navigate to the mount point directory by entering ```cd /mnt```

* Any mounted drives should be accessible from the mount point directory. Drives you'll see in /mnt include:
* d - Sata Drive Bay
* f - Storage for FTK
* h - Open FTK cases
* i - Codemeter access key
* y - DigArchDiskStation

* If the Y:\ drive is not visible in /mnt or /mnt/y appears to be empty then re-mount:
* Change to the top level directory by entering ```cd /```
* Enter the command ```sudo mount -t drvfs Y: /mnt/y```

```$ cd /mnt/y/Staging/faComponents```

Expand Down
2 changes: 1 addition & 1 deletion staging/Loading-Archivist-Workstation.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ Archivists processing electronic records without using FTK or ePADD will receive

## Confirming Archivist Workstation Processing

* Open CMS and navigate to the collection's media log.
* Open SPEC and navigate to the collection's object records.
* Check that all the media in the collection has been transferred as files.
* Consult with Digital Archives staff if there are disk images present or transfer is incomplete.

Expand Down
Loading

0 comments on commit 6c47d3f

Please sign in to comment.