From 17bbe20ac6b48e81a41e3d239c730b591deb1996 Mon Sep 17 00:00:00 2001 From: dreyjo <59994482+dreyjo@users.noreply.github.com> Date: Wed, 28 Jun 2023 17:37:24 -0400 Subject: [PATCH 01/17] added proposal drafting markdown --- tools/software_draft.md | 151 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 151 insertions(+) create mode 100644 tools/software_draft.md diff --git a/tools/software_draft.md b/tools/software_draft.md new file mode 100644 index 0000000..d6c67cc --- /dev/null +++ b/tools/software_draft.md @@ -0,0 +1,151 @@ +--- +title: Software (Draft) +layout: default +nav_order: 5 +parent: Tools +has_children: False +--- + +# Software +{: .no_toc } + +Digital Archives utilizes several software throughout our workflows. This page acts as a departmental glossary for these software and includes installation, detailed use instructions and links to official documentation whenever possible. Shorter contextual instructions for these software can be found on the associated pages, consider this page our definitive guides. + +## Table of contents +{: .no_toc .text-delta } + +1. TOC +{:toc} + +## Programs: + +Programs are defined as any non-internally developed out-of-the-box software used in Digital Archives workflows. + +### 7-zip +### apple3rtr +### AWS +### Bagger +### Bagit.py +### BitCurator +### Brunnhilde with disktype +### clamscan +### CiderPress +### dBpoweramp +### dd +### Disk Utility +### DiskPart +### disktype +### DosBox +### DROID +### Emailchemy +### ePADD +### FACTools +### Filemaker Pro 17 +### Forensic Toolkit +### FTK Imager +### gtar +### HFS Explorer +### Iromlab +### IsoBuster +### Java +### MediaInfo +### miniVmac +### openssl +### Oracle VM VirtualBox +### pip +### Python3 +### QuickTime Player +### QuickView Plus +### rclone: + +[*Rclone*](https://rclone.org/) is a command line tool for managing files on cloud storage, and is one tool used by Digital Archives for moving files between Digital Archives workstations, Google Shared Drives for Divisions, and long term storage. This page lists instructions for installing and configuring rclone. + +#### Installing rclone: + +For detailed installation instruction tailored to your operating system see the [rclone documentation](https://rclone.org/install/). + +For MacOS: +Use Homebrew to install rclone via the terminal by entering: +``` +brew install rclone +``` + +For Windows: +[Download and run Windows installation file.](https://rclone.org/downloads/) +#### Configuring rclone: + +Once installed, you must configure Rclone which involves setting the remote locations you will be accessing. Detailed configuration instruction can be found in the [rclone configuration docs](https://rclone.org/docs/) separated by cloud storage systems. This section details configuration instructions for MacOS users accessing Google Drive. + +* In terminal, type in ```rclone config``` + +* Select “n” for “New Remote” + +* Enter the name for your remote when Rclone prompts you to, try to keep the name as short as possible while still being descriptive and specific. + +* rclone will then ask you to select from a list of storage platforms. Enter the number that corresponds on the list to Google Drive (Note: Google Cloud Storage is different from Google Drive!) + +* rclone asks for a Google Application Client Id, this is optional and you can select enter to choose default. + +* rclone asks for an OAuth Client Secret Id, this is optional and you can enter to select enter to choose default. + * To set your own application and OAuth client IDs see instructions on [creating a google application client id](https://rclone.org/drive/#making-your-own-client-id) in the rclone docs. + +* rclone will provide a list of scopes of access ranging from full access to all files to read-only access to file metadata. For our purposes enter the number corresponding to full access to all files. + +* Rclone asks for a root folder id, this is optional and you can enter to select enter to choose default. + +* Rclone asks for service account credentials, this is optional and you can enter to select enter to choose default. + +* Rclone asks you to edit advanced configuration, select “n” for No. Follow up prompt asks if you would like to use auto configuration, enter “y” for Yes. + +* A browser window should then open asking you to log into your google account and allow access to Rclone. Move through the prompts in browser. Once you see the success window, return to the terminal + +* Rclone asks if you would like to configure this remote as a shared drive. Select Y for yes. This opens a list of the shared drives associated with your account. + +* Enter the number associated with the shared drive you will be uploading requested items to. + +* Rclone then asks if the configuration you’ve been doing is okay. Enter y for Yes and configuration is complete! + +To see all currently configured remotes (rclone term for any cloud storage locations) enter ```rclone listremotes``` into terminal. + +#### Using rclone: + +For detailed installation instruction tailored to your operating system see the [rclone documentation](https://rclone.org/install/). + +For MacOS: +Use Homebrew to install rclone via the terminal by entering: +``` +brew install rclone +``` + +For Windows: +[Download and run Windows installation file.](https://rclone.org/downloads/) + +### rsync +### Sha1deep +### Sheepshaver +### Siegfried +### tree +### VLC +### Windows Subsystem for Linux (WSL) +### WinUAE + +## Working Scripts: + +Working Scripts are defined as scripts written internally and maintained by Digital Archives staff for use in our workflows. + +### CMS metadata import +### iterative scripts +### kryofluxmove.sh +### makesips script +### moveimages script +### report ftk extents script +### report HDD extents script + +## Deprecated and Legacy Scripts: + +Deprecated and Legacy Scripts are any scripts which may have been previously used in workflows but are no longer in use, were planned but not completed or replaced by a Working Script. + +### ft.sh +### metadata.sh +### movemetadata script +### movephotograph script From c7fd9405330de626477df1d758d2d66b5f958835 Mon Sep 17 00:00:00 2001 From: dreyjo <59994482+dreyjo@users.noreply.github.com> Date: Thu, 6 Jul 2023 17:06:12 -0400 Subject: [PATCH 02/17] starting to add entries for software --- tools/software_draft.md | 316 +++++++++++++++++++++++++++++++++------- 1 file changed, 265 insertions(+), 51 deletions(-) diff --git a/tools/software_draft.md b/tools/software_draft.md index d6c67cc..d32152a 100644 --- a/tools/software_draft.md +++ b/tools/software_draft.md @@ -21,41 +21,225 @@ Digital Archives utilizes several software throughout our workflows. This page a Programs are defined as any non-internally developed out-of-the-box software used in Digital Archives workflows. -### 7-zip -### apple3rtr -### AWS -### Bagger -### Bagit.py -### BitCurator -### Brunnhilde with disktype -### clamscan -### CiderPress -### dBpoweramp -### dd -### Disk Utility -### DiskPart -### disktype -### DosBox -### DROID -### Emailchemy -### ePADD -### FACTools -### Filemaker Pro 17 -### Forensic Toolkit -### FTK Imager -### gtar -### HFS Explorer -### Iromlab -### IsoBuster -### Java -### MediaInfo -### miniVmac -### openssl -### Oracle VM VirtualBox -### pip -### Python3 -### QuickTime Player -### QuickView Plus +### 7-zip: + +7-zip is used in the lab to compress evidence and cases generated in Forensic Toolkit. It is also occasionally used to unpack collection material that is not supported by FTK. + +#### Installing 7-zip: + +[Download 7-zip](https://www.7-zip.org/){:target="_blank"} + +#### Configuring: +#### Using: + +### apple3rtr: + +Apple /// Ready-to-Run (apple3rtr) is a software bundle used for MAME Apple emulation. It has been used by Digital Archives staff to view Apple II files. + +#### Installing: +For detailed installation instructions vist the (apple3rtr github repo)[https://github.com/datajerk/apple3rtr]. + +#### Configuring: +#### Using: + +### AWS: + +#### Installing: +#### Configuring: +#### Using: + +### Bagger: + +#### Installing: +#### Configuring: +#### Using: + +### Bagit.py: + +#### Installing: +#### Configuring: +#### Using: + +### BitCurator: + +#### Installing: +#### Configuring: +#### Using: + +### Brunnhilde with disktype: + +#### Installing: +#### Configuring: +#### Using: + +### clamscan: + +#### Installing: +#### Configuring: +#### Using: + +### CiderPress: + +#### Installing: +#### Configuring: +#### Using: + +### dBpoweramp: + +#### Installing: +#### Configuring: +#### Using: + +### dd: + +#### Installing: +#### Configuring: +#### Using: + +### Disk Utility: + +#### Installing: +#### Configuring: +#### Using: + +### DiskPart: + +#### Installing: +#### Configuring: +#### Using: + +### disktype: + +#### Installing: +#### Configuring: +#### Using: + +### DosBox: + +#### Installing: +#### Configuring: +#### Using: + +### DROID: + +#### Installing: +#### Configuring: +#### Using: + +### Emailchemy: + +#### Installing: +#### Configuring: +#### Using: + +### ePADD: + +#### Installing: +#### Configuring: +#### Using: + +### FACTools: + +#### Installing: +#### Configuring: +#### Using: + +### Filemaker Pro 17: + +#### Installing: +#### Configuring: +#### Using: + +### Forensic Toolkit: + +#### Installing: +#### Configuring: +#### Using: + +### FTK Imager: + +#### Installing: +#### Configuring: +#### Using: + +### gtar: + +#### Installing: +#### Configuring: +#### Using: + +### HFS Explorer: + +#### Installing: +#### Configuring: +#### Using: + +### Iromlab: + +#### Installing: +#### Configuring: +#### Using: + +### IsoBuster: + +#### Installing: +#### Configuring: +#### Using: + +### Java: + +#### Installing: +#### Configuring: +#### Using: + +### MediaInfo: + +#### Installing: +#### Configuring: +#### Using: + +### miniVmac: + +#### Installing: +#### Configuring: +#### Using: + +### openssl: + +#### Installing: +#### Configuring: +#### Using: + +### Oracle VM VirtualBox: + +#### Installing: +#### Configuring: +#### Using: + +### pip: + +#### Installing: +#### Configuring: +#### Using: + +### Python3: + +#### Installing: +#### Configuring: +#### Using: + +### QuickTime Player: + +#### Installing: +#### Configuring: +#### Using: + +### QuickView Plus: + +#### Installing: +#### Configuring: +#### Using: + ### rclone: [*Rclone*](https://rclone.org/) is a command line tool for managing files on cloud storage, and is one tool used by Digital Archives for moving files between Digital Archives workstations, Google Shared Drives for Divisions, and long term storage. This page lists instructions for installing and configuring rclone. @@ -109,25 +293,55 @@ To see all currently configured remotes (rclone term for any cloud storage locat #### Using rclone: -For detailed installation instruction tailored to your operating system see the [rclone documentation](https://rclone.org/install/). -For MacOS: -Use Homebrew to install rclone via the terminal by entering: -``` -brew install rclone -``` +### rsync: -For Windows: -[Download and run Windows installation file.](https://rclone.org/downloads/) +#### Installing: +#### Configuring: +#### Using: + +### Sha1deep: + +#### Installing: +#### Configuring: +#### Using: + + +### Sheepshaver: + +#### Installing: +#### Configuring: +#### Using: + +### Siegfried: + +#### Installing: +#### Configuring: +#### Using: + +### tree: + +#### Installing: +#### Configuring: +#### Using: + +### VLC: + +#### Installing: +#### Configuring: +#### Using: + +### Windows Subsystem for Linux (WSL): + +#### Installing: +#### Configuring: +#### Using: + +### WinUAE: -### rsync -### Sha1deep -### Sheepshaver -### Siegfried -### tree -### VLC -### Windows Subsystem for Linux (WSL) -### WinUAE +#### Installing: +#### Configuring: +#### Using: ## Working Scripts: From 7c3eb29f4c642dd146a9cdda50732ce7877afeb6 Mon Sep 17 00:00:00 2001 From: dreyjo <59994482+dreyjo@users.noreply.github.com> Date: Fri, 21 Jul 2023 14:47:36 -0400 Subject: [PATCH 03/17] added to data file --- _data/program.csv | 13 +++++++++++++ _data/working.csv | 0 2 files changed, 13 insertions(+) create mode 100644 _data/program.csv create mode 100644 _data/working.csv diff --git a/_data/program.csv b/_data/program.csv new file mode 100644 index 0000000..b4bba90 --- /dev/null +++ b/_data/program.csv @@ -0,0 +1,13 @@ +name,desc,inst +7-zip,7-zip is used in the lab to compress evidence and cases generated in Forensic Toolkit. It is also occasionally used to unpack collection material that is not supported by FTK.,[Download 7-zip](https://www.7-zip.org/){:target='_blank'} +apple3rtr,Apple /// Ready-to-Run (apple3rtr) is a software bundle used for MAME Apple emulation. It has been used by Digital Archives staff to view Apple II files.,For detailed installation instructions visit the [apple3rtr github repo](https://github.com/datajerk/apple3rtr). +AWS,'Amazon Web Services (AWS) is a cloud computing platform offered by Amazon. AWS S3 features are used to store copies of born-digital material both before and after the collection has been processed by archivists.' +Bagger, Bagger is an applciation created by the U.S. Library of Congress to produce files packages following the Bagit specification. It has been used in the Digital Archives lab for transferring born-digital material +Bagit.py, Bagit.py is a command line tool used produce files packages following the Bagit specification, Check python version by entering ```python --version``` or ```python3 --version``` into terminal. Then install using pip by entering ```pip install bagit``` or ```pip3 install bagit``` +Bitcurator, BitCurator is a Linux distribution with a suite of open source digital forensics and data analysis tools used in the lab as both a virtual machine and as bootable OS alongside Windows. BitCurator is used for transfer and metadata extraction especially for formats unrecognized in Windows and Mac environments., Download software [BitCurator VM](https://github.com/BitCurator/bitcurator-distro/wiki/Releases) or install with Live CD. The [Virtualbox Extension Pack](https://www.virtualbox.org/wiki/Downloads) must be installed on the host for shared folder support, USB 2.0/3.0 support and support for some NVMe SSDs. Once you’ve installed VirtualBox, use the link below the main download on VirtualBox download site to install the Extension Pack on your host. +CiderPress, CiderPress is a Windows utility for managing Apple II disk images. It has been used in the Digital Archives Lab for viewing Apple file formats., +rclone,'For detailed installation instruction tailored to your operating system see the [rclone documentation](https://rclone.org/install/).

For MacOS: Use Homebrew to install rclone via the terminal by entering ```brew install rclone```

For Windows:[Download and run Windows installation file.](https://rclone.org/downloads/)' + + + + diff --git a/_data/working.csv b/_data/working.csv new file mode 100644 index 0000000..e69de29 From 1bf4e9a336f10bb54fa5a36110caac0ab7a0e159 Mon Sep 17 00:00:00 2001 From: dreyjo <59994482+dreyjo@users.noreply.github.com> Date: Wed, 26 Jul 2023 13:01:18 -0400 Subject: [PATCH 04/17] added all programs through D --- _data/program.csv | 10 +++++++++- tools/software_draft.md | 27 ++++++++++++++++++++++++--- 2 files changed, 33 insertions(+), 4 deletions(-) diff --git a/_data/program.csv b/_data/program.csv index b4bba90..6d47905 100644 --- a/_data/program.csv +++ b/_data/program.csv @@ -1,4 +1,4 @@ -name,desc,inst +name,desc,inst,conf,use 7-zip,7-zip is used in the lab to compress evidence and cases generated in Forensic Toolkit. It is also occasionally used to unpack collection material that is not supported by FTK.,[Download 7-zip](https://www.7-zip.org/){:target='_blank'} apple3rtr,Apple /// Ready-to-Run (apple3rtr) is a software bundle used for MAME Apple emulation. It has been used by Digital Archives staff to view Apple II files.,For detailed installation instructions visit the [apple3rtr github repo](https://github.com/datajerk/apple3rtr). AWS,'Amazon Web Services (AWS) is a cloud computing platform offered by Amazon. AWS S3 features are used to store copies of born-digital material both before and after the collection has been processed by archivists.' @@ -6,6 +6,14 @@ Bagger, Bagger is an applciation created by the U.S. Library of Congress to prod Bagit.py, Bagit.py is a command line tool used produce files packages following the Bagit specification, Check python version by entering ```python --version``` or ```python3 --version``` into terminal. Then install using pip by entering ```pip install bagit``` or ```pip3 install bagit``` Bitcurator, BitCurator is a Linux distribution with a suite of open source digital forensics and data analysis tools used in the lab as both a virtual machine and as bootable OS alongside Windows. BitCurator is used for transfer and metadata extraction especially for formats unrecognized in Windows and Mac environments., Download software [BitCurator VM](https://github.com/BitCurator/bitcurator-distro/wiki/Releases) or install with Live CD. The [Virtualbox Extension Pack](https://www.virtualbox.org/wiki/Downloads) must be installed on the host for shared folder support, USB 2.0/3.0 support and support for some NVMe SSDs. Once you’ve installed VirtualBox, use the link below the main download on VirtualBox download site to install the Extension Pack on your host. CiderPress, CiderPress is a Windows utility for managing Apple II disk images. It has been used in the Digital Archives Lab for viewing Apple file formats., +dbpoweramp, dbpoweramp is a program used in conjunction with [Iromlab](#Iromlab) for transferring born-digital material from optical media, +dd, dd is a command line utility used in the Digital Archives Lab to create disk images of optical media. In most cases transfers are made of optical media instead of disk images. +Disk Utility, Disk Utility is a MacOS utility for disk volume management. Disk Utility used by Digital Archives staff for mounting/unmounting remvable media.,,,enter ```list disk``` or ```list volume``` to confirm media is mounted +diskpart, diskpart is Windows command line utility for disk partitioning. diskpart is used by Digital Archives staff for confimring removable media are properly mounted. +disktype, disktype is a command line tool used by Digital Archives staff for extracting filesystem metadata liek sector format for floppy disks, For Mac OS:
In terminal use homebrew to install by entering ```brew install disktype```,, In terminal navigate to the directory containing floppy disk image(s) and enter ```disktype a/disk/image```. +DosBox, DosBox is an open-source emulator for teh MS-DOS operating system. DosBox has been used by Digital Archives staff to view DOS executables and word processing files. +DROID, DROID is file format identification tool developed by the UK National Archives. +Iromlab, Iromlab is software for automated imaging of optical media using [Nimbie](#tools/nimbie) disc robots, rclone,'For detailed installation instruction tailored to your operating system see the [rclone documentation](https://rclone.org/install/).

For MacOS: Use Homebrew to install rclone via the terminal by entering ```brew install rclone```

For Windows:[Download and run Windows installation file.](https://rclone.org/downloads/)' diff --git a/tools/software_draft.md b/tools/software_draft.md index d32152a..e518d9d 100644 --- a/tools/software_draft.md +++ b/tools/software_draft.md @@ -21,7 +21,26 @@ Digital Archives utilizes several software throughout our workflows. This page a Programs are defined as any non-internally developed out-of-the-box software used in Digital Archives workflows. -### 7-zip: +{% for tool in site.data.program %} + +### {{tool.name}}: +{{tool.desc}} + +#### Installing {{tool.name}}: +{{tool.inst}} + +{% comment %} +#### Configuring {{tool.name}}: +{{tool.conf}} + +#### Using {{tool.name}}: +{{tool.use}} +{% endcomment %} + +{% endfor %} + +{% comment %} +### 7-zip 7-zip is used in the lab to compress evidence and cases generated in Forensic Toolkit. It is also occasionally used to unpack collection material that is not supported by FTK. @@ -37,7 +56,7 @@ Programs are defined as any non-internally developed out-of-the-box software use Apple /// Ready-to-Run (apple3rtr) is a software bundle used for MAME Apple emulation. It has been used by Digital Archives staff to view Apple II files. #### Installing: -For detailed installation instructions vist the (apple3rtr github repo)[https://github.com/datajerk/apple3rtr]. +For detailed installation instructions visit the (apple3rtr github repo)[https://github.com/datajerk/apple3rtr]. #### Configuring: #### Using: @@ -256,6 +275,7 @@ brew install rclone For Windows: [Download and run Windows installation file.](https://rclone.org/downloads/) + #### Configuring rclone: Once installed, you must configure Rclone which involves setting the remote locations you will be accessing. Detailed configuration instruction can be found in the [rclone configuration docs](https://rclone.org/docs/) separated by cloud storage systems. This section details configuration instructions for MacOS users accessing Google Drive. @@ -362,4 +382,5 @@ Deprecated and Legacy Scripts are any scripts which may have been previously use ### ft.sh ### metadata.sh ### movemetadata script -### movephotograph script +### movephotograph script +{% endcomment %} From 37d9fc80943d47816176730a7550d5cfdca49392 Mon Sep 17 00:00:00 2001 From: dreyjo <59994482+dreyjo@users.noreply.github.com> Date: Thu, 27 Jul 2023 17:38:52 -0400 Subject: [PATCH 05/17] added programs D through M --- _data/program.csv | 26 +++++++++++++++++++++++++- 1 file changed, 25 insertions(+), 1 deletion(-) diff --git a/_data/program.csv b/_data/program.csv index 6d47905..379f125 100644 --- a/_data/program.csv +++ b/_data/program.csv @@ -13,9 +13,33 @@ diskpart, diskpart is Windows command line utility for disk partitioning. diskpa disktype, disktype is a command line tool used by Digital Archives staff for extracting filesystem metadata liek sector format for floppy disks, For Mac OS:
In terminal use homebrew to install by entering ```brew install disktype```,, In terminal navigate to the directory containing floppy disk image(s) and enter ```disktype a/disk/image```. DosBox, DosBox is an open-source emulator for teh MS-DOS operating system. DosBox has been used by Digital Archives staff to view DOS executables and word processing files. DROID, DROID is file format identification tool developed by the UK National Archives. +Emailchemy, Emailchemy has been used in the Digital Archives lab for migrating email fomats. +ePADD, ePADD is an open-source program developed by Stanford University's Special Collections and University Archives. The software ha been used for appraisal and arrangement of email. +Filemaker Pro 17, Filemaker is used by Preservation and Collection Services (PCP) as a collection management system. born-digital media is inventoried in Filemaker and PCP tracks projects through Filemaker., Installing Filemaker requires license keys. Make sure you have the proper keys and permissions before installing. +Forensic Toolkit, Forensic Toolkit is a computer forensics software used in the Digital Archives Lab to appraise and arrange born-digital material. +FTK Imager, FTK Imager is a disk imaging program associated with Forensic Toolkit. The software has been used in the Digital Archives lab to image and extract files from digital carriers. +gtar, Gtar is a command line program used on Digital Archives MacOS workstations to compress Kryoflux stream files., For MacOS: Install via Homebrew by entering ```brew install gnu-tar``` in terminal. +HFS Explorer, HFS Explorer is a program usd to view HFS formatted disks on Windows operating systems, Iromlab, Iromlab is software for automated imaging of optical media using [Nimbie](#tools/nimbie) disc robots, +IsoBuster, IsoBuster is data recovery software that has been used in the Digital Archives lab to image and extract files from optical media. +Java, Java is a software package which has baeen a dependency for some Digital Archives tools like DROID and Bagger,,, Check for the version of Java installed with commands ```which java``` or ```java -version``` in terminal on Mac or WSL on Windows. +MediaInfo, MediaInfo is an open-source program for extractign technical information from media files. The program has been used in the Digital Archives Lab to extract metadata from born-digital video, +Mediaconch, +miniVmac, +openssl, +Oracle VM VirtualBox, +pip, +Python3, +QuickTime Player, +QuickView Plus, rclone,'For detailed installation instruction tailored to your operating system see the [rclone documentation](https://rclone.org/install/).

For MacOS: Use Homebrew to install rclone via the terminal by entering ```brew install rclone```

For Windows:[Download and run Windows installation file.](https://rclone.org/downloads/)' - +Sha1deep, +Sheepshaver, +Siegfried, +tree, +VLC, +WSL, +WinUAE, From a95f953b20240fad79ea2853fc13624838a2a063 Mon Sep 17 00:00:00 2001 From: dreyjo <59994482+dreyjo@users.noreply.github.com> Date: Wed, 9 Aug 2023 17:02:31 -0400 Subject: [PATCH 06/17] added all programs --- _data/program.csv | 82 +++++----- tools/software_draft.md | 328 +--------------------------------------- 2 files changed, 40 insertions(+), 370 deletions(-) diff --git a/_data/program.csv b/_data/program.csv index 379f125..6d24ca5 100644 --- a/_data/program.csv +++ b/_data/program.csv @@ -1,45 +1,41 @@ name,desc,inst,conf,use -7-zip,7-zip is used in the lab to compress evidence and cases generated in Forensic Toolkit. It is also occasionally used to unpack collection material that is not supported by FTK.,[Download 7-zip](https://www.7-zip.org/){:target='_blank'} -apple3rtr,Apple /// Ready-to-Run (apple3rtr) is a software bundle used for MAME Apple emulation. It has been used by Digital Archives staff to view Apple II files.,For detailed installation instructions visit the [apple3rtr github repo](https://github.com/datajerk/apple3rtr). -AWS,'Amazon Web Services (AWS) is a cloud computing platform offered by Amazon. AWS S3 features are used to store copies of born-digital material both before and after the collection has been processed by archivists.' -Bagger, Bagger is an applciation created by the U.S. Library of Congress to produce files packages following the Bagit specification. It has been used in the Digital Archives lab for transferring born-digital material -Bagit.py, Bagit.py is a command line tool used produce files packages following the Bagit specification, Check python version by entering ```python --version``` or ```python3 --version``` into terminal. Then install using pip by entering ```pip install bagit``` or ```pip3 install bagit``` -Bitcurator, BitCurator is a Linux distribution with a suite of open source digital forensics and data analysis tools used in the lab as both a virtual machine and as bootable OS alongside Windows. BitCurator is used for transfer and metadata extraction especially for formats unrecognized in Windows and Mac environments., Download software [BitCurator VM](https://github.com/BitCurator/bitcurator-distro/wiki/Releases) or install with Live CD. The [Virtualbox Extension Pack](https://www.virtualbox.org/wiki/Downloads) must be installed on the host for shared folder support, USB 2.0/3.0 support and support for some NVMe SSDs. Once you’ve installed VirtualBox, use the link below the main download on VirtualBox download site to install the Extension Pack on your host. +7-zip,7-zip is used in the lab to compress evidence and cases generated in Forensic Toolkit. It is also occasionally used to unpack collection material that is not supported by FTK.,[Download 7-zip](https://www.7-zip.org/){:target='_blank'}., +apple3rtr,Apple /// Ready-to-Run (apple3rtr) is a software bundle used for MAME Apple emulation. It has been used by Digital Archives staff to view Apple II files.,For detailed installation instructions visit the [apple3rtr github repo](https://github.com/datajerk/apple3rtr)., +AWS, Amazon Web Services (AWS) is a cloud computing platform offered by Amazon. AWS S3 features are used to store copies of born-digital material both before and after the collection has been processed by archivists., +Bagger, Bagger is an applciation created by the U.S. Library of Congress to produce files packages following the Bagit specification. It has been used in the Digital Archives lab for transferring born-digital material., +Bagit.py, Bagit.py is a command line tool used produce files packages following the Bagit specification, Check python version by entering ```python --version``` or ```python3 --version``` into terminal. Then install using pip by entering ```pip install bagit``` or ```pip3 install bagit```. Use the link below the main download on the VirtualBox download site to install the Extension Pack on your host., CiderPress, CiderPress is a Windows utility for managing Apple II disk images. It has been used in the Digital Archives Lab for viewing Apple file formats., -dbpoweramp, dbpoweramp is a program used in conjunction with [Iromlab](#Iromlab) for transferring born-digital material from optical media, -dd, dd is a command line utility used in the Digital Archives Lab to create disk images of optical media. In most cases transfers are made of optical media instead of disk images. -Disk Utility, Disk Utility is a MacOS utility for disk volume management. Disk Utility used by Digital Archives staff for mounting/unmounting remvable media.,,,enter ```list disk``` or ```list volume``` to confirm media is mounted -diskpart, diskpart is Windows command line utility for disk partitioning. diskpart is used by Digital Archives staff for confimring removable media are properly mounted. -disktype, disktype is a command line tool used by Digital Archives staff for extracting filesystem metadata liek sector format for floppy disks, For Mac OS:
In terminal use homebrew to install by entering ```brew install disktype```,, In terminal navigate to the directory containing floppy disk image(s) and enter ```disktype a/disk/image```. -DosBox, DosBox is an open-source emulator for teh MS-DOS operating system. DosBox has been used by Digital Archives staff to view DOS executables and word processing files. -DROID, DROID is file format identification tool developed by the UK National Archives. -Emailchemy, Emailchemy has been used in the Digital Archives lab for migrating email fomats. -ePADD, ePADD is an open-source program developed by Stanford University's Special Collections and University Archives. The software ha been used for appraisal and arrangement of email. -Filemaker Pro 17, Filemaker is used by Preservation and Collection Services (PCP) as a collection management system. born-digital media is inventoried in Filemaker and PCP tracks projects through Filemaker., Installing Filemaker requires license keys. Make sure you have the proper keys and permissions before installing. -Forensic Toolkit, Forensic Toolkit is a computer forensics software used in the Digital Archives Lab to appraise and arrange born-digital material. -FTK Imager, FTK Imager is a disk imaging program associated with Forensic Toolkit. The software has been used in the Digital Archives lab to image and extract files from digital carriers. -gtar, Gtar is a command line program used on Digital Archives MacOS workstations to compress Kryoflux stream files., For MacOS: Install via Homebrew by entering ```brew install gnu-tar``` in terminal. -HFS Explorer, HFS Explorer is a program usd to view HFS formatted disks on Windows operating systems, -Iromlab, Iromlab is software for automated imaging of optical media using [Nimbie](#tools/nimbie) disc robots, -IsoBuster, IsoBuster is data recovery software that has been used in the Digital Archives lab to image and extract files from optical media. -Java, Java is a software package which has baeen a dependency for some Digital Archives tools like DROID and Bagger,,, Check for the version of Java installed with commands ```which java``` or ```java -version``` in terminal on Mac or WSL on Windows. -MediaInfo, MediaInfo is an open-source program for extractign technical information from media files. The program has been used in the Digital Archives Lab to extract metadata from born-digital video, -Mediaconch, -miniVmac, -openssl, -Oracle VM VirtualBox, -pip, -Python3, -QuickTime Player, -QuickView Plus, -rclone,'For detailed installation instruction tailored to your operating system see the [rclone documentation](https://rclone.org/install/).

For MacOS: Use Homebrew to install rclone via the terminal by entering ```brew install rclone```

For Windows:[Download and run Windows installation file.](https://rclone.org/downloads/)' -Sha1deep, -Sheepshaver, -Siegfried, -tree, -VLC, -WSL, -WinUAE, - - - +dbpoweramp, dbpoweramp is a program used in conjunction with [Iromlab](#Iromlab) for transferring born-digital material from optical media., +dd, dd is a command line utility used in the Digital Archives Lab to create disk images of optical media. In most cases transfers are made of optical media instead of disk images., +Disk Utility, Disk Utility is a MacOS utility for disk volume management. Disk Utility used by Digital Archives staff for mounting/unmounting remvable media.,,,enter ```list disk``` or ```list volume``` to confirm media is mounted., +diskpart, diskpart is Windows command line utility for disk partitioning. diskpart is used by Digital Archives staff for confimring removable media are properly mounted., +disktype, disktype is a command line tool used by Digital Archives staff for extracting filesystem metadata liek sector format for floppy disks, For Mac OS:
In terminal use homebrew to install by entering ```brew install disktype```,, In terminal navigate to the directory containing floppy disk image(s) and enter ```disktype a/disk/image```., +DosBox, DosBox is an open-source emulator for teh MS-DOS operating system. DosBox has been used by Digital Archives staff to view DOS executables and word processing files., +DROID, DROID is file format identification tool developed by the UK National Archives., +Emailchemy, Emailchemy has been used in the Digital Archives lab for migrating email fomats., +ePADD, ePADD is an open-source program developed by Stanford University's Special Collections and University Archives. The software ha been used for appraisal and arrangement of email., +Filemaker Pro 17, Filemaker is used by Preservation and Collection Services (PCP) as a collection management system. born-digital media is inventoried in Filemaker and PCP tracks projects through Filemaker., Installing Filemaker requires license keys. Make sure you have the proper keys and permissions before installing., +Forensic Toolkit, Forensic Toolkit is a computer forensics software used in the Digital Archives Lab to appraise and arrange born-digital material., +FTK Imager, FTK Imager is a disk imaging program associated with Forensic Toolkit. The software has been used in the Digital Archives lab to image and extract files from digital carriers., +gtar, Gtar is a command line program used on Digital Archives MacOS workstations to compress Kryoflux stream files., For MacOS: Install via Homebrew by entering ```brew install gnu-tar``` in terminal., +HFS Explorer, HFS Explorer is a program usd to view HFS formatted disks on Windows operating systems., +Iromlab, Iromlab is software for automated imaging of optical media using [Nimbie](#tools/nimbie) disc robots., +IsoBuster, IsoBuster is data recovery software that has been used in the Digital Archives lab to image and extract files from optical media., +Java, Java is a software package which has baeen a dependency for some Digital Archives tools like DROID and Bagger,,, Check for the version of Java installed with commands ```which java``` or ```java -version``` in terminal on Mac or WSL on Windows., +MediaInfo, MediaInfo is an open-source program for extractign technical information from media files. The program has been used in the Digital Archives Lab to extract metadata from born-digital video., +Mediaconch, MediaConch is an open source software project developed by MediaArea and utilizing MediaInfo. It is used by Digital Archives staff to develop programatic policies for checking born-digital audiovisual material against preservation standards., +Mini Vmac, Mini vMac is an emulator for running early Macintosh software., +openssl, OpenSSL is a software library used to ssh., +Oracle VM VirtualBox, Virtualbox is open soure virtualization software. It has been used by Digital Archives staff to run teh BitCurator environment as a virtual machine., Download and install the [current VirtualBox release and Extension Pack](https://www.virtualbox.org/wiki/Downloads)., +pip, Pip is package management software used for installing and upgrading Python modules,,,Pip should be installed with python3. Call pip by entering ```pip``` or ```pip3``` in terminal or command line., +Python3, Python3 is the latest version of the Python programming language, For Windows: Check for python versions by entering ```python --version``` on command line. Installers are avilable at python.org.
For MacOS: Check for python versions by entering ```python --version``` in terminal. Install your selected version of python with homebrew by entering ```brew install python@version.number```., +QuickTime Player, QuickTime is used in teh Digital Archives Lab to appraise Mcintosh formatted born-digital video., +QuickView Plus, QuickView Plus has been used in the Digital Archives Lab to view a range of legacy formats like word processing and photo formats., +rclone,'For detailed installation instruction tailored to your operating system see the [rclone documentation](https://rclone.org/install/).

For MacOS: Use Homebrew to install rclone via the terminal by entering ```brew install rclone```

For Windows:[Download and run Windows installation file.](https://rclone.org/downloads/)'., +Sha1deep, Sha1deep is a command line program for computing and comparing SHA-256 cryptographic hashes. Sha1deep is sued by Digital Archives Staff to identify duplicate material across digital media in a collection by comparing checksums, For MacOS: Install via Homebrew by entering ```brew install m5deep```, +Sheepshaver, Sheepshaver is an Apple Macintosh emulator. It has been used in teh Digital Archives Lab to view legacy Mac files., +Siegfried, Siegfried is a command line file format identification tool. Siegfried is used in the Digital Archives Lab to identify and validate file formats. Siegfried output also includes checksums., For Windows: Download [the latest Siegfried binary](https://github.com/richardlehane/siegfried/releases/download/v1.10.1/siegfried_1-10-1_win64.zip).
For MacOS: Install via homebrew by entering ```brew install richardlehane/digipres/siegfried``` into terminal,Current installations of siegfried use the config file to store preferred command options. Enter ```sf -setconf -csv -hash sha1 -z``` in terminal to set the preferred command options., For identifying a file format enter ```sf /path/to/file``` in terminal. Enter sf -help for a complete list of sf commands. +tree, tree is command line program for recursive directory listing and has been used in the Digital Archives Lab to identify contents of media including number of folders and files, For MacOS: Install via homebrew by entering ```brew install tree``` into terminal., +VLC, VLC is an open source media player used by Digital Archives staff to view and appraise born-digital audio and video., Download [the latest release of VLC Media Player](https://www.videolan.org/vlc/)., +WSL, Windows Subsystem for Linux (WSL) is a Windows feature used to run Linux environments on Windows operating systems, +WinUAE, WinUAE is an Amiga emulator and has been used in the Digital Archives Lab to view Amiga files., \ No newline at end of file diff --git a/tools/software_draft.md b/tools/software_draft.md index e518d9d..10aeede 100644 --- a/tools/software_draft.md +++ b/tools/software_draft.md @@ -29,340 +29,14 @@ Programs are defined as any non-internally developed out-of-the-box software use #### Installing {{tool.name}}: {{tool.inst}} -{% comment %} #### Configuring {{tool.name}}: {{tool.conf}} #### Using {{tool.name}}: {{tool.use}} -{% endcomment %} {% endfor %} -{% comment %} -### 7-zip - -7-zip is used in the lab to compress evidence and cases generated in Forensic Toolkit. It is also occasionally used to unpack collection material that is not supported by FTK. - -#### Installing 7-zip: - -[Download 7-zip](https://www.7-zip.org/){:target="_blank"} - -#### Configuring: -#### Using: - -### apple3rtr: - -Apple /// Ready-to-Run (apple3rtr) is a software bundle used for MAME Apple emulation. It has been used by Digital Archives staff to view Apple II files. - -#### Installing: -For detailed installation instructions visit the (apple3rtr github repo)[https://github.com/datajerk/apple3rtr]. - -#### Configuring: -#### Using: - -### AWS: - -#### Installing: -#### Configuring: -#### Using: - -### Bagger: - -#### Installing: -#### Configuring: -#### Using: - -### Bagit.py: - -#### Installing: -#### Configuring: -#### Using: - -### BitCurator: - -#### Installing: -#### Configuring: -#### Using: - -### Brunnhilde with disktype: - -#### Installing: -#### Configuring: -#### Using: - -### clamscan: - -#### Installing: -#### Configuring: -#### Using: - -### CiderPress: - -#### Installing: -#### Configuring: -#### Using: - -### dBpoweramp: - -#### Installing: -#### Configuring: -#### Using: - -### dd: - -#### Installing: -#### Configuring: -#### Using: - -### Disk Utility: - -#### Installing: -#### Configuring: -#### Using: - -### DiskPart: - -#### Installing: -#### Configuring: -#### Using: - -### disktype: - -#### Installing: -#### Configuring: -#### Using: - -### DosBox: - -#### Installing: -#### Configuring: -#### Using: - -### DROID: - -#### Installing: -#### Configuring: -#### Using: - -### Emailchemy: - -#### Installing: -#### Configuring: -#### Using: - -### ePADD: - -#### Installing: -#### Configuring: -#### Using: - -### FACTools: - -#### Installing: -#### Configuring: -#### Using: - -### Filemaker Pro 17: - -#### Installing: -#### Configuring: -#### Using: - -### Forensic Toolkit: - -#### Installing: -#### Configuring: -#### Using: - -### FTK Imager: - -#### Installing: -#### Configuring: -#### Using: - -### gtar: - -#### Installing: -#### Configuring: -#### Using: - -### HFS Explorer: - -#### Installing: -#### Configuring: -#### Using: - -### Iromlab: - -#### Installing: -#### Configuring: -#### Using: - -### IsoBuster: - -#### Installing: -#### Configuring: -#### Using: - -### Java: - -#### Installing: -#### Configuring: -#### Using: - -### MediaInfo: - -#### Installing: -#### Configuring: -#### Using: - -### miniVmac: - -#### Installing: -#### Configuring: -#### Using: - -### openssl: - -#### Installing: -#### Configuring: -#### Using: - -### Oracle VM VirtualBox: - -#### Installing: -#### Configuring: -#### Using: - -### pip: - -#### Installing: -#### Configuring: -#### Using: - -### Python3: - -#### Installing: -#### Configuring: -#### Using: - -### QuickTime Player: - -#### Installing: -#### Configuring: -#### Using: - -### QuickView Plus: - -#### Installing: -#### Configuring: -#### Using: - -### rclone: - -[*Rclone*](https://rclone.org/) is a command line tool for managing files on cloud storage, and is one tool used by Digital Archives for moving files between Digital Archives workstations, Google Shared Drives for Divisions, and long term storage. This page lists instructions for installing and configuring rclone. - -#### Installing rclone: - -For detailed installation instruction tailored to your operating system see the [rclone documentation](https://rclone.org/install/). - -For MacOS: -Use Homebrew to install rclone via the terminal by entering: -``` -brew install rclone -``` - -For Windows: -[Download and run Windows installation file.](https://rclone.org/downloads/) - -#### Configuring rclone: - -Once installed, you must configure Rclone which involves setting the remote locations you will be accessing. Detailed configuration instruction can be found in the [rclone configuration docs](https://rclone.org/docs/) separated by cloud storage systems. This section details configuration instructions for MacOS users accessing Google Drive. - -* In terminal, type in ```rclone config``` - -* Select “n” for “New Remote” - -* Enter the name for your remote when Rclone prompts you to, try to keep the name as short as possible while still being descriptive and specific. - -* rclone will then ask you to select from a list of storage platforms. Enter the number that corresponds on the list to Google Drive (Note: Google Cloud Storage is different from Google Drive!) - -* rclone asks for a Google Application Client Id, this is optional and you can select enter to choose default. - -* rclone asks for an OAuth Client Secret Id, this is optional and you can enter to select enter to choose default. - * To set your own application and OAuth client IDs see instructions on [creating a google application client id](https://rclone.org/drive/#making-your-own-client-id) in the rclone docs. - -* rclone will provide a list of scopes of access ranging from full access to all files to read-only access to file metadata. For our purposes enter the number corresponding to full access to all files. - -* Rclone asks for a root folder id, this is optional and you can enter to select enter to choose default. - -* Rclone asks for service account credentials, this is optional and you can enter to select enter to choose default. - -* Rclone asks you to edit advanced configuration, select “n” for No. Follow up prompt asks if you would like to use auto configuration, enter “y” for Yes. - -* A browser window should then open asking you to log into your google account and allow access to Rclone. Move through the prompts in browser. Once you see the success window, return to the terminal - -* Rclone asks if you would like to configure this remote as a shared drive. Select Y for yes. This opens a list of the shared drives associated with your account. - -* Enter the number associated with the shared drive you will be uploading requested items to. - -* Rclone then asks if the configuration you’ve been doing is okay. Enter y for Yes and configuration is complete! - -To see all currently configured remotes (rclone term for any cloud storage locations) enter ```rclone listremotes``` into terminal. - -#### Using rclone: - - -### rsync: - -#### Installing: -#### Configuring: -#### Using: - -### Sha1deep: - -#### Installing: -#### Configuring: -#### Using: - - -### Sheepshaver: - -#### Installing: -#### Configuring: -#### Using: - -### Siegfried: - -#### Installing: -#### Configuring: -#### Using: - -### tree: - -#### Installing: -#### Configuring: -#### Using: - -### VLC: - -#### Installing: -#### Configuring: -#### Using: - -### Windows Subsystem for Linux (WSL): - -#### Installing: -#### Configuring: -#### Using: - -### WinUAE: - -#### Installing: -#### Configuring: -#### Using: - ## Working Scripts: Working Scripts are defined as scripts written internally and maintained by Digital Archives staff for use in our workflows. @@ -383,4 +57,4 @@ Deprecated and Legacy Scripts are any scripts which may have been previously use ### metadata.sh ### movemetadata script ### movephotograph script -{% endcomment %} + From bd06e1fc78d789fd332a576b1fe2f583bc0f9e22 Mon Sep 17 00:00:00 2001 From: dreyjo <59994482+dreyjo@users.noreply.github.com> Date: Thu, 10 Aug 2023 11:47:19 -0400 Subject: [PATCH 07/17] added working scripts --- _data/working.csv | 5 +++++ tools/software_draft.md | 15 ++++++++------- 2 files changed, 13 insertions(+), 7 deletions(-) diff --git a/_data/working.csv b/_data/working.csv index e69de29..f4b38d7 100644 --- a/_data/working.csv +++ b/_data/working.csv @@ -0,0 +1,5 @@ +name,desc,inst,conf,use +kryofluxmove.sh,This bash script is used by Digital Archives staff to move floppy disk images transferred using kryoflux to folders with legacy CMS media IDs of the associated digital carrier., +SIPdir.sh,This bash script is used by Digital Archives staff to reate consecutive numbers of submission information packages for material transferred from digital carriers., +report_ftk_extents.py,This python script transforms XML reports exported from FTK after Processing Archivists have completed bookmarking into JSON files for import into ASpace. +report_HDD_extents.py,This python script collects the extents of finding aid components created by a Processing Archivist when processing work is done at a workstation with a hard drive. The script then produces an extents JSON file for import into ASpace. \ No newline at end of file diff --git a/tools/software_draft.md b/tools/software_draft.md index 10aeede..d2f5ea3 100644 --- a/tools/software_draft.md +++ b/tools/software_draft.md @@ -41,13 +41,14 @@ Programs are defined as any non-internally developed out-of-the-box software use Working Scripts are defined as scripts written internally and maintained by Digital Archives staff for use in our workflows. -### CMS metadata import -### iterative scripts -### kryofluxmove.sh -### makesips script -### moveimages script -### report ftk extents script -### report HDD extents script +{% for tool in site.data.working %} +### {{tool.name}}: +{{tool.desc}} + +#### Using {{tool.name}}: +{{tool.use}} + +{% endfor %} ## Deprecated and Legacy Scripts: From 74c2f1f602a835d763ab74d85b1cfa3ecba072ab Mon Sep 17 00:00:00 2001 From: dreyjo <59994482+dreyjo@users.noreply.github.com> Date: Thu, 24 Aug 2023 16:58:09 -0400 Subject: [PATCH 08/17] added if conditionals to programs and working scripts, added software hub page --- _data/program.csv | 2 +- _data/working.csv | 6 +++--- tools/rclone.md | 7 ++++--- tools/software_draft.md | 39 ++++++++++----------------------------- tools/working-scripts.md | 32 ++++++++++++++++++++++++++++++++ 5 files changed, 50 insertions(+), 36 deletions(-) create mode 100644 tools/working-scripts.md diff --git a/_data/program.csv b/_data/program.csv index 6d24ca5..6919a03 100644 --- a/_data/program.csv +++ b/_data/program.csv @@ -31,7 +31,7 @@ pip, Pip is package management software used for installing and upgrading Python Python3, Python3 is the latest version of the Python programming language, For Windows: Check for python versions by entering ```python --version``` on command line. Installers are avilable at python.org.
For MacOS: Check for python versions by entering ```python --version``` in terminal. Install your selected version of python with homebrew by entering ```brew install python@version.number```., QuickTime Player, QuickTime is used in teh Digital Archives Lab to appraise Mcintosh formatted born-digital video., QuickView Plus, QuickView Plus has been used in the Digital Archives Lab to view a range of legacy formats like word processing and photo formats., -rclone,'For detailed installation instruction tailored to your operating system see the [rclone documentation](https://rclone.org/install/).

For MacOS: Use Homebrew to install rclone via the terminal by entering ```brew install rclone```

For Windows:[Download and run Windows installation file.](https://rclone.org/downloads/)'., +rclone,rclone is a command line tool for managing and moving files between remote and local storage locations,For detailed installation and configuration instructions [see our dedicated rclone page](/tools/rclone.html). Sha1deep, Sha1deep is a command line program for computing and comparing SHA-256 cryptographic hashes. Sha1deep is sued by Digital Archives Staff to identify duplicate material across digital media in a collection by comparing checksums, For MacOS: Install via Homebrew by entering ```brew install m5deep```, Sheepshaver, Sheepshaver is an Apple Macintosh emulator. It has been used in teh Digital Archives Lab to view legacy Mac files., Siegfried, Siegfried is a command line file format identification tool. Siegfried is used in the Digital Archives Lab to identify and validate file formats. Siegfried output also includes checksums., For Windows: Download [the latest Siegfried binary](https://github.com/richardlehane/siegfried/releases/download/v1.10.1/siegfried_1-10-1_win64.zip).
For MacOS: Install via homebrew by entering ```brew install richardlehane/digipres/siegfried``` into terminal,Current installations of siegfried use the config file to store preferred command options. Enter ```sf -setconf -csv -hash sha1 -z``` in terminal to set the preferred command options., For identifying a file format enter ```sf /path/to/file``` in terminal. Enter sf -help for a complete list of sf commands. diff --git a/_data/working.csv b/_data/working.csv index f4b38d7..1fe23ec 100644 --- a/_data/working.csv +++ b/_data/working.csv @@ -1,5 +1,5 @@ -name,desc,inst,conf,use +name,desc,use kryofluxmove.sh,This bash script is used by Digital Archives staff to move floppy disk images transferred using kryoflux to folders with legacy CMS media IDs of the associated digital carrier., SIPdir.sh,This bash script is used by Digital Archives staff to reate consecutive numbers of submission information packages for material transferred from digital carriers., -report_ftk_extents.py,This python script transforms XML reports exported from FTK after Processing Archivists have completed bookmarking into JSON files for import into ASpace. -report_HDD_extents.py,This python script collects the extents of finding aid components created by a Processing Archivist when processing work is done at a workstation with a hard drive. The script then produces an extents JSON file for import into ASpace. \ No newline at end of file +report_ftk_extents.py,This python script transforms XML reports exported from FTK after Processing Archivists have completed bookmarking into JSON files for import into ASpace.,The script takes two arguments: ```-f``` or ```--file``` followed by the path to the XML report to be transformed and ```o``` or ```--output``` followed by the path the destination directory for JSON output. +report_HDD_extents.py,This python script collects the extents of finding aid components created by a Processing Archivist when processing work is done at a workstation with a hard drive. The script then produces an extents JSON file for import into ASpace.,The script takes one argument: -d or --dir for the finding aid components directory on a hard drive.
Run the script following the syntax: python3 path/to/report_ftk_extents.py -d /path/to/collection/er/directory \ No newline at end of file diff --git a/tools/rclone.md b/tools/rclone.md index a5d3d61..adee0d0 100644 --- a/tools/rclone.md +++ b/tools/rclone.md @@ -1,12 +1,13 @@ --- title: Rclone layout: default -nav_order: 1 -parent: Software +nav_order: 8 +parent: Software (Draft) grand_parent: Tools +has_children: False --- -# Using Rclone: +# Rclone: [*Rclone*](https://rclone.org/) is a command line tool for managing files on cloud storage, and is one tool used by Digital Archives for moving files between Digital Archives workstations, Google Shared Drives for Divisions, and long term storage. This page lists instructions for installing and configuring rclone. ## Installing Rclone: diff --git a/tools/software_draft.md b/tools/software_draft.md index d2f5ea3..4d943ee 100644 --- a/tools/software_draft.md +++ b/tools/software_draft.md @@ -3,15 +3,19 @@ title: Software (Draft) layout: default nav_order: 5 parent: Tools -has_children: False +has_children: True --- - -# Software +## Table of contents +{: .no_toc .text-delta } {: .no_toc } +# Software Digital Archives utilizes several software throughout our workflows. This page acts as a departmental glossary for these software and includes installation, detailed use instructions and links to official documentation whenever possible. Shorter contextual instructions for these software can be found on the associated pages, consider this page our definitive guides. -## Table of contents +* Navigate to [Programs](/tools/programs.html) for a glossary of out of teh box tools used by Digital Archives staff. +* Navigate to [Working Scripts](/tools/working-scripts.html) for information on any scripts written by Digital Archives staff specific to our workflows. + + diff --git a/tools/working-scripts.md b/tools/working-scripts.md new file mode 100644 index 0000000..7312bbd --- /dev/null +++ b/tools/working-scripts.md @@ -0,0 +1,32 @@ +--- +title: Working Scripts +layout: default +nav_order: 7 +parent: Software (Draft) +grand_parent: Tools +has_children: False +--- + +# Working Scripts: + +Working Scripts are defined as scripts written internally and maintained by Digital Archives staff for use in our workflows. + +{% for tool in site.data.working %} +### {{tool.name}}: +{{tool.desc}} + +{% if tool.use%} +#### Using {{tool.name}}: +{{tool.use}} +{% endif %} + +{% endfor %} + +## Deprecated and Legacy Scripts: + +Deprecated and Legacy Scripts are any scripts which may have been previously used in workflows but are no longer in use, were planned but not completed or replaced by a Working Script. + +### ft.sh +### metadata.sh +### movemetadata script +### movephotograph script From 66f565bf12eac9fc50380af7656ce84594df4947 Mon Sep 17 00:00:00 2001 From: dreyjo <59994482+dreyjo@users.noreply.github.com> Date: Thu, 31 Aug 2023 13:35:56 -0400 Subject: [PATCH 09/17] added docs link column to programs.csv, deleted old software page, fixed links --- _data/program.csv | 50 +++---- tools/rclone.md | 2 +- tools/software.md | 296 +-------------------------------------- tools/working-scripts.md | 2 +- 4 files changed, 33 insertions(+), 317 deletions(-) diff --git a/_data/program.csv b/_data/program.csv index 6919a03..732e314 100644 --- a/_data/program.csv +++ b/_data/program.csv @@ -1,41 +1,41 @@ -name,desc,inst,conf,use +name,desc,inst,conf,use,docs 7-zip,7-zip is used in the lab to compress evidence and cases generated in Forensic Toolkit. It is also occasionally used to unpack collection material that is not supported by FTK.,[Download 7-zip](https://www.7-zip.org/){:target='_blank'}., -apple3rtr,Apple /// Ready-to-Run (apple3rtr) is a software bundle used for MAME Apple emulation. It has been used by Digital Archives staff to view Apple II files.,For detailed installation instructions visit the [apple3rtr github repo](https://github.com/datajerk/apple3rtr)., -AWS, Amazon Web Services (AWS) is a cloud computing platform offered by Amazon. AWS S3 features are used to store copies of born-digital material both before and after the collection has been processed by archivists., -Bagger, Bagger is an applciation created by the U.S. Library of Congress to produce files packages following the Bagit specification. It has been used in the Digital Archives lab for transferring born-digital material., -Bagit.py, Bagit.py is a command line tool used produce files packages following the Bagit specification, Check python version by entering ```python --version``` or ```python3 --version``` into terminal. Then install using pip by entering ```pip install bagit``` or ```pip3 install bagit```. Use the link below the main download on the VirtualBox download site to install the Extension Pack on your host., -CiderPress, CiderPress is a Windows utility for managing Apple II disk images. It has been used in the Digital Archives Lab for viewing Apple file formats., +apple3rtr,Apple /// Ready-to-Run (apple3rtr) is a software bundle used for MAME Apple emulation. It has been used by Digital Archives staff to view Apple II files.,For detailed installation instructions visit the [apple3rtr github repo](https://github.com/datajerk/apple3rtr).,,,https://github.com/datajerk/apple3rtr +AWS, Amazon Web Services (AWS) is a cloud computing platform offered by Amazon. AWS S3 features are used to store copies of born-digital material both before and after the collection has been processed by archivists.,,,,https://docs.aws.amazon.com/ +Bagger, Bagger is an applciation created by the U.S. Library of Congress to produce files packages following the Bagit specification. It has been used in the Digital Archives lab for transferring born-digital material.,,,,https://github.com/LibraryOfCongress/bagger +Bagit.py, Bagit.py is a command line tool used produce files packages following the Bagit specification, Check python version by entering ```python --version``` or ```python3 --version``` into terminal. Then install using pip by entering ```pip install bagit``` or ```pip3 install bagit```. Use the link below the main download on the VirtualBox download site to install the Extension Pack on your host.,,,https://github.com/LibraryOfCongress/bagit-python +CiderPress, CiderPress is a Windows utility for managing Apple II disk images. It has been used in the Digital Archives Lab for viewing Apple file formats.,,,,https://a2ciderpress.com dbpoweramp, dbpoweramp is a program used in conjunction with [Iromlab](#Iromlab) for transferring born-digital material from optical media., -dd, dd is a command line utility used in the Digital Archives Lab to create disk images of optical media. In most cases transfers are made of optical media instead of disk images., -Disk Utility, Disk Utility is a MacOS utility for disk volume management. Disk Utility used by Digital Archives staff for mounting/unmounting remvable media.,,,enter ```list disk``` or ```list volume``` to confirm media is mounted., -diskpart, diskpart is Windows command line utility for disk partitioning. diskpart is used by Digital Archives staff for confimring removable media are properly mounted., -disktype, disktype is a command line tool used by Digital Archives staff for extracting filesystem metadata liek sector format for floppy disks, For Mac OS:
In terminal use homebrew to install by entering ```brew install disktype```,, In terminal navigate to the directory containing floppy disk image(s) and enter ```disktype a/disk/image```., -DosBox, DosBox is an open-source emulator for teh MS-DOS operating system. DosBox has been used by Digital Archives staff to view DOS executables and word processing files., -DROID, DROID is file format identification tool developed by the UK National Archives., -Emailchemy, Emailchemy has been used in the Digital Archives lab for migrating email fomats., -ePADD, ePADD is an open-source program developed by Stanford University's Special Collections and University Archives. The software ha been used for appraisal and arrangement of email., +dd, dd is a command line utility used in the Digital Archives Lab to create disk images of optical media. In most cases transfers are made of optical media instead of disk images.,,,,https://linux.die.net/man/1/dd +Disk Utility, Disk Utility is a MacOS utility for disk volume management. Disk Utility used by Digital Archives staff for mounting/unmounting remvable media.,,,enter ```list disk``` or ```list volume``` to confirm media is mounted.,https://support.apple.com/guide/disk-utility/welcome/mac +diskpart, diskpart is Windows command line utility for disk partitioning. diskpart is used by Digital Archives staff for confimring removable media are properly mounted.,,,,https://learn.microsoft.com/en-us/windows-server/administration/windows-commands/diskpart +disktype, disktype is a command line tool used by Digital Archives staff for extracting filesystem metadata liek sector format for floppy disks, For Mac OS:
In terminal use homebrew to install by entering ```brew install disktype```,, In terminal navigate to the directory containing floppy disk image(s) and enter ```disktype a/disk/image```.,https://disktype.sourceforge.net/doc/ +DosBox, DosBox is an open-source emulator for teh MS-DOS operating system. DosBox has been used by Digital Archives staff to view DOS executables and word processing files.,,,,https://www.dosbox.com/DOSBoxManual.html +DROID, DROID is file format identification tool developed by the UK National Archives.,,,,https://cdn.nationalarchives.gov.uk/documents/information-management/droid-user-guide.pdf +Emailchemy, Emailchemy has been used in the Digital Archives lab for migrating email fomats.,,,,https://www.weirdkid.com/products/emailchemy/doc/Emailchemy_User_Manual.pdf +ePADD, ePADD is an open-source program developed by Stanford University's Special Collections and University Archives. The software ha been used for appraisal and arrangement of email.,,,,https://github.com/ePADD/epadd/releases Filemaker Pro 17, Filemaker is used by Preservation and Collection Services (PCP) as a collection management system. born-digital media is inventoried in Filemaker and PCP tracks projects through Filemaker., Installing Filemaker requires license keys. Make sure you have the proper keys and permissions before installing., -Forensic Toolkit, Forensic Toolkit is a computer forensics software used in the Digital Archives Lab to appraise and arrange born-digital material., +Forensic Toolkit, Forensic Toolkit is a computer forensics software used in the Digital Archives Lab to appraise and arrange born-digital material. FTK Imager, FTK Imager is a disk imaging program associated with Forensic Toolkit. The software has been used in the Digital Archives lab to image and extract files from digital carriers., -gtar, Gtar is a command line program used on Digital Archives MacOS workstations to compress Kryoflux stream files., For MacOS: Install via Homebrew by entering ```brew install gnu-tar``` in terminal., -HFS Explorer, HFS Explorer is a program usd to view HFS formatted disks on Windows operating systems., -Iromlab, Iromlab is software for automated imaging of optical media using [Nimbie](#tools/nimbie) disc robots., +gtar, Gtar is a command line program used on Digital Archives MacOS workstations to compress Kryoflux stream files., For MacOS: Install via Homebrew by entering ```brew install gnu-tar``` in terminal.,,,https://www.gnu.org/software/tar/manual/tar.html +HFS Explorer, HFS Explorer is a program usd to view HFS formatted disks on Windows operating systems.,,,,https://catacombae.org/hfsexplorer/ +Iromlab, Iromlab is software for automated imaging of optical media using [Nimbie](#tools/nimbie) disc robots.,,,,https://catacombae.org/hfsexplorer/ IsoBuster, IsoBuster is data recovery software that has been used in the Digital Archives lab to image and extract files from optical media., Java, Java is a software package which has baeen a dependency for some Digital Archives tools like DROID and Bagger,,, Check for the version of Java installed with commands ```which java``` or ```java -version``` in terminal on Mac or WSL on Windows., MediaInfo, MediaInfo is an open-source program for extractign technical information from media files. The program has been used in the Digital Archives Lab to extract metadata from born-digital video., Mediaconch, MediaConch is an open source software project developed by MediaArea and utilizing MediaInfo. It is used by Digital Archives staff to develop programatic policies for checking born-digital audiovisual material against preservation standards., -Mini Vmac, Mini vMac is an emulator for running early Macintosh software., -openssl, OpenSSL is a software library used to ssh., -Oracle VM VirtualBox, Virtualbox is open soure virtualization software. It has been used by Digital Archives staff to run teh BitCurator environment as a virtual machine., Download and install the [current VirtualBox release and Extension Pack](https://www.virtualbox.org/wiki/Downloads)., +Mini Vmac, Mini vMac is an emulator for running early Macintosh software. +openssl, OpenSSL is a software library used to ssh.,,,,https://www.openssl.org/docs +Oracle VM VirtualBox, Virtualbox is open soure virtualization software. It has been used by Digital Archives staff to run teh BitCurator environment as a virtual machine., Download and install the [current VirtualBox release and Extension Pack](https://www.virtualbox.org/wiki/Downloads). pip, Pip is package management software used for installing and upgrading Python modules,,,Pip should be installed with python3. Call pip by entering ```pip``` or ```pip3``` in terminal or command line., Python3, Python3 is the latest version of the Python programming language, For Windows: Check for python versions by entering ```python --version``` on command line. Installers are avilable at python.org.
For MacOS: Check for python versions by entering ```python --version``` in terminal. Install your selected version of python with homebrew by entering ```brew install python@version.number```., QuickTime Player, QuickTime is used in teh Digital Archives Lab to appraise Mcintosh formatted born-digital video., QuickView Plus, QuickView Plus has been used in the Digital Archives Lab to view a range of legacy formats like word processing and photo formats., -rclone,rclone is a command line tool for managing and moving files between remote and local storage locations,For detailed installation and configuration instructions [see our dedicated rclone page](/tools/rclone.html). -Sha1deep, Sha1deep is a command line program for computing and comparing SHA-256 cryptographic hashes. Sha1deep is sued by Digital Archives Staff to identify duplicate material across digital media in a collection by comparing checksums, For MacOS: Install via Homebrew by entering ```brew install m5deep```, +rclone,rclone is a command line tool for managing and moving files between remote and local storage locations,For detailed installation and configuration instructions [see our dedicated rclone page](rclone.html).,,,https://rclone.org/docs/ +Sha1deep, Sha1deep is a command line program for computing and comparing SHA-256 cryptographic hashes. Sha1deep is sued by Digital Archives Staff to identify duplicate material across digital media in a collection by comparing checksums, For MacOS: Install via Homebrew by entering ```brew install m5deep```,,,https://linux.die.net/man/1/sha1deep Sheepshaver, Sheepshaver is an Apple Macintosh emulator. It has been used in teh Digital Archives Lab to view legacy Mac files., -Siegfried, Siegfried is a command line file format identification tool. Siegfried is used in the Digital Archives Lab to identify and validate file formats. Siegfried output also includes checksums., For Windows: Download [the latest Siegfried binary](https://github.com/richardlehane/siegfried/releases/download/v1.10.1/siegfried_1-10-1_win64.zip).
For MacOS: Install via homebrew by entering ```brew install richardlehane/digipres/siegfried``` into terminal,Current installations of siegfried use the config file to store preferred command options. Enter ```sf -setconf -csv -hash sha1 -z``` in terminal to set the preferred command options., For identifying a file format enter ```sf /path/to/file``` in terminal. Enter sf -help for a complete list of sf commands. +Siegfried, Siegfried is a command line file format identification tool. Siegfried is used in the Digital Archives Lab to identify and validate file formats. Siegfried output also includes checksums., For Windows: Download [the latest Siegfried binary](https://github.com/richardlehane/siegfried/releases/download/v1.10.1/siegfried_1-10-1_win64.zip).
For MacOS: Install via homebrew by entering ```brew install richardlehane/digipres/siegfried``` into terminal,Current installations of siegfried use the config file to store preferred command options. Enter ```sf -setconf -csv -hash sha1 -z``` in terminal to set the preferred command options., For identifying a file format enter ```sf /path/to/file``` in terminal. Enter sf -help for a complete list of sf commands.,https://www.itforarchivists.com/siegfried/ tree, tree is command line program for recursive directory listing and has been used in the Digital Archives Lab to identify contents of media including number of folders and files, For MacOS: Install via homebrew by entering ```brew install tree``` into terminal., VLC, VLC is an open source media player used by Digital Archives staff to view and appraise born-digital audio and video., Download [the latest release of VLC Media Player](https://www.videolan.org/vlc/)., -WSL, Windows Subsystem for Linux (WSL) is a Windows feature used to run Linux environments on Windows operating systems, +WSL, Windows Subsystem for Linux (WSL) is a Windows feature used to run Linux environments on Windows operating systems. WinUAE, WinUAE is an Amiga emulator and has been used in the Digital Archives Lab to view Amiga files., \ No newline at end of file diff --git a/tools/rclone.md b/tools/rclone.md index adee0d0..193a9fa 100644 --- a/tools/rclone.md +++ b/tools/rclone.md @@ -2,7 +2,7 @@ title: Rclone layout: default nav_order: 8 -parent: Software (Draft) +parent: Software grand_parent: Tools has_children: False --- diff --git a/tools/software.md b/tools/software.md index babbd6a..98d9de7 100644 --- a/tools/software.md +++ b/tools/software.md @@ -1,300 +1,16 @@ --- title: Software layout: default -nav_order: 3 +nav_order: 5 parent: Tools has_children: True --- - - -# Software -{: .no_toc } - ## Table of contents {: .no_toc .text-delta } +{: .no_toc } -1. TOC -{:toc} - -## Types of Software -The following is a list of software used in born-digital workflows at NYPL. -### 7-zip -7-zip is used in the lab to compress evidence and cases generated in Forensic Toolkit. It is also occasionally used to unpack collection material that is not supported by FTK. -#### Install 7-zip binary -[Download 7-zip](https://www.7-zip.org/){:target="_blank"} - -### AWS -Amazon Web Services features S3 and Glacier are used to store copies of born-digital material both before and after the collection has been processed by archivists. - -Using the AWS dashboard requires a login. - -### Bagit.py -Bagit.py is used to transfer born-digital material to storage. Bagit creates file manifests that help to ensure transfers are complete and free of errors. - -#### Installation -Check python version -``` python --version ``` -or -``` python3 --version ``` -then install -``` pip install bagit ``` -or -``` pip3 install bagit ``` - -### BitCurator -BitCurator is a Linux distribution with a suite of open source digital forensics and data analysis tools used in the lab as both a virtual machine and as bootable OS alongside Windows. BitCurator is used for transfer and metadata extraction, especially for formats unrecognized in Windows and Mac environments. - -#### Installation -Download software -[BitCurator VM or Live CD](https://github.com/BitCurator/bitcurator-distro/wiki/Releases){:target="_blank"} - -[Current release of VirtualBox](https://www.virtualbox.org/wiki/Downloads){:target="_blank"} - -The VirtualBox Extension Pack must be installed on the host for shared folder support, USB 2.0/3.0 support, and support for some NVMe SSDs. Once you’ve installed VirtualBox, use the link below the main download on VirtualBox download site to install the Extension Pack on your host. - -Unpack the BitCurator Virtual Machine. - -Once you’ve installed VirtualBox and the VirtualBox extension pack, start up VirtualBox. - -Add Virtual Machine -From the menu bar, select the menu item “Machine -> Add…”, and navigate to the folder containing .vbox file that you extracted. Choose that file, and the Virtual Machine should appear in the list within the manager. - -### dBpoweramp -dBpoweramp is used in conjunction with Iromlab and the Nimbie to transfer born-digital material from optical media to storage. - -### dd -The command line utility dd is used in the lab to create disk images of optical media. In most cases transfers are made of optical media instead of disk images. -#### Use -In the terminal type -``` dd if=/dev/sr0 of=/cygdrive/z/ingest/diskImages/CollID/MediaID/objects/MediaID.iso ``` -Hit return -Once complete the terminal will display bytes copied and return to a blank prompt. - -### Disk Utility -On MacOS Disk Utility is used in the lab to confirm that removable media is properly mounted. - -### DiskPart -On Windows DiskPart command line utility is used in the lab to confirm that removable media is properly mounted. -#### Use -Type -``` list disk ``` -or -``` list volume ``` -to confirm media is mounted. - -### disktype -Disktype is used on MacOS in the lab to extract filesystem metadata such as sector format for floppy disks. - -#### Installation -``` brew install disktype ``` -#### Use - ``` cd ``` - ``` diskype ``` - -### Filemaker Pro 17 -Filemaker is used by Preservation and Collection Services as a collection management system. Born-digital media is inventoried in Filemaker and PCP tracks projects through Filemaker. -#### Installation -Installing Filemaker requires license keys. Make sure you have the proper keys and permissions before installing. -#### Use - -### Forensic Toolkit -Forensic Toolkit is used in the lab to appraise, and arrange born-digital material. -#### Installation -Forensic Toolkit is licensed software and requires a dongle to open. The software can be downloaded from [AccessData](https://accessdata.com/product-download){:target="_blank"} -#### Use - -### gtar -Gtar is used on a Mac in the lab to compress KryoFlux stream files. It is called through the command line with the bash alias 'compress'. -#### Installation -``` brew install gnu-tar ``` -### Iromlab -Iromlab is used in conjunction with dBpoweramp and the Nimbie to transfer born-digital material from optical media to storage. -### Java -Java is required for tools like DROID and Bagger. -#### Use -Try commands -``` which java ``` -and -``` java -version ``` -if you are encountering java errors. You may need to put the path of the correct version of java in your bash profile. -In Windows you can use the search from the start menu to check the java version. Type 'about java' into search. Then click the About Java icon result. -### openssl -openssl is used in the lab to SSH between computers. SSH is used to run workflow scripts located on ARCHV Mac. -### Oracle VM VirtualBox -VirtualBox is used in the lab to run the BitCurator environment as a virtual machine. -#### Installation -Current release of [VirtualBox](https://www.virtualbox.org/wiki/Downloads){:target="_blank"} - -The VirtualBox Extension Pack must be installed on the host for shared folder support, USB 2.0/3.0 support, and support for some NVMe SSDs. Once you’ve installed VirtualBox, use the link below the main download on VirtualBox download site to install the Extension Pack on your host. -### pip -Pip is used in the lab to install and upgrade Python modules like bagit.py. -#### Use -Pip should be installed with python3. Call pip with -``` pip``` or ``` pip3 ``` -depending on your installation. -Check python version -``` python --version ``` -or -``` python3 --version ``` -### Python3 -Python3 is used in the lab to run a a variety of scripts, including bagit.py. -#### Installation -Windows: -Check python version -``` python --version ``` -Installers are available at [python.org](https://docs.python.org/3.8/using/windows.html#windows-full){:target="_blank"} - -Mac: -Check python version -``` python3 --version ``` -Replace the number below with the version you wish to install. -``` brew install python@3.8 ``` -BitCurator: -Check python version -``` python3 --version ``` -Replace the number below with the version you wish to install. -``` sudo apt-get install python3.8 ``` -### QuickTime Player -QuickTime is used in the lab to appraise Macintosh formatted born-digital video. - -### rclone -Rclone is used in the lab to transfer born-digital material between for Digital Archives workstations, Google Shared Drives for Divisions, and long term storage. -#### Installation -Windows: -[Download and install binary](https://rclone.org/downloads/){:target="_blank"} - -Mac: -```brew install rclone``` -### rsync -Rsync is used in the lab to transfer born-digital material when bagit.py is unable to complete a transfer. Rsync is used when transferring from a computer backup or a drive with a large number of system files. -#### Installation -Mac: -``` brew install rsync``` -### Siegfried -Siegfried is used in the lab to identify and validate file formats. Siegfried output also includes checksums, useful when rsync is used for transfer instead of bagit.py. -#### Installation -Windows: -[Download binary](https://github.com/richardlehane/siegfried/releases/download/v1.8.0/siegfried_1-8-0_win64.zip){:target="_blank"} -Mac: -```brew install richardlehane/digipres/siegfried``` - -#### Use -Current installations of siegfried use the config file to store preferred command options. -``` sf -setconf -csv -hash sha1 -z ``` -sets the preferred command options. -Find a complete list of sf commands with -``` sf -help ``` -### Sha1deep -Sha1deep is used in the lab to identify duplicate material across digital media in a collection. It is usually used in bash scripts. -#### Installation -Mac: -``` brew install m5deep ``` -#### Use -Here is an example of how to use sha1deep to transfer only unique files from media. -Write checksums of transfer volume to file -``` sha1deep -r [directory or volume] >> collnumber.sha1 ``` -Write path of unique files per directory or volume to file with mediaid -``` sha1deep -rx [path to checksums file] [directory or volume] >> [path to checksums for media] ``` -Remove ds_store from unique file list -```sed '/.DS_Store/d' [path to file list] > [file list]``` -change terminal path out of home directory -```cd ../../``` -Navigate to the media being transferred to avoid /Volumes/HD/folders being copied within media-id directory. -Make sure spaces are not escaped in paths. “ “ not “\ “ -Transfer unique files -```rsync -aPF --files-from=[path to file list] . [path to media]``` -### tree -Tree is used in bash scripts in the lab to identify contents of media including number of folders and files. -#### Installation -Mac: -```brew install tree``` -### VLC -VLC multimedia player is used in the lab to view and appraise born digital audio and video. -#### Installation -https://www.videolan.org/vlc/index.html -## Scripts -The following is list of scripts used in born-digital workflows at NYPL. The scripts are located in the [digarch_scripts](https://github.com/NYPL/digarch_scripts/blob/main/){:target="_blank"} Github repository. A link to each script is provided after the description. -### ft.sh -This script is used in the lab to create bagit transfers and Siegfried csvs for born-digital media. -[ft.sh ](https://github.com/NYPL/digarch_scripts/blob/main/Mac/ft.sh){:target="_blank"} -### kryofluxmove.sh -This script is used in the lab to move KryoFlux disk images to folders with the MediaIDs of the disks. -[kryofluxmove.sh](https://github.com/NYPL/digarch_scripts/blob/main/Mac/kryofluxmove.sh){:target="_blank"} -### metadata.sh -This script is used in the lab to move metadata from disk images to folders with the MediaIDs of the disks. -[metadata.sh](https://github.com/NYPL/digarch_scripts/blob/main/Mac/metadata.sh){:target="_blank"} -### Brunnhilde with disktype -This script is used in BitCurator to identify disk image type and pass the type to Brunnhilde to generate the correct output. -[Brunnhilde with disktype](https://github.com/NYPL/digarch_scripts/blob/main/BitCurator/brudt.sh){:target="_blank"} -### makesips script -This script is used to create a consecutive number of submission information packages for material from digital media. -[makesips script](https://github.com/NYPL/digarch_scripts/blob/main/Mac/SIPdir.sh){:target="_blank"} -Alternatively, ``mkdir`` command can be used to create SIPs. This works when SIPs aren't consecutively numbered. 0001 to 0009 require a different line from 0010 on. -```mkdir -p CollID/Media-000{1..9}/{metadata/submissionDocumentation,objects}``` -```mkdir -p CollID/Media-00{10..99}/{metadata/submissionDocumentation,objects}``` -```mkdir -p CollID/Media-000{1,5,7,9}/{metadata/submissionDocumentation,objects}``` -### moveimages script -This script is used in the lab to move a disk image into the objects folder of a SIP. -[moveimages script](https://github.com/NYPL/digarch_scripts/blob/main/Mac/kryofluxmove.sh){:target="_blank"} -### movemetadata script -This script is used in the lab to move metadata into the metadata folder of SIP. -[movemetadata script](https://github.com/NYPL/digarch_scripts/blob/main/Mac/metadata.sh){:target="_blank"} -### movephotograph script -This script is used in the lab to copy JPEGS of media from the photographs directory on the staging drive to the submissionDocumentation directory in the MediaID directory of SIPs. -[movephotograph script](https://github.com/NYPL/digarch_scripts/blob/main/Mac/movephotograph.sh){:target="_blank"} -### FACTools -This script is used in the lab to repackage Finding Aid Component packages. -[FACTools](https://github.com/NYPL/digarch_scripts/blob/main/Mac/qctools.sh){:target="_blank"} -### report ftk extents script -This script transforms XML reports exported from FTK after Processing Archivists have completed bookmarking into JSON files for import into ASpace. The script takes two arguments. -* ```-f``` or ```--file```: this argument accepts the path to the XML report to be transformed. -* ```-o``` or ```--output```: this arguments accepts the path to the destination directory for the transformed JSON. - -Run the script following the syntax: ```python3 path/to/report_ftk_extents.py -f /path/to/xml/report -o /path/to/json/destination/directory``` - -### report HDD extents script -This script collects the extents of finding aid components created by a Processing Archivist when processing work is done at a workstation with a hard drive. The script then produces an extents JSON file for import into ASpace. - -The script takes one argument: ```-d``` or ```--dir``` for the finding aid components directory on a hard drive. - -Run the script following the syntax: -```python3 path/to/report_ftk_extents.py -d /path/to/collection/er/directory``` +# Software +Digital Archives utilizes several software throughout our workflows. This page acts as a departmental glossary for these software and includes installation, detailed use instructions and links to official documentation whenever possible. Shorter contextual instructions for these software can be found on the associated workflow pages, consider this page our definitive guides. -### iterative scripts -Scripts are created per collection using a directory listing as input in a while loop. This allows previous move scripts to run per collection rather than one MediaID. -### CMS metadata import -This script is used in the lab to import metadata from transfers to the FileMaker Collection Management System. Currently, output from the Nimbie (Iromlab) is used. -## Previous and Occasional Software -The following is a list of software used in previous workflows. These tools can be used when transfer fails with current tools. -### Emailchemy -Emailchemy has been used in the lab to migrate email formats. -### ePADD -ePADD has been used in the lab to appraise and arrange email. -### DROID -DROID has been used in the lab to identify and validate file formats. -### MediaInfo -MediaInfo has been used in the lab to extract metadata from born-digital video. -### FTK Imager -FTK Imager has been used in the lab to image media and extract files from media. -### IsoBuster -IsoBuster has been used in the lab to image optical media and extract files from media. -### Bagger -Bagger has been used in the lab to transfer born-digital material. -## Legacy Format Appraisal -The following is a list of software used in the lab to view legacy formats not supported by FTK. -### WinUAE -WnUAE has been used in the lab to view Amiga files. -### apple3rtr -Apple3rtr has been used in the lab to view Apple II files. -### miniVmac -MiniVmac has been used in the lab to view legacy Mac files. -### DosBox -DosBox has been used in the to view DOS executables and word processing files. -### QuickView Plus -QuickView Plus has been used in the lab to view a range of legacy formats like word processing and photo formats. -### CiderPress -CiderPress has been used in the lab to view Apple file formats. -### HFS Explorer -HFS Explorer has been used in the lab to view files on HFS filesystems. -### Sheepshaver -Sheepshaver has been used in the lab to view legacy Mac files. \ No newline at end of file +* Navigate to [Programs](programs.html) for a glossary of out of the box tools used by Digital Archives staff. +* Navigate to [Working Scripts](working-scripts.html) for information on any scripts written by Digital Archives staff specific to our workflows. \ No newline at end of file diff --git a/tools/working-scripts.md b/tools/working-scripts.md index 7312bbd..db38579 100644 --- a/tools/working-scripts.md +++ b/tools/working-scripts.md @@ -2,7 +2,7 @@ title: Working Scripts layout: default nav_order: 7 -parent: Software (Draft) +parent: Software grand_parent: Tools has_children: False --- From 402e413ec1608567c601126439ccf7d0c85f278a Mon Sep 17 00:00:00 2001 From: dreyjo <59994482+dreyjo@users.noreply.github.com> Date: Thu, 31 Aug 2023 13:39:12 -0400 Subject: [PATCH 10/17] adding programs page --- tools/programs.md | 37 +++++++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+) create mode 100644 tools/programs.md diff --git a/tools/programs.md b/tools/programs.md new file mode 100644 index 0000000..965c72a --- /dev/null +++ b/tools/programs.md @@ -0,0 +1,37 @@ +--- +title: Programs +layout: default +nav_order: 6 +parent: Software +grand_parent: Tools +has_children: False +--- + +# Programs: + +Programs are defined as any non-internally developed out-of-the-box software used in Digital Archives workflows. Many programs used by Digital Archives staff can be installed via Homebrew or by downloading installation files from the associated website. Programs requiring more intensive installation and configuration instructions have been given dedicated pages. + +{% for tool in site.data.program %} + +## {{tool.name}}: +{{tool.desc}} + +{% if tool.docs %} +[See Official Documentation]({{tool.docs}}) +{% endif %} + +{% if tool.inst%} +### Installing {{tool.name}}: +{{tool.inst}} +{% endif %} + +{% if tool.conf%} +### Configuring {{tool.name}}: +{{tool.conf}} +{% endif %} + +{% if tool.use%} +### Using {{tool.name}}: +{{tool.use}} +{% endif %} +{% endfor %} \ No newline at end of file From 62eed1f10ae7aae7b25bf41b7a055348b438979e Mon Sep 17 00:00:00 2001 From: dreyjo <59994482+dreyjo@users.noreply.github.com> Date: Thu, 31 Aug 2023 13:41:07 -0400 Subject: [PATCH 11/17] deleting software draft --- tools/software_draft.md | 42 ----------------------------------------- 1 file changed, 42 deletions(-) delete mode 100644 tools/software_draft.md diff --git a/tools/software_draft.md b/tools/software_draft.md deleted file mode 100644 index 4d943ee..0000000 --- a/tools/software_draft.md +++ /dev/null @@ -1,42 +0,0 @@ ---- -title: Software (Draft) -layout: default -nav_order: 5 -parent: Tools -has_children: True ---- -## Table of contents -{: .no_toc .text-delta } -{: .no_toc } - -# Software -Digital Archives utilizes several software throughout our workflows. This page acts as a departmental glossary for these software and includes installation, detailed use instructions and links to official documentation whenever possible. Shorter contextual instructions for these software can be found on the associated pages, consider this page our definitive guides. - -* Navigate to [Programs](/tools/programs.html) for a glossary of out of teh box tools used by Digital Archives staff. -* Navigate to [Working Scripts](/tools/working-scripts.html) for information on any scripts written by Digital Archives staff specific to our workflows. - - From cff1d08ec42651284d2d8ae71b00a6ca24ae42ee Mon Sep 17 00:00:00 2001 From: arhonemus <34036628+arhonemus@users.noreply.github.com> Date: Thu, 7 Sep 2023 17:31:12 -0400 Subject: [PATCH 12/17] copy edited program.csv --- _data/program.csv | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/_data/program.csv b/_data/program.csv index 732e314..0a65c48 100644 --- a/_data/program.csv +++ b/_data/program.csv @@ -9,8 +9,8 @@ dbpoweramp, dbpoweramp is a program used in conjunction with [Iromlab](#Iromlab) dd, dd is a command line utility used in the Digital Archives Lab to create disk images of optical media. In most cases transfers are made of optical media instead of disk images.,,,,https://linux.die.net/man/1/dd Disk Utility, Disk Utility is a MacOS utility for disk volume management. Disk Utility used by Digital Archives staff for mounting/unmounting remvable media.,,,enter ```list disk``` or ```list volume``` to confirm media is mounted.,https://support.apple.com/guide/disk-utility/welcome/mac diskpart, diskpart is Windows command line utility for disk partitioning. diskpart is used by Digital Archives staff for confimring removable media are properly mounted.,,,,https://learn.microsoft.com/en-us/windows-server/administration/windows-commands/diskpart -disktype, disktype is a command line tool used by Digital Archives staff for extracting filesystem metadata liek sector format for floppy disks, For Mac OS:
In terminal use homebrew to install by entering ```brew install disktype```,, In terminal navigate to the directory containing floppy disk image(s) and enter ```disktype a/disk/image```.,https://disktype.sourceforge.net/doc/ -DosBox, DosBox is an open-source emulator for teh MS-DOS operating system. DosBox has been used by Digital Archives staff to view DOS executables and word processing files.,,,,https://www.dosbox.com/DOSBoxManual.html +disktype, disktype is a command line tool used by Digital Archives staff for extracting filesystem metadata like sector format for floppy disks, For Mac OS:
In terminal use homebrew to install by entering ```brew install disktype```,, In terminal navigate to the directory containing floppy disk image(s) and enter ```disktype a/disk/image```.,https://disktype.sourceforge.net/doc/ +DosBox, DosBox is an open-source emulator for the MS-DOS operating system. DosBox has been used by Digital Archives staff to view DOS executables and word processing files.,,,,https://www.dosbox.com/DOSBoxManual.html DROID, DROID is file format identification tool developed by the UK National Archives.,,,,https://cdn.nationalarchives.gov.uk/documents/information-management/droid-user-guide.pdf Emailchemy, Emailchemy has been used in the Digital Archives lab for migrating email fomats.,,,,https://www.weirdkid.com/products/emailchemy/doc/Emailchemy_User_Manual.pdf ePADD, ePADD is an open-source program developed by Stanford University's Special Collections and University Archives. The software ha been used for appraisal and arrangement of email.,,,,https://github.com/ePADD/epadd/releases @@ -21,8 +21,8 @@ gtar, Gtar is a command line program used on Digital Archives MacOS workstations HFS Explorer, HFS Explorer is a program usd to view HFS formatted disks on Windows operating systems.,,,,https://catacombae.org/hfsexplorer/ Iromlab, Iromlab is software for automated imaging of optical media using [Nimbie](#tools/nimbie) disc robots.,,,,https://catacombae.org/hfsexplorer/ IsoBuster, IsoBuster is data recovery software that has been used in the Digital Archives lab to image and extract files from optical media., -Java, Java is a software package which has baeen a dependency for some Digital Archives tools like DROID and Bagger,,, Check for the version of Java installed with commands ```which java``` or ```java -version``` in terminal on Mac or WSL on Windows., -MediaInfo, MediaInfo is an open-source program for extractign technical information from media files. The program has been used in the Digital Archives Lab to extract metadata from born-digital video., +Java, Java is a software package which has been a dependency for some Digital Archives tools like DROID and Bagger.,,, Check for the version of Java installed with commands ```which java``` or ```java -version``` in terminal on Mac or WSL on Windows., +MediaInfo, MediaInfo is an open-source program for extracting technical information from media files. The program has been used in the Digital Archives Lab to extract metadata from born-digital video., Mediaconch, MediaConch is an open source software project developed by MediaArea and utilizing MediaInfo. It is used by Digital Archives staff to develop programatic policies for checking born-digital audiovisual material against preservation standards., Mini Vmac, Mini vMac is an emulator for running early Macintosh software. openssl, OpenSSL is a software library used to ssh.,,,,https://www.openssl.org/docs @@ -33,8 +33,8 @@ QuickTime Player, QuickTime is used in teh Digital Archives Lab to appraise Mcin QuickView Plus, QuickView Plus has been used in the Digital Archives Lab to view a range of legacy formats like word processing and photo formats., rclone,rclone is a command line tool for managing and moving files between remote and local storage locations,For detailed installation and configuration instructions [see our dedicated rclone page](rclone.html).,,,https://rclone.org/docs/ Sha1deep, Sha1deep is a command line program for computing and comparing SHA-256 cryptographic hashes. Sha1deep is sued by Digital Archives Staff to identify duplicate material across digital media in a collection by comparing checksums, For MacOS: Install via Homebrew by entering ```brew install m5deep```,,,https://linux.die.net/man/1/sha1deep -Sheepshaver, Sheepshaver is an Apple Macintosh emulator. It has been used in teh Digital Archives Lab to view legacy Mac files., -Siegfried, Siegfried is a command line file format identification tool. Siegfried is used in the Digital Archives Lab to identify and validate file formats. Siegfried output also includes checksums., For Windows: Download [the latest Siegfried binary](https://github.com/richardlehane/siegfried/releases/download/v1.10.1/siegfried_1-10-1_win64.zip).
For MacOS: Install via homebrew by entering ```brew install richardlehane/digipres/siegfried``` into terminal,Current installations of siegfried use the config file to store preferred command options. Enter ```sf -setconf -csv -hash sha1 -z``` in terminal to set the preferred command options., For identifying a file format enter ```sf /path/to/file``` in terminal. Enter sf -help for a complete list of sf commands.,https://www.itforarchivists.com/siegfried/ +Sheepshaver, Sheepshaver is an Apple Macintosh emulator. It has been used in the Digital Archives Lab to view legacy Mac files., +Siegfried, Siegfried is a command line file format identification tool. Siegfried is used in the Digital Archives Lab to identify and validate file formats. Siegfried output also includes checksums., For Windows: Download [the latest Siegfried binary](https://github.com/richardlehane/siegfried/releases/download/v1.10.1/siegfried_1-10-1_win64.zip).
For MacOS: Install via homebrew by entering ```brew install richardlehane/digipres/siegfried``` into terminal,Current installations of siegfried use the config file to store preferred command options. Enter ```sf -setconf -csv -hash sha1 -z``` in terminal to set the preferred command options., Enter ```sf PATH > MediaID``` in terminal to create a report of file formats. Enter ```sf -help``` for a complete list of sf commands.,https://www.itforarchivists.com/siegfried/ tree, tree is command line program for recursive directory listing and has been used in the Digital Archives Lab to identify contents of media including number of folders and files, For MacOS: Install via homebrew by entering ```brew install tree``` into terminal., VLC, VLC is an open source media player used by Digital Archives staff to view and appraise born-digital audio and video., Download [the latest release of VLC Media Player](https://www.videolan.org/vlc/)., WSL, Windows Subsystem for Linux (WSL) is a Windows feature used to run Linux environments on Windows operating systems. From e9daac2e5d33ca8fc526e2f94f2e9df334989d13 Mon Sep 17 00:00:00 2001 From: arhonemus <34036628+arhonemus@users.noreply.github.com> Date: Thu, 7 Sep 2023 17:53:57 -0400 Subject: [PATCH 13/17] added m5deep to sha1deep summary --- _data/program.csv | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_data/program.csv b/_data/program.csv index 0a65c48..36d984b 100644 --- a/_data/program.csv +++ b/_data/program.csv @@ -32,7 +32,7 @@ Python3, Python3 is the latest version of the Python programming language, For W QuickTime Player, QuickTime is used in teh Digital Archives Lab to appraise Mcintosh formatted born-digital video., QuickView Plus, QuickView Plus has been used in the Digital Archives Lab to view a range of legacy formats like word processing and photo formats., rclone,rclone is a command line tool for managing and moving files between remote and local storage locations,For detailed installation and configuration instructions [see our dedicated rclone page](rclone.html).,,,https://rclone.org/docs/ -Sha1deep, Sha1deep is a command line program for computing and comparing SHA-256 cryptographic hashes. Sha1deep is sued by Digital Archives Staff to identify duplicate material across digital media in a collection by comparing checksums, For MacOS: Install via Homebrew by entering ```brew install m5deep```,,,https://linux.die.net/man/1/sha1deep +Sha1deep, Sha1deep is a command line tool for computing and cryptographic hashes. (M5deep | sha1deep | sha256deep are a set of programs to compute MD5 | SHA-1 | SHA-256 message digests on files.) Sha1deep is used by Digital Archives Staff to identify duplicate material across digital media in a collection by comparing checksums., For MacOS: Install via Homebrew by entering ```brew install m5deep```,,,https://linux.die.net/man/1/sha1deep Sheepshaver, Sheepshaver is an Apple Macintosh emulator. It has been used in the Digital Archives Lab to view legacy Mac files., Siegfried, Siegfried is a command line file format identification tool. Siegfried is used in the Digital Archives Lab to identify and validate file formats. Siegfried output also includes checksums., For Windows: Download [the latest Siegfried binary](https://github.com/richardlehane/siegfried/releases/download/v1.10.1/siegfried_1-10-1_win64.zip).
For MacOS: Install via homebrew by entering ```brew install richardlehane/digipres/siegfried``` into terminal,Current installations of siegfried use the config file to store preferred command options. Enter ```sf -setconf -csv -hash sha1 -z``` in terminal to set the preferred command options., Enter ```sf PATH > MediaID``` in terminal to create a report of file formats. Enter ```sf -help``` for a complete list of sf commands.,https://www.itforarchivists.com/siegfried/ tree, tree is command line program for recursive directory listing and has been used in the Digital Archives Lab to identify contents of media including number of folders and files, For MacOS: Install via homebrew by entering ```brew install tree``` into terminal., From 6b014b4e00391d43607463b7b0d7f0dcf44cbd7f Mon Sep 17 00:00:00 2001 From: arhonemus <34036628+arhonemus@users.noreply.github.com> Date: Fri, 8 Sep 2023 10:56:24 -0400 Subject: [PATCH 14/17] added tool.name to linkt text, removed new tab --- _data/program.csv | 2 +- tools/programs.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/_data/program.csv b/_data/program.csv index 36d984b..072ffa0 100644 --- a/_data/program.csv +++ b/_data/program.csv @@ -1,5 +1,5 @@ name,desc,inst,conf,use,docs -7-zip,7-zip is used in the lab to compress evidence and cases generated in Forensic Toolkit. It is also occasionally used to unpack collection material that is not supported by FTK.,[Download 7-zip](https://www.7-zip.org/){:target='_blank'}., +7-zip,7-zip is used in the lab to compress evidence and cases generated in Forensic Toolkit. It is also occasionally used to unpack collection material that is not supported by FTK.,[Download 7-zip](https://www.7-zip.org/), apple3rtr,Apple /// Ready-to-Run (apple3rtr) is a software bundle used for MAME Apple emulation. It has been used by Digital Archives staff to view Apple II files.,For detailed installation instructions visit the [apple3rtr github repo](https://github.com/datajerk/apple3rtr).,,,https://github.com/datajerk/apple3rtr AWS, Amazon Web Services (AWS) is a cloud computing platform offered by Amazon. AWS S3 features are used to store copies of born-digital material both before and after the collection has been processed by archivists.,,,,https://docs.aws.amazon.com/ Bagger, Bagger is an applciation created by the U.S. Library of Congress to produce files packages following the Bagit specification. It has been used in the Digital Archives lab for transferring born-digital material.,,,,https://github.com/LibraryOfCongress/bagger diff --git a/tools/programs.md b/tools/programs.md index 965c72a..292bd0a 100644 --- a/tools/programs.md +++ b/tools/programs.md @@ -17,7 +17,7 @@ Programs are defined as any non-internally developed out-of-the-box software use {{tool.desc}} {% if tool.docs %} -[See Official Documentation]({{tool.docs}}) +[{{tool.name}} Official Documentation]({{tool.docs}}) {% endif %} {% if tool.inst%} From 6efa85de218260b2ae507ccf88233800fb2ef697 Mon Sep 17 00:00:00 2001 From: arhonemus <34036628+arhonemus@users.noreply.github.com> Date: Fri, 8 Sep 2023 11:01:06 -0400 Subject: [PATCH 15/17] removed Filemaker entry --- _data/program.csv | 1 - 1 file changed, 1 deletion(-) diff --git a/_data/program.csv b/_data/program.csv index 072ffa0..ce1117d 100644 --- a/_data/program.csv +++ b/_data/program.csv @@ -14,7 +14,6 @@ DosBox, DosBox is an open-source emulator for the MS-DOS operating system. DosBo DROID, DROID is file format identification tool developed by the UK National Archives.,,,,https://cdn.nationalarchives.gov.uk/documents/information-management/droid-user-guide.pdf Emailchemy, Emailchemy has been used in the Digital Archives lab for migrating email fomats.,,,,https://www.weirdkid.com/products/emailchemy/doc/Emailchemy_User_Manual.pdf ePADD, ePADD is an open-source program developed by Stanford University's Special Collections and University Archives. The software ha been used for appraisal and arrangement of email.,,,,https://github.com/ePADD/epadd/releases -Filemaker Pro 17, Filemaker is used by Preservation and Collection Services (PCP) as a collection management system. born-digital media is inventoried in Filemaker and PCP tracks projects through Filemaker., Installing Filemaker requires license keys. Make sure you have the proper keys and permissions before installing., Forensic Toolkit, Forensic Toolkit is a computer forensics software used in the Digital Archives Lab to appraise and arrange born-digital material. FTK Imager, FTK Imager is a disk imaging program associated with Forensic Toolkit. The software has been used in the Digital Archives lab to image and extract files from digital carriers., gtar, Gtar is a command line program used on Digital Archives MacOS workstations to compress Kryoflux stream files., For MacOS: Install via Homebrew by entering ```brew install gnu-tar``` in terminal.,,,https://www.gnu.org/software/tar/manual/tar.html From 135e0b1df1861677a604f954e49d539d7043044d Mon Sep 17 00:00:00 2001 From: arhonemus <34036628+arhonemus@users.noreply.github.com> Date: Fri, 8 Sep 2023 11:05:28 -0400 Subject: [PATCH 16/17] copy edited working.csv --- _data/working.csv | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/_data/working.csv b/_data/working.csv index 1fe23ec..da9f464 100644 --- a/_data/working.csv +++ b/_data/working.csv @@ -1,5 +1,5 @@ name,desc,use kryofluxmove.sh,This bash script is used by Digital Archives staff to move floppy disk images transferred using kryoflux to folders with legacy CMS media IDs of the associated digital carrier., -SIPdir.sh,This bash script is used by Digital Archives staff to reate consecutive numbers of submission information packages for material transferred from digital carriers., -report_ftk_extents.py,This python script transforms XML reports exported from FTK after Processing Archivists have completed bookmarking into JSON files for import into ASpace.,The script takes two arguments: ```-f``` or ```--file``` followed by the path to the XML report to be transformed and ```o``` or ```--output``` followed by the path the destination directory for JSON output. -report_HDD_extents.py,This python script collects the extents of finding aid components created by a Processing Archivist when processing work is done at a workstation with a hard drive. The script then produces an extents JSON file for import into ASpace.,The script takes one argument: -d or --dir for the finding aid components directory on a hard drive.
Run the script following the syntax: python3 path/to/report_ftk_extents.py -d /path/to/collection/er/directory \ No newline at end of file +SIPdir.sh,This bash script is used by Digital Archives staff to create consecutive numbers of submission information packages for material transferred from digital carriers., +report_ftk_extents.py,This python script transforms XML reports exported from FTK after Processing Archivists have completed bookmarking into JSON files for import into ASpace.,The script takes two arguments: ```-f``` or ```--file``` followed by the path to the XML report to be transformed and ```-o``` or ```--output``` followed by the path the destination directory for JSON output. +report_HDD_extents.py,This python script collects the extents of finding aid components created by a Processing Archivist when processing work is done at a workstation with a hard drive. The script then produces an extents JSON file for import into ASpace.,The script takes one argument: ```-d``` or ```--dir``` for the finding aid components directory on a hard drive.
Run the script following the syntax: ```python3 path/to/report_ftk_extents.py -d /path/to/collection/er/directory``` \ No newline at end of file From 0c9178955a507850bc9dfa7efeda6ba740ddad82 Mon Sep 17 00:00:00 2001 From: arhonemus <34036628+arhonemus@users.noreply.github.com> Date: Fri, 8 Sep 2023 11:11:09 -0400 Subject: [PATCH 17/17] copy edited rclone --- tools/rclone.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tools/rclone.md b/tools/rclone.md index 193a9fa..455f782 100644 --- a/tools/rclone.md +++ b/tools/rclone.md @@ -39,9 +39,9 @@ Once installed, you must configure Rclone which involves setting the remote loca - rclone will provide a list of scopes of access ranging from full access to all files to read-only access to file metadata. For our purposes enter the number corresponding to full access to all files. -- Rclone asks for a root folder id, this is optional and you can enter to select enter to choose default. +- Rclone asks for a root folder id, this is optional and you can select enter to choose default. -- Rclone asks for service account credentials, this is optional and you can enter to select enter to choose default. +- Rclone asks for service account credentials, this is optional and you can select enter to choose default. - Rclone asks you to edit advanced configuration, select “n” for No. Follow up prompt asks if you would like to use auto configuration, enter “y” for Yes.